82.8%
Accuracy on HAM10000 using AdaCBM with GPT-generated concepts, outperforming linear classification and LaBo baselines.
Research & Publications
MICCAI 2024
AdaCBM: Explainable and accurate diagnosis with adaptive concept bottleneck models
Townim F. Chowdhury, Vu Minh Hieu Phan, Kewen Liao, Minh-Son To, Yutong Xie, Anton van den Hengel, Johan W. Verjans, Zhibin Liao
2024
Townim F. Chowdhury, Vu Minh Hieu Phan, Kewen Liao, Minh-Son To, Yutong Xie, Anton van den Hengel, Johan W. Verjans, Zhibin Liao
2024
Publication summary
AdaCBM addresses a key challenge in clinical AI: improving diagnostic performance while keeping model decisions explainable through human-understandable concepts. The paper builds on Concept Bottleneck Models, CLIP, and GPT-generated medical concepts.
The authors show that conventional CLIP-based concept bottleneck models have limited adaptation power because post-CBM fine-tuning mainly rescales and shifts concept contributions. To improve transfer learning without losing interpretability, AdaCBM introduces a learnable adapter module between CLIP image embeddings and the concept bottleneck model.
The framework also includes prompt-guided medical concept generation and concept utility selection using statistical criteria. Experiments on skin disease, blood cell cytology, and diabetic retinopathy datasets show AdaCBM can match or exceed non-interpretable baselines while producing concept-level explanations.
“AdaCBM demonstrates that explainability and accuracy do not need to be treated as opposing goals. By adapting the image representation before concept reasoning, the model preserves transparent concept-level decisions while improving diagnostic performance.”
Editorial research summary
OrthoAI research content adaptation
Key findings
74.3%
Accuracy on BCCD cytology images, closely matching the non-interpretable linear classification ceiling while retaining interpretability.
78.3%
Accuracy on diabetic retinopathy fundus images using only selected concept-based explanations.
Method
Adaptive concept bottleneck learning
AdaCBM re-examines CLIP-based Concept Bottleneck Models as linear classification systems and identifies that adapting the image embedding before concept scoring is more effective than only fine-tuning post-CBM weights.
A lightweight learnable adapter is inserted between the CLIP image encoder and the CBM layer. This improves downstream domain adaptation while preserving the concept bottleneck as the only inference path.
Prompt-guided concept generation and selection
GPT-4 is used to generate concise medical concepts describing visual properties of disease classes, including color, shape, size, and texture.
Concepts are filtered using task-utility criteria based on t-statistics and Pearson correlation, helping retain discriminative and non-redundant medical concepts for explainable diagnosis.
Key contributions
AdaCBM offers a complete recipe for explainable medical diagnosis using GPT-generated concepts, CLIP embeddings, and adaptive concept bottleneck learning.
- Reframes CLIP-based Concept Bottleneck Models as linear classification systems to identify where adaptation should occur.
- Introduces a lightweight adapter module between CLIP and CBM to improve transfer learning without sacrificing explainability.
- Uses GPT-4 prompt engineering to generate concise, clinically meaningful visual concepts.
- Applies statistical concept utility selection to retain discriminative and non-redundant medical concepts.
- Maintains concept-level interpretability while improving diagnostic classification performance.
- Validates the framework across dermatoscopic, cytology, and fundus image classification datasets.
Similar publications
view all
MICCAI 2023
MedIM: Boost Medical Image Representation via Radiology Report-guided Masking
Yutong Xie, Lin Gu, Tatsuya Harada, Jianpeng Zhang, Yong Xia, Qi Wu
CVPR 2023 · IEEE/CVF Conference on Computer Vision and Pattern Recognition
PEFAT: Boosting Semi-Supervised Medical Image Classification via Pseudo-loss Estimation and Feature Adversarial Training
Qingjie Zeng, Yutong Xie, Zilin Lu, Yong Xia