Model Insights & Explainability
Full transparency into neural decision boundaries through gradient-weighted class activation mapping (Grad-CAM) and game-theoretic SHAP tree attribution.
ResNet-18
512-dim Embedding
XGBoost
15-dim Multi-Softprob
Clinical Scaler
22-dim Standardized
Gated Cross-Modal
2-Head MultiheadAttention
ResNet18 Vision Backbone
Processes 224x224 chest radiographs through deep convolutional residual blocks. Frozen early layers maintain robust low-level spatial features while layer3/4 are fine-tuned for thoracic opacities. Extracts a dense 512-dimensional latent embedding.
Grad-CAM uses the gradients of the target concept flowing into the final convolutional layer (layer4[-1]) to generate a coarse 2D localization map highlighting the regions in the chest radiograph most critical for predicting consolidation.
ResNet-18 ROC Curve Analysis (AUC 0.9540)
SHAP (SHapley Additive exPlanations) computes Shapley values from cooperative game theory to quantify the exact marginal contribution of each symptom and vital sign toward the tree ensemble's final disease log-odds.
| Modality Stream | Architecture | Accuracy | Precision (Macro) | Recall (Macro) | F1-Score | ROC-AUC |
|---|---|---|---|---|---|---|
| Image-Only | ResNet18 Transfer Learning | 85.10% | 89.81% | 80.30% | 82.38% | 0.9540 |
| Symptom-Only | XGBoost Multi-Softprob | 87.20% | 87.21% | 87.20% | 87.20% | 0.9691 |
| Clinical-Only | XGBoost + Scaler | 87.10% | 87.07% | 87.10% | 87.08% | 0.9709 |
| Multimodal (Gated) | Gated Cross-Attention | 97.08% | 97.10% | 97.08% | 97.08% | 0.9880 |