General Symptoms Image Analysis Multimodal Assessment Model Insights About

Model Insights & Explainability

Full transparency into neural decision boundaries through gradient-weighted class activation mapping (Grad-CAM) and game-theoretic SHAP tree attribution.

Interactive Neural Pipeline Architecture Click any node to inspect
Vision

ResNet-18

512-dim Embedding

Symptoms

XGBoost

15-dim Multi-Softprob

Vitals

Clinical Scaler

22-dim Standardized

Attention

Gated Cross-Modal

2-Head MultiheadAttention

PyTorch • Transfer Learning • Grad-CAM Hooked

ResNet18 Vision Backbone

Processes 224x224 chest radiographs through deep convolutional residual blocks. Frozen early layers maintain robust low-level spatial features while layer3/4 are fine-tuned for thoracic opacities. Extracts a dense 512-dimensional latent embedding.

Radiology Explainability (Grad-CAM) Spatial Localization

Grad-CAM uses the gradients of the target concept flowing into the final convolutional layer (layer4[-1]) to generate a coarse 2D localization map highlighting the regions in the chest radiograph most critical for predicting consolidation.

ROC Evaluation ResNet-18 ROC Curve Analysis (AUC 0.9540)
Clinical Signal Attribution (SHAP) Game Theoretic

SHAP (SHapley Additive exPlanations) computes Shapley values from cooperative game theory to quantify the exact marginal contribution of each symptom and vital sign toward the tree ensemble's final disease log-odds.

Generating live SHAP feature attribution plot...
Model Performance & Evaluation Benchmark Real Test Set Validation
Modality Stream Architecture Accuracy Precision (Macro) Recall (Macro) F1-Score ROC-AUC
Image-Only ResNet18 Transfer Learning 85.10% 89.81% 80.30% 82.38% 0.9540
Symptom-Only XGBoost Multi-Softprob 87.20% 87.21% 87.20% 87.20% 0.9691
Clinical-Only XGBoost + Scaler 87.10% 87.07% 87.10% 87.08% 0.9709
Multimodal (Gated) Gated Cross-Attention 97.08% 97.10% 97.08% 97.08% 0.9880