MediScan built a binary classification model to detect a rare cardiac condition from routine screening data. The condition appears in only 2% of patients, and the clinical team is concerned that the current dashboard highlights accuracy even though missed cases can delay treatment.
The model was evaluated on 10,000 patients with 200 actual positive cases.
| Metric | Model A | Model B |
|---|---|---|
| Accuracy | 97.0% | 91.0% |
| Recall | 25.0% | 85.0% |
| Precision | 62.5% | 16.2% |
| F1 Score | 35.7% | 27.3% |
| False Negatives | 150 | 30 |
| False Positives | 150 | 870 |
Leadership initially prefers Model A because its accuracy is much higher. However, clinicians argue that Model B is safer because it identifies far more true cases. You need to determine when accuracy is misleading in an imbalanced setting and how recall should influence model selection.