BrightPath Education uses a binary classification model to identify students at risk of dropping out of an online certificate program so advisors can intervene early. The team reports that overall accuracy looks strong, but program directors are concerned that too many at-risk students are still being missed.
| Metric | Current Model | Previous Rules-Based Process |
|---|---|---|
| Accuracy | 0.89 | 0.81 |
| Precision | 0.74 | 0.52 |
| Recall | 0.58 | 0.71 |
| F1 Score | 0.65 | 0.60 |
| AUC-ROC | 0.84 | 0.69 |
| Students flagged per month | 1,120 | 1,860 |
| Actual at-risk students per month | 1,430 | 1,430 |
The new model is more accurate and more precise than the prior framework, but recall has fallen. As a result, advisors spend less time on false alarms, yet a meaningful share of at-risk students receive no intervention.