Rang Technologies is preparing to deploy a binary classification model in Rang Sales Hub to score inbound leads as likely or unlikely to convert within 30 days. The model will determine which leads are routed to the SDR team first. In offline testing, the model looks better than the legacy rules-based system on ranking quality, but several validation signals suggest deployment risk.
| Metric | Legacy Rules | Candidate Model | Validation Note |
|---|---|---|---|
| Accuracy | 0.81 | 0.89 | Inflated by class imbalance |
| Precision | 0.42 | 0.61 | Better lead quality for SDRs |
| Recall | 0.68 | 0.47 | More converters are missed |
| F1 Score | 0.52 | 0.53 | Only marginal net improvement |
| AUC-ROC | 0.71 | 0.86 | Strong ranking ability |
| Log Loss | 0.58 | 0.41 | Better probability estimates overall |
| Brier Score | 0.19 | 0.16 | Calibration still imperfect |
| Positive Rate | 0.18 | 0.18 | Base rate unchanged |
The VP of Revenue wants to launch the model next week because precision improved materially, but SDR leadership is concerned that recall dropped from 0.68 to 0.47, which could reduce total conversions if too many good leads are deprioritized. You need to determine whether the model is actually ready for deployment and what validation steps should happen before launch.