Choose Threshold for Lead Scoring | Dataford Interview Questions

Context

Insight Global uses a binary classification model in its recruiter workflow to predict whether a candidate lead in the Insight Global talent platform will respond positively to outreach within 7 days. Recruiters can only prioritize about 1,200 leads per day, so the operating threshold directly determines both recruiter workload and missed opportunities.

The current model was trained on 1.8M historical leads and outputs a probability score. The team is debating whether to keep the current threshold of 0.50 or move to a lower threshold to capture more likely responders.

Current Performance

Threshold	Precision	Recall	F1	Daily Leads Flagged	True Positives/Day	False Positives/Day
0.70	0.81	0.42	0.55	620	502	118
0.50 (current)	0.68	0.61	0.64	1,050	714	336
0.35	0.56	0.76	0.64	1,540	862	678
0.20	0.41	0.89	0.56	2,180	894	1,286

Additional validation set facts: positive class prevalence is 11.2%, AUC-ROC is 0.84, log loss is 0.39, and calibration analysis shows predicted probabilities above 0.70 are slightly overconfident.

The Problem

Recruiting leadership wants more candidate responses, but recruiter operations cannot exceed daily review capacity for long. You need to recommend an operating threshold and explain the tradeoff clearly.

Requirements

Identify which threshold you would recommend and why.
Quantify the precision-recall tradeoff using the table.
Explain how business costs of false positives vs. false negatives should influence threshold choice.
Discuss whether calibration matters before using the score operationally.
Propose how you would validate the threshold after launch.

Constraints

Recruiter capacity is ~1,200 prioritized leads/day.
A false positive wastes recruiter time and delays other outreach.
A false negative means a potentially responsive candidate is not prioritized.

Context

Current Performance

Threshold	Precision	Recall	F1	Daily Leads Flagged	True Positives/Day	False Positives/Day
0.70	0.81	0.42	0.55	620	502	118
0.50 (current)	0.68	0.61	0.64	1,050	714	336
0.35	0.56	0.76	0.64	1,540	862	678
0.20	0.41	0.89	0.56	2,180	894	1,286

Additional validation set facts: positive class prevalence is 11.2%, AUC-ROC is 0.84, log loss is 0.39, and calibration analysis shows predicted probabilities above 0.70 are slightly overconfident.

Requirements

Identify which threshold you would recommend and why.

Quantify the precision-recall tradeoff using the table.

Explain how business costs of false positives vs. false negatives should influence threshold choice.

Discuss whether calibration matters before using the score operationally.

Propose how you would validate the threshold after launch.