Choose Checkout Fraud Threshold | Dataford Interview Questions

Your question is Choose Checkout Fraud Threshold. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Context

Thrive Market uses a binary classification model to score checkout orders for payment fraud before order confirmation. The current threshold was set during launch to reduce chargebacks, but the Risk and CX teams now disagree on whether the model is too aggressive and blocking too many legitimate members.

Current Performance

Validation set: 200,000 Thrive Market checkout attempts, with 2,000 actual fraudulent orders (1.0% prevalence).

Threshold	Precision	Recall	F1	False Positive Rate	Orders Flagged	TP	FP	FN	TN
0.30	0.18	0.90	0.30	0.041	10,000	1,800	8,200	200	189,800
0.50	0.31	0.78	0.44	0.018	5,000	1,560	3,440	440	194,560
0.70	0.50	0.55	0.52	0.006	2,200	1,100	1,100	900	196,900
0.85	0.67	0.30	0.41	0.0015	900	600	300	1,400	197,700

The Problem

A missed fraud order costs Thrive Market an average of $95 in chargebacks and operational loss. A false positive costs about $8 in customer support, payment retry friction, and abandoned checkout risk. The current production threshold is 0.70.

Requirements

Recommend the most appropriate threshold for Thrive Market and justify it quantitatively.
Explain the precision-recall tradeoff and why accuracy alone would be misleading here.
Compare the business cost at each threshold using the table above.
Describe when you would choose a different threshold for a manual review queue versus an automatic block.
Propose how you would validate the chosen threshold before full rollout.

Constraints

Fraud prevalence is low and class imbalance is high.
CX wants to minimize friction for legitimate members.
Risk leadership cares more about total business cost than maximizing a single metric.