Context

StripeShield uses a binary classification model to flag card transactions for fraud review before approval. The current model has reduced chargebacks overall, but the Risk team is concerned that the threshold may be too conservative and is allowing costly fraud through.

Current Performance

The model was evaluated on 1,000,000 recent transactions with an observed fraud rate of 0.8%.

Confusion matrix counts:

The Problem

Each false negative results in an average fraud loss of $240. Each false positive sends a legitimate transaction to manual review or decline, costing $6 in operations and customer friction. The Head of Risk wants to know whether the model should be tuned for higher recall, even if precision drops.

Requirements

1. Interpret the trade-off between false positives and false negatives using the metrics above.
2. Quantify the business impact of current FP and FN errors.
3. Explain why accuracy is misleading in this setting.
4. Recommend whether to lower the decision threshold and what metric(s) should guide that choice.
5. Propose follow-up analyses to validate the recommendation.

Constraints

• Manual review capacity is limited to 8,000 flagged transactions per day.
• Customer experience is sensitive to unnecessary declines.
• Fraud labels arrive with a 21-day delay, so evaluation is retrospective.