Context

Sony PlayStation Store uses a binary classifier to detect potentially fraudulent digital purchases before fulfillment. The Risk team is deciding between two candidate models for production: Model P (precision-oriented) and Model R (recall-oriented).

Current Performance

Evaluation was run on a labeled holdout set of 200,000 transactions with a fraud rate of 2.0% (4,000 fraud, 196,000 legitimate).

The Problem

Sony can only send flagged purchases to a limited review queue before digital content is delivered. The business wants to reduce fraud losses without creating too much customer friction from unnecessary holds on legitimate purchases.

Requirements

1. Compare the two models using the metrics above and explain when a precision-oriented model is preferable versus when a recall-oriented model is preferable.
2. Quantify the business tradeoff using the review-cost and missed-fraud-loss assumptions.
3. Recommend which model Sony should deploy under the current operating constraints.
4. Identify what additional analysis you would request before making a final launch decision.
5. Explain whether threshold tuning could produce a better operating point than choosing either model as-is.

Constraints

• Manual review capacity: 4,000 transactions/day
• False positives delay legitimate purchases and increase support contacts
• False negatives create direct fraud loss and chargeback risk
• Decision must support near-real-time checkout flow in PlayStation Store