Evaluate Safety Review Precision-Recall Tradeoff

Context

SafeMarket uses a text classification model to flag user-generated product listings for manual safety review. The model decides which listings enter a limited human review queue, and leadership is concerned that the current setup may be missing too many unsafe listings while also consuming reviewer capacity.

Current Performance

The team evaluated one month of labeled listings after deployment. Unsafe content includes prohibited medical claims, dangerous product instructions, and regulated items requiring removal.

The Problem

The model is highly precise, but recall is materially lower than the previous operating point. Reviewers report that most flagged listings are truly unsafe, but policy teams are escalating missed violations found through user reports and audits.

Requirements

1. Define precision and recall clearly for this safety review workflow.
2. Interpret whether the current operating point is appropriate given the business objective.
3. Use the metrics to explain the tradeoff between reviewer efficiency and safety coverage.
4. Recommend how you would track these metrics over time by policy category, market, and model threshold.
5. Propose concrete changes to improve performance without overwhelming the review team.

Constraints

• Human review capacity is capped at 6,000 listings per day.
• False negatives carry regulatory and trust risk.
• False positives increase reviewer cost and delay legitimate listings.