Context

NimbusCRM uses a lead scoring model to rank inbound B2B leads so the sales team can prioritize outreach. The model predicts whether a lead will become a sales-qualified opportunity within 30 days, but leadership is unsure whether better offline metrics are translating into more pipeline and revenue.

Current Performance

The Problem

The model looks materially better than the old rules offline, and SDRs are contacting high-scored leads faster. However, closed-won revenue is slightly down and sales leadership believes the model may be over-prioritizing leads that convert to SQL but not to revenue.

Requirements

1. Assess whether the model is helping the business, not just improving ML metrics.
2. Explain what the current metrics suggest about ranking quality, threshold choice, and calibration.
3. Identify likely failure modes causing stronger lead prioritization but weaker downstream revenue.
4. Recommend how you would validate impact using holdouts, funnel analysis, or experiments.
5. Propose model and process changes to improve business outcomes.

Constraints

• SDR team can actively work only 20% of monthly inbound leads.
• Enterprise deals have 4x the revenue of SMB deals but longer sales cycles.
• Labels for closed-won revenue arrive 60-120 days after lead creation.