Evaluate Personal Loan Rate Regression | Dataford Interview Questions

Your question is Evaluate Personal Loan Rate Regression. Take a moment with it on the right.

Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).

Context

SoFi has deployed a regression model to predict the final APR offered to applicants for SoFi Personal Loans before the pricing engine applies policy adjustments. The model is used by the underwriting and pricing teams to estimate applicant-level rates, but recent monitoring shows larger-than-expected pricing errors for some borrower segments.

Current Performance

Metric	Validation Set	Last 30 Days Production	Change
RMSE	0.62 APR pts	0.91 APR pts	+46.8%
MAE	0.41 APR pts	0.58 APR pts	+41.5%
Median Absolute Error	0.29 APR pts	0.37 APR pts	+27.6%
R²	0.84	0.71	-0.13
Bias (Predicted - Actual)	+0.03 APR pts	-0.18 APR pts	-0.21
% within ±0.50 APR pts	78%	61%	-17 pts

Segment	MAE	Bias
---	---:	---:
Prime borrowers (FICO 740+)	0.34	+0.11
Near-prime (680-739)	0.57	-0.09
Thin-file applicants	0.89	-0.46
Debt consolidation loans > $30K	0.76	-0.31

The Problem

The model still looks acceptable on aggregate, but it systematically underpredicts APR for thin-file and larger-loan applicants. That creates downstream pricing corrections in the SoFi underwriting flow and a poorer member experience when quoted rates change.

Requirements

Evaluate the model using the metrics above and explain which metrics matter most.
Diagnose what the aggregate metrics may be hiding across segments.
Recommend a validation approach to confirm whether this is drift, poor calibration, or segment underfit.
Propose concrete improvements to the model and monitoring plan.

Constraints

APR prediction errors above 0.50 points materially increase quote-to-funded drop-off.
Underpricing risk is more costly than overpricing risk.
Any replacement model must be explainable enough for risk and pricing review.