HomeValue AI built a gradient boosting regression model to predict residential sale prices for mortgage pre-approval and agent pricing guidance. The model performs well on average, but regional teams report that errors are large for high-value homes and newer properties, creating underwriting risk and poor user trust.
| Metric | Validation Set | Last Month in Production | Change |
|---|---|---|---|
| RMSE | $41,800 | $58,600 | +40.2% |
| MAE | $24,900 | $31,400 | +26.1% |
| Median Absolute Error | $16,200 | $19,100 | +17.9% |
| R² | 0.89 | 0.81 | -0.08 |
| Mean Error (Pred - Actual) | -$1,200 | -$9,800 | More underprediction |
| % predictions within 10% of actual | 78% | 64% | -14 pts |
| Segment breakdown from production: |
| Segment | MAE |
|---|---|
| Homes < $500k | $18,700 |
| Homes $500k-$1M | $34,900 |
| Homes > $1M | $96,400 |
| New builds (< 2 years old) | $72,300 |
| Rural zip codes | $61,800 |
The VP of Risk wants to know whether the model is still acceptable for pricing support and where it is failing. You need to evaluate overall performance, explain what the metrics imply, and recommend how to improve the model and its deployment policy.