Optimize Underperforming Loan Default Model | Dataford Interview Questions

Context

LendWise uses a binary classification model to predict whether a small-business loan applicant will default within 90 days. The latest production model is underperforming against both the previous version and the team’s approval-risk targets, leading to higher credit losses and concern from risk operations.

Current Performance

Metric	Previous Model	Current Model	Target
Accuracy	0.84	0.79	0.83
Precision	0.71	0.64	0.70
Recall	0.68	0.52	0.65
F1 Score	0.69	0.57	0.67
AUC-ROC	0.81	0.74	0.80
Log Loss	0.46	0.59	<0.50
Default rate in eval set	18.0%	18.0%	—
Monthly credit loss	$1.9M	$2.8M	<$2.0M

The Problem

The current model is missing too many true defaulters while also generating more false alarms than expected. Risk leadership wants a structured plan to diagnose the underperformance and recommend improvements that can be implemented without materially slowing loan decisions.

Requirements

Interpret the metrics and explain what they imply about model behavior.
Identify likely root causes for the performance drop.
Propose an error analysis plan using the available outputs and segments.
Recommend specific optimization steps, including threshold and model changes.
Explain how you would validate that the next model is actually better.

Constraints

Loan decisions must be returned in under 300 ms.
False negatives are more costly than false positives, but excessive declines hurt revenue.
Retraining can happen weekly; new external bureau features are expensive to add.

Problem

Context

Current Performance

Metric	Previous Model	Current Model	Target
Accuracy	0.84	0.79	0.83
Precision	0.71	0.64	0.70
Recall	0.68	0.52	0.65
F1 Score	0.69	0.57	0.67
AUC-ROC	0.81	0.74	0.80
Log Loss	0.46	0.59	<0.50
Default rate in eval set	18.0%	18.0%	—
Monthly credit loss	$1.9M	$2.8M	<$2.0M

The Problem

Requirements

Interpret the metrics and explain what they imply about model behavior.
Identify likely root causes for the performance drop.
Propose an error analysis plan using the available outputs and segments.
Recommend specific optimization steps, including threshold and model changes.
Explain how you would validate that the next model is actually better.

Constraints

Loan decisions must be returned in under 300 ms.
False negatives are more costly than false positives, but excessive declines hurt revenue.
Retraining can happen weekly; new external bureau features are expensive to add.

Problem

Context

Current Performance

Metric	Previous Model	Current Model	Target
Accuracy	0.84	0.79	0.83
Precision	0.71	0.64	0.70
Recall	0.68	0.52	0.65
F1 Score	0.69	0.57	0.67
AUC-ROC	0.81	0.74	0.80
Log Loss	0.46	0.59	<0.50
Default rate in eval set	18.0%	18.0%	—
Monthly credit loss	$1.9M	$2.8M	<$2.0M

The Problem

Requirements

Interpret the metrics and explain what they imply about model behavior.
Identify likely root causes for the performance drop.
Propose an error analysis plan using the available outputs and segments.
Recommend specific optimization steps, including threshold and model changes.
Explain how you would validate that the next model is actually better.

Constraints

Loan decisions must be returned in under 300 ms.
False negatives are more costly than false positives, but excessive declines hurt revenue.
Retraining can happen weekly; new external bureau features are expensive to add.

Problem

Context

Current Performance

Metric	Previous Model	Current Model	Target
Accuracy	0.84	0.79	0.83
Precision	0.71	0.64	0.70
Recall	0.68	0.52	0.65
F1 Score	0.69	0.57	0.67
AUC-ROC	0.81	0.74	0.80
Log Loss	0.46	0.59	<0.50
Default rate in eval set	18.0%	18.0%	—
Monthly credit loss	$1.9M	$2.8M	<$2.0M

The Problem

Requirements

Interpret the metrics and explain what they imply about model behavior.
Identify likely root causes for the performance drop.
Propose an error analysis plan using the available outputs and segments.
Recommend specific optimization steps, including threshold and model changes.
Explain how you would validate that the next model is actually better.

Constraints

Loan decisions must be returned in under 300 ms.
False negatives are more costly than false positives, but excessive declines hurt revenue.
Retraining can happen weekly; new external bureau features are expensive to add.