Discuss Loan Default Model Vulnerabilities | Dataford Interview Questions

Business Context

NorthStar Bank uses a binary classification model to predict whether a personal loan applicant will default within 12 months. The model is already in production for pre-approval screening, and the hiring team wants to assess whether you can identify and mitigate common model vulnerabilities before they become business, compliance, or reliability issues.

Dataset

You are given a historical underwriting dataset used to train the current model.

Feature Group	Count	Examples
Applicant financials	12	annual_income, debt_to_income, revolving_utilization, credit_history_length
Credit bureau signals	9	delinquencies_2y, inquiries_6m, public_records, fico_band
Loan attributes	6	loan_amount, term_months, interest_rate, purpose
Demographics / geography	5	state, employment_type, home_ownership, age_band
Operational fields	4	application_channel, broker_flag, submission_hour, prior_customer

Rows: 240K applications from the last 24 months
Target: default_12m (1 if defaulted within 12 months, else 0)
Class balance: 11.5% positive
Missing data: 8% missing in employment-related fields, 3% missing in bureau features, and some categories unseen in recent months due to policy changes

Success Criteria

A strong solution should identify the main vulnerabilities of the model, explain how to test for them, and propose practical mitigations. “Good enough” means the model maintains stable out-of-time performance, avoids leakage, is reasonably interpretable for risk review, and does not materially degrade for key applicant segments.

Constraints

Decisions must be explainable to risk and compliance teams
Batch scoring latency must stay under 10 minutes for 100K applications
Retraining is allowed monthly, not daily
The bank prefers simple, auditable pipelines over highly complex ensembles

Deliverables

Train a baseline default classifier and evaluate it on an out-of-time test set.
Identify at least five model vulnerabilities you would be prepared to discuss in an interview.
Show how you would detect vulnerabilities such as overfitting, leakage, drift, instability, and unfair segment performance.
Recommend mitigations and monitoring for production deployment.
Explain the tradeoffs between a simpler interpretable model and a stronger but less transparent model.

Problem

Business Context

Dataset

You are given a historical underwriting dataset used to train the current model.

Feature Group	Count	Examples
Applicant financials	12	annual_income, debt_to_income, revolving_utilization, credit_history_length
Credit bureau signals	9	delinquencies_2y, inquiries_6m, public_records, fico_band
Loan attributes	6	loan_amount, term_months, interest_rate, purpose
Demographics / geography	5	state, employment_type, home_ownership, age_band
Operational fields	4	application_channel, broker_flag, submission_hour, prior_customer

Rows: 240K applications from the last 24 months
Target: default_12m (1 if defaulted within 12 months, else 0)
Class balance: 11.5% positive
Missing data: 8% missing in employment-related fields, 3% missing in bureau features, and some categories unseen in recent months due to policy changes

Success Criteria

Constraints

Decisions must be explainable to risk and compliance teams
Batch scoring latency must stay under 10 minutes for 100K applications
Retraining is allowed monthly, not daily
The bank prefers simple, auditable pipelines over highly complex ensembles

Deliverables

Train a baseline default classifier and evaluate it on an out-of-time test set.
Identify at least five model vulnerabilities you would be prepared to discuss in an interview.
Show how you would detect vulnerabilities such as overfitting, leakage, drift, instability, and unfair segment performance.
Recommend mitigations and monitoring for production deployment.
Explain the tradeoffs between a simpler interpretable model and a stronger but less transparent model.

Problem

Business Context

Dataset

You are given a historical underwriting dataset used to train the current model.

Feature Group	Count	Examples
Applicant financials	12	annual_income, debt_to_income, revolving_utilization, credit_history_length
Credit bureau signals	9	delinquencies_2y, inquiries_6m, public_records, fico_band
Loan attributes	6	loan_amount, term_months, interest_rate, purpose
Demographics / geography	5	state, employment_type, home_ownership, age_band
Operational fields	4	application_channel, broker_flag, submission_hour, prior_customer

Rows: 240K applications from the last 24 months
Target: default_12m (1 if defaulted within 12 months, else 0)
Class balance: 11.5% positive
Missing data: 8% missing in employment-related fields, 3% missing in bureau features, and some categories unseen in recent months due to policy changes

Success Criteria

Constraints

Decisions must be explainable to risk and compliance teams
Batch scoring latency must stay under 10 minutes for 100K applications
Retraining is allowed monthly, not daily
The bank prefers simple, auditable pipelines over highly complex ensembles

Deliverables

Train a baseline default classifier and evaluate it on an out-of-time test set.
Identify at least five model vulnerabilities you would be prepared to discuss in an interview.
Show how you would detect vulnerabilities such as overfitting, leakage, drift, instability, and unfair segment performance.
Recommend mitigations and monitoring for production deployment.
Explain the tradeoffs between a simpler interpretable model and a stronger but less transparent model.

Problem

Business Context

Dataset

You are given a historical underwriting dataset used to train the current model.

Feature Group	Count	Examples
Applicant financials	12	annual_income, debt_to_income, revolving_utilization, credit_history_length
Credit bureau signals	9	delinquencies_2y, inquiries_6m, public_records, fico_band
Loan attributes	6	loan_amount, term_months, interest_rate, purpose
Demographics / geography	5	state, employment_type, home_ownership, age_band
Operational fields	4	application_channel, broker_flag, submission_hour, prior_customer

Rows: 240K applications from the last 24 months
Target: default_12m (1 if defaulted within 12 months, else 0)
Class balance: 11.5% positive
Missing data: 8% missing in employment-related fields, 3% missing in bureau features, and some categories unseen in recent months due to policy changes

Success Criteria

Constraints

Decisions must be explainable to risk and compliance teams
Batch scoring latency must stay under 10 minutes for 100K applications
Retraining is allowed monthly, not daily
The bank prefers simple, auditable pipelines over highly complex ensembles

Deliverables

Train a baseline default classifier and evaluate it on an out-of-time test set.
Identify at least five model vulnerabilities you would be prepared to discuss in an interview.
Show how you would detect vulnerabilities such as overfitting, leakage, drift, instability, and unfair segment performance.
Recommend mitigations and monitoring for production deployment.
Explain the tradeoffs between a simpler interpretable model and a stronger but less transparent model.