Context

Data Society is evaluating a binary classification model that predicts whether a learner will complete a course within 30 days. A junior team member trained a gradient boosted tree model and reported strong training performance, but stakeholders are concerned that the model may not generalize well in production.

Current Performance

The Problem

You need to determine whether the gradient boosted model is overfitting or underfitting, explain how the metrics support that conclusion, and recommend what Data Society should do next.

Requirements

1. Diagnose whether the current model is overfitting, underfitting, or neither.
2. Use the train/validation/test metrics to justify your conclusion.
3. Compare the current model to the simpler baseline and explain which model you would prefer.
4. Recommend specific changes to improve generalization.
5. Explain what additional validation checks you would run before deployment.

Constraints

• The model must be explainable enough for internal review.
• Retraining can happen weekly, not daily.
• False negatives are more costly than false positives because missed at-risk learners reduce intervention opportunities.