Business Context
AdNova serves sponsored product ads across a mid-sized ecommerce marketplace and wants a neural network to predict click-through probability for ranking. The current model overfits quickly: training loss keeps falling while validation AUC plateaus after a few epochs.
Dataset
You are given a supervised binary classification dataset for ad click prediction.
| Feature Group | Count | Examples |
|---|
| Numerical behavior features | 18 | impressions_7d, clicks_7d, ctr_30d, avg_session_length |
| Categorical entity features | 9 | device_type, country, ad_category, campaign_type |
| Dense embedding inputs | 3 | user_embedding, item_embedding, query_embedding |
| Contextual features | 6 | hour_of_day, day_of_week, placement, page_type |
- Size: 1.2M ad impressions, 36 input features after preprocessing
- Target: Binary — clicked (1) vs not clicked (0)
- Class balance: 11.4% positive, 88.6% negative
- Missing data: ~4% missing in behavioral aggregates for new users; <1% missing in categorical fields
Success Criteria
A good solution should improve validation generalization using dropout, achieve AUC-ROC >= 0.78, log loss <= 0.42, and reduce the train-validation AUC gap to < 0.03.
Constraints
- Batch inference must score 5M impressions/hour
- Training must complete in under 2 hours on a single GPU
- The solution should remain simple enough for reliable retraining by the ads ML team
Deliverables
- Build a neural network baseline without dropout and a regularized version with dropout.
- Explain where dropout should be applied and why it helps reduce overfitting.
- Compare train/validation/test performance and quantify the generalization improvement.
- Tune dropout rate and at least two additional hyperparameters.
- Recommend a production-ready configuration, including monitoring for overfitting after deployment.
