Engineer Features for ServiceNow Escalations

Business Context

Steampunk uses ServiceNow to manage internal delivery and support workflows across client programs. You need to build a model that predicts whether an incident ticket will be escalated within 7 days, and the main focus of the exercise is deciding which features to engineer, why they are useful, and how to validate that they improve model performance without introducing leakage.

Dataset

You are given 14 months of historical ServiceNow incident data exported at the ticket-day grain.

• Size: 320K ticket-day records representing 92K unique incidents
• Target: Binary — escalated within the next 7 days (1) vs not escalated (0)
• Class balance: 11.4% positive, 88.6% negative
• Missing data: 18% missing in text-derived fields, 6% missing in requester/account attributes, and some workflow fields only populated after specific transitions

Success Criteria

A strong solution should improve over a no-feature-engineering baseline and achieve PR-AUC ≥ 0.42, ROC-AUC ≥ 0.80, and precision ≥ 0.35 at 70% recall on a held-out time-based test set.

Constraints

• Avoid target leakage from post-escalation or future workflow events
• Predictions run every 4 hours, so batch inference must stay under 5 minutes for 50K open tickets
• Operations leaders need interpretable drivers, not just a black-box score

Deliverables

1. Propose and implement a feature engineering strategy, including features you would include and explicitly exclude.
2. Train at least one baseline model and one model using engineered features.
3. Use a time-based validation scheme and justify it.
4. Evaluate the impact of engineered features with concrete metrics.
5. Explain which engineered features are most valuable and how you would maintain them in production.