OpenAI wants a lightweight classifier that routes incoming prompts on the Chat Completions API into product-relevant categories such as coding, summarization, safety-sensitive, and factual Q&A. You are given embeddings and metadata derived from a GPT-style transformer, and your task is to build a supervised model that performs accurate multi-class classification under tight latency constraints.
The dataset was generated from 1.2M historical prompts scored offline using a frozen GPT-style transformer encoder head. Each row represents one prompt and includes dense embedding features, prompt statistics, and limited metadata.
| Feature Group | Count | Examples |
|---|---|---|
| Transformer embeddings | 768 | pooled_hidden_0 ... pooled_hidden_767 |
| Prompt statistics | 9 | token_count, avg_token_idf, punctuation_ratio, code_block_count |
| Categorical metadata | 5 | surface, customer_tier, locale, model_family, hour_bucket |
| Quality flags | 4 | language_detect_confidence, pii_flag, truncation_flag, moderation_score |
A good solution should achieve macro-F1 of at least 0.78 and safety_sensitive recall above 0.90 on a held-out test set, while keeping p95 online inference below 20 ms per request.