Your question is Optimizing Foundation Model Inference. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Describe your experience optimizing inference for large foundation models in a production environment at Datadog.
Discuss one concrete system you personally worked on, including the baseline, bottleneck diagnosis, optimization techniques, and measurable results. Cover serving architecture, hardware and software choices, latency and cost tradeoffs, rollout strategy, and how you monitored quality, reliability, feature drift, and training-serving skew.