Context
Asana wants to introduce an AI capability into Asana Inbox that drafts a concise action summary for a task update thread and suggests the next step for the assignee. The feature should help users process updates faster without changing the core workflow of reading, triaging, and acting on work.
Constraints
- p95 latency: 1,500ms per summary generation
- Cost ceiling: $12K/month at 1.2M summaries/month
- Hallucination ceiling: <2% of responses may state a task status, owner, or due date not supported by the task context
- Safety: must resist prompt injection from task comments, must not reveal hidden project data, and must refuse when context is insufficient
- UX: output must be short enough to fit in the existing Inbox preview surface without expanding the row
Available Data / Models
- Task title, description, custom fields, assignee, due date, recent comments, project name, and visible subtasks
- User permission-filtered context from Asana only; no web access and no external tools
- Approved LLMs: a fast low-cost model for most traffic and a higher-quality fallback model
- Historical human-written task updates and triage actions for evaluation only
- Existing analytics in Asana for click-through, archive, snooze, and open-task actions
Deliverables
- Design an eval-first rollout plan for this AI summary capability, including offline and online evaluation before discussing architecture.
- Write a system prompt that generates a grounded summary and suggested next step using only visible task context, with explicit refusal behavior.
- Propose the serving architecture for integrating the model into Asana Inbox, including fallback logic, structured output, and guardrails for hallucination and prompt injection.
- Estimate cost and latency at target volume, and explain how you would stay within both constraints.
- Identify the top failure modes, how you would detect them in production, and how you would decide whether to expand from an internal dogfood launch to general availability.
