BrightDesk, a SaaS customer support platform, wants to improve article retrieval in its help center. Users often search with natural-language questions, while the current system relies on keyword matching and misses semantically relevant documents.
You are given a corpus of 180,000 English help-center articles and resolved support tickets, plus 25,000 historical user queries with clicked or manually judged relevant documents. Documents range from 20 to 1,200 words (median 180), and queries range from 2 to 40 words. Roughly 65% of queries are short keyword-style searches, while 35% are conversational or paraphrased questions. Relevance labels are sparse: each query has 1-5 positive documents and many unlabeled candidates.
A strong solution should clearly compare embedding-based retrieval and traditional keyword search on relevance, robustness to paraphrasing, latency, and operational complexity. Target NDCG@10 ≥ 0.72 and Recall@20 ≥ 0.85 on held-out queries, while keeping p95 retrieval latency under 150 ms.