Dataford
Popular roles
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Browse
Browse All RolesEvery role hub, from analyst to MLBrowse All CompaniesCompany-specific interview loopsAll Interview GuidesThe full guide library
Top questions by role
Software EngineerData AnalystData ScientistData EngineerBusiness AnalystAI EngineerMachine Learning EngineerProduct Manager
Top questions by skill
SQLPythonStatisticsMachine LearningA/B TestingSystem DesignGenerative AIProduct SenseMetricsBehavioral
Browse all questions →Try a mock interview
ExperiencesCoaching
New
Practice
Mock InterviewsTimed interview simulations with feedbackSuccess PathYour 6-week structured planModulesCurated lessons by topicWebinarsTalks from ex-Big Tech data leadsPlaygroundA free-form scratch editor
Learn
BlogInterview strategy and career adviceTech Job Market ReportHiring trends across data and AI rolesFor UniversitiesDataford for career centersAbout DatafordWho we are and how we build
Pricing
Get started

Design Intelligent API Rate-Limit Orchestrator

HardSystem Design00:00
Practice interviewer
In session
5 left
00:00

Whenever you're ready, walk me through how you'd approach this.

You need to log in / sign up to chat or submit.

Problem

Product Context

Meta exposes large-scale APIs such as the Graph API, Marketing API, and WhatsApp Business Platform to millions of external developers and enterprise integrations. Design an ML-driven rate-limit orchestration system that decides which requests to admit, defer, batch, or shed so client integrations remain reliable while Meta controls infrastructure cost and protects backend services.

Scale

SignalValue
Monthly active apps12M
Daily active apps3.5M
Peak inbound API QPS22M requests/sec
Distinct app-user tokens/day900M
Downstream protected services150+ internal services
Per-request decision latency budget (p99)25ms
Historical logs retained for training180 days

Task

  1. Clarify the product objective and define what “reliable” and “cost-efficient” mean for Meta and third-party developers.
  2. Design an end-to-end ML system that predicts request value/risk and makes real-time rate-limit decisions using a multi-stage architecture.
  3. Specify the online path, offline training pipeline, feature store design, and how the system handles tenant-level fairness, bursty traffic, and cold-start apps.
  4. Choose models for retrieval, ranking, and final policy/re-ranking, and explain why each stage is appropriate under the latency budget.
  5. Define offline and online evaluation, including guardrails for developer experience, backend protection, and cost.
  6. Identify major failure modes such as feature drift, training-serving skew, abuse spikes, and bad model rollouts, with detection and mitigation.

Constraints

  • Must support hard platform quotas and policy constraints; ML can optimize within policy, not override compliance rules.
  • Some labels are delayed: downstream errors, app retries, and developer churn may arrive hours or days later.
  • Decisions must be explainable enough for internal operations and enterprise support escalations.
  • The system should reduce unnecessary throttling while preventing cascading failures in Graph API and dependent services.
  • Cost target: keep incremental ML serving cost below $0.00002 per API request on average.