Problem
Product Context
Meta runs a large internal microservices platform where thousands of services consume secrets from a centralized secrets management system. Design an ML-driven control plane that prioritizes secret rotation, detects risky access patterns, and decides when to trigger immediate rotation versus scheduled rotation with minimal service disruption.
Scale
| Signal | Value |
|---|---|
| Internal services | 120,000 |
| Secrets under management | 1.8B active secrets |
| Secret reads/day | 45B |
| Secret write / rotation events/day | 220M |
| Peak policy-evaluation QPS | 650K |
| Peak rotation-decision QPS | 90K |
| End-to-end online latency budget | 120ms p99 |
| Regions | 12 global regions |
Task
- Clarify the product goals, threat model, and success metrics for an ML-based secret rotation system.
- Design the end-to-end architecture, including candidate generation, risk ranking, and final re-ranking / policy enforcement before rotation.
- Choose models and features for each stage, and explain what runs online versus batch.
- Define the data pipeline, labels, training cadence, and how to avoid training-serving skew.
- Propose offline and online evaluation, including security guardrails and rollout strategy.
- Identify key failure modes, especially around feature drift, false positives, stale features, and regional outages.
Constraints
- The system must integrate with Meta-style internal service identity, audit logging, and regional control planes.
- Some secrets back critical production paths and cannot be rotated synchronously without a fallback credential path.
- Access logs are high-volume and partially delayed across regions by up to 3 minutes.
- Security decisions must be explainable to infra and service owners.
- Cost matters: only the highest-risk 0.5% of secrets can trigger expensive immediate rotation workflows each day.
Practicing as: Security Engineer interview at MetaHi, I'll play your Meta interviewer for the Security Engineer role. Candidates describe these interviews as mostly positive and moderately difficult, so expect me to be friendly and conversational. Take your time with the question above and answer like we're in the room.
You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.
