Whenever you're ready, walk me through how you'd approach this.
Describe how you would design a scaling API application given specific constraints.
Focus on how you would size the system, choose the serving architecture, and handle latency, cost, and reliability tradeoffs. Assume the API must support both online requests and offline learning or optimization loops, with clear monitoring and fallback behavior.