Business Context

RoboFleet operates 2,000 autonomous mobile robots across fulfillment centers. The robotics team wants a reinforcement learning policy that improves navigation efficiency and collision avoidance in simulation before controlled deployment on real robots.

Dataset

Unlike supervised learning, this problem uses logged interaction trajectories collected from a simulator and prior robot controllers. You are given offline rollout data for initialization and an online training environment for policy improvement.

• Observation space: 64 numerical features per timestep
• Action space: 2 continuous controls
• Reward sparsity: moderate; dense shaping plus terminal success/failure rewards
• Failure rate: ~14% of episodes end in collision or timeout
• Missing data: ~3% missing sensor bins due to simulated dropout; some episodes have truncated logs

Success Criteria

A good solution should improve mean episodic return by at least 20% over the rule-based baseline, reduce collision rate below 5%, and maintain inference latency under 10 ms per control step on edge hardware.

Constraints

• Safety matters more than raw reward; unstable policies cannot be deployed
• Training is expensive: simulator budget is capped at 72 GPU-hours per experiment
• The policy must generalize across warehouse layouts and moderate sensor noise
• The final policy should be explainable at a high level to robotics engineers

Deliverables

1. Explain what Proximal Policy Optimization (PPO) is and why it fits this robotics setting.
2. Build a PPO training pipeline for continuous robot control using the provided simulator.
3. Describe preprocessing for observations, reward handling, and episode truncation.
4. Evaluate the learned policy against a rule-based baseline on held-out maps.
5. Recommend deployment safeguards, monitoring metrics, and retraining cadence.