Describe a time you faced a significant technical failure on a production system. What did you learn, and how did you prevent it from happening again?