Our services experienced a 6-minute outage due to AWS hardware failure. Despite having systems for database re-balancing and scaling, they underperformed and failed to recover properly. We are now actively taking steps to improve these solutions to ensure better resilience against future hardware failures.