Date: October 21st, 2025 5:50 PM Author: Buff hall sneaky criminal
Amazon's internal DNS server at East Coast 1 stopped working intermittently for west coast customers around 12-2am PST. Amazon uses some shit called DynamoDB to handle all the DNS calls. If there's even a brief outage, it causes a backlog of requests to the DynamoDB while the system gets rebooted. In this case they got the shit rebooted fast as fuck, but the intermittent outages created a sufficient backlog of requests that it took several hours for DynamoDB to fully populate its cache and respond to all the requests. That's the best I can explain it.
Date: October 21st, 2025 6:00 PM Author: Buff hall sneaky criminal
Yeah it was a window into how many companies are using EC2. Turns out it's such a fuckload that it took nine hours for Amazon servers to fully populate the EC2 DNS cache