AWS Web Hosting Service Reliability

AWS Reliability 2006–2025 We often have a biased focus — we notice the extraordinary when it breaks, but ignore the ordinary when it works quietly for years. The ~15-hour AWS outage on October 20, 2025, left a strong impression. In contrast, it’s hard to compare that with the nearly silent reliability of the previous years. Some teams, driven by emotion and lacking the bigger picture, even rush to "migrate ASAP." But not many people remember similar events. The previous outage of roughly the same length was the Kinesis incident in 2020. A longer one happened way back in 2011 — you can see those dips in the chart. If you calculate reliability using rolling windows (1-year and 5-year), you’ll see that typical AWS availability usually falls between three nines and four nines — like SQS (99.9%) and EC2 (99.99%). Most of the time, it’s closer to four nines. As of October 25, 2025, the rolling figures are 99.84% (1-year) and 99.95% (5-year). That last number — 99.95% — is on par with the published SLA for Lambda and EKS, which is, frankly, pretty great. Bottom line: We should learn from outages, but we should also understand the history and the actual numbers. Every extra "nine" comes at a high cost — in engineering effort, complexity, and money.

🛡️ How to Protect Your Business from Cloud Outages The AWS US-EAST-1 outage affected hundreds of services for 20+ hours. Here’s how to ensure your business stays resilient when the cloud fails: 1. Multi-Region Deployment Deploy across multiple AWS regions (US-EAST-1 + US-WEST-2). If one fails, traffic automatically routes to another. 2. Multi-Cloud Strategy Don’t put all eggs in one basket. Distribute critical workloads across AWS, Azure, and GCP. 3. Robust Monitoring Monitor everything. Use third-party tools, not just provider monitoring. Get alerts before customers complain. 4. Graceful Degradation Design systems to operate in reduced capacity mode. If authentication fails, allow cached credentials temporarily. 5. Database Resilience Replicate databases across regions. Test your failover regularly — untested backups are just hopes. 6. DNS Redundancy Use multiple DNS providers. DNS failures were a root cause of this outage. 7. Disaster Recovery Plan Document runbooks, define RTOs/RPOs, and conduct regular DR drills. Can you restore your app in a different region in under 1 hour? 8. Map Dependencies Know what depends on what. If AWS US-EAST-1 went down right now, do you know exactly what would break? 9. Status Page Keep customers informed during outages. Transparency builds trust. 10. Start Small You don’t need everything at once. Start with: • Dependency mapping • Monitoring & alerting• One backup region for critical services • Test your DR plan Final Thought 💭 The AWS outage reminded us that the cloud is not infallible. No matter how reliable your provider claims to be (AWS has 99.99% uptime SLA), outages will happen. The question isn’t if the next outage will occur, but when — and whether your business will be ready. What’s your organization doing to prepare for cloud outages? Share your strategies in the comments! 👇 #CloudComputing #AWS #DisasterRecovery #BusinessContinuity #DevOps #CloudResilience #SRE #TechStrategy #Infrastructure

Last month’s massive outage in AWS US East 1 was a reminder of something we all know but rarely act on: regions fail. Services disappear. Control planes become unreachable. And when that happens, most systems discover, too late, that their ingestion, indexing, or materialized view pipelines were never built for real world failure modes. Every Mach5 Software, Inc. customer has at least one deployment in US East 1. Not a single one experienced 𝗱𝗮𝘁𝗮 𝗹𝗼𝘀𝘀 during the outage. That was not luck. It was the result of two design decisions we made very early on, and I think they represent principles every modern data system should adopt. 𝟭. 𝗗𝘂𝗿𝗮𝗯𝗹𝗲 𝗱𝗮𝘁𝗮 𝗺𝘂𝘀𝘁 𝗹𝗶𝘃𝗲 𝗶𝗻 𝗼𝗯𝗷𝗲𝗰𝘁 𝘀𝘁𝗼𝗿𝗮𝗴𝗲, 𝗻𝗼𝘁 𝗲𝗽𝗵𝗲𝗺𝗲𝗿𝗮𝗹 𝗰𝗼𝗺𝗽𝘂𝘁𝗲. All indexed data, segments, and commit records in Mach5 are stored directly in object storage. Even when S3 itself became temporarily unavailable, the invariant held: once data is written and acknowledged, it stays written. Local disks, caches, or in-cluster replicas cannot make that guarantee under regional disruption. Object storage can. 𝟮. 𝗜𝗻𝗴𝗲𝘀𝘁𝗶𝗼𝗻 𝘀𝘁𝗮𝘁𝗲 𝗮𝗻𝗱 𝗱𝗮𝘁𝗮 𝗰𝗼𝗺𝗺𝗶𝘁𝘀 𝗺𝘂𝘀𝘁 𝗯𝗲 𝗮𝘁𝗼𝗺𝗶𝗰. Our transaction protocol commits two things together: • the data you just indexed • the exact source tracking state used to produce it This gives you 𝗲𝘅𝗮𝗰𝘁𝗹𝘆 𝗼𝗻𝗰𝗲 𝗶𝗻𝗴𝗲𝘀𝘁𝗶𝗼𝗻 and 𝗳𝘂𝗹𝗹𝘆 𝗿𝗲𝗰𝗼𝘃𝗲𝗿𝗮𝗯𝗹𝗲 𝗺𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝘃𝗶𝗲𝘄𝘀, even when ingestion pipelines crash mid flow or an entire region stalls. After the outage, deployments simply resumed from the last committed point with no duplicates, no gaps, and no drift between indexed data and source. Cloud reliability is not about avoiding outages. It is about engineering systems that remain correct when outages happen. If you are building a modern data platform, whether search, analytics, pipelines, or warehousing, two principles matter more than any other: • Use object storage as the source of truth. • Treat data and ingestion metadata as a single atomic unit. We learned these lessons the hard way, long before Mach5 existed. I am glad we built them into the system from day one. If you want to dive deeper into the mechanics, happy to share more.

𝗔𝗰𝗵𝗶𝗲𝘃𝗶𝗻𝗴 𝗛𝗶𝗴𝗵 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗼𝗻 𝗔𝗪𝗦 𝗪𝗶𝘁𝗵𝗼𝘂𝘁 𝗕𝗿𝗲𝗮𝗸𝗶𝗻𝗴 𝘁𝗵𝗲 𝗕𝗮𝗻𝗸 99.99% uptime is the goal, but the cost can quickly spiral out of control. • 𝗔𝗪𝗦 𝗥𝗲𝗴𝗶𝗼𝗻𝘀 𝘃𝘀. 𝗔𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗭𝗼𝗻𝗲𝘀: Not every service requires the expense of multi-region failover. Evaluate your RTO/RPO requirements and risk tolerance within the AWS region to determine the appropriate level of redundancy. • 𝗚𝗿𝗮𝗰𝗲𝗳𝘂𝗹 𝗙𝗮𝗶𝗹𝘂𝗿𝗲 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴: Implement circuit breakers and retry mechanisms with AWS services like API Gateway and Step Functions to mitigate transient issues and prevent cascading failures. • 𝗖𝗼𝘀𝘁-𝗘𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗥𝗲𝗱𝘂𝗻𝗱𝗮𝗻𝗰𝘆: Explore the use of Spot Instances and Reserved Instances (RIs) to achieve redundancy within an AWS region at a fraction of the cost compared to multi-region deployments. • 𝗟𝗲𝘃𝗲𝗿𝗮𝗴𝗲 𝗔𝗪𝗦 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀: Consider serverless computing options like AWS Lambda and AWS Fargate, which offer inherent scalability and high availability within an AWS region without extensive infrastructure management. 𝗣𝗿𝗼 𝗧𝗶𝗽: Utilize AWS Route 53 weighted routing to seamlessly balance traffic across your cost-effective failover configurations within an AWS region, ensuring a smooth user experience during disruptions. By carefully considering these strategies, you can build highly available systems on AWS that meet your business needs while optimizing your cloud spending. #AWS #awscommunity