Report: AWS, Azure and GCP evaluated for overall performance — which is best?

An annual report produced by Cockroach Labs finds a statistical dead heat among the big three cloud providers. The report evaluates the performance of AWS, Azure and GCP for common Online Transaction Processing (OLTP) workloads. Testing shows that all three cloud providers offer price-competitive options as well as top-tier performance for OLTP applications, suggesting a shift in the market as the industry matures operationally and moves to standardize linear pricing models.

This year, the report benchmarked 56 instance types and 107 discrete configurations and conducted 3,000+ runs across the following broad axes, including OLTP performance, CPU performance, network performance and storage I/O Performance. Additionally, this year’s report improved test design configuration for all tests and added testing for different-sized instance types; cross-region latency; storage tests with fsync; variable OLTP workload complexity; and 12x longer throughput test duration, from 60 seconds per test previously to 12 minutes per test.

The report reveals that AMD beats Intel on overall performance: machines with AMD Milan (EPYC Gen 3) processors claimed the top spots in both CPU benchmarking and OLTP testing for both large and small instance types. In past years, Intel led the pack in overall performance, with AMD competing on price-for-performance metrics. This year, both the overall performance leader and the price-for-performance leader were AMD-based instances.

Furthermore, all three clouds have price-competitive offerings: when it comes to price (and performance) all three clouds were in a statistical dead heat. Even instance and storage combinations that are a bit more expensive are potentially very competitive depending on the requirements of a specific workload.

Finally, storage and transfer costs have an outsized impact on total cost to operate: “hidden” costs of storage and data transfer can have a larger impact on total cost than the price of the instances themselves, especially when it comes to building a highly resilient stateful application.

Read the full report by Cockroach Labs.