Taking a look at benchmark results for AWS instances
In this post, we take a look at benchmark results for both single and multi core CPUs on AWS. We take an extract of published benchmark data provided by geekbench via their browser for CPU benchmark data to take a closer look at the performance characteristics of a select few machine types.
Single Core Performance of Burstable Machines (t3 Instances)
AWS t3 machines are defined as machines with burstable compute capabilities, that acquire compute credits while your utilization of the machine is low. The length of time you are able to perform burst operations on the machine is defined by the type, starting at 5% of baseline CPU for a t3.nano and working up to 40% for a t3.xlarge machine.
For single core performance, we take a look at the following machine types:
- t3.small (66 machines, avg 723)
- t3.medium (236 machines, avg 730)
- t3.large (111 machines, avg 642)
- c5.large (249 machines, avg 765)
- c5.xlarge (118 machines, avg 908)
This plot takes a look at the single core characteristics of t3 machines vs c5.large and c5.xlarge machines.
The error bars represented in the plot above represents 1 standard deviation from the mean represented across each machine type. For this particular sample of machines, we see that the average CPU score is around 698 if we average across the t3 CPU types. The t3.large machine appears to have lower compute power, but has a wider range of variance. The c5.large machine seems to have better performance compared to the t3 machines and the c5.xlarge machine is significantly better compared to the c5.large machine by 18.7%.
I've taken a look at the CPU specs for the t3 vs c5 machines while leads to the following information:
- t3 CPU: Up to 3.1 GHz Intel Xeon Platinum Processor
- c5 CPU: Custom 2nd generation Intel Xeon Scalable Processors (Cascade Lake) with a sustained all core Turbo frequency of 3.6GHz and single core turbo frequency of up to 3.9GHz
It appears that both the t3.small and t3.medium machines are using identical CPUs, t3.large could be using more varied CPU types which is resulting in a lower average with higher variance.
Multi Core Performance Between c5/c5a/m6a Machines
For single core performance, we take a look at the following machine types:
- c5.large (249 machines, avg 765)
- c5.xlarge (118 machines, avg 2047)
- c5a.large (39 machines, avg 1191)
- c5a.xlarge (25 machines, avg 2378)
- m6a.large (2 machines, avg 1537)
- m6a.xlarge (2 machines, avg 2963)
While the number of data points for the m6a machines had only 2 points, we find here that compute scores between the different machine generations ranked in the following manner: c5 < c5a < m6a.
This is expected as the following CPU generations use the following CPU types:
- c5 CPU: Custom 2nd generation Intel Xeon Scalable Processors (Cascade Lake) with a sustained all core Turbo frequency of 3.6GHz and single core turbo frequency of up to 3.9GHz
- c5a CPU: AMD EPYC 7000 series processors with an all core turbo clock speed of 2.5 GHz
- m6a CPU: Up to 3.6 GHz 3rd generation AMD EPYC processors
As of recently the AMD EPYC family of CPUs are more performant on a single threaded basis compared to the Intel Xeon CPUs. Also, as we would expect, the m6a is a faster CPU compared to the m5a.
When comparing the large vs the xlarge machines in terms of multi-core throughput, the AMD CPUs appear to be just shy of 2x performance when compared to the large vs xlarge machines (where the xlarge machines have 4 VCPUs compared to 2 for the large machine). The c5.large machines seem to lag in terms of their multi-core performance, being slightly under 3x slower compared to the c5.xlarge machines and this is consistent compared with the c5.large vs c5.xlarge single core performance – the c5.xlarge machines on average use better CPUs.
A Short Note on Cost
A quick comparison of cost between the different CPU generations (based on US West AZ):
- c5.large: $0.085 / hour
- c5a.large: $0.77 / hour
- m6a.large: $0.0864 / hour
If you are planning on using machines in AWS, it appears that AMD machines will provide the best bang for the buck as for the same CPU tier but an AMD CPU is cheaper than the Intel CPUS and the m6a AMD CPUs are nearly at price parity compared to an Intel CPU at the c5 generation.