AWS Fargate is Amazons solution to run containers without managing servers or clusters. For many aspects AWS Fargate is similar to AWS Elastic Container Service, but without having to deal with EC2 clusters. It is basically “server-less containers”.
Let’s take a look at AWS Fargate’s network performance!
TLDR; Sustained network throughput is very stable, but not symmetric and does not really grow with assigned container resources.
AWS Fargatewaslaunchedin Dublin/Ireland (eu-west-1) which is a good opportunity to take a look at some performance characteristics of that service.
There are various articles on AWS EC2 network performance, sometimes between instances, sometimes to other AWS services like S3 (e.g. byAndreas Wittig). For EC2, we know those numbers very well and we regularly check them in order to recommend correctStormForge cluster sizesto our customers.Known network performance is very important in order not to be mislead when doing performance testing.For EC2, raw network performance in terms of bandwidth depends on instance type and more importantly on instance size within one instance family. The general rule of thumb: The bigger the instance (and the more you pay), the better networking gets.
Back to AWS Fargate. We were wondering what kind of network performance can one expect from AWS Fargate? And how does the container’s resource sizing (CPU and memory) relate to its network performance? These might be important parameters to know if you have a bandwidth dependent workload.
Preparation and Test Setup
To asses network performance we are going to useiPerf3. To make the measurement not being limited on the iPerf server part, we are using beefy 72 corec5.18xlargeinstances, which are advertised with up to 25Gibt/s network performance. During preparation, we did sanity checks between twoc5.18xlargeinstances where we saw 22Gibt/s sustained throughput with peaks reaching close to 25Gibt/s. That should provide us with enough headroom for our experiments.
Our test target will be AWS Fargate containers launched ineu-west-1. Fargate allows five tiers of “CPU Units” with a range of memory (checkAWS Fargate documentation). Since we were interested in the relation of network performance and CPU/memory sizing, we decided to test 10 scenarios, the minimum and maximum memory values for each of the five CPU Unit tiers:
CPU Units
vCPU
Memory (MiB)
Price per hour (USD)
256
0.25
512
0.019
256
0.25
2048
0.03805
512
0.5
1024
0.038
512
0.5
4096
0.0761
1024
1
2048
0.076
1024
1
8192
0.1522
2048
2
4096
0.152
2048
2
16384
0.3044
4096
4
8192
0.304
4096
4
30720
0.5834
We also tested the same 10 configurations with iperf’s--reverseoption to verify symmetric network performance, resulting in a total of 20 scenarios.
All test scenarios are configured to omit the first 10 seconds of measurement to skip pastTCP slow-startwindow and also (initial) peak performance. Actual measurements are performed for 60 seconds using two parallel TCP streams. We are primarily interested in sustained network performance, ignoring shot throughput peaks.
Get Started with StormForge
Try StormForge for FREE, and start optimizing your Kubernetes environment now.
The results were quite interesting and quite a bit surprising to us. But without further ado, let’s take a look at the results.
Here is a plot of all tested configurations with the average bandwidth measured over 60 seconds. “Fargate out” refers to traffic being send from AWS Fargate (iPerf server) to our EC2 instance (iPerf client) done via iPerf’s--reverseflag. “Fargate in” is the other direction, without--reverse, sending data from EC2 to the Fargate container.
Here is the same result data:
CPU Units
vCPU
Memory (MB)
Outgoing (MBit/s)
Incoming (MBit/s)
256
0.25
512
268
136
256
0.25
2048
268
136
512
0.5
1024
268
136
512
0.5
4096
625
319
1024
1
2048
268
137
1024
1
8192
455
454
2048
2
4096
3404
319
2048
2
16384
637
636
4096
4
8192
2753
456
4096
4
30720
637
636
Although for most measurements the bandwidth is not super high, the measured bandwidth is very stable over time. For all tests there was an initial peak, but the sustained bandwidth was very solid with little to no variation. This is very nice as this makes it quite predictable.
In addition to the network throughput, we also took a look at the containers CPU utilization (normalized to 100%). This draws a very interesting picture which could at least explain why network performance is not symmetric (in higher then out in all cases):
Conclusions
We are not surprised by the relatively low throughput of the smaller container configurations (this is basically very similar to EC2 network performance). Two aspects strikes us as very odd:
Network throughput on AWS Fargate does not seem to be symmetric.Often there is “just” a 2x difference, but it goes up to over 10x.
Ingress performance of 2048 CPU / 4 GB memory configuration is way off at 3Gbits(same with 4096 CPU / 8 GB memory). And the throughput goes down again when we increased CPU/memory for the container.
This isreallystrange observation. The biggest container you can get with 4096 CPU Units and 30GB memory has roughly the same network performance as the50% cheaper 2048 CPU/16GB option.
The initial assumption was that network performance correlates with allocated resources like with EC2. At first this seems to be the case, until the 2048 CPU Unit tier where things got really odd. Compared to pricing and network performance for EC2 the price per MBit is actually not that bad (at least for ingress).
For a follow-up, one might look into more details like measuring all memory tiers (1GB increments currently), vary the amount of TCP streams in iPerf, etc. There might be configuration tweaking possible to increase network performance overall, but that does not explain the immensely (positive) performance outliers.
Do you have questions or remarks regarding AWS Fargate or other performance topics? Justdrop us a line🙂
More Details
In case you are interested in a bit more detail of the test setup, here you go!
We used an unofficialfargate CLI toolby John Pignata, which has a really nice and simple interface. We had topatch the toolto support the newly available regions, includingeu-west-1which we wanted to use for testing.
We used the followingDockerfileto set up the iPerf3 servers running in Fargate:
FROM alpine:latest
RUN apk --update add iperf3 \
&& rm -rf /var/cache/apk/*
ENTRYPOINT ["iperf3"]
CMD ["--server", "--json", "--verbose"]
Creating a AWS Fargate task based on thisDockerfilewith our desired parameters was very straight forward. This command will build the container image, upload it to your private Docker registry and launch a Fargate task with the given resource configuration:
$ bin/fargate task run iperf-test --region=eu-west-1 --cpu $CPU_UNITS --memory $MEMORY_UNITS
When the container is ready, you can start iPerf from the beefy EC2 instance like this:
After each test, we wait 20 seconds before starting the next experiment. This is go give both systems a bit to cool down.
The iPerf3 client (EC2 instance) and all Fargate containers were started in the same VPC using the same Security Group withineu-west-1region. No other user processes were running on the EC2 instance, besides network monitoring tools for sanity checking.