Jumbo Frames - Understanding, Building and Configuring
The course is part of this learning path
In this lecture we will discuss in detail the theory behind Jumbo Frames. We will study the internals and structure of Ethernet Frames and examine how Jumbo Frames when enabled effect the frame sizing to our advantage. We will review use cases and scenarios where Jumbo Frames are useful. We will review limitations of Jumbo Frames - highlighting invalid configurations such as Internet traversing network paths.
- [Instructor] So, what exactly are jumbo frames? Jumbo frames are defined by the MTU. Let's start by defining what an MTU is. The Maximum Transmission Unit, MTU, of a network connection is the size in bytes of the largest permissible packet that can be passed over the connection. The larger the MTU of a connection, the more data that can be passed in a single packet. Ethernet packets consist of the frame or the actual data you're sending and the network overhead information that surrounds it. Ethernet frames can come in different formats and the most common format is the standard Ethernet version two frame format. It supports 1,500 MTU which is the largest Ethernet packet size supported over most of the internet. The maximum supported MTU for instance depends on its instance type. All Amazon EC2 instance types support 1,500 MTU and many current instance sizes support 9,000 MTU or jumbo frames. An Ethernet frame as observed at layer two is composed of three main parts. One, the MAC header, 14 bytes in size, and contains the source and destination MAC addresses as its first two fields. The MAC header is completed with the Ether type field, two bytes in size. Two. The middle section of the Ethernet frame is payload data which includes any headers for other protocols, for example the internet protocol or IP in short form. Three, the Ethernet frame is finalized with a frame check sequence which is a four byte cyclic redundancy check used to detect any in-transit corruption of data. If we now look closely at the structure of a standard Ethernet frame, we can see that by definition its payload data section can range from 46 bytes up to 1,500 bytes. In comparison, the payload data section for a jumbo Ethernet frame can range from 46 bytes all the way up to 9,000 bytes. This is the key differentiator between standard Ethernet frames and jumbo Ethernet frames, the ability to carry up to six times more data in the payload data section. As earlier mentioned, the key differentiator between standard Ethernet frames and jumbo Ethernet frames is the size of the payload section. The maximum amount of data that can be carried in the payload section of a standard Ethernet frame is 1,500 bytes whereas for a jumbo frame, this value extends out to 9,000 bytes. An Ethernet frame uses a fixed size header. The header contains no user data and is overhead. Transmitting a larger frame is more efficient because the overhead to data ratio is improved. Utilizing jumbo frames can significantly improve network performance by providing less overhead and a higher payload capacity. As can be seen here, the jumbo frame has less than six times the overhead when compared to the standard frame and six times more payload. This enhanced payload capacity results in better and more efficient channel utilization. Let's now look at what a standard Ethernet frame looks like on the wire. The following screenshot is of a TCP dump packet capture file opened within Wireshark. We can see the following key points. The network configured MTU is 1,500 and hence the frame length for each captured frame is 1,512 bytes. The math behind this value is as follows, 14 bytes for the Ethernet header, the payload is 1,500 bytes which breaks down into 20 bytes for the IP header, 1,478 bytes for ICMP which itself breaks down into 16 bytes of ICMP header as this is an ICMP type eight message, leaving a remainder of 1,462 bytes for the data section. Taking a similar view now of how a jumbo Ethernet frame looks in the wire, you can see the following key points. The network configured MTU is 9,000 and hence the frame length for each captured frame is 9,012 bytes. The math behind this value is as follows, 14 bytes for the Ethernet header. The payload is now 9,000 bytes which breaks down into 20 bytes for the IP header, 8,978 bytes for ICMP which itself breaks down into 16 bytes of ICMP header for ICMP type eight message, leaving a remainder of 8,962 bytes for the data section. AWS supports jumbo frames with 9,000 MTU for the following EC2 instances. All other EC2 instances provide standard support of 1,500 MTU. Compute optimized, C3, C4, CC2. General purpose, M3, M4, T2. Accelerated computing, CG1, F1, G2, P2. Memory optimized, CR1, R3, R4, X1. Storage optimized, D2, HI1, HS1, I2, I3. As you can see in this diagram, using jumbo frames inside of VPC and such that traffic is optimized for where it's directed, you can either configure the MTU size by route or use multiple ENIs with different MTU sizes and different routes. We'll dive into this type of solution in more detail in the demonstration that we perform towards the end of this course. Finally, please note Direct Connect Connections, VPN connections and traffic sent out to an internet gateway currently do not support jumbo frames and are limited to 1,500 MTU. Certain use cases make sense for enabling jumbo frames. Make sure to use jumbo frames when you have a dedicated private network and you can configure an MTU of 9,000 on all equipment. Server clustering and backups where large amounts of data are being moved around on private networks fully under your control are ideal for jumbo frames as larger MTUs permit faster backups. Distributed file system protocols like NFS are good candidates for leveraging jumbo frames, 9,000 byte MTU to carry an 8192 NFS data block. For EC2 instances that are co-located inside a placement group, jumbo frames help to achieve the maximum network throughput possible. The largest frame supported by most of the internet today is 1,500 MTU. Larger frames consume extra Ethernet-linked transmission time. These can introduce more delay for packets that follow causing latency. Voice over IP applications are sensitive to this latency and as such are not good candidates for jumbo frames. As previously mentioned, AWS Direct Connect Connections, AWS VPN IPSec Connections, and traffic sent out to an internet gateway currently do not support jumbo frames and they're also limited to 1,500 MTU. Therefore, it's important to consider the network path as initiated from within your VPC as to whether jumbo frames are suitable.
Jeremy is a Content Lead Architect and DevOps SME here at Cloud Academy where he specializes in developing DevOps technical training documentation.
He has a strong background in software engineering, and has been coding with various languages, frameworks, and systems for the past 25+ years. In recent times, Jeremy has been focused on DevOps, Cloud (AWS, GCP, Azure), Security, Kubernetes, and Machine Learning.
Jeremy holds professional certifications for AWS, GCP, Terraform, Kubernetes (CKA, CKAD, CKS).