10 Gigabit Ethernet
Once an exotic, high end technology, 10 GbE is now mainstream and widely deployed both in the datacenter and in the campus environment. 10 gigabit Ethernet, however, has had a fairly lengthy process of development to get to where it is now, and with many older 10GbE platforms and modules still in use, there are a number of different optics types still seen in use today, both on legacy and current production platforms.
From left to right:
XENPAK – the original 10GbE pluggable optics. Presents SC connectors
X2 – the successor to the XENPAK. X2 optics had limited use in the market, with Cisco being the only major user of X2 based products. Presents SC connectors
XFP – the first of the small form factor 10GbE optics. XFP optics were primarily adopted in router products, and are still almost exclusively found in routers. Presents LC connectors
SFP+ – a 10GbE optics using the same physical form factor as a gigabit SFP. Because of this, many of the small SFP+ based 10GbE switches use 1G/10G ports, giving an added degree of flexibility. SFP+ optics have become, by far, the most commonly used of all 10 gigabit capable optics. Presents LC connectors
Within these form factors are many different types of optical and electrical specifications; the only requirement is that the optics type match. It is perfectly acceptable to connect an X2 to an SFP, or a XENPAK to an SFP+, or any other combination.
10GBase-CX1, also referred to as a direct attach cable (DAC), is the SFP+ copper standard. The standard has a maximum distance of 10 meters, though Cisco currently only offers lengths up to 5m. This is actually a cable with SFP+ ends, not a module with a separate cable.
Notice that the cable is permanently integrated into the SFP+ ends. Because of this, both devices must present SFP+ ports. While the cables are somewhat inconvenient to work with due to this integration, CX1 modules are used due to a very low cost ($150-$200 per cable), extremely low power consumption (0.25W per cable), and a negligible latency penalty. Because of these characteristics, DACs are used primarily for connectivity from servers to the top of rack switch, or for interconnects between devices in the same rack.
One of the major goals for 10GbE was to develop a connectivity standard suitable for use with standard Ethernet cabling and RJ45 ends. The 10GBase-T standard emerged from this, and within the last few years, advances in technology have allowed for 10GBase-T interfaces to be deployed with the same density as SFP+ based products. Like 10GBase-CU, the majority of 10GBase-T deployments are server to top of rack switch.
Late in 2014, an extension to 10GBase-T was proposed, called NBase-T, and in mid-2015, NBase-T switching has entered the market. NBase-T adds two intermediate speeds, 2.5 and 5Gbps, to the available speeds supported. Driving factors behind NBase-T are compatibility with existing Cat5e (2.5G maximum) and Cat6 (5G maximum) cabling plants at the standard maximum distance of 100m while being able to support data rates beyond 1Gbps for power users and high-performance 802.11ac (and beyond) wireless access points.
Due to power demands, there are currently no pluggable 10GBase-T or NBase-T SFP modules; all of the current products on the market are fixed interface only.
10GBase-SR is the original multimode optics specification, and is still by far the most commonly used. As it uses a single, low cost solid state laser assembly operating at 850nm, it is also the least expensive of the optical modules available for a 10GbE platform. However, 10GBase-SR is very sensitive to fiber type. Below is a list of cable specs and maximum distance with SR optics.
|62.5||160||26m||This is standard multimode fiber.|
|50||2000||300m||AKA: OM3 or 10GbE-optimized fiber|
|Core Size (microns)||Modal Bandwidth (MHz*km)||Max Distance||Notes|
Because of these distance limits, it is highly recommended that any new multimode fiber deployments use OM3 or higher grade fiber to ensure any 10GbE deployments do not run into any issues with distance.
For deployments where longer runs of traditional 62.5 micron multimode fiber are in use and cannot be upgraded, the 10GBase-LRM standard was developed. A single laser operating at 1310nm is used alongside a special mode-conditioning cable to achieve a maximum distance of 220m over 62.5 micron fiber.
10GBase-LR can reach up to 10km over singlemode fiber. There is no minimum distance for LR, either, so it is suitable for short connections over singlemode fiber as well.
10GBase-ER can reach up to 40km over singlemode fiber. Due to the laser power, attenuation is required for links less than 20km long.
ZR optics can reach up to 80km over singlemode fiber. Due to the very high transmit power, significant attenuation is needed for shorter links. Use of ZR optics should be preceded with an optical power test of the fiber span in question to ensure a problem-free deployment. Interestingly, 10GBase-ZR is actually not an IEEE standard, though most vendors offer a ZR option.
10G over DWDM
With 10GbE, it is possible to get optics modules that output at DWDM wavelengths, allowing for much simpler DWDM deployments, and with these optics no additional transponder hardware is required. Current innovations in 10GbE/DWDM optics include full C-band tunability for ease of sparing, ordering, and provisioning of DWDM channels, as well as features like forward error correction (FEC) and OTN/G.709 support integrated directly onto the optic.
40 and 100 Gigabit Ethernet
The QSFP 28 module uses the same physical form factor as the QSFP+ optics for 40GbE, but supports 4x 25Gbps lanes, allowing for 100Gbps of total throughput. QSFP 28 is rapidly becoming the standard for 100GbE over multimode fiber, as the small form factor allows for high port density as needed by datacenter switching and the use of 4 lane optics assemblies allows for the upgrade of any 40GBase-SR4 systems to 100GbE without needing to upgrade the physical plant.
Looking forward – 25 and 50 Gigabit Ethernet
In high-performance datacenter environments, the need for network performance at the server level is already straining 10GbE connectivity, especially once new technologies like object based storage and virtual SANs are added to the mix. To address this, the 25 and 50 gigabit Ethernet standards were created, and are just now arriving on the market. These standards use the same underlying technologies first developed for the 100GBase-SR4 and LR4 standards. A 25GbE connection uses a single 25Gbps lane, and 50GbE uses a pair of 25Gbps lanes. Initial deployments of 25 and 50GbE use a QSFP 28 optic with breakout cables – for 25GbE, these are the same cables used to convert a 40GbE interface to 4x 10GbE. Eventually, 25GbE will be available in an SFP+ form factor, while 50GbE will retain the QSFP form factor.
If you liked what you read in this tech guide, you might also enjoy this blog post.