It is important to keep in mind that all Caltech have done in this instance is
to test some commercial off-the-shelf 40 Gigabit Ethernet Network Interface
Cards (NICs) from Mellanox (specifically the ConnextX-3).
See product brochure here -
http://www.mellanox.com/related-docs...ochure_6.1.pdf
Last year, they had tested the predecessor to the ConnextX-3, the ConnectX-2 NIC,
but due to imitations with the servers they used which were only fitted with PCI
Express 2.0 buses, they could not exploit the full speed of the NICs were only
achieving 24Gbs in each direction.
This year, they upgraded their sever motherboards to those with PCI Express 3.0
compliant motherboards which they then fitted with dual-ported ConnextX-3 NIC's to
achieve the 98Gbps up and the 98Gbps down to achieve the all-round 186Gbps
speeds over a tuned network.
Where they faced some of their challenges were in aspects such as drivers
crashing. So not exactly at the leading edge of communications research
where engineers in labs are transferring data at much higher rates over optical
fibers but at the "bleeding edge" as far as being relatively early adopters
of this commercial technology in a academic WAN environment.
It is also important to keep in mind that 40 Gigiabit Ethernet and 100 Gigabit Ethernet
or 100Gbe is already an IEEE set of standards (IEEE 802.3ba) and that there
are several vendors already providing commercial offerings. If you want a WAN
at the office, data center or home providing these types of speeds, you can go out and
purchase the off-the-shelf-components today.
if you want to operate such a WAN over a distance of more than 10 meters, you will
want to be using optical fibers and the IEEE spec. already allows for 40km links
over single-mode fiber.
For example, some of the Australian ISP's have already upgraded their own
backbone networks to running 100Gbe in preparation for handling the huge data
volumes projected in the near future and for which the NBN is designed to
cater for.
The NBN engineers have specified single-mode fiber connections for the back-haul
(the mid to long distance transport of the data) and the cross-connects so it
is well positioned to exploit corresponding advances in technologies such as NICs.