High Energy Physicists Set New Record for Network Data Transfer
The record-setting demonstration was made possible through the use of twelve 10 Gbps links to SC08 provided by SCInet, CENIC, National Lambda Rail, Pacific Wave and Internet2, together with two fully populated Cisco 6509E switches, 10 gigabit Ethernet network interfaces provided by Myricom and Intel, two fiber channel S2A9900 storage platforms provided Data Direct Networks equipped with 8 Gbps host bus adapters from QLogic along with five X4500 and X4540 disk servers from Sun Microsystems. The server equipment consisted of 32 twin motherboards Supermicro systems using dual quad-core Intel Xeon processors.
Monalisa: 110 Gbps (114 Gbps peak)
SCInet Graphs
100G test with Ciena
Second major milestone was achieved by the HEP team working together with Ciena, who had just completed its first OTU-4 (112 Gbps) standard link carrying a 100 Gbps payload (or 200 Gbps bidirectional) with forward error correction. The Caltech and Ciena teams used an optical fiber cable with ten fiber-pairs linking their neighboring booths, Ciena’s system to multiplex and demultiplex ten 10 Gbps links onto the single OTU-4 wavelength running on an 80 km fiber loop, and some of Caltech’s nodes used in setting the wide area network records together with FDT, to achieve full throughput over the new link.
Thanks to FDT’s high throughput capabilities, and the error free links between the booths, the teams were able to achieve a maximum of 199.90 Gbps bi-directionally (memory-to-memory) within minutes of the start of the test, and an average of 191 Gbps during a 12 hour period that logged the transmission of 1.02 Petabytes overnight.
Before dismantling the exhibit at the end of the conference, Caltech and DDN worked together to quickly reach 69.3 Gbps over the fiber cable, limited by the disk speed and the kernel, reaching 92% of the full throughput capacity of the DDN platforms. The team expects to be able to reach 100% of the storage platforms’ capacity with further kernel-tuning.
