SuperComputing 2012
Enabling Scientific Discoveries with the LHC Data Distribution Over 100 Gigabit Networks

Tools used in SC '12

FDT - One of the key advances in this demonstration was Fast Data Transport (FDT; http://monalisa.cern.ch/FDT), an open source Java application developed by the Caltech team in close collaboration with the Polytehnica Bucharest team. FDT runs on all major platforms and uses the NIO libraries to achieve stable disk reads and writes coordinated with smooth data flow across long-range networks. The FDT application streams a large set of files across an open TCP socket, so that a large data set composed of thousands of files, as is typical in high-energy physics applications, can be sent or received at full speed, without the network transfer restarting between files. FDT works with Caltech's MonALISA system to dynamically monitor the capability of the storage systems as well as the network path in real-time, and sends data out to the network at a moderated rate that achieves smooth data flow across long range networks.

MonALISA - MonALISA, stands for Monitoring Agents using a Large Integrated Services Architecture, has been developed by Caltech and its partners with the support of the U.S. CMS software and computing program. The framework is based on Dynamic Distributed Service Architecture and is able to provide complete monitoring, control and global optimization services for complex systems.
The MonALISA system is designed as an ensemble of autonomous multi-threaded, self-describing agent-based subsystems which are registered as dynamic services, and are able to collaborate and cooperate in performing a wide range of information gathering and processing tasks. The system is designed to easily integrate existing monitoring tools and procedures and to provide this information in a dynamic, customized, self describing way to any other services or clients. http://monalisa.caltech.edu

RDMAs baed FTP Service - RFTP is a novel data transfer tool for high performance networks. It uses IB verbs through the InfiniBand, RoCE, and iWARP provided by InfiniBand Verbs library (libibverbs) and RDMA communication manager (librdmacm). RFTP is availble in RHEL/CentOS now. For more information visit: ftp100.cewit.stonybrook.edu/rftp/.