Speaker
Description
Lustre is the de-facto Storage solution in HPC, several Top 500 sites including NERSC, LLNL,ORNL are using or consider to deploy it over new storage hardware technologies such as fast NVMe SSD, 3D Xpoint SSD or proprietary Burst Buffers solutions. During my staying at SLAC for 3 years I deployed a High performance storage solution based on Lustre/ZFS for the US DoE LCLS project, which is the world’s first hard X-ray free-electron laser. The solution is based on a high performance storage prototype using NVMe SSD technology which will be eventually used for the LCLS II evolution where data taking will move from 120Hz to 1MHz, thus the need of even higher performance storage capabilities. Several clients (data reduction pipelines nodes) will need to access data from the DAQ system at a very high rate with locking contention. The prototype I realized is able to saturate 100GB/s with a bunch of clients reading and writing to a limited number of Lustre Object Storage servers and can scale up linearly just adding more storage nodes.
