Description
Nowadays, physics experiments are often a joint effort of many scientists working in different parts of the world. The amount of data collected by the experiments is so large that data are typically stored and analyzed on machines hosted in various computing centers spread around the planet. For these reasons, one of the challenges for modern scientific collaborations is an efficient data distribution and access system. Rucio is a framework originally developed by the ATLAS experiment for data management, access and distribution, which is now used by many more collaborations within the high energy physics domain (CMS, Belle II, Dune) and outside (ESCAPE, SKA, CTA). In the Gravitational Wave (GW) community it is used by the second-generation interferometers LIGO and VIRGO.
The adoption of this solution is being evaluated also for the future third-generation interferometer, the Einstein Telescope (ET). ET will observe a volume of the Universe one thousand times larger than the current interferometers and this will reflect on a larger data acquisition rate. Additionally, the signal candidates will need to be analyzed in the so-called low latency pipelines to produce prompt alerts for multi messenger astronomy. A secure and valid distributed data management system is crucial.
In this contribution, we briefly describe Rucio main features and its use in the current GW experiments. We report on the current R&D setup which includes a test instance of the Rucio server managed by INFN and two storage elements deployed at INFN Torino and UCLouvain computing centers. Finally, we discuss additional work on the customization of Rucio features for GW experiments: the implementation of a POSIX-like filesystem to provide the user with a more familiar structure of the data catalog, and the use of a streaming protocol to support data distribution for low latency pipelines.
