Extending Rucio with modern cloud storage support
Rattachement africain : ch, gb, us. Niveau de preuve : code pays fourni par la source.
Le résumé fourni par la source
Rucio is a software framework designed to facilitate scientific collaborations in efficiently organising, managing, and accessing extensive volumes of data through customizable policies. The framework enables data distribution across globally distributed locations and heterogeneous data centres, integrating various storage and network technologies into a unified federated entity. Rucio offers advanced features like distributed data recovery and adaptive replication, and it exhibits high scalability, modularity, and extensibility. Originally developed to meet the requirements of the high-energy physics experiment ATLAS, Rucio has been continuously expanded to support LHC experiments and diverse scientific communities. Recent R&D projects within these communities have evaluated the integration of both private and commercially-provided cloud storage systems, leading to the development of additional functionalities for seamless integration within Rucio. Furthermore, the underlying systems, FTS and GFAL/Davix, have been extended to cater to specific use cases. This contribution focuses on the technical aspects of this work, particularly the challenges encountered in building a generic interface for self-hosted cloud storage, such as MinIO or CEPH S3 Gateway, and established providers like Google Cloud Storage and Amazon Simple Storage Service. Additionally, the integration of decentralised clouds like SEAL is explored. Key aspects, including authentication and authorisation, direct and remote access, throughput and cost estimation, are highlighted, along with shared experiences in daily operations.
Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.
Le contrôle bibliographique ouvert
DOI retrouvé dans Crossref DOI retrouvé ; titre concordant.
- Titre Crossref
- Extending Rucio with modern cloud storage support
- Date Crossref
- 01/01/2024
- Éditeur
- EDP Sciences
- Type
- journal-article
Ce recoupement confirme des métadonnées liées au DOI. Il ne confirme ni la méthode ni les conclusions de l’étude, et il ne compte pas comme une seconde source scientifique indépendante.
Où se fait cette recherche
-
European Organization for Nuclear Research pays non établi dans la noticeStructure de recherche
-
SKA Observatory pays non établi dans la noticeStructure de recherche
-
The University of Texas at Austin pays non établi dans la noticeUniversité ou école supérieure
-
Brookhaven National Laboratory pays non établi dans la noticeStructure de recherche
-
University of Edinburgh pays non établi dans la noticeUniversité ou école supérieure
-
Square Kilometre Array Observatory (SKAO) pays non établi dans la noticeInstitution
-
University of Texas at Austin (UTA) pays non établi dans la noticeUniversité ou école supérieure
European Organization for Nuclear Research, SKA Observatory et The University of Texas at Austin, avec 4 autres affiliations.
Une affiliation ne permet pas de déduire la nationalité d’un auteur.