Blue-Cloud Data federation approach
With input by EMODnet, Copernicus Marine and several leading marine Research Infrastructures, the Horizon Europe Blue-Cloud is contributing to increasing the available data volume and to an easier discovery and access of blue data, open to researchers, industry and society from Europe and beyond to address issues related to climate change, food, biodiversity conservation, sustainable ocean economy, pollution and natural hazards. The Blue-Cloud data federation ultimately improves the quality of data available thanks to harmonisation, validation, qualification and assessed interoperability of the various sources as provided by each federated BDI.
The Blue-Cloud Data Discovery & Access Service is one of the two main components of the Blue-Cloud technical framework, next to the Blue-Cloud Virtual Research Environment (VRE). It facilitates the discovery and retrieval of marine data sets and data products for external users in stand-alone mode, also interoperable with the Blue-Cloud VRE. The Blue Cloud data sets are managed in blue data infrastructures (BDIs) that are connected to the Blue Cloud service to serve federated discovery and access. These include key EU infrastructures such as Copernicus Marine Service, EMODnet, SeaDataNet, and more.
Access the Blue-Cloud Data Discovery & Access Service
The Blue-Cloud Data Discovery & Access Service - Quality data for marine researchers.
Interview with Dick Schaap, MARIS, Blue-Cloud Technical Coordinator
Supporting marine data discovery and accessibility to enable cross-domain research
There are several research infrastructures or other data services running in Europe that cover a multitude of marine-related sciences, providing specific datasets coming from observations collected with different methods. These infrastructures constitute a diverse world, each looking at a piece of the big picture. To overcome this fragmentation, the Data Discovery & Access Service (DD&AS) has been implemented. This service facilitates the discovery and retrieval of data sets and data products for external users in stand-alone mode, also interoperable with other Blue-Cloud services, such as the Virtual Research Environment (VRE). More than 10 million data sets are managed in blue data infrastructures (BDIs) that are connected to this Blue-Cloud service to serve federated discovery and access. A common interface is provided for the discovery and retrieval of data sets and data products from each of the federated BDIs.
How does the Data Discovery & Access Service work?
Users can access it through an existing Marine ID. A common interface is provided for the discovery and retrieval of data sets and data products from each of the federated BDIs. The interface also includes facilities for mapping and viewing the locations of data sets, as this is part of the query dialogue. Moreover, the interface has a shopping mechanism, facilitating users to compose and submit mixed shopping baskets with requests for data sets from multiple BDIs.
The Blue-Cloud Data Discovery & Access service makes use of web services and APIs, following protocols such as CSW, OAI-PMH, ERDDAP, or otherwise, as provided and maintained by the BDIs. These are used to deploy machine-to-machine interactions for harvesting metadata, submitting queries, and retrieving resulting metadata, data sets and data products.
The query mechanism has a two-step approach:
- The first step enables users to identify interesting data collections, with free search, geographic and temporal criteria as main query operators on a common catalogue including entries from all federated BDIs;
- The second step enables users to drill down per interesting BDI to get more specific data sets at granule level, using again free search, geographic and temporal criteria, but this time at granule level, and including additional search criteria, specific for a selected BDI;
- Finally, users are able to compose and submit shopping requests at the granule level and after processing by the Blue-Cloud data broker to retrieve the data sets by downloading from their MyBlueCloud dashboard.
The Blue-Cloud Data Discovery & Access service helps marine data users:
- to search and discover relevant data sets
- to complete and submit a shopping basket with the required data sets
- to stay informed about the progress of the shopping requests
- to download the data sets once ready for downloading
- to ingest data sets into the VRE data pool for use in VRE applications
It also supports managers of blue data infrastructures in:
- staying informed about the shopping requests and associated users for their repository
- preparing periodic management reports
Access the Blue-Cloud Data Discover & Access Service
Optimising FAIRness of federated Blue Data Infrastructures: webinar
On 6 December, 2023 a webinar part of the Blue-Cloud Training Academy gave more details about the architecture and approach for the Blue-Cloud DD&AS. It provided insight how a few BDIs are managing their networks of data originators in order to provide harmonised and FAIR discovery and access to their collected data resources, using and promoting best practices and standards.
Re-watch the webinar and download the materials