Data quality improvement for ocean in-situ observations: quality control and duplicate checking

Enhancing Data Quality in Ocean In-Situ Observations

Over the past century, over 20 million varied-quality in-situ profiles were collected, creating a need for an efficient, accurate, automatic quality control algorithm to detect the outliers and remove the duplicates.

In order to deepen this topic and understand its root causes, the Blue-Cloud team organised a webinar on 25 May 2023 with Prof. Zhetao Tan. In this talk, Tan first shows a new automatic quality control system (called CODC-QC) to check ocean temperature and salinity profiles. This system doesn’t assume any data distribution and uses a time-varying threshold to minimise the possibility of mistakenly flagging good data. It works better on different ocean instruments. Using CODC-QC shows more stronger ocean warming rates and helps improve ocean science. Besides, the talk also shows a duplicate checking algorithm (called ‘DNA’ method) to detect potential duplicates in the WOD with more than 20 million profiles, creating a benchmark dataset with duplicate/non-duplicate flags and identifying the physical reasons behind duplication. All these can help improve the data quality of ocean in-situ observations.

Blue-Cloud EOV Workbenches

In Blue-Cloud 2026, advanced Workbenches (WBs) are being developed to streamline data handling for Essential Ocean Variables (EOV). These tools enhance data harmonisation, validation, and qualification from various sources. These novel Workbenches leverage cloud tech, AI, and advanced analytics for improved data handling. They are set to be adopted by key initiatives, consistently producing validated data collections, enhancing oceanic simulations. The EOV Workbenches cover Physics, Eutrophication, and Ecosystem-level Variables.

Would you like to watch the recording? Contact us!