New statistical method identifies anomalies in datasets

Researchers at the University of Vienna have developed a statistical method that can be used to identify hidden anomalies in scientific datasets. Described in the journal Patterns, the method can be applied automatically to large datasets, making it easier for researchers to identify problematic datasets in a targeted manner and thus improving the quality of data-driven research.

Modern scientific methods increasingly rely on large volumes of data. In modern drug discovery, for example, millions of experimental measurements are often collated from databases and fed into AI models to aid the discovery and development of new medicines. The quality of this data is crucial, as errors in the collection, processing or integration of data not only slow down subsequent analyses, findings and research processes; they also render them futile. However, the systematic verification of large datasets poses considerable challenges for researchers.

Uday Abu-Shehab, Matthias Welsch and Johannes Kirchmair, from Vienna’s Christian Doppler Laboratory for Molecular Informatics in the Biosciences, set out to develop a method capable of automatically identifying anomalous datasets, based on what is known as Benford’s Law. This law describes the observation that, in many naturally occurring datasets, certain digits occur more frequently as the first digit than others. If datasets deviate substantially from this pattern, this may indicate irregularities or potential quality issues.

“Our method does not provide direct evidence that data is flawed or falsified,” Kirchmair noted. “Rather, it helps to filter out datasets from large collections that should be subjected to closer scrutiny. This enables researchers to focus their attention specifically on potentially problematic datasets.”

To enable the prioritisation of datasets, the researchers developed the Simulation-Based Benfordness Estimation (SBBE) method. The approach combines extensive computer simulations with Bayesian statistics and provides not only an assessment of data quality but also an estimate of the associated uncertainty. This makes it possible to compare datasets more effectively than with previous methods.

“With many existing approaches, the results of quality analyses depend heavily on the number of available data points,” Abu-Shehab explained. “Our method explicitly takes this uncertainty into account, thereby enabling fair comparisons between datasets of different sizes. This makes the results substantially more robust.”

To assess the effectiveness of the approach, the researchers tested it using bioactivity data, which describes the strength of interaction between bioactive compounds and their target proteins and is essential in modern drug discovery. The analysis showed that large bioactivity databases generally exhibit high data quality, but certain datasets exhibit unusual statistical characteristics. The researchers were ultimately able to attribute these to particularities in the experimental design, data processing or data curation.

As SBBE is not limited to any specific field of research, the authors see potential for numerous further applications. Wherever large volumes of numerical data are generated, the method could help to reveal hidden irregularities and strengthen the basis for well-founded scientific decisions, as well as promoting the development of reliable AI applications.

“Tools for automated quality control are becoming increasingly important, particularly for data-driven research,” Welsch said. “Our aim is to provide researchers with an easy-to-use tool to identify anomalies at an early stage and further improve the quality of scientific data collections.”

Image credit: iStock.com/Sezeryadigar