Overrepresentation Bias Leads to Performance Overestimation in Blood-Brain Barrier Permeability Prediction Models: Characterization and Mitigation.

Overview

1. Biomedical Data Science Laboratory (BDSLab), Instituto de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València (UPV), Camí de Vera s/n, València 46022, Spain

Abstract

Recent advancements in blood–brain barrier permeability (BBBP) prediction of drug compounds have highlighted the growing role of machine learning, particularly deep learning. While considerable attention has been given to feature engineering and model design, their evaluation often receives insufficient attention despite its fundamental role in model credibility. In this work, we study a phenomenon we term overrepresentation bias, susceptible to be found in drug property databases, characterized by the presence of near-identical compounds with the same or nearly identical property values. Our findings reveal that overrepresentation bias leads to overly optimistic performance estimates in BBBP prediction models by significantly inflating test evaluation metrics13.3% in average for the area under curve and 16.44% in average for the macro F1-score. To address this bias, we propose (i) an automatic detection algorithm and (ii) a bias-aware data handling procedure. We recommend adopting this approach to ensure more reliable model evaluations. Given that overrepresentation bias can affect performance estimation more than feature selection, model architecture, or even training data, we urge both academic and industrial communities to acknowledge its significance and take proactive measures to identify and address this bias in future studies.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

The paper links to its data, not to its authors' code: see the Data section.

The paper's code and data availability statement is in the Data section.

Tracing map

A tracing map links a paper to the code its authors published: this paper has none, so it has no map.

Data

Datasets cited

• github.com/bdslab-upv/bbbp-overrepresentation-bias — at github.com; found in “Data Availability Statement”

Data Availability Statement

The source data, along with the processed data presented in this study, can be found at our GitHub repository (https://github.com/bdslab-upv/bbbp-overrepresentation-bias) (https://github.com/bdslab-upv/bbbp-overrepresentation-bias). The code for both the overrepresentation-aware data splitting and the automatic approximate collision threshold calculation is available at our GitHub repository (https://github.com/bdslab-upv/bbbp-overrepresentation-bias).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

• Publisher: — → American Chemical Society

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 6 MeSH terms, 1 funder, 52 references.

Cite

This paper

Ferri, P., & García-Gómez, J. M. (2026). Overrepresentation Bias Leads to Performance Overestimation in Blood-Brain Barrier Permeability Prediction Models: Characterization and Mitigation. Journal of chemical information and modeling, 66(12), 6837-6854. https://doi.org/10.1021/acs.jcim.5c02891

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.3390/pharmaceutics18060670

Predicting Blood-Brain Barrier Permeability from Experimental Data: An Interpretable and Externally Validated Machine Learning Framework.

Journal: Pharmaceutics

In common: methods / tools, 4 references

[2] doi:10.34133/csbj.0036 [code]

HYG-mol: An Interpretable Multimodal Hypergraph Framework for Molecular Property Prediction.

Journal: Computational and structural biotechnology journal

In common: 4 references

[3] doi:10.1016/j.namjnl.2026.100121 [code]

Predicting blood-brain barrier permeability of chemicals by machine learning modeling.

Journal: NAM journal

In common: 4 references

[4] doi:10.3390/ph19081319 [code]

DeepBBB: A Data-Composition-Aware Graph Screening Workflow for BBB-Focused CNS Library Construction and Prospective PAMPA-BBB Evaluation.

Journal: Pharmaceuticals (Basel, Switzerland)

In common: 3 references

[5] doi:10.3389/fonc.2026.1834400

BrainFusionNet: an attention-augmented deep convolutional framework with hybrid loss optimisation and test-time augmentation for multi-class brain tumour detection in magnetic resonance images.

Journal: Frontiers in oncology

In common: methods / tools, 3 references

[6] doi:10.34133/csbj.0184 [code]

Cross-Species Multitask Learning with Molecular and ADME Descriptors for Liver Microsomal Metabolic Stability.

Journal: Computational and structural biotechnology journal

In common: 2 references

[7] doi:10.1002/mrm.70431 [code]

SelExNet: A Self-Supervised Physics-Informed Framework for Multi-Channel Joint RF and Gradient Waveform Optimization in 2D Spatially Selective Excitation.

Journal: Magnetic resonance in medicine

In common: 2 references

[8] doi:10.1016/j.isci.2026.116355 [code]

Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion.

Journal: iScience

In common: 2 references

[9] doi:10.3389/fnins.2026.1837233

Integrated multi-omics and deep learning analysis reveals neurotransmitter metabolism regulatory mechanisms of Tianwang Buxin Dan.

Journal: Frontiers in neuroscience

In common: 2 references

[10] doi:10.1016/j.molcel.2026.07.006 [code]

DeorphaNN: Virtual screening of GPCR peptide agonists using AlphaFold-predicted active-state complexes and deep learning embeddings.

Journal: Molecular cell

In common: 2 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.