Monroe: A Molecular Model with In-Context Probabilistic Inference ✛
Context
Predicting the biological activity of small molecules is a key step in drug discovery, but bioassay data are expensive and limited. Around this task, a whole field of molecular foundation models has emerged: MiniMol, CheMeleon, MolFormer, and others, which are pretrained on large datasets and then adapted to specific tasks. Monroe joins this line but bets on in-context probabilistic inference instead of fine-tuning, promising more efficient use of limited data.
Methods
Monroe uses a graph transformer with an improved representation of stereochemistry: for E/Z double bonds and R/S chiral centers, auxiliary edges are added to distinguish isomers. The model is pretrained on 81 million molecules from the PM6 set (quantum-chemical properties) and 1.56 million molecules from PCBA (bioassays), totaling 1152 tasks. Losses include conformation denoising and embedding decorrelation. For adaptation to new tasks, TabPFN is used—a model trained on synthetic tabular data that, given molecular embeddings and their labels, produces a probabilistic prediction for a test molecule in a single forward pass.
Results
On 28 Polaris tasks, Monroe wins or is statistically indistinguishable from the best method in 100% of cases, with a mean rank of 3.71. On 30 MoleculeACE tasks, specifically targeting 'activity cliffs' (sharp activity changes with small structural differences), the model achieves the best overall RMSE of 0.629 and the best cliff RMSE of 0.737. Ablations show that key components—conformation denoising, stereochemical edges, decorrelation, and PCBA data—contribute significantly. Furthermore, applying TabPFN to existing models MiniMol and CheMeleon yields new strong variants MiniMolPFN and CheMeleonPFN, confirming the universality of the approach.
Implications
Monroe demonstrates that in-context probabilistic inference can replace fine-tuning for molecular foundation models, reducing computational costs and improving performance on small data. Unlike neighboring works that focus on scaling or new architectures, Monroe emphasizes the adaptation method, showing that even existing models benefit from TabPFN. This opens the way to more flexible and faster methods in cheminformatics.
Future development
Hybrid approaches combining fine-tuning and in-context inference are likely to emerge, as well as extensions to other types of molecular data, such as spectra or images. TabPFN might be adapted to handle non-standard dependency structures in molecular tasks. It is also interesting to explore how to improve activity cliff prediction—this task remains open.
Impact
The method could accelerate virtual screening in drug discovery, allowing faster selection of promising compounds without costly experiments. It is also useful for any molecular property prediction tasks in materials science and agrochemistry.
Next steps
The authors could expand pretraining to even larger datasets and test the model on more diverse benchmarks, including time-series tasks. They should also investigate how to improve conformation denoising, possibly with separate heads or different loss weighting schemes.
Key open problems
For cheminformatics, the problem of predicting activity cliffs—sharp activity changes with minimal structural differences—remains unsolved. Monroe improves results, but the cliff RMSE is still higher than the overall RMSE, indicating fundamental difficulty. Also open is the question of benchmark reliability: known errors in datasets can distort evaluations.
The knowledge machine's view
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
Monroe is a foundational molecular model pretrained on 81 million molecules with quantum-chemical properties and 1.56 million molecules from bioassays. It uses a graph transformer that accounts for stereochemistry and adapts to new tasks via TabPFN without fine-tuning. On the Polaris and MoleculeACE benchmarks, it outperforms similar models, especially in predicting sharp activity changes.
Where its strength is
Its key strength is in-context probabilistic inference via TabPFN, which replaces fine-tuning and allows efficient use of limited data, while also improving existing models.
Where it could go next
- It might be interesting to see how Monroe's approach can be applied to quantum-chemical calculations for large systems where computational efficiency is important.based on: Quantum Chemistry for Hundreds of Thousands of Atoms arXiv:2608.18623v1
- There is a related work on interpretability of AI predictions in chemistry; it may be worth investigating how to explain TabPFN's decisions based on Monroe's embeddings.based on: Transparent Chemistry: How AI Explains Its Predictions arXiv:2603.01274
- The archive contains ideas on efficient learning with smaller data; these could be useful for further improving Monroe's adaptation to small samples.based on: Training Neural Networks on Less, but Smarter arXiv:2602.23269
Worth trying
- Test Monroe with TabPFN on quantum-chemical property prediction tasks for large molecules where speed is critical.
- Use interpretation methods from a related work to analyze which parts of a molecule influence TabPFN's predictions.
- Explore training strategies on smaller data to improve Monroe's efficiency in limited bioassay settings.
Why this may matter
Monroe demonstrates that in-context probabilistic inference can be a universal way to adapt molecular models, which is important for accelerating drug discovery and reducing costs.
Nearby works in our archive
- Quantum Chemistry for Hundreds of Thousands of Atomssimilarity 0.61
- How to Tune a Molecular Motor Without Losing Your Mindsimilarity 0.60
- Neurons Reveal Their Secrets in Minutes, Not Hourssimilarity 0.59
- Neural Network Reads the Future of Plasmasimilarity 0.58
- Transparent Chemistry: How AI Explains Its Predictionssimilarity 0.57
- A Quantum Accelerator for Drug Designsimilarity 0.57
- Training Neural Networks on Less, but Smartersimilarity 0.57
The archive is sparse around this work: the closest review is only 0.61, against a usual 0.64. It looks like few have drilled nearby.
Original abstract
as written by the authors arXiv:2608.18982v1
Bioassay activity prediction is often data-limited because drug-discovery datasets rely on time-consuming and expensive wet-lab experiments for data generation and evaluation. This challenge has inspired recent research into molecular foundation models (MFMs), which aim to encode general-purpose chemical knowledge into molecular representations that generalize well in data-constrained scenarios. This paper presents Monroe, a new MFM with several innovations over the existing state of the art: increased scale allowing pre-training on over 81 million molecules from the PM6 quantum chemistry dataset; improved graph representation of stereochemistry; improved training losses including conformer denoising and embedding decorrelation; improved multi-task learning; and the use of a prior-data-fitted model (TabPFN) for downstream in-context prediction. Our evaluations use a principled pairwise comparison framework that measures statistically significant performance differences. Across established Polaris benchmarks, Monroe matches or exceeds existing MFMs, while on activity cliff benchmarks, designed to assess utility for molecular discovery, it achieves significant improvements over prior methods. Finally, ablation and transfer experiments show that PFN-based downstream predictors also substantially improve two leading existing models, MiniMol and CheMeleon, yielding new state-of-the-art variants we call MiniMol_PFN and CheMeleon_PFN, suggesting that our downstream adaptation strategy generalizes beyond Monroe. Source code is at github.com/blazejba/monroe.
The name Monroe is given in honor of Elizabeth Monroe Boggs, a pioneer in computational quantum chemistry. Pretraining took only 26 hours on four H100 GPUs.
Key numbers
- molecules in pretraining: 81 million
- pretraining tasks: 1152
- GPUs for pretraining: 4 × H100
- pretraining time: 26 hours
- win rate on Polaris: 100%