Ashton Petersen

Previously, I touched on both the capacity of autonomous AI labs to drive research as well as their limitations in bridging the gap between theory and our physical reality. I concluded by stating that AI labs help researchers narrow down possibilities, while the physical world still has the final say. That idea is where I want to begin today. Rather than focusing on whether autonomous AI can make research faster, I want to discuss the more important question of whether its results can be trusted.

There’s little doubt that autonomous research systems can perform complex experiments and help decide the direction of research in scientific domains ranging from materials science to genomics, as noted in a recent technical review in Chemical Reviews. Instead, I think time is better spent analyzing the reliability gap between the results that autonomous AI systems produce that appear precise, mathematical, and trustworthy at first glance.

Google DeepMind’s GNoME, a project aimed at advancing materials discovery through machine learning, demonstrated both this theoretical promise and potential pitfall by predicting new materials that might be stable enough to exist in the real world. The project reported 2.2 million stable crystal structures. Now, that is an impressive result, but it also shows why reliability checks are necessary. After all, how can scientists be totally confident in the results of a dataset far too vast for them to manually validate?

A stickier and far more alarming source of errors in self-driven AI research comes during measurement interpretation. Diffraction patterns, for example, are often analyzed to decide whether a predicted material was synthesized or not, but a sample might contain several phases, impurities, or patterns that resemble a different material. These patterns can be difficult enough to analyze for humans, let alone for an AI that must be trained on massive data sets just to gain some crude sense of pattern recognition. A 2024 PRX Energy paper on inorganic materials prediction and autonomous synthesis had this same conclusion, arguing that automated interpretation of X-ray diffraction is not yet reliable enough for unsupervised discovery.

Bad data from studies like these leads to bad conclusions. Bad conclusions can send a research project in the wrong direction, preventing scientists from making real progress in key fields. Bad conclusions leave engineers using potentially compromised materials to construct our infrastructure and means of transportation. Bad conclusions eventually show up in electronics, medicine, energy systems, and the like.

We need to concern ourselves with these bad conclusions and focus on mitigating them. The current, obvious first step in doing so is by keeping humans meaningfully involved rather than pretending the lab can become completely independent.

Another important solution is better data infrastructure. Just like any physical lab, autonomous labs need detailed metadata: sample data, calibration data, detailed procedural walkthroughs, etc. Without such information, another lab may not be able to reproduce the result to confirm their validity. This connects to the NIST AI Risk Management Framework, which identifies trustworthy AI qualities such as validity, reliability, safety, transparency, explainability, and accountability. I’d hazard that these qualities should not be optional. They should be treated like basic safety rules.

With these as preliminary guidelines, reliability standards will hopefully become a normal part of self-driving lab papers in the coming years, with students and researchers being trained not only to use AI tools but also to question it. If we can achieve this, the best autonomous labs will be not only fast, but also transparent enough that their discoveries can be challenged, corrected, and confirmed in the not-too-distant future.