Recent independent investigations by Epoch AI and Anthropic have evaluated the capabilities of current artificial intelligence models when performing scientific research tasks. According to reporting by The Decoder, models such as GPT-5.6 Sol and Claude Fable 5 demonstrate an ability to execute experiments, yet they suffer from significant limitations in genuine creative thinking and scientific self-criticism. In testing, the highest-performing model achieved only 15 percent of the human reference score, and that result relied entirely on pre-existing methods already known to human researchers. The primary hurdle identified across the evaluations is the current AI systems’ persistent inability to critically evaluate their own experimental outputs.
Evaluating AI Autonomy in Science
The findings provide a reality check regarding the push toward fully autonomous scientific discovery using machine learning models. While large language models and specialized reasoning agents have advanced rapidly in data processing and code generation, the scientific method relies heavily on healthy skepticism, hypothesis generation, and rigorous self-correction. The studies suggest that current architectures simply overstate their achievements and lack the foundational reasoning required to independently drive novel scientific breakthroughs without human intervention.
Unanswered Questions and Next Steps
Significant gaps remain regarding how developers plan to address these critical reasoning deficits. The reports did not specify what architectural changes might bridge the gap between running automated laboratory routines and exercising genuine scientific judgment. Observers will need to watch whether upcoming iterations of these models can incorporate robust self-critique mechanisms or if fundamentally new training paradigms will be required to achieve true research autonomy.
As laboratories continue to deploy AI agents for routine experimentation, human oversight remains indispensable for interpreting results and directing legitimate scientific inquiry.
Lead image: cottonbro studio / Pexels
