Discovery & AI · constellation

AI Systems That Discover, Not Just Summarize

Also called: Autonomous scientific discovery

Personal interest
Philosophical question
Working interpretation

I want to find out whether groups of AI agents, programs that can take steps on their own, can do more than summarize what is already known: whether they can suggest new ideas worth testing and find results that other people can check for themselves. The goal is a real new contribution, not just a lot of automated activity.

What would count as a truly new result, and who would be able to check it?

Why it attracts me

Most of what AI tools do today is rearrange knowledge that already exists. They summarize, translate and draft. That is useful, but it is not discovery. What I want to know is whether a group of AI agents, set up carefully, can go further: suggest a new idea, test it, and produce a result that someone else can check without trusting the system or me.

The question matters to me because it sits where several of my interests meet: mathematics, computer search, careful evidence and the wish to make a real contribution (Making a Meaningful Contribution).

The idea

Research has a rough cycle. You pick a precise question, propose an answer, try to break it, check it again independently, and compare it with what is already known. An autonomous system tries to run more of that cycle with less step-by-step human direction. The pieces have their own places on this map: AI workflows that plan and criticize (AI Workflows That Catch Their Own Mistakes), computer search (Computer Search and Rules of Thumb), proofs a machine can check (Proofs a Computer Can Check) and the search for earlier work (Checking Whether Someone Got There First).

The hard part is not producing output. It is producing a result that is both new and true.

What I think (and don't know)

I think the most promising designs keep finding and accepting separate. The program that proposes an answer should not be the only one allowed to approve it. I also think the strongest early results will come where a proposed answer is cheap to check exactly, even if finding it took a huge search.

What I do not know is how far this goes. A system can look impressively busy while producing nothing new. It can also produce something correct that turns out to be already known. I do not yet know where the line falls between AI doing research and AI being a very good research tool, or whether that line matters as much as the result itself.

Where it connects

This idea pulls against Curious Wandering and Lucky Discoveries. Directed research aims at a target, but some of the best connections turn up while wandering without one, and both have value. It also depends on Pairing AI's Reach With Human Judgment, because judging which results matter is still a human job. And it leads to Contributing Without Needing the Credit: the goal is the contribution, not the credit.

An example

My Agentic Maths side quest asks whether coding agents can find publishable mathematics if novelty, proof and attempts to break a claim are treated as separate gates. A candidate passes through steps that include reproducing known results, a hostile review, an independent rebuild, a search for earlier work and, finally, handoff to a human expert. The strongest result so far is a construction whose main theorem is checked by the Lean proof checker. It is marked ready for expert review, not peer reviewed, and the page is explicit that even a top-level result can still be wrong or already known. That honesty is the point. The system's job is to take a claim as far as it honestly can, then stop and hand it to people.

Questions I am still carrying

How do I measure progress when most attempts fail quietly?
How should credit and authorship work when AI does much of the search?
Will the best results come from fully automatic systems or from close teams of people and AI?

What this does not establish

Being interested in AI-led discovery does not show that current AI systems can make new, correct discoveries on their own. Each claimed result still needs independent checking, a search for earlier work and review by human experts.

Questions I'm still exploring

What is the smallest result that would count as a real discovery rather than a summary?
Which parts of research can be handed to AI, and which must stay with people?
How do I avoid mistaking a large volume of output for progress?

Sources and further reading

Bernardino Romera-Paredes et al., "Mathematical discoveries from program search with large language models", Nature 625 (2024): 468–475 — Describes FunSearch, which paired a language model with an automatic evaluator and found new mathematical constructions.
Alex Davies et al., "Advancing mathematics by guiding human intuition with AI", Nature 600 (2021): 70–74 — Machine learning pointed mathematicians toward patterns that they then proved themselves.

Working interpretation: drafted from my notes and interests for review. It is not a direct quotation, and I may still change it.

In short

Autonomous scientific discovery means using AI systems to run more of the research cycle with less step-by-step human direction: choosing a precise question, proposing a candidate answer, testing it, and reporting what survived. Most AI today is better at summarizing what people already know. The harder test is a result that is both new and correct, which means someone other than the AI must be able to check it.

Why this matters to me

Discovery is one of the main threads running through this map, and AI agents are the newest tool I have for it. What draws me is not the speed. It is the question of whether a well-designed system can produce something genuinely new that other people can verify without taking my word for it. I care about the difference between a real contribution and an impressive amount of automated activity.

Connections

Making a Meaningful ContributionMotivation

My wish to discover new things is tied to wanting to make a real contribution, not just to produce activity.

Curious Wandering and Lucky DiscoveriesTension

Research aimed at a clear target and open-ended wandering both have value. A directed search can miss connections that only turn up by chance.

AI Workflows That Catch Their Own MistakesPart of

AI workflows that plan, criticize and check their own work are the practical machinery that AI-led discovery would run on.

Unsolved Problems in MathematicsPart of

Unsolved math problems are one of the clearest tests for AI-led discovery, because the questions are exact, so a claimed answer can be checked rigorously.

Proofs a Computer Can CheckPart of

If AI is to discover anything, its claims need checks that do not rely on trust. Proofs a computer can check are among the strongest.

Checking Whether Someone Got There FirstPart of

A discovery has to be new as well as true, so AI-led research needs a search for earlier work before it claims anything.

Doing Research With Critics and ExpertsPart of

Even a highly automated discovery process ends with people: critics and specialists who test a result and judge whether it matters.

Computer Search and Rules of ThumbPart of

Much of AI-led discovery is search through enormous numbers of possibilities, guided by shortcuts and checked afterwards.

Results That Hold Up When RerunPart of

A discovery only counts as evidence if others can rerun it and get the same result without trusting the system that found it.

Pairing AI's Reach With Human JudgmentPart of

I think discovery systems work best as a partnership, with AI searching widely and people judging which results matter.

Taking Methods Into Unfamiliar FieldsPart of

The same computer methods often work across very different sciences, so AI-led discovery is not tied to one field.

Contributing Without Needing the CreditPart of

The point of discovery is to add something useful, whether or not anyone's name ends up attached to it.

All My Interests, One Changing SearchPart of

The wish to discover something new lives here: AI-assisted research, open problems in mathematics, and the careful checking that turns a promising idea into a result.