As someone 'in academia', I worry that tools like this fundamentally discard significant fractions of both the scientific process and why the process is structured that way.
The reason that we do research is not simply so that we can produce papers and hence amass knowledge in an abstract sense. A huge part of the academic world is training and building up hands-on institutional knowledge within the population so that we can expand the discovery space.
If I went back to cavemen and handed them a copy of _University Physics_, they wouldn't know what to do with it. Hell, if I went back to Isaac Newton, he would struggle. Never mind your average physicist in the 1600s! Both the community as a whole, and the people within it, don't learn by simply reading papers. We learn by building things, running our own experiments, figuring out how other context fits in, and discussing with colleagues. This is why it takes ~1/8th of a lifetime to go from the 'world standard' of knowledge (~high school education) to being a PhD.
I suppose the claim here is that, well, we can just replace all of those humans with AI (or 'augment' them), but there are two problems:
a) the current suite of models is nowhere near sophisticated enough to do that, and their architecture makes extracting novel ideas either very difficult or impossible, depending on who you ask, and;
b) every use-case of 'AI' in science that I have seen also removes that hands-on training and experience (e.g. Copilot, in my experience, leads to lower levels of understanding. If I can just tab-complete my N-body code, did I really gain the knowledge of building it?)
This is all without mentioning the fact that the papers that the model seems to have generated are garbage. As an editor of a journal, I would likely desk-reject them. As a reviewer, I would reject them. They contain very limited novel knowledge and, as expected, extremely limited citation to associated works.
This project is cool on its face, but I must be missing something here as I don't really see the point in it.
Fully agreed on point A, but I've heard the "but then humans won't be trained" argument before and don't buy it. It's already the case that humans can cheat or get buy without fully understanding the math or ideas they're working with.
This is what PhD defences are for, and what paper reviews are for. Yes, likely we need to do something to improve peer review, but that is already true without AI.
From a more philosophical point of view, if we did hypothetically have some AI assistant in science that could speed up discovery by say 2x, in some areas it seems almost unethical not to use it. E.g. how many more lives could be saved by getting medicine or understanding disease earlier? What if we obtained cleaner power generation or cleaner shipping technologies twice as fast? How many lives might be saved by curtailing climate change faster?
To me, accelerating science is likely one of the most fundamentally important applications of modern AI we can work on.
Scientific advancement IS the change in the understanding of humans.
Your fallacy lies in pretending science can progress in an objectively defined space where saying 'progress twice as fast' even makes sense.
But it does not. It progresses as people refine and improve their understanding of the universe. Books or LLMs don't contain understanding, only data. If less people grow a deep understanding of mathematics and the universe, science progress will slow down, even as metrics like numbers of graduates or papers published go up.
> If I can just tab complete my N body code, did I really gain the knowledge of building it?
Yes, because fixing it requires about the same effort as writing it from scratch. At least with current level of AI. When it works well, you just move it to a library and use it without worrying about implementation, like we do with all other library code.
Using AI doesn't make the problem any easier for the developer. The fact that it generates functions for you is misleading, code review and testing is harder than typing. In the end we use AI in coding because we like the experience, it doesn't fundamentally change what kind of code we can write. It saves us a few keystrokes and lookups.
AI might be useful in literature review stage, formal writing and formatting math. Who's gonna give millions worth of compute to blindly run the AI-Scientist? Most companies prefer to have a human in the loop, it's a high stakes scenario depending on cost.
The original purpose of science was to get closer to god. Things change.
Arts degrees were also once 7-8 years before the french fought for radical simplification to get them down. There is no law of nature that says PhDs cannot be simplified down more too.
The stupification argument comes kneejerk with every new medium, you don't have to believe everything you watch on TV or form a crutch to a screen oracle. Either way, we need not concern ourselves with the average scientist who fails forgettably by being stupid when the potential upsides are great scientists who succeed with it.
One point of an AI paper farm could just be to increase that discovery hit space a little while you sleep but no one claims it can or will replace every scientist, hopefully only the pessimistic ones.
PhD cannot be simplified further down without making it not "significant contribution to science/art/craft". However, DAs (Doctor of Arts) could be granted for hard sciences. The threshold for DA is "outstanding achivement", which would be fit for a post-AGI doctorate in hard sciences.
LLM have unleashed the dreamer in each and every young coder. Now, there is all sorts of speculation on what these machines can or cannot do. This is a natural process of any mania.
These folks must all do courses in epistemology to realize that all knowledge is built up of symbolic components and not spit out by a probabilistic machine.
Gradually, reality will sync (intentional misspelling) in, and such imaginations will be seen to be futile manic episodes.
> These folks must all do courses in epistemology to realize that all knowledge is built up of symbolic components and not spit out by a probabilistic machine.
Knowledge ends up as symbolic representation, but it ultimately comes from the environment. Science is search, searching the physical world or other search spaces, but always about an environment.
I think many people here almost forget that the training set of GPT was the hard work of billions of people over history, who researched and tested ideas in the real world and build up to our current level. Imitation can only take you so far. For new discoveries the environment is the ultimate teacher. It's not a symbolic processing thing, it's a search thing.
Everything is search - protein folding? search. DNA evolution? search. Memory? search. Even balancing while walking is search - where should I put my foot? Science - search. Optimizing models - search for best parameters to fit the data. Learning is data compression and search for optimal representations.
Symbolic representations are very important in search, they quantize our decisions and make it possible to choose in complex spaces. Symbolic representation can be copied, modified and transmitted, without it we would not get too far. Even DNA uses its own language of "symbols".
Symbols can encode both rules and data, and more importantly, can encode rules as data, so syntax becomes object of meta-syntax. It's how compilers, functional programming and ML models work - syntax creating syntax, rules creating rules. This dual aspect of "behavior and data" is important for getting to semantics and understanding.
my guy you're so confident yet you forget AlphaFold, it designs protein structures that don't exist.
Who's to say that a model can't eventually be trained to work within certain parameters the real word operates in and make new novel ideas and inventions much like a human does in a larger scope.
DeepMind is overselling their AI hand when they dont have to.
"whos to say that" - this could be a leading question for any "possibility" in the AI religion.
"whos to say that god doesnt exist" etc. questions for which there are no tests, and hence fall outside the realm of science and in the realm of religion.
If you go and look at the list of authors on those papers you will see that most of the authors have PhD doing something protein folding related. It's not that some computer scientists figured it out. It's that someone built the infrastructure then gave it to the subject matter experts to use.
AlphaFold doesn't solve the protein folding problem. It has practical applications, but IMO we still need to (And can!) build better ab-initio chemistry models that will actually simulate protein folding, or chemical reactions more generally.
Models like AlphaFold are very different beasts. There's definitely a place for tools that suggest verifiable, specific, products. Overarching models like 'The AI Scientist' that try to do 'end-to-end' science, especially when your end product is a paper, are significantly less useful.
We already substitute "good authority" (be it consensus or a talking head) for "empirical grounding" all the time. Faith in AI scientific overlords seems a trivial step from there.
The reason that we do research is not simply so that we can produce papers and hence amass knowledge in an abstract sense. A huge part of the academic world is training and building up hands-on institutional knowledge within the population so that we can expand the discovery space.
If I went back to cavemen and handed them a copy of _University Physics_, they wouldn't know what to do with it. Hell, if I went back to Isaac Newton, he would struggle. Never mind your average physicist in the 1600s! Both the community as a whole, and the people within it, don't learn by simply reading papers. We learn by building things, running our own experiments, figuring out how other context fits in, and discussing with colleagues. This is why it takes ~1/8th of a lifetime to go from the 'world standard' of knowledge (~high school education) to being a PhD.
I suppose the claim here is that, well, we can just replace all of those humans with AI (or 'augment' them), but there are two problems:
a) the current suite of models is nowhere near sophisticated enough to do that, and their architecture makes extracting novel ideas either very difficult or impossible, depending on who you ask, and;
b) every use-case of 'AI' in science that I have seen also removes that hands-on training and experience (e.g. Copilot, in my experience, leads to lower levels of understanding. If I can just tab-complete my N-body code, did I really gain the knowledge of building it?)
This is all without mentioning the fact that the papers that the model seems to have generated are garbage. As an editor of a journal, I would likely desk-reject them. As a reviewer, I would reject them. They contain very limited novel knowledge and, as expected, extremely limited citation to associated works.
This project is cool on its face, but I must be missing something here as I don't really see the point in it.