If we take an example of what is considered a priori as creativity, such as story telling, LLMs can do pretty well at creating novel work.
I can prompt with various parameters, plot elements, moral lessons, and get a de novo storyline, conflicts, relationships, character backstories, intrigues, and resolutions.
Now, the writing style tends to be tone-deaf and poor at building tension for the reader, and it is apparent that the storytelling has little “theory of mind” of the reader, but the material has elements that we would certainly consider to be creative if written by a student.
It seems we must either cede that LLMs can do some creative synthesis, as this and some other experiments of mine suggest, or we must decide that these tasks, such as “creative writing” are not in fact creative, but rather mostly or strictly derivative.
There is some argument to be had in assertions that storytelling is all derivative of certain patterns and variations on a fixed number of tropes and story arcs… but arguing this begs the question of whether humans actually do any “pure” creative work , or if in fact, all is the product of experience and study. (Training data)
Which leads me to the unpleasant conflict about the debate of AI creativity. Is the debate really pointing out an actual distinction, or merely a matter of degree? And what are the implications, either way?
I’m left with the feeling that LLMs can be as capable of creative work as most 8th grade students. What does this say about AI, or developing humans? Since most people don’t exceed an 8th grade level of literacy, what does this say about society?
Is there even such a thing as de novo idea synthesis?
To add to this pondering: we are discussing the state today, right now. We could assume this is as good as it's ever gonna get, and all attempts to overcome some current plateau are futile, but I wouldn't bet on it. There is a solid chance that 8th grade level writer will turn into a post-grad writer before long.
So far the improvements in writing have not been as substantial as those in math or coding (not even close, really). Is there something fundamentally “easier” for LLMs about those two fields?
Much more formal structure and generally code can be tested for correctness. Prose doesn't have that benefit. That said, given the right prompt and LLM, you can squeeze out surprisingly good stuff: https://bsky.app/profile/talyarkoni.com/post/3ldfjm37u2s2x
If we take an example of what is considered a priori as creativity, such as story telling, LLMs can do pretty well at creating novel work.
I can prompt with various parameters, plot elements, moral lessons, and get a de novo storyline, conflicts, relationships, character backstories, intrigues, and resolutions.
Now, the writing style tends to be tone-deaf and poor at building tension for the reader, and it is apparent that the storytelling has little “theory of mind” of the reader, but the material has elements that we would certainly consider to be creative if written by a student.
It seems we must either cede that LLMs can do some creative synthesis, as this and some other experiments of mine suggest, or we must decide that these tasks, such as “creative writing” are not in fact creative, but rather mostly or strictly derivative.
There is some argument to be had in assertions that storytelling is all derivative of certain patterns and variations on a fixed number of tropes and story arcs… but arguing this begs the question of whether humans actually do any “pure” creative work , or if in fact, all is the product of experience and study. (Training data)
Which leads me to the unpleasant conflict about the debate of AI creativity. Is the debate really pointing out an actual distinction, or merely a matter of degree? And what are the implications, either way?
I’m left with the feeling that LLMs can be as capable of creative work as most 8th grade students. What does this say about AI, or developing humans? Since most people don’t exceed an 8th grade level of literacy, what does this say about society?
Is there even such a thing as de novo idea synthesis?
Troubling questions abound.