I think the burden shifts towards being able to imagine and describe the fantasy. Novelty, and artistic creativity is still required. You can bring a horse to water, but you can’t force the horse to drink it. Many humans don’t use their imagination let alone have the eloquence to describe a search space that contains novelty.
How is this any different to working with a human artist? If I wanted the real world Salvador Dali to draw me a picture of a kebab being eaten by a badger I'd have to tell him that's what I want. I'd also need to educate baby Dali first, feed him all the art and information he can take so that he has a model of the world he's operating in. I'll need to supply Dali with context of prior art, educate him on styles, literature, language, and all the other things that shape a human mind.
As for the humans that don't use their imagination, maybe they never want to talk to an AI artist, just as many humans don't care about art at all. Millions of humans don't care about social news, and yet FaceBook algos pump out content for people all day long.
