I mean, human creativity is also based on inputs and association. Even if the inputs are a few degrees apart, having an algorithm query similarities to two concepts and merge them together contextually wouldn't be impossible. Nor would taking how many times it's seen a concept and trying to put a twist on it. Although they might be pretty difficult to program in.
We aren’t going to settle this question here, especially if we aren’t all speaking a common technical language about what LLMs can and can’t do. But a central question is whether creativity can be quantified, to the extent that you can measure success or failure, both key requirements in a machine learning model.
There are AI/ML devs and researchers who think that yes, you absolutely can quantify and solve for what we’d consider human creativity. But what that’s meant in practice has been solving for some kind of “novelty.” The result is basically random nonsense—art that’s expressionistic in style but that isn’t meant to invoke any particular emotions or thoughts, and garments that basically look like vomit.
One persistent issue within the larger field is whether AI engineers could benefit from a greater education in the humanities, soft sciences, etc. That sort of thing might prevent the sort of blunders that lead to ethical violations in models, but also the possible hubris of thinking that everything the human mind can do can be boiled down into inputs and features and math.
So it all gets very existential right away, but here’s how I see it: We still don’t really know how the brain works, so we can’t possibly recreate its most complex capabilities and responses. Until we can (and it might never happen—it could be as insurmountable as FTL) the real question is, do you think there’s some unique value in art, writing, or any other creative pursuit that can evoke emotions?
If no, then why isn’t everyone able to write or paint or create in evocative ways with enough training? Because of it was that quantifiable and reproducible, we would have already quantified and reproduced it at industrial scales.