That LLMs exist; that they are capable of forming coherent sentences in response to prompts; that they are in some genuine sense creative without intentionality, suggests that there is something importantly right about the arguments of structuralist linguistics. Language demonstrably can exist as a system independent of the humans who employ it, and exist generatively, so that it is capable of forming new combinations. […] much of what we commonly attribute to individual cognition is in fact carried out through the systems of signs that structure our social lives.
This is a much more interesting observation about linguistics than it is about AI
It’s ultimately also an observation about what LLMs can and can’t be, based on the structures they’re reproducing: