LLM Output will Poison Future LLM Input
TL;DR = "Garbage In = Garbage Out"
AI "will enter applications where future updates are critical to retain user value, but impossible without including substantial AI-generated texts of dubious accuracy."
"Iterative training [with inaccurate and hallucinated AI content] eventually produces . . . half-garbled nonsense, as tiny inaccuracies compound out-of-control."
"If 'access to genuine human-generated content is essential' to avoid Model Collapse, it is by extension essential to be able to sustain production of human-generated content and to be able to identify it. But LLMs are poised to make both conditions impossible."
"Perhaps economic incentives for not returning to human information systems will only pervade at the corporate level, making corporations unable to compete with smaller businesses that can resist the same incentives (admittedly a utopian hypothetical). Perhaps LLMs will so poison consolidated knowledge that localism is the only possible functional economic mode left."
Learn More