The Doom Spiral!

In issue 1 of this series, we saw that the Welsh language is short of quality data – and that the Token Crisis threatens smaller languages more than others. But what would happen if the quality of the sparse data that is being created started to decline too? This is where a new concept comes into play: the Doom Spiral.

Judah [1] describes the doom spiral. He shows how AI-generated or machine-translated content is increasingly added to public information sources such as Wikipedia, especially in smaller or vulnerable languages. The most vulnerable language communities often do not have a huge editorial team, so the information may contain culturally inappropriate language or biases. This is not necessarily the case for Wicipedia Cymraeg, of course. However it is difficult to argue that content that has been translated into Welsh mechanically or created by AI does not appear on the web. It is quite possible that those texts are full of bias and stereotypes etc.

The problem is that this content is then scraped to train future AI systems, meaning errors, misunderstandings and cultural distortions are fed back into the next generation of models. The process feeds on itself. Each cycle amplifies distortion — this is the Doom Spiral.

The logical conclusion of the iterative worsening is something called Model Collapse. Model Collapse occurs when AI systems are increasingly trained on their own outputs rather than diverse, real-world data. Over time, they lose the ability to represent rare or complex phenomena and become “poisoned” by their own narrow feedback loops [14]. They collapse. They die.

Poor models — though the implications for human language and cultural communities are far more serious.

“The spiralling decent” by sagesolar licenced under CC BY 2.0 .

The danger is that as the model gets worse, and before the model dies completely, the sub-standard output can develop into the dominant form of the language online, not because it reflects the way people use the language, but because it is easy for a machine to produce it.

Do we know what this would look like in the end? Well, yes, thanks to an experiment by Hintze et al. [15]! They connected one AI system that created images based on a text prompt with a system that created text based on a photo, then, after providing an initial prompt, let the systems ‘communicate’ with each other multiple times, allowing one system to create output based on the output of the other, without any additional human input. And what happened? Well after a number of iterations, the systems started to create extremely generic pictures, pretty pictures of lighthouses and cathedrals etc., regardless of the original text! Very interesting, but also cause for concern. According to Hintze et al.:

“The implication extends far beyond art generation. Many new AI applications use similar self-referential loops. If left unchecked, these systems could amplify the biases and redundancies already present in large datasets, reinforcing aesthetic and cultural uniformity.” [3]

They suggest that we can use two AI systems, as has been done in their experiment, but to reveal bias in training data. This is clearly a good thing. But for now, the concerns remain in terms of bias, quality, and the negative impact on culture. And this is especially true considering that, often, cultural communities that are affected do not have much control over what text is used for training, not to mention the self-referential process or the Doom Spiral.

Given these concerning trends, it would be good if we could readily identify bias. But how do we eliminate it in the first place? Inclusion may be a starting point. But we’ll discuss that in the next blog! Bye for now!

Bibliography

[1] Judah, J. (2025). “How AI and Wikipedia have sent vulnerable languages into a doom spiral.” Retrieved 01/12/25, o https://www.technologyreview.com/2025/09/25/1124005/ai-wikipedia-vulnerable-languages-doom-spiral.
[2] Shumailov, I., Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot ac R. Anderson (2024). The Curse of Recursion: Training on Generated Data Makes Models Forget.
[3] Hintze, A., F. Åström a J. Schossau (2025). “Autonomous language-image generation loops converge to generic visual motifs.” Patterns 7: 101451.