A new research paper posted to arXiv cs.CL investigates the impact of unlabeled AI-generated text appearing in web corpora used for pretraining language models. As web content increasingly features AI-generated material, researchers applied quality filtering to examine June and August web data, finding significant portions labeled as machine-generated. To understand how this phenomenon affects model training, the study involved pretraining 800 language models while varying the proportions of human and AI tokens.
The Growing Presence of Synthetic Text in Web Corpora
Pretraining large language models heavily relies on vast collections of scraped internet text. Historically, this data was assumed to be authored primarily by humans. However, the proliferation of generative tools means that internet text now routinely includes content produced by various AI systems. Unlike controlled synthetic data generation or academic model-collapse experiments, this wild AI text arrives unlabeled, multi-sourced, and intended for human audiences. Understanding how these tokens influence downstream model performance is a critical question for AI development, as pretraining datasets form the foundational knowledge base for commercial and open-source models alike.
Open Questions and Future Monitoring
While the research explores scaling laws through hundreds of pretrained models, the announcement did not specify the exact architectures or sizes of the 800 models tested, nor did it detail the full parameters of the scaling laws discovered. It remains to be seen how model developers will adjust data curation pipelines to account for rising volumes of wild AI text, or whether future datasets will incorporate specialized filters to detect and remove multi-source machine-generated content before training begins.
As the ratio of machine-generated content on the web continues to climb, researchers will need to monitor how pretraining corpora adapt to these shifting data dynamics.
Lead image: Brett Sayles / Pexels
