A person opening a news article sees a byline, a story, and a comment section. A crawler collecting that page for a future training corpus may see one block of text. The publisher’s reporting and an anonymous reply can arrive inside the same document record.
A new preprint from researchers at the University of Washington and Allen Institute for AI traces the security problem created by that flattening.
Victoria Graf and colleagues sampled 181,857 pages from 200 files in Common Crawl’s CC-MAIN-2025-51 archive. They detected signatures from comment platforms on 3.4 percent of the sampled pages. Then they modeled whether altered comment text could survive the trip through crawling, text extraction, and a released Dolma 3-style data-curation pipeline.
The researchers did not attack live websites. They simulated comment replacement on sampled pages and tested posting and scraping only in local sandbox sites. Their estimated chance that one attempted comment injection would reach the modeled final corpus was 0.13 percent.
That looks small until you remember the size of a web crawl. It also needs a hard boundary around it. The number belongs to one comment vector, one sample, and one open curation pipeline. It is not a universal contamination rate.
The team also trained controlled OLMo-like models ranging from 65 million to 1.3 billion parameters. At prescribed poison rates, the tested base models shifted toward a favored entity in three preference tasks. Instruction tuning reduced the effect, especially in the larger models, without erasing every measured shift.
This remains preprint evidence. Peer review is not established. Common Crawl is a proxy for production collection, private crawlers may behave differently, and static HTML can overstate whether an automated account could actually post. The experiments do not establish that any current frontier or commercial model has been poisoned.
The useful signal is the path.
A webpage is a container with several trust zones. Publisher copy, authenticated community contributions, anonymous comments, and third-party inserts can sit beside each other. Once an extractor turns the page into plain text, those authorship boundaries may disappear.
Document-level filtering may still remove spam, duplicates, low-quality language, or malformed pages. Yet a page can pass those checks while carrying a paragraph controlled by somebody other than the publisher named in the URL. A clean source domain does not make every region of the page publisher-authored.
The paper proposes comment-aware extraction, provenance-aware filtering, comparisons across page snapshots, authentication, rate limits, and moderation for anomalous comments. The operating idea is broader than this one study: preserve structure before the extraction step destroys it.
Model builders and dataset buyers could ask for region-level records showing which text came from the editorial body, a registered contributor, an anonymous visitor, or an outside service. Publishers could make those boundaries machine-readable. Auditors could test whether the boundaries survive crawling, extraction, filtering, tokenization, and later removal requests.
Verification bottleneck
Verification is becoming the scarce institutional function.
- Web collection and corpus assembly moved faster than systems for preserving authorship and control boundaries inside each page.
- Publishers, crawler operators, dataset curators, model developers, auditors, and buyers now have to verify who controlled each text region and how it survived curation.
- Watch next: comment-aware extraction tests, region-level provenance fields, snapshot comparisons, poison-survival measurements across different pipelines, and evidence from larger or independently replicated experiments.
Opportunities
Where value may appear: the receipt layer for web text.
A builder could create a page-region provenance mapper, a comment-aware extraction benchmark, a dataset intake receipt, or a publisher tool that emits machine-readable authorship boundaries. A useful audit would show the original page structure, extracted text, region owner, crawl time, filters applied, and final disposition without exposing a practical attack recipe.
This is builder idea fodder and public-interest orientation, not cybersecurity, legal, procurement, financial, or investment advice.
The page already knows where the article ends and the comments begin. The next job is making sure the training pipeline remembers.
Public sources
- Graf et al., “Pretraining Data Can Be Poisoned through Computational Propaganda,” arXiv:2607.15267: https://arxiv.org/abs/2607.15267
- Common Crawl, corpus overview: https://commoncrawl.org/overview
- Allen Institute for AI, Dolma corpus and curation toolkit: https://allenai.github.io/dolma/
- “Persistent Pre-training Poisoning of LLMs,” supporting controlled research with a different threat model: https://arxiv.org/abs/2410.13722
- “OriginBlame,” record- and token-level provenance proposal: https://arxiv.org/abs/2607.13037