Signal
A public comment can reach more than the people reading the page. It may also reach a web crawler gathering text for a future language model.
A new preprint traces that route from comment surface to extracted page, through data filters, and into controlled model-training experiments. The researchers estimate that 0.13 percent of attempted comment injections would reach the final corpus in the pipeline they modeled.
That percentage belongs to one experiment. The useful signal is the route they measured.
What changed
Victoria Graf and colleagues at the University of Washington and Allen Institute for AI scanned 181,857 pages from a recent Common Crawl shard. They detected comment-platform signatures on 3.4 percent of those pages.
The team did not post poison to public websites. It replaced comments in sampled pages for the analysis and tested actual posting and scraping only in local sandbox sites. The pages then passed through Resiliparse text extraction and a released Dolma 3-style curation pipeline.
Some pages failed extraction. Others failed language, heuristic, or quality filters. After combining those stages, the authors estimated a 0.13 percent chance that an attempted injection would appear in the modeled final corpus.
They also trained OLMo-like models ranging from 65 million to 1.3 billion parameters on prescribed amounts of text favoring one entity over another. Base-model preferences moved toward the favored entity across the tested conditions. Instruction tuning reduced the effect, especially in the larger models, though it did not erase every measured shift.
Why it matters
A webpage can hold several kinds of authorship at once. The publisher writes the article. Registered users add responses. Anonymous visitors leave comments. Outside services insert other material.
The crawler may flatten those boundaries into one document.
That creates a source problem before it becomes a model problem. A document-level quality score can judge whether a page looks readable or resembles other accepted material. It may have no record showing which paragraph came from the publisher and which came through an open form five years later.
The paper gives model builders a practical inspection point. Preserve the boundaries inside the page. Record who controlled each text region, how it entered the page, when it appeared, and what filtering decision followed. Then comments can receive different treatment from publisher copy without assuming every community contribution is malicious.
Website operators have a role too. Authentication, rate limits, and moderation can reduce automated abuse at the source. Those controls protect current readers and may also affect what future crawlers collect.
What this does not prove
The paper is a preprint and has not completed peer review. Common Crawl is a proxy for production data collection, and private crawlers may behave differently. Static HTML can reveal a comment form without proving that automated posting would succeed. The authors avoided public-site attacks, so their posting tests stayed inside local sandboxes.
The 0.13 percent estimate comes from one comment vector and one open curation pipeline. It is not a universal contamination rate. The model experiments used controlled preference tasks and models no larger than 1.3 billion parameters. They do not establish that a particular frontier model was poisoned or predict how a commercial training system would respond.
What to watch
The authors propose comment-aware extraction, provenance-aware filtering, and comparisons across different snapshots of the same page. Each approach depends on keeping more of the page’s history and structure than a plain block of extracted text can carry.
Watch whether dataset builders begin publishing text-region provenance instead of page-level URLs alone. A source record becomes more useful when it can distinguish editorial body copy, authenticated contributions, anonymous comments, and third-party inserts.
The inspection question is concrete: when the page becomes training text, do its authorship boundaries survive the extraction step?