Detecting AI-Generated Content in Swedish News Media
We built a classifier that detects AI-generated text in Swedish news media by learning stylistic differences between human and machine writing. The key design choice was to train on human articles from before ChatGPT existed, then generate AI versions of those same articles, so the model cannot rely on topic to separate the two classes.
Same topic, different author
We collected tens of thousands of human-written Swedish news articles published between 2020 and 2022, then used 9 different LLMs to generate AI versions of those same articles. Because both the human and AI texts cover identical topics from the same time period, the model is forced to learn stylistic differences rather than topical ones.
We fine-tuned ModernBERT-large (a 395M parameter model with Swedish language support) on this dataset. The AI texts were generated at three levels of granularity: full articles from a topic, paragraph-level rewrites, and sentence-level expansions. We used 9 models from OpenAI, Anthropic, and Google spanning 2022 to 2026, so the detector generalizes across model generations rather than overfitting to one particular style.
The result: 96.9% accuracy on a held-out test set, with an AUC-ROC of 0.994.
Running inference on 50,000 articles
We ran the trained model across roughly 50,000 articles published between January 2023 and February 2026 from seven major Swedish news sources: SVT, SR, DN, Aftonbladet, Expressen, SvD, and GP.
The detection threshold was calibrated at a 2% false positive rate using ~10,000 pre-ChatGPT articles the model never saw during training.
Through 2023 and most of 2024, the detection rate sits at or below the expected false positive baseline, consistent with little to no AI-generated content. Starting in late 2024, the signal begins to rise. By early 2026, nearly 4% of articles are flagged, roughly double the baseline.
Comparing with an LLM-as-judge
As a baseline, we ran the same articles through Claude Sonnet 4.5, one of the most capable LLMs available, using it as a classifier with a carefully engineered detection prompt.
The LLM flagged very few articles across all time periods. Its detection rate shows no variation over time. No temporal signal emerges from the LLM-as-judge approach.
The same training objective that makes LLMs produce natural text makes them poorly suited to distinguish it from human writing. A fine-tuned classifier learns statistical patterns that a general-purpose LLM does not attend to.
The detection paradox
Our detector was trained on AI text generated by models available between 2022 and 2026. AI writing quality improves with each generation. The texts our detector flags are, almost by definition, the ones that still carry detectable statistical signatures. The AI-generated articles that most successfully mimic human style are the ones we never catch.
This is a form of survivorship bias applied to detection. We observe the detectable cases, not the successful ones. The 3.7% we flag in early 2026 is a lower bound. The true share of AI-generated or AI-assisted content could be meaningfully higher, produced by newer models or more sophisticated prompting that our classifier has not learned to recognize.
The apparent acceleration from late 2024 onward likely understates the actual rate of change. As models improve, the gap between what exists and what we can detect will widen. Any detector trained today reflects the statistical signatures of current and past models, not future ones.
Caveats
These findings do not mean 4% of Swedish news is written by AI. "AI-flagged" is not the same as "AI-written." The detector picks up statistical signatures that could come from fully AI-written articles, AI-edited drafts, or AI-assisted writing. Our threshold is deliberately conservative. The signal is still small in absolute terms, and the vast majority of Swedish news content appears human-written.
This is also about style, not quality. AI-generated text is not inherently worse. What matters is transparency. Readers should know how the text they read was produced.
Method
- Model: AI-Sweden-Models/ModernBERT-large, fine-tuned for binary classification
- Training data: ~50,000 labeled articles (human and AI-generated) from 9 LLMs
- AI generation: 30 prompt templates, 3 granularity levels (full/paragraph/sentence), varying temperatures and personas
- Training: 3 epochs, BF16 mixed precision on NVIDIA DGX (A100)
- Threshold: 2% FPR calibrated on ~8,000 held-out pre-ChatGPT articles (top-3 mean paragraph score > 0.939)
- Sources: SVT, SR, DN, Aftonbladet, Expressen, SvD, GP (2020 to 2026)
- LLM baseline: Claude Sonnet 4.5 with a carefully engineered Swedish-language classification prompt