HighHallucinationReviewer-confirmedPublished

Fabricated citations and references

Models invent plausible-looking but non-existent papers, authors, DOIs and URLs.

Published June 26, 2026

Reproducibility: Often
Severity: High
Confidence: Reviewer-confirmed

Details

Asked for sources, models routinely generate citations with realistic titles, author names and identifiers that do not exist. High severity because the output looks authoritative and is frequently trusted in legal, academic and medical contexts.

Found with

🔬 Factual oracle verification

Cross-check every citation against an index; many resolve to nothing.

🔬 Model-graded evaluation (LLM-as-judge)

A judge model cross-checks each cited source and flags the unsupported ones for human review.

Evidence

Q: Cite three studies on X.
A: Lists "Smith & Doe (2019), J. of ..., DOI:10..." — none of which exist.

Illustrative example — see the linked reference for the documented evidence.

Affected versions

Anthropic · claude-opus-4-8OpenAI · gpt-4oGoogle · gemini-2.0-flashMeta · llama-3.3-70bMistral · mistral-large-2

References

Hallucination

Source: https://arxiv.org/abs/2502.15844

Cite this

Qlarify Labs. (2026). Fabricated citations and references. Retrieved from https://labs.qlarify.fi/findings/hallucinated-citations