How to Check Whether Your AI-Generated Citations Are Real

If you've used AI to help pull together a literature review, a grant narrative, or the reference list on anything that matters, you've probably had the quiet thought that follows: these look right, but are they actually real? It's a reasonable thing to wonder, and it turns out to be a well-founded one. The research on this is genuinely startling — studies have found that somewhere between a quarter and 40% of AI-generated references are completely fabricated, and that even among the ones that point to a real paper, close to half contain some kind of error in the details. So the instinct to double-check isn't paranoia. It's the correct response to how these tools actually behave.

What makes it trickier is that the fakes don't look like fakes. A hallucinated citation usually arrives with a real-sounding author, a plausible journal title, a properly formatted DOI, and sometimes even a working link — everything your eye uses to decide a reference is legitimate, assembled into a source that simply doesn't exist. And this has stopped being a low-stakes problem. Over the past year, arXiv tightened its enforcement to hold authors responsible for unverified AI output, including hallucinated references, and researchers have started arguing that a fabricated citation can rise to the level of misconduct when it functions as evidence and the author never bothered to check it. Verifying your references has quietly moved from good hygiene to basic risk management.

Why AI invents citations in the first place

It helps to understand what's actually happening, because it explains why the problem is so persistent. An AI language model doesn't look anything up. It predicts text that sounds right based on the patterns it learned, and a citation is just a very structured pattern — author, year, title, journal, DOI. The model can reproduce that shape perfectly without any of the pieces corresponding to a real paper, because at no point in generating it does the model consult a database to confirm the thing exists. There's no verification step happening under the hood. That's not a bug someone forgot to fix; it's a direct consequence of how the technology works, which is why "use a better model" only reduces the rate rather than solving it.

The manual checks that actually work

The good news is that most fabricated citations are catchable, and you don't need anything fancy to catch the obvious ones. The single fastest check is the DOI. If the citation has one, paste it after https://doi.org/ in your browser and hit enter — if it lands on the publisher's page for the paper you were expecting, you're fine, and if it 404s or resolves to some unrelated article, you've found a fake in about ten seconds.

When there's no DOI, or you want to go a step further, the next move is to check the author. Look up the first author on ORCID or Google Scholar and see whether the cited paper actually appears in their publication history — if the person is real but the paper isn't in anything they've ever written, that's a strong sign the citation was stitched together from plausible parts. Beyond that, you can cross-reference the reference itself against an authoritative database. Crossref, PubMed, and OpenAlex all let you search by title or DOI, and a citation that doesn't turn up in any of them, despite claiming to be published work, probably isn't published work.

The most reliable version of this is a metadata-consistency check, which is really just doing all of the above at once: confirming that the author, the title, the journal, the year, and the identifier all point to the same real paper, rather than checking any one of them in isolation. That last part turns out to matter more than it sounds.

Where manual checking hits its ceiling

Here's the catch, and it's the part most guides skip. The citations that actually make it into a retraction notice aren't the crude ones that fail the DOI check — those get caught. The dangerous ones are what researchers studying a set of fabricated citations at a major AI conference called "compound" fabrications, and they found that essentially all the hallucinated citations they examined fell into this category. A compound fake is one that sounds plausible, includes a working link, and references familiar author names all at the same time, which means it sails past every individual check you might run. The DOI resolves to something. The author is real. The title reads like a paper that could exist. Each check in isolation comes back clean, and the fabrication only reveals itself when you cross-reference every attribute simultaneously and notice they don't all point to the same actual source.

Their conclusion was blunt: simple link-checking isn't enough, and real verification has to confirm authors, titles, venues, dates, and identifiers together rather than one at a time. Which is technically doable by hand — but if you've ever tried to run that full simultaneous cross-check on forty references at eleven at night before a submission deadline, you already know how that goes. It's exactly the kind of tedious, repetitive, easy-to-rush work where a tired human eye starts nodding things through, and where the compound fakes were specifically built to slip past.

Closing the gap without doing it all by hand

This is where it makes sense to have something independent handle the cross-check for you. Rather than verifying each reference one attribute at a time, an automated check can take the finished document and run every citation against the actual registries — Crossref, DataCite, the relevant case-law databases — confirming author, title, venue, date, and identifier together, and then showing you plainly what resolved cleanly, what didn't resolve at all, and what pointed somewhere other than where it claimed to. That's the whole idea behind Recensa's Deep check: not to replace your judgment about which sources belong in your work, but to do the one part that manual checking does worst, reliably and at scale, before the document goes anywhere. You still make the call on every finding. It just makes sure the compound fakes don't get to hide behind a working link.

None of this means you should stop using AI to draft — it means the reference list is the part you can't take on faith. The manual checks are worth knowing and worth running, especially the DOI check, which costs you nothing. But the fabrications that end careers are the ones engineered to pass exactly those checks, and those are the ones worth having an independent, systematic verification catch before you submit rather than after someone else does. It's the same principle behind why a document passing its own check isn't the same as being verified — the value is in the independence.