How to use AI for research without getting fooled

Use AI to find leads, not to hand you findings. Published studies keep finding fabricated or wrong references in chatbot output, so open each cited source, confirm that it exists, and confirm that it says what the AI says it says. Prefer tools that link their sources, but check those links too: AI search tools with live web access still misattribute and break citations.

Why AI makes up citations

A language model writes the most plausible next words. A reference has a very regular shape: authors, year, title, journal, volume, pages. A model can produce text in that shape without looking up any record, so a citation can look perfect and point to nothing.

A 2025 paper by Adam Tauman Kalai and colleagues, "Why Language Models Hallucinate," argues that the problem is not a mystery. In the authors' account, the way models are trained and scored rewards a confident guess over an honest "I don't know." Most benchmarks score answers in a way that makes guessing pay. Until that changes, expect a model to fill a gap with something that sounds right.

What the studies found

Researchers have measured this directly. The numbers vary by model, task, and how strictly each study defines an error, but the direction is the same.

Study	What was tested	Result
Walters and Wilder, Scientific Reports, 2023	636 citations in 84 short literature reviews by ChatGPT-3.5 and GPT-4	55% of GPT-3.5 citations and 18% of GPT-4 citations were fabricated. Of the real ones, 43% (GPT-3.5) and 24% (GPT-4) had substantive errors.
Bhattacharyya et al., Cureus, 2023	115 references in 30 medical papers by ChatGPT-3.5	47% fabricated, 46% real but inaccurate, 7% real and accurate.
Chelli et al., JMIR, 2024	Asking models to find the papers for 11 published systematic reviews	Hallucinated references: 39.6% for GPT-3.5, 28.6% for GPT-4, 91.4% for Bard.
Tow Center, Columbia Journalism Review, 2025	1,600 queries to eight AI search tools, asking for the source of a news excerpt	More than 60% of answers were wrong overall, from 37% for Perplexity to 94% for Grok 3.
Magesh et al., Journal of Empirical Legal Studies, 2025	Legal AI research tools from LexisNexis and Thomson Reuters	Each hallucinated more than 17% of the time, though less than GPT-4.

Three lessons stand out.

Newer models are better, not safe. Walters and Wilder called GPT-4 "a major improvement," then added: "problems remain."
Real references can still be wrong. In the Cureus study, nearly half the references were real but inaccurate. The most common error was a wrong PubMed ID.
Search and retrieval reduce the problem but do not remove it. The legal tools retrieve real documents and still hallucinated. In the Tow Center tests, more than half of the responses from Gemini and Grok 3 cited fabricated or broken URLs, and ChatGPT signaled uncertainty in only 15 of 200 answers.

These are snapshots. Models change every few months, and a newer one may do better on the same test. The safe habit is to check every time, whatever the tool.

How to check a citation in five minutes

Find the record. Search the exact title in quotes in Google Scholar, or in PubMed for health topics. If the citation has a DOI, put https://doi.org/ in front of it. The DOI system looks up that string and sends you to the object it points to.
Match the details. Compare authors, year, journal, and title with the record. A small mismatch can mean a garbled real paper. No record at all means you should treat the citation as fabricated until you find it.
Open the paper and find the claim. Read the abstract, then search the full text for the specific finding. Ask one question: does this passage support the sentence the AI wrote?
Check its status. Look for a retraction or correction notice. The Retraction Watch database is now public through Crossref, and since January 2025 its retractions are in Crossref's API.
Check what kind of source it is. A preprint, a review, an opinion piece, and a randomized trial carry different weight. Our guide on how to read a research paper covers this.

If a claim has no source you can open, it is not evidence yet.

Prefer tools that show their sources

A tool that links each claim to a page lets you run the check above in seconds instead of minutes. That is a real advantage. But a link means you can check. It does not mean anyone did. The Tow Center found tools citing syndicated copies instead of the original publisher, and linking to error pages. Click through, and read the part of the page that matters.

If you compare AI search tools, our posts on Gregator vs Perplexity and Gregator vs ChatGPT describe how each shows its sources.

Read the source, not the summary

An AI summary can drop the hedges that the authors wrote. "In mice" becomes a claim about people. "Associated with" becomes "causes." "In a small pilot" disappears. When a claim matters:

Read the abstract and the limitations section of the original.
Check who and what was studied, and how many.
Look for the size of the effect, not only whether there was one.
Look for other sources that reach the same result on their own. Our source credibility checklist shows how.

Habits that keep you safe

Use AI where errors are cheap. Brainstorming search terms, explaining jargon, outlining a topic, and suggesting where to look are good uses. Final facts and citations are not.
Ask for links, then open them. A reference with no link costs you a database search. A reference with a link costs you a click.
Do not accept the model's word that a source is real. Check the record yourself, in a database or on the publisher's site.
Keep a short log. For each claim you plan to use, note the source, the passage, and the date you checked it.
Be strict with high-stakes topics. For health, law, and money, go to the primary source or to an institution you trust, every time.

Where Gregator fits

Gregator, our product, is an AI tool too, so the same rules apply to it. It researches the open web and writes a course of Chapters with inline footnotes, and the writer is told to place each marker right after the claim it supports; each Chapter ends with a Sources list you can open. Each source is also classified by an AI model for type and evidence quality, so treat those labels as a first read, not a verdict; the course shows a trust overview of the mix. Gregator does not fact-check, and it does not verify citations against claims. Run the five-minute check above on its footnotes too, especially for any claim you plan to repeat or rely on.

Frequently asked questions

Why does ChatGPT make up sources?

A model generates plausible text, and a citation has a predictable shape. Researchers argue that training and scoring reward confident guesses over admitting uncertainty, so a model may produce a reference that looks real without any record behind it.

Are AI search tools with citations reliable?

More reliable than tools with no sources, but not reliable enough to skip checking. A 2025 Tow Center test found eight AI search tools wrong on more than 60% of citation queries overall.

How can I tell if a citation is real?

Search the exact title in Google Scholar or PubMed, or resolve the DOI at doi.org. Then open the paper and confirm that it supports the claim.

Can I use AI to do a literature review?

Use it to brainstorm search terms and find leads, then search the databases yourself. In one 2024 study, the authors advised against using language models as the primary or exclusive tool for systematic reviews. For paper search tools, see our comparisons with Elicit and Consensus.

Do newer models still hallucinate references?

Yes, less often. GPT-4 fabricated far fewer citations than GPT-3.5 in one study, but 18% were still fabricated, and tools with retrieval still made errors.