How to check an AI answer against the document it came from

People now paste leases, insurance policies, job offers and lecture notes into AI tools and ask what they mean. That's a sensible thing to do. A 40-page document is hard to read, and a good tool can find the right paragraph quickly.

The catch is that these tools sometimes get it wrong, and they don't sound any different when they do. The US National Institute of Standards and Technology calls this "confabulation": a system confidently presenting content that is false (NIST AI 600-1, section 2.2). The same report warns that a tool can also produce made-up citations or reasoning that make a wrong answer look trustworthy.

You don't need to stop using AI to read documents. You need a fast way to check it. This guide gives you one.

Why answers about a document go wrong

Knowing the usual causes makes the mistakes easier to spot.

• The tool never read the part that matters. Long files may be cut short. Scanned pages, photos of pages and text inside images often have no readable text at all. Tables can be read out of order.
• It answered from general knowledge instead of your file. Ask "how much notice do I need to give?" and a tool may give a common answer for leases in general, not what your lease says.
• It missed an exception. Clauses often say "subject to clause 14", "unless", "except" or "provided that". The main sentence says one thing; the exception changes it.
• It missed a definition. Capitalised words like "Business Day", "Pre-existing Condition" or "Confidential Information" usually have a defined meaning near the front or back. The definition beats the everyday meaning.
• It blurred a number. "30 business days" becomes "30 days". "Two months' rent" becomes "one month". "Per claim" becomes "per year".
• It merged two documents or two versions. A draft and a signed copy, or a contract and a later variation, can say different things. So can a main policy and its schedule.
• Your question assumed something. Ask "what's the fee for ending the lease early?" and a tool may find a fee even if the lease doesn't allow early exit at all.
• The document is silent, and the tool filled the gap. This is the most dangerous one, because the answer sounds reasonable.

What a useful citation lets you do

A citation is only useful if it lets you check the answer yourself. Here's the difference.

One test catches most problems: copy a distinctive phrase from the quoted passage and search for it in your own copy (Ctrl+F on Windows, Cmd+F on a Mac). If it isn't there, stop and treat the answer as unchecked.

The five-step check

This takes two to five minutes per answer. Do it for anything you're going to act on: signing, paying, cancelling, claiming or missing a deadline.

1. Find the passage in your own copy

Open the cited page and find the sentence. Don't rely on a preview or the tool's summary of it. Read the original words.

If the tool can't point you to a passage, ask it to. "Quote the sentence that says this, and give me the page" is a fair follow-up. If it still can't, the answer is a guess.

2. Read the sentences around it

Read the whole clause, plus the sentence before and after. Look for:

• Exceptions: unless, except, provided that, subject to, other than.
• Conditions: if, only if, when, as long as, on the condition that.
• Who it applies to: the landlord or the tenant, the employer or the employee, you or "the insured".
• Cross-references: "as set out in Schedule 2" means Schedule 2 is part of the answer.

3. Check the definitions and the numbers

If the passage uses a capitalised term, find its definition. Then check every number in the answer against the passage:

• days, business days or calendar months
• dates and who starts the clock ("from the date of notice", "from the end of the term")
• amounts, percentages and currency
• "per claim", "per year", "in total"

A wrong number is the most common error that still looks right.

4. Look for anything elsewhere that changes it

Contracts and policies often say one thing in the main text and change it later. Search your copy for the key word (notice, deposit, renew, exclusion, terminate) and skim every hit. Check:

• schedules, annexes and appendices
• special conditions, endorsements or addenda
• amendments, variations or a newer version
• an "order of precedence" clause that says which document wins

5. Ask the question a second way

Ask the opposite question, or a narrower one. If you asked "can I end the lease early?", also ask "what happens if I leave before the end of the term?" and "is there a break clause?". If the answers disagree, read the passages behind both.

If the stakes are high, the last step is a person. Take the passage, not the AI answer, to your landlord, insurer, HR team or a lawyer, and ask them to confirm in writing.

A worked example

Here is a made-up example. A renter uploads their lease and asks: "How much notice do I need to give to move out?"

The tool answers: "You need to give 30 days' notice."

1. Find the passage. The cited clause says: "The Tenant must give at least 60 days' written notice before the end of the Term." The answer got the number wrong.
2. Read around it. The next sentence says that if neither side gives notice, the lease continues month to month, and then 30 days' notice is enough. So the tool mixed two rules together.
3. Definitions and numbers. "Term" is defined as the 12-month fixed period. "Written" notice rules out a text message, according to the notices clause.
4. Look elsewhere. A special condition on the last page adds a break fee of one month's rent if the tenant leaves during the fixed term.
5. Ask it another way. "What happens if I leave before the end of the Term?" brings up the break fee clause directly.

The first answer wasn't invented from nothing. It was close. But acting on it could have cost the renter a month's rent. Five minutes of checking found the real answer.

Red flags

Slow down and check harder when you see any of these:

• No source at all, or a source that is just "the document".
• A quote you can't find in your copy.
• Words like "typically", "usually", "most contracts" or "in general". These often mean the answer came from general knowledge, not your file.
• A number in the answer that doesn't appear in the passage.
• An answer that is more certain than the passage. The clause says "may"; the answer says "will".
• A detailed answer to a question your document doesn't seem to cover.
• Different answers when you ask the same question twice.
• A page number beyond the end of your document.

When "the document doesn't say" is the right answer

Many documents don't cover everything. A lease may say nothing about pets. A job offer may not mention overtime. When that's true, the honest answer is "this document doesn't cover it", and the next step is to ask the other party or check the law where you live.

A tool that always has an answer is more worrying than one that sometimes says it can't find one.

How FileAI shows its work

FileAI is built around this check. Upload a PDF or Word file, ask in plain English, and each answer comes with numbered sources. Click one to see the passage it came from, highlighted on the page in PDFs, so steps 1 and 2 take seconds. If the document doesn't cover your question, FileAI tells you so instead of filling the gap.

FileAI answers only from the files you give it. It can still be wrong: it can pick a less relevant passage, or miss a second clause somewhere else that changes the answer. So read the highlighted passage before you rely on it, and use steps 3 to 5 for anything important. It doesn't give legal or financial advice. Scanned PDFs without selectable text aren't supported yet.

Two places to see it for yourself:

• Try the sample contract demo. No signup. Pick a question, such as whether the contract renews automatically, and click the source to see the highlighted passage.
• Read our accuracy results. We test FileAI's paid document checks on real public documents against an answer key, and publish what they miss as well as what they find.

For contracts, leases and policies, see how FileAI helps with contracts. You can start free, and pricing shows what's included. Your files are never used to train AI.

Sources

• National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), July 2024, section 2.2 (confabulation)

This guide is general information, not legal advice. For decisions with legal or financial consequences, confirm the meaning of your document with the other party or a qualified professional.