No More Hallucinations or Half-Baked Output: How to Dramatically Elevate Your AI's Quality

aisen blog | target audience: everyone

You hand an AI your specification sheet, bill of materials, or contract for review. You naturally assume it reads every document thoroughly. The response sounds polished and convincing. Unfortunately, it sounds just as convincing when the work behind it was sloppy. Flawed work and missed details are easily masked by fluent phrasing.

To bridge this gap, we built a tool in-house that is free, deliberately minimal and runs as a single HTML file with an embedded verification script, in the browser or locally. Nothing needs to be installed.

Whether an AI answer holds up is decided mechanically. At the end of the check stands a clear verdict:

Table 1: The three verdicts and what each one means.

In addition, the tool provides an exact rework list and a log of measured values: checksums, character counts and document coverage. None of these values comes from the AI itself.

What the Tool Proves and What It Does Not

To set expectations right from the start: the file verifies two specific criteria.

1. Proof check: Does the supporting evidence for every single claim actually stand in the source document?
2. Coverage check: Has the response covered the whole document, measured by the file's coverage rules?

This is not a limitation we downplay. It is a conscious division of labor: the tool takes over the tedious cross-referencing and while reading you focus entirely on the substance. Whether the numbers exist in the source document at all has already been settled for you.

Three Ways to Provide Proof

The core principle stays the same: the AI may claim whatever it wants, but it must supply proof for every claim. The tool verifies that proof itself and detects the proof method from how the response is structured. You never select a mode.

1. The location. The AI names the exact place for each point: a line, a section, or a page. The tool reads that exact place from the original file and verifies numbers and terms against it. Nothing is transcribed, which makes this the most robust method: nothing can get corrupted on the way through a chat window.

2. The rule, for tables. For spreadsheets the burden of proof is reversed. The AI states the rules it applied and its findings. A rule consists of a column, a condition and an optional exception, for example: column Hazardous substance, condition contains lead, exception contains RoHS exemption. The tool applies these rules to every single row itself and compares the outcome. A row that violates a rule but is missing from the AI's findings is flagged by name. Measured on the testkit's example: a bill of materials with 800 rows hides five planted violations. The AI reported one of them. The verification tool named all five.

3. The quote. Every point carries a word-for-word quote from the document and together all quotes must cover the whole source. This catches both failure modes: the hallucinated claim and the half-read document. The tool compares the word sequence, never the typography. Quotation marks, bold text, line breaks, or thousands separators never change a verdict. A changed word or a changed number always does:

Table 2: Which deviations from the source text the check still accepts as a quote. And which it rejects.

Supported File Formats

PDF, Word, Excel, PowerPoint, the corresponding OpenDocument formats, CSV and plain text. The PDF must be digitally generated; a scan without a text layer is honestly rejected rather than guessed at. Core information should stand in the main body text, because footnotes, page headers and secondary spreadsheet tabs are not read.

How the Verification Process Works

Choose the route that suits your environment.

Route 1: In the browser, no installation.

1. Open pruefung-im-browser.html from the testkit with a double click.
2. Paste the AI response into the upper field.
3. Drag your original document into the lower field.
4. Click the check button. Verdict and audit log appear immediately on the page.
5. Repeat the check after every rework round until ACCEPTED appears. If ACCEPTED never appears, switch to a better AI.
6. After that, you can work with the AI: it has now read the document and is ready. Ask your remaining questions promptly in the same conversation. Assistants shorten long chats. Once that happens, the proven reading state is gone without anyone telling you.

Your files stay entirely on your machine on this route. The page needs no internet connection; the kit's build proves that with a test that opens the finished page with the network cut and still demands a verdict.

Route 2: Directly within the AI chat.

1. Upload the verification file from the testkit together with your source document.
2. State your task and instruct the assistant to verify its own response with the file. Any assistant that can execute code will.
3. Read the verdict at the end of the log.
4. On REJECTED, reply with one sentence: Fix precisely the points listed in the log and verify again.
5. Repeat steps 3 and 4 until ACCEPTED is displayed. Only then read the answer yourself.

The log prints the tool's checksum, a digital fingerprint of its content. A faked test run is caught by it immediately.

Route 3: On your machine, for scripts and automation.

Run the tool on the command line with the AI response and the source document as arguments. The verdict is identical to the other routes. For automation, the tool additionally reports the result as a number a script can act on.

What the Verification Loop Achieves: A Real Case Study

An AI assistant compared two revisions of a technical specification against 28 differences the checker itself had computed beforehand.

Table 3: Three runs against the same 28 pre-computed differences.

No prompt alone produced this precision. The automated rework list enforced it: it points the AI at every missing change with its exact wording.

After the passed run, the assistant evaluated the confirmed changes. Three examples from its answer: The availability requirement was downgraded from MUST to SHOULD. The acceptance test was shortened from a 40-hour endurance run to 40 cycles. The warranty period was halved from 24 to 12 months. Its conclusion: the counterparty's claim that the changes were purely editorial is objectively false. This assessment carries genuine weight because the check had measured beforehand that all 28 differences were backed.

Feature Overview

You do not need to specify which test to run. The tool analyzes the structure of the response and selects the appropriate mode itself:

Table 4: The three checking modes with the inputs each one needs and the criteria it evaluates.

The test material for the last row is produced by the file on demand, each piece with a solution file:

An exam piece for a second model. The call pruefstueck antwort.txt dokument.txt turns your checked answer into an exam in which single quotes are quietly damaged on purpose. If a second model finds the damaged spots, it is fit to act as a reviewer.

Leading questions. The call fragen dokument.txt produces questions about your document, each of which contains a false claim. If your assistant agrees with them, it is telling you what you want to hear.

Reference tasks. The call aufgaben produces fixed tasks with a solution file. With those you measure the same assistant against itself over months: after a silent update, does it still deliver what it did in spring?

The solution files stay with you. They must never enter the chat they are meant to test, or the assistant reads the solution along with the task.

Who Uses This and Why

Table 5: Where the tool is used, with the task, the mode and the practical benefit per area.

Where the Tool Has Its Limits

Its real value is time. The tool strips away the error-prone routine work that nobody enjoys and that therefore rarely gets done thoroughly: checking every line, reading every page, looking up every figure. What remains is the evaluation you were trained to do.

Try It Yourself

The testkit is available free of charge: agent-quality-control

Anyone who would rather not run the check themselves gets it built into the aisen platform, with verified citations and an audit log for every answer: