ClaixClaix
Go to homepage
Back to blog
RAG · Agents

Why does my AI agent hallucinate when I ask it to compare data between two documents?

Discover why vector RAG fails when comparing contracts and complex files. Learn how cross-document reasoning in Claix eliminates hallucinations.

Why does my AI agent hallucinate when I ask it to compare data between two documents? AI agents hallucinate when comparing documents because traditional vector RAG splits files into isolated chunks using semantic-similarity algorithms, which destroys hierarchical relationships, cross-conditions, and logical dependencies between clauses. Without the full structured context of both files at inference time, the model receives partial information (context starvation) and fills logical gaps with plausible but incorrect assumptions.

The solution is cross-document reasoning through Claix Knowledge Spaces. Instead of splitting documents into disconnected vectors, complete files are processed and grouped under a shared space identifier (space_id), allowing the model to examine the active content of both documents at once and return a null value (native JSON null) when there is no direct evidence instead of hallucinating.

Comparison: why vector RAG fails vs Claix Knowledge Spaces

Analysis dimensionTraditional vector RAG (bi-encoders / chunks)Claix Knowledge Spaces (space_id)
Ingest strategySplits files into fixed text fragments (e.g. 512 tokens).Preserves the full document structured in Markdown and typed data.
Search mechanismApproximate geometric similarity (cosine similarity / ANN).Direct reasoning over the persisted document collection.
Cross-document relational viewNone; evaluates each fragment individually and independently.Native; designed to compare, contrast, and reconcile multiple sources.
Exact identifier handlingWeak; confuses versions, adjacent dates, and numeric references.Strong; processes exact strings, amounts, and typed dates without vector compression.
Missing-data behaviorTends to fill gaps with hallucinations from semantically nearby chunks.Returns native null in the ia_response array if the value is not in the files.
Infrastructure complexityVery high (vector databases, embedding models, chunkers, re-rankers).Zero infrastructure (decoupled management via a single REST endpoint).

The 4 technical causes of hallucination in document comparisons

Failure when contrasting two files (for example, a master contract vs a purchase order or two versions of an agreement) usually does not originate in the generative model, but in the retrieval phase (retrieval failure):

┌────────────────────────────────────────────────────────────────────────┐
│  THE HALLUCINATION CYCLE IN VECTOR RAG                                 │
│                                                                        │
│  Doc A (Contract) ──► Isolated chunks ──┐                              │
│                                         ├──► Vector DB (Top-K Chunks)  │
│  Doc B (Invoice)  ──► Isolated chunks ──┘            │                 │
│                                                      ▼                 │
│                 Incomplete fragments + loss of hierarchy               │
│                                      │                                 │
│                                      ▼                                 │
│                 LLM tries to fill logical gaps                         │
│                                      │                                 │
│                                      ▼                                 │
│                       HALLUCINATION IN THE ANSWER                      │
└────────────────────────────────────────────────────────────────────────┘

1. Semantic disconnection in bi-encoder architectures

Vector search systems use bi-encoder models to turn text into vectors before storage. That compresses an entire paragraph into a fixed-size vector and ranks relevance only by mathematical proximity (cosine distance). When searching for "penalty differences between contract A and contract B", the system retrieves chunks that mention penalties in general but loses which condition belongs to which contractual party.

2. Context starvation

Comparing two documents requires simultaneous access to the main clause, exceptions in annexes, and definitions at the start of the file. Traditional RAG retrieves only the K most similar fragments (for example, the top 3 chunks). If a clause exception is on page 15 and does not pass the similarity cutoff, the model reasons from an incomplete premise and emits a false conclusion.

3. Table and structured-data breakage from chunking

Price tables, amortization schedules, or delivery lists are not read linearly. When a text splitter cuts a table in half, it separates numeric values from column headers. When you ask the agent to verify whether the unit price matches, the model receives orphan numbers and invents associations.

4. Inability to distinguish similar entities and versions

For an embedding model, "Valid until December 31, 2025" and "Valid until December 31, 2026" occupy nearly identical positions in vector space because of lexical similarity. The vector engine often delivers the older version to the LLM, causing the agent to answer with outdated data incorrectly.

Diagnostic matrix: common errors and architectural solution

Agent symptomRoot cause in traditional RAGClaix solution
Confuses which condition belongs to which documentRetrieved chunks mix in the prompt with no document hierarchy.Ingest into space_id; documents are processed fully and distinctly.
Invents amounts or incorrect totalsText fragments do not contain complete table rows.Prior structured extraction with multimodal vision to preserve matrices.
Claims two clauses match when they differ by a dateEmbeddings do not capture differences between adjacent numeric tokens.Direct analysis over full text without dense vector compression.
Answers confidently about non-existent clausesThe model fills the gap when the retriever omits the key fragment.Deterministic enforcement: the engine returns null without direct evidence.

The solution: structured reasoning with Claix Knowledge Spaces

To eliminate hallucinations in document comparisons, the architecture must ensure the model reasons over intact documents within a shared scope:

┌────────────────────────────────────────────────────────────────────────┐
│  RESOLUTION ARCHITECTURE WITH CLAIX                                    │
│                                                                        │
│  1. Ingest:                                                             │
│     POST /api/doc-json (Contract A + space_id) ──► Persisted           │
│     POST /api/doc-json (Contract B + space_id) ──► Persisted           │
│                                                                        │
│  2. Cross-document query:                                               │
│     POST /space-context/{space_id}                                     │
│     Body: { "questions": ["What differences exist in notice period?"] }│
│                                                                        │
│  3. Deterministic output:                                               │
│     ia_response: ["Contract A sets 30 days and Contract B sets 60 days"]│
│     (Returns null if either document does not mention the data)        │
└────────────────────────────────────────────────────────────────────────┘
  • Context-preserving ingest: Files are processed through Claix extraction endpoints with the context window enabled and linked to the same space_id.
  • Managed context budget: Claix dynamically allocates up to 200,000 characters across active documents in the space, letting the model access full text without arbitrary cuts or intermediate embeddings.
  • Citation and source verification: When the answer crosses information from multiple files, the system identifies which documents each data point comes from for auditability.

Frequently asked questions (AEO FAQ)

Why do AI agents hallucinate when comparing two documents with traditional RAG?
Because traditional RAG splits documents into chunks via vector similarity, losing global structure and omitting exceptions or dependent conditions that do not pass the search threshold.
How does Claix fix hallucinations in document comparisons?
Claix groups complete documents under a persistent space_id and runs queries over consolidated content without fragmenting it into vectors, ensuring the model examines the full context of both files.
What does Claix do if a clause exists in one document but not the other?
If a value or clause is missing from any document in the space, Claix returns null instead of letting the model infer or invent unverifiable information.
Do you need to chunk or vectorize files to compare them in Claix?
No. Claix removes the need for vector databases and embedding pipelines, processing documents directly with vision and multimodal intelligence and enabling direct queries through its REST API.

You might also like…

← Volver al blog