Hamza Ouhssaine
Contact

Sanad سند

Agentic RAG that shows the page it answered from

Sanad answers questions about a folder of documents. Every claim in an answer carries a marker, and every marker resolves to a page and a bounding box (or a block and character offsets): click it and the original file opens with the exact passage highlighted. When the documents don’t hold the answer, the agent says so instead of guessing.

Type
Personal project, 2026, designed and built end to end
Reads
PDF, DOCX, PPTX, XLSX, HTML, Markdown, text, JSON and images
Backend
Python, FastAPI, LangGraph, ChromaDB, SQLite FTS5, PyMuPDF, SQLModel
Models
Claude for answers and vision OCR; OpenAI or local embeddings (BGE, E5, MiniLM)
Frontend
Angular, with a live search trace and a PDF/HTML source viewer
Tests
Over 350 offline pytest tests
Sanad’s chat on the left answers “how is the OCR done in article OCR Unlimited?” with eight numbered source chips; on the right, the cited PDF is open on page 10 with the quoted sentences outlined.
An answer assembled from eight passages of one paper. Clicking source 8 opens the PDF on page 10 with the quoted sentences outlined.

The name

سَنَد, sanad, from the root س-ن-د, “to lean on”: the thing a claim rests on. The word has three meanings, and the project sits where they meet.

A support
What holds something up. An answer with nothing under it is just a plausible sentence.
A chain of transmission
In classical Islamic scholarship the sanad is the chain of narrators attached to a report. A report was judged on whether its chain traced back to a source, not on how convincing it sounded. That is this system’s contract: every claim carries a marker, every marker resolves to a page, and a quote that can’t be found in the passage it cites is flagged.
A supporting document
In commercial Arabic a sanad is the paper behind a transaction: سند التسليم, a delivery note; سند الطلب, a purchase order. The exact paperwork Sanad was built to read.

How a document becomes searchable

Every file in a project’s folder goes through the same pipeline, whatever its format. Follow one photographed delivery note.

One photographed delivery note, from the folder to the index
New file dropped in the folder Parse to blocks 9 formats, one type Vision OCR scans and photos Render HTML an id per block Chunk structure, then tokens Contextualise a sentence per chunk Embed the project’s model SQLite FTS5 BM25 keywords ChromaDB a collection per model New file dropped in the folder Parse to blocks 9 formats, one type Vision OCR scans and photos Render HTML an id per block Chunk structure, then tokens Contextualise a sentence per chunk Embed the project’s model SQLite FTS5 BM25 keywords ChromaDB a collection per model
  • Input or output
  • LLM call
  • Code
  • Data store
  1. New file A photographed delivery note lands in the project folder; the watcher queues it.
  2. Parse to blocks It is an image, so the parser hands it to vision.
  3. Vision OCR Claude vision transcribes the note and describes it, so a photo becomes a document like any other.
  4. Parse to blocks Back as blocks, with headings, tables and records kept. Non-PDF files are also rendered to HTML with a stable id per block, for the viewer.
  5. Chunk Headings and records start new chunks; text packs to about 700 tokens with 15% overlap, and a table row is never cut in two.
  6. Contextualise A cheap model writes one or two sentences placing each chunk in its document, for the embedding only. Cached by file hash.
  7. Embed Embedded with the project’s model; the same text goes to SQLite FTS5 for keyword search.
  8. ChromaDB Stored in the collection for this project and model, so vector spaces never mix. Paused work resumes at the chunk it stopped on.

  1. Parse every format into one block type

    PDF, Word, PowerPoint (speaker notes included), Excel, saved web pages, Markdown, JSON and images all become the same kind of block, with headings, tables and records kept. Scans and photos go through Claude vision, transcribed and described, so a photographed delivery note is a document like any other.

  2. Chunk on structure, then on tokens

    Headings, sheets and records start new chunks first; text is then packed to about 700 tokens with 15% overlap that never cuts a sentence. A table is always its own chunk; a large one splits by rows and repeats its header in every part, so no row is ever cut in two.

  3. Situate each chunk in its document

    Contextual Retrieval: a cheap model writes one or two sentences placing each chunk in its document, prepended to the embedded text only, never to what the user sees or what a citation is matched against. A row reading “| 80 | 6.00 | 480.00 |” becomes findable by a question about accessory pricing. Cached by file hash, so a re-ingest costs nothing.

  4. Render once, cite forever

    Every non-PDF file is converted at ingest into HTML with a stable id on each paragraph, row and record. The viewer then needs only two paths, PDF and HTML, and a chunk’s character offsets line up with the rendition because both came from the same parse.

  5. Embed into two indexes

    Vectors go to ChromaDB, in one collection per project and embedding model, so switching models never mixes vector spaces. The same text goes to SQLite FTS5 for keyword search.

  6. Keep the index true to the folder

    Each project owns a folder. A watcher ingests files as they appear, a reconciler rescans at startup, and a progress bar follows each file through parsing, chunking, contextualising and embedding. Any file, or the whole queue, can be paused and resumed at the chunk it stopped on, with no vision or LLM call paid twice, even across a restart.

Two searches, one ranking

Embeddings generalise but miss exact strings; a question about a part reference, an invoice total or a code like VT-COB/SKU98496 is lexical. BM25 gets those, embeddings get “what did the report conclude about growth”. Both run on every search.

Cosine similarity and BM25 scores live on incompatible scales, so they are fused with Reciprocal Rank Fusion, which only needs ranks: nothing to calibrate per corpus, and it stays correct when one retriever returns nothing. A cross-encoder then reads each of the 20 candidates together with the question, which is far more accurate and only affordable on a short list. The model is sent each chunk’s parent section: the chunk is what matched, the section is what answers.

Candidates kept at each stage of one search (default settings). Select a stage.

Embeddings find passages that mean the same thing, even with different words.

The 6 survivors are sent to the model as their parent sections, each tagged with file, page and section so it can cite them.

An agent that can decline

A LangGraph agent rather than single-shot RAG, because multi-part questions need more than one search, and because “I searched and the answer isn’t here” is a conclusion worth reaching on purpose.

One follow-up question through the LangGraph agent
Question a chat follow-up Rewrite standalone query Plan route the turn Small talk replies, no retrieval Decompose 2 to 4 sub-questions Agent calls search tools Hybrid search BM25, vectors, rerank Critique is this enough? Answer JSON, quote per claim Resolve citations quote to page and box Cited answer opens the source Question a chat follow-up Rewrite standalone query Plan route the turn Small talk replies, no retrieval Decompose 2 to 4 sub-questions Agent calls search tools Hybrid search BM25, vectors, rerank Critique is this enough? Answer JSON, quote per claim Resolve citations quote to page and box Cited answer opens the source
  • Input or output
  • LLM call
  • Code
  1. Question “And the second one?”, asked right after a question about two product references.
  2. Rewrite Uses the last six turns to make it standalone: “unit price of reference 40564101 in produits.xlsx”.
  3. Plan Classifies the turn as a lookup, so there is nothing to decompose. Small talk would end here, without retrieval.
  4. Agent The first round is forced to search, so the model can’t answer from memory. It calls the search tool.
  5. Hybrid search BM25 and vectors return 30 candidates each; Reciprocal Rank Fusion keeps 20 and the cross-encoder keeps 6, sent as their parent sections.
  6. Agent Has evidence and stops calling tools.
  7. Critique Judges whether the evidence answers the question: sufficient.
  8. Answer Writes JSON: the answer with a marker on every claim, and a verbatim quote for each citation.
  9. Resolve citations Finds each quote in its chunk (exact, normalised, then fuzzy) and maps it to page coordinates. Below a score of 72 it is shown as unverified.
  10. Cited answer The answer with a source chip per claim; clicking one opens the file with the quoted lines highlighted.

Other routes through the graph

  • Small talk: the plan node routes to a reply and the graph ends, with no retrieval.
  • Multi-hop and comparison questions: decomposed into 2 to 4 parts, each searched separately.
  • Tools ran but found nothing: the agent searches once more with different wording.
  • Critique finds the evidence insufficient: back to the agent with a better query, at most 3 rounds in all.
  • The model stops calling tools, hits the round limit or errors: straight to the answer, which says what is missing.

The answer node returns structured JSON. Every claim carries a marker, and every quote is copied character for character so it can be found again:

{
  "answer_markdown": "Le total est 82 900,00 DH [1], conforme au bon de livraison [2].",
  "citations": [
    { "marker": 1, "source": 1, "quote": "Total : 3 240 unites - 82 900,00 DH" }
  ],
  "sufficient": true
}

Every citation is checked

Models quote almost verbatim: they straighten quotation marks, collapse spaces, drop a final period. So each quote is located in its source chunk in three passes, and the score decides how it is shown.

How a quote is matched to its source
PassScore
Exact substring100
Normalised substring (whitespace, curly quotes, dashes)97
Fuzzy sliding window, then refinedits ratio

In a PDF the quote becomes a page and bounding boxes, trying shorter fragments when a line break gets in the way; in every other format it becomes a block id and character offsets in the HTML rendition. Below a score of 72 the viewer highlights the whole chunk instead: a confident highlight on the wrong line is worse than a vague one.

  • Verified citations

    A quote that matches its chunk with a score of 72 or more is verified. Below that it is shown in amber, marked unverified, and the answer gets a “2 of 3 sources verified” badge. Unverified citations are never deleted: a near-miss is usually a real source lightly paraphrased, and hiding it would hide a retrieval problem.

  • Uncited figures

    A sentence that states a number without a marker is counted. On invoices and delivery notes an invented total is the failure that matters, and a citation count alone can’t see it.

  • Orphan markers

    A marker in the text with no matching source is caught and logged.

Measured, not asserted

The evaluation harness scores retrieval with recall@k, MRR and nDCG per category, and full answers with groundedness and refusal accuracy. Retrieval-only runs call no LLM, so sweeping six embedding models costs nothing. Gold questions include unanswerable ones on purpose: a system that answers everything scores perfectly on recall and is useless.

Mean reciprocal rank on the sample corpus, by retriever and embedding model
all-MiniLM-L6-v2 bge-small-en-v1.5
Vector only 0.53 0.81
BM25 only 0.57 0.57
Hybrid + rerank 0.94 0.94

Scale 0 to 1; higher means the right passage ranked nearer the top.

The table shows why the retriever being scored matters. Through the full pipeline a weak embedder and a strong one score the same, because BM25 and the reranker rescue the weak one. To choose an embedding model, score vectors alone; to judge the system, score it whole. Comparisons are per question, so a flat average hiding swings in both directions shows up as the regression it is.

The weakest category is aggregation (recall@6 0.50, MRR 0.10 with MiniLM): RAG retrieves passages, it doesn’t add up three totals across three files. A compute tool over retrieved table rows is the fix, and the harness exists so that it gets built on a number, not an assertion.

Engineering choices

  • Models are a runtime choice

    The embedding model is chosen per project (a folder of French and Arabic delivery notes wants BGE-M3, English docs do better on BGE-base); the answer model is set instance-wide. Both switch without a restart, and a re-index runs in the background while chat keeps answering on the old model.

  • English-only models say so

    They don’t fail loudly on Arabic or French, they retrieve nonsense confidently, so the picker flags them in amber.

  • Query and documents share one model

    The model is resolved once per query and feeds both the embedder and the collection name. If a collection’s metadata disagrees, the search raises an error rather than returning wrong passages.

  • A paused document is invisible

    Keeping paid-for work during a pause must not make half a document answerable, so vectors wait in a spool and reach the index in one write.

  • Shows its work

    Each search reports its stages live (embedded, semantic, keyword, merged, reranked), then folds into “how this was found” under the answer, stored with the message.

  • Runs offline

    With a local embedding model, ingestion and retrieval need no embedding API key; only answer generation calls an LLM. Every endpoint call and internal action is logged to the database.

Stack

Agent
LangGraph, tool calling, structured JSON answers, SQLite checkpoints, rapidfuzz quote matching
Retrieval
ChromaDB, SQLite FTS5 (BM25), Reciprocal Rank Fusion, bge-reranker-v2-m3
Ingestion
PyMuPDF, python-docx / python-pptx / openpyxl, Claude vision OCR, tiktoken, watchdog
Backend
Python, FastAPI, Server-Sent Events, SQLModel, Alembic
Frontend
Angular 20 (signals), Tailwind CSS, Server-Sent Events reader
Quality
pytest, golden question sets, recall@k, MRR, nDCG, groundedness

Building something similar?

I’m happy to talk through retrieval over messy business documents, citation you can audit, or how to evaluate a RAG system before trusting it.

Email me Back to the CV