how your AI finds the right page

RAG vs STAIR

RAG loose chunks STAIR climb the table of contents vs
Opening: same goal — feed the LLM the right text. Totally different way of finding it.
STAIR = “STructure Aware Information Retriever” · IBM · arXiv 2609.03874 · Sep 2026
1 · the problem both are solving

The book doesn’t fit

your docs · 1,000 pages LLM context window too small — pick a slice!
Retrieval = deciding which slice of the corpus the model gets to read.
The whole fight: RAG and STAIR only disagree on how to pick the slice.
2 · the classic recipe

RAG: chop → embed → match

book chop ✂ chunks (by length) embed → numbers vector DB 0.12, −0.83, … “what is X?” embed the query too top-k “nearest” chunks LLM writes answer chapters? headings? thrown away!
Retrieval by similarity math: “which chunks look like the query?”
3 · where RAG hurts

3 classic failures

answer cut in half!
Chunk boundary splits the fact → neither half retrieves well
“similar” ✓ relevant picked this… …answer was here
Similar ≠ relevant — embeddings match words, not intent
vector DB embedding model chunker + re-index a whole stack to babysit
Docs change → re-chunk, re-embed, re-index. Forever.
Root cause: chopping by length destroys the meaning the author already gave you — the structure.
4 · the STAIR idea

Read it like a human

Contents 1. Basics 2. Methods 2.1 Old way 2.2 New way 3. Results 4. Advanced 4.2 Edge cases “that’s the one” STAIR ( query , ToC ) → section one LLM call. it reads the outline, names the leaf section that would contain the answer.
The retriever navigates the author’s outline — like you would with a real book.
Key move: the index is not a database — it’s just the table of contents, in the prompt.
5 · the STAIR recipe

Route → fetch → answer

“what is X?” ToC LLM #1 the router § 4.2 just a name! fetch its text LLM #2 answers, as usual vector DB · embeddings · chunker not needed
Retrieval by navigation: “which section would contain the answer?”
Bonus: force LLM #1 to output only valid section names (an enum) → it can’t hallucinate a section. Paper measured <0.05%.
6 · the whole difference, one picture

Look-alike vs would-contain

the query RAG asks: “which chunks LOOK LIKE these words?” cosine( query , chunk ) — geometry STAIR asks: “which section WOULD CONTAIN the answer?” LLM reasons over the outline — comprehension
Similarity search vs. reading comprehension of the outline. Everything else follows from this.
7 · a real experiment — with a real LLM

Meet the tiny book

A Tiny Field Guide
to Coffee
1 Basics 1.1 What coffee is 1.2 Freshness & storage 2 Brewing 2.1 Dose & strength 2.2 Water & temperature 3 Troubleshooting 3.1 Bitter coffee 3.2 Sour coffee 4 Gear 4.1 Grinders 4.2 Kettles & scales
8 sections · ~300 words — small enough to watch every single step under the microscope 🔬
Keep an eye on §2.1: it holds the answer to “how do I make my coffee stronger?” — 75 g per litre. We’ll ask exactly that.
8 · what RAG actually does to it

The book, after chunking

cut every 125 characters — titles gone, chapter boundaries ignored. This is the real output of the chunker:
Spot the crime: chunk #3 ends “…instead raise the” and chunk #4 starts “dose to seventy-five grams…” — the answer got cut in half. And chunk #2 is a lexical trap: “strong smells… strong spices”.
Then each card becomes numbers. Real embedding of chunk #3 (text-embedding-3-small): — 1,536 dimensions per chunk. Meaning became geometry.
9 · the microscope — follow one question through both pipelines

Ask the book

● checking… the easy one the trap ⚠ the paraphrase

🔵 the RAG journey

🟠 the STAIR journey

10 · does it work at scale? (paper’s numbers)

Scoreboard — Recall@1

STAIR STAIR — 82.6% 82.6 DSI (fine-tuned) DSI fine-tuned — 76.9% 76.9 DPR (dense RAG) DPR — 68.7% 68.7 BM25 (keywords) BM25 — 59.5% 59.5 Mistral 0-shot Mistral 7B zero-shot — 13.8% 13.8 significantly ahead ↑ same 7B model, no ToC, no training — the ToC is doing the work
% of questions where the #1 retrieved section was the right one · SearchTome benchmark (18 textbooks, 6 domains) · fine-tuned Mistral-7B
Also: on rare sections with few training examples, STAIR degrades much less than DSI — the outline generalizes.
11 · the fine print

Caveats — read before adopting

no ToC? no STAIR.
Needs real structure. Chat logs, tickets, note piles → classic RAG still wins (or build a ToC first).
a whole section… …vs a chunk $$ tokens
Coarse retrieval. You get whole sections — long ones cost context. Fix: route to the section, then rank chunks inside it.
prompt 10k docs?
The ToC must fit the prompt. Huge corpora → route in two hops: pick document, then pick section.
18 tidy textbooks ≠ your wiki unproven at scale
Eval is narrow. Clean books only — messy enterprise corpora untested. Synthetic training questions, too.
12 · the interesting bits

Extra tips ✓

Tip 1 — you probably don’t need the fine-tune. The 13.8% baseline is a tiny 7B model. A frontier model reading a ToC zero-shot is already a strong router → prototype in an afternoon, distill to a small model later if volume demands.
Tip 2 — constrain the router’s output to the list of real section IDs (enum / structured output) and retrieval hallucination is zero by construction — no training required for that property.
Tip 3 — it composes with RAG, not against it: ToC-route to the section, then embed-rank inside the section. Best of both on long sections.
Tip 4 — no structure? Induce it: cluster + summarize your corpus into a synthetic ToC (that’s the RAPTOR lineage). The paper lists this as future work.
Big picture — routing over a fixed set of section IDs is a pure classification problem. It doesn’t inherently need a generative model at all — any fast, calibrated classifier over ToC leaves can be the retriever.
13 · which one, when?

Cheat sheet

does the corpus have real structure? yes → no → STAIR-style routing manuals · docs sites · legal codes long sections? rank chunks inside classic RAG chat logs · tickets · note piles …or induce a ToC, then route
Rule of thumb: if a human would reach for the table of contents, so should your retriever.

That’s the whole idea

  • RAG = chop by length, match by similarity — works anywhere, loses structure.
  • STAIR = keep the outline, let an LLM navigate it — sharper when structure exists.
  • They compose: route the section, rank the chunks.
Try it: zero-shot router + constrained output = a weekend project. No vector DB required.
paper: arXiv 2609.03874 (IBM) · community repro: model / dataset on Hugging Face