Offline Datasheet RAG: The Pipeline I Argued Against
I argued against RAG for hardware docs, then an offline job made me build one. What survived: SVD parsing, structural chunks, hybrid search, multi-query.
On this page
I wrote a whole post on grep vs RAG for hardware docs saying to skip RAG and use file tools. For a small set of documents and a cloud LLM, I still think that is right. Grep and read win there. Then a job came to me: search 47 datasheets on an air-gapped factory network. No Claude API. No Gemini. Only a local 7B model, 8K of context, and no way out to the internet.
Grep across 47 PDFs returns hundreds of hits for a common word like “clock” or “prescaler.” An 8K window cannot hold that many. So I needed retrieval, and there was no cloud API to fall back on.
So I built the same kind of RAG pipeline I had spent the previous post arguing against. Most of it failed the way I expected it to. The parts that survived had one thing in common. They were built for register tables and cross-references, not for blog posts and novels.
Ingestion: Parse the Structure Before You Chunk It
Every RAG tutorial begins with “load your PDFs.” For hardware docs, that is the exact step where things start to go differently.
PDF text extraction turns this register map from RM0090:
USART_BRR Offset: 0x08 Reset: 0x0000
Bits 15:4 DIV_Mantissa Mantissa of USARTDIV
Bits 3:0 DIV_Fraction Fraction of USARTDIV
into something like:
USART_BRR Offset: 0x08 Reset: 0x0000 Bits 15:4 DIV_Mantissa
Mantissa of USARTDIV Bits 3:0 DIV_Fraction Fraction of USARTDIV
The column alignment is gone. What used to be a table reaches the embedding model as a jumble of words. Cut that into 512-token fixed windows and DIV_Mantissa can end up in a different chunk from its bit range.
Two changes did most of the work.
Table detection on PDF text. Before chunking anything, I scan for whitespace-aligned columns and rewrite them as markdown pipe tables. Register maps, pin tables, electrical spec tables: they all follow predictable column patterns. Rewritten as | Field | Bits | Description |, the structure gets through chunking and embedding in one piece. That gave me more retrieval accuracy than any model change I tried.
SVD parsing. Every ARM Cortex-M vendor ships SVD files. They are XML, one entry per peripheral register with its base address, bit fields, and reset value. I parse them into structured markdown:
## USART2 (0x40004400)
### BRR (offset: 0x08)
| Field | Bits | Reset | Description |
|--------------|-------|-------|-------------------------|
| DIV_Mantissa | 15:4 | 0x000 | Mantissa of USARTDIV |
| DIV_Fraction | 3:0 | 0x0 | Fraction of USARTDIV |
Compare that with the broken PDF extraction above. Register-address retrieval went from ~60% on PDF-only chunks to 100% on SVD chunks. RespCode ran four LLMs on register lookups and ended up in the same place. SVD injection brought all four to 100%. SVD is the single highest-impact input you can add to a hardware RAG pipeline.
SVD does not replace the PDF, though. The reference manual still holds the initialization sequences, the timing constraints, the clock tree behavior. But when the question is “what address is this register” or “what bits do I set,” SVD is the ground truth.
Chunking: Give Each Chunk a Location
Standard RAG cuts chunks at a fixed token count. On hardware docs, that runs into two problems.
A chunk of APB1 prescaler bit descriptions looks the same whether it came from the RCC chapter or the USART chapter. The embedding has no idea where the chunk sits in the document, so it cannot connect related pieces across chapters.
The fix is to label each chunk with its place in the document hierarchy before embedding it.
[Document: RM0090.pdf]
Section: RCC > Clock configuration > APB1 prescaler
The APB low-speed prescaler (APB1) configures the APB1 clock
frequency. Bits 12:10 of RCC_CFGR set the division factor...
Now the heading path ties the embedding to “RCC” and “clock configuration.” On my 20-query test set against RM0090, cross-reference retrieval accuracy rose from 35% to 65%.
The second problem is chunk size, and neither choice is free. Small chunks retrieve precisely but lose the surrounding context. Large chunks keep the context but blur the embedding. So I index small, around 256-512 tokens, for precise hits, and then widen each hit at query time to include its neighbors. That way I get the exact register description that matched, along with the introduction and notes around it.
Search: Hybrid Vector + Keyword
Pure vector search breaks down on register notation. Ask “what clock source is the MCU using?” against a chunk that contains RCC_CFGR[SWS]: bits 3:2, read-only and the similarity score stays near zero. Embedding models learned natural language, not register shorthand.
None of the embedding models I tried (I ran three that fit on CPU) reliably found register definitions from natural-language questions.
Hybrid search fixed it. It combines vector similarity with full-text keyword search and merges the results from both. The keyword side catches what the embeddings miss. Search “SPI1_CR1 CPOL” and the keyword index finds the exact chunk even when the embedding model scores it low. Keyword search is exact. The term is either in the chunk or it is not.
Two adjustments helped:
Acronym boost. Queries with uppercase terms (SPI, UART, DMA) get extra weight on the keyword side. Hardware questions depend heavily on acronyms, and keyword search handles them well. A one-line change raised the hit rate on register queries from 72% to 81%.
Peripheral-aware scoping. I detect peripheral names in the query (UART1, SPI2, TIM3) and boost chunks whose heading path names that peripheral. “USART2” brings up chunks filed under “UART” headings. It stays a soft boost, not a hard filter, so cross-peripheral results still come through.
Across 20 register-lookup queries, hybrid search with peripheral boosting reached 85% accuracy. Vector-only search reached 52%.
One practical note. I keep all the embeddings in SQLite through sqlite-vec. It pages to disk, so the full embedding set for 47 datasheets never has to sit in RAM. On an 8GB machine that is already running a 7B model, that matters.
Multi-Query Decomposition for Cross-Peripheral Questions
The USART baud rate bug from the grep vs RAG post still got past everything above. A single query for “USART baud rate” finds the BRR register and never finds the APB1 clock prescaler in a different chapter. In meaning, the two terms are far apart. No amount of embedding or keyword tuning connects them.
What worked: before searching, give the question to the local LLM and have it split the question into sub-queries. “Configure USART2 at 9600 baud on STM32F407” becomes:
- USART2 BRR register configuration
- APB1 bus clock frequency
- RCC prescaler settings for APB1
Each sub-query goes through hybrid search, and the results are merged and deduplicated. Three retrievals where one had failed.
On 10 cross-peripheral test queries, single-query retrieval scored 30%. Multi-query reached 80%. All of that gain came from queries that span two peripherals, the ones where the model needs the register plus the clock or the interrupt source.
The cost is 2-3x the search lookups plus one extra LLM call, roughly 1-2 seconds on a 7B model. With an 8K context window, top-2 results per sub-query (6 total) did better than top-3 (9 total, which brings in too much noise).
What Still Breaks
Deep dependency chains. Multi-query handles a 2-hop query fine (register + clock source). Many embedded questions go deeper than that. “Set up DMA for USART2 RX” needs the DMA stream mapping table, the interrupt vector assignment, and the NVIC priority configuration. That is three hops, and the LLM cannot guess them in advance because each one depends on the result of the one before. This calls for iterative retrieval (search, read, search again), which is a different architecture. For now I add the hints by hand.
Timing diagrams. The pipeline handles text only. Most of the critical timing parameters also appear in tables, and table detection picks those up. What gets lost is the signal ordering and the edge relationships that live in waveform diagrams. A local model that can see images would help, but it needs another 4-5GB of RAM.
Niche MCUs. When the embedding model has never seen “SAMD21 GCLK” or “GD32F450 ENET,” vector search gets worse. The keyword component catches the exact matches as a fallback.
The Setup That Survived
The stack: nomic-embed-text-v1.5 for embeddings (CPU, no GPU needed), SQLite with sqlite-vec for vector storage, FTS5 for keyword search, and any 7B+ GGUF model for generation and query decomposition. It fits in 8GB RAM.
This pipeline runs inside NeuroTerm, the tool I’m building.
If I were starting from zero again, I would parse the SVD files first. That one change gave me the largest accuracy jump I saw, from ~60% to 100% on register lookups, and it came before I had changed chunking or search at all. Get your MCU’s SVD from cmsis-svd-data, convert it to structured markdown, and index it. Everything else in this post is tuning built on top of that.
Your Turn
Want this running over your own datasheets? Paste the goal into Claude Code’s /goal and keep the spec in your repo. The constraints are the whole point: offline, 8GB, 8K context. Hit those three and the pipeline below is what’s left standing.
Goal: Build an offline RAG over a folder of MCU datasheets + SVD files that runs
on an 8GB machine with no internet and a local 7B GGUF model. Done when, on a
20-query register-lookup set, SVD-backed register-address lookups return 100%
correct addresses, hybrid search hits at least 85% retrieval accuracy, and on a
10-query cross-peripheral set multi-query decomposition hits at least 80% -- with
no cloud API call anywhere in the pipeline.Spec (constraints first):
- Offline / air-gapped: no cloud LLM or embedding API, ever.
- 8GB RAM shared with a running 7B model; embeddings page to disk, not RAM.
- 8K context: retrieve few, tight chunks (top-2 per sub-query beat top-3).
- CPU-only embeddings, no GPU.
Requirements:
- Parse vendor SVD (XML) into structured markdown register tables first.
- Detect whitespace-aligned tables in PDF text, convert to markdown pipe tables.
- Prepend each chunk's document/section heading path before embedding.
- Index small (256-512 tokens); expand each hit to its neighbors at query time.
- Hybrid retrieval: vector similarity + FTS5 keyword, merged. Boost uppercase
acronyms and chunks whose heading path matches a peripheral named in the query.
- Multi-query: ask the local model to split a cross-peripheral question into
sub-queries, run each through hybrid search, merge and dedupe.
Non-goals:
- Vision / timing-diagram parsing (text-only pipeline).
- Iterative 3+ hop retrieval (add hints manually for now).
Acceptance:
- Register-address lookups 100% with SVD chunks (vs ~60% PDF-only).
- Hybrid + peripheral boost at least 85% on 20 register queries (vs 52% vector-only).
- Multi-query at least 80% on 10 cross-peripheral queries (vs 30% single-query).
Implementation Plan:
1. SVD to structured markdown (from cmsis-svd-data). Exit: register-address lookups 100%.
2. PDF table detection to markdown tables. Exit: register tables survive chunking.
3. Heading-path-prefixed chunking, small chunks + neighbor expansion. Exit: cross-ref queries up from ~35%.
4. Hybrid vector (nomic-embed-text-v1.5) + FTS5, SQLite/sqlite-vec storage, acronym + peripheral boost. Exit: at least 85% on register queries.
5. Multi-query decomposition, top-2 per sub-query. Exit: at least 80% on cross-peripheral queries.
Stack: nomic-embed-text-v1.5 (CPU), SQLite + sqlite-vec, FTS5, any 7B+ GGUF model. Fits 8GB RAM.
Starter prompt: "Build the pipeline in the spec, phase by phase. Start with phase 1
(SVD to structured markdown) and do not proceed until its exit check passes."
Comments
Loading comments...