← Essays /Post · 27 of 28 · Methods

Hardware Docs for LLMs: Why I Went Back to grep

RAG cut the cross-references that make hardware docs useful. After wrestling with a pipeline, I went back to handing the LLM my SVD and PDF as file tools.

·9 min read · · · #rag#ai-agents#embedded-systems#stm32#datasheets
Hardware Docs for LLMs: Why I Went Back to grep
On this page

I asked Claude to set up USART2 for 9600 baud on an STM32F407. The code compiled fine. The serial terminal printed nothing but garbage.

Looking back, the problem was not in the LLM’s reasoning. It was in retrieval. The USART baud rate formula depends on the peripheral clock frequency, and that value is written in a chapter far away from the USART registers. My RAG pipeline brought back one chapter and missed the other. The LLM filled the empty space by inventing a clock frequency.

I spent two weeks on a proper RAG system just for RM0090: Docling for PDF parsing, ChromaDB for hybrid search, a routing layer on top. Then I realized, a little flatly, that I could have handed the LLM the file paths and let it grep.

When grep wins, when RAG wins

The rest of this post is my case against RAG for hardware docs. Before I make it, I want to mark where it stops, because it is a much narrower claim than “RAG is bad.”

SituationWhat to use
A handful of docs, cloud LLM, large context windowgrep and read, as file tools
Air-gapped or local model with a small context windowretrieval, built around the document structure
Dozens of datasheets, “which MCU has X” queriesan index of some kind, not necessarily RAG

Row one is what most firmware work looks like, and it is what this post is about. Row two is a real constraint I ran into later on an air-gapped factory network, and there I needed a retrieval pipeline rebuilt for datasheets. What separates the two is not the type of document. It is whether you can put the pages in front of the model at all.

Why RAG Breaks on Hardware Docs

Most RAG research tests on a different kind of document from the start. LaRA (Feb 2025, 2,326 test cases) covered novels, academic papers, and financial statements. Self-Route (Jul 2024) used NLP papers, multi-field documents, and multi-hop QA benchmarks. The Databricks study ran 2,000 experiments on product docs, financial, and general QA datasets. None of them tested on hardware documentation, and hardware documentation fails in ways that are different at the level of structure.

Chunking destroys cross-references. RM0090 Section 30.3.4 holds the baud rate formula. The clock frequency that formula needs is in Section 7.2, hundreds of pages away. Standard RAG retrieves each chunk on its own. The Late Chunking paper (Sep 2024) named exactly this problem: phrases that lean on earlier context end up with much weaker embeddings once they are cut into chunks. In hardware docs, “this register” pointing back to a definition three chapters earlier is ordinary.

Embedding models can’t index register tables. A description like RCC_CFGR[SWS]: bits 3:2, read-only packs a great deal of meaning into 15 tokens. Embedding models trained on natural language give these entries low similarity scores. Ask “what clock source is the MCU using?” and cosine similarity will not bring up the SWS field.

A wrong bit position is not “roughly correct.” If a system returns bit 13 where bit 12 belongs, the code writes to the wrong register field and the peripheral simply does nothing. RespCode (2026) recorded this across four LLMs. Gemini hallucinated SYSCON_BASE 0x50000000 for the LPC55S69 (the correct value is 0x40000000). 3 of the 4 models got the CTIMER2 clock enable bit wrong. All four compiled cleanly under arm-none-eabi-gcc. None of them ran on real hardware.

The LlamaIndex benchmark measured this gap head-on. On a small set of research papers (5 documents, 22-52 pages each), a filesystem agent (grep + read) scored 8.4/10 for correctness and RAG scored 6.4/10. Their own explanation: “RAG is bound to context loss due to chunking and sub-optimal retrieval calls, so the LLM generating the final answer has access to limited, sometimes incorrect context.”

Hardware docs make each of these worse. The information is denser, the cross-references carry the structure of the document, and the tolerance for error is zero.

The USART Baud Rate Bug

Almost everything that goes wrong when RAG meets hardware docs shows up in this one example.

I asked for USART2 on an STM32F407 at 168MHz: 9600 baud, 8N1, RX interrupt. RAG fetched the baud rate register description from RM0090 Section 30.3.4, with the USARTDIV formula attached. It missed the RCC configuration in Section 7.2, the part that says USART2 sits on APB1 at SYSCLK/4 = 42MHz. The LLM did see f_CK in the formula, but no chunk told it what f_CK is for USART2, so it guessed.

// What RAG generated (WRONG - assumes f_CK = 168MHz)
USART2->BRR = 168000000 / 9600;  // BRR = 17500

// Correct (f_CK = 42MHz for APB1 peripherals)
USART2->BRR = 42000000 / 9600;   // BRR = 4375
// Simplified formula for 16x oversampling (OVER8=0, default).
// Full: Baud = f_CK / (8 * (2 - OVER8) * USARTDIV). RM0090 Section 30.3.4.

It compiles. It runs. Then you connect at 9600 baud and garbage comes out. Because the code looks fine, this is the kind of bug that takes hours.

Changing the chunk size does not fix it. The baud rate formula and the clock tree live in different chapters. Cosine similarity cannot fetch “USART baud rate” and “RCC APB1 prescaler” together, because in meaning the two passages look unrelated. I don’t see this as a tuning problem. It is a problem of structure.

You Don’t Need RAG. You Need File Access.

The answer I reached was not a better RAG pipeline. It was to let the LLM open the documentation files directly and search them when it needs to.

Tool-based MCU documentation access architecture: project files and local docs (SVD, PDF) connected to LLM agent via tool calls (grep, read), producing verified register code

Architecture: the agent greps SVD and PDF files on demand instead of retrieving pre-chunked embeddings.

For a firmware engineer, two files cover almost everything.

SVD (System View Description): Every ARM Cortex-M silicon vendor ships one of these XML files. It holds the peripheral registers, base addresses, bit fields, reset values, and enumerated values for a specific MCU. The STM32F407 SVD alone has more than 1,000 register definitions across 58 peripherals. The cmsis-svd-data repository collects SVDs from many vendors in one place. RespCode parsed 2,177 SVD files, and once they gave the LLM the SVD, its accuracy on register value lookups went from a ~25% baseline to 100% on the tested fields.

PDF reference manual: RM0090 for the STM32F407 runs past 1,700 pages. It covers the clock tree, peripheral behavior, timing diagrams, initialization sequences, and electrical characteristics. The procedural knowledge is all here: how to bring up DMA, what breaks when you touch a prescaler, which GPIO alternate function maps to which peripheral.

The LLM does not need these files packed into its context window. It needs tool access: grep to search them, read to load only the section that matters. Claude Code (I covered its embedded Linux workflow in the embedded Linux AI agents post), Cursor, and similar tools already do this. You only have to tell them where the files are.

Why Tool Access Beats Full Context

You could put the whole reference manual into the context window. The cost is hard to justify. RM0090 is roughly 1.5M tokens. At Gemini 2.5 Pro pricing (long context at $2.50/M input tokens), one query costs about $3.75. Even with Gemini 2.5 Flash-Lite ($0.10/M) it is $0.15 per query, and you wait 30-60 seconds for the O(n^2) attention computation.

Tool-based access reads only what it needs. A typical query touches about 20-50K tokens of SVD data and PDF sections. That is 30-75x fewer tokens per query. Instead of taking in one huge context at once, the LLM makes several small reads, and its own search judgment decides where each one points.

The table below compares the approaches at Claude Sonnet 4.6 pricing ($3/M input, as of March 2026). In practice, loading the full context needs Gemini’s 1-2M context window.

ApproachTokens per queryEst. costLatencyAccuracy
RAG (chunked)~3K~$0.011-2sLow on cross-peripheral
Full context~1,500K~$4.5030-60sHigh but expensive
Tool access (grep+read)~20-50K~$0.06-0.1510-20sHigh

The cost estimates come from published API pricing and rough token counts. The tool access numbers assume 3-5 read/grep rounds per query.

The LlamaIndex benchmark points the same way. On small documents, agentic file search beat RAG by 2 points on correctness (8.4 vs 6.4). Those were research papers, not hardware docs. The direction still holds, and I expect the gap to grow on hardware docs, where cross-references are so dense.

How the Agent Solves the Baud Rate Bug

With tool access, the same USART2 request goes very differently. The agent greps the SVD for USART2 and BRR:

<!-- grep result from STM32F407.svd -->
<peripheral>
  <name>USART2</name>
  <baseAddress>0x40004400</baseAddress>
  ...
  <register>
    <name>BRR</name>
    <addressOffset>0x08</addressOffset>
    <size>0x20</size>
    <fields>
      <field><name>DIV_Mantissa</name><bitOffset>4</bitOffset><bitWidth>12</bitWidth></field>
      <field><name>DIV_Fraction</name><bitOffset>0</bitOffset><bitWidth>4</bitWidth></field>
    </fields>
  </register>
</peripheral>

Now it has the exact register address and field layout. The agent reads the PDF section on USART baud rate and sees f_CK in the formula. It does not guess. It greps the PDF for “APB1” and “USART2” and finds Section 7.2 with APB1 = SYSCLK/4 = 42MHz. Three tool calls, and the answer is right.

RAG can’t do this, because cosine similarity does not follow a cross-reference. The agent decides what to search next from what it just found, so it can follow the reference to the end.

Setting It Up

Minimal setup

Download your MCU’s SVD file from cmsis-svd-data, and get the reference manual PDF from the vendor site. Put both in your project’s docs/ folder. Then add this to your CLAUDE.md (or whatever your project context file is):

## Hardware Reference
- Target: STM32F407VG (Cortex-M4, 168MHz)
- SVD: docs/STM32F407.svd (register definitions, bit fields, addresses)
- Reference Manual: docs/RM0090.pdf (peripheral behavior, clock tree, init sequences)
- Datasheet: docs/DS8626.pdf (pinout, electrical characteristics, absolute max ratings)

When writing register-level code, always grep the SVD file for exact
register addresses and bit field definitions. Do not guess register values.
For clock configuration, read RM0090 Section 7 (RCC).

That is all of it. The agent now reads the same documentation I would have open in a PDF viewer. The only difference is that it can search it as a program.

What the agent still gets wrong

For popular MCUs like STM32, ESP32, and nRF52, the LLM has already seen most of the relevant documentation in training. It often gets the register configuration right with no documents at all. In those cases the SVD file is less a primary source than a way to check.

Where documentation access really decides the result is somewhere else: minor or niche MCUs the LLM has never seen, recent silicon revisions where the register layout changed, vendor-specific errata. In those cases RAG would have to ingest the documents anyway. Tool access handles them as they are, with no preprocessing pipeline.

What This Doesn’t Cover

Large-scale retrieval across many documents. Put 50 datasheets side by side and ask “which MCU has a CAN-FD peripheral with at least 64 message buffers,” and tool-based grep does get slow. A lightweight index helps there (it does not have to be RAG). But that is a product selection question, not a firmware development one. It is a different problem.

Diagrams and schematics. Tool-based PDF access does better here than I expected. Claude and Gemini process PDF pages as images, so the agent can read a specific page and interpret block diagrams, clock tree figures, and pin assignment tables by looking at them. The tool pattern stays the same: the agent greps the PDF for “Figure 23,” reads that page, and takes what it needs.

Timing diagrams are another matter. General-purpose VLMs still struggle to read multi-signal waveforms. GPT-4o misreads signals and misses protocol-specific transitions like AHB wait states. TD-Interpreter (July 2025) fine-tuned a vision-LLM only for timing diagrams, reached expert-level accuracy, and beat GPT-4o, but it needs custom training on synthesized timing diagram data. On the MMMU multimodal benchmark (30 subjects, engineering diagrams included), earlier-generation VLMs scored 60-70% while human experts scored ~89%. Newer models are closing the gap, but they still fall behind on timing diagrams. The harder MMMU-Pro variant pushes scores lower still. For setup/hold times and protocol timing, I still read the numbers off the datasheet myself.

Validation. Giving the LLM the right documentation does not guarantee the right code. Checking the output against the SVD database after generation, the way RespCode does, catches errors the LLM makes even with perfect context. That is a separate problem from retrieval.

Skip RAG. Use the Files.

RAG was built for natural language documents, where approximate retrieval is good enough. Hardware documentation is not that kind of document. Register addresses, bit fields, clock trees, and cross-peripheral dependencies need exact retrieval with the full context kept whole.

After all that, where I ended up was the simplest option: put the SVD and PDF files in the project, tell the LLM where they are, and let it grep. No embedding pipeline, no vector database, no chunk-size tuning. The files are the ground truth. The agent only has to read them.

ecro
Written by ecro

Created an LLM benchmark for firmware. EmbedEval →

Building a project-shaped agent harness for Claude Code, Cursor, and Codex. harness-maker →

Building a serial and SSH terminal where coding agents work on the board beside you. NeuroTerm →

Comments

Loading comments...