AI-Written Firmware: What Breaks Outside the Function
AI-written firmware breaks where a single function can't show it. I found 3 patterns visible in the diff and 3 that only surface after 49 days.
On this page
After I caught an LLM-generated CAN logger quietly killing eMMC, I started collecting failures like it one by one. I ran 220+ test cases against Sonnet 4.6 and Haiku 4.5 on Zephyr 3.6 with arm-none-eabi-gcc 12.2. Some checks failed on every model I tried, and 6 of those 8 landed in just two buckets. One was error handling (missing cleanup when init fails). The other was concurrency (state shared between an ISR and a thread).
The table below holds the measured pass rates for the 6 patterns covered in this post.
| Pattern | Sonnet 4.6 | Haiku 4.5 | Basis |
|---|---|---|---|
| Error path cleanup | both failed | both failed | Same TC linux-driver-006 (n=1) |
| volatile + ISR | 33% | 22% | isr-concurrency category, 9 TCs |
| DMA cache coherency | 44% | 0% | dma category, 9 TCs, lowest overall |
| Counter overflow | ~5% | ~5% | Likely underrepresented in public repos |
| Float NaN bypass | ~5% | ~5% | Likely underrepresented in public repos |
| Strict aliasing | ~10% | ~10% | Wrong patterns dominate public repos |
What held my attention longer than the numbers was how the failures behaved. Some show up right in the code diff. If you know what to look for, review catches them. The rest don’t. They pass review, pass tests, and only appear weeks or months later: when a device stays powered on, or when you write against -O0 and ship on -O2.
The two kinds have one thing in common. Both are nonlocal, which means the bug doesn’t show up inside its own function. To see the error path cleanup you have to follow a resource graph across the init stages. DMA cache coherency only becomes visible once you know the hardware bypasses the CPU cache, a fact that lives in the datasheet and never in the code. Counter overflow asks you to picture 2^32 milliseconds of uptime. None of these stand out within the scope of one function.
LLMs are good at code that reads well where it sits. Embedded bugs, though, are mostly nonlocal, and that mismatch is what opens the 35%p gap between explicit prompts (the answer sits in the instruction) and implicit ones (you have to infer the domain knowledge). The model is not failing to write correct code. The correct code depends on context that lives outside the function.
Patterns Visible in Diff
1. Error Path Cleanup
This was the most common failure across every model I tested. In a multi-step init, if step N fails, you have to unwind the resources from steps 1 through N-1 in reverse order, and the model skips that.
/* Sonnet 4.6 output, prompt: "write sensor init for I2C + GPIO + DMA" */
int sensor_init(void) {
i2c_init(&config);
gpio_configure(SENSOR_PIN, GPIO_INPUT);
dma_setup(&rx_channel);
return 0; /* what if gpio_configure() failed? i2c stays acquired */
}
/* with goto cleanup */
int sensor_init(void) {
int ret;
ret = i2c_init(&config);
if (ret < 0)
return ret;
ret = gpio_configure(SENSOR_PIN, GPIO_INPUT);
if (ret < 0)
goto err_gpio;
ret = dma_setup(&rx_channel);
if (ret < 0)
goto err_dma;
return 0;
err_dma:
gpio_release(SENSOR_PIN);
err_gpio:
i2c_deinit(&config);
return ret;
}
On readability alone, the first version is much cleaner. But it leaks the I2C bus on every error path, and once failures pile up the bus runs out entirely.
Sonnet and Haiku tripped over the same test case, side by side. It was a Zephyr driver init sequence, internal tag linux-driver-006. Ask for goto cleanup explicitly and compliance climbs to nearly 100%. So the model knows how to write it. It just doesn’t unless you ask.
Coverity’s resource-leak checker flags this. Without it, one review rule is enough: be suspicious of any init function that has no cleanup label. cppcheck --enable=warning catches some of them too, though it is nowhere near as thorough as path-sensitive analysis.
2. volatile and ISR
An ISR sets a flag and the main loop reads it. Without volatile, the compiler caches that read in a register at -O2 and never looks at memory again.
int flag = 0; /* Sonnet 4.6, prompt: "ISR sets flag, main reads" -- no volatile */
volatile int flag = 0; /* one keyword is all it takes */
It runs fine in a debug build. In release it hangs in an infinite loop. One keyword separates the two.
LLMs will also put k_mutex_lock() or k_sleep() into an ISR as if it were nothing. On Zephyr, both of those end in deadlock or crash. The compiler doesn’t warn about it. You find out when the board freezes.
The first time I hit this I was sure it was a mutex deadlock, and I dug through lock ordering for a while. It never occurred to me that the ISR itself was holding the mutex. After that I added a k_is_in_isr() assertion to every locking wrapper, and now it trips on the spot. Plain C gives you no such runtime check, so in the end you fall back on code review.
3. DMA Cache Coherency
The DMA engine can’t read the data the CPU just wrote to a buffer. The CPU writes into cache, and the DMA engine reads from main memory. There is no error and no warning. The wrong bytes just go out on the wire.
/* no cache flush -- DMA reads stale data from main memory */
uint8_t tx_buf[256];
fill_buffer(tx_buf);
HAL_SPI_Transmit_DMA(&hspi1, tx_buf, 256);
/* STM32H7 (Cortex-M7, D-cache on): flush before DMA */
uint8_t __aligned(32) tx_buf[256];
fill_buffer(tx_buf);
SCB_CleanDCache_by_Addr((uint32_t *)tx_buf, 256);
HAL_SPI_Transmit_DMA(&hspi1, tx_buf, 256);
I was reading sensor values over SPI on an STM32H7 board, and the data came back corrupted now and then. My first suspect was SPI clock timing. I put a scope on it and the waveform was clean. After half a day of searching, the cause turned out to be a missing cache invalidate on the Rx buffer.
Any MCU with a data cache is exposed: Cortex-M7 class, STM32F7/H7. The knowledge you need is written down only in the datasheet and silicon errata. It was the lowest category in the whole benchmark, and one model scored zero on the cache coherency checks.
There is almost no useful tooling to automate this. Your options are to mark the DMA buffer region as non-cacheable through the MPU, or to check for yourself whether your HAL’s DMA wrapper handles cache management internally.
Patterns Exposed by Time or Environment
The next three you can find with grep. In code review, though, they read as “code that works.” They only fail once uptime accumulates, an edge-case input arrives, or the toolchain changes underneath you.
4. The 49.7-Day Counter Overflow
A 32-bit millisecond counter wraps at 2^32 ms. In days, that’s 49.7.
if (millis() > next_event) { /* handle event */ }
On day 49.7, millis() rolls back to 0. next_event is still a large value, so the condition stays false from then on. The timer never fires again.
/* Wrap-safe: unsigned subtraction handles overflow */
if ((uint32_t)(millis() - last_event) >= interval) { ... }
This happened in shipped products. macOS shipped a 32-bit counter and networking died on machines that passed 50 days of uptime. Zephyr’s OpenThread stack produced the same class of bug on the nRF52840. Every device hung at exactly day 49, and the issue got tagged “showstopper for commercial products.”
Catching it is almost disappointingly simple. One line of rg 'millis\(\)\s*>' src/ does it:
$ rg 'millis\(\)\s*[><=]' --glob '*.c' | head -5
drivers/heartbeat.c:47: if (millis() > next_beat) {
lib/ota/timeout.c:112: while (millis() < deadline) {
app/sensor_poll.c:83: if (millis() > sample_time) {
The output is illustrative, but the pattern looks the same in a real project. Rewrite every > comparison as a subtraction. < deadline is the same trap.
5. Float NaN Bypasses Safety Checks
Divide by zero and the safety comparison after it stops working entirely.
float distance, elapsed_time;
/* ... */
float speed = distance / elapsed_time;
if (speed > MAX_SPEED) emergency_stop();
When distance and elapsed_time are both 0, 0.0f / 0.0f is NaN. NaN has an awkward property: every comparison it takes part in returns false. NaN > MAX_SPEED? False. NaN < 0? False. NaN == NaN? Even that is false. So emergency_stop() never gets called, not once.
/* NaN/Inf guard (requires <math.h>) */
float speed = distance / elapsed_time;
if (isnan(speed) || isinf(speed)) {
emergency_stop();
return;
}
if (speed > MAX_SPEED) emergency_stop();
I add isinf() alongside it. If only distance is nonzero the result comes out as +-Inf, and +Inf > MAX_SPEED is true, so you do get an emergency stop (a spurious one, but a stop). The trouble is that -Inf > MAX_SPEED is false, which brings back the exact same bypass as NaN.
Feed zeros and negatives into your tests on purpose (fault injection) and it shows up right there. On a safety-critical path, avoiding float altogether and going fixed-point is the more fundamental fix.
6. Strict Aliasing
If you read memory through a pointer of a different type, the compiler is free to delete the read itself.
struct sensor_reading *r = (struct sensor_reading *)buf; /* UB -- ubiquitous in training data */
struct sensor_reading r;
memcpy(&r, buf, sizeof(r)); /* defined behavior */
It runs fine at -O0. At -O2 the compiler assumes “these two pointers can’t possibly point at the same memory” (the strict aliasing rule) and skips the read through buf. The struct is left holding garbage. And if buf also fails the struct’s alignment requirement, on a Cortex-M0 that path leads to a HardFault.
On an STM32F4 project I moved GCC from 10 to 12 and the sensor driver broke. The code had run cleanly at -O2 for three years. TBAA optimization had become more aggressive, and the read through the pointer cast was gone. Switching to memcpy fixed it, and when you open it with arm-none-eabi-objdump -d it compiles down to the same register move anyway.
The defense belongs in CI. Turn on -fstrict-aliasing -Wstrict-aliasing=2, then verify at runtime with UBSan. On a small MCU it is more realistic to run UBSan in host unit tests than on the full firmware image. At minimum, passing both the -O0 and the -O2 build is the line I won’t go below.
The “Clean Code” Trap That Training Rewards
There are more than these six. eMMC wear (part 1), sensor plausibility and radio stack corruption belong on the list too. That doesn’t mean the models are bad at everything. Boilerplate, driver stubs, state machine scaffolding and single-function logic they handle well. The failures only appear when the domain knowledge is implicit.
When I ask why it is the implicit things in particular, I keep coming back to the training data. An LLM’s training data is mostly open-source code from GitHub. The code that collects stars there is clean and terse. Add a volatile and it looks like a redundant qualifier. Reach for goto and it gets marked a bad habit. Put cleanup on every error path and it reads as noise. RLHF pushes the same way, because human raters give higher scores to short, readable answers. That is how the model gets trained toward “clean code.” But safe embedded code isn’t clean. What the training rewards and what the domain requires point in opposite directions.
Catching Them With a Reviewer Agent
So I keep a reviewer agent that sweeps for these patterns as soon as code comes out. Wire firmware-safety-reviewer in as a hook or custom command in Claude Code (or Cursor) and it asks the same questions every time the LLM emits code. They are the six patterns from this post, word for word: is there reverse cleanup on the error paths, does the ISR-shared variable carry volatile, is there a cache op on the DMA buffer, is the 32-bit counter wrap-safe, is there a NaN guard after the float division, did it use memcpy instead of a pointer cast.
The knowledge is already inside the LLM. Ask explicitly and it hits ~95%. What’s missing is the step where it applies that knowledge. The reviewer pulls that step out of the model on every generation.
When I review AI-written code, the first question I ask isn’t “does it compile?” It’s “what does this code miss because it can’t see past its own function?”
To prevent these patterns outright with context engineering, see Context Engineering.
The next post covers the benchmark that measures these patterns systematically: 233 test cases, a 5-layer evaluation pipeline, and a per-model scorecard.
Comments
Loading comments...