AI-Written Firmware: Where It Breaks

I caught an LLM-generated CAN logger quietly killing eMMC, and after that I started collecting these failures one by one. I ran 220+ test cases against Sonnet 4.6 and Haiku 4.5 on Zephyr 3.6 with arm-none-eabi-gcc 12.2. Some checks failed on every model I tried, and 6 of those 8 landed in just two buckets: error handling (missing cleanup when init fails) and concurrency (state shared between an ISR and a thread).

Measured pass rates for the 6 patterns covered in this post:

Pattern	Sonnet 4.6	Haiku 4.5	Basis
Error path cleanup	both failed	both failed	Same TC linux-driver-006 (n=1)
volatile + ISR	33%	22%	isr-concurrency category, 9 TCs
DMA cache coherency	44%	0%	dma category, 9 TCs, lowest overall
Counter overflow	~5%	~5%	Likely underrepresented in public repos
Float NaN bypass	~5%	~5%	Likely underrepresented in public repos
Strict aliasing	~10%	~10%	Wrong patterns dominate public repos

The numbers were less interesting than how the failures behaved. Some show up right in the code diff. If you know what to look for, review catches them. The rest are different. They pass review, pass tests, and only show their face weeks or months later: when a device stays powered on, or when you write against -O0 and ship on -O2.

The two kinds share one thing. Both are nonlocal. The bug doesn’t show up inside its own function. To see the error path cleanup you have to follow a resource graph across the init stages. DMA cache coherency only becomes visible once you know the hardware bypasses the CPU cache, a fact that lives in the datasheet and never in the code. Counter overflow needs you to picture 2^32 milliseconds of uptime in your head. Nothing here pokes out within the scope of one function.

LLMs are good at code that reads well locally. Embedded bugs, though, are mostly nonlocal, and that mismatch is what opens the 35%p gap between explicit prompts (the answer sits in the instruction) and implicit ones (you have to infer the domain knowledge). The model isn’t failing to write correct code. The correct code just depends on context that lives outside the function.

Patterns Visible in Diff

1. Error Path Cleanup

The most common failure across every model I tested. In a multi-step init, if step N fails, you have to unwind the resources from steps 1 through N-1 in reverse order. The model skips that.

/* Sonnet 4.6 output, prompt: "write sensor init for I2C + GPIO + DMA" */
int sensor_init(void) {
    i2c_init(&config);
    gpio_configure(SENSOR_PIN, GPIO_INPUT);
    dma_setup(&rx_channel);
    return 0;  /* what if gpio_configure() failed? i2c stays acquired */
}

/* with goto cleanup */
int sensor_init(void) {
    int ret;

    ret = i2c_init(&config);
    if (ret < 0)
        return ret;

    ret = gpio_configure(SENSOR_PIN, GPIO_INPUT);
    if (ret < 0)
        goto err_gpio;

    ret = dma_setup(&rx_channel);
    if (ret < 0)
        goto err_dma;

    return 0;

err_dma:
    gpio_release(SENSOR_PIN);
err_gpio:
    i2c_deinit(&config);
    return ret;
}

On pure readability, the first version wins easily. The catch is that every error path leaks the I2C bus, and once failures pile up the bus is exhausted outright.

Sonnet and Haiku tripped over the same test case, side by side: a Zephyr driver init sequence, internal tag linux-driver-006. Ask for goto cleanup explicitly and compliance climbs to nearly 100%. So the model knows how to write it. It just won’t unless you ask.

Coverity’s resource-leak checker flags this. Without it, one review rule covers you: be suspicious of any init function that has no cleanup label. cppcheck --enable=warning catches some of them too, though it’s nowhere near as thorough as path-sensitive analysis.

2. volatile and ISR

An ISR sets a flag and the main loop reads it. Drop volatile and the compiler caches that read in a register at -O2 and stops looking at memory again.

int flag = 0;                /* Sonnet 4.6, prompt: "ISR sets flag, main reads" -- no volatile */
volatile int flag = 0;       /* one keyword is all it takes */

Runs fine in a debug build. Infinite loop in release. One keyword’s worth of difference.

LLMs will also drop k_mutex_lock() or k_sleep() straight into an ISR without blinking. On Zephyr, both of those are either deadlock or crash. The compiler stays silent about it. You find out when the board freezes.

The first time I hit this I was sure it was a mutex deadlock and dug through lock ordering for a while. It never crossed my mind that the ISR itself was holding the mutex. After that I dropped a k_is_in_isr() assertion into every locking wrapper, and now it trips on the spot. Plain C gives you no such runtime check, so in the end you fall back on code review.

3. DMA Cache Coherency

The DMA engine can’t read the data the CPU just wrote to a buffer. The CPU writes into cache and the DMA engine reads from main memory. No error, no warning. It just puts the wrong bytes on the wire.

/* no cache flush -- DMA reads stale data from main memory */
uint8_t tx_buf[256];
fill_buffer(tx_buf);
HAL_SPI_Transmit_DMA(&hspi1, tx_buf, 256);

/* STM32H7 (Cortex-M7, D-cache on): flush before DMA */
uint8_t __aligned(32) tx_buf[256];
fill_buffer(tx_buf);
SCB_CleanDCache_by_Addr((uint32_t *)tx_buf, 256);
HAL_SPI_Transmit_DMA(&hspi1, tx_buf, 256);

I was pulling sensor values over SPI on an STM32H7 board and the data kept coming back corrupted, intermittently. My first suspect was SPI clock timing. I put a scope on it and the waveform was clean. Half a day of thrashing later, the culprit turned out to be a missing cache invalidate on the Rx buffer.

Any MCU with a data cache is exposed: Cortex-M7 class, STM32F7/H7. The knowledge you need is written down only in the datasheet and silicon errata. It was the lowest category in the whole benchmark, and one model scored zero on the cache coherency checks.

There’s basically no useful tooling to automate this away. Your options are to mark the DMA buffer region as non-cacheable through the MPU, or to go check for yourself whether your HAL’s DMA wrapper handles cache management internally.

Patterns Exposed by Time or Environment

The next three you can find with grep. In code review, though, they read as “code that works.” They only blow up once uptime accumulates, an edge-case input arrives, or the toolchain changes underneath you.

4. The 49.7-Day Counter Overflow

A 32-bit millisecond counter wraps at 2^32 ms. In days, that’s 49.7.

if (millis() > next_event) { /* handle event */ }

On day 49.7, millis() rolls back to 0. next_event is still a large value. The condition is false forever after. The timer never fires again.

/* Wrap-safe: unsigned subtraction handles overflow */
if ((uint32_t)(millis() - last_event) >= interval) { ... }

This is a shipped-product story. macOS shipped a 32-bit counter and networking died on machines that passed 50 days of uptime. Zephyr’s OpenThread stack threw the same class of bug on the nRF52840. Every device hung at exactly day 49, and the issue got tagged “showstopper for commercial products.”

Catching it is almost anticlimactically simple. One line of rg 'millis\(\)\s*>' src/:

$ rg 'millis\(\)\s*[><=]' --glob '*.c' | head -5
drivers/heartbeat.c:47:    if (millis() > next_beat) {
lib/ota/timeout.c:112:    while (millis() < deadline) {
app/sensor_poll.c:83:    if (millis() > sample_time) {

The output is illustrative, but the pattern looks the same in a real project. Rewrite every > comparison as a subtraction. < deadline is the same trap.

5. Float NaN Bypasses Safety Checks

Divide by zero and the safety comparison downstream is disabled wholesale.

float distance, elapsed_time;
/* ... */
float speed = distance / elapsed_time;
if (speed > MAX_SPEED) emergency_stop();

When distance and elapsed_time are both 0, 0.0f / 0.0f is NaN. And NaN behaves nastily. Every comparison it touches returns false. NaN > MAX_SPEED? False. NaN < 0? False. NaN == NaN? Even that is false. So emergency_stop() never gets called, not once.

/* NaN/Inf guard (requires <math.h>) */
float speed = distance / elapsed_time;
if (isnan(speed) || isinf(speed)) {
    emergency_stop();
    return;
}
if (speed > MAX_SPEED) emergency_stop();

Wire up isinf() alongside it. If only distance is nonzero the result comes out as +-Inf, and +Inf > MAX_SPEED is true, so you do get an emergency stop (a spurious one, but a stop). The problem is -Inf > MAX_SPEED is false, which recreates the exact same bypass as NaN.

Push zeros and negatives into your tests on purpose (fault injection) and it surfaces right there. On a safety-critical path, avoiding float altogether and going fixed-point is the more fundamental fix.

6. Strict Aliasing

Read memory through a pointer of a different type and the compiler is free to delete the read itself.

struct sensor_reading *r = (struct sensor_reading *)buf;  /* UB -- ubiquitous in training data */

struct sensor_reading r;
memcpy(&r, buf, sizeof(r));                               /* defined behavior */

Runs fine at -O0. At -O2 the compiler assumes “these two pointers can’t possibly point at the same memory” (the strict aliasing rule) and skips the read through buf. The struct is left holding garbage. And if buf also fails the struct’s alignment requirement, on a Cortex-M0 that path leads to a HardFault.

On an STM32F4 project I bumped GCC from 10 to 12 and the sensor driver broke. Code that had run cleanly at -O2 for three years. TBAA optimization got more aggressive, and the read through the pointer cast vanished completely. Switching to memcpy cleared it, and if you crack it open with arm-none-eabi-objdump -d it compiles down to the same register move anyway.

You defend against this in CI. Turn on -fstrict-aliasing -Wstrict-aliasing=2, then verify at runtime with UBSan. On a small MCU it’s more realistic to run UBSan in host unit tests rather than on the full firmware image. Passing both the -O0 and the -O2 build, at minimum, is the line you don’t cross below.

The “Clean Code” Trap That Training Rewards

There are more than these six. eMMC wear (part 1), sensor plausibility, radio stack corruption, the list keeps going. The flip side is that the models aren’t bad at everything. Boilerplate, driver stubs, state machine scaffolding, single-function logic: they handle all of that well. The failures only surface when the domain knowledge is implicit.

Why implicit things specifically? An LLM’s training data is mostly open-source off GitHub. The code that racks up stars there is clean and terse. Add a volatile and it looks like a redundant qualifier. Reach for goto and it gets marked a bad habit. Put cleanup on every error path and it reads as noise. RLHF pushes the same way, because human raters hand higher scores to short, readable answers. That’s how the model gets trained toward “clean code.” But safe embedded code isn’t clean. The training incentive and the domain requirement point in opposite directions.

Catching Them With a Reviewer Agent

So I keep a reviewer agent that sweeps for these patterns the moment code comes out. Wire firmware-safety-reviewer in as a hook or custom command in Claude Code (or Cursor) and it fires the same questions automatically every time the LLM emits code. The six patterns from this post, verbatim: is there reverse cleanup on the error paths, does the ISR-shared variable carry volatile, is there a cache op on the DMA buffer, is the 32-bit counter wrap-safe, is there a NaN guard after the float division, did it use memcpy instead of a pointer cast.

The knowledge is already inside the LLM. Ask explicitly and it hits ~95%. What’s missing is the application step. The reviewer drags that application out of the model on every generation.

When you review AI-written code, the first question isn’t “does it compile?” It’s “what does this code miss because it can’t see past its own function?”

To prevent these patterns outright with context engineering, see Context Engineering.

---

Next post: the benchmark that measures these patterns systematically. 233 test cases, a 5-layer evaluation pipeline, and a per-model scorecard.