AI-Written Firmware: How Far Can You Trust It?

The problem with LLM-generated embedded code isn’t that it’s wrong. It’s that it goes wrong quietly. Tell it exactly what to do and it usually delivers. But the moment implicit domain knowledge is needed, the pass rate falls off a cliff. Existing coding benchmarks don’t catch this gap. A logging daemon I flagged in code review is what taught me.

Code That Silently Kills Your eMMC

I was reviewing a junior engineer’s code on an i.MX8M Mini BSP project. A CAN logging daemon. You could tell at a glance it was AI-generated. Over 200 lines, with signal handling, daemonization, PID file management, nothing missing, all production-grade structure. No junior writes at that level of completeness. Here’s the core loop.

/* AI-generated CAN logging daemon (core loop) */
#define DEFAULT_LOGFILE  "/var/log/can.log"
#define LOG_FLUSH_INTERVAL 10  /* seconds */

FILE *fp = fopen(logfile, "a");   /* logfile = "/var/log/can.log" */
alarm(LOG_FLUSH_INTERVAL);        /* SIGALRM every 10s to flush */

while (running) {
    struct can_frame frame;
    ssize_t nbytes = read(sock, &frame, sizeof(frame));
    if (nbytes < 0) {
        if (errno == EINTR) {
            if (flush_flag) { fflush(fp); flush_flag = 0; alarm(LOG_FLUSH_INTERVAL); }
            continue;
        }
        break;
    }
    log_frame(fp, iface, &frame, &ts);

    if ((frame_count + err_count) % 1000 == 0)
        fflush(fp);  /* flush every 1000 frames */
}

It’s well built. Instead of hammering fflush() on every frame, it buffers on a 10-second interval plus every 1000 frames. Error handling is there, graceful shutdown via signals works. I nearly hit approve and moved on.

Then my hand stopped. /var/log/can.log. That’s sitting on eMMC.

On our system, CAN messages arrive every 100ms. With 1000-frame buffering, that’s a flush roughly every 100 seconds. Add the 10-second alarm and you’re looking at thousands of eMMC writes a day. Appending to one file keeps hammering the same block group even with wear leveling in place. The P/E cycles on MLC/TLC eMMC grind down bit by bit, and on a bad day the eMMC flips to read-only a few months later, or takes the boot partition down with it.

The fix is simple. Write to tmpfs instead of /var/log/, and only flush down to eMMC on a schedule.

/* Fix: tmpfs buffer + periodic eMMC flush */
#define DEFAULT_LOGFILE  "/tmp/can_buffer.log"  /* tmpfs (RAM) */
#define EMMC_FLUSH_INTERVAL 3600                /* save to eMMC once per hour */

Thousands of daily writes drop to 24. Wire up logrotate and old files get cleaned too. If I’d missed this in review, we’d have found out months after deployment, from devices dying in the field.

The file I/O itself was flawless. Signal handling, buffering, error handling, all production quality. The trouble is that nothing in the code reflects the fact that it runs on eMMC. On a server this code is perfect. On embedded it kills the hardware. Nobody told the AI tool to “think about eMMC wear,” so of course it didn’t.

The bigger problem is the junior didn’t know this either. The AI wrote it cleanly, so it must be right. Without a senior to catch it in review, this ships to the field as-is.

35 Percentage Points: The Gap Benchmarks Hide

This didn’t stop at one logging daemon. I got curious whether the pattern repeated, so I built about 200 embedded test cases across 20 categories. Then I ran each one in two versions.

• Explicit: “Declare a volatile int flag and set it in the ISR” — most pass
• Implicit: “Communicate between the ISR and main thread via a flag” (no mention of volatile) — pass rate drops noticeably

In my tests the gap came out around 35 percentage points. Granted, verification was regex-based heuristics rather than a real runtime run, and the test cases lean heavily toward Zephyr. The exact number isn’t the point. The pattern is. The gap between explicit and implicit prompts stays wide and consistent.

Those tests later grew into EmbedEval, a 233-case benchmark with a real compile and runtime layer. Sonnet 4.6 lands at 68.0% there, Haiku 4.5 at 56.9%, and the implicit-knowledge categories are still where both of them fall apart.

The existing coding benchmarks (HumanEval, SWE-bench) are all explicit style. “Implement this function,” spelled out precisely. But embedded’s real difficulty lives in what you don’t write in the prompt: volatile, eMMC wear, cleanup ordering, timing margins. Almost nothing measures that.

When you see “the LLM scores 90%,” you have to ask: is that 90% with the answer already sitting in the prompt, or 90% where it has to infer the domain knowledge on its own?

The Same Gap Shows Up in Papers

Other people’s benchmarks show the same thing. EmbedAgent (2025, arXiv preprint) reports LLM pass@1 at 73.8% on MicroPython (Raspberry Pi Pico), dropping to 29.4% on ESP-IDF. MicroPython APIs like pin.value() are blatantly explicit. ESP-IDF is a different story: esp_err_t checks, the component system, menuconfig conventions, a pile of implicit rules. Same implicit knowledge problem.

IoT-SkillsBench (2026) goes a step further. Injecting LLM-generated “helper context” into the prompt actually made Zephyr performance worse. The LLM just reinforced its own wrong assumptions. Context written by a human expert, on the other hand, pushed the score up to 39/42. Dump in any old context and it turns toxic. Only context pulled from real failure experience actually helps.

What I Changed

After that review I changed exactly one thing. I put a silent failure checklist in the project config file (CLAUDE.md).

## Silent Failure Checklist
- eMMC writes -- high-frequency writes must use tmpfs buffering. No direct fflush
- Error paths -- LLM almost never does reverse-order cleanup on init failure
- IRQ context -- LLM inserts blocking calls (mutex, sleep) in interrupt handlers
- Device Tree -- pinctrl, clock required properties get omitted
- Timing margins -- LLM sets timeout = period with zero margin

A new line gets tacked on every time I catch one in review. Three months with this checklist, dozens of generated modules later, and the same failure types haven’t come back. Five lines in a file beat dumping the whole datasheet into RAG. The LLM can’t fish “watch the eMMC write endurance” out of a 512-page reference manual. But it reads a one-line checklist entry just fine.

Is it a complete fix? No. I can only write down the traps I know about. The ones I don’t know are still hiding inside that gap. Still, as the traps pile up, CLAUDE.md grows into “implicit knowledge made explicit.” The more experience the engineer has, the thicker the file gets, and the wider the LLM’s safe zone becomes.

When you look at LLM-written embedded code, the first question isn’t “does it compile?” The first question is “what did I not tell it?” If you can’t answer that for a given chunk of code, you’re not ready to hand it to an LLM yet.

For a concrete way to solve this with context engineering, see Context Engineering.