← Essays /Post · 28 of 28 · Making

Why I Started EdgeLog

I wanted to know what happens when LLMs, zero-trust security, and modern frameworks meet real embedded hardware. A research log from the edge.

·4 min read · · · #edge-computing#edge-ai#embedded-systems
Why I Started EdgeLog
On this page

Every hot AI technology is born in the cloud. LLMs, RAG pipelines, zero-trust security, real-time ML inference: they all debut on servers with plenty of RAM and a network that stays up. Then someone asks, “can this run on the device?”

Most documentation stops right there.

The Question Nobody’s Answering

What happens when you drop a quantized LLM onto an edge device with 512MB of RAM? How do you fit zero-trust security onto a Linux SBC that has to boot in under two seconds? Can Zephyr’s Kenning runtime really replace a bare-metal inference pipeline? Does RAG make any sense when the whole knowledge base sits on a 4GB eMMC? Can a WASM runtime do anything useful on a Cortex-A53 with a 2W power budget?

These are problems from real projects, not thought experiments. Anyone who pushes new tech onto constrained hardware runs into them. What differs is that nobody writes down what they found.

Existing embedded blogs cover the fundamentals well: Interrupt for firmware engineering, Shawn Hymel for MCU-class edge AI. But the frontier moves fast, and the distance between “this technology exists” and “here’s what happened when I put it on real hardware” grows every month.

What EdgeLog Is

EdgeLog is a research log. I put emerging tech onto embedded and edge devices, see what actually runs, and then write down what happened.

Every post tries to answer one question: what happens when this meets real hardware constraints?

The topics follow the frontier wherever it goes. These are the ones I am working through right now.

On-Device Intelligence. I put a Q4_K_M-quantized Qwen2.5-1.5B on a Raspberry Pi 5 (8GB) and ran llama.cpp. It does about 6 tokens/sec. That is slow. But it is enough for local command parsing and log summarization without calling a cloud API. The questions I care about start after “it runs”: memory bandwidth bottlenecks (the Pi’s 50 GB/s LPDDR4X vs. a datacenter GPU’s 2 TB/s HBM), swap pressure under concurrent inference, RAG with SQLite + vector extensions on eMMC, on-device fine-tuning with LoRA adapters small enough to fit in flash, and hybrid setups where the edge device decides what is worth sending to the cloud.

Next-Gen Frameworks & Runtimes. Zephyr 4.x supports 900+ boards, with Kenning runtime, LiteRT, and IREE handling ML inference. As of December 2025, Rust has shed the experimental label in the Linux kernel, and the DRM subsystem plans to require it for new drivers. WASM runtimes like WasmEdge and Ocre (Linux Foundation, 128KB footprint on Zephyr) are moving past experiments into production. Hybrid OS setups (Zephyr for real-time control, Linux for networking and UI) are quietly becoming the default on complex edge devices. What I want to see is which of these survive real deadlines and real power budgets.

AI-Powered Embedded Development. I use Claude every day for Yocto recipe debugging and device tree review. It is good at catching missing compatible strings and pointing out where RDEPENDS needs a fix. It is bad at register-level driver code. It will hand me plausible C that compiles but writes to the wrong offset, because it hallucinated the register map. I want to record where that line falls for different embedded tasks, with concrete pass/fail examples. (I started with spec-driven TDD and context engineering for sensor drivers.)

Edge Security, Privacy & CRA Compliance. The EU Cyber Resilience Act reporting obligations begin on September 11, 2026. If you sell embedded Linux products in the EU, you have to report actively exploited vulnerabilities to ENISA within 24 hours. Full compliance is due by December 2027. Most of the Yocto-based products I have seen have no automated CVE scanning in their build pipeline. An SBOM that ENISA would accept is further away still. Zephyr already generates Build SBOMs automatically, and Yocto is still catching up. I am working out what a realistic compliance setup looks like for meta-security-based builds, and what secure boot and anti-rollback OTA actually demand on custom hardware.

This list will keep changing. I meant it to.

How I’ll Write

Experiment first, write second. Every post starts from real hardware, real code, real measurements. If I have not built it, flashed it, and watched it run (or crash), I say so. This post is the only one on the blog without a terminal screenshot. Every post after it comes with evidence.

Honest about what doesn’t work. I think half the value lives in the dead ends. A technology that looks great in the README but falls apart on a memory-tight device is exactly what is worth writing down. I once spent two weeks trying to get WASM-based ML inference running on a Cortex-A53 board, then gave up and went back to native C. That is a post too.

Respect your time. No filler, no hype, no buzzword padding. Code you can run, numbers you can compare, trade-offs you can weigh against your own project. That is all I mean to offer.

What Comes Next

The first experiments are already running.

Some engineers light up when someone asks “can we run this on the device?” This blog is written for them. Subscribe via RSS.

The cloud had its decade. Now try fitting it into 256KB.

ecro
Written by ecro

Created an LLM benchmark for firmware. EmbedEval →

Building a project-shaped agent harness for Claude Code, Cursor, and Codex. harness-maker →

Building a serial and SSH terminal where coding agents work on the board beside you. NeuroTerm →

Comments

Loading comments...