← All posts
hardware

Altar: a small screen that asks before it lets anything happen

· 6 min read · by aft3r_d0t

Duotone ink illustration of a calm, heavy-lidded face dissolving into dots on one side, with pillar candles standing below it.

On my desk there’s a small square screen with a face on it. The face changes depending on who’s in the house working and what they’re doing. We call it Altar. On paper it’s a desk companion; in practice it’s where the house asks me for permission.

What’s inside

The hardware is an off-the-shelf ESP32-S3-BOX-3: a small touchscreen box with 16 MB of flash and 16 MB of PSRAM, a microphone and a speaker. On the Mac there’s a hub that talks to it over a USB serial cable and decides what the screen shows: faces, widgets, a menu, a ticker, a calendar chip, a “while you were away” card.

The faces are short video clips, one set per agent, cut into JPEG frames and packed into one file. The target is 12 frames per second. That’s 83.3 ms per frame, and the board decodes a frame in 18–21 ms, so decoding was never the problem. Measured frame time on the device: 83.0 ms against the 83.3 ms target, zero errors. The device was already fast enough to do the work. It just wasn’t being asked to do it.

The 1 fps mystery

Played from internal flash the faces were smooth. Played from the SD card, where there’s room for all of them, they crawled at about 1 frame per second.

The plan listed half a dozen fixes, from a faster card clock to prefetching frames into PSRAM to skipping frames when late. We’d already ruled out the card’s cluster size. And the fix that actually worked was the least glamorous one: the code read the file through a C stdio buffer that refilled one byte at a time. Replace that with a plain read() — one seek and one read per frame — and the card plays at 12 fps. The new path landed within 0.4 ms of the best one we could measure, so nothing had to be reversed.

Most of embedded performance work is finding the one line that’s doing something silly a million times.

The numbers say how silly. Through the stdio path, reading one frame took 603,265 microseconds and cost 52.3 disk reads per frame — the file was refilled in single bytes, so a 26 KB frame turned into roughly 25,800 calls at about 23 microseconds each. The plain read() path took 2,396 microseconds; the FATFS paths 2,343 and 2,001. Raw sector reads ran at 13,973 KB/s, which says the card itself was never slow. We were asking it for one byte at a time, then complaining that it was slow.

Two lessons came out of that, both about how we’d been working rather than about the card. The first measurement was taken against the wrong library: my first search for the stdio implementation picked the picolibc archive, while the map file from the actual build named the toolchain’s flash libc. Same symptom, different cause; the honest answer after re-checking was “the mechanism was right, the library was wrong”. The second lesson was about ordering. Two of the proposed fixes — prefetching and frame-skipping — were written into the spec before anyone had a single number for what a read cost. When the number arrived, both were withdrawn, and the rule became: instrument, read the number, then specify the fix.

The other direction is slower and nobody has fixed it: loading a full set of faces and voice lines onto that card over the same serial cable took 1,627 seconds for 478 MB — about 27 minutes, 0.29 MB/s. It happens rarely, and for now it’s fine. But the upload isn’t atomic. An earlier attempt died at 20 % when the cable was pulled, and the retry started from zero rather than from where it stopped — resume didn’t survive the board losing power. The old pack stayed playable throughout, so a failed upload costs an evening, not the faces.

Approve / Deny

Here’s the part I like most. When an agent wants to run a command that needs a human, the request can show up on Altar as a card with two buttons. Tap Approve and the waiting agent gets its answer in about 2 ms; Tap Deny and it gets “denied on the device” in about 4 ms. If the hub times out, the request simply stays waiting — no default, no silent yes.

We tested it live: Deny on the screen, and the command was blocked and the folder stayed put. Approve, and it ran. Approve from chat instead, and the card on the desk closed by itself.

The funny bit: for days after it was wired up, the screen showed zero real approval requests. The agents run in a “smart” mode that auto-approves commands it judges safe, so only the risky or escalated ones ever reach a human. The button works. It’s just rarely needed, which is exactly what you want from an emergency brake. Later the whole thing grew a switch of its own, so approvals can be forced on or off for everyone; that switch came with ten deliberately planted faults, and the test suite killed all ten.

It talks back now

There’s a push-to-talk disc at the bottom of the screen. Hold it, say something in Russian, and the hub transcribes it, passes it to an agent and reads the reply aloud. Measured on this Mac: a cold transcription takes about 1.39 seconds, a warm one 0.489; the reply is spoken in roughly 0.74; the first chunk of audio lands about two seconds after you let go of the button, and the whole reply leg takes around a second. There’s a quiet mode too — when a reply shouldn’t be spoken aloud, it arrives as a banner on the screen instead of audio, with no sound at all. The reply isn’t taken on trust either: what comes back is compared byte for byte with an independent render of the same text, because a truncated reply would still match its own checksum. Only then does the round trip count as working.

We discussed muting voice lines when nobody’s at the desk. I said no: it’s harmless, and if I’m home but across the room, at least I’ll hear it.

What broke along the way

A device that sits on a desk and talks to the house collects its own pile of failures, and none were theoretical.

The first boot after one of the flashes panicked once with a stack overflow in the main task — the microphone had been opened there, and the task didn’t have room for it. The second boot was clean, and after that the boot timings became a thing we read like a heartbeat: splash at about 300 ms, first face under three seconds, fade a few hundred ms later.

Then the USB link. Over one morning with the room empty, the board reconnected eleven times, and every reconnection replayed an old crash dump on hello. No resets, no data loss we could see. The suspects are a cable or a hub, or macOS power management while the Mac idles — and since none was proven, none got fixed. It’s the kind of fault that waits a week for being ignored before it gets an afternoon.

It’s called Altar because that’s what it looks like at night: a small glowing shrine where the house comes to ask. And the whole point is the boring rule underneath — a request that needs a human is never answered by the thing that made the request.