ESP32-P4 https://git.0x4200.cafe/esp32p4.git/atom?h=main 2026-09-16T14:28:40Z 2026-09-16T14:28:40Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T16:28:33Z urn:sha1:b42ecaed412be2e30b9e780eb7c9e46e1535f26f 2026-08-26T15:00:20Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T15:00:20Z urn:sha1:85cf1a7dd912db4a5fc3ac2cb9d8550050ffbdb0 The firmware build already rebuilds pardes's object when the grid changes, for the reason that changing it in one directory and linking in another silently links the previous answer. The chrome fade is the same shape of setting - baked into the object, wanted from here - so it travels the same path: -Dtheme-animation is forwarded to the nested build, and like the geometry it is skipped entirely when -Dpardes-obj names an object explicitly. Measured either way on the die: the fade costs 1,097 ms of wire per theme change against 215 without it, and compiling it out takes 2,336 bytes off the image. Both numbers are in the comment above the option in ../02-pardes-code/build.zig. Board reflashed with the default (md5 018a9554e4a8ae51); p4-bench --check 4/4; host tests green. 2026-08-26T14:12:04Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T14:12:04Z urn:sha1:adb23572bc2d7aae09cf8605825f6d48ca23ba4a ## -Dcols / -Drows The geometry is baked into pardes's object, and that object is built by the other repository - so changing the grid was two commands in two directories, and the second one silently linked whatever the first had left behind. That is the shape of mistake that ends with a firmware whose grid is not the grid you think you flashed. `-Dcols` and `-Drows` now drive the editor's own build before linking it. A nested `zig build`, not a package dependency: the seam between these repositories stays a file, for the reasons the comment above it has always given. This only reaches across to ask for the file to be made a particular way, and it stands aside entirely when `-Dpardes-obj` names an object explicitly, because then the caller has already said which one they want. Verified by the segment table: `-Dcols=80 -Drows=24` takes the loaded segment from 20,408 to 49,944 bytes, which is the shadow grid going from 784 cells to 1,920. ## Poke and Peek, checked against each other Two more checks in `p4-bench --check`, and they are checked against each other because that is the only way to check either one without a second debugger: a Peek alone cannot tell a correct read from a stuck one, and a Poke alone cannot tell a write from a no-op. Both oracles were wrong before they were right, and both failures are worth recording. The Escape check asked whether an `x` came back on the wire. That was true until the boot buffer gained a line beginning "x selects a line", at which point a passing check began failing for a reason that had nothing to do with Escape. It now reads the CURSOR COLUMN after `0`, which the document cannot spell: 8 in normal mode, three further right if the `0` had been inserted instead. The Poke and Peek checks searched the raw byte stream for `deadbeef` and could not find it - because this renderer emits only the cells that CHANGED, jumping between runs with absolute cursor positioning, so a word on screen is frequently not a word on the wire. `stripAnsi` removes the escapes first. The Poke assertion then had to be loosened from one phrase to three facts, because the message row is 49 columns here and a wrapped line puts a row boundary inside whichever phrase straddles it. Four checks, all passing on the die: lone Escape still leaves insert mode ok (cursor col 8) a click split byte-by-byte lands at 18 ok (cursor 3,18) Poke writes a word and reads it back ok and a later Peek still finds it ok 2026-08-26T12:56:28Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T12:56:28Z urn:sha1:55743cc5d564f6ef6d6f8a0eb1a614a74b021d3a ## zig build selftest Seventeen checks, on the board, chosen by one rule: a check belongs there only if the die can answer it and a host cannot. Every serious bug this port has produced was invisible to a host test. `std.mem.eql` compares a byte at a time on this target, so the firmware's largest read was three times slower than it needed to be and nothing on a laptop could tell. A lone ESC resolves to the Escape key, which is right when a kernel hands over a whole sequence and wrong when a 115200 line hands over one byte every 87 us. A full transmit FIFO stopped anything draining the receiver, and the FIFO depth is a hardware number. So: the volatile promise (two reads of a live counter are two reads, which is what `Peek` rests on), word-wise equality checked against `std.mem.eql` itself at every difference position and both alignments, the input rescue against a fake port with a FIFO that loses what arrives into a full one, the allocator on real L2MEM, and the cycle counter against the systimer - which is clocked from the crystal and therefore cannot be flattered by a wrong CPU divider. Anything that is pure logic stays in `zig build test`, which is faster and needs no hardware. Duplicating those here would make the suite longer and no stronger. Not a `zig test` binary, deliberately: Zig's runner wants an OS and `std.testing.allocator` is a debug allocator over the page allocator, which on freestanding is either a compile error or a lie. The harness is thirty lines. `selftest` has its own application, image and flash chain so it is one command with no flags to remember, and it makes the board's verdict the build's - a suite whose result a human has to read out of a scrolling log is a suite that gets ignored on the first busy afternoon. Proven both ways: 17/17 with exit 0, and exit 1 naming the check when one is deliberately inverted. The clock check earned its place immediately. The first version read `config.cpu_mhz` and compared the die against it without ever performing the raise, so `-Dcpu-mhz=360` failed with `khz=90001 want=360000`. The check was right and the expectation was wrong; it now calls `setCpuFreq` itself, which makes it a test of the raise rather than a tautology. 90001 kHz at 90, 360004 at 360. ## zig build layout, and a loader error that explains itself Both of these exist because of an hour I spent that they would have saved. `NotTwoMappedSegments` said the count was wrong and nothing about what the segments were, which is the only thing that says which section grew, shrank or stopped being emitted. I diagnosed one by hand with readelf on an artifact that turned out to be a stale install, then guessing at the linker script. The image step now prints the segment table with the error - it has to happen there, because a rejected image is never written, so no later step can show it - and `zig build layout` prints the same table for an ELF plus the loader's verdict as TEXT, which `size` cannot do because it reads a finished image and the moment you need the table is when there isn't one. They immediately paid for themselves. The reason the suite would not build was that it had no `_start`: `-fentry=_start` found no such symbol, `--gc-sections` discarded every function as unreachable, and `.flash.text` was empty. `layout` prints `entry 0x0` for exactly that, in one line. The second failure - a silent board - was a missing app descriptor, which is the same class of thing and now has a comment where it happened. ## The panic handler already existed, and was printing past the end of its message `msg` is a Zig slice and `%s` reads until a NUL, so handing `msg.ptr` to the ROM's printf printed the message and then whatever followed it in memory. String literals get away with it; std's own panics do not, because they are formatted into a buffer - "index out of bounds: index 5, len 3" - and carry no terminator. It now goes out through `uart.write`, which takes a length, with the fault address after it so addr2line can find the line. 2026-08-26T04:13:28Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T04:13:28Z urn:sha1:e1526395233aad15dc9e2fdb49d5842de3e7e77b Reported as "a key is stuck and is only sent when I send a new event". It was neither stuck nor late - it was gone, and a later frame repainting those cells is what made it look like it arrived eventually. The loop is read, apply, render, write, and `uart.write` blocks while the transmit FIFO is full. That wait is real backpressure and should stay: dropping half an escape sequence leaves the host terminal in the wrong colour for the rest of the session. But NOTHING drained the receive FIFO during it, and that FIFO is 128 bytes - 11 ms of wire at 115200. Measured on the die, typing a burst in one host write and counting what the firmware's loop actually took off the UART: burst before after after + chunked input 128 128 128 128 200 197 200 200 300 257 300 300 600 478 600 600 1200 - 1200 1200 2400 - 2316 2400 4096 - 3611 4096 Two windows had to close, and the second was only visible once the first was shut. `input_rescue.pump` drains the receiver on every iteration of the wait for transmitter room. That is the big one, and it is the whole reason this policy lives in its own file: `uart.zig` cannot be tested without the chip because every line of it is an MMIO access, while `pump` takes its port as `anytype` and runs against a fake with a two-byte transmit FIFO and an eight-byte receive FIFO in `zig build test`. The fake models the receive FIFO the way the hardware behaves - a byte arriving into a full FIFO is simply gone - so the test fails by 67 lost bytes with the rescue removed, which is the die's 88-of-200 in miniature. It also caught a flaw in its own first draft: a fake whose transmit FIFO drains as fast as it fills never blocks, so `pump` never waits and the test proves nothing. The second window was APPLYING the input. A keystroke costs 44 us on an empty line and 63 us at 640 characters, so handing the editor a full 128-byte batch is up to 8 ms in which nothing drains the receiver - against 11 ms of FIFO. The loop now feeds the editor eight bytes at a time and rescues between chunks. Splitting a burst at an arbitrary byte is already safe, because `pardes_p4_input` keeps whatever it could not parse; that is how it survives an escape sequence split across two UART reads. One render still happens per loop iteration, so this costs no extra wire. Eight rather than thirty-two by measurement: 32 left 2400 and 4096 lossy, 8 does not. Beyond 4096 bytes in one burst the editor genuinely cannot keep up, and the ring reports what it abandoned instead of losing it silently - `rxdrop` in the PROF line, alongside a running count of received bytes. That counter is the other lesson here: the first attempt at measuring this counted characters on the reconstructed screen, which cannot distinguish "never arrived" from "arrived but off the edge of the viewport", and it disagreed with the hardware in both directions. No cost to latency: round trip median 3687 us over 60 trials against 3687 before, maximum 3866, screen byte-identical to the vaxis reference, host tests green. 2026-08-26T00:11:24Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-26T00:11:24Z urn:sha1:aa524eae414f4875794638df0586c011c6b0f2c5 The board was executing at 90 MHz because the stock second-stage bootloader is built with CONFIG_BOOTLOADER_CPU_CLK_FREQ_MHZ=90 (bootloader_clock_init.c:27-37). Measured here against the systimer, which is XTAL/2.5 and therefore an independent reference: 4,500,367 cycles in 50,004 us = exactly 90 MHz. The CPLL is ALREADY at 360 MHz - 90 is 360/4 - so this is a divider change and nothing else. No PLL to enable, no lock to wait for, and the P4 has no per-frequency voltage step to order it against (rtc_clk_init.c:58-80 sets HP_ACTIVE DBIAS once from efuse). `hal/clkrst.zig:setCpuFreq` writes the four dividers in ESP-IDF's upscale order - APB, SYS, MEM, then CPU, with a bus-update handshake after each - because IDF's own comment says the other order passes through a state where APB or MEM violates its timing. Then it calls the ROM's `ets_update_cpu_frequency`, without which every `ets_delay_us` in the image is wrong by exactly the frequency ratio. Measured after: 359,991 kHz. Nothing else moved, which is the reason this is safe from a running console: UART0's baud clock comes from XTAL (hal/uart.zig:116-139), the systimer from XTAL/2.5, and the flash interface from SPLL 480 MHz - none of them from the CPU. The `cycle` CSR simply counts faster, and the board never converts it, so only the divisor in experiments/ had to move. ## What it bought step fixed per char at 160 chars ReleaseSmall 16.99 ms 54.3 us 25.56 ms ReleaseFast 14.85 ms 34.7 us 20.30 ms 0.79x + ASCII grapheme 14.56 ms 12.0 us 16.46 ms 0.64x + ASCII print 14.27 ms 6.9 us 15.36 ms 0.60x + shadow grid 8.87 ms 7.3 us 10.02 ms 0.39x + byte compare 8.37 ms 7.1 us 9.48 ms 0.37x + 360 MHz 4.37 ms 1.9 us 4.67 ms 0.18x Compute went 6.42 -> 2.44 ms: 2.6x for a 4x clock, not 4x, and the shortfall is the point. At 360 MHz the grid walk reads 27 KB per frame in 226 us, about 6 cycles a byte, so that stage is bounded by L2MEM bandwidth and does not care how fast the core is. The prediction that this would happen was made before the measurement and held. ## The goal was 4 ms and this is 4.37 Short by 372 us, and the remaining budget is known: ~2.0 ms of host and USB latency that no firmware change touches (measured independently against the protocol responder), plus 2.4 ms of board compute of which vaxis's own diff is 631 us, pardes's Surface rebuild ~505 us and our grid walk 226 us. vaxis's diff is the only item large enough to close the gap alone, and it is redundant work - `present` already computes exactly which cells moved - so emitting ANSI straight from the shadow grid would do it. I did not, because it is a from-scratch renderer and the honest verification for it needs more than the harness currently proves. ## Verification, and a bug in my own instrument Raising a core clock 4x is exactly the change that corrupts a screen quietly, so the A/B compares screens across clocks as well as across the shadow-grid flag. The first attempt REPORTED A DIFFERENCE at 360 MHz, and it was the verifier: it hashed the raw SGR parameters applied to each cell, which is history-dependent, and a faster board splits the same keystrokes across different frames. Decoding SGR into actual state - resolved foreground, background and attribute set per cell - it is identical: same characters and same style everywhere, both across clocks and across the flag. Also checked and found innocent: `rtt.zig` polled with a 1 ms timeout, which looked like it would quantise every sample. It does not - poll(2) returns when data arrives, not when the timeout expires - and switching to a non-blocking spin moved the measured round trip by 0 us. The comment now says so, since the next reader will wonder too. snap 95/95, hxdiff 481 cases 0 mismatches, hxparity 561 cases 0 mismatches, unit-test, zig-p4 host tests, tty and p4 both build. 90 MHz remains the default; -Dcpu-mhz=360 is opt-in because every number in experiments/ up to this commit was taken at 90. 2026-08-25T22:16:25Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-25T21:49:57Z urn:sha1:1935944a0352e9d176f0718063327e686ece1948 The board half: the -Dprof attribution that overturned the conclusion, its data, and the report correction. `-Dprof` adds two cycle-counter reads around `pardes_p4_input` and `pardes_p4_render` and prints both. Off by default: it puts a line on the wire per frame, which is the very resource being measured, so it answers "where did the 15 ms go" and not "how fast is it". It answered. Input is flat at ~220 us regardless of document size - 1.5% of a keystroke - and the entire ~15 ms floor plus every microsecond of the per-character slope live in `render`. The edit-path fix that the source reading implied (committed next door in 02-pardes-code) is worth 20% on a 19 MB file and, measured here over 5 conditions x 7 trials, exactly 0% on this board. `experiments/report.typ` gains Experiment 3 and a correction: Experiment 2's mechanism claim was wrong, says so, and carries the disproof beside it. The ranked recommendations are reordered with the renderer at #1. Also here: `--sweep position` in p4-bench, which holds the document fixed at one 320-character line and moves only the cursor. Column 320 costs 33.9 ms and emits 28 bytes; column 0 costs 26.0 ms and emits 81. Latency and output size are inverted on this board - the signature of a walk from the start of a line. 2026-08-25T21:41:18Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-25T21:41:18Z urn:sha1:1cef9c2e4bd873ebe13f5df635899231bcc467d2 "Too slow for interactive use" is a real complaint and not a number. This adds the number, and the number says the wire is innocent. ## The instrument `tools/perfproto.zig` is a small framed protocol - "P4", op, length, CRC-32 of the payload, payload - shared VERBATIM by the host tool and `examples/uartperf.zig`, so a frame one writes and the other parses cannot drift. It is imported as a module by both, not copied. The checksum is the whole point. RX overrun on this UART is undetected in hardware and uncounted in the driver, so a byte that never arrived is indistinguishable from a late one; a throughput figure that is not checksummed is a guess about how fast data was corrupted. `sink` accumulates a CRC over every payload byte the board received and `report` hands it back, so the host can prove that what arrived is what it sent. `tools/rtt.zig` is the two timing functions: `roundTrip` and `measure`. Round trip is to the FIRST response byte, deliberately. A renderer that starts drawing in 8 ms and finishes in 130 ms feels immediate; one that thinks for 130 ms and then draws in 8 ms feels broken; waiting for the wire to fall quiet cannot tell them apart. Time to the last byte is recorded separately as `settle`. Microseconds, because at 115200 one byte is 87 us and a millisecond clock quantises the answer into buckets eleven bytes wide. `tools/bench_main.zig` is `p4-bench`: `--link` for the ceiling, `--editor` for how much of it the editor uses, `--sweep` for one controlled variable at a time with `--csv` raw per-trial output. ## What it measured The link is essentially perfect: 11,496 B/s up and 11,413 B/s down, 99.8% of capacity in both directions, CRC verified over 32,768 B each way, zero corruption. Typing at 6 to 100 keys/s loses nothing and never uses more than 9% of the wire, so H5 - "typing loses input" - is refuted. Latency is compute per input event, not transmission. A 40-byte motion and a 206-byte insert-and-escape cost the SAME round trip to within 0.3 ms, across a five-fold range of output. That is why raising the baud cannot fix typing: there is almost no wire in it. And an edit costs the whole document. Round trip against characters already in the line is a straight line at 54.3 us per character per keystroke - 17.0 ms at an empty line, 25.6 ms at 160. On a ~90 MHz core that is ~5,000 cycles per character, far more than a copy alone, so the full-buffer copy the source does is accompanied by at least one more full pass. One controlled intervention: building the editor object ReleaseFast instead of ReleaseSmall cuts the fixed cost 13% and the per-character cost 36%, for 35% more flash (809,536 B of a 1,536,000 B partition). Its advantage grows with the document. Nothing else measured comes close to that ratio. ## Three bugs found while building it The responder printed garbage and looked dead: it read `.rodata` before evicting the bootloader's stale cache lines. `flushFlashCache` moved from `src/pardes/app.zig` to `soc.zig` with its measured evidence, since every application that touches `.rodata` after hand-over needs it and exactly one file knew that. Then it booted, printed its marker and went silent after ten seconds: `rst:0x10 (CHIP_LP_WDT_RESET)`. The bootloader arms the RTC watchdog and expects the application to take it over. Only the editor ever did. `serial.Port.drain()` drains INPUT, not output - so timing a transfer to it reported 202% of the wire's capacity and ate the reply. Added `flushOutput` (tcdrain), named so the two cannot be confused again. Also: Zig 0.16 emits an explicit `+` for a non-negative SIGNED integer whenever a width is given (std/Io/Writer.zig:1548-1559), which put a `+` in front of every number in the first tables. ## The report `experiments/report.typ` reads the raw CSVs and computes its own figures, so a re-run changes the document instead of contradicting it. It states five hypotheses, settles each against one experiment, and is explicit about the one that failed: the geometry sweep is confounded, because characters accumulated across conditions and the length experiment then proved that matters. It is reported as unsupported rather than dressed up as a result. 2026-08-25T19:17:41Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-25T19:16:15Z urn:sha1:0b11540923d5ec2de57fc199c4c9309f838133f5 `zig build interact` shredded the editor's screen with fragments of `[11/13] steps +- console`. The cause is structural, not cosmetic: an interactive step lives for as long as the human does, and `std.Progress` redraws the build runner's step tree on stderr every 80 ms for all of it. std already solves this, and the solution is a lock rather than a flag. `Step.Run` with `stdio == .inherit` holds `io.lockStderr()` for the entire lifetime of the child (std/Build/Step/Run.zig:1588-1592) - the same lock `Progress` must take to draw. A hand-rolled step gets none of that unless it takes the lock itself, and `ConsoleStep` did not. So the console is now a real host program, `tools/console_main.zig`, and `console`/`interact` are `Run` steps on it. `--color off` is not needed and is no longer suggested anywhere. Measured under a real pty (a pipe hides the bug, because progress only draws to a terminal - which is why every earlier end-to-end test here looked clean): zero bytes of progress output across an 11-second session, 3 frames, the editor's `^` modified-marker landing at row 2 after two keystrokes. The second reason for a program is that "just connect" should not imply a build. Once the firmware is in flash the board runs it across resets, so the common case during use is to open the port and nothing else: zig-out/bin/p4-console # the board is already programmed zig-out/bin/p4-console --no-reset # ...and leave a live session running `--no-reset` is the interesting one and it is proven on the die: attaching to the running editor produced 309 bytes with no ESP-ROM banner and zero `boot:` lines, then a `^` frame in response to typing. The session survived a detach and reattach with no repaint, which is exactly what a 11.9 KB/s link wants. `zig build console` is 4 steps and builds no image, no app and no object. It installs `p4-console` itself rather than going through `installArtifact`, so `zig build` alone still lands exactly one file in `zig-out` - verified from an empty tree. Also here: * `tools/console.zig`'s `ModeWatch` tests were reachable by nothing. `zig build test` runs them now; the matcher has to resynchronise when the byte that broke a match is the next match's ESC, which is worth a regression test. * The port-permission advice existed twice, in `failPort` and in the new program. It now lives once, in `tools/serial.zig` beside the code that opens ports, and says nothing about the path so each caller can name its own on the first line. 2026-08-25T19:17:41Z Gabriel Schneider gbrls@0x4200.cafe 2026-08-25T18:42:03Z urn:sha1:6b176c598e55a48572e5e508a213491eec122bf4 AccessDenied on /dev/ttyUSB0 is the first thing anyone hits, and "cannot open /dev/ttyUSB0: AccessDenied" does not help. The steps that open the port now share one failure path that names the group, and names the part that actually confuses people: adding yourself with usermod -aG does NOT affect a shell that is already running, because credentials are captured at login. So the same error greets someone who has already fixed it. The message offers newgrp for this shell and sg for one command.