pardes as P4 firmware: the seam, and a flash-mapping bug in this toolchain

The editor arrives as one freestanding OBJECT exporting a seven-function C ABI
(src/pardes/app.zig declares it, ../02-pardes-code/src/p4.zig implements it), not
as a package dependency. A build.zig.zon path dependency was built first and
reverted: merely DECLARING it nested pardes's ~30-package graph under this one and
broke every build here - std/Build.zig:2091 exceeded its 1000-branch comptime
quota via ghostty's lazyImport, seven cached tree_sitter versions use APIs removed
in 0.16, and the fetch wrote 2.6 GB across 42,736 files into this working copy.

The seam is bytes in and bytes out, which is what a serial line is anyway: the
editor owns vaxis and the ANSI encoding, this side owns the UART, the heap and the
clock, and neither names the other's types. It is versioned, because linkers do
not type-check C symbols and a drifted signature would link cleanly and then
corrupt the stack.

THE BUG WORTH THE COMMIT. .flash.text was ALIGN(64), and the image builder's
anchor makes two mapped segments share an MMU page safely - as long as rodata does
not END inside the page where text BEGINS. With a 578 KB image it does. A volatile
read of a string literal at 0x4004a1d1 returned 37 09 fa 4f, which disassembles to
"lui s2, 0x4ffa0": this image's own .flash.text. Every literal in that last shared
page read as code, so the first thing the firmware tried to print was machine code
and it died on an instruction access fault. .flash.text is now ALIGN(0x10000),
making the segments page-disjoint. The packing trick this project opened with only
ever mattered when the alternative was 64 KiB of zeros in a 1 KB image.

Two more findings, both recorded in README.md:

  * A linker symbol declared as an anyopaque OBJECT gives the optimiser a
    zero-sized object, so ordinary stores through a pointer derived from its
    address are dead code it may drop - and did, silently. The allocator's first
    block header read back as size=2988759312 next=0x14284684 and the free-list
    walk never terminated. @extern with a many-pointer has no size to lose.
    examples/memprobe.zig could not have caught it: it writes through a volatile
    pointer, which the optimiser must leave alone.

  * The RTC watchdog is armed at handover. Every example here had been resetting on
    a ten-second cycle, invisibly, because no run had ever lasted eight seconds.

State, honestly: the firmware boots, clears .bss, brings up the console, disables
the watchdog, starts the systimer, checks the ABI version, initialises the 384 KiB
heap and calls into the editor, which sets up its sink and its environment. It then
faults inside pardes_p4_init on the first allocation. The cause is measured but not
fixed: a load from .flash.rodata page 3 returns the contents of the page 0x50000
higher - exactly the vaddr distance between the rodata and text segments - while
pages 0, 2 and 4 read correctly. The bisect markers that localised it are still in
place, deliberately, because the next step needs them.


--- correction, measured after the above was written ---

Two mapped segments is NOT a choice, and the earlier comment in tools/image.zig
was right for a reason I initially got wrong and then measured.

I first read bootloader_utility.c's `#else` branch, which classifies segments by
address window with two independent ifs - and since the P4's DROM and IROM windows
are the identical range (soc.h:146-149), I concluded the last mapped segment wins
both roles and the first is never mapped. That branch does not run on this chip.
The P4 takes the SOC_MMU_DI_VADDR_SHARED branch (bootloader_utility.c:805-851),
whose own comment says it: "On chips with shared D/I external vaddr, we don't
divide them into either D or I, as essentially they are the same." It collects
mapped segments POSITIONALLY into rom_addr[2] and ends with

    assert(rom_index == 2);

Shipping a one-segment image proved it, on the board:

    Assert failed in unpack_load_app, bootloader_utility.c:842 (rom_index == 2)

So the split stays, image.zig keeps enforcing exactly two - turning that boot-time
abort into a build-time error - and both are now documented with the branch that
actually runs and the assert that actually fires.

What DOES change is alignment. .flash.text was ALIGN(64). Two mapped segments may
share a 64 KiB MMU page only if they also share a flash page, which the image
builder's anchor guarantees - and that holds right up until an application is large
enough for rodata to END inside the page where text BEGINS. With a 578 KB image it
does. Measured on the die: a volatile read of a string literal at 0x4004a1d1
returned 37 09 fa 4f, which disassembles to "lui s2, 0x4ffa0" - this image's own
.flash.text. Every literal in that shared page read as code, so the first thing the
firmware tried to print was machine code, and it died on an instruction access
fault. .flash.text is now ALIGN(0x10000), which makes the segments page-disjoint.
It costs up to 64 KiB of image padding against a 1.5 MiB partition; the packing
trick this project opened with only mattered when the alternative was 64 KiB of
zeros in a 1 KB image.

With that fixed the firmware gets much further: entry, .bss cleared, console up,
watchdog disabled, systimer running, ABI version checked, the 384 KiB heap
initialised, into the editor, its sink and environment ready - and the literal at
0x4004a1d1 now reads back correctly.

Still open, and characterised rather than guessed: pardes_p4_init faults on its
first allocation. The allocator struct crosses the seam intact (its function
pointers land in .flash.text), but the std.mem.Allocator vtable at 0x40035a1c reads
back as instruction bytes, and the dispatch at .flash.text+0xade2 jumps through it.
Ruled out with measurements: the ELF and the image agree at that address, the flash
is MD5-verified against the image, the wrong bytes are identical across three
resets and two reflashes (so not a stale cache), the corruption is a contiguous run
rather than 64-byte lines, and mmu_hal_map_region's arithmetic
(page_num = ceil(len/page), entry from vaddr) is correct for the segments as now
laid out. The next measurement is the one that settles it: read the MMU entry
registers from the running application and print vaddr -> flash for every page. The
register model in src/soc.zig can do that; the bisect markers are left in place for
it.

1 files changed, 239 insertions, 0 deletions

diff --git a/tools/console.zig b/tools/console.zig
new file mode 100644
index 0000000..2e924e3
--- /dev/null
+++ b/tools/console.zig
@@ -0,0 +1,239 @@
+//! An interactive terminal on the far side of the serial port.
+//!
+//! `MonitorStep` prints the board's output for N seconds and never sends anything. That is the
+//! right tool for a program that only reports. It is the wrong tool for a program the human is
+//! supposed to *use*, which needs the host's keystrokes on the wire and the host's terminal out of
+//! the way. So this is the other half: raw-mode stdin forwarded to the UART, UART forwarded to
+//! stdout, until the escape byte.
+//!
+//! The division of labour is the interesting part. The board runs the application and emits ANSI;
+//! the host terminal emulator (ghostty here) does the actual terminal work - fonts, scrollback,
+//! selection - and answers the application's capability queries itself. This process is a wire, and
+//! deliberately almost transparent: a `\x1b[?1049h` from the board reaches ghostty, ghostty's reply
+//! to a `CSI c` reaches the board, and neither end needs to know there are 3 metres of USB cable
+//! and a CH340 in between.
+//!
+//! Almost transparent, because of one thing a wire cannot pass through: **size**. A terminal
+//! application learns its window size from `ioctl(TIOCGWINSZ)`, and firmware has no ioctl. DEC mode
+//! 2048 ("in-band resize") solves half of it - a terminal that has been sent `\x1b[?2048h` reports
+//! every subsequent resize as `CSI 48 ; rows ; cols ; ypix ; xpix t`, and those reports flow down
+//! the wire like any other bytes. The half it does not solve is the *first* size, because a change
+//! notification is only sent on a change. This process owns the real tty, so it is the only party
+//! that can answer, and it injects that same sequence itself:
+//!
+//!   * once at attach, for a board that never asks;
+//!   * on SIGWINCH, because the host's window changed and mode 2048 may not be enabled;
+//!   * and on seeing `\x1b[?2048h` come *back* from the board - which is the exact moment the
+//!     application has declared itself ready to understand one. That is the deterministic trigger;
+//!     the other two are belt and braces.
+//!
+//! Injecting a resize report the host terminal would also have sent is harmless: it carries the
+//! size as data, so a duplicate is idempotent, and `vaxis`'s parser (`src/Parser.zig:528-551`)
+//! treats both identically because they are byte-for-byte the same sequence.
+
+const std = @import("std");
+const posix = std.posix;
+const linux = std.os.linux;
+const serial = @import("serial.zig");
+
+/// Ctrl-] , telnet's escape and not a key any full-screen application binds. Ctrl-C, Ctrl-Q and
+/// Ctrl-Z all had to be rejected: an editor wants every one of them, and a bridge that swallowed
+/// them would be lying about being transparent.
+pub const escape_byte: u8 = 0x1d;
+
+/// Set by the SIGWINCH handler, read by the loop. `volatile` rather than atomic because a signal
+/// handler on the same thread is not a concurrent writer - it is an interruption - and this only
+/// needs the compiler to stop caching the load.
+var winch_pending: bool = false;
+
+/// The handler takes `posix.SIG`, not an int: std's `Sigaction.handler_fn` is
+/// `*align(1) const fn (SIG) callconv(.c) void` (std/os/linux.zig:6026).
+fn onWinch(_: posix.SIG) callconv(.c) void {
+    @as(*volatile bool, &winch_pending).* = true;
+}
+
+const Winsize = extern struct { row: u16, col: u16, xpixel: u16, ypixel: u16 };
+const TIOCGWINSZ = 0x5413;
+
+fn windowSize() Winsize {
+    var ws: Winsize = .{ .row = 24, .col = 80, .xpixel = 0, .ypixel = 0 };
+    // A failure here is not fatal: 80x24 is a defensible terminal, and the alternative is refusing
+    // to attach because the size could not be read.
+    _ = linux.ioctl(0, TIOCGWINSZ, @intFromPtr(&ws));
+    if (ws.row == 0) ws.row = 24;
+    if (ws.col == 0) ws.col = 80;
+    return ws;
+}
+
+/// The in-band resize report, in the form `vaxis` parses: `CSI 48 ; rows ; cols ; ypix ; xpix t`.
+/// Note the pixel fields are height-then-width, which is the opposite order from the
+/// `struct winsize` they come out of - `Parser.zig:536-539` reads height first.
+fn sendWinsize(port: *serial.Port, ws: Winsize) void {
+    var buf: [64]u8 = undefined;
+    const seq = std.fmt.bufPrint(&buf, "\x1b[48;{d};{d};{d};{d}t", .{
+        ws.row, ws.col, ws.ypixel, ws.xpixel,
+    }) catch return;
+    port.write(seq) catch {};
+}
+
+/// Matches `\x1b[?2048h` in the board's output stream one byte at a time, because the sequence can
+/// be split across reads. Returns true on the byte that completes it.
+const ModeWatch = struct {
+    const want = "\x1b[?2048h";
+    at: usize = 0,
+
+    fn feed(m: *ModeWatch, byte: u8) bool {
+        if (byte == want[m.at]) {
+            m.at += 1;
+            if (m.at == want.len) {
+                m.at = 0;
+                return true;
+            }
+        } else {
+            // Restart, and allow this byte to be a fresh start - otherwise "\x1b\x1b[?2048h" is
+            // missed.
+            m.at = if (byte == want[0]) 1 else 0;
+        }
+        return false;
+    }
+};
+
+pub const Options = struct {
+    /// Pulse reset so the application starts from boot with the console already attached. Without
+    /// it, attaching to a board that has been running for a while shows a screen mid-session with
+    /// no redraw until something changes.
+    reset: bool = true,
+    /// Print the escape-key hint. Suppressed for scripted runs, whose output is being asserted on.
+    banner: bool = true,
+};
+
+/// Forward bytes both ways until the escape byte arrives on stdin.
+///
+/// Returns normally on escape; the terminal is always restored, including on error, because the
+/// alternative is handing the user back a shell with no echo.
+pub fn attach(port_path: []const u8, baud: serial.Baud, opts: Options) !void {
+    var port = try serial.Port.open(port_path, baud);
+    defer port.close();
+
+    // No `isatty`: `tcgetattr` answers the same question with the same syscall this needs anyway,
+    // and a null here means "stdin is a pipe" - which is a supported way to run this, for scripted
+    // sessions whose input is a file.
+    const saved: ?posix.termios = posix.tcgetattr(0) catch null;
+    if (saved) |prev| {
+        var raw = prev;
+        // The same raw mode `serial.Port.open` builds for the port, for the same reason: every byte
+        // the user types has to reach the board unmodified, including the ones the line discipline
+        // would otherwise interpret. ISIG off is what lets Ctrl-C reach the application instead of
+        // killing this process.
+        raw.lflag.ICANON = false;
+        raw.lflag.ECHO = false;
+        raw.lflag.ISIG = false;
+        raw.lflag.IEXTEN = false;
+        raw.iflag.IXON = false;
+        raw.iflag.ICRNL = false;
+        raw.iflag.INLCR = false;
+        raw.iflag.BRKINT = false;
+        raw.oflag.OPOST = false;
+        try posix.tcsetattr(0, .FLUSH, raw);
+    }
+    defer if (saved) |prev| posix.tcsetattr(0, .FLUSH, prev) catch {};
+
+    // Installed after raw mode so a resize during setup cannot be missed-but-flagged.
+    posix.sigaction(posix.SIG.WINCH, &.{
+        .handler = .{ .handler = onWinch },
+        .mask = posix.sigemptyset(),
+        .flags = 0,
+    }, null);
+
+    // stdin/stdout as `std.Io.File`, because std 0.16 has no `posix.read`/`posix.write` any more -
+    // byte traffic goes through std.Io. The `io` is borrowed from the port, which already holds the
+    // single-threaded instance `serial.Port.open` created.
+    const io = port.io;
+    const stdin: std.Io.File = .{ .handle = 0, .flags = .{ .nonblocking = false } };
+    const stdout: std.Io.File = .{ .handle = 1, .flags = .{ .nonblocking = false } };
+
+    if (opts.banner) {
+        var hint: [96]u8 = undefined;
+        const line = std.fmt.bufPrint(&hint, "[zig-p4 console @ {d} baud - Ctrl-] to detach]\r\n", .{
+            baud.rate(),
+        }) catch "[zig-p4 console - Ctrl-] to detach]\r\n";
+        stdout.writeStreamingAll(io, line) catch {};
+    }
+
+    if (opts.reset) try port.resetToRun(.{});
+    sendWinsize(&port, windowSize());
+
+    var watch: ModeWatch = .{};
+    var from_board: [4096]u8 = undefined;
+    var from_user: [256]u8 = undefined;
+
+    while (true) {
+        if (@as(*volatile bool, &winch_pending).*) {
+            @as(*volatile bool, &winch_pending).* = false;
+            sendWinsize(&port, windowSize());
+        }
+
+        var pfd = [_]posix.pollfd{
+            .{ .fd = 0, .events = posix.POLL.IN, .revents = 0 },
+            .{ .fd = port.file.handle, .events = posix.POLL.IN, .revents = 0 },
+        };
+        // A bounded wait rather than an infinite one so a SIGWINCH that lands between the check
+        // above and the poll below is still serviced promptly; poll reports the signal itself as
+        // an interrupt, which is handled as "go round again".
+        const ready = posix.poll(&pfd, 200) catch continue;
+        if (ready == 0) continue;
+
+        // POLL.IN is not the only thing poll reports, and ignoring the rest is a hot spin, not a
+        // no-op: unplug the CH340 mid-session and the port's revents carries HUP|ERR|NVAL forever.
+        // poll then returns immediately with a non-zero count, neither branch below matches because
+        // both test POLL.IN, and the loop burns a core with stdin still in raw mode and ISIG off.
+        // `std.posix.poll` cannot surface it as an error either - it maps INVAL to `unreachable`
+        // (std/posix.zig:1007-1017) because a dead descriptor is reported in `revents`, not errno.
+        const gone = posix.POLL.HUP | posix.POLL.ERR | posix.POLL.NVAL;
+        if (pfd[1].revents & gone != 0) return error.PortDisconnected;
+        // stdin dying is ordinary: a pipe ran out, or the terminal closed. Detach quietly.
+        if (pfd[0].revents & gone != 0) return;
+
+        if (pfd[1].revents & posix.POLL.IN != 0) {
+            const n = port.read(&from_board) catch 0;
+            if (n > 0) {
+                stdout.writeStreamingAll(io, from_board[0..n]) catch {};
+                for (from_board[0..n]) |b| {
+                    if (watch.feed(b)) sendWinsize(&port, windowSize());
+                }
+            }
+        }
+
+        if (pfd[0].revents & posix.POLL.IN != 0) {
+            const n = stdin.readStreaming(io, &.{&from_user}) catch 0;
+            if (n == 0) return; // stdin closed: a pipe ran out, so detach
+            if (std.mem.indexOfScalar(u8, from_user[0..n], escape_byte)) |cut| {
+                // Everything before the escape still belongs to the board.
+                if (cut > 0) port.write(from_user[0..cut]) catch {};
+                if (opts.banner) stdout.writeStreamingAll(io, "\r\n[detached]\r\n") catch {};
+                return;
+            }
+            port.write(from_user[0..n]) catch {};
+        }
+    }
+}
+
+test "ModeWatch completes only on the full sequence" {
+    var m: ModeWatch = .{};
+    for ("\x1b[?2048") |b| try std.testing.expect(!m.feed(b));
+    try std.testing.expect(m.feed('h'));
+}
+
+test "ModeWatch resynchronises on a false start" {
+    var m: ModeWatch = .{};
+    // A prefix that dies, then the real thing immediately after: the naive reset-to-zero misses
+    // this because the byte that broke the match is itself the next match's ESC.
+    for ("\x1b[?20") |b| try std.testing.expect(!m.feed(b));
+    for ("\x1b[?2048") |b| try std.testing.expect(!m.feed(b));
+    try std.testing.expect(m.feed('h'));
+}
+
+test "ModeWatch ignores unrelated traffic" {
+    var m: ModeWatch = .{};
+    for ("hello \x1b[?1049h world \x1b[0m") |b| try std.testing.expect(!m.feed(b));
+}