# The daemon: one pardes per machine, many clients per session > I want to have a global pardes daemon ... holding file descriptors for > ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a > cpu+file server ... the main test: multiple clients to a single session, > and a client here and the session on freebsd and very deep and thorough > performance testing and optimizations to make it feel like its local and > snappy. Designed 2026-10-29 in discussion (the user, the coordinator, a survey agent). Nothing here is built yet. Paths are relative to the repo. ## Decided by the user - **A deterministic core, replicated.** The session is the authority: it orders every client's input and every host result, and only it performs effects. Each client runs a full core replica, applies its own input at once, and rolls back and replays when the order differs. Visible rollback is acceptable. - **Kakoune-style clients.** Each client has its own view (layout, focus, selections, scroll) over shared buffers and terminals. Registers are session-wide. A terminal shown at two sizes is sized by its last typist. - **Performance order.** First, the regular tty and GUI get no slower. Then the detached and remote clients reach that local performance, with the session on the FreeBSD box included. - **A host step may show up one RTT late, but is never wrong.** This covers Look, open and exec, as sam's hostlock does. - **Join is complete.** The joining client pays for it in time; the authority and the other clients do not. - **One long-lived daemon per user per machine.** It supervises one child process per session, postgres-style. Nothing is left dangling, and the core is not de-globalised. ## What today's seam is, and why it goes `src/detached/server.zig` is a third Host: it runs the core and the ptys in a single poll loop and sends cell frames to the frontends, at the smallest common grid across them. - **The frames are mostly not diffed.** Every layer's cells are re-sent on each frame (wire.zig:734, :755). - **Several things never cross the wire.** Images, PDF pixels, rail ticks and pane animations stay behind (surface.zig:483-509). - **Every keystroke is a round trip,** which no amount of optimisation can make feel local at 30-80 ms. - **Each remote 9P operation dials afresh,** costing about 6 RTTs (9p_io.zig:2361-2385). ## The design - **Processes.** - The daemon is subreaper, spawner, fd holder, registry and 9P root. - Every session is a child of the daemon: a GUI window, a tty session behind a stub, or a headless session. - A session is always an in-process authority with its own local view, so the regular path does not change. Replicas exist only for extra clients or remote ones. - **The tty stub.** - `pardes` in a terminal passes its terminal fd to the daemon (SCM_RIGHTS), along with cwd, env and argv. - The session child drives that fd directly. The stub forwards SIGWINCH and exits with the session's status. - Detach: the session drops the fd and the stub exits. No fork happens. - **GUI detach.** The window is destroyed and the process carries on headless. Closing a window that has no other client attached quits it. - **Shells survive a session crash.** After each `forkShell`, the session sends the daemon a dup of the pty master, so the shells do not get SIGHUP. Being subreaper only decides who reaps them. - **The recorder.** It is null when no client is attached. When one is, it appends what the authority consumed, in order: - every step's host tape (the results of the ~30 `turn.yield()` sites); - each `serveFs` mutation. Turn order is log order, so the local path changes by one null check. - **The protocol.** One ordered stream: events, host tapes, digests and terminal state. Framing, the hello and the Event codec are kept from wire.zig; the frames go. - **Terminals cross as authoritative screen state,** not as pty bytes: - metadata plus immutable row blobs, assigned in absolute terms; - scrollback fetched on demand. A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the authority, and the local path keeps reading ghostty directly. - **The reflective walker.** One comptime `@typeInfo` walk gives: - the digest; - the complete dump, which also serves join, Restore and Handover; - the rollback copy. Fields are annotated local, authority-only or derived. A hash of the type graph versions it. The hello carries a semantics hash (version, features, grammars), and a mismatched build is refused. - **Rollback.** - When the first speculative event goes pending, the replica copies what speculation can write: views, buffers, registers, counters and terminal metadata. Blobs are pinned rather than copied. - On a divergence it restores that copy, then replays the confirmed events and its own pending ones. - The authority never rolls back. - **Join.** The authority forks a writer child at a step boundary; the child streams the dump. The authority buffers the events after it for the joiner. - **Remote.** - `ssh -L` of the daemon's one socket. - The daemon hands a replication connection's socket to its session. - A remote daemon mounts as `/n/`. ## Determinism leaks (the survey's census) | Leak | Where | Fix | |---|---|---| | Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak | | 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order | | Host syscalls in a step | the ~30 `turn.yield()` sites | host tape | | Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done | | Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local | | Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ | | Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced | | getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state | | Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash | | ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals | ## Phases Each phase ships on its own and keeps `zig build perf --base` level (with an A/A run). All 110 snapshots stay green throughout. 1. **Determinism and the replay-divergence harness.** - Work: the recorder and tape; the leak fixes, the render one first; the semantics hash; the walker's digest. - Check: replay each recorded log into two cores at different addresses and one headless core. - Gate: zero divergences on the snapshot scripts and on N monkey seeds. 2. **A complete dump on the walker.** It replaces dump.zig for Restore and Handover. - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint. 3. **The daemon and the tty stub.** - Work: in-place detach and reattach; pty master dups; the frame wire deleted. - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves its shells adoptable; no stale sockets after N cycles. 4. **Views.** A second window on the same session, in one process. - Gate: logs with several views replay without divergence. 5. **Replicas.** - Work: speculation, rollback, screen-state terminals, join, socket handoff, `ssh -L`. - Gates: - a 30/80 ms delay relay keeps remote within noise of local; - the rollback rate stays low under the fuzzer; - a lagging client leaves the authority flat; - a join costs the authority under one frame. 6. **Remote amortisation.** A content cache by hash, prefetch, zstd, predicted echo in TTY mode, and session upgrade by re-exec. New measurements to add before phase 1 lands: keystroke-to-present in the tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file open. ## Still open (the coordinator's position in brackets) 1. Delete the frame wire at phase 3, leaving no remote attach until phase 5. [yes: it was never used daily] 2. The stream as its own framed protocol, or as a 9P file. [framed: one ordered stream with no per-read round trip; 9P stays for scripting and the relay] 3. Rollback by walker copy or by arena copy-on-write. [whichever measurement of the mutable state's size decides] Decided: the walker copy, about 100 KB and 0.05 ms once file texts are pinned (Phase 5 groundwork notes). 4. How the daemon starts: spawning itself or socket activation. [itself, by double fork; FreeBSD has no systemd. A user unit can be added if a terminal's scope ever kills it] 5. The join writer: a fork child, or a metadata copy serialised on a worker. [fork first, measure the page-table copy] Decided: the copy on a worker. A fork of 1 GB stalls the authority 13 to 20 ms, and copy-on-write taxes it for as long as the join streams (Phase 5 groundwork notes). ## Phase 3 notes Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`). Decisions taken at the forks, with the ponytail lens: - **Sessions are not the daemon's children; every one registers.** The design had the daemon spawn a terminal's session from the fd, cwd, env and argv the stub passes. That proved wrong here: - the user's terminals each run in a `9ns --mntgen` user and mount namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon spawned would run in whichever namespace the daemon was started from, its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits, umask and stdin (`pardes -`) would be the daemon's too, or have to be carried and re-execed; - startup would pay an IPC round trip and an exec of a 100+ MB binary; - the snapshot harness's idleness probe walks the launched process's tree, and a daemon-spawned session is outside it. So `pardes` in a terminal forks its session off itself before anything starts (`daemon.forkSession`): the child calls `setsid` and draws on the `/dev/tty` the parent opened; the parent stays as its stub. GUI windows and `--detach` sessions just register. The daemon spawns nothing, so it is no subreaper: a detached session is reparented to the system's reaper (systemd --user or init), which reaps it, as it reaps a crashed session's shells. Nothing dangles: the daemon lists every session (its connection) and holds every master, and exits a second after it holds neither. - **Starting the daemon.** The first session takes the flock, binds and listens itself and queues its own connection; `pardes --daemon` is then double-forked and exec'd onto that listener and lock (fds 3 and 4), by a terminal's stub after the session's fork, so no session waits for it: its hello sits queued until the daemon runs, and is not waited on (the daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s). With the spawn before the fork, a cold start measured +0.38 ms (+7.6%). The daemon's reader thread likewise starts after the first frame. Another starter waits on the lock; a stale socket is the lock holder's to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and the fds it carries. - **pty masters.** `forkShell` sends a dup; a host's close sends `close`; a session's end sends `quit` (its shells hang up); a master whose shell is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT handlers send `quit` too (one `send`, async-signal-safe), so only a session killed outright or crashed leaves shells behind. - **Adoption is a word, `Adopt`,** calling the host through a vtable entry (no Effect): the daemon hands over every orphaned master, and each becomes a pane through the `Tty` word, its spawn taking the pty instead of forking; a 1x1 size then the pane's makes it redraw. Adopting at startup was rejected: an `EDITOR='pardes --wait'` session would take the shells and hang them up when the editor exits. - **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and ends with the session's status (a forked stub `waitpid`s; an attaching one reads `x`). A HUP is the terminal's end: kept by a daemon, a session takes HUP on its signal thread (sigwait, beside WINCH) and detaches, as on EIO; HUP never ends a kept session, since the terminal's own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane (the terminal is raw, as before); a TSTP sent to the stub is ignored, since a stopped stub hands the terminal back under a session still drawing; a background start stops at a `tcsetattr` before the fork, as the old raw mode did. `--wait`, nested forwarding and `--tty` run before any of this and are unchanged. - **`--detach` is a session started detached:** the tty host with no terminal, the GUI with a hidden window closed at once. With the frames gone, `detached/server.zig` was a third host with nothing left to do that the other two could not. - **`Attach` (the word) is deleted:** it switched a session's own terminal to another session through the frames. - **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a session as before: in the terminal's own process, ended by its hangup, for a harness that kills by pid and closes the terminal. A GUI test window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`. - **Known gaps.** A forked session takes its stub's death, a SIGKILL too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and detaches; a `pardes --attach` stub is no parent of its session, and one killed with SIGKILL while its terminal stays open still leaves the session drawing there until the terminal closes or Detach. A reattached terminal keeps the first one's environment (TERM, COLORTERM); vaxis queries its capabilities again. A daemon killed outright leaves its sessions unregistered, and Detach then refuses. A daemon of an older protocol answers `version`, and a session then runs without one. A GUI started in a terminal's foreground stays that terminal's job. Measured (ReleaseFast, base main a7369739 against these changes, both live at once and their samples alternating A, B, A, B; A/A the base against itself; p50 ratio new/base): | | tty | A/A | GUI (test mode) | A/A | |---|---|---|---|---| | startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 | | startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | | | key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 | | key echoed by bash, to its frame | 1.016 | 1.005 | | | What is left of the tty's startup is the stub's fork (about 70 us), which the design cannot do without: the job the shell waits on must be able to end while the session runs on. Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout), and overran the stack, clobbering a neighbouring local in the forked child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one. ## Phase 4 notes Built 2026-10-29 (`src/views.zig`, the tty and GUI hosts, `daemon.zig`, the snapshot harness and the monkey). Decisions taken at the forks, with the ponytail lens: - **A view is swapped in, not pointed at.** The ~1,900 references to the core's layout, focus and pane state stay as they are: the core sees one view, the live one, in the fields it always had. A view's share of the session is a list of field names (`views.fields`, ~60) made into a struct by one comptime `@Struct` (`Part`), and of each pane a struct beside it (`PaneView`, in `Pane.pv`); going into view `k` copies the live one's out (`park`) and `k`'s in (`unpark`). Copies, not swaps: a slot is stale while its view is live and written whenever it goes out, so no two places own a selection's heap. One view never parks: the single client pays `views.several` (an AND and a compare) in the pump, the step's sync and an edit, and one store a key (`typedInto`). Rejected: a `*View` the core reads through (every reference rewritten, an indirection on every access); a twin pane per view (Zerox exists only for files, and a terminal's pty is its pane's slot). - **Where each piece of state went.** - Each view's own (`views.fields`): the screen size and cell metrics; the columns, their weights, panes, serials, tags and the finder; rects; `active`, `active_column` and the header focus; the jump list (kakoune's is per client); the pointer, hover, drag, mouse grab, clicks and chords; key prefixes (`"x` register name, leader, Ctrl-w, window pick); a pending clipboard read; the presentation and the animations (lifts, cursor glide); `windowed`, `native_images` and `native_image_patches`, which the census had as watched host writes: a tty view on kitty and one on a plain terminal differ. The 223 comptime platform branches stay comptime: a session is one build, which the semantics hash says. - Each pane's, each view's own (`PaneView`): `vweight`, `collapsed` and `attached_to` (a layout's); `focus`; the body's mode, cursor, selections, sticky column, pending normal command and find; a document's scroll, `hscroll` and size; and what the step measured for that view (`tag_rows`, `tag_top`, `tag_scroll`, `view`, `notices`), so a view drawn without a step of its own draws its own geometry. - The session's, every view's: the panes' text, undo, terminals and their scroll (tmux shares a pane's copy mode too), tags (acme's are per window; one tag per buffer is one text, its caret included), the workspace tag, prompts, messages, registers (decided), the macro being recorded, look-hover (one pointer at a time), mouse sweeps, query panes, search state, settings and theme, the effect and input queues, `needs_frame` (any change draws every view). A `.name` prompt (a file's name drafted in its tag) is the view's that opened it (`Pane.prompt_view`): only that view's focus leaving the pane drops it, and its closing. - **A terminal shown at two sizes** takes the size of the view that typed into it last (`Pane.sized_by`, set by a key or a paste; a new pane's is its maker's; a closed view's goes to the next to sync it). The others show it at that size: `sync` resizes no pty for them, and `pane.cols/rows` stay the terminal's own, so the terminal code's invariant (pane size is the emulator's) holds. A view shorter than the terminal is passed down its top rows as far as keeps its cursor's row in sight, as tmux shows a bigger window in a smaller client (`Pane.pan`, terminal.panFor: per view, made in the step and again when a view is drawn out of turn, read where the body's rows start, `Pane.scroll` and the body walk), an arrow in its gutter's first row saying rows are above; a shell's prompt, at the bottom, is never out of sight. ponytail: a panned view's mouse reports to the terminal's program and its kitty images are placed as unpanned. - **Panes come and go under parked views.** A view keeps the serial of each slot when it goes out; unparked, the ids whose serial changed leave its layout and its keyboard moves to its first pane left. A pane focused in a view that does not show it (a Look at a file another view opened, a jump, a 9P focus) is placed in the view's active column (`views.show`, kakoune's buffer shown in the window that asked). A text edited in one view moves every other view's cursor and scroll with the splice, as a Zerox twin's (`File.ParkedPlaces`); their selections go, as a twin's do. - **Drawing the other views stays out of the digest.** The pump draws each other view by unparking it, rendering and presenting, and restores the live one exactly (`drawOthers`): no digested field moves, so the headless replica, which draws nothing and skips it, agrees. What drawing a view makes of its own (its animations, its presentation) is kept; what its unpark normalised (a cursor clamped inside a changed text, a pruned layout) is not, and is done again when it next goes live by an input, in every replica alike. `parked` and `pv` are `.local`: a parked view is checked when it is live again. Found by `pardes-monkey --views=3 --tape`: a drawn view's `post_present` (acknowledgePanelPresentation, which moves the pointer's hover) was applied by the headless replica to the live view; other views take no post_present, their panels acknowledged when they are live and drawn. - **Inputs.** `toView`, `openView` and `closeView` are taped. A host goes into a view before the input its client sent: the tty's terminal readers post through a `Poster` that puts a `from` ahead of a terminal's input whenever another terminal posted last (so one terminal posts none, and coalesced motions never cross views); the GUI asks SDL which window an event is for (`route`), which also takes a window's resize to its view (the GUI sizes the live view from its window each frame). - **The tty.** Each terminal is a `Screen` of its own: its vaxis, input reader, frame answer (`FrameAck`, was global), kitty images and shm probe (were global), glide, trail and flash, its stub and its Killed slot. The primary's stub stays `daemon.stub`. A SIGWINCH asks every terminal its size (attach stubs forward theirs; another view's terminal is no one's controlling terminal); a SIGHUP asks each whether its terminal or stub's pipe hung up, where it used to mean the one terminal. Detach, or a terminal's end, with views left closes only that view and tells its stub `d`; the session's end tells every stub. ponytail: a bracketed paste in progress is the shell's, not a terminal's: keys from another terminal during it join it. - **The GUI.** One `Gui` (device, atlas, fonts, pipelines) and its window's share swapped like the core's (`window_fields`: the SDL window and its presenting, scroll and glide, scene and post chain, pointer, blink, captures); `Shell.follow` keeps the Gui on the core's live view. Closing a window while others stay closes its view; Restore closes the others. ponytail: a second window's compositor hit test reads whichever window's state is in the Gui (the workspace tag is in the same place in every view). - **The daemon.** `attach` names an attached session too: it takes another view (a terminal, or a window). With no name, the detached sessions are asked first, then the only session of that kind. - **Harnesses, extended not new.** A snapshot script's `start ... views=N` gives the app N-1 more ptys (PARDES_TEST_VIEWS, opened as attached ones from the start), `view ` types into and captures terminal k, and every wait reads all of them (a pty nobody reads would block the session's frame); a `resize` there SIGWINCHes the app, as a stub would. `pardes-monkey --views=N` switches among N terminals a quarter of its steps, each with its own size; `zig build replay` runs its seeds at one view and at three. The GUI's test feed takes `ESC]777;view;BEL`, and PARDES_TEST_VIEWS=N opens N more hidden windows, captured under view-k. - **Not done, and why.** - A view of the other kind (a terminal on a window session, a window on a terminal's): each process links one frontend; the daemon still matches kinds. - 9P acts in the live view, the one whose client spoke last: no view is named in a path yet (a script's focus or layout is that view's). - Every change draws every view (`needs_frame` is the session's); a view's own dirtiness waits for a measurement that wants it. - Restore and `-l` keep one view, the live one; a dump has every view (phase 2 notes), and Handover carries panes, which join the views of the window they land in. - **What proved wrong in the design.** "Views" in the design were a struct the core reads through; the swap made them cheaper and kept every reference. Its rollback copy (phase 5) is the same list of fields. The design's view-local state assumed a step per view; the step's measurements (`tag_rows`, `notices`, the pane's geometry key) had to become per view too, or a view drawn after another's step drew the other's tag heights. And the census's single "frontend kind" leak is not per process but per client: kitty graphics are a terminal's. Measured: ReleaseFast, the base binaries from before the rework against this line on phase 1's merge (vvmuwntp, before main's tty throughput fix), interleaved base, new, base ... in one run on the perf cores, then an A/A of the base against itself the same way; the ratio of the medians of runs, new/base (4 runs of new, 5 of base; `--reps 25` for the core, `--live` and `--live-gui` through the 32 MiB live driver): | | new/base | A/A | |---|---|---| | core, medium: render, keys, PageDown, wheel | 0.97-1.02 | 0.99-1.01 | | core, medium: edit-char, edit-twin, deep-sel | 0.90-0.96 | 0.99-1.00 | | core, medium: open, body-2m, body-appends | 0.94-1.02 | 0.88-1.08 | | tty, keystroke to present p50 (5 gestures) | 0.94-1.11 | 0.90-1.02 | | tty, the core's part of it p50 | 0.95-1.04 | 0.86-0.99 | | tty, `yes \| head -c 32M` wall; 1M-line open | 0.96; 0.98 | 1.09; 1.02 | | GUI, keystroke to present p50 (present-bound, 30 ms) | 0.98-1.01 | 1.03-1.10 | | GUI, the core's part of it p50 | 0.84-1.01 | 1.01-1.19 | Every cell is inside its A/A; the widest, the tty's PageDown at 1.11 (runs 386-506 us against 410-451), is 0.94 best against best. What one view pays is a byte test in the pump, the step's sync and an edit (`views.several`), a store a key, an index for the live terminal or window, and a `from` never posted. A first A/B, of this line before the rebase onto phase 1's merge (its render still made the geometry afresh every frame), read the GUI's core at 0.89-1.16 with no A/A to set it against; it is not the one reported. Two views (not gated): every frame draws both, so a change costs two renders; a view switch copies ~60 fields and each pane's share, a few microseconds. Gates, on the line rebased onto main's screen-state terminals (mnlwlpvk): - `zig build replay -Dplatform=tty`: 113 snapshot scripts pass (the 110, views.snap, views-term.snap and views-pan.snap), their 131 tapes replay in all three replicas (393 runs, 29,436 digests); 20 monkey seeds at one view (60 runs, 14,004 digests) and 20 at three views (60 runs, 9,666 digests); no divergence, no panic. views.snap: 75 digests a replica, views-term.snap 35. core-test: 1117 of 1135, 18 skipped. - Unit tests (on qzsouslm): tty 1478 of 1498 pass, 20 skipped; GUI 1549 of 1560, 9 skipped and 2 failed, the same two that fail on phase 1's line (`tagline pointer mapping retains physical grips...` and `a tag caret past its row's end glides...`). The views' own: 8 (own cursor, scroll, mode, layout and focus; an edit and a register shared; a terminal sized by its last typist; views closing; the other views drawn leave the digest; a view keeps its place in a text another edits; a short view keeps a tall terminal's cursor in sight). views-pan.snap: a shell sized for the tall view, its prompt and last lines in the short one under an arrow, then typed into from the short one and sized for it. - test/fs.py passes (with a TMPDIR short enough for the 9P sockets' paths). test/daemon.py: 20 detach and attach cycles, then a second terminal on the attached session, both drawn, detached from inside and then closed outright, the first drawing on, and the kill and Adopt; `--gui`: 10 window cycles, then a second window on the open one, detached alone. - `pardes-monkey --views=3 --tape` found the one divergence fixed here (the drawn views' post_present). ## FreeBSD notes Groundwork for phase 5's main test, a client here and the session on the FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and runs there. The GUI is not built for it, since frontends stay local. The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is the host's root shell; port 2222 is a git jail and is never touched. Everything of ours lives in `/root/pardes-test`, nothing outside it. - **Building.** Everything, the C deps included, cross-compiles here: ```sh export PATH=$HOME/.local/bin:$PATH E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P' $E zig build -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/bin/pardes-freebsd-x86_64 $E zig build test-install -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files ``` The `.15.0` matters: Zig's default FreeBSD (14.0) links `libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs the test-build programs instead of running them. A test binary takes no filter when it runs, so running one test alone means a rebuild with `-Dtest-filter=` (and `--prefix ` to keep the full set). - **Deploying.** `-F /dev/null` is required here, because /etc/ssh is owned by nobody: ```sh S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519' zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all scp $S p.zst root@65.109.81.103:/root/pardes-test/bin/pardes.zst ssh $S root@65.109.81.103 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst' ``` The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh ` (a build, PARDES_* stripped, logged there), `deploy.sh [name]` and `bx.sh ''` (a command run there with env.sh sourced). `sshcheck.py` drives a session through `ssh -tt` from a local pty, and `fwd9p.py` times 9P through `ssh -L`. - **The box's layout.** Under `/root/pardes-test`: - `bin/`: pardes, and `python3` linked to python3.12; - `run/`: the runtime directory (sockets, the daemon's lock); - `home/` and `tmp/`; - `src/`: the tracked tree, which the tests need: they read docs/typ, and make their temporary directories in `.zig-cache/tmp` under the working directory; - `test-bin/test/` and `logs/`; - `env.sh`, which points every path pardes uses there (HOME carries the XDG defaults): ```sh T=/root/pardes-test export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color} ``` - **Running.** - A session: `ssh -t $S root@65.109.81.103 '. /root/pardes-test/env.sh; pardes notes.txt'`, and `pardes --attach=` the same way. - The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`. Some tests name `/tmp/pardes-*` outright and leave seven empty directories there (`pardes-arg-save` and the like): rmdir them after. - The e2e scripts, from `src/`: `python3 -B test/daemon.py /root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way. - Kill every process you start there by its pid, and check `ps -axww` afterwards. - **What FreeBSD does differently.** Each item was found on the box, and each is fixed where pardes met it unless it says otherwise: - A write to a terminal that is gone fails with ENXIO, not EIO. std does not expect that errno, and a Debug build traced it to stderr, which is the same dead terminal, recursing until the stack ran out: a session whose ssh dropped hung. Tracing is off on FreeBSD (main.zig). - A signal whose action is to be ignored is dropped when it is sent, sigwait or not, so WINCH never reached the session's watcher. A handler that does nothing keeps it (tty.zig). - waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64 bits. With Linux's numbers a command pane's exit was never seen. - There is no /proc. A shell's directory and program come from sysctl instead (kern.proc.cwd, kern.proc.pathname). - `ttyTaken` answers taken for any foreground group but the shell's, so a nested shell counts as a job. - `signalSession` stays Linux-only, so Kill leaves a command's `&` jobs running there. - File watching uses FreeBSD 15's inotify: Linux's events, with its own IN_CLOEXEC and IN_NONBLOCK. - `shm_open` refuses O_NONBLOCK; its names are not files, so there is no FIFO to fear. A `write` does not grow a shared memory object. - A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a stat by path now comes first. - A Unix socket whose listen queue is full refuses a connect with ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep that judged by connecting could remove a live session's socket while its queue overflowed. A session's liveness is now its lock: it holds `.lock` flocked from before it binds until after it unlinks (9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they unlink only under it. A connect judges only a socket with no lock, one of a pardes from before the locks. The daemon's own start was already decided by its flock. - **The tests there.** The box runs them as root, with no zls, fish or fonts installed. - main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no zls), and two tests that make a file or directory unreadable, which root reads anyway. - panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43 passed. The small programs all pass but fonts (no fonts). - Off Linux, the parts of tests whose refusing path is Linux's /proc/1 are skipped, and so is the builtins JSON check, since the committed lists are Linux builds' (Tty9p included). - test/daemon.py passes there at 20 cycles, its killed stub included (procctl's death signal). One earlier run failed at cycle 3, one process short in its count (most likely ps -e failing to read an environment). - test/fs.py runs past its `pardes -` +Pager checks now, as far as the SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty after the first frame, FreeBSD's pty fills before the session's frame and reset are written, and the session waits in `ttyout` (a binary from before these fixes too). Open. - **At 207 ms, for the replicas.** - In today's model every key is a round trip. Measured through `ssh -tt` (min, median and max): - a key typed into a file was on screen in 213, 229 and 346 ms; - a key typed into a shell pane in 218, 219 and 224 ms. That is one RTT and about 15 ms. - 9P over `ssh -L` of the session's socket costs one RTT a message: 670 ms to connect, 1035 ms for a file read (walk, open, read, read, clunk), 830 ms for a ctl write. A replica cannot fetch its state file by file. - The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L` cannot forward it (tried: EPROTOTYPE here, connect failed there), and no fd would cross anyway. The replication connection has to be a stream of its own, as the 9P socket is, which the daemon hands on locally. - A dropped connection is a hangup: the session detaches, and `pardes --attach` takes it back, as after a Detach. - sun_path is 104 bytes there, not 108. ## Phase 1 notes Built 2026-10-29 (`src/walk.zig`, `src/tape.zig`, the hooks across the core; `pardes-snap --tape`, `pardes-monkey --tape`, `zig build replay`). Decisions taken at the forks, with the ponytail lens: - **The tape is a tree recorded at the host boundary, both ways.** Each call the host makes into the core (`tape.Input`: update, serveFs, pump, perform, the shell's acknowledgements and failures, a Restore...) is recorded with its arguments; each call the core makes out -- a Host function, wrapped generically over the vtable's fields, or a syscall of its own (`tape.site`, which yields around it; every old `turn.yield()` is one) or a read of the clock or environment (`tape.value`) -- with its result. `done` and `ret` close each, so a 9P request served while a step is out lands inside that step's syscall: turn order is tape order. A replica drives the same functions and its host answers from the tape; one that makes a call the recording did not diverges on the spot, before any digest. `turn.yield` panics in a replica, so an untaped syscall is a stack trace. - **The replay is a mode of the same binary** (`PARDES_REPLAY= pardes`), not a harness of its own: an error travels as this build's number, and the code replayed is the code that recorded. Three replicas run one after another: two that draw (the first's memory held while the second runs, so it lies elsewhere) and one that never renders. The headless one steps over the presents it does not make but takes what the host told the core inside them (`acknowledge_panel_presentation`). - **Fields a host writes between calls** (`native_images`, `present_skipped`, `active`...) are found by a diff of a short list at every crossing, not by touching the frontends. - **The digest** is taken after every call the host made at the top (a pump, in the tty), as one per field of the core, its stream and each pane, its file and places, so a divergence names the field. `PARDES_RECORD_EVERY` digests after every input, nested ones too, to find the step one starts at. - **The walker needed a fourth annotation, `len`**: a fixed array written only as far as a length field says (`jumps`/`njumps`, a message's text) is undefined beyond it, 0xAA in a safe build and a fault where it holds a slice. Integers are hashed by value (a `u21`'s padding bits are not its value). - **View geometry** is `File.geometry`, a lazy cache keyed by a hash of what it is made of: revision, length, path, scroll, hscroll, the pane's size, tags and query, tables as far as read, context declarations, the output origin, and the pane's `view` (wrap, layered context, row metrics, rect height), which `sync` sets. A hundred readers a frame ask for it, so the key is one hash of those facts laid out as words (fed field by field it cost 4% of every frame). A step reading it (`wrapAt`, `wrapRow`, `ensureCursorVisible`, a body drag, `zz`) makes it on demand, and the render draws from what the step made: made afresh for every frame as well (to show an input the key missed only as a headless divergence), it cost a scroll a third walk of the rows, 4% of a PageDown. An input the key misses shows as a stale frame in a golden. - **Render wrote more than the census found**, all moved into the step: context declarations, the notice bands and their chips' columns (a click on a chip was routed by the last frame's), the header's caret reveal (`tagnav.golden` moved: after `g l B B` typed in one burst the bar stays scrolled, as a human typing it sees it), the tables' reading (a first look of 4096 lines, a count, wherever it is asked; the rest a frame's taped clock budget a tick), the tty filter palette and body chrome (derived). A view's paint-ahead stays the render's own prefetch: instead of queuing an effect it is asked of the host once the frame is presented (`Stream.askAhead`, in the pump), so a replica that draws nothing asks for none; query rows' paints count apart from it. (Asked inside the render, its worker's thread start cost a held `j` 10% in the live tty measurement.) - **Clock.** `advance`'s wakes are the recording session's (`tape.value(.wake, nextWake)`): a picture decoding or a fade asks a frame, and a replica that draws none must still take the session's ticks. The budgets of Places, Names and Tables are taped verdicts (`tape.over`), as are a message's time of day (TZ included), File.mtime and the pump's clock read (made whether or not it waits). A message's stamp and an unattended session's expiry of it read the clock through the tape too, not the session clock: that is the last pump's, and stamped by it a message written over 9P after an idle spell was as old as the spell and went at once (fs.py's `unattended_messages_expire`). - **A pointer seen through a sliding pane** is taped (`Pardes.pointerCell`): what was under it is the frame the user saw. - **The host's facts**: the session directory (getcwd, asked once at init), `$SHELL` and whether it is there, `$HOME` for `~`, a `Shell` that is not there, a missing interpreter, a directory refused above, a file deleted on disk (`File.deleted` is an input). The rest of the core's getenv (PARDES_DUMP and the XDG dirs in dump.zig, TMPDIR for kitty media, PARDES_TEST_CLOCK in the render) is the host's or authority-only. Per read rather than "captured once into session state": less code, and the same for a replica. - **Authority-only**: worker futures and their Io (a future's presence too), the 9P server's state (`fs`: fids, opens, the log; serveFs itself is replayed in turn order, 9p_io's own writes into `fs` are not), the store file every window of the user shares, which a replica keeps in memory, and Recent read from it. **Local**: the surface, presentation, frame stats, animations, the effect and event queues (drained by every pump before its digest). **Derived**: the geometry, notices, highlights, line index, tables, node trees, rail ticks, a compiled regex, an image's decoded pixels, PDF rasters. - **Terminals** replay from their taped pty bytes (they are `output` events) and are digested as their active screen, cursor and scroll numbers; their kitty images are left out until phase 5. - **Worker results**: the census's render-polled ones (image decode, PDF lanes) are derived state; the rest already arrive as events. - **Unit tests that mutate and render** without a step now step (`p.sync()`) first, as every host does; three of them had passed on the last frame's stale state: a prompt chip a body click had already closed, a `.center` landing measured on no wrap map at all (it centres by lines, so under wrap a line above can now push the cursor down where `ensureCursorVisible` takes the view back: that test is unwrapped now), and a context row count a scroll had not refreshed. The perf gate's `deep-sel` cell was one more: its view, placed from the last frame's map, stopped six lines short of the 2 MB line it selects in. In view, that line showed `File.rawAtDisplay` measuring a line's whole rest before walking to a column (20 ms a frame, Debug); it walks to the column only now, and the cell is the baseline's (91 us against 106). What proved wrong or missing in the design: - The census missed the host writes between calls, the pointer through the presentation, the render-time writes above, `Stream.start` (host code doing core work, made harmless by the futures being authority's), getcwd, `access(2)` for `$SHELL`, File.mtime's clock, and a deadlock in vaxis's queue (a `yes` stopped being read; fixed on this line, found by the new measurement). - `windowed`/`native_images` are watched host writes for now, not View state (phase 4); the semantics hash in the hello is not done (there is no hello in phase 1), so a tape from another build is not refused yet. - Deferred: the GUI's own host-to-core calls (font requests, opacity, `core.advance`/`render` are inputs, the rest not yet), PDF lanes and Handover yields (never reached by the gate; a replica would panic on them, which is the point), `Stream.runHere`'s yield (a host with no workers). The gate on the final line (on main vukknslx): `pardes-snap --tape`, 110/110 scripts, 128 tapes, 384 replica runs, 33,093 digests, no divergence; `pardes-monkey --tape`, 50 seeds of 70 steps, 150 replica runs, 37,800 digests, none; `zig build replay` (the 110 scripts and 20 seeds) clean; `snap` 110/110 with no golden changed but `tagnav`; unit-test tty, pane-test, 9p-io-test and fs-test pass, unit-test gui fails only its two known tests. Perf, base (main with perf --live and the queue fix) against this line, ReleaseFast, interleaved A B A ... A, medians of runs, the base's own spread as the A/A noise. The perf harness's gesture cells (render, keys, PageDown, wheel, edit, deep-sel) 0.97-1.03 at an A/A of 2-7%, and on the final line, measured on the shared perf cores, 0.91-0.97 at 10-31%. The live tty on a quiet machine: core time p50 0.96-0.97, keystroke-to-present p50 0.98-1.03 (A/A 1-23%), `yes | head -c 1G` 0.96, a million-line open 1.01. The live GUI's core p50 1.00-1.04 (A/A 2-18%), present-bound at the window's 30 ms. Two costs the A/B found are gone (the geometry key, the paint-ahead's thread start in the render). ## Tty throughput notes Found and fixed 2026-10-29 (`src/tty/tty.zig`, `waitInput`). - **What the 1 GB `yes` cell measures is mostly the kernel.** A pty's output processing (ONLCR, a `\n` made `\r\n`) costs the writer about 0.4 us a newline here, and `yes` writes one every two bytes. Every reader of a pty is capped by it: `script(1)` drains 100 MB of `yes` at 8.6 MB/s with OPOST on and at 650 MB/s with it off, a Python drain at 8.0 MB/s, and tmux at 5.4 MB/s. The tty pardes was at 5.7 to 6.1 MB/s, three quarters of that ceiling, before any change. The 179 s were the kernel's. - **Where pardes itself was slow:** with 200-byte lines (`yes` of a long word), the ceiling rises to 264 MB/s, tmux reaches 55 MB/s, and pardes managed 18.6 MB/s. A flat profile (perf at paranoid 2 allows no call graphs) put 98.7% of the samples in pardes: `Vaxis.render` 29%, `body_layer.renderBody` 17%, `Shell.present` 10%, memcpy 8%, ghostty's parse 1.5%. The tty drew a frame for every pty read, which is 4 KB, nearly all of them overwritten before anyone could see them (about 690k frames for the 1 GB cell). - **The fix: output is drawn once per frame tick at most** (16 ms), with whatever came in until then, as the GUI drains its queue once per vsync. Two things are drawn at once, as before: - an input's frame; - the focused pane's output within a second of an input (its echo; a remote shell's arrives an RTT later). Neither waits while it is undrawn, including when FrameAck held its frame for the terminal's answer. Snapshot runs (the virtual clock) are not paced. - **And a pane has two reads queued at most** (`readPty`): its reader waits for the loop to take one before it reads on. ghostty parses `yes` at about 9 MB/s, 0.45 ms a 4 KB read, slower than this machine's pty delivers it. So a flood filled the loop's queue (512 events): a key and its echo waited behind its reads, and which blocked writer a free slot woke was luck, which gave half-second peaks. Paced but unbounded, the wait was steady at about 20 reads, 9 ms for the key and 9 more for its echo. A frame dump lined up with each key's write showed it (PARDES_FRAMES and `flood.py` with FRAMES=). What a flood writes beyond two reads now waits in its pty; the loop could not have kept up with it anyway. - **Measured** (ReleaseFast, `.scratch/daemon/term/yes.py` through a real 120x40 pty, all interleaved, with measurements pinned to two physical cores and so four CPUs; medians of 3 rounds, MB/s): | | `yes`, 32 MiB | 200-byte lines, 256 MiB | |---|---|---| | base (before the rework) | 5.5 | 14.5 | | phase 1's tip | 4.8 | 15.0 | | pacing alone (the queue limit came after) | 4.6 | 117 | | tmux | 5.4 | 47.6 | | raw pty drain (Python) | 7.7 | 239 | | GUI, test mode | 0.9 | | Unpinned, before the pinning: 200-byte lines 18.6 to 247 MB/s (1 GB in 53.7 s and in 4.0 s), and `yes` 5.7 to 6.9 MB/s. The GUI could only be measured in its test mode (a hidden window), which downloads every frame from the GPU, so 0.9 MB/s says nothing of a real window. There is no headless compositor here, and a real window must not open on the user's session. In a later interleaved run, the change as committed against pacing alone gave 4.8 and 6.0 against 4.8 and 5.4 MB/s for `yes`, and 102 and 117 against 124 and 119 MB/s for long lines: the same within the noise. `pardes-perf --live`'s `yes` cell drew 2.3k frames for 32 MiB against 11k before, in the same time: it is kernel-bound. - **A held key while another pane floods** (`flood.py`: a key every 30 ms into `cat` in one pane of `-n 3`, timed from its write to the pty to its echo on screen, with pyte in the loop; `yes` in another pane): in one interleaved run of two rounds (p50, p90, worst, in ms): | | under the flood | idle | |---|---|---| | phase 1's tip | 3.0, 212 to 214, 445 to 470 | 2.2 to 2.3 | | pacing alone | 10.3 to 10.6, 19 to 20, 26 to 33 | 2.3 | | this change | 3.3 to 4.2, 4.3 to 5.6, 6.0 to 7.8 | 2.3 to 3.2 | The driver's own cost (pyte) is in every number. With 16 KiB queued a pane instead of two reads, the flood's p50 was 7.5 to 7.8 ms for no throughput gained, so two reads it is. - **Keystroke-to-present** (`pardes-perf-live --live`, base and this change interleaved, 5 and 4 runs; A/A between the base runs' halves): p50 against base is 0.92 for key-down, 0.89 key-right, 0.85 edit-char, 0.80 page-down and 0.93 wheel. The A/A was 0.73 to 0.92. The 32 MiB `yes` took 1.00 the time with 0.18 the frames, and the big open 1.10 (A/A 1.05). Two earlier runs of pacing alone, 3 and 5 rounds, were within their A/A too. - **Unrelated, found on the way:** `test/fs.py` failed `unattended_messages_expire` (its first `shown()`) on phase 1's tip as well; pmxpuolt on main (a message's stamp and expiry read the clock again) is that fix. fs.py also needs a short TMPDIR: under `.scratch/daemon/term/tmp`, `directory_panes`'s socket path is 108 bytes, one past the limit, and the session starts with no 9P. Two GUI unit tests fail on phase 1's tip as well (`tagline pointer mapping ...`, `a tag caret past its row's end ...`). A suite started as a shell's `&` job runs with SIGINT ignored, so four host_io tests whose Ctrl-C must stop a `sleep` fail there; started otherwise, they pass. ## Screen-state terminal notes Built 2026-10-29 (`src/screen_state.zig`), a pure encode and apply pair over byte buffers for the phase 5 stream; no transport yet. Only tests reference it, so the tty and GUI binaries carry none of it: no symbol of it is linked into the tty built from this tree. The local path's A/B, base against that tty, is the Tty throughput notes' keystroke-to-present. - **Both kinds of blob are ghostty's own snapshot records,** not a cell format of ours. The terminal-wide state (modes, colours and palette, title, pwd, tab stops, mouse modes, scrolling region) is a TERMINAL record. A row is the PAGE record of a one-row page the row is cloned into, which carries its styles, graphemes, hyperlinks, wide cells, wrap and prompt marks exactly. The replica decodes them with ghostty's decoders and writes a row with `Screen.clonePartialRowGrowCapacity`. - **A row's address is a hash of what it shows** (Wyhash of its flags, its width and its cells by value), not of its blob: ghostty keeps the source page's style ids when it clones a row, so equal rows give blobs that differ. The width is in the address, because a blank row is the same at every width. - **A delta** is a header (cursor, kitty keyboard flags, the active area's first row as an absolute row, the history's extent), the TERMINAL record's address, an address for each active row, an address for each row that scrolled off since the acknowledged state (the tail, up to a screen of them), and the blobs for the addresses the peer does not hold. An unchanged row costs 8 bytes. - **One delta per peer is in flight,** against the state it acknowledged, as FrameAck keeps one frame in flight for the tty. A slow peer skips the states in between. The peer holds exactly the blobs of the state it acknowledged, at most two screens of rows and the meta, so the authority knows what it holds without asking. At 207 ms this means at most about 5 deltas a second to the FreeBSD box; phase 6's predicted echo is meant to cover typing there. - **Absolute rows** come from a tracked pin at the last active top. The rows scrolled since are the distance to the new top; if the pin's page was pruned, the count jumps past every row the replica could hold. A resize renumbers (an epoch), and the peer starts afresh. Scrollback is fetched by absolute row (`encodeRows`, `applyRows`). - **The replica writes into a ghostty Terminal,** a terminal pane's own, so `src/terminal.zig` draws it unchanged. A test renders a replica's terminal pane and its authority's in two cores: the bodies are equal, cell for cell. - **Gates.** - The soak drives an authority with pty bytes and lands deltas at random, both skipping states and delaying a delta while the authority moves on. Resizes and fetches of holes also come at random. After each delta it checks the replica's screen against the authority's as encoded, and every history row the replica holds outside its holes against the authority's row at that number. - It passed with synthetic shell, TUI and `yes` streams at 4 seeds (in the unit tests, with a 64 KB scrollback so it prunes). It also passed with 197 recorded streams at 3 seeds each (the test that PARDES_SCREEN_STATE= turns on): the 192 pane outputs of the snapshot scripts' tapes, plus real vim, nvim, htop, less and btop sessions recorded through a pty. Its corpus and recorder are in `.scratch/daemon/term`. - **Numbers** (ReleaseFast: `PARDES_SCREEN_STATE= zig build core-test -Dplatform=tty -Doptimize=ReleaseFast -Dtest-filter=screen_state`, which runs under std.testing's checking allocator): - Encode, 200x60, every row new: p50 250 us, p99 487 us, 29.8 KB. - Encode, 200x60, nothing new: p50 71 us, p99 139 us, 534 bytes. That is the 60 rows hashed and the TERMINAL record encoded to hash it. - Under `yes` as fast as ghostty takes it in-process (9 to 12 MB/s), with a delta every 16 ms: 62 deltas a second, 63 KB/s on the wire, 1 KB a delta (60 rows and the tail's 60, all held). That is bounded by the frame rate, not the output. - Under varied text at 36 to 55 MB/s: 355 KB/s on the wire, 5.6 KB a delta. - The replica applies a `yes` delta in 2.4 ms, most of it pushing the rows scrolled past into its history (a row at a time). A delta of varied text applies in 0.24 ms under the C allocator and 0.95 ms under the checking one. - **What proved wrong in the design.** - "Scrollback fetched on demand" alone loses rows. A row that changed after the last delta and then scrolled off would sit stale in the replica's history. The tail fixes the rows that scrolled within a screen. Older ones, which the replica never saw or saw before they last changed, are holes: rows it must fetch before it shows them. - The address had to include the row's width, as above. The soak found this: a replica reused an 80-column blank row after a resize to 10. - ghostty narrows pages lazily, so an active row can be wider than its terminal: only the terminal's columns are hashed and encoded. - **Not done, or a gap.** - Kitty images in terminals. - pardes's own state derived from the stream is not carried: its `MarkedHandler` prompt phase, command-output pins and the replies it writes to the pty. ghostty's prompt marks on rows do cross. - The view's scroll offset is per client and is not sent. - A replica's core must not resize its pane's terminal itself; the authority sizes it. - The replica's history extent follows its own pruning under the authority's limit, so it can be a page off, and its oldest rows can be ones the authority has dropped. - A fetch sends every row as a blob (phase 6's cache will send addresses). No zstd. - A replica pushes a flood's rows into its history one at a time. - The integration into the stream and the cores (a replica pane's State fed by `apply` instead of `.output`) is the next piece. ## Phase 5 groundwork notes Built 2026-10-29: the delay relay (`test/delay_relay.py`), the replica socket (`src/daemon.zig`, `src/detached/wire.zig`), and two measurements for open points 3 and 5 (`zig build rollback-bench`, which runs `test/rollback_bench.zig`, and `test/fork_stall.py`). Decisions taken at the forks, with the ponytail lens: - **The delay relay is a Python script.** Each direction is a reader that stamps every chunk with a due time and a writer that sends it then. The due time is the chunk's arrival plus the delay. With a rate cap, it is the moment the link has carried what came before, plus the delay. So throughput is not serialised behind the delay, and the bytes stay in order. Either side is a Unix or TCP socket. ponytail: its buffer is unbounded, which makes a link with an infinite bandwidth-delay product; on a real link, ssh's window is the bound. - **Two sockets, where the design had "the daemon's one".** The fd-carrying SEQPACKET socket stays local. Once the daemon holds its lock, it binds `pardes-replica.sock` (SOCK_STREAM) beside it, and unlinks it at exit. `ssh -L` forwards that one. - **The daemon judges the hello, so a session never sees one it refuses.** - Every session's registration carries its semantics hash (daemon protocol 2), and the daemon compares the client's with the session's. - It refuses in words: another protocol; another build, naming both hashes and pointing at `pardes --version` to compare them; or a session nobody has. - It reads exactly the hello's length (the header, then the payload), so whatever the client sends after it is left for the session. - Up to 16 hellos may be on their way at once. ponytail: a client that never sends one holds its slot until it hangs up. - **The welcome comes from the session**, with its pid and name, over the connection it was handed. That tells the client the handoff happened. The test checks that the session's fd count went up and the daemon's did not. - **The semantics hash** (`wire.semantics`) is computed at comptime and covers: - the protocol, the release and the commit; - the platform and the build options; - every dependency the manifest pins, by its content hash (build.zig's `deps` option). The design said grammars, but ghostty and the Unicode tables also make a core step differently, so it takes all of them; - the walker codec's assumptions (pointer width, byte order). It leaves out the OS. The main test is a Linux client of a FreeBSD session, and what differs between those is the host's, which the tape carries. The two builds of one tree hash alike (1087257bf166588a on both), and the box's session welcomed the client from here. ponytail: an unstamped build (Debug with `-Dplatform`, so every test build) has no commit, so two dev trees with one release and the same options hash alike. - **The hello lost its grid**: a replica's view takes a resize like any other input. A ping (client tag 0x03) and three server tags (welcome, refused, pong) are new. The wire is version 14. - **The session's half is a stub**: a thread per replica that sends the welcome, answers pings, and ends at bye or a hangup. Measured. Timings ran alone through the perf lock, pinned to cores 6-7 while builds ran niced on the others. The local A/B ran interleaved, with an A/A run beside it. - **The relay's self-check** added this much over a direct connection, at each delay (median RTT, then throughput through it): | one-way delay | wanted | added | throughput | |---|---|---|---| | 15 ms | 30 ms | 30.26 ms | 3.2 GB/s | | 40 ms | 80 ms | 80.27 ms | 2.3 GB/s | | 103.5 ms | 207 ms | 207.26 ms | 2.3 GB/s | Direct, the same pair gave 9.9 GB/s. The extra 0.13 ms each way is the threads' wake-ups. A 20 MB/s cap held at 19.98 MB/s. - **A replica's round trips.** Each row is a connect and hello, then 50 pings, to a detached session of this build. The session's thread answers each ping. | path | hello | ping min / median / p90 | |---|---|---| | direct (Unix socket) | 0.26 ms | 0.01 / 0.01 / 0.01 ms | | relay, 30 ms RTT | 30.95 ms | 30.26 / 30.34 / 30.40 ms | | relay, 80 ms RTT | 80.73 ms | 80.25 / 80.31 / 80.39 ms | | relay, 207 ms RTT | 207.62 ms | 207.22 / 207.35 / 207.41 ms | | FreeBSD box, `ssh -L` | 411 to 476 ms | 203.2 / 204.9 to 210.4 / 210 to 221 ms | The box's rows cover six runs over two passes. In the same passes, a TCP connect to the box's port 22 took 203.8 to 205.3 ms at best and 206.4 to 208.3 ms at the median. The relay rows are the second pass; the first pass had p90s up to 5 ms worse. So a ping through `ssh -L` and the daemon's handoff costs the RTT and nothing more measurable. The hello costs two RTTs: ssh confirms the channel before it forwards a byte. A refused hello over the link came back in 412 ms, also two RTTs. That is the floor phase 5 builds on. A replica connects once, so it pays the second RTT once. - **Open point 3, rollback: the walker copy.** `zig build rollback-bench -Doptimize=ReleaseFast` builds two sessions in process: - typical: a shell with 1 MB of scrollback, three 60 KB files and a 3 MB one, 40 edits; - big: 14 panes, an 18 MB file, four 3 MB files and eight small ones, 8 MB of scrollback, 200 edits. For each, it measures the state the digest walk visits (what speculation can write) and the share of it that is blobs. Blobs are a file's text and its undo snapshots. `File.setContentSpan` puts a new buffer in place of the old one at every edit, and a snapshot is a copy, so both are immutable once made and a copy can pin them. The walk is then timed again with them taken out. | session | walked | of it blobs | left to copy | walk | walk, blobs out | copy (walk + memcpy) | |---|---|---|---|---|---|---| | typical | 36.3 MB | 36.2 MB | 80 KB | 3.1 to 3.8 ms | 0.019 to 0.023 ms | 0.020 to 0.024 ms | | big | 282.5 MB | 282.4 MB | 96 KB | 23 to 27 ms | 0.043 to 0.044 ms | 0.044 to 0.045 ms | A copy is about 0.1 to 0.3% of a 16.7 ms frame, so the walker wins: arena copy-on-write would buy nothing. Three conditions come with it: - a blob a copy holds must not be freed until the copy is let go, whether by a refcount or by a free list drained at confirmation; - caches derived from a blob (`line_starts`, highlights) are also made anew at each edit, so they can be pinned with it, or remade on demand, since the geometry is keyed by revision; - a terminal's history is not in the walk, because under screen-state terminals it is the authority's pinned rows. The walk with the blobs out is the stand-in for a copy walk: phase 2's dump walk is the real one, and this step re-measures it when it lands. - **Open point 5, the join writer: not a fork.** The authority's stall, as median fork time, with 64 MB written by the authority while the child lives: | written memory | Linux 7.2 (THP always) | FreeBSD 15.1 | |---|---|---| | 100 MB | 1.8 to 2.7 ms | 4.1 ms | | 600 MB | 8.6 to 10.8 ms | 14.0 ms | | 1 GB | 13.3 to 19.6 ms (max 24) | 20.4 ms (max 26) | | 4 GB (one Linux pass) | 62 ms (4 KiB pages: 162) | 52 ms | | then 64 MB written under the child | 26 to 32 ms | 19 to 27 ms | Linux is two passes on the pinned cores, beside the builds; the box was idle. The big session forked in process at 431 MB RSS took 3.4 and 8.3 ms. The user's windows hold 450 and 630 MB. The gate, "a join costs the authority under one frame", fails twice: - fork itself, at 1 GB on both systems (and at 600 MB once a frame is 8.3 ms, at 120 Hz); - copy-on-write after it. Every page the authority writes while the child streams costs a fault, about 1.3 µs per 4 KiB page. The child lives as long as the dump takes to cross, which is seconds at 207 ms. So the join is the same walker copy as rollback: tens of microseconds at a step boundary, blobs pinned. A worker thread then serialises the copy and streams the pinned blobs, which are immutable and so need no lock, while the authority buffers the events after it for the joiner. This is the coordinator's alternative, "a metadata copy serialised on a worker". The authority pays the copy, and the blobs stay pinned while they stream. - **No local change.** tty startup and keys, the phase 1 tip (a4b85cb7, ReleaseFast) against this, both live at once with their samples alternating; the p50 ratio new/base: | | three A/B runs | two A/A runs | |---|---|---| | startup | 1.040, 0.965, 1.024 | 1.053, 0.978 | | key in a file, to its frame | 1.084, 1.025, 1.027 | 1.025, 1.013 | | key echoed by bash, to its frame | 0.996, 0.972, 1.010 | 1.015, 0.998 | The daemon and the hello are off the hot path, and every ratio is within the spread of the A/A runs beside it, whose second slot reads 1 to 5% slow against the same binary. So there is no measurable change. The absolute times are not comparable with phase 3's 4.73 ms: the startup p50 was 6.5 to 26 ms here, on two cores shared by both live sessions and the driver, with builds running on the others. What proved wrong in the design: - "`ssh -L` of the daemon's one socket": there are two. The one that carries fds cannot cross ssh, and no fd would cross anyway. - "The authority forks a writer child": fork's page-table copy is over a frame at 1 GB on both systems, and the copy-on-write tax lasts as long as the join streams. Once blobs are pinned, a copy of what speculation can write is about 100 KB, small enough for a join to take one too. - "When the first speculative event goes pending, the replica copies views, buffers, ...": a buffer's text never needs copying, since every edit replaces it. Only its handle and the views are copied. - "The hello carries a semantics hash (version, features, grammars)": it hashes every dependency, and leaves the OS out on purpose. - Pre-existing on the phase 1 tip, and not this line's: test/fs.py's `mounted-startup` session hangs at startup (its main thread waits on a futex), the same with the base binary, and still on main after phase 4 (Phase 5 part A notes). A ReleaseFast `exit_answers` run loses its Rclunk race about 1 time in 7 on base, and 1 in 40 here. ## Phase 5 part A notes Built 2026-10-29, in one process and with no transport: the rollback copy (`src/rollback.zig`), the replica engine (`src/replica.zig`), the tape's speculation mode (`src/tape.zig`), and an in-process harness (`test/replicas.zig`, `zig build replicas`). `zig build rollback-bench` now times the real copy. Decisions taken at the forks, with the ponytail lens: - **The copy saves bytes and puts them back where they were.** Its walk is the digest's (src/walk.zig): views, buffers, registers, counters and terminal metadata. It saves the bytes of every allocation the walk reaches, and puts each back over its allocation. That works because the replica core's allocator (`rollback.Epoch`) changes three things while a copy is held: - a free of anything older than the copy is put off; - growing such an allocation in place is refused, so the core moves it; - anything allocated meanwhile belongs to the epoch. So every pointer in the saved bytes still points at what it pointed at when the copy was taken. Putting the copy back frees the epoch's allocations and forgets the put-off frees. Letting the copy go makes them. Nothing has to know who owns what. - **Three blobs are pinned instead of copied** (4 KiB and up): a file's text, its line index and its undo snapshots. An edit puts a new one in place of the old (File.setContentSpan) and never writes one, so the copy keeps a reference and the put-off free keeps it alive. A safe build checks every pin is unchanged when the copy goes. - **More than the digest's fields are saved.** Within a struct the copy saves every field: the derived caches, and this process's own state too. It skips the queues a step drains (asserted empty) and the frame statistics, and cuts an array at its `len`. Both were measured wrong first: - a safe build's `free` writes 0xAA over what it frees, so a cache the speculation freed came back as that (a line index full of 0xAA, then a crash); - this process's own fields (the frame arena, the surface) point at what the epoch then frees. For the same reason, file texts and line indexes are now freed raw everywhere (`memory.freeRaw`), as setContentSpan already did. - **A speculation is checked against the authority's tape of the same input.** The replica applies its own input as the authority will (an update, then a pump) and records it with its own tape. If the authority's unit for that input has the same records (inputs, host calls and their answers, digests aside), the head goes: that is the fast path. The digests are left out of the comparison because the starting states are equal by induction. A digest checks every unit a replica replays. - **Each client has its own view** (src/views.zig). The authority goes into the client's view before its input (`toView`, which is taped), and the replica's speculation does the same. So a unit records the switch only when another view was live, alike on both sides. The views not live (`parked`, each pane's `pv`) are this process's own fields, and the copy saves them with the rest. - **A host call or a syscall stops a speculation, but cannot stop it midway.** Zig has no unwinding. The call is answered with a harmless value (an error of its set, null, zeroes, or a union's first zeroable case), the input runs to its end, and then it is undone: the copy is put back and the earlier pending inputs applied again. That input, and those after it, wait for the authority's answer. A read of the clock or the environment (`tape.value`) is a guess made locally, which the comparison checks. `watched` is answered false, as the authority's host says too: a replica draws outside the step. - **An emulator is the authority's.** Its writers in terminal.zig (feed, resize, scroll, follow, create, deinit) stop a speculation before they write. A safe build asserts that a clean speculation left every terminal's digest as it was. Tree-sitter's trees live on the C heap, which the epoch cannot see. While a copy is held, `syntax.held_trees` puts off their frees and frees the epoch's own. - **The view's events wait** (a resize, the pointer): phase 4. - **Messages are bytes.** A client's input is a u16 client id, a u32 sequence number, and the event as detached/wire.zig frames it. A unit is the input's origin (0 for the authority's own: a pty's output, a setup), its sequence number, and the authority's tape of it. - **The harness** is a discrete-event simulation. One authority and three replicas share one view, each client typing monkey.zig's mix (keys, typing, pastes, a few commands, resizes and clicks) at 50-200 ms with pauses. Each link takes half the RTT plus up to 10% of it in jitter, in order. The session is the tty's own boot (a raw shell over a text pane) with three files looked at. The authority's pty echoes what it is written 2 ms later, which makes units of its own. Found on the way: phase 1's replay ran Debug, whose 0xAA fill made every read of unwritten memory alike in two cores. In a release build they read what the heap had, and replicas diverged at once, from: - `col_panes`, digested whole and undefined past each column's panes (now zeroed); - `restore_buf`, the same (now zeroed); - `terminal.State.reply_dropped`, never set, since `create` leaves fields as the heap had them; - `Text.sels` and `PendingPipe.ranges`, digested past their length (a `len` now cuts a slice too). `PARDES_REPLICA_FILL=0xff` fills a replica's allocations, so that what is left shows at once. Any replica is a release build, so these mattered. Measured on main with views (lyntonns). The ReleaseFast runs were alone through the perf lock; the correctness sweeps ran beside builds. - **Correctness.** Three clients each typed 100 inputs per seed, at RTTs of 30, 80 and 207 ms. No replica diverged, and every replica ended on the authority's digest, in all three runs: - Debug: 12 seeds, a view each; - ReleaseFast with `PARDES_REPLICA_FILL=0xff`: 20 seeds, a view each; - ReleaseFast, timed: 20 seeds, a view each and in one shared view. - **What happened to the inputs, and what it cost** (ReleaseFast, a view each; times in µs, p50 / p99): | RTT | rollbacks per input (mispredicted + another's) | speculative apply | copy (1.3 MB) | rollback | |---|---|---|---|---| | 30 ms | 31% (106 + 1745) | 35 / 183 | 75 / 198 | 342 / 818 | | 80 ms | 76% (257 + 4285) | 54 / 168 | 94 / 447 | 455 / 1318 | | 207 ms | 129% (423 + 7335) | 50 / 174 | 91 / 373 | 569 / 1561 | - Of 6000 inputs a run, 259 to 348 stopped at a host call and waited, and 434 to 449 were the view's own (resizes and clicks). - In one shared view, the copy is half the size (0.66 MB), the costs are half (copy 37 / 152, rollback 184 to 259 at p50), and the rollbacks are the same: 30%, 70% and 121%. - The worst rollback, 1.6 ms, is a tenth of a 16.7 ms frame. A speculative apply is 0.2 ms at worst. - **The copy alone** (`rollback-bench`, ReleaseFast, single view): | session | saved | pinned | take | put back after an edit | |---|---|---|---|---| | typical (5 panes) | 980 KB | 36.6 MB | 0.044 ms | 0.036 ms | | big (14 panes) | 1.49 MB | 286.5 MB | 0.071 ms | 0.051 ms | Letting a copy go costs nothing measurable. What proved wrong: - "Views cut the conflicts": they cut what a rollback shows, not how often one happens. Any unit of another client's, in any view, comes before this replica's pending inputs in the authority's order, so the copy goes back and the pending inputs are applied again. Each costs a third of a millisecond, a tenth of a frame at worst. Telling which units commute with which pending inputs is the next lever, not a needed one. - "A copy of about 100 KB": it is 1 to 1.5 MB. The core struct itself is most of it (the presentation, the surface, the settings and the column tags), plus the views' parked state, now that the copy saves all of a struct and not only what a digest takes. It is still under 0.1 ms. - "A misprediction is rare": every key that leaves a message stamps it with the time of day (`HH:MM:SS`). So a key confirmed in another second than it was applied is recorded differently: 2 to 7% of inputs here. In the harness both cores share one clock, so this is the floor. Across machines an offset between the clocks raises it. Part B should guess the authority's clock (its last confirmed reading, plus the time since locally), not read its own. - views.zig left a pane's slot for the live view unwritten, and a slot holding the live body's selections. A pane closed before that view parked again freed whatever its slot held. This is fixed in its own change, before this one. Every replica switches views on every input, and found it at once. Not done, and why: - No transport: part B. - The view's events (a resize, a click) still wait for the authority. With views they could be applied at once. - A pty write stops a speculation, so a shell's echo is shown late. Phase 6 predicts the echo. - The authority's own units (a shell's echo) roll back a replica that has pending inputs. - The epoch allocator is single-threaded. A replica core's workers (images, search) must allocate elsewhere, or hold a lock, in part B. - A replica keeps a timing sample for every input it applies (the harness reads them). A long-lived one keeps only counters. - test/fs.py's `mounted-startup` still hangs on main (lyntonns). The same hang shows with main's own binary in a workspace of its own, so it is not this line's. ## Phase 2 notes Built 2026-10-29 (`src/walk.zig` Dump/Load, `src/dump.zig`, terminal.zig's dump hooks; `pardes-snap --dump`, `pardes-monkey --dump`, and both in `zig build replay`). Decisions taken at the forks, with the ponytail lens: - **The dump is the digest's preimage.** It walks exactly what the digest walks (local, derived and authority fields stay out), so `digest(load( dump(x))) == digest(x)` is the definition of complete, and the gate. The dump is `magic`, a table of every type the roots' walk reaches (each struct's and union's fields by name with their types' places, each enum's names and values, each number's width), then the values as the tape lays them out but for an enum or union tag by its place in the table, and a pointer or slice met again written as a reference to the first. The table is a comptime graph of the roots: a session's is 324 types in 23 KB, of a fresh session's 96 KB dump. - **Crossing builds: by name, and positional when it need not be.** A table byte-identical to this build's is read field after field, no name asked (the join's same-build path: no separate strict format). Any other is matched by name: a field added takes its default, one removed is passed over, an enum value by its name (Builtin's ordinals shift with every builtin added), numbers widened or narrowed by value, arrays as far as they fit, one that came to hold a `len` read either way. A field whose kind changed takes its default and is said; with no default the value it is in is refused, and an element of an array of optional pointers (the session's panes) is then left out with its place in the layout (`dropMissing`), never more. What was let go is the session's first message after the load. A pane whose terminal another build could not read is a fresh terminal (`ensureKind`). - **Roots, not wrappers.** A session's dump is five roots: the core, read *into* a core `Pardes.create` made (its local, derived and authority fields stay create's: no defaults to invent for an arena or a host), its 9P mounts (state the user made, inside the 9P server's authority `fs`), which file panes' files were on disk when it was made (one gone by the load is said deleted, so Del asks: its text is nowhere else; one never on disk comes back as it was), the theme's name, and the views not live. A pane's dump (the store's, a handover's) is a Pane alone. - **Every view is in the dump** (decided after phase 4 merged). The live view is the core's own fields, so `view` and `views_open` were dumped already, and a session dumped with several views loaded into a core that said so with no parked slot: its first view switch, close or drawing of the others unwrapped a null. Leaving them out would have needed `load` to fold them anyway, and the join (phase 5) needs them: a replica applies the other clients' `toView`. So `views.dumped` writes each open view but the live one (the live one's slot is stale by design): its share of the session but what drawing keeps (`Pardes.replica.local`, computed from `views.fields`: its panels as presented, its animations), its slots' serials, and its share of every pane (`pv_known` says which shares own their selections, not which exist: a parked view has one of every pane). `views.undumped` parks them again in `load`, last, so nothing after it rolls back what it moved; a pane a view has no share of gets the live view's, a view the core says open that the dump lacks is closed, a dump with no views root (before this) has one view. They stay out of the digest as before (parked views are checked when live again); the gate is the re-dump byte for byte, now also on the three-view monkey seeds. A session *revived* for one client (Restore, `-l`) is one view, the live one as view 0 (`views.only`, what the hosts already assumed), and a pane carried or reopened takes the view it lands in (`sized_by`, `prompt_view`). Cost: the views root is empty for one view; with several, a share is about 100 bytes a pane a view. - **Aliasing by reference.** Zerox twins share their text and undo again, an `inherited` cwd points at its pane again. Keyed by address, length and type, which is also what made twins dump their text once. - **Terminals are ghostty 1.4's snapshot** (`terminal.State.dumpWrite`): both screens, history, styles, links, and the kitty images with their pixels and placements (the known gap in test/kitty.zig closed), plus a walk dump of pardes's own parts (unsent replies, the view, the OSC 133 command cycle, its output pins as screen points). The snapshot is at the ground state: a sequence the program had begun at the dump's moment is lost (ponytail: the stream would have to track its continuation, a cost on every pty read). ghostty calls its format a work in progress: a ghostty bump that changes it makes a terminal another build cannot read, which degrades as above. - **Clean files' text is in the dump.** A join cannot read the authority's disk, and the session is what it holds. A file changed on disk since comes back as the watch a load emits finds it (as on open: a clean pane reloads, an unsaved one is marked changed), not by a read in the load. The store keeps a closed pane's record under 8 MiB by a slim form (no terminal history, no undo, a clean file's text read again on reopen), as it kept terminals without history before. - **Load and revive are two halves.** `load` makes the core and nothing else (what the gate checks, and what a replica takes); `revivePane` and `reviveSession` give it to a host: a new shell under each terminal's history (alternate screen left, keys, mouse and paste modes reset, the banner; the old shell's REPL, Tty+ word and sent command dropped; a command pane finished), watches, PDFs opened again (MuPDF's document is derived: `Document.handle` is optional now), Recent and the store, the ThemeFile asked for again, the startup config. The screen stays the dump's until the host resizes: setting this process's `opts` size first reflowed twice and lost where a terminal's view was. - **`opts` is local** (how this process was started: allocators, socket names, store path), and so are `dump_out` and the effect and input queues' counters (the queues were local, their heads and lengths were walked: a loaded core read a garbage effect). Pipe's runs, an Edit's, Handover's dump in flight and its carrier, and the host's fallback file table are authority: host errands, not state. - **Restore keeps serials.** Renumbering would rewrite every serial the session holds (attachments, query owners, places, messages, LSP and callstack requests); Restore hangs up every connection anyway. The next serials are past the old session's; `restored`/`restoredcol` records are gone. A Restore over an older session keeps the host's facts (cell pixels, row metrics, native images) and the higher of the request sequences. - **A failed load leaves nothing**: the loader records each allocation it made and frees them all (`rollback`), and a `dumpFree` for a custom type's insides. ghostty's snapshot decode itself leaks when an allocation inside it fails: the OOM Restore test runs without a terminal. - **Handover** dumps `Carried` (the kind, a column's tag, each pane's dump, weights, owners) behind `pardes-carried\n`; a new window's `-l` of it is an empty core that adopts the panes, its settings this process's (the old carried dump brought the source window's theme and settings along). Panes carried now keep their undo. - **No v1 reader.** A ZON dump is refused before anything is replaced (`not a pardes dump: it does not begin as one`); a ZON record in the store is an old build's session's, which no new one lists (below). Dumps are `.dump` files now. What proved wrong or missing in the design: - The walker's list rule (exactly `items` and `capacity`) never matched in a safe build: Zig 0.17's ArrayList has a `pointer_stability` lock there, so phase 1's digest hashed every list's capacity and a dump wrote it. Both take a list's items now (`walk.isList`). - Merged onto phase 5 part A (sqsnvoux): a `len` may cut a slice now (a text's room for its ranges, made whole and used `nsel` deep). The dump writes such a slice whole, room and all, and a load gives it the room it had; only one of plain numbers may be cut so (every bit pattern a value past the cut), else no build. - Phase 4's other views lost their selections whenever they were drawn: `unpark` hands a slot's selections to the body (main's fix for a double free) and `drawOthers` never gave them back, so a drawing session took them from every view not live while a headless replica kept them. The three-view monkey seeds with the headless replay found it (seeds 5 and 35 of 50 diverged at the next key typed at several cursors); `drawOthers` gives them back after each frame. - "pty masters passed as fds where Handover passes them today": Handover passes none (it refuses terminals: their shells are the window's children), and phase 3 made Detach happen in place, so no dump carries a master. A loaded terminal gets a new shell; the daemon's Adopt is the way back to the old ones. - There are no crash dumps: crash.zig records the panic message only. None was added: a panic handler would walk state that may be what broke and allocate inside the handler. The path that exists is Dump, rebuild, Restore, which the by-name load is for (a session read by name is a unit test over the whole type graph). - Two behaviours of the old dump were its approximations, and goldens moved to the real session: a terminal's view comes back where it was (load.snap's old view was not the dumped one), its prompt rows stay hidden in text mode (OSC 133 marks are in the snapshot), its Filter, a pane's message and typed text over its history are kept, and a query pane comes back a live query, not plain text. - The web shell had not built on main for a while (u64 counts on a 32-bit target, threads, stb_image and the tape's host calls). It builds again and loads a native build's dump by name: `-Ddump=` embeds test/web-snapshots/touch.dump, made by tools/web_fixture.py, in place of the hand-written ZON fixture. A field this build compiled out (void on wasm) is passed over, and the theme goes by its name, a fourth session root, since the theme table differs between builds. web-snap's touch check passes; its drift was the test's (13 tag-layer fields, helix's retuned colours, and a pane resized by its grip, there being no rule between panes to drag). The web shell's own `chromeTarget` still calls an upper pane's last row a handle, which the mouse code no longer resizes on. - Not done: the website's replay (pardes-website/replays/00) still runs its July wasm; the dump it was built from is gone. `node --test test/web_driver_test.mjs` fails "CDP closes a stalled connection after its deadline" every time, a test of the harness's CDP client, not of pardes. A dump is read whole, not decoded as it arrives (the join can stream it; decoding as it arrives needs a reader over a stream, not a slice). Native byte order: every host pardes runs on is little-endian. Upgrading from a build before phase 2, a user loses: - A ZON dump: Restore and `-l` refuse it, the session they are in unchanged. A pane carried between an old window and a new one stays where it was (a failed handover). - Nothing in the store. Its framing and records are as they were; only a closed pane's dump inside its record changed. A closed pane is listed only by the session that closed it (a session's id is new each process), so a new build never lists, opens or reports an old build's records, and an old window still running brings back its own. Both builds' compactions keep the records of every running session and drop those of sessions that ended, as before. Checked on a copy of the user's real store (14.4 MB: Recent's 234 paths, 140 ZON records of 8 sessions, three of them running), the copy under the workspace's scratch, the real one never opened: a new build starts on it with no error or message, /recent lists the same 200 entries as the base build's on the same copy, Panes shows none of the old records, a pane closed and brought back comes back. A compaction forced by a record cut short (as a writer that died leaves one) gives the same file in both builds but for each one's own records: the running sessions' 72 records byte for byte, Recent folded to the same 200, the ended sessions' 68 dropped. The base build then runs on a store the new one wrote, with its walker records in it, without an error. One cost moved: a closed small pane's record is 14.5 KB, not 1.5 KB, most of it the type table each self-describing dump carries (ponytail: bounded by 64 a session and the compaction; a table once per store generation if it matters). The gates on the line rebased onto main with views and phase 5 part A (sqsnvoux). `zig build replay -Dreplay-seeds=50`, with `--dump` in all three parts: the 113 scripts (132 sessions, 29,430 digests over their first, second and headless replicas), 50 monkey seeds of 70 steps (37,722 digests) and 50 at three views (29,148), no divergence, and every session's dump at its end loaded to its digest (all 232, field by field) and dumped again byte for byte (14.4, 5.4 and 6.7 MB of dumps). The snapshots pass with five goldens moved (above). `zig build replicas -- --seeds 12` (Debug, three clients, 30/80/207 ms): none diverged or unequal. unit-test tty passes; unit-test gui fails only phase 1's two known tests; test/fs.py and test/daemon.py (20 cycles) pass; the web shell builds and its touch check passes. The cross-build check is walk.zig's test of two hand-written versions of a struct (a field added, one removed, one renamed, enums renumbered, a width changed, a pane whose required field changed kind left out), both ways, and dump.zig's of a real session read by name. The real-store check (above) holds on this line. Measured (ReleaseFast, the base binaries from before the rework against this line, interleaved A B A ... A on the perf cores, medians, the base's own spread as the A/A): the perf harness's core cells 0.92-0.99 of base (p90 0.80-1.03) at an A/A of 15-70%. A session of 16 small files, a 100 MB file and a terminal with `seq 1 3000000` of scrollback, through 9P in a real pty: | | base | new | A/A | |---|---|---|---| | Dump, the big file clean | 8.8 ms, 0.2 MB | 83 ms, 105 MB | 21% | | Restore of it | 69 ms | 105 ms | 4% | | Dump, the big file unsaved | 514 ms, 140 MB | 191 ms, 210 MB | 1% | | Restore of it | 1845 ms | 310 ms | 0% | | a closed pane kept: the terminal | 131 ms | 143 ms | 4% | | the 100 MB file | 433 ms | 127 ms | 3% | | a small file, after the 100 MB one | 9.6 ms | 17 ms | 44% | | a small file, alone | 1.0 ms | 1.0 ms | 7% | A clean file's text is in the dump now (what makes it complete), and an unsaved one's undo (the 100 MB file's one edit keeps the text before it); binary, not base64 in ZON, so the rest is faster. The small close after the big one is not the small pane's: it is the kernel freeing the 100 MB pane's memory on the 9P connection's thread (panes are freed two syncs after they close), 208 MB against the base's 108, since the restored unsaved file's undo holds a copy of its text (found with the access tracer, perf and schedstat, .scratch/daemon/trace). A small close alone is level (three after two others: 1.0 ms both; an isolated one, median of ten: base 0.77 ms, new 0.92). Of the 0.15 ms left, two thirds were outside the walk and are gone: savePane kept the whole 88 KB Pane aside and put it back, and built a 41 KB empty undo history when it was not slim; it saves only the fields it changes now. The type table is made once a process (it was made at every store record: 123 µs to 33 µs a small pane's record in Debug). ponytail: Store.close's CRC32 over the record is most of what is left, and a record repeats its 14.6 KB type table; a table once per store file would shrink both, a change to the store's format left until a close's cost matters. The live keystroke cells were not run (perf rule: `--live` only for a change to terminal output); a dump is off the hot path. ## Phase 5 part B terminal notes Built 2026-10-29: screen-state terminals wired into the replicas (`src/screen_state.zig`, `src/replica.zig`, `src/tape.zig`, `src/terminal.zig`; `zig build replicas`). Transport-free: bytes in, bytes out, proven in part A's in-process harness; the remote agent's stream carries them. Decisions taken at the forks, with the ponytail lens: - **What crosses.** The authority's units, as in part A, but a pty's output is recorded without its bytes. Then, per peer, its terminals' messages. Each one is u8 pane, u32 the pane's serial (a slot's next pane is another), u8 kind, and the kind's bytes: - to a replica (the remote's 0x06): 1, a delta; 2, rows (`encodeRows`); - to the authority (0x04): 1, an acknowledgement (u32 seq); 2, a request for rows (u64 lo, u64 hi, absolute); 3, a delta refused: start this terminal afresh. - **Where they go in the stream.** Each replica steps every unit on its own core, and a step in a terminal pane reads the emulator (rows, cursor, modes, marks). So before the authority steps any input that is not a pty's output, and once a frame, it sends a peer its own steps since the last unit (`Authority.pending`, below) and then what the peer is owed of the terminals (`Screens.flush`); after a step that read a terminal's history, before its unit, the rows the peer lacks (`Screens.fill`). A replica takes them in that order between units, and a step it replays reads the terminal its authority's step read. Nothing goes after an output step: a flood is paced by the frame rate, not by pty reads. - **The authority's own steps are batched.** One unit a pty read was 2000 units a second under `yes`: each rolled back every replica that had pending input, and each carried a digest of one word a field, about 12 KB (28 MB/s a peer). `Authority.output` steps into an open batch without a digest; `pending` sends the batch as one unit with one digest, before any other input is stepped and once a frame. Under screen state a digest is one word (the harness's trace mode keeps the fields, to name one). Units under the flood went from 28 MB/s to 0.1 MB/s a peer. - **Several deltas in flight, against the version last sent.** Each state sent is a version of its terminal (`Track`), made once for every peer: a hash of each active row, the TERMINAL record, and what pardes keeps of the stream (the OSC 133 marks, `terminal.marksOf`). A peer's delta is against the version it was last sent. The stream is ordered and loses nothing, so the peer has that one by the time it applies the next, and the replica holds exactly that version's blobs. Up to 32 are in flight (13 at 207 ms and 60 Hz). A peer at the bound gets nothing until an acknowledgement frees room, then the newest version, the ones between skipped. Peers at the same base get the same bytes (one encode for all). A delta against a base the replica does not have (one lost, or it started afresh) is refused, and its acknowledgement asks for a reset. - Rejected: mosh's base at the last *acknowledged* state with a throwaway number. On a stream that loses nothing it only makes deltas bigger and the replica keep more. - **The replica's terminal is the authority's, to the row.** - Its history is exactly as long as the authority's: no byte limit of its own, trimmed to the authority's count at each delta. Row numbers and the scrollbar then say the same on both sides, which steps read. - It takes the authority's viewport, its cursor's semantic content (a prompt's input, which `promptInputReady` reads) and its marks. - Its emulator is written by `apply` and fetched rows alone. A replica's core never ingests, resizes (the authority sizes it; a resize's own core work, its pointer selections cleared, still runs) or replies to its program; its pty writes are off the tape on both sides. - No digest covers an emulator under screen state (`State.digest`); the harness compares screens instead. - **What an ingest tells pardes is taped.** Over a terminal, pardes keeps positions: the text-mode cursor, selections, the edit copy, the pointer's rows. An ingest moves them when it prunes or erases history or clears the screen. The authority records what it told them (each tracked row's row after it and whether it was erased, the clear, whether the pages lived: `Site.positions`); a replica, which ingests nothing, takes that, and the rest of the logic runs alike on both. - **History a step reads is sent before its unit.** Scrolled past more than a screen at once between deltas, rows are holes on the replica, blank until fetched. A step in text mode reads a terminal's whole grid, and the replica's replay of that step must read what the authority's read, which a fetch an RTT later cannot give. The authority keeps each peer's holes as the replica does (`Peer.holes`, from what it sent), and after a step that read a terminal's rows (`State.read`, set outside the render) sends each peer those it lacks, before the step's unit (`Screens.fill`). A history row never changes, so each goes once. Scrolled into view, holes are blanked and asked for (`Replica.screenRequest`); the answer fills them an RTT later. - **A flood costs the replica a counter, not its rows.** When a delta scrolls more than the replica's screen and history hold, nothing it holds is in the new state: its rows stay where they are, renumbered, the active ones and the tail written from the delta, the rest holes. Growing and trimming a history's rows one by one was the 2.4 ms a `yes` delta took. - **Speculation stays off terminals.** An input whose live pane is a terminal is not speculated: its step reads the emulator, the last delta's here and the pty's newer one there. It waits for the authority (a key in tty mode already did, at its pty write). Phase 6 predicts the echo. - **Rollback never brings a terminal's past back.** - The rollback copy never held an emulator (State is skipped), and a replica's emulators, their State included, are allocated on the C heap, outside its rollback epoch (`terminal.emulator_gpa`, `State.gpa`). Deltas write them while a copy is held, and nothing of them is a copy's to save, keep or put back. Without that, a rollback freed a delta's pages (the test below crashes). A first fix, the epoch paused while a delta was written, was not enough: the copy also saved emulator memory reached through a stale pointer (below), and put its old bytes back. - The core's cache of a terminal's rows is marked stale after a delta, dropped when a copy is taken and emptied when one is put back: it is a grid's worth of text, and the copy's would be of an older grid (without the mark, the test reads it). Found on the way: - **A view's slot that is live is stale, and the copy follows it.** views.zig keeps a parked view's share as copies, stale while that view is live (a column tag's text, `Text.own`, freed by the live view since). The rollback copy walks every slot: a stale pointer that lands on a live allocation saves it, and a rollback puts its old bytes back. Part A was safe while every free under a held copy was put off, which holds again now that a delta's writes are outside the epoch. A ReleaseFast harness run found it (heap corruption after 20 seeds), a safe build's check that named what was freed under the copy pinned it. ponytail: the copy still follows the stale slots; skipping the live view's slot diverged (part of it is read), and is the next fix in views.zig, not here. - A pane's `cwd` inherited from a terminal's pane is a pointer, and the walker follows it: a digest of a pane next to a shell digested the shell's emulator. It mattered when a digest was taken outside the authority's tape (`pending`'s first version). - `resizeGrid` does core work before it resizes (the pointer selections of a narrowed terminal): skipping it whole on a replica diverged. Measured. Correctness ran beside builds; the timings alone on the perf cores (ReleaseFast). - **Correctness.** - The soak (unit tests, and the 197 recorded streams at 3 seeds): several deltas in flight, landing late and in order, one in 50 lost (refused, then a reset), random resizes and fetches. After every delta the replica's screen, history extent and viewport are the authority's as encoded, and every history row it holds outside its holes is the authority's. None failed, Debug or ReleaseFast. - `zig build replicas`: three clients typing in their own views over 30, 80 and 207 ms links while the shell runs a TUI, a shell's session with clears, then 8 MB of `yes` read 4 KB at a time. In ReleaseFast, 20 seeds at each RTT, there was no divergence, every replica's digest and every terminal at quiescence was the authority's, and each of the 23,508 deltas applied matched the authority's screen at its version. The same held in Debug: 4 seeds at each RTT, and 20 at 30 and at 80 ms; and in ReleaseFast with every replica allocation filled with 0xff (`PARDES_REPLICA_FILL`), 20 seeds at each RTT. - **The terminals in the harness** (ReleaseFast, 20 seeds, three clients; a peer's numbers while the flood runs): | | 30 ms | 80 ms | 207 ms | |---|---|---|---| | deltas a replica applied a second | 76.3 | 75.7 | 71.7 | | delta bytes a second | 10.3 KB | 10.5 KB | 10.0 KB | | unit bytes a second | 101.6 KB | 102.0 KB | 101.5 KB | | encode a frame, every peer, p50 / p99 | 15 / 36 us | 16 / 42 us | 16 / 31 us | | the same, one peer | 13 / 28 us | 14 / 30 us | 15 / 30 us | | apply a delta, p50 / p99 | 37 / 494 us | 35 / 442 us | 34 / 400 us | | rows fetched on demand, all runs | 1103 | 649 | 786 | | rows sent before a step that read them, all runs | 18.0 MB | 18.5 MB | 10.0 MB | One delta in flight would have been one an RTT: 4.8 a second at 207 ms. The rate is the frame's (60) and the inputs' (a delta before each). - **The bench** (`PARDES_SCREEN_STATE= zig build core-test -Doptimize=ReleaseFast -Dtest-filter=screen_state`, 200x60, the floods under the C allocator): - encode a version, every row new: p50 273 us, p99 394 us, 29.8 KB; nothing new: 71 / 95 us, 549 bytes; - `yes` at 19-20 MB/s: 63 deltas a second, 65 KB/s a peer (1038 bytes a delta), encode a frame 28 us at one peer and 29 at three, apply p50 65 to 72 us and p99 202 to 265 us (it was 2.4 ms); - varied text at 59-64 MB/s: 356 KB/s a peer, encode a frame 52 / 54 us, apply p50 468 to 493 us. - **Part A's costs, with terminals** (the harness, three clients): speculative apply p50 28 to 42 us; copy p50 95 to 151 us (1.3 to 1.5 MB); rollback p50 331 to 442 us, p99 1.6 ms at 80 and 207 ms. At 30 ms the copy's p99 is 7.8 ms and a rollback's 17 ms: a register holding a terminal's whole grid (`%` then `y` in a shell's text, 2 MB after the flood) is copied whole by every speculation. Registers could be pinned as file texts are. (Done since: the follow-ups below.) - **No local regression.** `pardes-perf-live --live`, A and the new tty interleaved A B A ... A (4 runs of the new one, 5 of A) on the perf cores; the median of the runs' p50s, new/A, present and core: | | against main (kuovmsxw) | against base | A/A (base) | |---|---|---|---| | key-down | 1.02 / 1.02 | 0.94 / 0.94 | 1.00 / 1.01 | | key-right | 1.02 / 1.03 | 1.00 / 0.95 | 0.99 / 1.01 | | edit-char | 1.01 / 1.00 | 0.99 / 0.99 | 1.00 / 1.00 | | page-down | 0.92 / 0.95 | 1.22 / 1.05 | 1.00 / 0.99 | | wheel | 0.83 / 0.85 | 1.13 / 1.10 | 1.00 / 1.00 | | `yes \| head -c 32M` wall | 0.92 | 1.12 | 0.94 | | a 1M-line open | 1.03 | 1.14 | 0.97 | Against main, this line's own A/B, every cell's runs overlap main's. Page-down and wheel against the base read 1.21 and 1.12 in an earlier run too, and 0.92 and 0.83 against main: main's since the base, not this line's. What the local path gained: a counter a pty read, a resize and a scroll; a flag a frame and a store when rows are built; and, for a terminal with positions over it, a taped value (a null check) and two stack arrays. Not done, and why: - Kitty images in terminals still do not cross. - Units under a flood still cost about 0.1 MB/s a peer: an output step is about 45 bytes of tape (its input, the pump, a host call), 2000 a second. Phase 6's zstd takes most of it (each step's records are alike); or a host could feed a pane's queued reads to the core as one. - The first step in text mode after a flood sends its terminal's whole history of holes, up to a few MB, ahead of its unit (ponytail: a step reads every row, its motion surface). - A 9P run waiting on a replica's marks (`noteMarks`) reads them from the last delta, not from the output step; its writes are the authority's, but `sent_command` could differ. The output pins (`out_start`, `out_end`) stay the authority's. - Text reads outside `buildRows` (a pointer selection's copy in tty mode, 9P's screen reads) do not ask for the rows; 9P's are the authority's anyway. - **A hazard for part A** (fixed since, below): the rollback copy followed a view's stale slot. - Echo prediction (phase 6): a key into a terminal waits a round trip. - A replica steps every output step of the authority's (bytes-less), about 2000 a second under a flood, each a core step. - The GUI was not measured: its host is unchanged, and the core's terminal path is the tty's. - The other agent wires it in: the tty and GUI hosts, `Screens` per attached peer, and a replica frontend drawing after `screen`. Follow-ups (2026-10-29): - **A live view's slot keeps no pointer.** When a view goes live (`to`, `close`, `only`, and `drawOthers` putting the live one back), its parked share is emptied (views.zig `emptied`): it held the pointers it handed the live fields (a column tag's text, a chord's argument, the presentation's) until it parked again, and the copy, walking every slot, saved and put back whatever memory had since taken their place. A pane's share holds no pointer but its selections, which `unpark` already hands over. A test frees a tag's text under the stale slot, hands its memory to another allocation, writes it under a held copy and puts the copy back: it fails without the fix. - **Registers and location rows are pinned, not copied.** A register's values and joined text, and a file's location rows (with the ones an edit dropped and a query's saved), are replaced whole and never written, so the copy pins them as it does a file's text, freed raw. Rows pin whole with what each owns (its path, its colours: `pinnedDeep`). In the harness at 30 ms the big copies were not the register but a file's location rows, 5 to 6 MB after the flood. ReleaseFast, 20 seeds, three clients, p99: copy 5.1 ms to 0.22 ms at 30 ms (0.19 to 0.20 ms at 80 and 207); rollback 14.9 to 9.5 ms at 30 ms (1.3 to 1.4 ms at 80 and 207). What is left of a rollback's tail is the replay's digests: half the harness's CPU is Wyhash over the whole core at every replayed unit, the pinned blobs included (part A's digest). Hashing a pinned blob once, while it lives, would take most of it. - **The live perf harness ends what it starts.** Every process a live run starts inherits its private runtime directory: after each cell, and in the cleanup every path takes, the harness sends TERM, then KILL, to each whose environment names it (`liveSweep`). With a wrapper that leaves a daemon-kept detached session (as the remote agent's client-live.sh does), the old harness left three sessions and their daemon a run; this one leaves none. The gate's baselines are recorded again from base (main and the new perf.zig), and .scratch/daemon/base/pardes-perf-live rebuilt from it. - **A blob's digest is a word, made once.** The walker's digest, taken at every unit a replica is sent and every unit it replays, hashed every blob whole each time: a 100 MB file was 5 ms a digest, two digests a unit, on the user's own session. A blob (a field annotated `.blob`: a file's text and line index, an undo snapshot, a register's values and joined text, a file's location rows, the ones an edit dropped, a query's saved) is now hashed once, its word kept by its address (walk.zig), and the digest folds that word in. The rollback copy pins what the same annotation names (`pinned` and `pinnedDeep` are gone). - **Exact while every free drops the word:** every such blob is freed raw, through `memory.freeRaw` or `freeUnfilled` (the pins needed raw frees), and those drop its word, so an address used again is hashed again. The first ReleaseFast harness run found one that did not: an edit freed the old text with `rawFree` itself, the next text of the same length came back at the same address, and the old word was folded in (162 of 180 replica runs diverged, in `file.content`, named with PARDES_REPLICA_TRACE); it goes through `freeUnfilled` now. A replica's blobs born under a copy are never digested (a speculation records none), so the epoch's own frees leave no word behind. A register's values array was the one written in place; `put` now makes a new one. ponytail: the word is kept by address, so a new free path that skips those two misses it; a word carried in the struct beside its blob if that bites again. - **Written in place, or freed past its word:** a Debug build hashes a blob's bytes at every digest and folds those in, and panics where the word kept at that address and length is not their hash: a blob written in place (which the pins forbid too) or freed without dropping its word. The tests, the replicas harness and replay run Debug and go through it (and pass); walk.zig's test writes a blob in place and sees its kept word is no longer its hash. A release build (ReleaseSafe, the user's own, and ReleaseFast) takes the word and never panics on a stale one: at worst it shows as a digest divergence. Not recovered yet: a replica that diverges records why (`Replica.diverged`) and takes no more units; nothing resets it or joins it again. That is the remote client's to add (a fresh join from the authority's dump), not a crash. - Measured (ReleaseFast, alone on the perf cores; PARDES_DIGEST_BENCH=100 zig build core-test -Dtest-filter="big file open"): with a 100 MB file open, a unit of a key into it (two digests) went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the file's word), and a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients, ReleaseFast): rollback p99 at 30 ms went from 9.5 to 2.6 ms (1.5 and 1.4 ms at 80 and 207 ms, about as before), copy p99 0.27 ms (0.22 before, within the run's noise); none diverged, none unequal, no delta wrong. ## Phase 6: upgrade notes Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`, `main.zig`, `dump.resumeFromDump`; `test/daemon.py --upgrade`). A running session, and the daemon, move onto the pardes installed since they started, in place: text, undo, views and layout, terminals with their running shells, and their terminals, windows and daemon link. Decisions taken at the forks, with the ponytail lens: - **The trigger is the word, `Upgrade`** (`SPC s U`, any tag, `/ctl` and `/pane/N/exec` over 9P). It moves the daemon too: the session sends it `upgrade` just before its own exec, so one word moves both, and no word of the daemon's own was needed. Nothing upgrades by itself. A focus-in (one comes back to the terminal or window from rebuilding elsewhere) stats the start path, and a file there that is not the one the process started from is said once, in a line: `a newer pardes is installed: Upgrade (SPC s U) moves this session onto it`. Rejected: a watch on the binary's directory (an inotify entry and an event source for a line), and upgrading on its own (a build half-written, or one the user means to test first, would take every session). - **Which binary: the path this process was started from**, read once just after its first frame (`std.process.executablePath`, a readlink, and a stat for the notice: off a startup's way, and no build is installed within a frame of a start), exec'd again at the upgrade. `install(1)` and `zig build` put a new file at the path, renamed over the running one, which `/proc/self/exe` then names `(deleted)`. The record carries the path exec'd, so a process that fell back to the old binary (whose own exe is that deleted file) upgrades from the same path next time. Not argv[0] and PATH: a session's PATH is the one it started with, and the readlink is what the daemon's spawn already used. - **Asked first, ` --version`**: run as it is, it must exit 0 with a `pardes ` line within 10 s. A file not there, not a pardes, or one that dies at start is found before anything is let go, and said ("Upgrade: exited 3 at --version; this session stays as it is"). 9P is served while it runs (the turn rested), so a client's clunk after its `Upgrade` write, and anything else, is answered; after it, nothing is. - **At a step boundary**: after a pump, the pty readers are canceled (a canceled tty reader posts an end of file, passed over while upgrading), the terminals' input readers stopped, the loop's queue applied, and the core settled (`Pardes.settle`: queued inputs, then effects). In the GUI the queue is drained, the readers canceled, and drained again (a reader waiting for room gets it). Bytes read are never lost; keys typed during the exec wait in the terminal's kernel buffer for the new process. - **The dump goes to the DumpDir, `pardes-upgrade-.dump`**, not a memfd and not the runtime directory: a refused load leaves it where a relative `Restore` finds it, on a disk and not a tmpfs (a big file's session dumps 100 MB and up). Never $PARDES_DUMP, which is a Dump's. A successful load deletes it. - **The record is one argument**, `--upgraded=`, ahead of the arguments the session was started with, which configure it again (`--9p=`, `--tty-toggle`, `--geometry`, `--nested`); its file, stdin, directory, `--detach` check, forwarding and daemon fork are skipped. A line a word: `link`, `stub`, `old` (fds), `window`, `pty `, `tty `, `exe`, `dump`, and `refused` in a fallback; a word another build adds is passed over. Every fd it names is close-on-exec but across the one exec (`Record.closeOnExec`), and the new process sets them back at once, so no shell it forks inherits a master or a terminal. - **The pid stays**: the shells stay the session's children, the stub keeps waiting on it, the 9P name of a session named by its pid is the same, and PDEATHSIG (set again) still means the stub's death. - **Terminals.** A kept terminal pane is not revived (`resumeFromDump` with `kept`): no reset, no banner, no new shell; ghostty's state is the dump's, the program's own, and its pty keeps its size (no 1x1 nudge: the screen is already right). The host puts the master back in the pane's slot and reads on (a command pane's exit watched again). A terminal pane with no live master (its shell gone) is revived as a Restore revives it. - **The tty's terminals stay raw, their cooked termios in the record**: put back to cooked between the two processes, a key would echo and a ^C (forwarded by the stub) could kill the new process before its handlers. What is a stack is unwound: the kitty keyboard flags are popped (the new process pushes them once its query is answered), its kitty images deleted and its shm names unlinked (the pid is the same, and so would the names be). The alternate screen, mouse and paste modes stay on; the new process sets them again and draws everything, one full frame over the old one in place. The old image's prompt rc files (the running shells have read them) are swept by the new one, which makes its own: at its sweep, any of its pid are the old image's. They stay for an exec that failed. - **Views come back whole.** `resumeFromDump` keeps every view the dump has (no `views.only`) and does not run the startup config again (the session's settings are the dump's). Each terminal's view is in its `tty` line: the primary one is `run`'s, the others are entered as `--attach` enters one, with their stubs; a view that lost its terminal closes. The GUI opens a window for each open view, the live one first (`openAnotherWindow(k)`): its windows are the one flash, as decided. - **9P is cut and comes back.** What the connections had to send is let go first (`Listener.flushOut`, the 200 ms the quit's path already had), then the exec closes the listener and its lock, and the new process claims the name again. ponytail: a dial during the exec and the load is refused (one round of a client's retry); passing the listener through as well would keep it queued. - **Replicas are told in words.** Each connection a session serves gets a `refused` frame ("the session is upgrading to another build of pardes: connect again with that build") and a shutdown before the exec. Their hello's semantics hash would not match the new build anyway, and the session registers its new hash on the same link (a second `hello`, which the daemon takes as an update). The reconnect, and a local client of the same path re-exec'ing itself, are phase 5B's: there is no replica client yet. - **The daemon re-execs the same way**: its own start path, asked `--version` first, the listener and lock (fds 3 and 4), the replicas' listener, every session's link and every held master kept, its registry on its command line (`--daemon --upgraded=` with a `c` line a connection, its slot, kind, pid, state, semantics and name, and an `h` line a master and its owner's slot); a replica whose hello is still on its way is let go and dials again. Sessions never see it: their links are the same sockets. - **Failure never loses a shell.** 1. The binary does not answer `--version`: nothing changes, said. 2. The exec fails: the fds close-on-exec again, the readers and the terminals' keys start again, the kitty flags are pushed back, the dump deleted, said. 3. The new build cannot load the dump: it execs the old binary again, from `/proc/self/exe` opened before the exec (`old`, Linux), with `refused` set; that one loads its own dump and says "Upgrade: the new build could not load this session (); it goes on as ". 4. No old binary to go back to (FreeBSD has no /proc by default), or it refuses too: a fresh session adopts every kept master as a terminal pane (the Adopt path) and says "Upgrade: no build could load the session (); its shells are here, and Restore brings the rest back". The dump stays. 5. A crash after the exec is the crash Phase 3 covers: the daemon's dups keep the shells for an Adopt, and the dump is in the DumpDir. `PARDES_TEST_UPGRADE_REFUSE=1` makes a start refuse the load as such a build would (the first start only), `=all` the fallback's start too: test/daemon.py installs a script that sets it and execs the new build. Found on the way: - A fresh session's first pane has no directory until its own shell's spawn is performed, so the `Tty` word that adopts a shell refused ("Tty: : no such directory") and the shell was hung up with "no room": the kept shells are taken after the startup effects, and those they emit performed then. - A test reconnecting after the word could reach the old process, still asking `--version` (and serving 9P meanwhile): the test waits for the exec (the session's exe, or on FreeBSD its new semantics in the daemon's registry) before it reconnects. - An `Adopt` (or a `Detach`) served in a session's first pump, before its first frame, is refused as if no daemon kept it: the reader that takes the daemon's `ok` starts after that frame, and `kept()` waits 2 s for it. The test's next session waits for its first frame. Pre-existing, left as it is. Gates (Debug, `A` this line unstamped, `B` the same with `-Dstamp-commit`, a trivial visible change: `pardes 0.24 ()` and another semantics hash; both installed in turn at a private prefix by a rename, as `zig build` and install(1) do): - `python3 -B test/daemon.py --upgrade A B`, under a private runtime dir. A tty session with an unsaved edit, a shell running `sleep 1000; echo done-after-upgrade` and a second terminal (a second view), and a replica of A's build connected: - B installed, a focus-in says the newer build; `Upgrade` written to `/ctl`: the session's 9P answers again 123 ms later (81 to 83 ms on the box). The session's pid is the same, `/proc//exe` is B (not deleted), and so is the daemon's, the same pid. - The edit is there, the pane dirty, and `Undo` takes it back. - The shell's pid is the same, still the session's child, its `sleep` running; killed, the shell goes on and `done-after-upgrade` shows. - Both terminals are drawn again (each its own view); the stubs wait on. - The replica of A's build is told `the session is upgrading to another build...`; a hello of B's build is welcomed by the daemon, one of A's refused (the daemon lists the session as B). - The fds level: the session's as before but the replica's, the daemon's the same; the session's prompt files as many as before, the new image's. - A refusing build (a script that sets `PARDES_TEST_UPGRADE_REFUSE=1` and execs B) installed and asked: the session goes back to the binary it ran, its exe that inode again, the shell and text as they were. - One that refuses everywhere (`=all`): a fresh session, the shell a pane of it (`echo $$` there is its pid), the dump left in the DumpDir. - The session killed outright: the daemon, upgraded three times, still holds the shell's master, and a new session adopts it. - Nothing left: no process, no socket. Passed 3 runs of 3 (and every run since). - `--upgrade --gui A-gui B-gui`: a window session in test mode (hidden windows, captured) with a second window from `pardes-gui --attach`, the same edit and job: after `Upgrade` the pid is the same and the exe B's, the daemon B, the edit and undo there, the shell and its job the session's, both windows' captures written again, B's hello welcomed; the session ends with nothing left. Passed 5 runs of 5. - FreeBSD 15.1 (the box, `pardes-ua` and `pardes-ub` cross-built from this line): the same tty test but the fallback to a kept copy (there is none). The new build is told by its semantics in the daemon's registry, and the daemon's re-exec by `--upgraded=` on its command line; the fds level by procstat. 81 to 83 ms from the word to 9P. Passed 9 runs of 10 (the last two on this line's final builds); the one failure was the test's first-frame race above, fixed, and it passed after. - The existing gates: `test/daemon.py` (20 cycles) and `--gui` (10) pass; `test/fs.py` passes (49 s); `zig build snap` 113 of 113; `zig build replay` passes (the 113 scripts, 20 seeds at one view and 20 at three, their dumps loaded to their digests); unit-test tty 1479 of 1499, 20 skipped; unit-test gui 1550 of 1561, 9 skipped, and the two known failures (`tagline pointer mapping...`, `a tag caret past its row's end...`). core-test's new tests: the record read back as written (and refused when cut), and an upgraded dump keeping its views, a running terminal's alternate screen and no new shell, and settings the startup config would change. Measured. The upgrade itself, Debug, a 300 MB binary: from the word written to `/ctl` to the session's 9P answering again, 123 to 130 ms, one run beside builds 299 ms (the `--version` ask, the drain, a dump of a small session, the exec and the load); on the box, 81 to 83 ms with its 125 MB binary. No local change (ReleaseFast tty, main kuovmsxw against this line, both live at once, their samples alternating, through the perf lock; lat.py's startup is a spawn with no daemon running to its first frame's end): | | A/B | A/A | |---|---|---| | startup, 100 a build: p50 | 0.944 | 1.010 | | the same: mean | 0.919 | 1.061 | | the same: median of the fast mode (and its share) | 0.964 (54 to 52) | 0.996 (69 to 73) | | startup, 40 a build, two runs: p50 | 1.001, 1.266 | 0.998, 1.010 | | key in a file, to its frame, p50, two runs | 0.997, 1.002 | 0.958, 0.990 | | key echoed by bash, to its frame, p50, two runs | 0.997, 0.991 | 1.004, 0.999 | The startup times have two modes on this machine, about 4.5 and about 12 ms, and the share in each moves between runs and between the builds of an A/A alike, so a p50 that falls between them jumps (the 1.266). Within a mode the builds are the same. A first A/B, made while the path was still read at the start (a readlink and a stat before the first frame, some microseconds), read p50 1.04 to 1.13 against an A/A of 0.98 to 1.00 on runs that overlapped builds; the read was moved after the first frame anyway, so a start does nothing it did not. Not done, and why: - Remote replicas reconnect on their own, and a local client of the same path re-execs itself first: phase 5B's (no replica client exists yet). What they get today is the `refused` frame above, then the hangup. - The 9P listener is not kept across the exec: a dial in that window (a hundred milliseconds or so) is refused. Clients that dial afresh (the hosts' Client) retry; a mounted session sees its connection end. - On FreeBSD a build that cannot load the dump starts afresh (shells adopted, dump kept): there is no `/proc/self/exe` to go back to by default. Keeping an fd of the binary open from the start would do it, at an fd a session. - A crash of the new build after it starts is a crash: the shells wait in the daemon for an Adopt and the dump is in the DumpDir, but nothing restarts the session on its own (the stub, which runs the old binary, could). - An escape sequence a program had half written when the dump was made is lost (the snapshot is at ground state, Phase 2 notes). - Language servers and `|` pipes running at the upgrade are let go (their pipes close at the exec; they end), and the shells of panes closed just before it, still waiting to be reaped, are reaped only when the session ends. A daemon note (a terminal attaching, adopted shells) read by the old process's thread but not yet taken by its loop is lost; the stub of such a terminal sees its pipe end. - A detached session's upgrade (no terminal) was checked by hand (scratch script), not in test/daemon.py. - GUI windows come back at the size and place the compositor gives them. - A daemon protocol bump across an upgrade is not handled: the sessions of the older build keep talking theirs. - A session name with a newline would break the daemon's registry line. - An upgraded process records no tape (a recording ends at the upgrade).