# The daemon: one pardes per machine, many clients per session > I want to have a global pardes daemon ... holding file descriptors for > ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a > cpu+file server ... the main test: multiple clients to a single session, > and a client here and the session on freebsd and very deep and thorough > performance testing and optimizations to make it feel like its local and > snappy. Designed 2026-10-29 in discussion (the user, the coordinator, a survey agent). Nothing here is built yet. Paths are relative to the repo. ## Decided by the user - **A deterministic core, replicated.** The session is the authority: it orders every client's input and every host result, and only it performs effects. Each client runs a full core replica, applies its own input at once, and rolls back and replays when the order differs. Visible rollback is acceptable. - **Kakoune-style clients.** Each client has its own view (layout, focus, selections, scroll) over shared buffers and terminals. Registers are session-wide. A terminal shown at two sizes is sized by its last typist. - **Performance order.** First, the regular tty and GUI get no slower. Then the detached and remote clients reach that local performance, with the session on the FreeBSD box included. - **A host step may show up one RTT late, but is never wrong.** This covers Look, open and exec, as sam's hostlock does. - **Join is complete.** The joining client pays for it in time; the authority and the other clients do not. - **One long-lived daemon per user per machine.** It supervises one child process per session, postgres-style. Nothing is left dangling, and the core is not de-globalised. ## What today's seam is, and why it goes `src/detached/server.zig` is a third Host: it runs the core and the ptys in a single poll loop and sends cell frames to the frontends, at the smallest common grid across them. - **The frames are mostly not diffed.** Every layer's cells are re-sent on each frame (wire.zig:734, :755). - **Several things never cross the wire.** Images, PDF pixels, rail ticks and pane animations stay behind (surface.zig:483-509). - **Every keystroke is a round trip,** which no amount of optimisation can make feel local at 30-80 ms. - **Each remote 9P operation dials afresh,** costing about 6 RTTs (9p_io.zig:2361-2385). ## The design - **Processes.** - The daemon is subreaper, spawner, fd holder, registry and 9P root. - Every session is a child of the daemon: a GUI window, a tty session behind a stub, or a headless session. - A session is always an in-process authority with its own local view, so the regular path does not change. Replicas exist only for extra clients or remote ones. - **The tty stub.** - `pardes` in a terminal passes its terminal fd to the daemon (SCM_RIGHTS), along with cwd, env and argv. - The session child drives that fd directly. The stub forwards SIGWINCH and exits with the session's status. - Detach: the session drops the fd and the stub exits. No fork happens. - **GUI detach.** The window is destroyed and the process carries on headless. Closing a window that has no other client attached quits it. - **Shells survive a session crash.** After each `forkShell`, the session sends the daemon a dup of the pty master, so the shells do not get SIGHUP. Being subreaper only decides who reaps them. - **The recorder.** It is null when no client is attached. When one is, it appends what the authority consumed, in order: - every step's host tape (the results of the ~30 `turn.yield()` sites); - each `serveFs` mutation. Turn order is log order, so the local path changes by one null check. - **The protocol.** One ordered stream: events, host tapes, digests and terminal state. Framing, the hello and the Event codec are kept from wire.zig; the frames go. - **Terminals cross as authoritative screen state,** not as pty bytes: - metadata plus immutable row blobs, assigned in absolute terms; - scrollback fetched on demand. A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the authority, and the local path keeps reading ghostty directly. - **The reflective walker.** One comptime `@typeInfo` walk gives: - the digest; - the complete dump, which also serves join, Restore and Handover; - the rollback copy. Fields are annotated local, authority-only or derived. A hash of the type graph versions it. The hello carries a semantics hash (version, features, grammars), and a mismatched build is refused. - **Rollback.** - When the first speculative event goes pending, the replica copies what speculation can write: views, buffers, registers, counters and terminal metadata. Blobs are pinned rather than copied. A file's text and its line index are neither: an edit writes them in place, and the copy's allocator notes what each write into one that was there took out, which a rollback writes back, the newest first (src/rollback.zig `wrote`). - On a divergence it restores that copy, then replays the confirmed events and its own pending ones. - The authority never rolls back. - **Join.** The authority forks a writer child at a step boundary; the child streams the dump. The authority buffers the events after it for the joiner. - **Remote.** - `ssh -L` of the daemon's one socket. - The daemon hands a replication connection's socket to its session. - A remote daemon mounts as `/n/`. ## Determinism leaks (the survey's census) | Leak | Where | Fix | |---|---|---| | Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak | | 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order | | Host syscalls in a step | the ~30 `turn.yield()` sites | host tape | | Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done | | Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local | | Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ | | Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced | | getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state | | Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash | | ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals | ## Phases Each phase ships on its own and keeps `zig build perf --base` level (with an A/A run). All 110 snapshots stay green throughout. 1. **Determinism and the replay-divergence harness.** - Work: the recorder and tape; the leak fixes, the render one first; the semantics hash; the walker's digest. - Check: replay each recorded log into two cores at different addresses and one headless core. - Gate: zero divergences on the snapshot scripts and on N monkey seeds. 2. **A complete dump on the walker.** It replaces dump.zig for Restore and Handover. - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint. 3. **The daemon and the tty stub.** - Work: in-place detach and reattach; pty master dups; the frame wire deleted. - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves its shells adoptable; no stale sockets after N cycles. 4. **Views.** A second window on the same session, in one process. - Gate: logs with several views replay without divergence. 5. **Replicas.** - Work: speculation, rollback, screen-state terminals, join, socket handoff, `ssh -L`. - Gates: - a 30/80 ms delay relay keeps remote within noise of local; - the rollback rate stays low under the fuzzer; - a lagging client leaves the authority flat; - a join costs the authority under one frame. 6. **Remote amortisation.** A content cache by hash, prefetch, zstd, predicted echo in TTY mode, and session upgrade by re-exec. New measurements to add before phase 1 lands: keystroke-to-present in the tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file open. ## Still open (the coordinator's position in brackets) 1. Delete the frame wire at phase 3, leaving no remote attach until phase 5. [yes: it was never used daily] 2. The stream as its own framed protocol, or as a 9P file. [framed: one ordered stream with no per-read round trip; 9P stays for scripting and the relay] 3. Rollback by walker copy or by arena copy-on-write. [whichever measurement of the mutable state's size decides] Decided: the walker copy, about 100 KB and 0.05 ms once file texts are pinned (Phase 5 groundwork notes). 4. How the daemon starts: spawning itself or socket activation. [itself, by double fork; FreeBSD has no systemd. A user unit can be added if a terminal's scope ever kills it] 5. The join writer: a fork child, or a metadata copy serialised on a worker. [fork first, measure the page-table copy] Decided: the copy on a worker. A fork of 1 GB stalls the authority 13 to 20 ms, and copy-on-write taxes it for as long as the join streams (Phase 5 groundwork notes). ## Phase 3 notes Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`). Decisions taken at the forks, with the ponytail lens: - **Sessions are not the daemon's children; every one registers.** The design had the daemon spawn a terminal's session from the fd, cwd, env and argv the stub passes. That proved wrong here: - the user's terminals each run in a `9ns --mntgen` user and mount namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon spawned would run in whichever namespace the daemon was started from, its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits, umask and stdin (`pardes -`) would be the daemon's too, or have to be carried and re-execed; - startup would pay an IPC round trip and an exec of a 100+ MB binary; - the snapshot harness's idleness probe walks the launched process's tree, and a daemon-spawned session is outside it. So `pardes` in a terminal forks its session off itself before anything starts (`daemon.forkSession`): the child calls `setsid` and draws on the `/dev/tty` the parent opened; the parent stays as its stub. GUI windows and `--detach` sessions just register. The daemon spawns nothing, so it is no subreaper: a detached session is reparented to the system's reaper (systemd --user or init), which reaps it, as it reaps a crashed session's shells. Nothing dangles: the daemon lists every session (its connection) and holds every master, and exits a second after it holds neither. - **Starting the daemon.** The first session takes the flock, binds and listens itself and queues its own connection; `pardes --daemon` is then double-forked and exec'd onto that listener and lock (fds 3 and 4), by a terminal's stub after the session's fork, so no session waits for it: its hello sits queued until the daemon runs, and is not waited on (the daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s). With the spawn before the fork, a cold start measured +0.38 ms (+7.6%). The daemon's reader thread likewise starts after the first frame. Another starter waits on the lock; a stale socket is the lock holder's to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and the fds it carries. - **pty masters.** `forkShell` sends a dup; a host's close sends `close`; a session's end sends `quit` (its shells hang up); a master whose shell is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT handlers send `quit` too (one `send`, async-signal-safe), so only a session killed outright or crashed leaves shells behind. - **Adoption is a word, `Adopt`,** calling the host through a vtable entry (no Effect): the daemon hands over every orphaned master, and each becomes a pane through the `Tty` word, its spawn taking the pty instead of forking; a 1x1 size then the pane's makes it redraw. Adopting at startup was rejected: an `EDITOR='pardes --wait'` session would take the shells and hang them up when the editor exits. - **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and ends with the session's status (a forked stub `waitpid`s; an attaching one reads `x`). A HUP is the terminal's end: kept by a daemon, a session takes HUP on its signal thread (sigwait, beside WINCH) and detaches, as on EIO; HUP never ends a kept session, since the terminal's own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane (the terminal is raw, as before); a TSTP sent to the stub is ignored, since a stopped stub hands the terminal back under a session still drawing; a background start stops at a `tcsetattr` before the fork, as the old raw mode did. `--wait`, nested forwarding and `--tty` run before any of this and are unchanged. - **`--detach` is a session started detached:** the tty host with no terminal, the GUI with a hidden window closed at once. With the frames gone, `detached/server.zig` was a third host with nothing left to do that the other two could not. - **`Attach` (the word) is deleted:** it switched a session's own terminal to another session through the frames. - **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a session as before: in the terminal's own process, ended by its hangup, for a harness that kills by pid and closes the terminal. A GUI test window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`. - **Known gaps.** A forked session takes its stub's death, a SIGKILL too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and detaches; a `pardes --attach` stub is no parent of its session, and one killed with SIGKILL while its terminal stays open still leaves the session drawing there until the terminal closes or Detach. A reattached terminal keeps the first one's environment (TERM, COLORTERM); vaxis queries its capabilities again. A daemon killed outright leaves its sessions unregistered, and Detach then refuses. A daemon of an older protocol answers `version`, and a session then runs without one. A GUI started in a terminal's foreground stays that terminal's job. Measured (ReleaseFast, base main a7369739 against these changes, both live at once and their samples alternating A, B, A, B; A/A the base against itself; p50 ratio new/base): | | tty | A/A | GUI (test mode) | A/A | |---|---|---|---|---| | startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 | | startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | | | key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 | | key echoed by bash, to its frame | 1.016 | 1.005 | | | What is left of the tty's startup is the stub's fork (about 70 us), which the design cannot do without: the job the shell waits on must be able to end while the session runs on. Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout), and overran the stack, clobbering a neighbouring local in the forked child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one. ## Phase 4 notes (Superseded by "Architecture step 5 notes" below: the session's share of a view is indexed now, and a second local terminal or window is a replica client; the in-process extra views described here are gone.) Built 2026-10-29 (`src/views.zig`, the tty and GUI hosts, `daemon.zig`, the snapshot harness and the monkey). Decisions taken at the forks, with the ponytail lens: - **A view is swapped in, not pointed at.** The ~1,900 references to the core's layout, focus and pane state stay as they are: the core sees one view, the live one, in the fields it always had. A view's share of the session is a list of field names (`views.fields`, ~60) made into a struct by one comptime `@Struct` (`Part`), and of each pane a struct beside it (`PaneView`, in `Pane.pv`); going into view `k` copies the live one's out (`park`) and `k`'s in (`unpark`). Copies, not swaps: a slot is stale while its view is live and written whenever it goes out, so no two places own a selection's heap. One view never parks: the single client pays `views.several` (an AND and a compare) in the pump, the step's sync and an edit, and one store a key (`typedInto`). Rejected: a `*View` the core reads through (every reference rewritten, an indirection on every access); a twin pane per view (Zerox exists only for files, and a terminal's pty is its pane's slot). - **Where each piece of state went.** - Each view's own (`views.fields`): the screen size and cell metrics; the columns, their weights, panes, serials, tags and the finder; rects; `active`, `active_column` and the header focus; the jump list (kakoune's is per client); the pointer, hover, drag, mouse grab, clicks and chords; key prefixes (`"x` register name, leader, Ctrl-w, window pick); a pending clipboard read; the presentation and the animations (lifts, cursor glide); `windowed`, `native_images` and `native_image_patches`, which the census had as watched host writes: a tty view on kitty and one on a plain terminal differ. The 223 comptime platform branches stay comptime: a session is one build, which the semantics hash says. - Each pane's, each view's own (`PaneView`): `vweight`, `collapsed` and `attached_to` (a layout's); `focus`; the body's mode, cursor, selections, sticky column, pending normal command and find; a document's scroll, `hscroll` and size; and what the step measured for that view (`tag_rows`, `tag_top`, `tag_scroll`, `view`, `notices`), so a view drawn without a step of its own draws its own geometry. - The session's, every view's: the panes' text, undo, terminals and their scroll (tmux shares a pane's copy mode too), tags (acme's are per window; one tag per buffer is one text, its caret included), the workspace tag, prompts, messages, registers (decided), the macro being recorded, look-hover (one pointer at a time), mouse sweeps, query panes, search state, settings and theme, the effect and input queues, `needs_frame` (any change draws every view). A `.name` prompt (a file's name drafted in its tag) is the view's that opened it (`Pane.prompt_view`): only that view's focus leaving the pane drops it, and its closing. - **A terminal shown at two sizes** takes the size of the view that typed into it last (`Pane.sized_by`, set by a key or a paste; a new pane's is its maker's; a closed view's goes to the next to sync it). The others show it at that size: `sync` resizes no pty for them, and `pane.cols/rows` stay the terminal's own, so the terminal code's invariant (pane size is the emulator's) holds. A view shorter than the terminal is passed down its top rows as far as keeps its cursor's row in sight, as tmux shows a bigger window in a smaller client (`Pane.pan`, terminal.panFor: per view, made in the step and again when a view is drawn out of turn, read where the body's rows start, `Pane.scroll` and the body walk), an arrow in its gutter's first row saying rows are above; a shell's prompt, at the bottom, is never out of sight. ponytail: a panned view's mouse reports to the terminal's program and its kitty images are placed as unpanned. - **Panes come and go under parked views.** A view keeps the serial of each slot when it goes out; unparked, the ids whose serial changed leave its layout and its keyboard moves to its first pane left. A pane focused in a view that does not show it (a Look at a file another view opened, a jump, a 9P focus) is placed in the view's active column (`views.show`, kakoune's buffer shown in the window that asked). A text edited in one view moves every other view's cursor and scroll with the splice, as a Zerox twin's (`File.ParkedPlaces`); their selections go, as a twin's do. - **Drawing the other views stays out of the digest.** The pump draws each other view by unparking it, rendering and presenting, and restores the live one exactly (`drawOthers`): no digested field moves, so the headless replica, which draws nothing and skips it, agrees. What drawing a view makes of its own (its animations, its presentation) is kept; what its unpark normalised (a cursor clamped inside a changed text, a pruned layout) is not, and is done again when it next goes live by an input, in every replica alike. `parked` and `pv` are `.local`: a parked view is checked when it is live again. Found by `pardes-monkey --views=3 --tape`: a drawn view's `post_present` (acknowledgePanelPresentation, which moves the pointer's hover) was applied by the headless replica to the live view; other views take no post_present, their panels acknowledged when they are live and drawn. - **Inputs.** `toView`, `openView` and `closeView` are taped. A host goes into a view before the input its client sent: the tty's terminal readers post through a `Poster` that puts a `from` ahead of a terminal's input whenever another terminal posted last (so one terminal posts none, and coalesced motions never cross views); the GUI asks SDL which window an event is for (`route`), which also takes a window's resize to its view (the GUI sizes the live view from its window each frame). - **The tty.** Each terminal is a `Screen` of its own: its vaxis, input reader, frame answer (`FrameAck`, was global), kitty images and shm probe (were global), glide, trail and flash, its stub and its Killed slot. The primary's stub stays `daemon.stub`. A SIGWINCH asks every terminal its size (attach stubs forward theirs; another view's terminal is no one's controlling terminal); a SIGHUP asks each whether its terminal or stub's pipe hung up, where it used to mean the one terminal. Detach, or a terminal's end, with views left closes only that view and tells its stub `d`; the session's end tells every stub. ponytail: a bracketed paste in progress is the shell's, not a terminal's: keys from another terminal during it join it. - **The GUI.** One `Gui` (device, atlas, fonts, pipelines) and its window's share swapped like the core's (`window_fields`: the SDL window and its presenting, scroll and glide, scene and post chain, pointer, blink, captures); `Shell.follow` keeps the Gui on the core's live view. Closing a window while others stay closes its view; Restore closes the others. ponytail: a second window's compositor hit test reads whichever window's state is in the Gui (the workspace tag is in the same place in every view). - **The daemon.** `attach` names an attached session too: it takes another view (a terminal, or a window). With no name, the detached sessions are asked first, then the only session of that kind. - **Harnesses, extended not new.** A snapshot script's `start ... views=N` gives the app N-1 more ptys (PARDES_TEST_VIEWS, opened as attached ones from the start), `view ` types into and captures terminal k, and every wait reads all of them (a pty nobody reads would block the session's frame); a `resize` there SIGWINCHes the app, as a stub would. `pardes-monkey --views=N` switches among N terminals a quarter of its steps, each with its own size; `zig build replay` runs its seeds at one view and at three. The GUI's test feed takes `ESC]777;view;BEL`, and PARDES_TEST_VIEWS=N opens N more hidden windows, captured under view-k. - **Not done, and why.** - A view of the other kind (a terminal on a window session, a window on a terminal's): each process links one frontend; the daemon still matches kinds. - 9P acts in the live view, the one whose client spoke last: no view is named in a path yet (a script's focus or layout is that view's). - Every change draws every view (`needs_frame` is the session's); a view's own dirtiness waits for a measurement that wants it. - Restore and `-l` keep one view, the live one; a dump has every view (phase 2 notes), and Handover carries panes, which join the views of the window they land in. - **What proved wrong in the design.** "Views" in the design were a struct the core reads through; the swap made them cheaper and kept every reference. Its rollback copy (phase 5) is the same list of fields. The design's view-local state assumed a step per view; the step's measurements (`tag_rows`, `notices`, the pane's geometry key) had to become per view too, or a view drawn after another's step drew the other's tag heights. And the census's single "frontend kind" leak is not per process but per client: kitty graphics are a terminal's. Measured: ReleaseFast, the base binaries from before the rework against this line on phase 1's merge (vvmuwntp, before main's tty throughput fix), interleaved base, new, base ... in one run on the perf cores, then an A/A of the base against itself the same way; the ratio of the medians of runs, new/base (4 runs of new, 5 of base; `--reps 25` for the core, `--live` and `--live-gui` through the 32 MiB live driver): | | new/base | A/A | |---|---|---| | core, medium: render, keys, PageDown, wheel | 0.97-1.02 | 0.99-1.01 | | core, medium: edit-char, edit-twin, deep-sel | 0.90-0.96 | 0.99-1.00 | | core, medium: open, body-2m, body-appends | 0.94-1.02 | 0.88-1.08 | | tty, keystroke to present p50 (5 gestures) | 0.94-1.11 | 0.90-1.02 | | tty, the core's part of it p50 | 0.95-1.04 | 0.86-0.99 | | tty, `yes \| head -c 32M` wall; 1M-line open | 0.96; 0.98 | 1.09; 1.02 | | GUI, keystroke to present p50 (present-bound, 30 ms) | 0.98-1.01 | 1.03-1.10 | | GUI, the core's part of it p50 | 0.84-1.01 | 1.01-1.19 | Every cell is inside its A/A; the widest, the tty's PageDown at 1.11 (runs 386-506 us against 410-451), is 0.94 best against best. What one view pays is a byte test in the pump, the step's sync and an edit (`views.several`), a store a key, an index for the live terminal or window, and a `from` never posted. A first A/B, of this line before the rebase onto phase 1's merge (its render still made the geometry afresh every frame), read the GUI's core at 0.89-1.16 with no A/A to set it against; it is not the one reported. Two views (not gated): every frame draws both, so a change costs two renders; a view switch copies ~60 fields and each pane's share, a few microseconds. Gates, on the line rebased onto main's screen-state terminals (mnlwlpvk): - `zig build replay -Dplatform=tui`: 113 snapshot scripts pass (the 110, views.snap, views-term.snap and views-pan.snap), their 131 tapes replay in all three replicas (393 runs, 29,436 digests); 20 monkey seeds at one view (60 runs, 14,004 digests) and 20 at three views (60 runs, 9,666 digests); no divergence, no panic. views.snap: 75 digests a replica, views-term.snap 35. core-test: 1117 of 1135, 18 skipped. - Unit tests (on qzsouslm): tty 1478 of 1498 pass, 20 skipped; GUI 1549 of 1560, 9 skipped and 2 failed, the same two that fail on phase 1's line (`tagline pointer mapping retains physical grips...` and `a tag caret past its row's end glides...`). The views' own: 8 (own cursor, scroll, mode, layout and focus; an edit and a register shared; a terminal sized by its last typist; views closing; the other views drawn leave the digest; a view keeps its place in a text another edits; a short view keeps a tall terminal's cursor in sight). views-pan.snap: a shell sized for the tall view, its prompt and last lines in the short one under an arrow, then typed into from the short one and sized for it. - test/fs.py passes (with a TMPDIR short enough for the 9P sockets' paths). test/daemon.py: 20 detach and attach cycles, then a second terminal on the attached session, both drawn, detached from inside and then closed outright, the first drawing on, and the kill and Adopt; `--gui`: 10 window cycles, then a second window on the open one, detached alone. - `pardes-monkey --views=3 --tape` found the one divergence fixed here (the drawn views' post_present). ## FreeBSD notes Groundwork for phase 5's main test, a client here and the session on the FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and runs there. The GUI is not built for it, since frontends stay local. The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is the host's root shell; port 2222 is a git jail and is never touched. Everything of ours lives in `/root/pardes-test`, nothing outside it. - **Building.** Everything, the C deps included, cross-compiles here: ```sh export PATH=$HOME/.local/bin:$PATH E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P' $E zig build -Dplatform=tui -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/bin/pardes-freebsd-x86_64 $E zig build test-install -Dplatform=tui -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files ``` The `.15.0` matters: Zig's default FreeBSD (14.0) links `libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs the test-build programs instead of running them. A test binary takes no filter when it runs, so running one test alone means a rebuild with `-Dtest-filter=` (and `--prefix ` to keep the full set). - **Deploying.** `-F /dev/null` is required here, because /etc/ssh is owned by nobody: ```sh S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519' zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all scp $S p.zst root@65.109.81.103:/root/pardes-test/bin/pardes.zst ssh $S root@65.109.81.103 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst' ``` The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh ` (a build, PARDES_* stripped, logged there), `deploy.sh [name]` and `bx.sh ''` (a command run there with env.sh sourced). `sshcheck.py` drives a session through `ssh -tt` from a local pty, and `fwd9p.py` times 9P through `ssh -L`. - **The box's layout.** Under `/root/pardes-test`: - `bin/`: pardes, and `python3` linked to python3.12; - `run/`: the runtime directory (sockets, the daemon's lock); - `home/` and `tmp/`; - `src/`: the tracked tree, which the tests need: they read docs/typ, and make their temporary directories in `.zig-cache/tmp` under the working directory; - `test-bin/test/` and `logs/`; - `env.sh`, which points every path pardes uses there (HOME carries the XDG defaults): ```sh T=/root/pardes-test export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color} ``` - **Running.** - A session: `ssh -t $S root@65.109.81.103 '. /root/pardes-test/env.sh; pardes notes.txt'`, and `pardes --attach=` the same way. - The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`. Some tests name `/tmp/pardes-*` outright and leave seven empty directories there (`pardes-arg-save` and the like): rmdir them after. - The e2e scripts, from `src/`: `python3 -B test/daemon.py /root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way. - Kill every process you start there by its pid, and check `ps -axww` afterwards. - **What FreeBSD does differently.** Each item was found on the box, and each is fixed where pardes met it unless it says otherwise: - A write to a terminal that is gone fails with ENXIO, not EIO. std does not expect that errno, and a Debug build traced it to stderr, which is the same dead terminal, recursing until the stack ran out: a session whose ssh dropped hung. Tracing is off on FreeBSD (main.zig). - A signal whose action is to be ignored is dropped when it is sent, sigwait or not, so WINCH never reached the session's watcher. A handler that does nothing keeps it (tty.zig). - waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64 bits. With Linux's numbers a command pane's exit was never seen. - There is no /proc. A shell's directory and program come from sysctl instead (kern.proc.cwd, kern.proc.pathname). - `ttyTaken` answers taken for any foreground group but the shell's, so a nested shell counts as a job. - `signalSession` stays Linux-only, so Kill leaves a command's `&` jobs running there. - File watching uses FreeBSD 15's inotify: Linux's events, with its own IN_CLOEXEC and IN_NONBLOCK. - `shm_open` refuses O_NONBLOCK; its names are not files, so there is no FIFO to fear. A `write` does not grow a shared memory object. - A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a stat by path now comes first. - A Unix socket whose listen queue is full refuses a connect with ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep that judged by connecting could remove a live session's socket while its queue overflowed. A session's liveness is now its lock: it holds `.lock` flocked from before it binds until after it unlinks (9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they unlink only under it. A connect judges only a socket with no lock, one of a pardes from before the locks. The daemon's own start was already decided by its flock. - **The tests there.** The box runs them as root, with no zls, fish or fonts installed. - main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no zls), and two tests that make a file or directory unreadable, which root reads anyway. - panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43 passed. The small programs all pass but fonts (no fonts). - Off Linux, the parts of tests whose refusing path is Linux's /proc/1 are skipped, and so is the builtins JSON check, since the committed lists are Linux builds' (Tty9p included). - test/daemon.py passes there at 20 cycles, its killed stub included (procctl's death signal). One earlier run failed at cycle 3, one process short in its count (most likely ps -e failing to read an environment). - test/fs.py runs past its `pardes -` +Pager checks now, as far as the SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty after the first frame, FreeBSD's pty fills before the session's frame and reset are written, and the session waits in `ttyout` (a binary from before these fixes too). Open. - **At 207 ms, for the replicas.** - In today's model every key is a round trip. Measured through `ssh -tt` (min, median and max): - a key typed into a file was on screen in 213, 229 and 346 ms; - a key typed into a shell pane in 218, 219 and 224 ms. That is one RTT and about 15 ms. - 9P over `ssh -L` of the session's socket costs one RTT a message: 670 ms to connect, 1035 ms for a file read (walk, open, read, read, clunk), 830 ms for a ctl write. A replica cannot fetch its state file by file. - The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L` cannot forward it (tried: EPROTOTYPE here, connect failed there), and no fd would cross anyway. The replication connection has to be a stream of its own, as the 9P socket is, which the daemon hands on locally. - A dropped connection is a hangup: the session detaches, and `pardes --attach` takes it back, as after a Detach. - sun_path is 104 bytes there, not 108. ## Phase 1 notes Built 2026-10-29 (`src/walk.zig`, `src/tape.zig`, the hooks across the core; `pardes-snap --tape`, `pardes-monkey --tape`, `zig build replay`). Decisions taken at the forks, with the ponytail lens: - **The tape is a tree recorded at the host boundary, both ways.** Each call the host makes into the core (`tape.Input`: update, serveFs, pump, perform, the shell's acknowledgements and failures, a Restore...) is recorded with its arguments; each call the core makes out -- a Host function, wrapped generically over the vtable's fields, or a syscall of its own (`tape.site`, which yields around it; every old `turn.yield()` is one) or a read of the clock or environment (`tape.value`) -- with its result. `done` and `ret` close each, so a 9P request served while a step is out lands inside that step's syscall: turn order is tape order. A replica drives the same functions and its host answers from the tape; one that makes a call the recording did not diverges on the spot, before any digest. `turn.yield` panics in a replica, so an untaped syscall is a stack trace. - **The replay is a mode of the same binary** (`PARDES_REPLAY= pardes`), not a harness of its own: an error travels as this build's number, and the code replayed is the code that recorded. Three replicas run one after another: two that draw (the first's memory held while the second runs, so it lies elsewhere) and one that never renders. The headless one steps over the presents it does not make but takes what the host told the core inside them (`acknowledge_panel_presentation`). - **Fields a host writes between calls** (`native_images`, `present_skipped`, `active`...) are found by a diff of a short list at every crossing, not by touching the frontends. - **The digest** is taken after every call the host made at the top (a pump, in the tty), as one per field of the core, its stream and each pane, its file and places, so a divergence names the field. `PARDES_RECORD_EVERY` digests after every input, nested ones too, to find the step one starts at. - **The walker needed a fourth annotation, `len`**: a fixed array written only as far as a length field says (`jumps`/`njumps`, a message's text) is undefined beyond it, 0xAA in a safe build and a fault where it holds a slice. Integers are hashed by value (a `u21`'s padding bits are not its value). - **View geometry** is `File.geometry`, a lazy cache keyed by a hash of what it is made of: revision, length, path, scroll, hscroll, the pane's size, tags and query, tables as far as read, context declarations, the output origin, and the pane's `view` (wrap, layered context, row metrics, rect height), which `sync` sets. A hundred readers a frame ask for it, so the key is one hash of those facts laid out as words (fed field by field it cost 4% of every frame). A step reading it (`wrapAt`, `wrapRow`, `ensureCursorVisible`, a body drag, `zz`) makes it on demand, and the render draws from what the step made: made afresh for every frame as well (to show an input the key missed only as a headless divergence), it cost a scroll a third walk of the rows, 4% of a PageDown. An input the key misses shows as a stale frame in a golden. - **Render wrote more than the census found**, all moved into the step: context declarations, the notice bands and their chips' columns (a click on a chip was routed by the last frame's), the header's caret reveal (`tagnav.golden` moved: after `g l B B` typed in one burst the bar stays scrolled, as a human typing it sees it), the tables' reading (a first look of 4096 lines, a count, wherever it is asked; the rest a frame's taped clock budget a tick), the tty filter palette and body chrome (derived). A view's paint-ahead stays the render's own prefetch: instead of queuing an effect it is asked of the host once the frame is presented (`Stream.askAhead`, in the pump), so a replica that draws nothing asks for none; query rows' paints count apart from it. (Asked inside the render, its worker's thread start cost a held `j` 10% in the live tty measurement.) - **Clock.** `advance`'s wakes are the recording session's (`tape.value(.wake, nextWake)`): a picture decoding or a fade asks a frame, and a replica that draws none must still take the session's ticks. The budgets of Places, Names and Tables are taped verdicts (`tape.over`), as are a message's time of day (TZ included), File.mtime and the pump's clock read (made whether or not it waits). A message's stamp and an unattended session's expiry of it read the clock through the tape too, not the session clock: that is the last pump's, and stamped by it a message written over 9P after an idle spell was as old as the spell and went at once (fs.py's `unattended_messages_expire`). - **A pointer seen through a sliding pane** is taped (`Pardes.pointerCell`): what was under it is the frame the user saw. - **The host's facts**: the session directory (getcwd, asked once at init), `$SHELL` and whether it is there, `$HOME` for `~`, a `Shell` that is not there, a missing interpreter, a directory refused above, a file deleted on disk (`File.deleted` is an input). The rest of the core's getenv (PARDES_DUMP and the XDG dirs in dump.zig, TMPDIR for kitty media, PARDES_TEST_CLOCK in the render) is the host's or authority-only. Per read rather than "captured once into session state": less code, and the same for a replica. - **Authority-only**: worker futures and their Io (a future's presence too), the 9P server's state (`fs`: fids, opens, the log; serveFs itself is replayed in turn order, 9p_io's own writes into `fs` are not), the store file every window of the user shares, which a replica keeps in memory, and Recent read from it. **Local**: the surface, presentation, frame stats, animations, the effect and event queues (drained by every pump before its digest). **Derived**: the geometry, notices, highlights, line index, tables, node trees, rail ticks, a compiled regex, an image's decoded pixels, PDF rasters. - **Terminals** replay from their taped pty bytes (they are `output` events) and are digested as their active screen, cursor and scroll numbers; their kitty images are left out until phase 5. - **Worker results**: the census's render-polled ones (image decode, PDF lanes) are derived state; the rest already arrive as events. - **Unit tests that mutate and render** without a step now step (`p.sync()`) first, as every host does; three of them had passed on the last frame's stale state: a prompt chip a body click had already closed, a `.center` landing measured on no wrap map at all (it centres by lines, so under wrap a line above can now push the cursor down where `ensureCursorVisible` takes the view back: that test is unwrapped now), and a context row count a scroll had not refreshed. The perf gate's `deep-sel` cell was one more: its view, placed from the last frame's map, stopped six lines short of the 2 MB line it selects in. In view, that line showed `File.rawAtDisplay` measuring a line's whole rest before walking to a column (20 ms a frame, Debug); it walks to the column only now, and the cell is the baseline's (91 us against 106). What proved wrong or missing in the design: - The census missed the host writes between calls, the pointer through the presentation, the render-time writes above, `Stream.start` (host code doing core work, made harmless by the futures being authority's), getcwd, `access(2)` for `$SHELL`, File.mtime's clock, and a deadlock in vaxis's queue (a `yes` stopped being read; fixed on this line, found by the new measurement). - `windowed`/`native_images` are watched host writes for now, not View state (phase 4); the semantics hash in the hello is not done (there is no hello in phase 1), so a tape from another build is not refused yet. - Deferred: the GUI's own host-to-core calls (font requests, opacity, `core.advance`/`render` are inputs, the rest not yet), PDF lanes and Handover yields (never reached by the gate; a replica would panic on them, which is the point), `Stream.runHere`'s yield (a host with no workers). The gate on the final line (on main vukknslx): `pardes-snap --tape`, 110/110 scripts, 128 tapes, 384 replica runs, 33,093 digests, no divergence; `pardes-monkey --tape`, 50 seeds of 70 steps, 150 replica runs, 37,800 digests, none; `zig build replay` (the 110 scripts and 20 seeds) clean; `snap` 110/110 with no golden changed but `tagnav`; unit-test tty, pane-test, 9p-io-test and fs-test pass, unit-test gui fails only its two known tests. Perf, base (main with perf --live and the queue fix) against this line, ReleaseFast, interleaved A B A ... A, medians of runs, the base's own spread as the A/A noise. The perf harness's gesture cells (render, keys, PageDown, wheel, edit, deep-sel) 0.97-1.03 at an A/A of 2-7%, and on the final line, measured on the shared perf cores, 0.91-0.97 at 10-31%. The live tty on a quiet machine: core time p50 0.96-0.97, keystroke-to-present p50 0.98-1.03 (A/A 1-23%), `yes | head -c 1G` 0.96, a million-line open 1.01. The live GUI's core p50 1.00-1.04 (A/A 2-18%), present-bound at the window's 30 ms. Two costs the A/B found are gone (the geometry key, the paint-ahead's thread start in the render). ## Tty throughput notes Found and fixed 2026-10-29 (`src/tui/tui.zig`, `waitInput`). - **What the 1 GB `yes` cell measures is mostly the kernel.** A pty's output processing (ONLCR, a `\n` made `\r\n`) costs the writer about 0.4 us a newline here, and `yes` writes one every two bytes. Every reader of a pty is capped by it: `script(1)` drains 100 MB of `yes` at 8.6 MB/s with OPOST on and at 650 MB/s with it off, a Python drain at 8.0 MB/s, and tmux at 5.4 MB/s. The tty pardes was at 5.7 to 6.1 MB/s, three quarters of that ceiling, before any change. The 179 s were the kernel's. - **Where pardes itself was slow:** with 200-byte lines (`yes` of a long word), the ceiling rises to 264 MB/s, tmux reaches 55 MB/s, and pardes managed 18.6 MB/s. A flat profile (perf at paranoid 2 allows no call graphs) put 98.7% of the samples in pardes: `Vaxis.render` 29%, `body_layer.renderBody` 17%, `Shell.present` 10%, memcpy 8%, ghostty's parse 1.5%. The tty drew a frame for every pty read, which is 4 KB, nearly all of them overwritten before anyone could see them (about 690k frames for the 1 GB cell). - **The fix: output is drawn once per frame tick at most** (16 ms), with whatever came in until then, as the GUI drains its queue once per vsync. Two things are drawn at once, as before: - an input's frame; - the focused pane's output within a second of an input (its echo; a remote shell's arrives an RTT later). Neither waits while it is undrawn, including when FrameAck held its frame for the terminal's answer. Snapshot runs (the virtual clock) are not paced. - **And a pane has two reads queued at most** (`readPty`): its reader waits for the loop to take one before it reads on. ghostty parses `yes` at about 9 MB/s, 0.45 ms a 4 KB read, slower than this machine's pty delivers it. So a flood filled the loop's queue (512 events): a key and its echo waited behind its reads, and which blocked writer a free slot woke was luck, which gave half-second peaks. Paced but unbounded, the wait was steady at about 20 reads, 9 ms for the key and 9 more for its echo. A frame dump lined up with each key's write showed it (PARDES_FRAMES and `flood.py` with FRAMES=). What a flood writes beyond two reads now waits in its pty; the loop could not have kept up with it anyway. - **Measured** (ReleaseFast, `.scratch/daemon/term/yes.py` through a real 120x40 pty, all interleaved, with measurements pinned to two physical cores and so four CPUs; medians of 3 rounds, MB/s): | | `yes`, 32 MiB | 200-byte lines, 256 MiB | |---|---|---| | base (before the rework) | 5.5 | 14.5 | | phase 1's tip | 4.8 | 15.0 | | pacing alone (the queue limit came after) | 4.6 | 117 | | tmux | 5.4 | 47.6 | | raw pty drain (Python) | 7.7 | 239 | | GUI, test mode | 0.9 | | Unpinned, before the pinning: 200-byte lines 18.6 to 247 MB/s (1 GB in 53.7 s and in 4.0 s), and `yes` 5.7 to 6.9 MB/s. The GUI could only be measured in its test mode (a hidden window), which downloads every frame from the GPU, so 0.9 MB/s says nothing of a real window. There is no headless compositor here, and a real window must not open on the user's session. In a later interleaved run, the change as committed against pacing alone gave 4.8 and 6.0 against 4.8 and 5.4 MB/s for `yes`, and 102 and 117 against 124 and 119 MB/s for long lines: the same within the noise. `pardes-perf --live`'s `yes` cell drew 2.3k frames for 32 MiB against 11k before, in the same time: it is kernel-bound. - **A held key while another pane floods** (`flood.py`: a key every 30 ms into `cat` in one pane of `-n 3`, timed from its write to the pty to its echo on screen, with pyte in the loop; `yes` in another pane): in one interleaved run of two rounds (p50, p90, worst, in ms): | | under the flood | idle | |---|---|---| | phase 1's tip | 3.0, 212 to 214, 445 to 470 | 2.2 to 2.3 | | pacing alone | 10.3 to 10.6, 19 to 20, 26 to 33 | 2.3 | | this change | 3.3 to 4.2, 4.3 to 5.6, 6.0 to 7.8 | 2.3 to 3.2 | The driver's own cost (pyte) is in every number. With 16 KiB queued a pane instead of two reads, the flood's p50 was 7.5 to 7.8 ms for no throughput gained, so two reads it is. - **Keystroke-to-present** (`pardes-perf-live --live`, base and this change interleaved, 5 and 4 runs; A/A between the base runs' halves): p50 against base is 0.92 for key-down, 0.89 key-right, 0.85 edit-char, 0.80 page-down and 0.93 wheel. The A/A was 0.73 to 0.92. The 32 MiB `yes` took 1.00 the time with 0.18 the frames, and the big open 1.10 (A/A 1.05). Two earlier runs of pacing alone, 3 and 5 rounds, were within their A/A too. - **Unrelated, found on the way:** `test/fs.py` failed `unattended_messages_expire` (its first `shown()`) on phase 1's tip as well; pmxpuolt on main (a message's stamp and expiry read the clock again) is that fix. fs.py also needs a short TMPDIR: under `.scratch/daemon/term/tmp`, `directory_panes`'s socket path is 108 bytes, one past the limit, and the session starts with no 9P. Two GUI unit tests fail on phase 1's tip as well (`tagline pointer mapping ...`, `a tag caret past its row's end ...`). A suite started as a shell's `&` job runs with SIGINT ignored, so four host_io tests whose Ctrl-C must stop a `sleep` fail there; started otherwise, they pass. ## Screen-state terminal notes Built 2026-10-29 (`src/screen_state.zig`), a pure encode and apply pair over byte buffers for the phase 5 stream; no transport yet. Only tests reference it, so the tty and GUI binaries carry none of it: no symbol of it is linked into the tty built from this tree. The local path's A/B, base against that tty, is the Tty throughput notes' keystroke-to-present. - **Both kinds of blob are ghostty's own snapshot records,** not a cell format of ours. The terminal-wide state (modes, colours and palette, title, pwd, tab stops, mouse modes, scrolling region) is a TERMINAL record. A row is the PAGE record of a one-row page the row is cloned into, which carries its styles, graphemes, hyperlinks, wide cells, wrap and prompt marks exactly. The replica decodes them with ghostty's decoders and writes a row with `Screen.clonePartialRowGrowCapacity`. - **A row's address is a hash of what it shows** (Wyhash of its flags, its width and its cells by value), not of its blob: ghostty keeps the source page's style ids when it clones a row, so equal rows give blobs that differ. The width is in the address, because a blank row is the same at every width. - **A delta** is a header (cursor, kitty keyboard flags, the active area's first row as an absolute row, the history's extent), the TERMINAL record's address, an address for each active row, an address for each row that scrolled off since the acknowledged state (the tail, up to a screen of them), and the blobs for the addresses the peer does not hold. An unchanged row costs 8 bytes. - **One delta per peer is in flight,** against the state it acknowledged, as FrameAck keeps one frame in flight for the tty. A slow peer skips the states in between. The peer holds exactly the blobs of the state it acknowledged, at most two screens of rows and the meta, so the authority knows what it holds without asking. At 207 ms this means at most about 5 deltas a second to the FreeBSD box; phase 6's predicted echo is meant to cover typing there. - **Absolute rows** come from a tracked pin at the last active top. The rows scrolled since are the distance to the new top; if the pin's page was pruned, the count jumps past every row the replica could hold. A resize renumbers (an epoch), and the peer starts afresh. Scrollback is fetched by absolute row (`encodeRows`, `applyRows`). - **The replica writes into a ghostty Terminal,** a terminal pane's own, so `src/terminal.zig` draws it unchanged. A test renders a replica's terminal pane and its authority's in two cores: the bodies are equal, cell for cell. - **Gates.** - The soak drives an authority with pty bytes and lands deltas at random, both skipping states and delaying a delta while the authority moves on. Resizes and fetches of holes also come at random. After each delta it checks the replica's screen against the authority's as encoded, and every history row the replica holds outside its holes against the authority's row at that number. - It passed with synthetic shell, TUI and `yes` streams at 4 seeds (in the unit tests, with a 64 KB scrollback so it prunes). It also passed with 197 recorded streams at 3 seeds each (the test that PARDES_SCREEN_STATE= turns on): the 192 pane outputs of the snapshot scripts' tapes, plus real vim, nvim, htop, less and btop sessions recorded through a pty. Its corpus and recorder are in `.scratch/daemon/term`. - **Numbers** (ReleaseFast: `PARDES_SCREEN_STATE= zig build core-test -Dplatform=tui -Doptimize=ReleaseFast -Dtest-filter=screen_state`, which runs under std.testing's checking allocator): - Encode, 200x60, every row new: p50 250 us, p99 487 us, 29.8 KB. - Encode, 200x60, nothing new: p50 71 us, p99 139 us, 534 bytes. That is the 60 rows hashed and the TERMINAL record encoded to hash it. - Under `yes` as fast as ghostty takes it in-process (9 to 12 MB/s), with a delta every 16 ms: 62 deltas a second, 63 KB/s on the wire, 1 KB a delta (60 rows and the tail's 60, all held). That is bounded by the frame rate, not the output. - Under varied text at 36 to 55 MB/s: 355 KB/s on the wire, 5.6 KB a delta. - The replica applies a `yes` delta in 2.4 ms, most of it pushing the rows scrolled past into its history (a row at a time). A delta of varied text applies in 0.24 ms under the C allocator and 0.95 ms under the checking one. - **What proved wrong in the design.** - "Scrollback fetched on demand" alone loses rows. A row that changed after the last delta and then scrolled off would sit stale in the replica's history. The tail fixes the rows that scrolled within a screen. Older ones, which the replica never saw or saw before they last changed, are holes: rows it must fetch before it shows them. - The address had to include the row's width, as above. The soak found this: a replica reused an 80-column blank row after a resize to 10. - ghostty narrows pages lazily, so an active row can be wider than its terminal: only the terminal's columns are hashed and encoded. - **Not done, or a gap.** - Kitty images in terminals. - pardes's own state derived from the stream is not carried: its `MarkedHandler` prompt phase, command-output pins and the replies it writes to the pty. ghostty's prompt marks on rows do cross. - The view's scroll offset is per client and is not sent. - A replica's core must not resize its pane's terminal itself; the authority sizes it. - The replica's history extent follows its own pruning under the authority's limit, so it can be a page off, and its oldest rows can be ones the authority has dropped. - A fetch sends every row as a blob (phase 6's cache will send addresses). No zstd. - A replica pushes a flood's rows into its history one at a time. - The integration into the stream and the cores (a replica pane's State fed by `apply` instead of `.output`) is the next piece. ## Phase 5 groundwork notes Built 2026-10-29: the delay relay (`test/delay_relay.py`), the replica socket (`src/daemon.zig`, `src/detached/wire.zig`), and two measurements for open points 3 and 5 (`zig build rollback-bench`, which runs `test/rollback_bench.zig`, and `test/fork_stall.py`). Decisions taken at the forks, with the ponytail lens: - **The delay relay is a Python script.** Each direction is a reader that stamps every chunk with a due time and a writer that sends it then. The due time is the chunk's arrival plus the delay. With a rate cap, it is the moment the link has carried what came before, plus the delay. So throughput is not serialised behind the delay, and the bytes stay in order. Either side is a Unix or TCP socket. ponytail: its buffer is unbounded, which makes a link with an infinite bandwidth-delay product; on a real link, ssh's window is the bound. - **Two sockets, where the design had "the daemon's one".** The fd-carrying SEQPACKET socket stays local. Once the daemon holds its lock, it binds `pardes-replica.sock` (SOCK_STREAM) beside it, and unlinks it at exit. `ssh -L` forwards that one. - **The daemon judges the hello, so a session never sees one it refuses.** - Every session's registration carries its semantics hash (daemon protocol 2), and the daemon compares the client's with the session's. - It refuses in words: another protocol; another build, naming both hashes and pointing at `pardes --version` to compare them; or a session nobody has. - It reads exactly the hello's length (the header, then the payload), so whatever the client sends after it is left for the session. - Up to 16 hellos may be on their way at once. ponytail: a client that never sends one holds its slot until it hangs up. - **The welcome comes from the session**, with its pid and name, over the connection it was handed. That tells the client the handoff happened. The test checks that the session's fd count went up and the daemon's did not. - **The semantics hash** (`wire.semantics`) is computed at comptime and covers: - the protocol, the release and the commit; - the platform and the build options; - every dependency the manifest pins, by its content hash (build.zig's `deps` option). The design said grammars, but ghostty and the Unicode tables also make a core step differently, so it takes all of them; - the walker codec's assumptions (pointer width, byte order). It leaves out the OS. The main test is a Linux client of a FreeBSD session, and what differs between those is the host's, which the tape carries. The two builds of one tree hash alike (1087257bf166588a on both), and the box's session welcomed the client from here. ponytail: an unstamped build (Debug with `-Dplatform`, so every test build) has no commit, so two dev trees with one release and the same options hash alike. - **The hello lost its grid**: a replica's view takes a resize like any other input. A ping (client tag 0x03) and three server tags (welcome, refused, pong) are new. The wire is version 14. - **The session's half is a stub**: a thread per replica that sends the welcome, answers pings, and ends at bye or a hangup. Measured. Timings ran alone through the perf lock, pinned to cores 6-7 while builds ran niced on the others. The local A/B ran interleaved, with an A/A run beside it. - **The relay's self-check** added this much over a direct connection, at each delay (median RTT, then throughput through it): | one-way delay | wanted | added | throughput | |---|---|---|---| | 15 ms | 30 ms | 30.26 ms | 3.2 GB/s | | 40 ms | 80 ms | 80.27 ms | 2.3 GB/s | | 103.5 ms | 207 ms | 207.26 ms | 2.3 GB/s | Direct, the same pair gave 9.9 GB/s. The extra 0.13 ms each way is the threads' wake-ups. A 20 MB/s cap held at 19.98 MB/s. - **A replica's round trips.** Each row is a connect and hello, then 50 pings, to a detached session of this build. The session's thread answers each ping. | path | hello | ping min / median / p90 | |---|---|---| | direct (Unix socket) | 0.26 ms | 0.01 / 0.01 / 0.01 ms | | relay, 30 ms RTT | 30.95 ms | 30.26 / 30.34 / 30.40 ms | | relay, 80 ms RTT | 80.73 ms | 80.25 / 80.31 / 80.39 ms | | relay, 207 ms RTT | 207.62 ms | 207.22 / 207.35 / 207.41 ms | | FreeBSD box, `ssh -L` | 411 to 476 ms | 203.2 / 204.9 to 210.4 / 210 to 221 ms | The box's rows cover six runs over two passes. In the same passes, a TCP connect to the box's port 22 took 203.8 to 205.3 ms at best and 206.4 to 208.3 ms at the median. The relay rows are the second pass; the first pass had p90s up to 5 ms worse. So a ping through `ssh -L` and the daemon's handoff costs the RTT and nothing more measurable. The hello costs two RTTs: ssh confirms the channel before it forwards a byte. A refused hello over the link came back in 412 ms, also two RTTs. That is the floor phase 5 builds on. A replica connects once, so it pays the second RTT once. - **Open point 3, rollback: the walker copy.** `zig build rollback-bench -Doptimize=ReleaseFast` builds two sessions in process: - typical: a shell with 1 MB of scrollback, three 60 KB files and a 3 MB one, 40 edits; - big: 14 panes, an 18 MB file, four 3 MB files and eight small ones, 8 MB of scrollback, 200 edits. For each, it measures the state the digest walk visits (what speculation can write) and the share of it that is blobs. Blobs are a file's text and its undo snapshots. `File.setContentSpan` puts a new buffer in place of the old one at every edit, and a snapshot is a copy, so both are immutable once made and a copy can pin them. The walk is then timed again with them taken out. | session | walked | of it blobs | left to copy | walk | walk, blobs out | copy (walk + memcpy) | |---|---|---|---|---|---|---| | typical | 36.3 MB | 36.2 MB | 80 KB | 3.1 to 3.8 ms | 0.019 to 0.023 ms | 0.020 to 0.024 ms | | big | 282.5 MB | 282.4 MB | 96 KB | 23 to 27 ms | 0.043 to 0.044 ms | 0.044 to 0.045 ms | A copy is about 0.1 to 0.3% of a 16.7 ms frame, so the walker wins: arena copy-on-write would buy nothing. Three conditions come with it: - a blob a copy holds must not be freed until the copy is let go, whether by a refcount or by a free list drained at confirmation; - caches derived from a blob (`line_starts`, highlights) are also made anew at each edit, so they can be pinned with it, or remade on demand, since the geometry is keyed by revision; - a terminal's history is not in the walk, because under screen-state terminals it is the authority's pinned rows. The walk with the blobs out is the stand-in for a copy walk: phase 2's dump walk is the real one, and this step re-measures it when it lands. - **Open point 5, the join writer: not a fork.** The authority's stall, as median fork time, with 64 MB written by the authority while the child lives: | written memory | Linux 7.2 (THP always) | FreeBSD 15.1 | |---|---|---| | 100 MB | 1.8 to 2.7 ms | 4.1 ms | | 600 MB | 8.6 to 10.8 ms | 14.0 ms | | 1 GB | 13.3 to 19.6 ms (max 24) | 20.4 ms (max 26) | | 4 GB (one Linux pass) | 62 ms (4 KiB pages: 162) | 52 ms | | then 64 MB written under the child | 26 to 32 ms | 19 to 27 ms | Linux is two passes on the pinned cores, beside the builds; the box was idle. The big session forked in process at 431 MB RSS took 3.4 and 8.3 ms. The user's windows hold 450 and 630 MB. The gate, "a join costs the authority under one frame", fails twice: - fork itself, at 1 GB on both systems (and at 600 MB once a frame is 8.3 ms, at 120 Hz); - copy-on-write after it. Every page the authority writes while the child streams costs a fault, about 1.3 µs per 4 KiB page. The child lives as long as the dump takes to cross, which is seconds at 207 ms. So the join is the same walker copy as rollback: tens of microseconds at a step boundary, blobs pinned. A worker thread then serialises the copy and streams the pinned blobs, which are immutable and so need no lock, while the authority buffers the events after it for the joiner. This is the coordinator's alternative, "a metadata copy serialised on a worker". The authority pays the copy, and the blobs stay pinned while they stream. - **No local change.** tty startup and keys, the phase 1 tip (a4b85cb7, ReleaseFast) against this, both live at once with their samples alternating; the p50 ratio new/base: | | three A/B runs | two A/A runs | |---|---|---| | startup | 1.040, 0.965, 1.024 | 1.053, 0.978 | | key in a file, to its frame | 1.084, 1.025, 1.027 | 1.025, 1.013 | | key echoed by bash, to its frame | 0.996, 0.972, 1.010 | 1.015, 0.998 | The daemon and the hello are off the hot path, and every ratio is within the spread of the A/A runs beside it, whose second slot reads 1 to 5% slow against the same binary. So there is no measurable change. The absolute times are not comparable with phase 3's 4.73 ms: the startup p50 was 6.5 to 26 ms here, on two cores shared by both live sessions and the driver, with builds running on the others. What proved wrong in the design: - "`ssh -L` of the daemon's one socket": there are two. The one that carries fds cannot cross ssh, and no fd would cross anyway. - "The authority forks a writer child": fork's page-table copy is over a frame at 1 GB on both systems, and the copy-on-write tax lasts as long as the join streams. Once blobs are pinned, a copy of what speculation can write is about 100 KB, small enough for a join to take one too. - "When the first speculative event goes pending, the replica copies views, buffers, ...": a buffer's text never needs copying, since every edit replaces it. Only its handle and the views are copied. - "The hello carries a semantics hash (version, features, grammars)": it hashes every dependency, and leaves the OS out on purpose. - Pre-existing on the phase 1 tip, and not this line's: test/fs.py's `mounted-startup` session hangs at startup (its main thread waits on a futex), the same with the base binary, and still on main after phase 4 (Phase 5 part A notes). A ReleaseFast `exit_answers` run loses its Rclunk race about 1 time in 7 on base, and 1 in 40 here. ## Phase 5 part A notes Built 2026-10-29, in one process and with no transport: the rollback copy (`src/rollback.zig`), the replica engine (`src/replica.zig`), the tape's speculation mode (`src/tape.zig`), and an in-process harness (`test/replicas.zig`, `zig build replicas`). `zig build rollback-bench` now times the real copy. Decisions taken at the forks, with the ponytail lens: - **The copy saves bytes and puts them back where they were.** Its walk is the digest's (src/walk.zig): views, buffers, registers, counters and terminal metadata. It saves the bytes of every allocation the walk reaches, and puts each back over its allocation. That works because the replica core's allocator (`rollback.Epoch`) changes three things while a copy is held: - a free of anything older than the copy is put off; - growing such an allocation in place is refused, so the core moves it; - anything allocated meanwhile belongs to the epoch. So every pointer in the saved bytes still points at what it pointed at when the copy was taken. Putting the copy back frees the epoch's allocations and forgets the put-off frees. Letting the copy go makes them. Nothing has to know who owns what. - **Three blobs are pinned instead of copied** (4 KiB and up): a file's text, its line index and its undo snapshots. An edit puts a new one in place of the old (File.setContentSpan) and never writes one, so the copy keeps a reference and the put-off free keeps it alive. A safe build checks every pin is unchanged when the copy goes. - **More than the digest's fields are saved.** Within a struct the copy saves every field: the derived caches, and this process's own state too. It skips the queues a step drains (asserted empty) and the frame statistics, and cuts an array at its `len`. Both were measured wrong first: - a safe build's `free` writes 0xAA over what it frees, so a cache the speculation freed came back as that (a line index full of 0xAA, then a crash); - this process's own fields (the frame arena, the surface) point at what the epoch then frees. For the same reason, file texts and line indexes are now freed raw everywhere (`memory.freeRaw`), as setContentSpan already did. - **A speculation is checked against the authority's tape of the same input.** The replica applies its own input as the authority will (an update, then a pump) and records it with its own tape. If the authority's unit for that input has the same records (inputs, host calls and their answers, digests aside), the head goes: that is the fast path. The digests are left out of the comparison because the starting states are equal by induction. A digest checks every unit a replica replays. - **Each client has its own view** (src/views.zig). The authority goes into the client's view before its input (`toView`, which is taped), and the replica's speculation does the same. So a unit records the switch only when another view was live, alike on both sides. The views not live (`parked`, each pane's `pv`) are this process's own fields, and the copy saves them with the rest. - **A host call or a syscall stops a speculation, but cannot stop it midway.** Zig has no unwinding. The call is answered with a harmless value (an error of its set, null, zeroes, or a union's first zeroable case), the input runs to its end, and then it is undone: the copy is put back and the earlier pending inputs applied again. That input, and those after it, wait for the authority's answer. A read of the clock or the environment (`tape.value`) is a guess made locally, which the comparison checks. `watched` is answered false, as the authority's host says too: a replica draws outside the step. - **An emulator is the authority's.** Its writers in terminal.zig (feed, resize, scroll, follow, create, deinit) stop a speculation before they write. A safe build asserts that a clean speculation left every terminal's digest as it was. Tree-sitter's trees live on the C heap, which the epoch cannot see. While a copy is held, `syntax.held_trees` puts off their frees and frees the epoch's own. - **The view's events wait** (a resize, the pointer): phase 4. - **Messages are bytes.** A client's input is a u16 client id, a u32 sequence number, and the event as detached/wire.zig frames it. A unit is the input's origin (0 for the authority's own: a pty's output, a setup), its sequence number, and the authority's tape of it. - **The harness** is a discrete-event simulation. One authority and three replicas share one view, each client typing monkey.zig's mix (keys, typing, pastes, a few commands, resizes and clicks) at 50-200 ms with pauses. Each link takes half the RTT plus up to 10% of it in jitter, in order. The session is the tty's own boot (a raw shell over a text pane) with three files looked at. The authority's pty echoes what it is written 2 ms later, which makes units of its own. Found on the way: phase 1's replay ran Debug, whose 0xAA fill made every read of unwritten memory alike in two cores. In a release build they read what the heap had, and replicas diverged at once, from: - `col_panes`, digested whole and undefined past each column's panes (now zeroed); - `restore_buf`, the same (now zeroed); - `terminal.State.reply_dropped`, never set, since `create` leaves fields as the heap had them; - `Text.sels` and `PendingPipe.ranges`, digested past their length (a `len` now cuts a slice too). `PARDES_REPLICA_FILL=0xff` fills a replica's allocations, so that what is left shows at once. Any replica is a release build, so these mattered. Measured on main with views (lyntonns). The ReleaseFast runs were alone through the perf lock; the correctness sweeps ran beside builds. - **Correctness.** Three clients each typed 100 inputs per seed, at RTTs of 30, 80 and 207 ms. No replica diverged, and every replica ended on the authority's digest, in all three runs: - Debug: 12 seeds, a view each; - ReleaseFast with `PARDES_REPLICA_FILL=0xff`: 20 seeds, a view each; - ReleaseFast, timed: 20 seeds, a view each and in one shared view. - **What happened to the inputs, and what it cost** (ReleaseFast, a view each; times in µs, p50 / p99): | RTT | rollbacks per input (mispredicted + another's) | speculative apply | copy (1.3 MB) | rollback | |---|---|---|---|---| | 30 ms | 31% (106 + 1745) | 35 / 183 | 75 / 198 | 342 / 818 | | 80 ms | 76% (257 + 4285) | 54 / 168 | 94 / 447 | 455 / 1318 | | 207 ms | 129% (423 + 7335) | 50 / 174 | 91 / 373 | 569 / 1561 | - Of 6000 inputs a run, 259 to 348 stopped at a host call and waited, and 434 to 449 were the view's own (resizes and clicks). - In one shared view, the copy is half the size (0.66 MB), the costs are half (copy 37 / 152, rollback 184 to 259 at p50), and the rollbacks are the same: 30%, 70% and 121%. - The worst rollback, 1.6 ms, is a tenth of a 16.7 ms frame. A speculative apply is 0.2 ms at worst. - **The copy alone** (`rollback-bench`, ReleaseFast, single view): | session | saved | pinned | take | put back after an edit | |---|---|---|---|---| | typical (5 panes) | 980 KB | 36.6 MB | 0.044 ms | 0.036 ms | | big (14 panes) | 1.49 MB | 286.5 MB | 0.071 ms | 0.051 ms | Letting a copy go costs nothing measurable. What proved wrong: - "Views cut the conflicts": they cut what a rollback shows, not how often one happens. Any unit of another client's, in any view, comes before this replica's pending inputs in the authority's order, so the copy goes back and the pending inputs are applied again. Each costs a third of a millisecond, a tenth of a frame at worst. Telling which units commute with which pending inputs is the next lever, not a needed one. - "A copy of about 100 KB": it is 1 to 1.5 MB. The core struct itself is most of it (the presentation, the surface, the settings and the column tags), plus the views' parked state, now that the copy saves all of a struct and not only what a digest takes. It is still under 0.1 ms. - "A misprediction is rare": every key that leaves a message stamps it with the time of day (`HH:MM:SS`). So a key confirmed in another second than it was applied is recorded differently: 2 to 7% of inputs here. In the harness both cores share one clock, so this is the floor. Across machines an offset between the clocks raises it. Part B should guess the authority's clock (its last confirmed reading, plus the time since locally), not read its own. - views.zig left a pane's slot for the live view unwritten, and a slot holding the live body's selections. A pane closed before that view parked again freed whatever its slot held. This is fixed in its own change, before this one. Every replica switches views on every input, and found it at once. Not done, and why: - No transport: part B. - The view's events (a resize, a click) still wait for the authority. With views they could be applied at once. - A pty write stops a speculation, so a shell's echo is shown late. Phase 6 predicts the echo. - The authority's own units (a shell's echo) roll back a replica that has pending inputs. - The epoch allocator is single-threaded. A replica core's workers (images, search) must allocate elsewhere, or hold a lock, in part B. - A replica keeps a timing sample for every input it applies (the harness reads them). A long-lived one keeps only counters. - test/fs.py's `mounted-startup` still hangs on main (lyntonns). The same hang shows with main's own binary in a workspace of its own, so it is not this line's. ## Phase 2 notes Built 2026-10-29 (`src/walk.zig` Dump/Load, `src/dump.zig`, terminal.zig's dump hooks; `pardes-snap --dump`, `pardes-monkey --dump`, and both in `zig build replay`). Decisions taken at the forks, with the ponytail lens: - **The dump is the digest's preimage.** It walks exactly what the digest walks (local, derived and authority fields stay out), so `digest(load( dump(x))) == digest(x)` is the definition of complete, and the gate. The dump is `magic`, a table of every type the roots' walk reaches (each struct's and union's fields by name with their types' places, each enum's names and values, each number's width), then the values as the tape lays them out but for an enum or union tag by its place in the table, and a pointer or slice met again written as a reference to the first. The table is a comptime graph of the roots: a session's is 324 types in 23 KB, of a fresh session's 96 KB dump. - **Crossing builds: by name, and positional when it need not be.** A table byte-identical to this build's is read field after field, no name asked (the join's same-build path: no separate strict format). Any other is matched by name: a field added takes its default, one removed is passed over, an enum value by its name (Builtin's ordinals shift with every builtin added), numbers widened or narrowed by value, arrays as far as they fit, one that came to hold a `len` read either way. A field whose kind changed takes its default and is said; with no default the value it is in is refused, and an element of an array of optional pointers (the session's panes) is then left out with its place in the layout (`dropMissing`), never more. What was let go is the session's first message after the load. A pane whose terminal another build could not read is a fresh terminal (`ensureKind`). - **Roots, not wrappers.** A session's dump is five roots: the core, read *into* a core `Pardes.create` made (its local, derived and authority fields stay create's: no defaults to invent for an arena or a host), its 9P mounts (state the user made, inside the 9P server's authority `fs`), which file panes' files were on disk when it was made (one gone by the load is said deleted, so Del asks: its text is nowhere else; one never on disk comes back as it was), the theme's name, and the views not live. A pane's dump (the store's, a handover's) is a Pane alone. - **Every view is in the dump** (decided after phase 4 merged). The live view is the core's own fields, so `view` and `views_open` were dumped already, and a session dumped with several views loaded into a core that said so with no parked slot: its first view switch, close or drawing of the others unwrapped a null. Leaving them out would have needed `load` to fold them anyway, and the join (phase 5) needs them: a replica applies the other clients' `toView`. So `views.dumped` writes each open view but the live one (the live one's slot is stale by design): its share of the session but what drawing keeps (`Pardes.replica.local`, computed from `views.fields`: its panels as presented, its animations), its slots' serials, and its share of every pane (`pv_known` says which shares own their selections, not which exist: a parked view has one of every pane). `views.undumped` parks them again in `load`, last, so nothing after it rolls back what it moved; a pane a view has no share of gets the live view's, a view the core says open that the dump lacks is closed, a dump with no views root (before this) has one view. They stay out of the digest as before (parked views are checked when live again); the gate is the re-dump byte for byte, now also on the three-view monkey seeds. A session *revived* for one client (Restore, `-l`) is one view, the live one as view 0 (`views.only`, what the hosts already assumed), and a pane carried or reopened takes the view it lands in (`sized_by`, `prompt_view`). Cost: the views root is empty for one view; with several, a share is about 100 bytes a pane a view. - **Aliasing by reference.** Zerox twins share their text and undo again, an `inherited` cwd points at its pane again. Keyed by address, length and type, which is also what made twins dump their text once. - **Terminals are ghostty 1.4's snapshot** (`terminal.State.dumpWrite`): both screens, history, styles, links, and the kitty images with their pixels and placements (the known gap in test/kitty.zig closed), plus a walk dump of pardes's own parts (unsent replies, the view, the OSC 133 command cycle, its output pins as screen points). The snapshot is at the ground state: a sequence the program had begun at the dump's moment is lost (ponytail: the stream would have to track its continuation, a cost on every pty read). ghostty calls its format a work in progress: a ghostty bump that changes it makes a terminal another build cannot read, which degrades as above. - **Clean files' text is in the dump.** A join cannot read the authority's disk, and the session is what it holds. A file changed on disk since comes back as the watch a load emits finds it (as on open: a clean pane reloads, an unsaved one is marked changed), not by a read in the load. The store keeps a closed pane's record under 8 MiB by a slim form (no terminal history, no undo, a clean file's text read again on reopen), as it kept terminals without history before. - **Load and revive are two halves.** `load` makes the core and nothing else (what the gate checks, and what a replica takes); `revivePane` and `reviveSession` give it to a host: a new shell under each terminal's history (alternate screen left, keys, mouse and paste modes reset, the banner; the old shell's REPL, Tty+ word and sent command dropped; a command pane finished), watches, PDFs opened again (MuPDF's document is derived: `Document.handle` is optional now), Recent and the store, the ThemeFile asked for again, the startup config. The screen stays the dump's until the host resizes: setting this process's `opts` size first reflowed twice and lost where a terminal's view was. - **`opts` is local** (how this process was started: allocators, socket names, store path), and so are `dump_out` and the effect and input queues' counters (the queues were local, their heads and lengths were walked: a loaded core read a garbage effect). Pipe's runs, an Edit's, Handover's dump in flight and its carrier, and the host's fallback file table are authority: host errands, not state. - **Restore keeps serials.** Renumbering would rewrite every serial the session holds (attachments, query owners, places, messages, LSP and callstack requests); Restore hangs up every connection anyway. The next serials are past the old session's; `restored`/`restoredcol` records are gone. A Restore over an older session keeps the host's facts (cell pixels, row metrics, native images) and the higher of the request sequences. - **A failed load leaves nothing**: the loader records each allocation it made and frees them all (`rollback`), and a `dumpFree` for a custom type's insides. ghostty's snapshot decode itself leaks when an allocation inside it fails: the OOM Restore test runs without a terminal. - **Handover** dumps `Carried` (the kind, a column's tag, each pane's dump, weights, owners) behind `pardes-carried\n`; a new window's `-l` of it is an empty core that adopts the panes, its settings this process's (the old carried dump brought the source window's theme and settings along). Panes carried now keep their undo. - **No v1 reader.** A ZON dump is refused before anything is replaced (`not a pardes dump: it does not begin as one`); a ZON record in the store is an old build's session's, which no new one lists (below). Dumps are `.dump` files now. What proved wrong or missing in the design: - The walker's list rule (exactly `items` and `capacity`) never matched in a safe build: Zig 0.17's ArrayList has a `pointer_stability` lock there, so phase 1's digest hashed every list's capacity and a dump wrote it. Both take a list's items now (`walk.isList`). - Merged onto phase 5 part A (sqsnvoux): a `len` may cut a slice now (a text's room for its ranges, made whole and used `nsel` deep). The dump writes such a slice whole, room and all, and a load gives it the room it had; only one of plain numbers may be cut so (every bit pattern a value past the cut), else no build. - Phase 4's other views lost their selections whenever they were drawn: `unpark` hands a slot's selections to the body (main's fix for a double free) and `drawOthers` never gave them back, so a drawing session took them from every view not live while a headless replica kept them. The three-view monkey seeds with the headless replay found it (seeds 5 and 35 of 50 diverged at the next key typed at several cursors); `drawOthers` gives them back after each frame. - "pty masters passed as fds where Handover passes them today": Handover passes none (it refuses terminals: their shells are the window's children), and phase 3 made Detach happen in place, so no dump carries a master. A loaded terminal gets a new shell; the daemon's Adopt is the way back to the old ones. - There are no crash dumps: crash.zig records the panic message only. None was added: a panic handler would walk state that may be what broke and allocate inside the handler. The path that exists is Dump, rebuild, Restore, which the by-name load is for (a session read by name is a unit test over the whole type graph). - Two behaviours of the old dump were its approximations, and goldens moved to the real session: a terminal's view comes back where it was (load.snap's old view was not the dumped one), its prompt rows stay hidden in text mode (OSC 133 marks are in the snapshot), its Filter, a pane's message and typed text over its history are kept, and a query pane comes back a live query, not plain text. - The web shell had not built on main for a while (u64 counts on a 32-bit target, threads, stb_image and the tape's host calls). It builds again and loads a native build's dump by name: `-Ddump=` embeds test/web-snapshots/touch.dump, made by tools/web_fixture.py, in place of the hand-written ZON fixture. A field this build compiled out (void on wasm) is passed over, and the theme goes by its name, a fourth session root, since the theme table differs between builds. web-snap's touch check passes; its drift was the test's (13 tag-layer fields, helix's retuned colours, and a pane resized by its grip, there being no rule between panes to drag). The web shell's own `chromeTarget` still calls an upper pane's last row a handle, which the mouse code no longer resizes on. - Not done: the website's replay (pardes-website/replays/00) still runs its July wasm; the dump it was built from is gone. `node --test test/web_driver_test.mjs` fails "CDP closes a stalled connection after its deadline" every time, a test of the harness's CDP client, not of pardes. A dump is read whole, not decoded as it arrives (the join can stream it; decoding as it arrives needs a reader over a stream, not a slice). Native byte order: every host pardes runs on is little-endian. Upgrading from a build before phase 2, a user loses: - A ZON dump: Restore and `-l` refuse it, the session they are in unchanged. A pane carried between an old window and a new one stays where it was (a failed handover). - Nothing in the store. Its framing and records are as they were; only a closed pane's dump inside its record changed. A closed pane is listed only by the session that closed it (a session's id is new each process), so a new build never lists, opens or reports an old build's records, and an old window still running brings back its own. Both builds' compactions keep the records of every running session and drop those of sessions that ended, as before. Checked on a copy of the user's real store (14.4 MB: Recent's 234 paths, 140 ZON records of 8 sessions, three of them running), the copy under the workspace's scratch, the real one never opened: a new build starts on it with no error or message, /recent lists the same 200 entries as the base build's on the same copy, Panes shows none of the old records, a pane closed and brought back comes back. A compaction forced by a record cut short (as a writer that died leaves one) gives the same file in both builds but for each one's own records: the running sessions' 72 records byte for byte, Recent folded to the same 200, the ended sessions' 68 dropped. The base build then runs on a store the new one wrote, with its walker records in it, without an error. One cost moved: a closed small pane's record is 14.5 KB, not 1.5 KB, most of it the type table each self-describing dump carries (ponytail: bounded by 64 a session and the compaction; a table once per store generation if it matters). The gates on the line rebased onto main with views and phase 5 part A (sqsnvoux). `zig build replay -Dreplay-seeds=50`, with `--dump` in all three parts: the 113 scripts (132 sessions, 29,430 digests over their first, second and headless replicas), 50 monkey seeds of 70 steps (37,722 digests) and 50 at three views (29,148), no divergence, and every session's dump at its end loaded to its digest (all 232, field by field) and dumped again byte for byte (14.4, 5.4 and 6.7 MB of dumps). The snapshots pass with five goldens moved (above). `zig build replicas -- --seeds 12` (Debug, three clients, 30/80/207 ms): none diverged or unequal. unit-test tty passes; unit-test gui fails only phase 1's two known tests; test/fs.py and test/daemon.py (20 cycles) pass; the web shell builds and its touch check passes. The cross-build check is walk.zig's test of two hand-written versions of a struct (a field added, one removed, one renamed, enums renumbered, a width changed, a pane whose required field changed kind left out), both ways, and dump.zig's of a real session read by name. The real-store check (above) holds on this line. Measured (ReleaseFast, the base binaries from before the rework against this line, interleaved A B A ... A on the perf cores, medians, the base's own spread as the A/A): the perf harness's core cells 0.92-0.99 of base (p90 0.80-1.03) at an A/A of 15-70%. A session of 16 small files, a 100 MB file and a terminal with `seq 1 3000000` of scrollback, through 9P in a real pty: | | base | new | A/A | |---|---|---|---| | Dump, the big file clean | 8.8 ms, 0.2 MB | 83 ms, 105 MB | 21% | | Restore of it | 69 ms | 105 ms | 4% | | Dump, the big file unsaved | 514 ms, 140 MB | 191 ms, 210 MB | 1% | | Restore of it | 1845 ms | 310 ms | 0% | | a closed pane kept: the terminal | 131 ms | 143 ms | 4% | | the 100 MB file | 433 ms | 127 ms | 3% | | a small file, after the 100 MB one | 9.6 ms | 17 ms | 44% | | a small file, alone | 1.0 ms | 1.0 ms | 7% | A clean file's text is in the dump now (what makes it complete), and an unsaved one's undo (the 100 MB file's one edit keeps the text before it); binary, not base64 in ZON, so the rest is faster. The small close after the big one is not the small pane's: it is the kernel freeing the 100 MB pane's memory on the 9P connection's thread (panes are freed two syncs after they close), 208 MB against the base's 108, since the restored unsaved file's undo holds a copy of its text (found with the access tracer, perf and schedstat, .scratch/daemon/trace). A small close alone is level (three after two others: 1.0 ms both; an isolated one, median of ten: base 0.77 ms, new 0.92). Of the 0.15 ms left, two thirds were outside the walk and are gone: savePane kept the whole 88 KB Pane aside and put it back, and built a 41 KB empty undo history when it was not slim; it saves only the fields it changes now. The type table is made once a process (it was made at every store record: 123 µs to 33 µs a small pane's record in Debug). ponytail: Store.close's CRC32 over the record is most of what is left, and a record repeats its 14.6 KB type table; a table once per store file would shrink both, a change to the store's format left until a close's cost matters. The live keystroke cells were not run (perf rule: `--live` only for a change to terminal output); a dump is off the hot path. ## Phase 5 part B terminal notes Built 2026-10-29: screen-state terminals wired into the replicas (`src/screen_state.zig`, `src/replica.zig`, `src/tape.zig`, `src/terminal.zig`; `zig build replicas`). Transport-free: bytes in, bytes out, proven in part A's in-process harness; the remote agent's stream carries them. Decisions taken at the forks, with the ponytail lens: - **What crosses.** The authority's units, as in part A, but a pty's output is recorded without its bytes. Then, per peer, its terminals' messages. Each one is u8 pane, u32 the pane's serial (a slot's next pane is another), u8 kind, and the kind's bytes: - to a replica (the remote's 0x06): 1, a delta; 2, rows (`encodeRows`); - to the authority (0x04): 1, an acknowledgement (u32 seq); 2, a request for rows (u64 lo, u64 hi, absolute); 3, a delta refused: start this terminal afresh. - **Where they go in the stream.** Each replica steps every unit on its own core, and a step in a terminal pane reads the emulator (rows, cursor, modes, marks). So before the authority steps any input that is not a pty's output, and once a frame, it sends a peer its own steps since the last unit (`Authority.pending`, below) and then what the peer is owed of the terminals (`Screens.flush`); after a step that read a terminal's history, before its unit, the rows the peer lacks (`Screens.fill`). A replica takes them in that order between units, and a step it replays reads the terminal its authority's step read. Nothing goes after an output step: a flood is paced by the frame rate, not by pty reads. - **The authority's own steps are batched.** One unit a pty read was 2000 units a second under `yes`: each rolled back every replica that had pending input, and each carried a digest of one word a field, about 12 KB (28 MB/s a peer). `Authority.output` steps into an open batch without a digest; `pending` sends the batch as one unit with one digest, before any other input is stepped and once a frame. Under screen state a digest is one word (the harness's trace mode keeps the fields, to name one). Units under the flood went from 28 MB/s to 0.1 MB/s a peer. - **Several deltas in flight, against the version last sent.** Each state sent is a version of its terminal (`Track`), made once for every peer: a hash of each active row, the TERMINAL record, and what pardes keeps of the stream (the OSC 133 marks, `terminal.marksOf`). A peer's delta is against the version it was last sent. The stream is ordered and loses nothing, so the peer has that one by the time it applies the next, and the replica holds exactly that version's blobs. Up to 32 are in flight (13 at 207 ms and 60 Hz). A peer at the bound gets nothing until an acknowledgement frees room, then the newest version, the ones between skipped. Peers at the same base get the same bytes (one encode for all). A delta against a base the replica does not have (one lost, or it started afresh) is refused, and its acknowledgement asks for a reset. - Rejected: mosh's base at the last *acknowledged* state with a throwaway number. On a stream that loses nothing it only makes deltas bigger and the replica keep more. - **The replica's terminal is the authority's, to the row.** - Its history is exactly as long as the authority's: no byte limit of its own, trimmed to the authority's count at each delta. Row numbers and the scrollbar then say the same on both sides, which steps read. - It takes the authority's viewport, its cursor's semantic content (a prompt's input, which `promptInputReady` reads) and its marks. - Its emulator is written by `apply` and fetched rows alone. A replica's core never ingests, resizes (the authority sizes it; a resize's own core work, its pointer selections cleared, still runs) or replies to its program; its pty writes are off the tape on both sides. - No digest covers an emulator under screen state (`State.digest`); the harness compares screens instead. - **What an ingest tells pardes is taped.** Over a terminal, pardes keeps positions: the text-mode cursor, selections, the edit copy, the pointer's rows. An ingest moves them when it prunes or erases history or clears the screen. The authority records what it told them (each tracked row's row after it and whether it was erased, the clear, whether the pages lived: `Site.positions`); a replica, which ingests nothing, takes that, and the rest of the logic runs alike on both. - **History a step reads is sent before its unit.** Scrolled past more than a screen at once between deltas, rows are holes on the replica, blank until fetched. A step in text mode reads a terminal's whole grid, and the replica's replay of that step must read what the authority's read, which a fetch an RTT later cannot give. The authority keeps each peer's holes as the replica does (`Peer.holes`, from what it sent), and after a step that read a terminal's rows (`State.read`, set outside the render) sends each peer those it lacks, before the step's unit (`Screens.fill`). A history row never changes, so each goes once. Scrolled into view, holes are blanked and asked for (`Replica.screenRequest`); the answer fills them an RTT later. - **A flood costs the replica a counter, not its rows.** When a delta scrolls more than the replica's screen and history hold, nothing it holds is in the new state: its rows stay where they are, renumbered, the active ones and the tail written from the delta, the rest holes. Growing and trimming a history's rows one by one was the 2.4 ms a `yes` delta took. - **Speculation stays off terminals.** An input whose live pane is a terminal is not speculated: its step reads the emulator, the last delta's here and the pty's newer one there. It waits for the authority (a key in tty mode already did, at its pty write). Phase 6 predicts the echo. - **Rollback never brings a terminal's past back.** - The rollback copy never held an emulator (State is skipped), and a replica's emulators, their State included, are allocated on the C heap, outside its rollback epoch (`terminal.emulator_gpa`, `State.gpa`). Deltas write them while a copy is held, and nothing of them is a copy's to save, keep or put back. Without that, a rollback freed a delta's pages (the test below crashes). A first fix, the epoch paused while a delta was written, was not enough: the copy also saved emulator memory reached through a stale pointer (below), and put its old bytes back. - The core's cache of a terminal's rows is marked stale after a delta, dropped when a copy is taken and emptied when one is put back: it is a grid's worth of text, and the copy's would be of an older grid (without the mark, the test reads it). Found on the way: - **A view's slot that is live is stale, and the copy follows it.** views.zig keeps a parked view's share as copies, stale while that view is live (a column tag's text, `Text.own`, freed by the live view since). The rollback copy walks every slot: a stale pointer that lands on a live allocation saves it, and a rollback puts its old bytes back. Part A was safe while every free under a held copy was put off, which holds again now that a delta's writes are outside the epoch. A ReleaseFast harness run found it (heap corruption after 20 seeds), a safe build's check that named what was freed under the copy pinned it. ponytail: the copy still follows the stale slots; skipping the live view's slot diverged (part of it is read), and is the next fix in views.zig, not here. - A pane's `cwd` inherited from a terminal's pane is a pointer, and the walker follows it: a digest of a pane next to a shell digested the shell's emulator. It mattered when a digest was taken outside the authority's tape (`pending`'s first version). - `resizeGrid` does core work before it resizes (the pointer selections of a narrowed terminal): skipping it whole on a replica diverged. Measured. Correctness ran beside builds; the timings alone on the perf cores (ReleaseFast). - **Correctness.** - The soak (unit tests, and the 197 recorded streams at 3 seeds): several deltas in flight, landing late and in order, one in 50 lost (refused, then a reset), random resizes and fetches. After every delta the replica's screen, history extent and viewport are the authority's as encoded, and every history row it holds outside its holes is the authority's. None failed, Debug or ReleaseFast. - `zig build replicas`: three clients typing in their own views over 30, 80 and 207 ms links while the shell runs a TUI, a shell's session with clears, then 8 MB of `yes` read 4 KB at a time. In ReleaseFast, 20 seeds at each RTT, there was no divergence, every replica's digest and every terminal at quiescence was the authority's, and each of the 23,508 deltas applied matched the authority's screen at its version. The same held in Debug: 4 seeds at each RTT, and 20 at 30 and at 80 ms; and in ReleaseFast with every replica allocation filled with 0xff (`PARDES_REPLICA_FILL`), 20 seeds at each RTT. - **The terminals in the harness** (ReleaseFast, 20 seeds, three clients; a peer's numbers while the flood runs): | | 30 ms | 80 ms | 207 ms | |---|---|---|---| | deltas a replica applied a second | 76.3 | 75.7 | 71.7 | | delta bytes a second | 10.3 KB | 10.5 KB | 10.0 KB | | unit bytes a second | 101.6 KB | 102.0 KB | 101.5 KB | | encode a frame, every peer, p50 / p99 | 15 / 36 us | 16 / 42 us | 16 / 31 us | | the same, one peer | 13 / 28 us | 14 / 30 us | 15 / 30 us | | apply a delta, p50 / p99 | 37 / 494 us | 35 / 442 us | 34 / 400 us | | rows fetched on demand, all runs | 1103 | 649 | 786 | | rows sent before a step that read them, all runs | 18.0 MB | 18.5 MB | 10.0 MB | One delta in flight would have been one an RTT: 4.8 a second at 207 ms. The rate is the frame's (60) and the inputs' (a delta before each). - **The bench** (`PARDES_SCREEN_STATE= zig build core-test -Doptimize=ReleaseFast -Dtest-filter=screen_state`, 200x60, the floods under the C allocator): - encode a version, every row new: p50 273 us, p99 394 us, 29.8 KB; nothing new: 71 / 95 us, 549 bytes; - `yes` at 19-20 MB/s: 63 deltas a second, 65 KB/s a peer (1038 bytes a delta), encode a frame 28 us at one peer and 29 at three, apply p50 65 to 72 us and p99 202 to 265 us (it was 2.4 ms); - varied text at 59-64 MB/s: 356 KB/s a peer, encode a frame 52 / 54 us, apply p50 468 to 493 us. - **Part A's costs, with terminals** (the harness, three clients): speculative apply p50 28 to 42 us; copy p50 95 to 151 us (1.3 to 1.5 MB); rollback p50 331 to 442 us, p99 1.6 ms at 80 and 207 ms. At 30 ms the copy's p99 is 7.8 ms and a rollback's 17 ms: a register holding a terminal's whole grid (`%` then `y` in a shell's text, 2 MB after the flood) is copied whole by every speculation. Registers could be pinned as file texts are. (Done since: the follow-ups below.) - **No local regression.** `pardes-perf-live --live`, A and the new tty interleaved A B A ... A (4 runs of the new one, 5 of A) on the perf cores; the median of the runs' p50s, new/A, present and core: | | against main (kuovmsxw) | against base | A/A (base) | |---|---|---|---| | key-down | 1.02 / 1.02 | 0.94 / 0.94 | 1.00 / 1.01 | | key-right | 1.02 / 1.03 | 1.00 / 0.95 | 0.99 / 1.01 | | edit-char | 1.01 / 1.00 | 0.99 / 0.99 | 1.00 / 1.00 | | page-down | 0.92 / 0.95 | 1.22 / 1.05 | 1.00 / 0.99 | | wheel | 0.83 / 0.85 | 1.13 / 1.10 | 1.00 / 1.00 | | `yes \| head -c 32M` wall | 0.92 | 1.12 | 0.94 | | a 1M-line open | 1.03 | 1.14 | 0.97 | Against main, this line's own A/B, every cell's runs overlap main's. Page-down and wheel against the base read 1.21 and 1.12 in an earlier run too, and 0.92 and 0.83 against main: main's since the base, not this line's. What the local path gained: a counter a pty read, a resize and a scroll; a flag a frame and a store when rows are built; and, for a terminal with positions over it, a taped value (a null check) and two stack arrays. Not done, and why: - Kitty images in terminals still do not cross. - Units under a flood still cost about 0.1 MB/s a peer: an output step is about 45 bytes of tape (its input, the pump, a host call), 2000 a second. Phase 6's zstd takes most of it (each step's records are alike); or a host could feed a pane's queued reads to the core as one. - The first step in text mode after a flood sends its terminal's whole history of holes, up to a few MB, ahead of its unit (ponytail: a step reads every row, its motion surface). - A 9P run waiting on a replica's marks (`noteMarks`) reads them from the last delta, not from the output step; its writes are the authority's, but `sent_command` could differ. The output pins (`out_start`, `out_end`) stay the authority's. - Text reads outside `buildRows` (a pointer selection's copy in tty mode, 9P's screen reads) do not ask for the rows; 9P's are the authority's anyway. - **A hazard for part A** (fixed since, below): the rollback copy followed a view's stale slot. - Echo prediction (phase 6): a key into a terminal waits a round trip. - A replica steps every output step of the authority's (bytes-less), about 2000 a second under a flood, each a core step. - The GUI was not measured: its host is unchanged, and the core's terminal path is the tty's. - The other agent wires it in: the tty and GUI hosts, `Screens` per attached peer, and a replica frontend drawing after `screen`. Follow-ups (2026-10-29): - **A live view's slot keeps no pointer.** When a view goes live (`to`, `close`, `only`, and `drawOthers` putting the live one back), its parked share is emptied (views.zig `emptied`): it held the pointers it handed the live fields (a column tag's text, a chord's argument, the presentation's) until it parked again, and the copy, walking every slot, saved and put back whatever memory had since taken their place. A pane's share holds no pointer but its selections, which `unpark` already hands over. A test frees a tag's text under the stale slot, hands its memory to another allocation, writes it under a held copy and puts the copy back: it fails without the fix. - **Registers and location rows are pinned, not copied.** A register's values and joined text, and a file's location rows (with the ones an edit dropped and a query's saved), are replaced whole and never written, so the copy pins them as it does a file's text, freed raw. Rows pin whole with what each owns (its path, its colours: `pinnedDeep`). In the harness at 30 ms the big copies were not the register but a file's location rows, 5 to 6 MB after the flood. ReleaseFast, 20 seeds, three clients, p99: copy 5.1 ms to 0.22 ms at 30 ms (0.19 to 0.20 ms at 80 and 207); rollback 14.9 to 9.5 ms at 30 ms (1.3 to 1.4 ms at 80 and 207). What is left of a rollback's tail is the replay's digests: half the harness's CPU is Wyhash over the whole core at every replayed unit, the pinned blobs included (part A's digest). Hashing a pinned blob once, while it lives, would take most of it. - **The live perf harness ends what it starts.** Every process a live run starts inherits its private runtime directory: after each cell, and in the cleanup every path takes, the harness sends TERM, then KILL, to each whose environment names it (`liveSweep`). With a wrapper that leaves a daemon-kept detached session (as the remote agent's client-live.sh does), the old harness left three sessions and their daemon a run; this one leaves none. The gate's baselines are recorded again from base (main and the new perf.zig), and .scratch/daemon/base/pardes-perf-live rebuilt from it. - **A blob's digest is a word, made once.** The walker's digest, taken at every unit a replica is sent and every unit it replays, hashed every blob whole each time: a 100 MB file was 5 ms a digest, two digests a unit, on the user's own session. A blob (a field annotated `.blob`: a file's text and line index, an undo snapshot, a register's values and joined text, a file's location rows, the ones an edit dropped, a query's saved) is now hashed once, its word kept by its address (walk.zig), and the digest folds that word in. The rollback copy pins what the same annotation names (`pinned` and `pinnedDeep` are gone). - **Exact while every free drops the word:** every such blob is freed raw, through `memory.freeRaw` or `freeUnfilled` (the pins needed raw frees), and those drop its word, so an address used again is hashed again. The first ReleaseFast harness run found one that did not: an edit freed the old text with `rawFree` itself, the next text of the same length came back at the same address, and the old word was folded in (162 of 180 replica runs diverged, in `file.content`, named with PARDES_REPLICA_TRACE); it goes through `freeUnfilled` now. A replica's blobs born under a copy are never digested (a speculation records none), so the epoch's own frees leave no word behind. A register's values array was the one written in place; `put` now makes a new one. ponytail: the word is kept by address, so a new free path that skips those two misses it; a word carried in the struct beside its blob if that bites again. - **Written in place, or freed past its word:** a Debug build hashes a blob's bytes at every digest and folds those in, and panics where the word kept at that address and length is not their hash: a blob written in place (which the pins forbid too) or freed without dropping its word. The tests, the replicas harness and replay run Debug and go through it (and pass); walk.zig's test writes a blob in place and sees its kept word is no longer its hash. A release build (ReleaseSafe, the user's own, and ReleaseFast) takes the word and never panics on a stale one: at worst it shows as a digest divergence. Not recovered yet: a replica that diverges records why (`Replica.diverged`) and takes no more units; nothing resets it or joins it again. That is the remote client's to add (a fresh join from the authority's dump), not a crash. - Measured (ReleaseFast, alone on the perf cores; PARDES_DIGEST_BENCH=100 zig build core-test -Dtest-filter="big file open"): with a 100 MB file open, a unit of a key into it (two digests) went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the file's word), and a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients, ReleaseFast): rollback p99 at 30 ms went from 9.5 to 2.6 ms (1.5 and 1.4 ms at 80 and 207 ms, about as before), copy p99 0.27 ms (0.22 before, within the run's noise); none diverged, none unequal, no delta wrong. ## Phase 5 part B notes Built 2026-10-29 and 30: the stream over the replica socket (src/replica.zig `Peers`, and its client half, `dial` and `readStream`; detached/wire.zig version 15), the client process (`pardes --attach=:` and `--attach=:`, tty.zig `remote`), the session's side of it in the tty host, rterm's screen-state terminals and the echo agent's predicted echo carried by it, a client's rejoin, and test/daemon.py's replica clients. Decisions taken at the forks, with the ponytail lens: - **A client's input is a unit of its own, settled by `step`.** On the authority, at the step boundary after each pump (`Peers.boundary`), each input that came is `toView(its view)`, `update(it)` and `step()`: the middle of a pump (queued inputs and effects), with no wait, no clock read and no frame. The client's speculation is the same three calls, so the two tapes read alike and the fast path takes the unit as it came. With the pump in its place (part A's harness) every speculation stopped at the tty host's `wait_input` and read the clock in ns. - **The stream.** Framed as wire.zig frames everything, a tag and a u32le length: - the session's: 0x01 welcome (pid, name, the client's id and view, the dump's length, the session's offset from UTC, its digest as dumped and its build's semantics), 0x02 refused (words), 0x04 a piece of the dump (1 MiB), 0x05 a unit (origin, seq, the authority's realtime and monotonic clocks when it was made, its tape), 0x06 a terminal's message (rterm's: a delta or rows), 0x07 a check's answer; - the client's: its inputs as the existing event tags (their seq is the n-th on the connection), 0x04 a terminal's message (rterm's: an acknowledgement, rows asked, a reset), 0x05 check. A unit carries one digest, at its end (`unit_digests`); under screen state it is one word (`whole_digest`, PARDES_REPLICA_TRACE for one a field). Blobs are hashed once and folded in as a word (main's walk.zig), so a digest a unit is cheap. The draft's lengths-only digest went with that. - **Terminals cross as screen state, in rterm's order.** The session's Peers keep a `Screens`, a peer each. At each boundary: the rows the pump's steps read that a client holds as holes (`fill`, then `filled`), the pump's unit, then the deltas (`flush`). Before each client input's step its deltas again, after it the rows it read, then its unit. A joiner's terminals start afresh after its dump. The client applies a delta or rows in stream order, acknowledges, and asks for the history rows a frame shows that it lacks. Units carry no pty bytes. - **A wait takes one step of a person's input, and none after a pty's output** (tty.zig `split`), while clients replicate: a step reads the terminals as the deltas after each unit bring them, and in a pump that took a pty's output and then a key, the key would read on the session what the client has only after the unit. The next is held for the next wait. Only while a client is attached. - **A step that reads a terminal's prompt after its output is taped** (tape.zig `.prompt`): the first prompt's greeting (`ls`, after a pty resize) and a command waiting for the prompt. Found at once by a client opening a shell of its own: the session's layout after the shell's first output saw the prompt, and the client's replay, its emulator a delta behind, did not. - A dump-loaded client's emulators are on the C heap, as `terminal.create` makes a screen replica's (walk.zig `.alloc` of a pane's `terminal`, dump.zig `terminal_gpa`). - **Predicted echo is the echo agent's** (src/echo.zig), wired as its notes say: the client's core points at its replica's `echo`, a frame judges it before drawing, and a wake comes a frame later while predictions are out. PARDES_NO_ECHO turns it off. - **Join, at a step boundary, the step paying for the mutable state alone.** The view is opened and the session dumped without its blobs (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted where they go) and without its terminals' emulators (their size: the first delta brings each). The client's writer thread writes the welcome, then the dump with the blobs put back where they go: what the client reads is `dumpBytes`' dump byte for byte but for the terminals (a test splices one of a session without any and compares), and the client is unchanged. Each blob is pinned until it is written (memory.zig `pin`: its free is put off; a blob is never written in place and is freed raw). The texts the session never digested (it records nothing until a client comes) would be hashed by the join's first digest, 250 MB for a 100 MB file and its undo: they are hashed first on a thread of their own (walk.zig `unhashed`, `makeWords`), pinned meanwhile, and the join waits a boundary for them; a second round joins whatever is left. - **A joiner checks its load.** The welcome carries the session's digest as dumped; a client whose load differs says so and ends. The first unit after a join says the host and its writes again (`tape.resync`), which a core made from a dump has not. Found on the way: the dump's reader took the first unit with the dump's last piece and dropped it (the client diverged at its first digest about one join in three); what it read past the dump is the stream's now (`Framer`). - **A diverged client joins again**, the same join as the first: its link ended, the session dialled again, its dump loaded, its view's size sent, in the same terminal, and a line over its frame until its next key says why ("joined again: this client diverged from the session (...)"). It gets a new view: the old one went with its link. PARDES_TEST_DIVERGE= puts a client's core off the session's (a digested counter) once quiet after its n-th unit. A divergence shows only at a unit the client replays (another's input, or its own it did not apply): the fast path compares records, not digests. - **An Upgrade lets its clients go in words** (`Peers.refuseAll`, before the dump): each is sent `refused` ("the session is upgrading to another build of pardes") after what it was sent, for up to 200 ms, its view closed. The client says it in its line and dials again, every 100 ms for 20 s (the exec, the load, the daemon's own exec). A daemon's refusal for another build, or a welcome whose semantics are not its own (the daemon judged the hello by the hash it last heard, which an upgrade under way may not have said yet), ends it with "session upgraded to a newer pardes: upgrade this client too". "A newer pardes is installed" is now a Host call the core makes in its focus step (`newer_build`), so a session's clients take the line from the tape. - **The session's clock, guessed.** A unit carries the authority's clocks when it was made; a speculation's clock reads answer the last unit's reading plus the time since it came plus the client's round trip (`tape.Guess`): a file's mtime (realtime seconds), a message's time of day (in the session's zone, its offset in the welcome) and a message's monotonic ms. Never the client's own wall clock. - **Mostly no rollback for the session's own pumps.** Every client input woke the session's pump before its own unit, and that pump advanced the clock: a foreign unit, a rollback a key. Two changes: a pump woken only by a replica's post reads no new time (`mail_only`: the tty's `now` answers the core's own), and a replica keeps what it applied over a unit of another's that ends on the digest the last one ended on (`neutral`; such a unit is replayed when the copy is next put back). A unit that says what the host wrote, or which host it is, is never neutral. - **The copy is taken while idle.** The client takes its rollback copy after drawing, when nothing is queued (`Replica.ready`), and lets it go when a unit comes first: a key no longer pays a megabyte's copy (memcpy was a third of the client's profile). The epoch's and the copy's maps hash an address by a multiply, not Wyhash (the other sixth). - **Each client a view, drawn by its own process.** `Pardes.undrawn` marks a replica client's view: the session's pump and `drawOthers` skip it. The client draws its own view between steps (`views.drawOne`), presenting through the tty Shell's own `present`; what that writes in the core (a frame skipped, one owed) is put back. A drawn parked view gives its selections back to its slot (`drawParked`). - **The client is the tty host with its inputs fed elsewhere.** Shell gains `remote`; `feed` sends a terminal's input to the replica engine and the session instead of to the core; quit, hangup and the size are the link's. Its replica core's allocator is the rollback epoch; images' and PDFs' are the C heap's, whose workers allocate on threads of their own. ponytail: a picture's pixels freed while a copy is held are not put off. - **Every kind of input is speculated** but one into a terminal (rterm's rule: its step reads the emulator), a view's own (a resize, the pointer) too. - **`unattended` is a watched field**, and a session with a client is attended: its messages go by keys, not the clock. - **Detach in a client ends that client**, a client's death too (its link's end); the session and the others carry on. A Restore lets every client go (they attach again). - **A lagging client never holds the session up.** Each client has a writer thread and a queue the session appends to (units coalesce there into one write); one whose queue passes 64 MiB is cut off. rterm's deltas skip states within that (32 in flight). - **ssh: a terminal asked for, not -N.** `--attach=:` runs `$PARDES_SSH -tt -q -L : 'exec sleep ...'`: only an interactive connection gets TCP_NODELAY from ssh and sshd. With -N, Nagle held each key of a burst for the last one's ACK, and a shell's echo through the box took two round trips (413 ms) where it takes one. `$PARDES_REMOTE_RUNTIME` names the remote runtime directory when it is not this one's path. - **`ssh -C` by default** when `$PARDES_SSH` is not set (it is then the whole command). Through the box a 3.9 MB join took 7.8 to 7.9 s plain and 5.4 to 5.8 s compressed. Of a join's time ssh's own forward is 2.0 s (connect, key exchange, authentication, a pty) and opening a channel 380 ms; a ControlMaster in the user's ssh config would skip the first. A rejoin keeps its forward. Measured. ReleaseFast; timings alone on the perf cores (lock.sh), the box's through its own link (207 ms, 170 to 400 KB/s from it). The harness scripts are in `.scratch/daemon/remote/t2/`. - **The main test** (`e2e_box2.py`): a session on the FreeBSD box with a 400-line file, two replica clients here through `ssh -L` each, and a terminal on the box itself attached as a view (`pardes --attach` there over `ssh -tt`). Each types into the file; A and B move and scroll; A opens a shell, runs a command in it and exits it; B is hung up; A and the box's terminal detach. A's and B's digests were the session's at every quiet point (every check agreed), the session's text had all three's words, the shell's answer was on A's screen 214 to 224 ms after Enter, and nothing was left on the box once its idle daemon had gone (a few seconds). The clients' first frames came 3.4 to 4.0 s after they started (ssh's forward, below). - **Keystroke to present on a client, local-only actions** (base's `pardes-perf-live --live`, its keys on a 50k-line file; the client of a session detached on that file, through test/delay_relay.py at each RTT and through the box's link; two rounds interleaved; p50 / p99, µs): | an idle machine, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | |---|---|---|---|---|---| | a local session, this line | 2070 / 2067 (2926) | 1507 / 1502 (1895) | 4744 / 4708 (7141) | 1997 / 2033 (5486) | 2098 / 2126 (2885) | | a local session, base | 2059 / 2068 (2678) | 1517 / 1536 (2391) | 4755 / 4695 (7464) | 1916 / 1927 (5441) | 2100 / 2108 (2642) | | client, direct | 1903 / 1917 (6813) | 1644 / 1650 (2136) | 4477 / 4413 (7912) | 1994 / 2244 (7405) | 1934 / 2063 (6793) | | client, relay 30 ms | 1849 / 1838 (6910) | 1576 / 1557 (2695) | 4465 / 4300 (15560) | 1755 / 1712 (7682) | 1860 / 1757 (7163) | | client, relay 80 ms | 1799 / 1753 (7281) | 1532 / 1501 (2034) | 3085 / 2654 (13705) | 1778 / 1743 (6386) | 2106 / 1870 (7692) | | client, relay 207 ms | 1778 / 1814 (6721) | 1453 / 1478 (1887) | 4309 / 4333 (18092) | 4640 / 1745 (7562) | 1963 / 1978 (7315) | | client, the box, ssh -C | 1773 (6947) | 1471 (2138) | 4441 (14018) | 3219 (7503) | 2163 (7692) | The same harness while builds ran on the other cores (the line before its last two changes, which do not touch a client's keys; one round): | builds on the other cores, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | |---|---|---|---|---|---| | a local session, this line | 464 (574) | 343 (417) | 1190 (1714) | 370 (1251) | 471 (638) | | a local session, base | 463 (629) | 349 (499) | 1188 (1663) | 399 (2078) | 2096 (2591) | | client, direct | 433 (1266) | 389 (441) | 1381 (2071) | 1183 (1400) | 451 (1360) | | client, relay 30 ms | 413 (1243) | 364 (537) | 1332 (4083) | 1163 (1254) | 421 (1293) | | client, relay 80 ms | 421 (1379) | 357 (500) | 1104 (1815) | 1143 (1359) | 1112 (1487) | | client, relay 207 ms | 398 (1228) | 339 (444) | 1145 (6882) | 1151 (1263) | 615 (1469) | - **A client's local-only key is as fast as a local session's at any RTT, the box's link included**: its p50s are the local session's, whatever the delay. Its p99 is 1 to 3x the local one's: the rollbacks (another's unit before its pending inputs; p99 1.6 to 17 ms in the client's own count) land between some keys and their frames. - **The machine's state moves every cell together** by up to 4.5x: on an idle machine the cores sleep deeply between keys (the `powersave` governor), and with builds on the other cores they do not; in a run made while one build ran, half the cells were in each state. Within one run, base and this line's local session read alike, and the clients alike with them. - Page-down on a client costs 1.15 ms where a local one costs 0.37 when the machine is awake: a client draws outside the pump, so the render's paint-ahead (`Stream.askAhead`, asked of the host after a pump's present) is never asked, and the rows a page-down brings are highlighted in its frame. - **Terminal echo** (`echo2.py`: a key written to the client's terminal, to the first output holding its glyph; 30 keys; p50 / p90, ms): | | file | shell, predicted | shell, not predicted (PARDES_NO_ECHO) | |---|---|---|---| | a local session, base | 0.5 / 0.5 | | 0.8 / 0.8 | | a local session, this line | 0.5 / 0.5 | | 0.8 / 0.8 | | client, relay 30 ms | 0.6 / 0.6 | 0.8 / 0.9 | 31.4 / 31.4 | | client, relay 80 ms | 0.5 / 0.5 | 0.5 / 0.8 | 81.4 / 81.6 | | client, relay 207 ms | 0.5 / 0.5 | 0.5 / 0.5 | 208.4 / 208.6 | | client, the box, ssh | 0.5 / 0.6 | 0.5 / 0.6 | 206.9 / 207.4 | | client, the box, ssh -C | 0.5 / 0.5 | 0.5 / 0.6 | 215.8 / 216.9 | Predicted, a key into a shell at a prompt shows as fast as a local one (it is drawn by the client's own frame); not predicted, it is one round trip and a millisecond and a half. Before this round (the draft's) the echo through the box was 211 ms p50. - **The visible rollback rate.** A frame drawn after one of the client's own inputs was recorded otherwise is counted as a rollback that may have been seen (an upper bound: the frame shows the inputs after it too). In the keys harness, of 669 inputs a run (two runs each): none at 0 and 30 ms; 2 and 3 (0.4%) at 80 ms; 11 and 13 (1.6 to 1.9%) at 207 ms, of 25 and 29 mispredicted (a key the session recorded otherwise than the client applied it). Every other rollback (another's unit before the client's pending inputs) is put back and replayed before the next frame. - **Join cost on the session** (`join2.py`: three clients join a session one after another, through the Unix socket; the session's worst boundary, which holds the join, and the client's first frame): | session | its dump | of it, made in the step | worst step, before | worst step, after | first frame | |---|---|---|---|---|---| | typical: 4.1 MB of files, 1 MB of scrollback, 6 panes | 8.1 MB | 113 KB | 12.4 ms | 1.6 ms | 22 to 46 ms | | big: 38.6 MB of files, 8 MB of scrollback, 14 panes | 75.6 MB | 122 KB | 51.3 ms | 1.5 ms | 87 to 117 ms | | a 100 MB file beside typical's (130 MB of files) | 254 MB | 114 KB | 172.5 ms | 1.6 ms | 247 to 288 ms | Each join itself took 0.3 to 1.4 ms of a step. The dump is about twice the files: their undo snapshots are blobs too. - **Join through the box** (`boxjoin.py`: a session there with five of the tree's source files open, 3.5 MB, and a shell with 1 MB of scrollback; a 3.9 MB dump; two joins each): the client's first frame 7.8 and 7.9 s through plain ssh, 5.4 and 5.8 s through `ssh -C` (the default now); the session's worst step 0.9 ms. A 16 KB session's first frame takes 3.4 to 4.0 s of which ssh's forward is 2.0 s and its channel's opening 380 ms. - **The session's step while a client lags** (`lag2.py`: a shell printing a line a millisecond for 12 s, client A typing a key every 30 ms, client B stopped with SIGSTOP and cut off at a 256 KB lag; µs, p50 / p99 / max): A's inputs 47 / 150 / 213 and the boundaries 66 / 208 / 681 with B stopped (cut off: yes), against 60 / 134 / 309 and 69 / 276 / 4072 with B running. - **Digests at quiescence**: every client of every test above ended on the session's digest (PARDES_REPLICA_CHECK): the main test's, the clients test's four (one rejoined after a forced divergence), and the upgrade test's client that joined again. - **No local regression.** With no client attached the recorder costs an atomic load a pump (`Peers.boundary`), a flag test an event (`split`) and one a raw free (`memory.pinned`); a session's own core has no echo. ReleaseFast, interleaved base new base new base on the perf cores, the median of each side's runs, new/old, beside an A/A of the old: - keystroke to present (base's `pardes-perf-live --live`, against base, present / core): | | new/base | A/A | |---|---|---| | key-down | 0.98 / 0.98 | 0.99 / 1.01 | | key-right | 0.86 / 0.84 | 1.00 / 0.99 | | edit-char | 0.76 / 0.86 | 1.01 / 1.00 | | page-down | 0.81 / 0.81 | 0.91 / 0.93 | | wheel | 0.89 / 0.88 | 0.96 / 1.00 | | `yes \| head -c 32M` wall | 0.98 | 1.06 | | a 1M-line open | 1.03 | 1.32 | - the core in process (`pardes-perf --reps 3`, each tree's own build of it), against main (ykrsossp): the keys, page-down and wheel 0.92 to 1.08 over the small, medium and large files; the edits 0.91 to 1.05; the opens 0.93 to 1.02; body-2m 1.02 to 1.26 (small's 1.26 alone); the scrollback cells 0.93 to 1.05. The A/A read 0.79 to 1.46 (most within 0.9 to 1.1). Against base the keys read 1.11 to 1.20 and the scrollback renders 1.68 to 2.18, but that pair was wrong: both of this line's core benchmarks were built at the default `-Doptimize` (their `--json` identity says `core_optimize` debug and `terminal_optimize` safe; the driver alone is ReleaseFast) and base's is ReleaseFast throughout. Built ReleaseFast, main's file gestures are level with base, and a terminal's frame is faster than base's since `terminal.cellAt` (the final gate's notes, at the end). The gates, once at the end (Debug unless said): - `zig build unit-test -Dplatform=tui` passes, its perf gate too; `-Dplatform=gui` 1563 of 1575, 10 skipped, and the two that fail on main already (`tagline pointer mapping...`, `a tag caret past its row's end...`). - `zig build snap`: 113 of 113. `zig build replay`: the 113 scripts and 20 monkey seeds, none panicked or diverged. `zig build replicas`: 4 seeds at each RTT, none diverged, none unequal, no delta wrong. - `test/daemon.py` (20 cycles, its replica clients' test included: four clients, one through the relay, one rejoined after a forced divergence, one a shell's, a terminal attached while a client's view was live), `test/daemon.py --upgrade` (an unstamped build to a stamped one, its two replica clients included) and `test/fs.py` (49 s) pass. - core-test's new tests: a join's dump with its blobs spliced back is the session's dump byte for byte; a pinned blob's free is put off until its last pin goes; and the replica tests (`-Dtest-filter=replica`). Found on the way: - **A terminal attached while a client's view is live drew nothing.** The tty's attach took the live view's screen, and a client's view has none in the session: it returned without a word, its stub waiting (the main test's terminal on the box). It takes the session's own now; the clients test attaches one while a client's view is live. - **The session's layout read a terminal's prompt after its output** (above, `.prompt`): a client opening a shell diverged at its first prompt. - **A divergence shows only at a unit the client replays.** The fast path compares a speculation's records with the unit's, not digests, so a client off the session's state that only types on stays off until a unit it did not apply comes (another's input, or its own it waited for). The forced-divergence test has another client type. - **Harness traps** (the scripts, not pardes): a terminal on the box through `ssh -tt` from a pty must have its size before ssh starts, else the remote pty is 0 by 0 and the session draws nothing in it; an Esc and the next key written within the input parser's lone-Esc window read as an Alt chord; a ReleaseFast build stamped with another commit has another semantics hash than the box's of the same tree (`-Dstamp-commit =false` for both); keys a test types into a client whose live pane is a shell in tty mode go to the shell (end it with a hangup). Not done, and why: - **A GUI client**, and a window session serving replicas (refused in words). The GUI host feeds its core from 52 places, each to route through the replica as the tty's `feed` does; its frame loop is the pump's (the client draws between steps, outside it); and its own presentation (the window's surface, its animations) would have to come out of the core's step as the tty's present does. A window client of a tty session is the main test's other half; it is the next piece. - Paint-ahead on a client (above: page-down 1.15 ms where a local one is 0.37): its draw would ask for it as a pump does, the worker's rows taken in as the client's own events, outside the session's tape. - A rejoin gets a new view: the old one's layout and focus go with its link. A rejoin that kept the view would take the view's number in the hello. - A notice of the session's own while clients replicate goes through the tape (the upgrade's line); a host that writes core state outside a step (the GUI's notice) would diverge them, and the GUI serves none. - The join waits for the session's texts to be hashed once a session (its first join, or after new texts): the joiner pays some milliseconds more, never the session. - A held paste (`split`) is dropped if the session ends with it held. - A `--attach=:` client ended by a hangup leaves its forward's socket file in $TMPDIR (its ssh goes with the terminal; the unlink is on the normal way out only). ## Phase 6: upgrade notes Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`, `main.zig`, `dump.resumeFromDump`; `test/daemon.py --upgrade`). A running session, and the daemon, move onto the pardes installed since they started, in place: text, undo, views and layout, terminals with their running shells, and their terminals, windows and daemon link. Decisions taken at the forks, with the ponytail lens: - **The trigger is the word, `Upgrade`** (`SPC s U`, any tag, `/ctl` and `/pane/N/exec` over 9P). It moves the daemon too: the session sends it `upgrade` just before its own exec, so one word moves both, and no word of the daemon's own was needed. Nothing upgrades by itself. A focus-in (one comes back to the terminal or window from rebuilding elsewhere) stats the start path, and a file there that is not the one the process started from is said once, in a line: `a newer pardes is installed: Upgrade (SPC s U) moves this session onto it`. Rejected: a watch on the binary's directory (an inotify entry and an event source for a line), and upgrading on its own (a build half-written, or one the user means to test first, would take every session). - **Which binary: the path this process was started from**, read once just after its first frame (`std.process.executablePath`, a readlink, and a stat for the notice: off a startup's way, and no build is installed within a frame of a start), exec'd again at the upgrade. `install(1)` and `zig build` put a new file at the path, renamed over the running one, which `/proc/self/exe` then names `(deleted)`. The record carries the path exec'd, so a process that fell back to the old binary (whose own exe is that deleted file) upgrades from the same path next time. Not argv[0] and PATH: a session's PATH is the one it started with, and the readlink is what the daemon's spawn already used. - **Asked first, ` --version`**: run as it is, it must exit 0 with a `pardes ` line within 10 s. A file not there, not a pardes, or one that dies at start is found before anything is let go, and said ("Upgrade: exited 3 at --version; this session stays as it is"). 9P is served while it runs (the turn rested), so a client's clunk after its `Upgrade` write, and anything else, is answered; after it, nothing is. - **At a step boundary**: after a pump, the pty readers are canceled (a canceled tty reader posts an end of file, passed over while upgrading), the terminals' input readers stopped, the loop's queue applied, and the core settled (`Pardes.settle`: queued inputs, then effects). In the GUI the queue is drained, the readers canceled, and drained again (a reader waiting for room gets it). Bytes read are never lost; keys typed during the exec wait in the terminal's kernel buffer for the new process. - **The dump goes to the DumpDir, `pardes-upgrade-.dump`**, not a memfd and not the runtime directory: a refused load leaves it where a relative `Restore` finds it, on a disk and not a tmpfs (a big file's session dumps 100 MB and up). Never $PARDES_DUMP, which is a Dump's. A successful load deletes it. - **The record is one argument**, `--upgraded=`, ahead of the arguments the session was started with, which configure it again (`--9p=`, `--tty-toggle`, `--geometry`, `--nested`); its file, stdin, directory, `--detach` check, forwarding and daemon fork are skipped. A line a word: `link`, `stub`, `old` (fds), `window`, `pty `, `tty `, `exe`, `dump`, and `refused` in a fallback; a word another build adds is passed over. Every fd it names is close-on-exec but across the one exec (`Record.closeOnExec`), and the new process sets them back at once, so no shell it forks inherits a master or a terminal. - **The pid stays**: the shells stay the session's children, the stub keeps waiting on it, the 9P name of a session named by its pid is the same, and PDEATHSIG (set again) still means the stub's death. - **Terminals.** A kept terminal pane is not revived (`resumeFromDump` with `kept`): no reset, no banner, no new shell; ghostty's state is the dump's, the program's own, and its pty keeps its size (no 1x1 nudge: the screen is already right). The host puts the master back in the pane's slot and reads on (a command pane's exit watched again). A terminal pane with no live master (its shell gone) is revived as a Restore revives it. - **The tty's terminals stay raw, their cooked termios in the record**: put back to cooked between the two processes, a key would echo and a ^C (forwarded by the stub) could kill the new process before its handlers. What is a stack is unwound: the kitty keyboard flags are popped (the new process pushes them once its query is answered), its kitty images deleted and its shm names unlinked (the pid is the same, and so would the names be). The alternate screen, mouse and paste modes stay on; the new process sets them again and draws everything, one full frame over the old one in place. The old image's prompt rc files (the running shells have read them) are swept by the new one, which makes its own: at its sweep, any of its pid are the old image's. They stay for an exec that failed. - **Views come back whole.** `resumeFromDump` keeps every view the dump has (no `views.only`) and does not run the startup config again (the session's settings are the dump's). Each terminal's view is in its `tty` line: the primary one is `run`'s, the others are entered as `--attach` enters one, with their stubs; a view that lost its terminal closes. The GUI opens a window for each open view, the live one first (`openAnotherWindow(k)`): its windows are the one flash, as decided. - **9P is cut and comes back.** What the connections had to send is let go first (`Listener.flushOut`, the 200 ms the quit's path already had), then the exec closes the listener and its lock, and the new process claims the name again. ponytail: a dial during the exec and the load is refused (one round of a client's retry); passing the listener through as well would keep it queued. - **Replicas are told in words.** Each connection a session serves gets a `refused` frame ("the session is upgrading to another build of pardes: connect again with that build") and a shutdown before the exec. Their hello's semantics hash would not match the new build anyway, and the session registers its new hash on the same link (a second `hello`, which the daemon takes as an update). The reconnect, and a local client of the same path re-exec'ing itself, are phase 5B's: there is no replica client yet. - **The daemon re-execs the same way**: its own start path, asked `--version` first, the listener and lock (fds 3 and 4), the replicas' listener, every session's link and every held master kept, its registry on its command line (`--daemon --upgraded=` with a `c` line a connection, its slot, kind, pid, state, semantics and name, and an `h` line a master and its owner's slot); a replica whose hello is still on its way is let go and dials again. Sessions never see it: their links are the same sockets. - **Failure never loses a shell.** 1. The binary does not answer `--version`: nothing changes, said. 2. The exec fails: the fds close-on-exec again, the readers and the terminals' keys start again, the kitty flags are pushed back, the dump deleted, said. 3. The new build cannot load the dump: it execs the old binary again, from `/proc/self/exe` opened before the exec (`old`, Linux), with `refused` set; that one loads its own dump and says "Upgrade: the new build could not load this session (); it goes on as ". 4. No old binary to go back to (FreeBSD has no /proc by default), or it refuses too: a fresh session adopts every kept master as a terminal pane (the Adopt path) and says "Upgrade: no build could load the session (); its shells are here, and Restore brings the rest back". The dump stays. 5. A crash after the exec is the crash Phase 3 covers: the daemon's dups keep the shells for an Adopt, and the dump is in the DumpDir. `PARDES_TEST_UPGRADE_REFUSE=1` makes a start refuse the load as such a build would (the first start only), `=all` the fallback's start too: test/daemon.py installs a script that sets it and execs the new build. Found on the way: - A fresh session's first pane has no directory until its own shell's spawn is performed, so the `Tty` word that adopts a shell refused ("Tty: : no such directory") and the shell was hung up with "no room": the kept shells are taken after the startup effects, and those they emit performed then. - A test reconnecting after the word could reach the old process, still asking `--version` (and serving 9P meanwhile): the test waits for the exec (the session's exe, or on FreeBSD its new semantics in the daemon's registry) before it reconnects. - An `Adopt` (or a `Detach`) served in a session's first pump, before its first frame, is refused as if no daemon kept it: the reader that takes the daemon's `ok` starts after that frame, and `kept()` waits 2 s for it. The test's next session waits for its first frame. Pre-existing, left as it is. Gates (Debug, `A` this line unstamped, `B` the same with `-Dstamp-commit`, a trivial visible change: `pardes 0.24 ()` and another semantics hash; both installed in turn at a private prefix by a rename, as `zig build` and install(1) do): - `python3 -B test/daemon.py --upgrade A B`, under a private runtime dir. A tty session with an unsaved edit, a shell running `sleep 1000; echo done-after-upgrade` and a second terminal (a second view), and a replica of A's build connected: - B installed, a focus-in says the newer build; `Upgrade` written to `/ctl`: the session's 9P answers again 123 ms later (81 to 83 ms on the box). The session's pid is the same, `/proc//exe` is B (not deleted), and so is the daemon's, the same pid. - The edit is there, the pane dirty, and `Undo` takes it back. - The shell's pid is the same, still the session's child, its `sleep` running; killed, the shell goes on and `done-after-upgrade` shows. - Both terminals are drawn again (each its own view); the stubs wait on. - The replica of A's build is told `the session is upgrading to another build...`; a hello of B's build is welcomed by the daemon, one of A's refused (the daemon lists the session as B). - The fds level: the session's as before but the replica's, the daemon's the same; the session's prompt files as many as before, the new image's. - A refusing build (a script that sets `PARDES_TEST_UPGRADE_REFUSE=1` and execs B) installed and asked: the session goes back to the binary it ran, its exe that inode again, the shell and text as they were. - One that refuses everywhere (`=all`): a fresh session, the shell a pane of it (`echo $$` there is its pid), the dump left in the DumpDir. - The session killed outright: the daemon, upgraded three times, still holds the shell's master, and a new session adopts it. - Nothing left: no process, no socket. Passed 3 runs of 3 (and every run since). - `--upgrade --gui A-gui B-gui`: a window session in test mode (hidden windows, captured) with a second window from `pardes-gui --attach`, the same edit and job: after `Upgrade` the pid is the same and the exe B's, the daemon B, the edit and undo there, the shell and its job the session's, both windows' captures written again, B's hello welcomed; the session ends with nothing left. Passed 5 runs of 5. - FreeBSD 15.1 (the box, `pardes-ua` and `pardes-ub` cross-built from this line): the same tty test but the fallback to a kept copy (there is none). The new build is told by its semantics in the daemon's registry, and the daemon's re-exec by `--upgraded=` on its command line; the fds level by procstat. 81 to 83 ms from the word to 9P. Passed 9 runs of 10 (the last two on this line's final builds); the one failure was the test's first-frame race above, fixed, and it passed after. - The existing gates: `test/daemon.py` (20 cycles) and `--gui` (10) pass; `test/fs.py` passes (49 s); `zig build snap` 113 of 113; `zig build replay` passes (the 113 scripts, 20 seeds at one view and 20 at three, their dumps loaded to their digests); unit-test tty 1479 of 1499, 20 skipped; unit-test gui 1550 of 1561, 9 skipped, and the two known failures (`tagline pointer mapping...`, `a tag caret past its row's end...`). core-test's new tests: the record read back as written (and refused when cut), and an upgraded dump keeping its views, a running terminal's alternate screen and no new shell, and settings the startup config would change. Measured. The upgrade itself, Debug, a 300 MB binary: from the word written to `/ctl` to the session's 9P answering again, 123 to 130 ms, one run beside builds 299 ms (the `--version` ask, the drain, a dump of a small session, the exec and the load); on the box, 81 to 83 ms with its 125 MB binary. No local change (ReleaseFast tty, main kuovmsxw against this line, both live at once, their samples alternating, through the perf lock; lat.py's startup is a spawn with no daemon running to its first frame's end): | | A/B | A/A | |---|---|---| | startup, 100 a build: p50 | 0.944 | 1.010 | | the same: mean | 0.919 | 1.061 | | the same: median of the fast mode (and its share) | 0.964 (54 to 52) | 0.996 (69 to 73) | | startup, 40 a build, two runs: p50 | 1.001, 1.266 | 0.998, 1.010 | | key in a file, to its frame, p50, two runs | 0.997, 1.002 | 0.958, 0.990 | | key echoed by bash, to its frame, p50, two runs | 0.997, 0.991 | 1.004, 0.999 | The startup times have two modes on this machine, about 4.5 and about 12 ms, and the share in each moves between runs and between the builds of an A/A alike, so a p50 that falls between them jumps (the 1.266). Within a mode the builds are the same. A first A/B, made while the path was still read at the start (a readlink and a stat before the first frame, some microseconds), read p50 1.04 to 1.13 against an A/A of 0.98 to 1.00 on runs that overlapped builds; the read was moved after the first frame anyway, so a start does nothing it did not. Not done, and why: - Remote replicas reconnect on their own, and a local client of the same path re-execs itself first: phase 5B's (no replica client exists yet). What they get today is the `refused` frame above, then the hangup. - The 9P listener is not kept across the exec: a dial in that window (a hundred milliseconds or so) is refused. Clients that dial afresh (the hosts' Client) retry; a mounted session sees its connection end. - On FreeBSD a build that cannot load the dump starts afresh (shells adopted, dump kept): there is no `/proc/self/exe` to go back to by default. Keeping an fd of the binary open from the start would do it, at an fd a session. - A crash of the new build after it starts is a crash: the shells wait in the daemon for an Adopt and the dump is in the DumpDir, but nothing restarts the session on its own (the stub, which runs the old binary, could). - An escape sequence a program had half written when the dump was made is lost (the snapshot is at ground state, Phase 2 notes). - Language servers and `|` pipes running at the upgrade are let go (their pipes close at the exec; they end), and the shells of panes closed just before it, still waiting to be reaped, are reaped only when the session ends. A daemon note (a terminal attaching, adopted shells) read by the old process's thread but not yet taken by its loop is lost; the stub of such a terminal sees its pipe end. - A detached session's upgrade (no terminal) was checked by hand (scratch script), not in test/daemon.py. - GUI windows come back at the size and place the compositor gives them. - A daemon protocol bump across an upgrade is not handled: the sessions of the older build keep talking theirs. - A session name with a newline would break the daemon's registry line. - An upgraded process records no tape (a recording ends at the upgrade). ## Phase 6: 9P round trips notes Built 2026-10-30 (`src/9p_io.zig` Client, `src/fs.zig`, the Mount word in `src/builtins.zig`, a paragraph of `docs/typ/reference.typ`). pardes's own 9P client is what mounts, the Mount word, the forwarding CLI, `--wait` and Handover use; scripting clients (9p(1), 9ns, test/ninep.py) are not changed here. Decisions at the forks, with the ponytail lens: - **The server already takes a connection's requests in order; nothing there changes.** cloud9's engine has one job slot: it starts the next frame only once the request before it is answered. What parks leaves that slot, and the frames after it go ahead: a read with nothing yet, and, while the editor is out in a syscall, a write or a truncating open (`tree.needsQuiet`; an OTRUNC open is a setattr first). So a walk and the open of its fid can be sent together (a walk never parks), and so can an open and its first read (an OREAD open never parks). Two things cannot: - a Twrite behind a truncating Topen that parks is refused `bad use of fid` (not open yet); - a Tclunk behind a parked Twrite interrupts it. The client therefore sends a write or read that came back `bad use of fid` again after its open's answer: that refusal is the engine's, made before the backend, so nothing was done. It sends its Tclunk only after the last answer, and does not wait for the Rclunk. The park test ("a change waits while the editor is out mid-step") takes the resend path. - **One flight per operation.** The flight is `[Tversion Tattach] Twalk [Twalk] Tstat Topen Tread` for a read, or `... Topen Twrite` for a write, each request with its own tag (`Flight`, `Session.land`). Past 16 names, the second walk goes from the fid to itself, so no fid is in between. - **Version and attach pipelined by a poke.** cloud9's client submits nothing while a Tversion is out. `greet` sets its `versioning` and `msize` around the submits, taking the asked msize until the Rversion lands. Only a pardes is dialled here, and every pardes offers `ninep.msize`. A write in that first flight is kept under `min_msize - 23`. The fix belongs in cloud9's client (taking requests while it versions). It is not made here, since this round pushes nothing (`// ponytail:` at `greet`). - **The end of a read.** 9P has no end of file but an empty read. read(5) says a reply "may be less than the requested amount", and a directory's next offset must be the last plus what it returned, so neither the end nor a directory's next read can be sent ahead. The Tstat in the flight gives a file's length: a read that comes back short at that length is the end. pardes reports real lengths for bodies, `/os` files, ctl, tag and the like. A file of no stated length, or a directory, costs one round trip more: index, layout, screen and log are generated whole by each read and say 0 by design (tree.zig `topSize`). An end flag would be a protocol extension, which is not worth one round trip on generated files. - **Read ahead.** With the length known, the Treads for the rest go at once, 14 at a time (the 16-tag window, less a clunk still out and a spare). A 1.3 MB file takes 3 round trips where it took 22. A file of no stated length is not read ahead (`// ponytail:`). - **Reads are timed by their progress.** Each Rread gives the rest the 2 s budget again, as each Twrite already did. Found on the way: on main, every read of one operation shared one 2 s budget, so a remote file of 1.3 MB failed to open (`dial failed: timed out`) at 80 and 207 ms. On the box without `-C`, the 77 KB `src/fs.zig` failed too. - **A mount keeps its connection** (`kept`), as Plan 9's mount keeps its channel: an operation through it is the flight alone, with no dial, version or attach. - There are eight slots, keyed by dial. A session is taken out of the table while it is in use, so two users of one dial each get their own. - Unmount forgets it (`Client.forget`). - Before reuse, a poll takes what arrived while it waited (a Rclunk owed); an EOF, or anything else, means the peer let go, and a new connection is dialled. - A read on a kept connection that hangs up all the same is asked once more on a new one. A write is not, since it may have been done. - Only `readMount`, `writeMount` and `probeMount` (the Mount word) keep one, since each costs the peer one of its 16 connections while kept. The CLI, `--wait` and Handover dial afresh as before, with their version, attach and walk pipelined. - **No qid or stat cache.** The trace shows that a Look through a mount is one operation with one walk. With walk, stat, open and read in one flight, a cache would save bytes but no round trip, and a cached stat is wrong once the peer changes the file. - **msize and iounit stay.** They are already 64 KiB, with iounit = msize - 24, so a body of up to 65,525 bytes is one Tread. A larger msize would cost each of a session's 16 connections five buffers of it. test/ninep.py asks 8192 by its own choice, so its body reads of more than that cost it more messages. - **`/n/`.** The daemon serves no 9P yet (phase 3), so there is no remote daemon to mount. A remote session mounts today with two commands, now in reference.typ: `ssh -C -N -L /tmp/box.sock: host`, then `Mount box /tmp/box.sock`. A relay in the daemon that lists a host's sessions under one mount waits for the daemon to serve 9P. `-C` is there because the box's link is slow in that direction (below). Measured. The harness is `.scratch/daemon/ninep/rt.py`. It runs a peer session and a local one that Mounts it through a relay. The relay holds each chunk a fixed delay after it arrives, as test/delay_relay.py does, and it also splits the stream into 9P frames, counting the T-messages and the rounds (runs of T-messages with no answer delivered between them). `box.sh` runs the same harness against a detached session on the box through `ssh -L`. Builds are ReleaseFast (main ymwmwrvx against this line), 3 reps each, medians. The RTTs column counts the round trips the client waited on. pardes's own client sends its last Tclunk without waiting, before and after; the python client waits on every message. The box columns are `ssh -L` at 207 ms, plain and with `ssh -C`; "fails" means `dial failed: timed out`. | operation (pardes's client) | msgs | RTTs | 30 ms | 80 ms | 207 ms | box | box `-C` | |---|---|---|---|---|---|---|---| | connect: `Mount` (version, attach, root read) | 5 → 7 | 4 → 1 | 124 → 33 | 324 → 81 | 831 → 208 | 1032 → 426 | 1024 → 410 | | file read: Look `/n/peer/index` (no length) | 7 → 6 | 6 → 2 | 185 → 61 | 486 → 161 | 1247 → 415 | 1441 → 426 | 1434 → 409 | | directory listing: Look `/n/peer/pane/` | 7 → 6 | 6 → 2 | 185 → 62 | 483 → 161 | 1247 → 415 | 1437 → 427 | 1430 → 410 | | ctl write: Save to `/n/peer/pane/1/ctl` | 6 → 4 | 5 → 1 | 155 → 32 | 402 → 81 | 1039 → 208 | 1237 → 214 | 1233 → 206 | | pane body read: Look `/n/peer/pane/1/body` (30 KB) | 7 → 5 | 6 → 1 | 186 → 33 | 483 → 81 | 1247 → 208 | 1843 → 641 | 1432 → 206 | | a remote file open: Look `/n/peer/os/.../fs.zig` (77 KB) | 8 → 6 | 7 → 2 | 213 → 64 | 563 → 162 | 1454 → 416 | fails → 1300 | 1844 → 617 | | a remote file open, 1.3 MB | 27 → 25 | 26 → 3 | 807 → 103 | fails → 244 | fails → 625 | fails → 5334 | fails → 1273 | | the python client (unchanged, for scale) | msgs, RTTs | 207 ms | box | |---|---|---|---| | connect | 2 | 415 | 614 to 635 (with ssh's channel open) | | read `/index`, list `/pane` | 5 | 1037 | 1023 to 1067 | | ctl write | 4 | 829 | 818 to 852 | | body read, 30 KB | 5 | 1037 | 1024 (`-C`) to 1283 (plain) | - **Each operation is one round trip.** A directory, or a file of no stated length, takes two. The first operation through a new `ssh -L` forward pays one more, the channel opening (the Mount, 426 ms on the box). - **On the box, bytes cost more than the 9P rounds.** The link from the box to here carries 170 to 400 KB/s through ssh. A 1 MiB `dd` takes about 6 s and 4 MiB about 10 s, where 4 MiB up takes 2.4 s. A 64 KiB answer arrives 1 to 2 RTTs after the request, so the 30 KB body reads in 641 ms plain and 206 ms with `-C`. The 1.3 MB file's Rreads come in at about one a round trip (the timeline is in `rt.py`'s `TIMELINE=1`). `blast.py` measured all 21 of its Treads in flight at once at 1.9 to 3.0 s, against 5.3 s for this client's 14 and then 7. A sliding window instead of batches would buy that, but only for large files on this link (not done). Compression helps every size: `-C` is in the documented recipe, and the replica stream's zstd (phase 6) will face the same link. No local regression. Local 9P has no delay. lab.py runs the base and the new build at once with their samples alternating, 1000 each, under the perf lock, and gives the p50 ratio new/base. The server is not changed, so its rows are level. The mount rows are faster because they dial nothing. | local, p50 | base | new | A/B | A/A | |---|---|---|---|---| | python client: read a 9 KB body (server path) | 88 µs | 89 µs | 1.004 | 1.000 | | python client: ctl write `clean` (server path) | 80 µs | 81 µs | 1.011 | 0.992 | | Look `/n/peer/pane/1/body` through a local mount | 329 µs | 255 µs | 0.776 | 1.009 | | Save to `/n/peer/pane/1/ctl` through a local mount | 214 µs | 141 µs | 0.660 | 1.000 | The gates: - test/fs.py, `zig build 9p-io-test`, unit-test tty, the snapshots and test/daemon.py (20 cycles) pass. - New: `a mount's connection is kept between operations, and one the peer hung up is dialled again` (a write longer than a message on a kept connection, a miss, a hangup, a redial and a forget). It also passes on the box (FreeBSD test build). Not done, and why: - **cloud9's client should take requests while it versions;** the poke in `greet` stands in for that until cloud9 is pushed and re-pinned. 9ns (cloud9's) still waits one round trip per message. - **A write longer than one message** sends its chunks after the first one at a time: a Save of a 1 MB file through a mount is 16 round trips. Pipelined Twrites could run in the wrong order once one parks; that does not matter for `/os` (pwrite at an offset) but would for some pane files. - **A file of no stated length** is not read ahead, and its end (like a directory's) costs a round trip more. - **The CLI's `ask`** (`pardes file` from a pane, `pardes -`) and `--wait` pipeline only version, attach and walk. Their open, writes and reads go one at a time, on a connection of their own. - **A peer's Restore cuts its clients up to 200 ms after it starts.** A request on a kept connection in that window is refused `Input/output error`, where a new dial would already have been the replacement's. - **Scripting from here to a remote session** through test/ninep.py or 9p(1) is still one round trip per message (1 s for a read at 207 ms). A python helper for remote scripting could pipeline the same way; nobody scripts that way yet. ## Phase 6: predicted echo notes Built 2026-10-30 (`src/echo.zig`; hooks in `src/replica.zig`; a field on the core and one call at the end of `body_layer.renderBody`; `zig build replicas -- --echo`). Transport-free like part B: proven in part A's in-process harness, carried by the remote client. A key typed into a terminal pane goes to the authority, and its echo comes back as a delta a round trip later: 207 ms a character in a shell on the FreeBSD box. The replica now draws it at once, mosh's way, and the authority's screen confirms or takes it back. Decisions at the forks, with the ponytail lens: - **Presentation, the replica's own.** The overlay is `Replica.echo`; the core points at it (`Pardes.echo`, `.local`, and in rollback.zig's `unsaved`, so the copy neither saves nor puts it back). No digest, dump or tape sees it, and the core's emulator stays the authority's. Its only write into what is drawn is `Echo.paint`, at the end of `renderBody` for a terminal pane: the cells predicted and the cursor after them, over the body as drawn. A terminal's body is painted straight onto the grid (a body layer is a file's with context rows), and the tty and the GUI both draw the grid's cells and cursor, so both get it. A session's own core has `echo` null: one null test a terminal pane a frame, and no overlay. - **What is predicted:** a printable ASCII character at the cursor, Backspace, Left and Right, typed into the view's terminal in tty mode (keys going to its program), on the primary screen, at the live bottom, with the cursor shown. Never for a chord, Enter, Tab, Esc, Up and Down, a paste, or anything on the alternate screen (vim never sees one). More narrowly, so as never to draw what the line editor would not: - a character only over a blank cell or the same character (typing along a fish suggestion); a line edited in its middle, or a suggestion left, is the shell's to redraw; - not in the last column (wrap is the program's); - Backspace only at the end of the line, over a prompt's input or a predicted character; Left over the same; Right over typed text, a prompt's input in the colour of the cell before it (past the text's end fish's Right takes its suggestion). Each prediction is a cell (an absolute row, the authority's numbering that the delta's header carries) and the cursor after it; a newer key's cell replaces an older one's, as mosh's overlay does. - **Judged once the authority has the key and its echo has had time.** Each unit of the replica's own says which of its keys the authority stepped (`Echo.confirmed`). A stream that loses nothing keeps the order the authority sent in, and it sends what it owes of the terminals before it steps an input, so a delta that arrives after that unit was made after the key was written. 50 ms after the unit (`grace_ns`, mosh's ECHO_TIMEOUT: the shell's echo, then the authority's next frame) each applied delta judges it: a cell the screen shows goes (confirmed), one it does not is a misprediction and every prediction goes. A predicted space (Backspace's too) is judged neither way, as mosh's blanks are; the cursor after the newest key is judged with it. With no delta at all (nothing changed: no echo), a frame judges it 250 ms later (`stale_ns`). - **A key that is not predicted** (Enter, Tab, a chord, or one the rules above refuse) is the next to wait for: once the authority has it, every prediction made before it goes, judged neither way (the deltas from then show that key's doing, which nothing predicted). The keys typed behind it wait (`deferred`) until its grace has passed; then those the authority does not have yet are predicted, in order, over the screen that shows its doing (`catchUp`). One the authority has whose grace is not over puts that off a little. So the characters typed straight on after Tab are drawn where the completion left the cursor once it shows (a round trip and 50 ms after Tab), the rest at once, and never where the cursor was before it. Found by the harness: the first version predicted nothing until the wait was over, and every key typed in that time became a new wait, so a word typed after Tab or Enter at 207 ms was all late (p90 242 ms). An input that is not a key into this terminal (a click, a command) has no deferral: where the keys after it go is not known here until the authority has it. - **Shown where the echo is vouched for.** At a shell's prompt, pardes's OSC 133 marks (phase input: a B and no C since) and the cursor's own semantic content say its line editor echoes: drawn at once. Elsewhere (`cat`, a shell without pardes's marks) only once an echo was confirmed there since the last Enter, Tab or chord (`trusted`), so the first character of each line waits a round trip. A key before the last one not predicted vouches for nothing after it: the harness found the characters of `sudo ls`, confirmed after its Enter, vouching for the password prompt that Enter brought up. After a misprediction that was drawn, nothing is drawn until an echo is confirmed again (`doubt`). A password prompt or a program that ignores what is typed confirms none, so nothing typed there is ever drawn. - **Underlined off a prompt only.** At a prompt the marks vouch for the echo, and an underline trailing every character for a round trip at 207 ms is noise; off one the trust is mosh's inference, and the underline says so. After a misprediction nothing is drawn at all, which mosh's adaptive underline approximates. - **No protocol change.** No echo acknowledgement crosses (mosh's `echo_ack`): the replica's own units stand for it, and the grace for its 50 ms. No termios flag crosses either: a program's echo is inferred from confirmations, as mosh does. The hook for the remote client (its `Replica` and stream as they are): - `Replica.init(..., screens = true)` points its core at its echo (`core.echo = &r.echo`). A replica made another way (the remote line's `initDump`) does that itself once its core exists. - Per key: nothing more. `r.input(ev)` predicts after its settle, in `r.view`. - Per unit: nothing more. `r.confirm(unit)` tells it which of its keys the authority has. - Per delta: nothing more. `r.screen(msg)` judges after each delta it applies. - Per frame, before drawing: `if (r.echoFrame()) dirty = true;` judges what is stale, predicts the keys deferred whose wait is over, and says whether the overlay changed since the last call; `r.echo.changed` says the same between frames (after `r.input` and `r.screen`), for a client that draws only when its state moved (the remote line's `moved`). - Draw its own view (`core.view == r.view`): the overlay is drawn in no other, and over a terminal in tty mode only. - The clock is the monotonic one (`FrameStats.nowNs`) unless `r.echo.now` is set (the harness's virtual clock). Measured. - **The harness** (`zig build replicas -Dplatform=tui -- --echo`): one authority and one replica over part A's delayed, jittered link, the authority's pty a shell played in the harness (`FakeShell`): a line editor at a prompt with pardes's marks and Tab completion, and the programs the corpus runs in it. Each key's echo comes 2 ms after the authority writes it. The client types a corpus of 222 keys at a typist's pace (60 to 180 ms a key) and in bursts (12 to 60 ms): lines at a prompt, Backspace, Left and Right, Tab typed straight on, `sudo` and its password, vim, a program that ignores what is typed, `cat` (the kernel's echo, no marks), and a line typed before the last one's prompt. Each client frame (60 Hz, its own phase) is what its screen shows: its core rendered, the overlay where it is drawn. Ground truth is the authority's screen right after each key's echo. - **Key to its character on the client's screen** (the simulation's clock; 20 seeds, 3060 keys the shell echoed at each RTT, ms, without → with): | | 30 ms | 80 ms | 207 ms | |---|---|---|---| | p50 | 49.8 → 8.7 | 102.3 → 8.7 | 239.0 → 8.9 | | p90 | 59.0 → 15.2 | 111.8 → 15.4 | 247.9 → 15.8 | | max | 66.9 → 65.3 | 121.5 → 114.5 | 261.7 → 251.6 | With it, a key waits for the client's next frame, 0 to 16.7 ms. The tty host draws an input's frame at once, so a real client's is the frame's cost. The tail is by design: at 207 ms the p90 of the characters typed straight on after Tab is 38 ms, of a line typed ahead of its prompt 48 ms, and of `cat`'s 179 ms (its first characters of a line wait for an echo to be confirmed); every other part's is 14 to 15 ms. - **Accuracy** (the same runs, 4440 keys at each RTT): 3718 to 3720 predicted (84%); 714 to 722 not, by the rules (each Enter, Tab, Ctrl-C, Ctrl-D and Esc, everything typed in vim, and the characters behind a key not predicted that the screen showed by the time it was over). Of those predicted, 3238 to 3240 were what the authority's screen showed right after the echo; the other 480, at each RTT, are exactly the password's and the ignoring program's 24 a seed, and none was drawn. Wrong characters drawn: none, at every RTT (0 keys, 0 frames). Frames a password's or the ignoring program's key was drawn: 0. vim's keys predicted: 0. The overlay judged 97 to 132 cells against the screen as mispredictions, all of them undrawn ones (the password's, the ignoring program's), and no cursor. Every frame drawn with a prediction shown (13,083 to 21,009 a cell) had each predicted cell where it was predicted in the rendered surface. - **Costs** (ReleaseFast): `paint` p50 50 to 63 ns, p99 86 to 155 ns; the replica's render with a prediction shown p50 54 µs; a key's `typed` is a few array scans. - **No divergence**: the replicas harness (ReleaseFast, 20 seeds x 120 inputs x 3 clients): none diverged, none unequal, 23,547 deltas checked and none wrong; its deltas a second (76.3, 75.7, 71.7) are part B's to the decimal. - **No local regression.** A session's own core draws no overlay (`echo` is null: a unit test says the authority's core has none). `pardes-perf-live --live` (base's driver), main (vtzpkxwu) and this line interleaved A B A ... A on the perf cores (3 runs of the new one, 4 of main), ReleaseFast; the median of the runs' p50s, new/main: | | new/main | A/A (main) | |---|---|---| | key-down, present / core | 0.99 / 0.99 | 1.00 / 0.99 | | key-right | 0.99 / 1.00 | 1.00 / 1.01 | | edit-char | 0.97 / 0.98 | 0.99 / 0.98 | | page-down | 0.85 / 0.92 | 0.82 / 0.85 | | wheel | 0.97 / 0.99 | 0.98 / 0.98 | | `yes \| head -c 32M` wall | 1.11 | 1.04 | | a 1M-line open | 0.93 | 1.01 | Every cell's runs overlap main's. The `yes` cell is the kernel's (Tty throughput notes): main's own runs read 4.5 to 7.1 s in the A/B and 5.8 to 7.6 s in the A/A, this line's 5.0 to 7.2 s. What the local path gained: a null test a terminal pane a frame, and 8 bytes of core. The gates: - Unit tests (`zig build core-test -Dtest-filter=echo`): keys typed at a prompt drawn at once in the rendered surface, unflagged, the cursor past them, then confirmed by the echo, Backspace too, and the replica's digest the authority's; a password prompt draws nothing typed and lets it go when stale, and vim's alternate screen predicts nothing; a misprediction takes every prediction back and hides the next until an echo is confirmed, and after Tab the keys behind it are drawn where the completion left the cursor, or not at all if the screen already shows them. - `zig build replicas -- --echo` fails on any divergence, unequal end, wrong delta, drawn password or ignored key, predicted vim key, or cell rendered otherwise than predicted; it passes at 20 seeds, Debug and ReleaseFast. `zig build replicas` passes at 20 seeds, ReleaseFast (and 4 in Debug). - Unit tests tty (`zig build unit-test -Dplatform=tui`, its perf gate too), the full `core-test`, and the 113 snapshot scripts pass; the GUI build's echo tests pass (`core-test -Dplatform=gui`). Not done, and why: - ASCII only: a width table would let the rest of Unicode in (`kindOf`). - A line edited in its middle (insert, Backspace), Delete, Home, End, the readline chords, word moves and a paste are not predicted: the line editor redraws what is right of the cursor, which a prediction would have to guess. Each waits a round trip, and the keys typed straight on behind it are drawn when it is over. - A key typed after an input that is not a key into this terminal (a click, a command) and that the replica could not apply itself waits for the authority, and so do the keys after it until a pause: where they go is not known here until the authority has it. - The echo is inferred, not told. No termios flag and no echo acknowledgement cross. A password prompt is safe because nothing typed there is ever confirmed. A shell at its prompt that does not echo a key (zsh's vi command mode) has it drawn until its grace passes, about a round trip and 50 ms, and then nothing more until an echo is confirmed; so does a program that turns echo off mid-line in `cat`'s plain echo. - An echo later than about 30 ms on the authority's host (a loaded box) makes a drawn character go when judged and come back with the echo: a flicker, never a wrong character. The grace could follow the measured echo delay. - A shell without pardes's marks (fish's own marks, a shell started without the integration) is trusted only by confirmation, so each line's first character waits a round trip, as in `cat`. - The GUI was not run: it draws the grid's cells and cursor that the harness checks in the rendered surface, and its build compiles and passes the echo tests. - The remote client (the other line) wires it: `initDump`'s core, the frame hook, its `moved`. (Done since: Phase 5 part B notes, with the echo through the FreeBSD box's link.) ## The final gate notes Run 2026-10-30 on main with every phase merged (rqzuvput, then the three fixes below), in the default workspace. Scripts and logs are in `.scratch/daemon/final/`. Found and fixed on the way: - **A terminal's paint walked the page list once a cell** (uvprzmxw). `recolorAnsi` read each cell through `pages.getCell(viewport x, y)`, which walks from the viewport's top down to the row. The main build no longer inlined it. `terminal.cellAt` reads the cell from the row's own pin instead. Phase 5B's "regression against base" was something else: its core benchmark was a Debug core beside base's ReleaseFast one (the `--json` identity says which). - **glibc's `tcgetattr` overran Zig's termios** (qoprvkln). glibc's struct is 60 bytes and Zig 0.17's `std.c.termios`/`std.posix.termios` is the kernel's 44, so every call wrote 16 bytes past its caller's struct (vaxis's `makeRaw` too). The ReleaseSafe tty trapped at its first terminal (`BADF => unreachable`); Debug and ReleaseFast survived on the stack's luck. On linux-gnu, main.zig exports `tcgetattr`/`tcsetattr` as the kernel's TCGETS/TCSETS* ioctls, which touch only the 36 bytes both layouts share. FreeBSD's libc struct is 44 bytes, so nothing overruns there (checked on the box). - **Restore of a session with a command pane failed in ReleaseSafe** (xquktpqo): "end of dump". The walker loaded an optional by setting it to an undefined payload and reading into `.?`. A `?[]u8`'s null is its address, so the optional itself was undefined, and ReleaseSafe's optimizer took it as null. `Pane.command` read one byte long and every field after it fell out of step. Pointer-like payloads now load through a local. Found by tracing (type.field, offset) from the loader in a Debug and a ReleaseSafe build and diffing for the first divergence. The last two were invisible to every Debug suite. The gate therefore runs the installed (ReleaseSafe) binaries through the suites that start a real pardes. Gates, Debug unless said: - `core-test`, `unit-test -Dplatform=tui` (its perf gate too), `snap` (113/113), `replay`, `replicas` (no divergence, no wrong delta at 30/80/207 ms), `9p-io-test`, `fs-test`, `test/daemon.py` (20 cycles), `test/daemon.py --upgrade`, `test/fs.py`: all pass. - `unit-test -Dplatform=gui`: 1563 of 1575, 10 skipped, and the two that failed on main already. - ReleaseSafe, the binaries installed: a tty in a pty with the daemon on and off, `test/daemon.py` 20 cycles, `test/fs.py`, `snap -Doptimize= ReleaseSafe` (113/113), `--upgrade` Debug to ReleaseSafe and back, the GUI 5 s offscreen: all pass. - The FreeBSD main test (ReleaseSafe on both sides, `e2e_box.py`): a session on the box, two replica clients here and a terminal there, all three typing. Every digest agreed (72 checks), first frames at 3.8 and 3.9 s, a shell's answer 223 ms after Enter, 1 misprediction and none visible. No local regression, ReleaseFast, interleaved base/new on the perf cores, new/base of the medians: - The core in process, on an idle machine (`--only medium|small --reps 25`, three rounds): every cell 0.91 to 1.02. Run beside builds, the big-buffer cells (`body-2m`, `body-appends`, `open`) spread 1.6x run to run on base alone, so that run's 1.10 to 1.16 there is noise. The idle rerun is the measure. - A terminal's frame (the scrollback cells, beside builds): render 0.86 to 0.90, output 0.89 to 0.91, edit-char 0.87 to 0.91. - Startup: boot 0.93 to 1.00, first frame 0.94 to 0.96, peak memory the same. - Live tty keystroke-to-present: level within each CPU state. With the machine busy, key-down is 459/469 µs against 452/465 and page-down 370/378 against 371. Idle, it is 1553/2060 against 2033 and 1939/2102 against 1921 (the `powersave` governor). The medians across both states read 0.33 to 1.70 and mean nothing. - Live GUI: present 32 to 38 ms on both (the offscreen software present), core 220 to 560 µs on both. Installed 2026-10-30 05:27 into ~/.local/bin: pardes and pardes-gui 0.24 (bbcebf80c45e), ReleaseSafe. The previous ones (0.24 caffd9c1f874, 2026-10-29 08:57) are `*.bak-pre-daemon`, and pardes-v9fs is unchanged. ## Phase 6: content cache and zstd notes Built 2026-10-30 (`src/replica.zig`: `Peer.farJoin`, the `Framer`'s envelopes, `Have` and `Cache`; detached/wire.zig version 16; `walk.blobs`; tty.zig `remote`; main.zig `remoteAttach`; libzstd in build.zig). A far client, one through ssh, is sent a join's blobs by hash and only those it holds nowhere, and everything the session sends it after its welcome goes in zstd. Decisions at the forks, with the ponytail lens: - **Far or near, the client says.** A client dialling through ssh (`--attach=:`) is far; one on this machine's own socket is near and joins as before, byte for byte: there a hash and a compression cost more than the bytes they save (a 100 MB file's first join takes 1.5 s far against 0.25 s near, the table below). `PARDES_REPLICA_FAR=1` makes a socket client far (the tests, the local measurements), `=0` an ssh one near. The daemon reads no further than the hello, so the word is the client's first message after it (`far`), in the same write: the session's writer reads it before anything after the welcome, with no round trip. - **The protocol: the session lists, the client asks.** After the welcome (which now says how many blobs the dump holds) a far client's writer sends the list (`blobs`: each blob's place in the state, its length and its BLAKE3 hash, 48 bytes), then the state with the blobs cut out (its pieces), then flushes and waits for the client's `want` (a bit a blob), then sends the wanted blobs, in order. The client resolves the list as soon as it is whole and answers while the state is still on its way (a first join pays that round trip: not done, below). A blob listed twice (an undo step that changed nothing) is asked for once and copied. The client says what it lacks, not what it holds: its cache can hold thousands of blobs, the list of a join a few hundred. The step never waits for any of it: the list is hashed on the session's writer thread for that client, from the blobs part B pins for it. ponytail: hashed again at each far join (BLAKE3, 3.6 GB/s); a session's later joins could keep the hashes beside its words. - **BLAKE3 from Zig's std, 256 bits.** A collision or a planted file must never become wrong content, so the client hashes what it reads from disk again before it uses it, and a file of another length or content (cut short, corrupted, planted, longer) is a miss: deleted, and the blob is asked for. What comes over the link is not hashed (the session said its hash); the cache file it is written to is checked when it is read. - **Where a far client holds blobs.** In order: the core it is leaving (a rejoin after a divergence or an Upgrade: the old core is kept until the new dump is whole, its blobs looked at by length and each hashed once when one of its length is asked for), then the cache on disk. ponytail: a scan of the old core's blobs a lookup, fine to some thousands of blobs. - **The cache: `$XDG_CACHE_HOME/pardes/blobs` (else `~/.cache/...`), a file a blob named by its hash in hex, the directory 0700.** Not the runtime directory: the blobs are worth keeping across a logout and a reboot (a client started tomorrow) and safe to lose, and they can be hundreds of megabytes, which the runtime directory (a tmpfs, often capped at a tenth of memory, emptied at logout) is not for. A file is written aside and renamed into place, so a second client never reads half of one. A far client writes what it was sent on a thread after its load (the first frame waits for none of it), and at its end, the blobs of its core the cache lacks: what it made since it joined (an edit's text, its undo snapshots), so a client started again is not sent them. A hangup now ends a remote client the normal way, for that (and its ssh forward's socket is unlinked: part B's last note). - **Bounded: 1 GiB, `PARDES_BLOB_CACHE_MB` to change it; the least recently used go first.** A hit makes its file's time now; before a store, the directory is scanned and the oldest files deleted until what comes fits. A join bigger than the cap stores what fits. ponytail: the scan and sort at each store, fine to tens of thousands of files, where an index would take over. - **zstd, decided by its numbers (below), on the whole stream after the welcome, not the join alone.** The session's writer puts every byte it sends a far client through one compressor and writes envelopes (`zstd`), each the flush of at most 1 MiB of input, so a reader unpacks one whole into a bounded buffer; the client's `Framer` unpacks them and hands their messages on as if they had come bare, so the dial and the stream's reader are unchanged. The join is one frame at level 6 over an 8 MiB window: an undo snapshot is a near copy of its text, a file's length before or after it, and zlib's 32 KiB window never sees the copy. What follows (units, terminal deltas) is a frame at level 3 over 1 MiB. The client's direction is not compressed. With all of that, `ssh -C` only spends both ends' time, and is no longer in the default command. ponytail: a big paste through a slow link goes as it is; it would be an envelope of its own. - **libzstd 1.5.7, vendored as the upstream tarball** (one copy; nothing else in the build bundles one; Zig's std has a decompressor but no compressor). Its compressor and decompressor only, single-threaded, without its x86 assembly, the block compressors of levels 3 and 6 alone (`ZSTD_EXCLUDE_*`), its unused functions dropped at link; every context made by `ZSTD_create*_advanced` with a `ZSTD_customMem` over c_heap, so a test's allocations are its testing allocator's. The zstd decision, measured. A join's stream as a client reads it, taped by a tee relay (`.scratch/daemon/amort/h/capture.py`): `real` is part B's box session made here (pardes.zig, host_io.zig, terminal.zig, look.zig and 3 MB of the tree's sources joined, a shell with 1 MB of scrollback, 4.0 MB of stream), `edited` the same after three edits to each text (each text's undo snapshots: 15.8 MB), `typical` and `big` join2.py's synthetic files (their lines differ by a counter, so everything compresses far better than source does). Compressed whole with the CLIs, here (one core); the box's CPU takes 0.03 s, 0.11 s and 1.04 s for real at -3, -9 and -19: | stream | raw | gzip -6 (ssh -C's) | zstd -3 | zstd -3, 8 MiB window | zstd -6, 8 MiB | zstd -9 | zstd -19 | |---|---|---|---|---|---|---|---| | real | 4.04 MB | 1.00 MB, 91 ms | 1.02 MB, 19 ms | 0.95 MB, 23 ms | 0.85 MB, 49 ms | 0.82 MB, 58 ms | 0.72 MB, 1005 ms | | edited | 15.82 MB | 3.97 MB, 336 ms | 3.36 MB, 63 ms | 0.96 MB, 30 ms | 0.86 MB, 61 ms | 0.85 MB, 87 ms | 0.73 MB, 967 ms | | typical | 8.33 MB | 1.22 MB, 98 ms | 0.72 MB, 19 ms | | | 0.35 MB, 39 ms | 0.20 MB, 723 ms | | big | 77.4 MB | 11.17 MB, 768 ms | 6.94 MB, 136 ms | 4.10 MB, 131 ms | 3.71 MB, 336 ms | 4.80 MB, 467 ms | 1.82 MB, 22.4 s | The box's link (each payload `cat` from the box through ssh, the time from its first byte here to its last, the median of three interleaved): | payload | through `ssh -C` | zstd -3, plain ssh | zstd -9, plain ssh | zstd -19, plain ssh | |---|---|---|---|---| | real (4.04 MB raw, 10.0 s through plain ssh) | 4.32 s | 4.82 s | 4.06 s | 4.14 s | | edited (15.82 MB raw) | 8.90 s | 7.03 s | 4.51 s | 3.87 s | - **Without undo history zstd buys nothing** that the link shows: 0.82 MB against zlib's 1.00 MB, 4.06 s against 4.32 s, within the runs' spread (3.9 to 5.0 s). The link is not the bottleneck its 170 to 400 KB/s suggests for a payload this size: a megabyte or less takes about 4 s whatever it is, TCP's window growing over a 207 ms round trip. - **With it, zstd halves the transfer** (8.90 s to 4.51 s): an undo snapshot is a near copy of its text, which zstd's window of megabytes matches and zlib's 32 KiB does not, and a session one works in for an hour holds up to 256 of them a file. The window decides it, not the level: at -3 with 8 MiB, edited is 0.96 MB where -3's own 2 MiB leaves 3.36 MB (the 3 MB file's snapshots are 3 MB apart). - **So zstd, at level 6 over 8 MiB**: within 4% of -9's size at about two thirds of its time; -19 saves a little more and costs a second of the session's CPU every 4 MB (22 s for big). The cache does not replace it: a first join has nothing cached, and an edit made while no client was attached is new. The benefit, measured (`h/joincache.py` here, `h/boxcache.py` through the box, both on `h/cachejoin.py`; ReleaseFast here and ReleaseSafe for the box's pair, timings alone on the perf cores). Four joins of one session, each client with its own terminal and check file, one cache between them: the first (the cache empty); a second client while the first is attached; that second one put off the session's state (PARDES_TEST_DIVERGE), which joins again from the core it leaves; and one started after both ended. The first frame is from the client's start, ssh's forward included (2.0 s of it on the box, part B); a rejoin keeps its forward, and its time is its dial and load (PARDES_REPLICA_STATS `join_ms`), its frame drawn after. - **Through the box** (ms; `real` 3.9 MB of files, a 3.9 MB dump; `edited` the same after three edits a text, a 15.8 MB dump; main is the final gate's ReleaseSafe pair, bbcebf80, through its default `ssh -C`): | session | build and ssh | first join | second client | rejoin, dial and load | started again | |---|---|---|---|---|---| | real | main, `ssh -C` | 6535 | 7166 | a whole join | 6952 | | real | this line, `ssh` (the default) | 7069 | 2484 | 434 | 2619 | | real | this line, `ssh -C` | 7623 | 2449 | 430 | 2409 | | edited | main, `ssh -C` | 11931 | 11257 | a whole join | 10921 | | edited | this line, `ssh` (the default) | 7079 | 2442 | 452 | 2482 | | edited | this line, `ssh -C` | 6824 | 2412 | 460 | 2437 | - A client that holds the blobs is drawn 2.4 to 2.6 s after it starts where main took 7.0 to 11.9 s: past ssh's 2.0 s, 0.4 to 0.5 s, whatever the session's size. Its link carries the state (113 KB before zstd) and the list. - A first join of a session with undo history: 7.1 s where main took 11.9 s (zstd's window). Without one, level with main (7.1 s against 6.5 s, within the link's spread): zstd's 15% fewer bytes are worth about the round trip the list's answer costs. - `ssh -C` over zstd: 7.6 and 6.8 s against 7.1 and 7.1 for the first join, the same for the others; noise either way, and dropped. - The session's step while each joined (its stats, real): 0.3 to 1.2 ms on the box, main's 0.3 to 0.9 ms. - **Here, the socket** (ms; join2.py's shapes: typical 4.1 MB of files and 1 MB of scrollback, big 38.6 MB and 8 MB, huge a 100 MB file beside typical's; three edits a text; near as before, far with PARDES_REPLICA_FAR=1): | session | client | first join | second client | rejoin, dial and load | started again | the session's worst step during the joins | |---|---|---|---|---|---|---| | typical | near | 22 | 22 | 9 | 22 | 0.6 | | typical | far | 42 | 32 | 17 | 32 | 0.5 | | big | near | 104 | 103 | 66 | 83 | 1.2 | | big | far | 517 | 153 | 127 | 153 | 0.8 | | huge | near | 254 | 244 | 170 | 285 | 0.7 | | huge | far | 1514 | 517 | 450 | 506 | 2.3 | - Near is part B's join unchanged (22 to 46, 87 to 117 and 247 to 288 ms then), and the default on this machine's socket: far there costs more than the bytes it saves (the session's hashing and compression on a first join, the client's read and hash of what it holds after). - A join costs the session's step at most 2.3 ms with a 100 MB file open, far or near (the gate: under 16.7 ms). Gates (Debug unless said; scripts and logs in `.scratch/daemon/amort/`): - **Correctness.** `test/daemon.py --far` (in its main run too): far clients of a detached session with two files (one 1.3 MB) and edits; the first sent every blob (1104 KB), a second client on this machine from disk (3 of 4, the fourth the text the first had just made), its rejoin after a forced divergence from the core it left (4 of 4), one started after both ended from disk (6 of 6: what they made, kept at their end), one after every cache file was corrupted or cut in half by hand sent them all again (5470 KB), and two with a 1 MiB cap held by older files: those went first and the cache stayed under its cap. Every client's digest was the session's at every quiet point (11 to 17 checks a run, none differed), Debug and ReleaseSafe. Through the box, every check of the four joins above agreed (28 to 41 a run). `zig build replicas`: 4 seeds at 30, 80 and 207 ms, none diverged, none unequal, no delta wrong. `test/daemon.py`'s replica tests pass, near and far. - **New tests** (core-test, `-Dtest-filter=blob`): a cached blob is hashed again as it is read, and one cut short, corrupted, planted or longer is a miss and goes; the cache stays under its cap, the least recently used going first (a read makes a file the newest; a file not a blob's counts and goes by its time); a far join over a socket pair, the session's writer and reader threads against the client's dial, is sent each blob once: again with them on disk, or in the core it leaves, the state alone, and a cached file gone bad sends that blob again; a near join's dump is the session's byte for byte. - **Suites**, once at the end: `core-test` (its run beside the box measurements had the 9P failure above; again, all pass), `unit-test -Dplatform=tui` (its perf gate, and shared-libs-check, which now refuses a second zstd too), `snap` 113 of 113, `replay` (113 scripts, 20 monkey seeds, none panicked or diverged), `replicas`, `9p-io-test`, `fs-test`, `test/daemon.py` 20 cycles (its far clients included), `--upgrade` (unstamped Debug to stamped), `test/fs.py` (49 s): pass. `unit-test -Dplatform=gui`: 1568 of 1580, 10 skipped, and the two that fail on main (`tagline pointer mapping...`, `a tag caret past its row's end...`). - **ReleaseSafe** (`zig build -Doptimize=ReleaseSafe -Dhome-install=false`, the binaries the box measurements ran): a tty in a pty with the daemon on and off, `test/daemon.py` 20 cycles, `test/daemon.py --far`, `test/fs.py` (39 s), `snap -Doptimize=ReleaseSafe` (113 of 113) and the cache tests (`core-test -Doptimize=ReleaseSafe -Dtest-filter=blob`): pass. - **No local regression.** ReleaseFast, interleaved base new base new base new base on the perf cores (`core_optimize` fast on both sides), the median of each side's runs, new/base, beside base's own A/A; builds ran on the other cores throughout, so every cell is in that state: - the core in process (`pardes-perf --reps 15`): the opens 0.90 to 0.99, render 0.94 to 0.99, the keys, page-down and wheel 0.80 to 1.03, the edits 0.90 to 1.05, body-2m 0.99 to 1.04, body-appends 0.92 to 1.07 (its A/A 0.99 to 1.30), deep-sel 0.89 to 0.97 (and longline's 364 ms on base, 0.1 ms on main already); a terminal's cells (sb-64k to sb-8m) 0.87 to 1.01. The A/A elsewhere 0.94 to 1.05. - live tty keystroke to present (base's `pardes-perf-live --live`, present p50 / core p50): key-down 1.01 / 0.99, key-right 1.01 / 0.95, edit-char 0.97 / 0.98, page-down 0.99 / 0.92, wheel 1.08 / 1.04 (A/A 1.05 to 1.16); a 1M-line open 1.06 (A/A 0.92); `yes | head -c 32M` 1.22 (A/A 1.13), its runs 4.1 to 7.3 s on this line and 5.4 to 6.7 on base, as main's 7.0 to 7.3 beside base's 6.0 to 7.3 were in the final gate. A session's own path is untouched by this change: a session with no far client hashes and compresses nothing, and a near one is sent its dump as before. Found on the way: - **PARDES_REPLICA_CHECK keeps a session's units coming.** With two clients checking, an idle session sends a unit about every 300 ms on main too (`h/idle.py`: 27 checks in 8 s, near or far, main's build alike): a check wakes the session's pump, whose unit makes the client's state "changed", and the client asks again once quiet. Only the test knob does it; a harness that expects a divergence to wait for a key it sends finds the client joined again before (the measurements above wait for the first link's stats line instead). - **A rejoin's "joined again" line can be gone before it is drawn**: it lasts until the next key, and keys typed during a long rejoin (a 100 MB file's) come first. As designed; the harnesses do not look for it. - **Harness traps.** A terminal's frame request split across two reads went unanswered (test/daemon.py's `Term` now keeps the last bytes of a read); keys written right after a first frame raced the terminal's answer to it, an answer behind an Esc in one read being an Alt chord (the far test waits 0.3 s, and writes the Esc on its own); a `pkill -f` whose pattern was in the shell's own argv killed the shell. - The core-test run of the suites had one failure, `9p_io`'s "a change waits while the editor is out mid-step", with the box measurements and builds running; alone three times, and the whole core-test again, it passed. Not done, and why: - **A first join pays the list's round trip** (207 ms on the box): the wanted blobs wait for the client's answer, and the state (113 KB before zstd) no longer hides it. A client with nothing cached could say so in its `far` word and be sent everything at once; it is a first join on a new machine only. - **A blob is hashed again at every far join** on the session's writer thread (a 254 MB dump: 70 ms there). The hashes could be kept beside the blob words. - **What a client sends is not compressed** (its keys, and a paste): with `-C` gone, a megabyte's paste crosses the box's link raw. - **A client killed outright keeps on disk only what it was sent**: what it made since (its edits' texts and undo) is written at its end, a hangup's included, but not on a kill -9; the next join gets those again, in zstd. - **The cache is per user, unencrypted, 0700**: a blob of a file the user could read elsewhere, kept until evicted or the cache is cleared. ## Phase 6: GUI replica notes Built 2026-10-30 (`src/gui/gui.zig` `remote` and the window session's `Peers`, `src/replica.zig` `Link`, `src/views.zig` `within`, the `builtins.zig` registry's `here`, `Event.frontend`, wire 16; `test/daemon.py --window-clients`). A window is now a replica client, of a terminal's session or a window's, and a window's session serves replicas of either kind. Decisions at the forks, with the ponytail lens: - **The window client is the tty's, with a window for a terminal.** `pardes-gui --attach=:` and `--attach=:` (ssh, main.zig `remoteAttach`, now the frontend's own `remote`) make the window as a session's is (`runNative`) and run `remoteSession` instead of `localSession`: the dial, the dump's replica checked against the session's digest, a reader thread, the rejoin after a divergence or an Upgrade's `refused`, the end in words. What is frontend-free in the tty's `Remote` moved to `replica.Link` and both use it (`input`, `take`, `drawn`, `quietCheck`, `rejoins`, `redial`, the stats and checks); the tty's keeps its draw. `pardes-gui --attach` and `--attach=` (no colon) re-open a local session's window through the daemon as before. - **The window's 52 writes into its core are one funnel, read over the client's own view.** The window feeds its core through `Input.post` (dispatch, the wheel, the touch and gamepad paths, the timedemo, the test feed), and a handful of frame chores call `core.update` themselves (its size, the pointer under a frame that moved, the smooth scroll). In a client `Input.post` appends to the client's list (`Client.post`, texts copied) and nothing writes the core. Mapping a pointer reads the core's view fields (its rects, its presentation, its row metrics), and another client's input may have made the replica's live view another's: so the window's work runs inside `Replica.within` (`views.within`: the live view parked, the client's unparked, put back after exactly as it was, as `drawOne` draws), and its inputs are applied after, one `Link.input` each. With one client the live view is its own and `within` is a call. - **No pump in the client.** Its core moves only by its own inputs and the session's units; it draws its view between steps (`Replica.draw` through the window's own `present`), puts back what presenting writes in the core (a skipped image, a frame owed) and records its frames as the tty client does. The window's per-frame host chores that write the core stay out of a client: opacity, font requests and their answers, a scene effect the GPU refused, user shader files (their compiles' messages). ponytail: a client's window keeps its own font, opacity and shader files; the built-in scene effects follow the session's settings. - **The wheel in a client moves a row a turn.** The window's glide and a touchpad's fraction run on the core's clock, which in a client is the session's and moves only with its units. ponytail: whole rows, no glide; the local window's is unchanged. - **A client says its frontend first** (`Event.frontend`: windowed, native images and patches; wire 16, client tag 0x20): the view the session opens for a joiner copies the live view's, and a window's view of a terminal's session would have drawn pictures as cells and the window's words would have been refused there. A terminal's client says it is no window, and draws pictures as cells (ponytail: a kitty-graphics terminal client was drawing none before, its shm never probed). - **The terminal's build and the window's step alike.** The semantics hash named the platform, so a window could not join a terminal's session (the FreeBSD box has terminals only) nor a terminal a window's. Their cores differ in the step only by the window's own words (Fonts, Pet, Font, TaglineSize, WindowOpacity, Ligatures, Lift and its family, the scene effects, their leader paths `SPC t f` and `SPC t r`) and the Settings report. Three ways: refuse cross-build joins (the box and a window's clients impossible); accept known divergences (Help's index, `SPC t f` and Settings would each make the other build's clients join again); or give both builds every word and let the *view's frontend* decide (kakoune's way: each client its own). The last: a desktop build's registry is the window's (`builtins.capabilities`), and `builtins.registry.here` says whether the live view takes a word (a window's own words in a window's view alone, `Pardes.windowed`, which a window's build starts true and a client says). `runBuiltinFrom` refuses the rest in the words the terminal said before ("Lift is GUI-only, invalid here"), and the leader paths, Help's index, `/commands`, the root ctl's settings and the Settings report list the view's own. The render keeps the build's own frontend (`builtins.frontend`: the tty draws no lift). Local behaviour is unchanged: the 113 snapshots, which hold the tty's Help index and leader groups, pass as they were. The hash names a family (`desktop`) for the two; the tty binary carries fonts.zig and the window's builtins (a few KB). - **A window's session takes replicas at its step boundary,** as the tty's does: `Peers` in the window's Shell, its wake an SDL event, `boundary` after each pump, a Restore and an Upgrade letting clients go (`dropAll`, `refuseAll`), Detach in a client's view ending that client. What the tty's session does with clients, the window's does too: one person's input a wait (`split`: the next SDL input held for the next wait, so a pump is one unit of it), a pump woken only by a client's post reads no new time (`mail_only`), and its last window closed with clients attached is a Detach, not the session's end. - **A client's view is no window here.** The window's Shell kept the live view's window in the Gui (`follow`); a client's view live (its input the last) has none, so `follow` keeps the window it holds (`windowed`, a mask of the views that are windows), and what the window does to its own view makes that view live first (`windowView`: an SDL event of the window, `Input.post`, the frame chores when one is due: its size, its scroll, a font). Found by the first test with a window typing beside a client: the window's keys went into the client's view. - **What the window writes into the core outside a step is taped.** The window's font (observed, requested, taken, acknowledged, refused), its opacity's answer and a refused scene effect are core calls now (`tape.Input`), so a window session's clients replay them; the rest of its writes were taped already (sizes, pointers, panel acknowledgements, messages). ponytail: an adopted shell's pane (`Adopt`) and a terminal reader's failure still write the core directly; neither is taped. - **The far path is the tty's** (amort's line, merged under this one): through ssh the window client is far as the tty's is (`PARDES_REPLICA_FAR` either way), its join's blobs found by hash in the core it leaves or the cache on disk, its stream in zstd, what it made kept at its end (`replica.Have`, the same calls in `remoteSession` as in tty.zig's `remote`, through `Link.load` and `Link.redial`). One wire version, 16, carries amort's far word (the first message after the hello) and the frontend (a client's first input). - **A check once a quiet while, not once a wake.** The tty's client asked the session's digest on its first wake after a change; a window session in test mode pumps a frame a tick, a unit each, so no wake ever came first. A client now asks once it is quiet, 300 ms after its last input and its last check (`quietCheck`); the answer is in stream order, so it holds with units flowing. Found on the way: - **A pty/run's answer read the replica's emulator** (vnolzlvq). A run's end (`ninep/pty.zig finish`) took the command's output off the terminal in the step; a screen-state replica's emulator is its deltas', which a lagging client's are states behind, and replaying the end read pins into pages it never had: ghostty's `leftWrap` overflowed (a window client stopped while a shell flooded the session aborted 3 runs in 8). The output is taped now (`.prompt`, the same site as a prompt read after output); none in 6 since. - **A client cut off while lagging could end in silence** (ussmwtkk): its own write found the link shut before its reader did, which ended the loop as a quit, exit 0, no words. A failed write is the session's hangup. - **The rejoin's drain** (the coordinator's ymzywopz, on main): the window client joins its reader before it drains, so it never had that race. - **A second's turn is a misprediction** (ssruwvql): an edit stamps its file with the clock's second, which a replica guesses (its last unit's time, the time since and the link's round trip); a key the session takes in another second than the guess goes the slow way, replayed. Right, and rare live (the round trip's jitter against a second); but a test's replica has no round trip yet and its authority takes the keys after all are typed, and `another client's input first` counted exact fast confirmations: under load in a Debug window build (a slower step) they straddled a second in 11 runs of 40. The two tests that count them start early in a second now: 0 of 30. - **Harness traps:** a window's test feed is a pipe, whose lone ESC waits for the next byte (the parser cannot tell it from a sequence's start), so the tests send Escape as `CSI 27 u`; a terminal client's Esc is resolved by the 50 ms wait for more input, which the test terminal's answers to frames (`CSI 0 n`) can keep refilling while other clients' units arrive (main's `normal_again`); a window session in test mode starts no daemon unless PARDES_DAEMON says so. Measured. ReleaseFast, unstamped (one semantics here and on the box), the timings alone on the perf cores (lock.sh), the windows offscreen (SDL's offscreen driver, software present); the harness scripts are in `.scratch/daemon/guirep/live/`. - **Keystroke to present on a window client, local-only actions** (base's `pardes-perf-live --live-gui`, its timedemo's keys on a 50k-line file, through `gui-live.sh`: a local window, or a window client of a terminal's session detached on that file, through test/delay_relay.py at each RTT and through the box's link; two rounds; p50 (p99), ms): | | key-down | key-right | edit-char | page-down | wheel | |---|---|---|---|---|---| | a local window, this line | 31.1 (40.3) / 31.5 (45.5) | 34.8 (44.2) / 35.3 (41.0) | 35.9 (44.4) / 36.5 (46.2) | 37.0 (50.0) / 37.8 (50.3) | 36.5 (47.3) / 36.9 (52.8) | | a local window, base | 29.0 (35.9) / 31.5 (37.5) | 35.1 (45.9) / 35.3 (42.2) | 36.2 (42.2) / 36.5 (45.1) | 37.2 (48.1) / 37.5 (52.5) | 36.5 (50.0) / 36.8 (44.2) | | client, direct | 31.3 (40.4) / 28.6 (36.2) | 35.1 (47.3) / 35.1 (54.4) | 36.7 (41.1) / 35.8 (44.0) | 36.0 (46.4) / 35.4 (40.8) | 35.8 (45.1) / 35.2 (45.8) | | client, relay 30 ms | 29.9 (54.6) / 29.9 (42.7) | 34.9 (45.8) / 36.4 (53.6) | 35.7 (54.0) / 35.8 (45.8) | 34.9 (52.1) / 35.1 (47.3) | 35.1 (44.0) / 35.1 (42.6) | | client, relay 80 ms | 32.7 (39.5) / 30.5 (36.6) | 38.1 (46.6) / 34.9 (44.0) | 39.1 (57.4) / 35.6 (46.1) | 37.7 (56.5) / 35.1 (43.9) | 38.0 (46.7) / 34.8 (44.5) | | client, relay 207 ms | 31.1 (35.9) / 30.6 (36.2) | 35.6 (46.3) / 34.6 (59.9) | 35.9 (45.4) / 35.5 (55.3) | 27.9 (56.6) / 34.9 (47.8) | 36.9 (55.3) / 34.6 (50.8) | | client, the box (far, zstd) | 33.7 (48.1) / 28.3 (43.9) | 31.1 (43.4) / 30.2 (46.4) | 30.9 (44.6) / 29.4 (39.3) | 33.8 (51.0) / 29.6 (48.4) | 28.7 (49.9) / 30.1 (43.2) | - **A window client's local-only action is as fast as a local window's at every RTT, the box's included.** Both are the offscreen software present's (30 to 38 ms, the final gate's GUI numbers): what the harness can tell apart is the core's share, the client's render 0.4 to 0.5 ms p50 (0.1 to 0.9 through the box) against a local window's update and render 0.24 to 0.6 ms. The client's render runs on its rollback epoch's allocator, which a local core's does not. - **Visible rollbacks** (a frame drawn after an input the session recorded otherwise, an upper bound; of 664 inputs a run): none direct and at 30 ms, 10 and 0 at 80 ms, 0 and 9 at 207 ms, 15 to 24 through the box (2.3 to 3.6%). Every other rollback (the session's own pumps' units before the client's pending inputs: 536 to 610 a run here, 268 to 295 through the box) is put back and replayed before the next frame, p99 2.6 to 8.9 ms. - **Terminal echo** (`echo-gui.py`: a key read off the window's test feed, to the end of the GPU frame that shows it, PARDES_TEST_LATENCY's notes; 40 keys; p50 / p90, ms): | | file (insert) | shell, predicted | shell, not predicted (PARDES_NO_ECHO) | |---|---|---|---| | a local window | 2.8 / 4.6 | | | | client, direct | 2.5 / 4.5 | 4.3 / 9.7 | 25.5 / 29.7 | | client, relay 30 ms | 2.6 / 4.7 | 5.1 / 11.0 | 60.4 / 64.8 | | client, relay 80 ms | 3.1 / 4.6 | 4.7 / 4.9 | 101.9 / 121.9 | | client, relay 207 ms | 2.7 / 8.2 | 4.4 / 10.0 | 236.8 / 248.9 | | client, the box (far, zstd) | 4.2 / 5.2 | 4.2 / 4.8 | 242.5 / 243.3 | (Through the box, the harness's ssh had `-C` beside the stream's zstd; a predicted key never waits for the link.) Every key of the shell's was predicted and confirmed (40 of 40, at each RTT): a key at a prompt shows in the window's next frame, as a key into a file does, the 0.5 ms of the tty client's being a terminal's write where a window's is a frame (its software present's 2.5 ms). Not predicted, it is a round trip and the session's paced output. - **Through the box, its join** (far, amort's path): the main test's 400 lines, 4 blobs, 51 KB sent, dial and load 0.63 s (ReleaseSafe 0.99 s), its first frame about 2.6 s after it started (ssh's forward is 2.0 s); its rejoin after a forced divergence from the core it left, 4 blobs of 4, nothing sent, 0.42 s. The live harness's 3.1 MB file: 3126 KB, 2.3 to 2.4 s. - **The session's step while a window client lags** (`lag-gui.py`: a shell printing a line a millisecond for 12 s, a terminal's client typing a key every 30 ms, the window stopped with SIGSTOP and cut off at a 256 KB lag; µs, p50 / p99 / max; ReleaseSafe): the terminal client's inputs 30 / 75 / 289, the boundaries 49 / 148 / 764, the window cut off (`cut 1`), and let go on it said "let this client go" and ended. Gates (Debug unless said; scripts and logs in `.scratch/daemon/guirep/`): - **Correctness.** `test/daemon.py --window-clients` (new): window clients of a terminal's session at 0, 30, 80 and 207 ms beside a terminal's client, every check of every client the session's digest (28 a run, none differed); one put off the session's state joined again, and again across an Upgrade (agreed 7 times), and detached; its far client's first join 9 blobs, 57 KB, its rejoins 9 of 9 and 10 of 10 from the core it left, nothing sent; a window's session served a terminal's client and a window's (agreed 9 and 7 times); nothing left behind. Debug and ReleaseSafe. `zig build replicas` at 30, 80 and 207 ms: none diverged, 1545, 1590 and 1512 deltas checked, none wrong. Through the box (`live/e2e_box_gui.py`: a session on the box, a window client and a terminal's client from here, a terminal on the box attached beside them, a file's edits, scrolls and moves, a shell's command, a second window far, forced off the session's state and joined again): every check agreed (the window 128, the terminal's 132; ReleaseSafe 131 and 134), none mispredicted, the second window's first join 4 blobs 51 KB, its rejoin 4 of 4 from the core it left, nothing sent; nothing left on the box. - **The window's own reopen** (`test/daemon.py --gui`, 5 detach and attach cycles, a second window detached alone): pass, Debug and ReleaseSafe (SDL's offscreen driver). - **Suites**, once at the end (before the tests' fix and the rebase below): `core-test -Dplatform=tui`, `unit-test -Dplatform=tui` (its perf gate), `snap` 113 of 113, `replay` (113 scripts, 20 monkey seeds, none panicked or diverged), `replicas`, `9p-io-test`, `fs-test`, `test/daemon.py` 20 cycles, `--window-clients`, `--upgrade` (unstamped Debug to stamped), `test/fs.py` (49 s): pass. `unit-test -Dplatform=gui`: 1568 of 1580, 10 skipped, and the two that fail on main (`tagline pointer mapping...`, `a tag caret past its row's end...`); `core-test -Dplatform=gui`: the caret one, and in two of its full runs a replica test's count of fast confirmations (below, ssruwvql; after it, the core-test runs pass but for the caret one). After the rebase onto main's `--ps` (prvoxvzk): `core-test` tty and gui, `test/daemon.py` 20 cycles (its `manage` included), `--window-clients` (its `--ps` checks: a terminal's session with window clients listed alone as `(tty, detached)`, a window's serving clients as `(gui, attached)`), `--gui` 5 cycles: pass, `core-test -Dplatform=gui` but for the caret one. - **ReleaseSafe** (`-Doptimize=ReleaseSafe -Dhome-install=false`, the binaries the box's second run used): `--version`, a tty in a pty with the daemon on and off, a window session offscreen for 5 s, `test/daemon.py` 20 cycles, `--window-clients`, `--gui`, the lag test (`live/lag-gui.py`, above), `test/fs.py` (38 s), `snap -Doptimize=ReleaseSafe` 113 of 113: pass. Again after the rebase, built afresh: `test/daemon.py` 20 cycles, `--window-clients` (its `--ps` checks), `--manage`, `--gui` 5 cycles, `test/fs.py` (38 s): pass. - **No local regression.** ReleaseFast, interleaved base new base new base on the perf cores (`core_optimize` fast on both sides; base is `.scratch/daemon/base`'s), the median of each side's runs, new/base, beside base's own A/A; the new binaries are this line before its rebase onto main's last two (a test's rest and `--ps`, neither on these paths): - the core in process (`pardes-perf --reps 15`, idle cores, the second of two runs): the opens 0.98 to 1.03, render 0.96 to 0.98, the keys, page-down and wheel 0.95 to 1.02 (longline's 0.79 to 0.87, main's already), the edits 0.90 to 1.03, body-2m 0.95 to 1.02, body-appends 0.95 to 1.02, deep-sel 0.92 to 0.99 (longline's 360 ms on base, 0.1 ms on main already), a terminal's cells (sb-64k to sb-8m) 0.88 to 1.02. The first run's two outliers did not come again: body-appends on the board's fixture 1.29 (A/A 0.96; then 1.02 alone, A/A 0.99, and 1.06 with the other cores busy) and resize-cols at 1M lines 1.10 (then 1.02). - live tty keystroke to present (base's `pardes-perf-live --live`, the other cores kept busy by spinners at nice 19, as idle cores let the powersave governor put the package to sleep between keys: base's runs read 347 to 1651 µs; present p50 / core p50): key-down 1.03 / 1.03, key-right 1.00 / 0.94, edit-char 1.00 / 0.99, page-down 1.01 / 0.95, wheel 1.05 / 1.05 (A/A 1.00 to 1.02; base's runs 389 to 469 µs, this line's 444 to 480), a 1M-line open 1.01, `yes | head -c 32M` 0.63 (A/A 1.03). - live window keystroke to present (`--live-gui`, offscreen; present / core): idle 0.93 to 0.98 / 0.94 to 1.03 (A/A 0.95 to 1.05), busy 0.90 to 0.97 / 0.98 to 1.03 (A/A 0.85 to 0.97 / 0.97 to 1.10). Not done, and why: - **A window client paints nothing ahead.** A key whose frame the session owes (an image's, a PDF's lane) waits for its unit; a local window's paint-ahead is the core's, which a client's moves only by units. - **No glide in a client, and a touchpad's fractions round to rows** (the core's clock is the session's there, above). - **The notice a client gets is in its title and its log, not its frame** (the session's words: joined again, let go, refused). The tty's draws them on its notice band, which the window's frame has no slot for outside the core. - **A client's window keeps its own font, opacity and shader files**: the session's `Font`, `Fonts` and `WindowOpacity` move the session's own window, and a client's follows them only through its core (the scene effects do; the font does not). `Fonts` reads the machine's font directories untaped, so its list is the session machine's. - **An EffectCode's sources differ by backend** (Vulkan or GL): a client of another build's window reads the session's. - **A terminal's client draws a window session's pictures as cells**, never through kitty graphics (no shm across machines). - **Adopt and a terminal reader's failure write the core untaped** (above): a client replaying them would diverge and join again. - **A rejoin opens a new view**: the old one's place, selection and scroll are the session's to keep, and it lets the old view go. - **A hit test reads the live view**, which `within` makes the client's for its own events; a frame chore outside one (a pointer's hover after a unit moved the frame) reads it as the unit left it. - **A window client has no SIGHUP to quit on**: its ssh's end is the link's, read as a hangup. - **One window a client**: a session's second window through one client would be a second client. ## The final gate, again, on everything merged Run 2026-10-30 on main lprqtsom (the content cache and zstd, the GUI replica client, `--ps`/`--kill`, the four flaky tests' fixes) with the same scripts (`.scratch/daemon/final/final2.sh`). - **Debug:** - `core-test` tty passes. `core-test` gui and `unit-test -Dplatform=gui` fail only on the two tests main already failed, 1568 of 1580 with 10 skipped. - `unit-test` tty, `snap` (113/113), `replay`, `replicas`, `9p-io-test`, `fs-test`, `test/daemon.py` (20 cycles, `--manage` and `--far` included), `--upgrade`, `--window-clients`, `--gui` (5 cycles) and `test/fs.py` pass. - **ReleaseSafe, the binaries installed:** - a tty started in a pty, daemon on and off; - `test/daemon.py` 20 cycles, `--window-clients`, `--gui` 5, `--upgrade` from Debug, `test/fs.py`, `snap -Doptimize=ReleaseSafe` (113/113). All pass. - **The FreeBSD main test:** ReleaseSafe at both ends. Two replica clients here and a terminal on the box. Every one of 130 checks agreed, with 0 mispredicted. First frames came 2.5 and 2.6 s after start (3.8 s before the content cache and zstd), and a shell answered 224 ms after Enter. - **The A/B against base:** ReleaseFast, interleaved, idle machine, `core_optimize` fast on both sides. - The core: medium 0.89 to 1.11, small 0.93 to 1.08. The edit cells were the high end. Rechecked on the medium file with 40 reps and four rounds, edit-twin reads 0.99, edit-char 1.00, key-down 0.99 and render 0.95: the noise this cell's A/A showed before (0.79 to 1.46). - Live tty keystroke to present: each run within base's own spread (key-down 1769 to 2071 µs against 1586 to 2007). - Live window: present 32.7 to 46.9 ms against 32.3 to 48.6, core 227 to 568 µs against 229 to 581. - **Found on the way, not fixed:** once, a window client of a session on the box diverged at every rejoin, each time at the first unit's digest (tape byte 119), for the 25 s it ran. The session was a directory and two shells. - Not reproduced since: not by the same launch, not by four session shapes, and not by the session run with per-field digests (PARDES_REPLICA_TRACE, which never diverged). - The single-word digest is `walk.digest` over the whole core. The per-field parts cover the core's fields, its stream and each pane's Pane, File.State and Places. So the word covers something the parts do not. - The next step is a client that, when a word differs, asks the session for that unit's parts, so the field is named the first time. - The client heals by rejoining, at the cost of a redraw each time. ## The remote bugs the user hit > due to the ping I know it won't be possible to have the same snappy > feeling as the local version, but these kinds of errors have nothing to > do with this limitation, you should do it much better, and measure it > too since I don't want to catch those myself The user's window client of the box blinked, lost its panes and jumped its cursor: each was a divergence, and the rejoin after it opened a fresh view. An adversarial soak of replica clients (`test/soak.py`, a session here or on the box, terminal and window clients through a jittered delay relay, a seeded monkey of a person's actions) found the causes; the fixers took them by root cause, each with a test that fails before its fix. Ids are the hunt's (`.scratch/daemon/hunt/findings.md`). ### fix4: a kind's open, and the bands a joiner's kind was told Two the kinds work found (2026-11-10, `.scratch/arch/kinds9-report.md`), both the rule already written above, broken at one site each. - **A kind's open was the one call a replica guessed.** Every other call into a kind (a key, the mouse, an address, a search, a ctl, the tick) is authority-only (`authorityOnly`): the replica has no kind's code. The open (`Kind.reopen`) was not, so a replica's guess of a word that opens a kind's pane with no file (Snake, Life, Fake, by command, Look or leader) ran with `ops == null` and defaulted its outcome, where the authority's was the kind's: mispredicted every time. Fix: one line at the top of `Kind.reopen`, which a Look's open, a revived placeholder and a kind's reload all pass through. Test at the end of `src/replica.zig`. - **A view's facts read derived state a joiner had not derived.** `Kind.viewOf` told a kind its view's body rows minus its bands, read from `Pane.notices` (the live view) or `Pane.pv[k].notices` (another), both `derived` in the walk and so never dumped: a client joined from a dump had them empty until a step recomputed them, its kind was told other rows than the session's, and the join diverged (`digest 0 differs in panes pane 1.addon`; sequence: open, second view, resize, a message on the kind's pane, an input of the second client, join, tick). Fix: `Messages.collectBands` became the pure `Messages.bands(p, view, pane, tag_rows, rect, id)` from dumped state alone (the pane's messages and prompt, the view's leader, rect and tag rows); `viewOf` calls it per view and `collectNotices` assigns it. Replicating the notices was rejected: `draw.place` writes `pane.notices` outside a step, a divergence of its own waiting; it still does, harmless now that no fact reads it. No wire or digest cost. Test at the end of `src/replica.zig`. - The rule, restated: a fact told to a kind, or to anything a digest covers, is derived again from dumped state at the call, never read from a copy a step or a park left behind. ### fix3: what the replica stream carried in its build's own words A unit of the replica stream is its session's tape, written by `walk.encode`, which said "same build only": a window client (pardes-gui) of a terminal's session, or of the box's, read another build's bytes. Found by the variant hunt (V2, E), with real ReleaseSafe executables. - **An error crossed as its number** (`@intFromError`, read back with `@errorFromInt`). The numbers are a build's own: 451 of 505 shared names are numbered otherwise in the Linux window's build than in the terminal's, 244 between the Linux and the FreeBSD terminal's (FileNotFound is 62 in both terminals and 60 in the window, whose 62 is NoDefaultValue). A terminal's tape replayed by pardes-gui panicked at its first failed read (`unexpected error code, found error.NonLocalPath`); a replica read the wrong error, and diverged. - Fix: an error is taped by its name. A set decodes it among its own names; a name this build lacks is `error.NoSuchError`, a divergence, never another error. `anyerror` keeps no list of names, so the tape's five (a host's reports: `shell_failed`, `save_failed`, ...) read back as one of `tape.Failure`, the union of the sets a host reports from. ponytail: kept by hand; one outside it panics a Debug build as it is taped, and is a release build's divergence. - **A stat crossed as `std.Io.File.Stat`** (the `os_stat` site, /os over 9P), whose `nlink` is 4 bytes on Linux and 8 on FreeBSD and `permissions` 4 and 2: a Linux client read a FreeBSD session's stat out of step. Fix: the site returns `fs.OsStat` (kind, size, mtime), the same everywhere. - **The guard:** `tape.shape`, what every type the tape carries is laid out as (each field, element and tag, an enum's names and numbers; an error as an error), goes into `wire.semantics`. Builds whose tapes differ in shape refuse each other's replicas in words at the hello, as builds of other options do; a site whose result `tape.results` does not list is a compile error. The sweep that set it (every type the tape reaches, from the Linux terminal, the Linux window and the FreeBSD terminal builds): the stat was the only one laid out otherwise. All three builds now hash to one semantics; with the stat taped as before, the Linux and the FreeBSD build hash to two. No wire version for it: every build's semantics changes with it, so an older build is refused in words. - **A search's answer was the build mode's (E).** regexp.zig's step budget was 4M in Debug and 32M in a release build, and semantics names no mode: `\s*(\w+)\s*=` over 200 letters was TooSlow in a Debug replica and no match in its ReleaseSafe session. Fix: 32M in every mode (a Debug build's worst search takes about two seconds). - **Found by the cross-build replay, the same class:** a core's first view was a window's by the build's default (`windowed = platform == .gui`), so a window's build replaying a terminal's session started otherwise (its init file's window settings applied); now an option (`Options.windowed`), taped with the rest. And a window's font words (`observe_font` and the three beside it) were no-ops in a terminal's build, so a terminal's replica of a window's session diverged at the window's first font; every hosted build takes them now. - **Tests:** walk.zig's "an error is taped by its name" and regexp.zig's "a search's answer is every build mode's" (from the variant hunt; it fails in Debug before). `test/cross_replay.py ...`: a session that meets the host's failures (a Look of a fifo and of a directory, a Save into a file, a Restore of no dump, a stat and a read under /os), recorded by one build and replayed by the others, and `--record` and `--replay` alone for the box. Another frontend's tape must agree in the replica that draws nothing, as a replica client draws none of its session's frames: the two that draw meet the other build's drawing, which owes frames for animations of its own (on 132 snapshot tapes, a window's build agrees in all three on 13). `zig build replay -Dreplay-with=` replays every snapshot script's tape with the window's build as well. - Before: pardes-gui panicked replaying a terminal's tape, and live, a window client (main's ReleaseSafe) of a session on the box (main's) panicked the moment the session looked at a fifo: `unexpected error code, found error.NonLocalPath`. - After: every direction agrees, tty to gui and back, the FreeBSD build's tape replayed here by both Linux builds and theirs on the box; live, a window and a terminal client here of a session on the box agree after the same failures. - **The second round (H19, the linger leak, H15, the web build):** - **A Linux client could not join a FreeBSD session holding a +LocationsConfig (H19):** `the session's dump loaded as another session`. An enum was digested by its number, and `registry.Builtin`, made at compile time, has Tty9p in a Linux build only, so the pane's origin (`Output.State.from = .{ .cmd = .LocationsConfig }`) was another number in each. Found by one dump loaded in both builds, its digests printed field by field. Fix: an enum is digested by its name's word (a fixed-width hash, looked up by number), as the dump already reads it by name; one numbered by hand stays its number. Before, main's window client of main's session on the box could not join; after, a terminal and a window client here join a FreeBSD session holding a +Search, a directory, a renamed file and a +LocationsConfig, and agree. cross_replay.py's session holds the last two (a FreeBSD tape replayed here diverged at `pane 2.file.output`). - **A detached session's messages went in the host's wait** (tty waitInput, gui waitDetached), untaped, so a replay, and every replica of a session nobody attached, kept a message its session had let go; with `MessageLinger 0` (test/daemon.py's sessions) every tape whose session said a failure diverged at pane 0's message. They go in the pump now, after its wait, the clock read taped. cross_replay.py's session says `MessageLinger 0` again. - **Adopt and a window's failed terminal reader wrote the core between steps (H15):** the Tty word run and the greeting cleared straight on the core, and a pane's mode set; every client of a session that took in a dead session's shells joined again. Now inputs (`adoptShell`, `readerFailed`); replica.zig's test fails with the old writes. - **`zig build web` failed** on main at terminal.zig:601 (`c_allocator` with no libc), and at walk.zig's blob lists behind it: both are a hosted build's alone now. The web build passes `touch.mjs`; the web driver's own CDP-deadline test fails as it did, untouched. - **A Grep or Find a 9P write runs** searches on the loop (Stream.runHere, so that the write's answer says whether it found anything), and its reads were no tape site: every replica of a session a script greps in joined again (`a syscall that is no tape site, made replaying`). The search is a site now (`search`), its batches what a replica takes, as it takes a worker's (Event.search). cross_replay.py's session greps and finds; a terminal client of a session a script greps in agrees. - **A +Search's rows were recoloured in place** (`Query.recolor`, as a paint lands): the rows are a blob, whose word a digest keeps by address and a rollback copy pins, so a release session kept the word of their colours before; an edit moved them into `dropped_marks` with it. A replica that hashed them at another time diverged there (fixer 1's soak, `pane 5.file.dropped_marks`), and a client loading the session's dump hashed the colours it carried: "loaded as another session". They are made again now (`locations.cloneRows`). `dropped_marks` stays in the digest: an undo back to its text reads it. Stream's test fails in Debug before (`written in place`). ### fix2: what a step read that only one side had, and units that ended undrained Each of these made every client of a session diverge at the same unit, so the session's step and its replicas' replay disagreed. None was a lag. - **A step read what only the authority holds** (H4, H8, H11, H12, H13, and the variant hunt's list). The store's closed panes and Recent are the authority's (`.authority`: every window of the user writes the store's file); a replica keeps its own, which has no file and only what it replayed since it joined. Steps made digested state of them: - the Panes picker's closed rows and Recent's rows (H4: every client diverged at the key that opened them; H13: a client on another machine read its own store); - `trackJump`, at every step, dropped a closed file's jumps when Recent did not know the file (H11: once a pane had closed, every client that joined diverged at its first unit, forever, in `jumps njumps jcur`); - `noteClosed` looked in Recent before it asked the host whether the file is there (`recentKeeps`, a taped stat), so the session skipped a syscall the replica made (H12: every Del of a file crashed the client); - a closed pane brought back (`Store.reopen`: a Look at `@pN`, a Panes row), a jump to a closed file and the +Jumps rows (`recentByHash`); - and, the same class: a Look or exec handed to a script reading a pane's events (`pane.fs.readers`, the 9P server's), Tty9p's socket check, and the session's config directory and startup file (`opts`, `.local`: ThemeFile, DumpThemes, Config, DumpConfig, an init file saved). **The model:** what only the authority holds stays the authority's, and a step reads it through `tape.authority` (src/tape.zig): a `value` read, its answer taped; a replay takes the answer, and a speculation that reaches one waits for it, as at a host call (one RTT, never wrong). A replica's own store and Recent are never read by a step. Rejected: the store and Recent as session state (dumped, digested, copied), which needs the store's file I/O on the tape call by call (it reads other windows' records at every step that writes), for the same rows. - `trackJump` no longer reads Recent: a closed file's jump stays, and the jump to it drops that file's entries when Recent has forgotten it, as when the file is gone. - `noteClosed` asks the host first, on every side. - The tty toggle, read on every key (which cannot wait a round trip), is session state: `Pardes.tty_toggle`, from the options when the core is made, dumped. - **A rollback put back pointers to freed memory** (H8, the segfault in the Panes picker). The copy saves a struct's bytes field by field, the `.authority` ones included, but walks only what the digest walks. The store and Recent live on the host's heap (`recentGpa`), whose frees the epoch never puts off: a speculated Del grew the closed list, the rollback put the old header back, and the next listing read a freed name. `store` and `recent` are never saved now (rollback.zig `unsaved`); a step never reads a replica's own. - **A unit taped outside a step ended with effects queued** (H6, H10). A leave is the session's `closeView`, whose sync sizes again a terminal the closed view sized and queues its pty's resize; the session did it at its next pump, in another unit. A replica that replayed the close kept the effect queued, and its next idle copy asserted (H10: a client killed when another left). A client joining at that boundary was dumped with it queued, which a loaded core has none of, and diverged at the session's next pump (H6: `the core called poll_frame where the tape has a host (12)`). `closeView` and `openView` settle inside their own input now. - **The lowered copy of a searched text held heap bytes** (H5's and H17's `places.hits|hay`). A smart-case search lowers the text a piece at a time into `Places.hay`, made with `memory.unfilled`; the digest and a join's dump take the whole slice, so the pieces not searched yet were whatever each process's heap held. Zeroed where it is made: 20 µs for 1 MB, 0.4 ms for 10 MB, 10 ms for 100 MB in one frame (ReleaseFast, page faults included; lowering the same text is four times that, over frames), the order of the copy an edit of that text already makes. - **A view drawn outside a step left the live slot aliasing** (H5's hypothesis, in a path it did not name). `withParked` (a replica client drawing or reading its own view while another is live) parked the live view and copied it back without emptying its slot or its panes' shares of selections, which the rollback copy walks. Not the cause of H5's signatures, which were the jumps (Recent) and the places (the lowered copy) above; the slot was saved and put back all along. - **A step that resized a terminal read rows its replicas could not be sent** (H18). A client's first key into a terminal sizes it to its view (`typedInto`, `sizes`); one that also read the shell's history in that step (an edit copy over its text, `ovl`) read rows the replicas held as holes. The session's `sync` reflowed its emulator inside the step, so the rows `Screens.fill` sent before the unit were the reflowed grid's, in the resize's new numbering (`Rows.epoch`); a replica replaying the step on the grid its last delta made drops rows of another numbering and read blanks. Every client diverged at that unit (`pane N.ovl`, and `jumps … pane N.body` where the rest of the step read the reflowed grid; `registers pane 2.body pane 2.ovl` in fixer 1's soak, a `d` in a terminal's text whose step resized it). Seen with an instrumented build on the hunt's seed 5 at 80 ms: the replica's `rows pane 2 epoch 69/68`. - The fix is fixer 1's (`terminal.resizeGrid` notes `resize_to` on an authority with screen-state replicas, made after the unit in `Peers.unit` and at `Peers.stop`), found on both lines at once; fix2's was the same and is dropped. With it, `terminal.create` must make `resize_to` null: it sets every field by hand, and the heap's bytes there reflowed an emulator to 43690x43690 in a Debug run (0xAA). - Found on the way: `shellRows` marked a terminal read only when it built its rows, not when it handed out the kept ones, so a client that came since (its history all holes) was sent none of what a step read. - Not a stale round trip, as first supposed: a delta across a renumbering is a reset already. The hunt's H7 fix (fixer 1's `Rows.update`) is under this one. - The gate on fix2's version of it (main mqkxnsqu, fixer 1's line): Debug `core-test` and `unit-test` tty, `snap`, `replay`, `replicas` (none diverged or unequal at 30, 80 and 207 ms), `test/daemon.py` 20, `--far`, `--manage`, `--upgrade`, `--window-clients`, `--gui` 5, `test/fs.py`; gui `core-test` and `unit-test` on main's two; ReleaseSafe `test/daemon.py` 20, `--far`, `--window-clients`, `--gui` 5, `--upgrade`, `test/fs.py`. Once, `test/daemon.py` 20 failed in its far clients' end: a session asked to Exit twice closed its 9P link before the second write's clunk was answered, an `EOFError` the test takes for no `OSError`; twice again it passed. Tests, each failing before its fix: src/rollback.zig ("a rollback leaves the store and Recent as a speculation left them", "a view drawn outside a step leaves the live view's slot holding nothing"), src/replica.zig ("a replica joined after a pane closed", with the variant hunt's Recent written by another window; "a replica's Del of a file its session's Recent knows"; "a client leaves, and one joins"), src/Places.zig ("a lowered copy's pieces not searched yet are the same in every process"; src/replica.zig's "a step that reads a terminal's rows as the core keeps them"). The hunt's repros (`panes_closed.py`, `del_file.py`, `recent_store.py` with the client's own XDG_STATE_HOME, `leave_resize.py`) agree on the fixed builds; on the installed 0.24 `panes_closed.py` still rejoins 16 times. `zig build replicas` now looks at files, closes panes, opens the Panes and Recent pickers and jumps. The gate, on fix2's five changes (`.scratch/daemon/fix2/gate.sh`): - **Debug:** `core-test` tty; `core-test` gui and `unit-test -Dplatform=gui` fail only on main's two (`tagline pointer mapping...`, `a tag caret past its row's end...`); `unit-test` tty, `snap` (113/113), `replay`, `replicas`, `test/daemon.py` 20, `--far`, `--manage`, `--upgrade`, `--window-clients`, `--gui` 5 and `test/fs.py` pass. - **ReleaseSafe:** `test/daemon.py` 20, `--far`, `--window-clients`, `--gui` 5, `--upgrade` from Debug and `test/fs.py` pass. - **Rebased onto fix3's line** (myulmrpt; its `tape.results` now lists what `tape.authority` returns): `core-test` and `unit-test` tty, `snap`, `replicas`, `test/daemon.py` 20, `--far`, `--window-clients`, `--gui` 5 and `test/fs.py` pass again in Debug; `test/daemon.py` 20, `--window-clients`, `--gui` 5 and `test/fs.py` in ReleaseSafe. - **The copy** (`zig build rollback-bench`, ReleaseFast, main's rollback.zig against fix2's): a typical session's 980.3 KB to 980.1 KB, a big one's 1487.9 to 1487.8 (the Store's and Recent's own bytes; their heap was never saved). Its take 0.045 and 0.063 ms, 0.075 and 0.077; its restore 0.036 and 0.034 ms, 0.053 and 0.054: noise, one run each. The soak (`test/soak.py local --minutes 5 --rtts 0,30,80,207 --clients tty,win --trace`, ReleaseSafe, the mouse on, Upgrades, resizes, detaches and scripts among the moves), twice: | cell | client | min | inputs | links | visible | mispred | p50 / p99 ms | cut short by | |---|---|---|---|---|---|---|---|---| | 0 ms | tty | 1.4 | 330 | 2 | 0 | 0 | 1.5 / 1298 | H7 (session) | | 0 ms | window | 1.4 | 225 | 3 | 6 | 2 | 4.5 / 12.4 | H7 | | 30 ms | tty | 0.6 | 101 | 1 | 0 | 0 | 43.2 / 1283 | H7 | | 30 ms | window | 0.6 | 56 | 1 | 0 | 0 | 4.8 / 25.4 | H7 | | 80 ms | tty | 5.1 | 962 | 1 | 6 | 30 | 14.4 / 1570 | | | 80 ms | window | 5.1 | 911 | 3 | 63 | 35 | 5.9 / 117 | | | 80 ms | window, attached later | 0.8 | 35 | 0 | 0 | 0 | 2.0 / 3.6 | its replay panic (H9) | | 207 ms | tty | 0.8 | 98 | 1 | 0 | 0 | 42.3 / 1879 | H7 | | 207 ms | window | 0.8 | 84 | 1 | 0 | 0 | 6.3 / 18.7 | H7 | - Every divergence of that run was the pointer's: `pointer_tag_hit look_hover_wait` (H3, fixer 1's: a tag hit made from the process's own frame), which takes every client of the session at once, and a window client's replay that met the `pointer` site where the tape had none (H9's panic, the same class). None of this section's classes. - H7 (unassigned: `integer overflow` in ghostty's PageList under `Screens.flush`, after a unit with a `pty_resize`) ended the session in three of four cells; the first run's cells, of the same seeds, did not meet it. - The first run (its table lost to the harness, which raised when a session with unsaved text would not Exit; fix2's copy now ends it by pid and counts a breach): 0 ms, three rejoins on H3 alone; 30 ms, one window client's replay panic at the `pointer` site; 80 ms, three on H3 and three of H18's (`jumps … pane N.body` at units with a `pty_resize`, every client together). - H17's own configuration (30 ms, seed 22, 4 minutes, where both clients had died at `n` on different places): no divergence in `places`. Its rejoins were H3's (five) and H18's (two units, `jumps … pane 0.body` and `pane 0.body pane 0.ovl`, each with a `pty_resize`). Not done, and found on the way: - **Two edges of the same class, left:** a pipe or an Edit in flight when a client joins (`Pipe.wait`, `.authority`, not dumped: the joiner drops its answer), and a 9P batch held at a join (`fs.batch`). Both need a join in the instant between a request and its answer. - **A Debug session with `PARDES_REPLICA_TRACE` is no soak:** its per-field digests rehash every blob at every unit (`walk.rehash`, Debug's check), a whole undo history of a big file a unit, and a boundary took 62 s. Soak ReleaseSafe, or Debug without `--trace`. - **std's segfault handler panics while it symbolizes** (H8's S3: `SelfInfo.Elf.findModule`, index out of bounds), so a crash record says that and not the segfault. It is std's, not pardes's (`crash.zig` takes panics only). ### fix: the session's abort, the window's rejoins, a client's end The user's window client of the box's session (`pardes-gui --attach=root@65.109.81.103:freebsd`, 2026-10-30): the cursor not keeping up, the window blinking, panes closing just after they opened, then a freeze. The first three were the client joining again, 15 times in its 25 s; the freeze was the session's abort at about its 16th far join, and the client's 20 s of redials after it. An adversarial hunt (a soak harness and a variant sweep) found the rest below. Each fix has a test that failed before it. - **The session aborted after 15 far joins (H2).** `c_heap: more distinct allocators than it can name`, from zstd's context on a peer's writer thread. The core's table held 64 copies of one allocator, `ptr` 0 and c_allocator's vtable. `zstd.gpa` is `std.heap.c_allocator`, whose `ptr` is undefined, and nothing outside a test writes it: a release build made it a constant and folded `known.ptr == current.ptr` to false, so every malloc named a new allocator (about four a join) and the 65th panicked. A client's decompressor (one a join) would have aborted after about 30 rejoins; the web shell's heap over `wasm_allocator` trapped the same way. - Fix: c_heap reads `current` through a volatile pointer, as its bytes are, so no heap compares a value the compiler may take for anything. - Tests: c_heap's own, which unit-test now runs built ReleaseSafe too (it panicked at the 65th malloc); `test/daemon.py --far` makes 200 far joins of one session, each read whole, and checks its fds, threads and RSS. The installed 0.24 aborted with c_heap's panic within that run. Now, ReleaseSafe: 200 joins in 1.7 s, fds and threads level (8, 8), RSS +108 KB; on the box (FreeBSD ReleaseSafe, through ssh): 200 in 185 s, fds 13 and threads 11 level, RSS 48964 to 49168 KB. - **A window's pointer over a tag diverged it at once (H3, the user's `tape byte 119`), and a click there aborted it (H9).** A step resolved a frontend's pointer hit again on `p.surface`, this process's last frame, which is `.local`: the session never draws a client's view, so the same motion gave the client a tag hit and the session none (`pointer_tag_hit`, `look_hover_wait`). Body hits did the same through `reprojectBodyHit` and a sweep's clamp (the variant hunt's F). - Fix: a hit carries the place it was made on (`Layer.Place`: the layer's viewport, columns, rows and compact rows) and the step resolves it from that, never from a frame; a sweep keeps its body's place to clamp onto. The wire carries the place and a tag hit's line: version 17. - Tests: `--window-clients` sweeps the pointer over the tags and the body and clicks there, at 0 and 207 ms (before: joined again, `tape byte 119`); mouse.zig's two tests from the variant hunt. The hunter's repros: 15 rejoins in 3 s of hovering a tag before, none after; clicks, drags and B3 in a window aborted it before, none after. - **A replay that met a site its tape lacked panicked (H9, H12).** Fix: a site or value the replay does not find is a divergence and returns what a speculation gets (`poisonOf`): any later leak costs a rejoin, never the process. Test: tape.zig's. - **A view closing left a terminal's resize queued (H10, H6).** `closeView` syncs, which sizes a terminal the closed view sized and queues its `resize_pty`, with no step after it to drain it: a replica's next copy asserted, and a speculation over the queue lost the effect when it rolled back, so the next pump's replay found no `pty_resize` where the tape had one. With the user's config (a font of their own sizes their view) every rejoin diverged at its first keys. - Fix: `closeView` settles in its own unit (fix2's, above, which took the place of a drain here), and so does `leaveView` below; a copy is refused (`error.Queued`), not asserted, over a queue, and the input waits for the authority. - Tests: replica.zig's in-process one (two clients at two sizes, one leaves; the other copies, types and agrees). The user's config with a random mix of keys and pointer: 18 rejoins in a minute before, none in four minutes after. - **A reflow overflowed the session's screen state (H7).** A resize to fewer columns puts more rows above the last active top than ever scrolled past it, so `top - history` underflowed where a peer starts afresh; a window client's font made such resizes. Fix: `Rows.update` keeps `top` at or past the history's length (a resize renumbers anyway). Test: screen_state.zig's, 200 long lines narrowed. - **A rejoin dropped the client's layout.** Its new view copied the live one, the session's own as it was before the client came. Fix: a session's last client, while nobody else watches the session, leaves it its layout (`leaveView`: columns, focus, scrolls, cursors and size), which a rejoin (an Upgrade's, a divergence's) copies back. Test: views.zig's. - **A client's end.** One refused at a redial exited from inside itself and left its ssh and `/tmp/pardes-ssh-.sock`; one rejoining after a divergence redialled a gone session for 20 s; a session that panicked kept its clients' links open for the seconds its trace took. - Fixes: the ssh is ended at `atexit` and dies with its client (PDEATHSIG); a redial waits only after an Upgrade's `refused` and else says "could not join again"; the panic and segfault handlers shut every client's link first. - Tests: `--far` attaches through an ssh stand-in, refused and detached, and finds both gone; `test/daemon.py` kills a session under a replica client, which ends within 2 s, in words; replica.zig's redial test. - **A far join's leave held the session's step** while the writer hashed the rest of its blobs (the hunter's review, unconfirmed). Fix: the writer stops at the next blob once it is let go. ponytail: one blob's hash (about 30 ms for 100 MB) is the ceiling. - **Kept as diagnostics:** the trace's unit descriptions give a key's and a pointer's words, and a confirmed speculation's state is checked against its unit's digest, which names the input that diverged rather than a later unit replayed. - **The user's symptoms, rechecked** on the box (ReleaseSafe at both ends, a window here with the user's own config through ssh, a monkey of keys, pointer, wheel, clicks and shell commands, 2 minutes), on this line over fix2's and fix3's: - Rejoins: none in 760 actions, 20 checks all agreed. (Before fix2's line, one in 799, at a `.kind` site its tape lacked, which had aborted the client.) Panes no longer close: a rejoin keeps the layout anyway. - The freeze: the session no longer aborts, and a client of one that does ends at once, saying so. - The blinking and the cursor: 4 visible rollbacks in the 2 minutes (4 inputs of 2375 mispredicted; 45 and 32 of 2369 before fix2's line). Each is a clock guess: an edit's file time a second off across a second's turn, or the millisecond clock the steps of Ctrl-W, Ctrl-O and Ctrl-U read. Not fixed: a guess of the session's clock is the design's ceiling there; reading a coarser clock in those steps is the next step. - **Two leaks the soak showed after that (its local80 and local207 cells):** - A client typing into a terminal at another size than its last typist sized it in that key's update (sync), and the session's emulator reflowed at once while its replicas' wait for the delta. The rows the step read that a replica held as holes then went to it in the new grid's numbering, which it cannot take, so its replay read them blank (`jumps`, `pane 1.body` at a `b`). Fix: an authority with screen-state replicas makes the size after the step's unit (`resize_to`, in `Peers.unit`); the step reads the grid it had on both sides, and the next flush brings the new one; `create` sets `resize_to` null, as it sets every field by hand (fixer 2 found a Debug session's terminals asked to be 43690 by 43690, the heap's 0xAA). Tests: replica.zig's, which reads history in the resizing step in the session's order of flush, fill and unit; terminal.zig's, a terminal made from 0xAA memory. - A client that joined had its panes' places loaded without their regex (`re` is derived: in no dump and no copy), took a search for a list's and found nothing an edit made, so it diverged at the next edit and at every rejoin after (`pane 3.places.hits`; `pane 17.cwd` too, which inherits pane 2's and hashes it whole). Fix: the regex is made again from the pattern when a set holds none. Test: Places.zig's, a dump loaded and the same edit made in both. - Not confirmed: "panes [5, 8] left its view while still open" in local0 came at 104 s, before any client of that cell had left (the rejoins' joins were at units 2028 and 2029, near its end), so neither `leaveView` nor a closed view's settle moved them. The soak compares pane ids (/index, /layout), and a slot is taken again by the next pane made: two panes the terminal's view showed, closed, and the window's Looks just before (Enter on a row) opening panes into those slots, in the window's view only as a view's new pane goes, reads the same. A serial a pane in /layout would tell the two apart. - **Gates, on this line over main npzpposw (fix2's and fix3's), again; then over wxpkrpwo, the builds, `core-test` tty, `replicas`, `test/daemon.py` 20 and `--window-clients` in Debug and ReleaseSafe and `test/fs.py` ReleaseSafe, all passing:** Debug `core-test` tty and gui, `unit-test` tty and gui (only the two GUI tests main already failed: the tagline pointer mapping and the tag caret), `snap` 113/113, `replay`, `replicas`, `test/daemon.py` 20 cycles, `--far`, `--window-clients`, `--manage`, `--gui`, `--upgrade` and `test/fs.py`; then `test/daemon.py` 20, `--far`, `--window-clients`, `--manage`, `--gui` 5, `--upgrade` from Debug and `test/fs.py` against ReleaseSafe builds. All pass. ## Phase 6: instant interaction notes Built 2026-10-30 (`src/replica.zig`, `src/tape.zig`, `src/terminal.zig`, `src/echo.zig`, `src/detached/wire.zig`'s `stamp`, one line in `src/pardes.zig`; PARDES_TEST_LATENCY's `I` and `U` lines in `src/gui/gui.zig`). The user: "make sure to use the mosh tricks so that user interaction to type/scroll/move the cursor feels instant specially when they don't require a request going to the daemon", and "measure it too since I don't want to catch those myself". The measure is test/soak.py's `instant` mode (the hunter's, per action class against a local session's own frontend) and a bench of my own that says what stopped each input (`.scratch/daemon/instant/harness/classes.py`). Decisions at the forks, with the ponytail lens: - **What stopped an input is counted by name.** A speculation that meets a host call, a site or a poison records which (`Tape.poisoned_by`), and the client's PARDES_REPLICA_STATS line ends `stops name:n,...` (PARDES_REPLICA_TRACE says it with the input). Every number below that says why an input waited comes from it. - **pardes's own steps over a terminal are speculated (H14).** Until now every input of a client whose keyboard was in a terminal waited a round trip (`a terminal has the keyboard`): normal-mode keys over its history, the wheel, Ctrl-u. The blanket poison goes. What reaches the program (a key in tty mode, a paste, a mouse report) is a pty write, a host call, and stops as before; the predicted echo covers it. The rest reads the emulator and may move its view: - **Its view moves here.** A screen-state replica's speculation may scroll a terminal (`terminal.viewOnly`: `scrollGrid`, `followOutput`); nothing else of an emulator. The rollback copy holds no emulator, so the replica keeps each terminal's view as the copy was taken (`Replica.views`), a delta sets it the authority's, and a rollback puts it back, between the units it replays in the order the deltas came among them (a delta's view comes after the units it shows: a neutral unit of another's that scrolled, replayed over it, scrolled twice; the harness's per-delta check found it). - **Applied again over each delta.** What it did depends on the emulator, here the last delta's and there what the pty said since. The stream sends a client its terminals before the authority steps any input and the rows a step read before its unit, so a speculation over a terminal (`Pending.terminal`: it read a terminal's rows, moved a view, or a terminal had the keyboard) is put back and applied again over each delta or rows that come while it waits (`overTerminals`): when its unit comes it read the emulator the authority's step read. - **No digest in a speculation.** A first version kept a terminal speculation's digest to check its unit's against; but a blob born under the copy and digested keeps its word past the epoch's free of it (src/walk.zig), and a later blob at that address read as it (the harness's Debug blob check, seed 11 at 207 ms). Gone: it is confirmed by its records, as any speculation is. - **Never kept to replay later.** A unit confirmed on the fast path with more applied on top is kept in `since`, to replay at the next rollback, over the deltas that came after it. One over a terminal is taken at once instead, over the emulator its step read, and those on top applied again. - Closing a terminal stops (`retirePane`: its pty's end is the authority's); the blanket poison had covered it, and the in-process harness's host has no `close_pty` to stop at. - Found on the way, others' to fix and fixed in main since: with more of a terminal's steps speculated, the replicas harness met H18 more often (a client typing at another size than the last typist: the session's emulator reflowed inside the step, the rows sent for the replicas' holes came renumbered, their replay read blanks; seeds 10, 11 and 17 at 207 ms, `pane 0.ovl`, `pane 0.sel_snap`). Main met it too (seed 53 of 60 at 207 ms). Fixer 1's `resize_to` and fixer 2's `read` through the rows cache (wwkwwlzt, tysmmuzx) fix it: on them, 20 seeds at each RTT, none diverged. - **A pane closes in a speculation.** `deinitEmulator` asked the authority before it looked for an emulator, so the reap after any close (a file's pane, a picker's, a preview's) stopped the input after it: 4 of the finder's 6 stops. - **The authority's yes or no is guessed (mosh's way).** Fixer 2's `tape.authority` makes a step that reads what only the authority holds wait for it: whether a script reads a pane's events, which every Look and click's command asks, put every Look a round trip late. A bool answer is now guessed in a speculation from the authority's last answer to the same question (`Tape.answers`, filled as units replay), recorded as the answer, so its unit confirms the guess or rolls it back. Lists (the closed panes the Panes picker shows, Recent) change with what the user does and are still asked: opening the Panes picker waits a round trip (its typing does not). - **An input's time is its client's.** A speculation guessed the session's clocks (`Tape.guess`: the last unit's reading, the time since, a round trip), and a step that read one recorded the guess; the authority read its own, so a message's millisecond (Ctrl-W's, a jump's) always and an edit's second across a turn sometimes recorded otherwise: a rollback each, which a window counts as seen (fixer 1: 1 to 8 a minute, window clients worst). Now the input carries the guess (wire 18's `stamp` frame before its event; `Pending.guess`, the same at every reapply), and the authority's step reads it as its clocks (`Tape.stamp`, `stampOf`; within a second of its own, else its own), so the two record the same time. A message's linger is then off by the guess's error, tens of milliseconds. - **The echo's lost predictions wait for their keys.** Found by measuring at 207 ms: the shell's typing was predicted (24 of 26) and drawn almost nowhere, key to output p50 65 ms. A drop with keys still in flight (a delta that moved the screen's top, here the pty's resize at the first key; or a misprediction) restarted the predictions at the screen's cursor, where those keys' echo was about to land: every key after it was judged wrong (5 of 6), and once wrong, nothing was drawn until an echo was confirmed, which none could be. Now the keys typed after a loss wait for the newest key in flight to show (`lose`, the `deferred` path keys behind Tab take), then are predicted over it. - **mosh's rules, checked** (its source is in ~/05-genizah/mosh now, src/frontend/terminaloverlay.cc `new_user_byte`): a printable character (inserted, the rest of the row shifted right), Backspace (the rest shifted left), Left and Right; CR moves the predicted cursor to the next line, tentative; everything else (Home, End, Delete, Up, Down, any other CSI or control) makes it tentative, nothing predicted. pardes predicts the same less the mid-line insert and delete (a line editor redraws what is right of the cursor; fish's suggestions mispredict a shift) and CR. Nothing mosh does is missing that would raise the measured share: at a prompt every letter is predicted. Measured. Before is main at uxwrlzlw with the counting (`base2`), after is this line on mqkxnsqu (`new2`; fixer 1's line between them changes none of what is measured here, and the line has since moved onto vplsqtyz); both ReleaseFast, unstamped, on the perf cores, the box's session the same build cross-compiled. The hunter's `instant` mode (test/soak.py, its xvmytsym copy pointed at my box directories): a class's 25 inputs a frame or more apart in one client, against a local session's own frontend measured in the same run. `at once` is the share that neither stopped nor showed later than half a round trip (a window's two frames). - **A terminal's history, by keys (Ctrl-b, k, Ctrl-u) and by the wheel** (tty; at once, then key to output p99 ms; local p99 0.6 to 1.2 ms): | | 0 ms | 30 ms | 80 ms | 207 ms | the box | |---|---|---|---|---|---| | term_scroll_keys | 0% 15.2 → 100% 1.1 | 0% 54.3 → 100% 1.5 | 0% 101.2 → 100% 0.9 | 0% 232.2 → 100% 0.9 | 4% 428.6 → 100% 1.4 | | term_scroll_wheel | 0% 1.5 → 100% 0.6 | 0% 48.6 → 100% 0.6 | 0% 99.1 → 100% 0.6 | 0% 226.3 → 100% 0.6 | 0% 207.7 → 100% 0.6 | The window client's, the same at once: 0% (4% on the box) → 100% at every cell. (A window's p99 in this mode is the first frame after the read, which a window draws whether or not the input waited: its at once is the measure.) - **Shell typing at a prompt** (tty, p99 ms; every letter predicted before and after, 25 of 25): 0.8 → 1.4 at 0 ms, 1.5 → 1.1 at 30, 0.8 → 0.9 at 80, 49.0 → 27.7 at 207, 210.6 → 18.0 on the box. My own bench's shell class at 207 ms (lines of `echo` and three words, Backspace, Enter): p50 65 ms (main, ReleaseFast) → 1.8 ms (this line, Debug), the echo judged wrong 5 of 6 → 0. - **Every file class stays at once** (type, move by keys and by click, scroll by keys and the wheel, select, the picker's typing): 100% before and after at every cell and on the box, tty p99 0.5 to 1.5 ms after (a local session's 0.6 to 0.9; its wheel 48 ms, the glide). - The gate: `instant: pass` at 0, 30, 80, 207 ms and through the box (before: 4 to 6 breaches a cell, each a terminal class's, 28 in all). - What stopped inputs, by name (my bench, 207 ms, before; `stops` in PARDES_REPLICA_STATS): the shell's keys `pty_write` (their echo is predicted), a terminal's history `scroll a terminal` 23 and `a terminal has the keyboard` 3 of 27, the finder `close a terminal` 4 and `preview` 2, a Look the authority's `readsEvents` 11 of 12 and `resolve` (the first open of a file) 1, a Save `kind` (and then its write), the Panes picker's opening the authority's closed panes; typing in a file, moving, scrolling a file, selecting: none. After: the shell's `pty_write`, the finder's `preview`, a Look's first `resolve`, a Save's `kind`, the picker's opening. - **The soak's blink** (test/soak.py `local --minutes 2.5`, its gate, a tty and a window client a cell at 0, 30, 80 and 207 ms, far): before (`base2`) visible rollbacks 0.0 to 1.0 a minute, the window at 80 ms 4.2 (13 in 3.1 min, a breach), mispredicted inputs 1 to 13 a client, and that line's crashes and rejoins (fixer 1's since). Fixer 1's own build, with them fixed: windows 2.2 to 2.7 a minute, ttys 0 to 1.3, clock guesses all. The final line (this on xymsrmso): 0 visible, 0 mispredicted, 0 rejoins at every cell, tty and window; `soak: pass`. - **The replicas harness** (ReleaseFast, 20 seeds x 120 inputs x 3 clients): inputs that stopped 731/649/790 at 30/80/207 ms → 453/390/398; confirmed on the fast path 6249/6246/5953 → 6695/6755/6753; mispredicted 220/305/458 → 15/15/21; none diverged, none unequal (before, main at uxwrlzlw: seed 9 at 207 ms unequal at quiescence, as main at vplsqtyz still was). - **The final line** (this on xymsrmso, fixer 1's line): the gate at 207 ms passes again, every class at once in both clients (tty p99 1.7 to 4.4 ms, a local session's 0.5 to 1.4 ms; shell typing 25 of 25 predicted, p99 28 ms); and through the box, every class at once in both clients (tty p99 0.5 to 2.4 ms, a local session's 1.6 to 3.6; shell typing p99 1.7 ms, 210.6 before). The gates (on the final line): Debug `core-test` tty, `unit-test` tty, `snap` 113/113, `replay` (20 seeds), `replicas` (4 seeds) and `--echo`, `test/daemon.py` 20 cycles, `--far`, `--window-clients`, `--manage`, `--gui`, `--upgrade`, `test/fs.py`: all pass; `core-test` gui and `unit-test` gui fail only main's two (the tag caret, the tagline pointer mapping). ReleaseSafe (`zig build -Doptimize=ReleaseSafe`): `test/daemon.py` 20 cycles, `--far`, `--window-clients`, `--manage` and `test/fs.py` pass. The new unit tests: a terminal's history by keys at once with a delta while they wait (replica.zig); keys still on their way when their predictions go (echo.zig); an edit's second across a turn confirmed on the fast path (replica.zig, which fails without the stamp). Not done, and why: - Opening the Panes picker and Recent wait a round trip (their lists are the authority's, and change with what the user does); the finder's preview reads the file it previews (a host call); a Save waits for its `kind` site and then its write; the first Look of a pane waits once (no answer to guess from yet). Each is one round trip, never wrong. - The first key typed into a shell scrolled back waits a round trip: typing snaps the view to the bottom, but the key is a pty write and its speculation, the snap included, is undone; the echo predicts only at the live bottom. - Under the in-flight bound (32 deltas) a step can read a terminal state the client was not sent: replayed, its unit diverges (a rejoin), as another client's always could; a speculation of one is confirmed by its records, so it is caught at the next digest a replay checks (fixer 1's check when nothing is left on top), not at once. - A window counts every misprediction followed by a present as seen, whatever it drew (a tty only one that wrote to its terminal): an upper bound, now 0 in the soak. - A terminal's history is not fetched ahead of its view: a scroll back (at once now) into rows a flood skipped shows them blank for a round trip. Asking for two screens around the view diverged a replica in the harness (seed 10 at 30 ms) before H18's fix, which may have been H18 met in the other order of events it made; dropped, not tried since. - The window's present is its frame (a software present in the test, up to a frame): a prediction is drawn in the same pass as its key, so it cannot come sooner than the window's own frame. After the line, the hunt's H21 (a client's normal end aborted): - **A closed terminal brought back put its emulator on the rollback epoch.** Both clients still alive at a box soak's end died at their hangup with `double free or corruption (!prev)` in `Epoch.deinit`; a local soak at 207 ms did it in 2.5 min, `free(): invalid pointer` in a terminal's screen replica's teardown at a rejoin. A terminal closed and brought back (Panes, `Look @p`) is loaded by `dump.loadPane`, which named no `.terminal` allocator, so on a screen-state replica its emulator was made with the core's, the epoch's, against `State.gpa`'s rule. Its screen replica's blobs (`apply`, with `State.gpa`) were the epoch's then, freed with the C heap's at the end behind its back, and freed again by the epoch; a delta under a held copy made them born in it, and a rollback freed them, and the emulator's pages, from under it. - Fix: `terminal.emulatorGpa`, `create`'s rule, is `loadPane`'s `.terminal` too. - Test: replica.zig's closed terminal brought back on a screen-state replica, its State not the epoch's (failed before). The local soak at 207 ms, ReleaseSafe (before, on lppumkvm, both its clients aborted): 2.5 min, both end 0; 4 min at seed 32, four clients, none aborts (its one breach, a window's panes gone from its view, is not this one). **A rejoin's view is its own again** (2026-10-30; `src/views.zig`'s `away`, `Pardes.awayView`/`rejoinView`, wire 19). Every rejoin gave the client a new view, a copy of the session's: a divergence's, and above all an Upgrade's, which every client makes after each install (the hunt's 80 ms cell: the terminal client lost `+Search` and `+New` from its view; likely the user's "some panes just close randomly right after I open them"). `leaveView` covered only a session's last client, while no one else watched it. - **A view kept for its client.** An Upgrade's clients (`Peers.refuseAll`) and one whose bye says it joins again (a divergence's: `bye.again`) leave their view open as it is, kept for them (`views_away`, a session field: digested, and dumped, so the new build has it). The client names the view it had in its next hello (`Hello.view`; the daemon hands it on with the connection, `replica `), and its join takes that view back (`rejoinView`, settled as `openView` is) rather than a new one. Both are taped inputs. Any other end closes the view as before. - **The live view is never one.** One kept going live sends the session into another, or into a copy of its own. `views.count` counts the views someone looks at, so the last of those stays, a tty's Detach is "one of several" only beside another, and `closeKeeping` never writes into a kept one. Drawing skips them, and an Upgrade's new build opens no window for one and closes none for want of a terminal (`resumeTerminals`, `resumeWindows`). - **Bounded by room.** A kept view goes when a join finds every view taken (`openView`, the lowest first): a client that never comes back (one of the old build, told to upgrade too) leaves one. ponytail: no timer; until then it costs a parked slot, and a terminal it sized last keeps that size until another view types into it. ponytail: a divergence's redial that reached the session before its old link's end takes a new view. - **Tests.** views.zig's: two clients' views through an Upgrade's dump, each back in its own, and a join with no room lets one go. replica.zig's: a divergence's rejoin, its pane in its view again, and the other client agrees. `test/daemon.py --upgrade`: two replica clients each open a pane in their own view, and after the Upgrade's rejoin each one's `/layout` holds its own pane and not the other's; and the plain run's client D, put off now at its 100th unit, opens a pane in its own view first, and after its divergence's rejoin its view holds it and its keys go into it. main's binaries fail both (`('its own view again', 4, 5, ...)`, `('client D in its own view again', 2, ...)`). - **Not this one:** the H21 soak's "panes [...] left its view while still open, with no rejoin since the last quiet point" (local 207 ms, seed 32, a window client with one link all through) is no rejoin's. - **Gates** (on zponxort): Debug `core-test` tty, `replicas`, `replay`, `test/daemon.py` 20 cycles, `--upgrade` (against a `-Dstamp-commit` build), `--window-clients` and `--gui`; ReleaseSafe `--upgrade` (from the Debug build) and `--window-clients`: all pass. After the line, the box soak's divergences and its clean quit: - **The session that quit at its 1093rd input was asked to.** Seed 32's window typed `Ctrl-W h`, then `imike quebec `. Window mode stays open until Enter, Esc, `:` or a label, so `i`, `m` and `i` went nowhere. `k` at a column's top went to its tag, and there `e SPC q` ran Exit, with nothing unsaved. The soak now leaves window mode with Enter. - **A FreeBSD session's +Help had one row fewer than its Linux clients'.** The box soak's rejoins after a divergence were `pane 4.file.content`, and `.splice` over a +Help made again. Each was at a `?` after SPC, which runs Help. Help's rows are the build's words, and Tty9p was a Linux build's alone (`enabled` asked the OS), so a Linux replica made a `SPC n 9 Tty9p` row its session never had. H19 made the enum's digest the same in every build; the set of words was still not. - Fix: Tty9p is a word of every hosted build. Whether the host is Linux is the authority's answer (`tape.authority`), so a replica of a FreeBSD session refuses Tty9p as its session does. - Test: test/cross_replay.py's session runs Help. Main's FreeBSD tape, replayed by a Linux ReleaseSafe build, diverged at it (`pane 5.file.content`); with the fix, 30 digests agree in Debug and in ReleaseSafe. - **A Debug replica of a release session diverged at every Grep's query pane** (`pane 2.query`: cross_replay.py's ReleaseSafe tape replayed by Debug). `Names.query_buf` was digested whole, and past `query_len` it is undefined: 0xAA in Debug, something else in a release build. Fix: a replica `len` on it. The same tape now agrees in Debug (30 digests). - **The hunt's `pane 2.file.scroll` (seed 41) went with the Help fix.** On main (zponxort) it came again at the same unit: tty207-1's `SPC ?` (1/596, 1/597) made a +Help, a window client joined, and tty207-1's `%` (1/713) diverged that window at `pane 2.file.scroll`. With the fix, the same keys and units pass with no rejoin, on lppumkvm and on zznqlptu. Not traced: which field the joined window got wrong, and why the window that replayed the Help unit itself did not diverge there. - **Box** (traced, ReleaseSafe at both ends): seed 32 (win,tty), 3 min. On main: four rejoins in three clients. The traces put three of them at the two Helps; the fourth, in the window that typed the first Help, gave no reason. With the fix: no rejoin. Seed 41 (tty,win), 3 min, the same before and after (one rejoin on main, none with the fix). - **Gates** (on zznqlptu): Debug `core-test` tty, `replicas` (none diverged or unequal), `test/daemon.py` 20 cycles and `--window-clients`, cross_replay.py with tty and window recorders, and the FreeBSD tape replayed by Debug tty, ReleaseSafe tty and Debug gui; ReleaseSafe `test/daemon.py` 20 cycles: all pass. After the line, a window that joined late: - **It diverged at the next key that moved a pane of another view** (`pane 23.file.scroll`, the box soak's seed 7: win207-3 at its join 466 s in; the same field in the two windows that joined at 16 and 17 min of an earlier run, not kept). A pane's `view` (`Pane.SessionView`: Wrap, the context rows' style, the body's and the tag's row heights, its rect's height) is written by `sync`, and was derived: never dumped. A key's move reads it before its step syncs (`halfPageMove`, `ensureCursorVisible`, through `File.geometry`), and a core loaded from a dump had the default, Wrap on and no row heights. The tty had typed `SPC t w` at 243 s, so the session's lines no longer wrapped and the joiner's did; its first stepped unit after the dump was the tty's Ctrl-D (1/2018, after `open_view` and a pump), and it scrolled further. A window there all along had made `view` in its own steps and agreed. A parked view's share (`PaneView.view`) was read the same way at a `to_view`, before the step syncs. Window row heights with TreeContext's context rows diverged it too, with Wrap on. - Fix: `view` is digested and dumped, in a pane and in a parked view's share. A dump of an older build has none and takes the default, as before. - Test: replica.zig's "a client joined from a dump moves through another view's pane as the session does, its lines not wrapped" (Wrap off, the cursor near the view's bottom, a join, Ctrl-D). Main fails it at `pane 0.file.scroll`. A throwaway exploration (120 seeds: two views, window row heights, TreeContext, Wrap, keys from either view after a join) diverged 20 times on main (`file.scroll`, `hscroll`), none with the fix. - ponytail: `notices` and `pan` stay derived. `unpark` makes `pan` again; a pane's notices are read by a click on its bands. - **Not this one:** the box run with the fix (seed 7, 21 min, seven clients) showed no `scroll` divergence in the four windows that joined late and no `view` difference anywhere, but win207-4 diverged once at its own key in a query pane (`pane 21.file.location_rows`, unit 4/447: the session's tape had the same host answers), and after its rejoin a speculated Enter panicked in `Query.land` (`f.location_rows[c + 1]`, len 0). Both left open. - **Gates** (on vtoozmwn): Debug `core-test` tty, `replay`, `replicas` (none diverged or unequal), `test/daemon.py` 20 cycles and `--window-clients`; ReleaseSafe `test/daemon.py` 20 cycles and `--window-clients`: all pass. The box soak, ReleaseSafe at both ends: 0 rejoins in six of seven clients, the seventh's the query pane's above. After it, the query pane's two: - **A speculated Enter panicked in `Query.land`** (`f.location_rows[c + 1]`, `index 1, len 0`: win207-4 after its rejoin). An edit of a frozen query pane's rows drops them with the text they were of (`setContentSpan`'s dropped marks), while its items and its current row stay, and a later search in the pane keeps them so in its `saved` for an Esc. `land` indexed the rows by the current item all the same; every other reader of them asks first. In the soak it was line 1 of a live pane (another view's SPC b b and Esc, and the window's own keys, in another order than the session took them), which the session never ran: the input panicked before it was sent. A session's own keys reach it too: SPC b b, Enter, then `jxd` on the pane's rows and Enter. - Fix: a current item without its row is none: the keyboard to the owner, as with no current row. - Test: Query.zig's "Enter on a frozen query pane's rows after its text was edited: no row to land on, the keyboard to its owner". Main panics in it. - **A window joined from a dump made a `/` query's rows plain where its session had coloured them** (`pane 21.file.location_rows`, win207-4's `/r`, unit 4/447: the session's tape had the same host answers). A hit row's colours are the owner's highlights where a frame painted them, else the locations cache's: both derived, made by what the process has drawn and painted, and a joiner has neither. - Fix: a row's colours are in no digest (Row's own `digest`); a dump still keeps them. They are how a row looks, not where it points. - Test: replica.zig's "a client joined from a dump makes a `/` query's rows as the session does, though only the session has drawn its owner". Main fails it at `pane 1.file.location_rows`. - **Gates** (on nopmlypk): Debug `core-test` tty, `replay`, `replicas` (none diverged or unequal), `test/daemon.py` 20 cycles and `--window-clients`; ReleaseSafe `test/daemon.py` 20 cycles and `--window-clients`; the box soak, seed 7, ReleaseSafe at both ends, 21 min and six clients (late windows and a late terminal among them): no rejoin, no differing check, no panic. All pass. ## Architecture step 6 notes: the rollback and the digest through the journal The clean-room plan's step 6 (`.scratch/clean-room/architecture.md` 9.4) was the rollback copy and the join through the text journal, deleting what the journal made redundant. What landed, and what did not and why: - **A text is written in place in a replica's core too.** The journal made a session's edit write its text and line index in place; a replica's still made a whole new text and index at every key, so that its rollback copy could pin the old ones. Now the copy keeps none of a text's bytes (File.State's `content` and `line_starts` are `.journal`): the epoch notes what each write into a text or an index that was there when the copy was taken took out (`rollback.Epoch.wrote`), and putting the copy back writes that back, the newest first (`unwrite`), before the saved bytes go back. A write into a buffer the epoch made is not noted: the copy put back frees that buffer, and the one it grew from was never written after the move. The three things the architect's watermark had to handle are the copy's already: the journal's arrays (its steps, its `done`, its entries) are saved allocations, and an entry merged into, or the redo entries a speculation dropped, are put-off frees. - **A digest takes a blob of 4 KiB and up by its length.** The blob words (a big text hashed once, its word kept by its address) went at every in-place edit, so every unit's digest after an edit hashed the whole text again. A text's revision says what it holds; a smaller blob is hashed whole. The join's hashing thread, which made the words of texts a session had not digested yet, went with them. The wire's version is 20. - **What stays, and why.** `Epoch` and `Copy`'s walk: the core's state is not the architecture's `Small` (pointer-free, one memcpy) but a graph of allocations (panes, views, rows, registers, the journal's arrays); a memcpy of the core struct brings none of what its pointers reach back, and the epoch is what keeps every allocation of before the copy where it was. They go when `Small` exists. memory.zig's pins: a join's writer thread still sends a session's texts after the step that dumped it, and a pinned text is copied on write meanwhile. The join's stream: "the texts as they are now and the journal entries since R" would have the writer read texts the loop writes in place, and the journal cannot say what changed since a revision (typing merges into one entry over many revisions, an entry carries none, an undo turns entries in place, the ring drops old steps). So the byte walker stays too (the join's stream, the closed panes, the `.dump` files of the build before): an image (the dump's form) would be BLAKE3 of every text inside the joining step. - **Costs** (`zig build replicas -Doptimize=ReleaseFast`, 4 seeds x 120 inputs from each of 3 clients, at 30/80/207 ms; two runs of the build before): a copy p99 83-140 us where it was 85-123; a rollback p99 1.17, 1.76 and 2.23 ms where it was 1.12-1.20, 1.68-1.82 and 2.27-2.44 (the 207 ms cell was past 2 ms before too). A join, in a Debug session with the notes and a 1.1 MB file the session edited, ten clients one after the other: 0.86-1.04 ms of the step (the first 2.9) where it was 5.0-6.0, and a client's input step p50 0.32 ms where it was 2.46: a unit's digest no longer hashes the edited file. ## Architecture step 5 notes: one client path Done 2026-11-06 (the `client` line, §9.4 step 5 of the clean-room architecture). A session shows one frontend of its own; every further view is a replica client. - **The views indexed.** A view's share of the session is `Pardes.View` (the ~42 fields `views.fields` used to name), one of `Pardes.views[4]` (null while closed), the live one `views[live]` through `Pardes.view()`. Going into another view is the index; nothing of the session's is copied. The references were rewritten from the compiler's errors (`p.active` is `p.view().active`, with `const v = p.view()` hoisted where a function uses it three times or more). The views are in the digest (a closed one is null, so a dump and a digest pay for the open ones only); a pane closing leaves every view's layout at once (`views.gone`), where each parked view used to keep serials and prune on its way back. An image from before (the live view as the core's own fields, the others under `views`) loads into `views[]` once. - **What stayed swapped.** Each view's share of a pane (`PaneView`: focus, weights, fold, the body's cursor, ranges, mode, a document's scroll and size, the step's tag rows and bands) is still the pane's own fields for the live view and `Pane.pv[k]` for the others, parked and unparked as before, and still out of the digest. Indexing it as the session's share was measured and not done: `pane.body` alone is 1,340 references, the body is a `Text` whose methods serve tags too, a document's scroll lives in `File.State` (102 references behind a `*State`), and a document's `cols`/`rows` share their 600 references with a terminal's pty size, which every view shows; none of the pane code has the live index to hand. - **One frontend a session.** The daemon answers `other ` for an attach to a session that is attached already (as it did for one of the other kind), so the second `pardes --attach` or `pardes-gui --attach` on this machine is a replica client through the replica socket, as a remote one is. The tty's per-terminal `Screen`s, the `Poster` and its `from` input, the Gui's window-fields swap (`window_fields`, `GuiWindow`, `follow`, `route`, `openAnotherWindow`, `closeOtherWindows`), the pump's `drawOthers` and the harnesses' extra terminals (`start ... views=N`, `view `, PARDES_TEST_VIEWS, the GUI feed's `view;`, the monkey's `--views`) are gone. The pump draws the one view this process shows (`views.shown`: open, no client's, kept for none), through `drawOne` when a client's input was the last. - **`--ps`.** A session tells the daemon its clients' pids (SO_PEERCRED on each replica socket; 0 where it is not Linux) whenever one joins or leaves; `--ps` lists `client ` under its session. - **An Upgrade from an older build** with a second terminal of its own lets that terminal go, its stub told it is detached; the view closes.