# The daemon: one pardes per machine, many clients per session > I want to have a global pardes daemon ... holding file descriptors for > ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a > cpu+file server ... the main test: multiple clients to a single session, > and a client here and the session on freebsd and very deep and thorough > performance testing and optimizations to make it feel like its local and > snappy. Designed 2026-10-29 in discussion (the user, the coordinator, a survey agent). Nothing here is built yet. Paths are relative to the repo. ## Decided by the user - **A deterministic core, replicated.** The session is the authority: it orders every client's input and every host result, and only it performs effects. Each client runs a full core replica, applies its own input at once, and rolls back and replays when the order differs. Visible rollback is acceptable. - **Kakoune-style clients.** Each client has its own view (layout, focus, selections, scroll) over shared buffers and terminals. Registers are session-wide. A terminal shown at two sizes is sized by its last typist. - **Performance order.** First, the regular tty and GUI get no slower. Then the detached and remote clients reach that local performance, with the session on the FreeBSD box included. - **A host step may show up one RTT late, but is never wrong.** This covers Look, open and exec, as sam's hostlock does. - **Join is complete.** The joining client pays for it in time; the authority and the other clients do not. - **One long-lived daemon per user per machine.** It supervises one child process per session, postgres-style. Nothing is left dangling, and the core is not de-globalised. ## What today's seam is, and why it goes `src/detached/server.zig` is a third Host: it runs the core and the ptys in a single poll loop and sends cell frames to the frontends, at the smallest common grid across them. - **The frames are mostly not diffed.** Every layer's cells are re-sent on each frame (wire.zig:734, :755). - **Several things never cross the wire.** Images, PDF pixels, rail ticks and pane animations stay behind (surface.zig:483-509). - **Every keystroke is a round trip,** which no amount of optimisation can make feel local at 30-80 ms. - **Each remote 9P operation dials afresh,** costing about 6 RTTs (9p_io.zig:2361-2385). ## The design - **Processes.** - The daemon is subreaper, spawner, fd holder, registry and 9P root. - Every session is a child of the daemon: a GUI window, a tty session behind a stub, or a headless session. - A session is always an in-process authority with its own local view, so the regular path does not change. Replicas exist only for extra clients or remote ones. - **The tty stub.** - `pardes` in a terminal passes its terminal fd to the daemon (SCM_RIGHTS), along with cwd, env and argv. - The session child drives that fd directly. The stub forwards SIGWINCH and exits with the session's status. - Detach: the session drops the fd and the stub exits. No fork happens. - **GUI detach.** The window is destroyed and the process carries on headless. Closing a window that has no other client attached quits it. - **Shells survive a session crash.** After each `forkShell`, the session sends the daemon a dup of the pty master, so the shells do not get SIGHUP. Being subreaper only decides who reaps them. - **The recorder.** It is null when no client is attached. When one is, it appends what the authority consumed, in order: - every step's host tape (the results of the ~30 `turn.yield()` sites); - each `serveFs` mutation. Turn order is log order, so the local path changes by one null check. - **The protocol.** One ordered stream: events, host tapes, digests and terminal state. Framing, the hello and the Event codec are kept from wire.zig; the frames go. - **Terminals cross as authoritative screen state,** not as pty bytes: - metadata plus immutable row blobs, assigned in absolute terms; - scrollback fetched on demand. A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the authority, and the local path keeps reading ghostty directly. - **The reflective walker.** One comptime `@typeInfo` walk gives: - the digest; - the complete dump, which also serves join, Restore and Handover; - the rollback copy. Fields are annotated local, authority-only or derived. A hash of the type graph versions it. The hello carries a semantics hash (version, features, grammars), and a mismatched build is refused. - **Rollback.** - When the first speculative event goes pending, the replica copies what speculation can write: views, buffers, registers, counters and terminal metadata. Blobs are pinned rather than copied. - On a divergence it restores that copy, then replays the confirmed events and its own pending ones. - The authority never rolls back. - **Join.** The authority forks a writer child at a step boundary; the child streams the dump. The authority buffers the events after it for the joiner. - **Remote.** - `ssh -L` of the daemon's one socket. - The daemon hands a replication connection's socket to its session. - A remote daemon mounts as `/n/`. ## Determinism leaks (the survey's census) | Leak | Where | Fix | |---|---|---| | Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak | | 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order | | Host syscalls in a step | the ~30 `turn.yield()` sites | host tape | | Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done | | Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local | | Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ | | Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced | | getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state | | Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash | | ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals | ## Phases Each phase ships on its own and keeps `zig build perf --base` level (with an A/A run). All 110 snapshots stay green throughout. 1. **Determinism and the replay-divergence harness.** - Work: the recorder and tape; the leak fixes, the render one first; the semantics hash; the walker's digest. - Check: replay each recorded log into two cores at different addresses and one headless core. - Gate: zero divergences on the snapshot scripts and on N monkey seeds. 2. **A complete dump on the walker.** It replaces dump.zig for Restore and Handover. - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint. 3. **The daemon and the tty stub.** - Work: in-place detach and reattach; pty master dups; the frame wire deleted. - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves its shells adoptable; no stale sockets after N cycles. 4. **Views.** A second window on the same session, in one process. - Gate: logs with several views replay without divergence. 5. **Replicas.** - Work: speculation, rollback, screen-state terminals, join, socket handoff, `ssh -L`. - Gates: - a 30/80 ms delay relay keeps remote within noise of local; - the rollback rate stays low under the fuzzer; - a lagging client leaves the authority flat; - a join costs the authority under one frame. 6. **Remote amortisation.** A content cache by hash, prefetch, zstd, predicted echo in TTY mode, and session upgrade by re-exec. New measurements to add before phase 1 lands: keystroke-to-present in the tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file open. ## Still open (the coordinator's position in brackets) 1. Delete the frame wire at phase 3, leaving no remote attach until phase 5. [yes: it was never used daily] 2. The stream as its own framed protocol, or as a 9P file. [framed: one ordered stream with no per-read round trip; 9P stays for scripting and the relay] 3. Rollback by walker copy or by arena copy-on-write. [whichever measurement of the mutable state's size decides] 4. How the daemon starts: spawning itself or socket activation. [itself, by double fork; FreeBSD has no systemd. A user unit can be added if a terminal's scope ever kills it] 5. The join writer: a fork child, or a metadata copy serialised on a worker. [fork first, measure the page-table copy] ## Phase 3 notes Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`). Decisions taken at the forks, with the ponytail lens: - **Sessions are not the daemon's children; every one registers.** The design had the daemon spawn a terminal's session from the fd, cwd, env and argv the stub passes. That proved wrong here: - the user's terminals each run in a `9ns --mntgen` user and mount namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon spawned would run in whichever namespace the daemon was started from, its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits, umask and stdin (`pardes -`) would be the daemon's too, or have to be carried and re-execed; - startup would pay an IPC round trip and an exec of a 100+ MB binary; - the snapshot harness's idleness probe walks the launched process's tree, and a daemon-spawned session is outside it. So `pardes` in a terminal forks its session off itself before anything starts (`daemon.forkSession`): the child calls `setsid` and draws on the `/dev/tty` the parent opened; the parent stays as its stub. GUI windows and `--detach` sessions just register. The daemon spawns nothing, so it is no subreaper: a detached session is reparented to the system's reaper (systemd --user or init), which reaps it, as it reaps a crashed session's shells. Nothing dangles: the daemon lists every session (its connection) and holds every master, and exits a second after it holds neither. - **Starting the daemon.** The first session takes the flock, binds and listens itself and queues its own connection; `pardes --daemon` is then double-forked and exec'd onto that listener and lock (fds 3 and 4), by a terminal's stub after the session's fork, so no session waits for it: its hello sits queued until the daemon runs, and is not waited on (the daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s). With the spawn before the fork, a cold start measured +0.38 ms (+7.6%). The daemon's reader thread likewise starts after the first frame. Another starter waits on the lock; a stale socket is the lock holder's to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and the fds it carries. - **pty masters.** `forkShell` sends a dup; a host's close sends `close`; a session's end sends `quit` (its shells hang up); a master whose shell is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT handlers send `quit` too (one `send`, async-signal-safe), so only a session killed outright or crashed leaves shells behind. - **Adoption is a word, `Adopt`,** calling the host through a vtable entry (no Effect): the daemon hands over every orphaned master, and each becomes a pane through the `Tty` word, its spawn taking the pty instead of forking; a 1x1 size then the pane's makes it redraw. Adopting at startup was rejected: an `EDITOR='pardes --wait'` session would take the shells and hang them up when the editor exits. - **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and ends with the session's status (a forked stub `waitpid`s; an attaching one reads `x`). A HUP is the terminal's end: kept by a daemon, a session takes HUP on its signal thread (sigwait, beside WINCH) and detaches, as on EIO; HUP never ends a kept session, since the terminal's own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane (the terminal is raw, as before); a TSTP sent to the stub is ignored, since a stopped stub hands the terminal back under a session still drawing; a background start stops at a `tcsetattr` before the fork, as the old raw mode did. `--wait`, nested forwarding and `--tty` run before any of this and are unchanged. - **`--detach` is a session started detached:** the tty host with no terminal, the GUI with a hidden window closed at once. With the frames gone, `detached/server.zig` was a third host with nothing left to do that the other two could not. - **`Attach` (the word) is deleted:** it switched a session's own terminal to another session through the frames. - **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a session as before: in the terminal's own process, ended by its hangup, for a harness that kills by pid and closes the terminal. A GUI test window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`. - **Known gaps.** A forked session takes its stub's death, a SIGKILL too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and detaches; a `pardes --attach` stub is no parent of its session, and one killed with SIGKILL while its terminal stays open still leaves the session drawing there until the terminal closes or Detach. A reattached terminal keeps the first one's environment (TERM, COLORTERM); vaxis queries its capabilities again. A daemon killed outright leaves its sessions unregistered, and Detach then refuses. A daemon of an older protocol answers `version`, and a session then runs without one. A GUI started in a terminal's foreground stays that terminal's job. Measured (ReleaseFast, base main a7369739 against these changes, both live at once and their samples alternating A, B, A, B; A/A the base against itself; p50 ratio new/base): | | tty | A/A | GUI (test mode) | A/A | |---|---|---|---|---| | startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 | | startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | | | key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 | | key echoed by bash, to its frame | 1.016 | 1.005 | | | What is left of the tty's startup is the stub's fork (about 70 us), which the design cannot do without: the job the shell waits on must be able to end while the session runs on. Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout), and overran the stack, clobbering a neighbouring local in the forked child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one. ## Phase 4 notes Built 2026-10-29 (`src/views.zig`, the tty and GUI hosts, `daemon.zig`, the snapshot harness and the monkey). Decisions taken at the forks, with the ponytail lens: - **A view is swapped in, not pointed at.** The ~1,900 references to the core's layout, focus and pane state stay as they are: the core sees one view, the live one, in the fields it always had. A view's share of the session is a list of field names (`views.fields`, ~60) made into a struct by one comptime `@Struct` (`Part`), and of each pane a struct beside it (`PaneView`, in `Pane.pv`); going into view `k` copies the live one's out (`park`) and `k`'s in (`unpark`). Copies, not swaps: a slot is stale while its view is live and written whenever it goes out, so no two places own a selection's heap. One view never parks: the single client pays `views.several` (an AND and a compare) in the pump, the step's sync and an edit, and one store a key (`typedInto`). Rejected: a `*View` the core reads through (every reference rewritten, an indirection on every access); a twin pane per view (Zerox exists only for files, and a terminal's pty is its pane's slot). - **Where each piece of state went.** - Each view's own (`views.fields`): the screen size and cell metrics; the columns, their weights, panes, serials, tags and the finder; rects; `active`, `active_column` and the header focus; the jump list (kakoune's is per client); the pointer, hover, drag, mouse grab, clicks and chords; key prefixes (`"x` register name, leader, Ctrl-w, window pick); a pending clipboard read; the presentation and the animations (lifts, cursor glide); `windowed`, `native_images` and `native_image_patches`, which the census had as watched host writes: a tty view on kitty and one on a plain terminal differ. The 223 comptime platform branches stay comptime: a session is one build, which the semantics hash says. - Each pane's, each view's own (`PaneView`): `vweight`, `collapsed` and `attached_to` (a layout's); `focus`; the body's mode, cursor, selections, sticky column, pending normal command and find; a document's scroll, `hscroll` and size; and what the step measured for that view (`tag_rows`, `tag_top`, `tag_scroll`, `view`, `notices`), so a view drawn without a step of its own draws its own geometry. - The session's, every view's: the panes' text, undo, terminals and their scroll (tmux shares a pane's copy mode too), tags (acme's are per window; one tag per buffer is one text, its caret included), the workspace tag, prompts, messages, registers (decided), the macro being recorded, look-hover (one pointer at a time), mouse sweeps, query panes, search state, settings and theme, the effect and input queues, `needs_frame` (any change draws every view). A `.name` prompt (a file's name drafted in its tag) is the view's that opened it (`Pane.prompt_view`): only that view's focus leaving the pane drops it, and its closing. - **A terminal shown at two sizes** takes the size of the view that typed into it last (`Pane.sized_by`, set by a key or a paste; a new pane's is its maker's; a closed view's goes to the next to sync it). The others show it at that size: `sync` resizes no pty for them, and `pane.cols/rows` stay the terminal's own, so the terminal code's invariant (pane size is the emulator's) holds. A view shorter than the terminal is passed down its top rows as far as keeps its cursor's row in sight, as tmux shows a bigger window in a smaller client (`Pane.pan`, terminal.panFor: per view, made in the step and again when a view is drawn out of turn, read where the body's rows start, `Pane.scroll` and the body walk), an arrow in its gutter's first row saying rows are above; a shell's prompt, at the bottom, is never out of sight. ponytail: a panned view's mouse reports to the terminal's program and its kitty images are placed as unpanned. - **Panes come and go under parked views.** A view keeps the serial of each slot when it goes out; unparked, the ids whose serial changed leave its layout and its keyboard moves to its first pane left. A pane focused in a view that does not show it (a Look at a file another view opened, a jump, a 9P focus) is placed in the view's active column (`views.show`, kakoune's buffer shown in the window that asked). A text edited in one view moves every other view's cursor and scroll with the splice, as a Zerox twin's (`File.ParkedPlaces`); their selections go, as a twin's do. - **Drawing the other views stays out of the digest.** The pump draws each other view by unparking it, rendering and presenting, and restores the live one exactly (`drawOthers`): no digested field moves, so the headless replica, which draws nothing and skips it, agrees. What drawing a view makes of its own (its animations, its presentation) is kept; what its unpark normalised (a cursor clamped inside a changed text, a pruned layout) is not, and is done again when it next goes live by an input, in every replica alike. `parked` and `pv` are `.local`: a parked view is checked when it is live again. Found by `pardes-monkey --views=3 --tape`: a drawn view's `post_present` (acknowledgePanelPresentation, which moves the pointer's hover) was applied by the headless replica to the live view; other views take no post_present, their panels acknowledged when they are live and drawn. - **Inputs.** `toView`, `openView` and `closeView` are taped. A host goes into a view before the input its client sent: the tty's terminal readers post through a `Poster` that puts a `from` ahead of a terminal's input whenever another terminal posted last (so one terminal posts none, and coalesced motions never cross views); the GUI asks SDL which window an event is for (`route`), which also takes a window's resize to its view (the GUI sizes the live view from its window each frame). - **The tty.** Each terminal is a `Screen` of its own: its vaxis, input reader, frame answer (`FrameAck`, was global), kitty images and shm probe (were global), glide, trail and flash, its stub and its Killed slot. The primary's stub stays `daemon.stub`. A SIGWINCH asks every terminal its size (attach stubs forward theirs; another view's terminal is no one's controlling terminal); a SIGHUP asks each whether its terminal or stub's pipe hung up, where it used to mean the one terminal. Detach, or a terminal's end, with views left closes only that view and tells its stub `d`; the session's end tells every stub. ponytail: a bracketed paste in progress is the shell's, not a terminal's: keys from another terminal during it join it. - **The GUI.** One `Gui` (device, atlas, fonts, pipelines) and its window's share swapped like the core's (`window_fields`: the SDL window and its presenting, scroll and glide, scene and post chain, pointer, blink, captures); `Shell.follow` keeps the Gui on the core's live view. Closing a window while others stay closes its view; Restore closes the others. ponytail: a second window's compositor hit test reads whichever window's state is in the Gui (the workspace tag is in the same place in every view). - **The daemon.** `attach` names an attached session too: it takes another view (a terminal, or a window). With no name, the detached sessions are asked first, then the only session of that kind. - **Harnesses, extended not new.** A snapshot script's `start ... views=N` gives the app N-1 more ptys (PARDES_TEST_VIEWS, opened as attached ones from the start), `view ` types into and captures terminal k, and every wait reads all of them (a pty nobody reads would block the session's frame); a `resize` there SIGWINCHes the app, as a stub would. `pardes-monkey --views=N` switches among N terminals a quarter of its steps, each with its own size; `zig build replay` runs its seeds at one view and at three. The GUI's test feed takes `ESC]777;view;BEL`, and PARDES_TEST_VIEWS=N opens N more hidden windows, captured under view-k. - **Not done, and why.** - A view of the other kind (a terminal on a window session, a window on a terminal's): each process links one frontend; the daemon still matches kinds. - 9P acts in the live view, the one whose client spoke last: no view is named in a path yet (a script's focus or layout is that view's). - Every change draws every view (`needs_frame` is the session's); a view's own dirtiness waits for a measurement that wants it. - Dump, Restore and Handover keep one view, the live one. - **What proved wrong in the design.** "Views" in the design were a struct the core reads through; the swap made them cheaper and kept every reference. Its rollback copy (phase 5) is the same list of fields. The design's view-local state assumed a step per view; the step's measurements (`tag_rows`, `notices`, the pane's geometry key) had to become per view too, or a view drawn after another's step drew the other's tag heights. And the census's single "frontend kind" leak is not per process but per client: kitty graphics are a terminal's. Measured: ReleaseFast, the base binaries from before the rework against this line on phase 1's merge (vvmuwntp, before main's tty throughput fix), interleaved base, new, base ... in one run on the perf cores, then an A/A of the base against itself the same way; the ratio of the medians of runs, new/base (4 runs of new, 5 of base; `--reps 25` for the core, `--live` and `--live-gui` through the 32 MiB live driver): | | new/base | A/A | |---|---|---| | core, medium: render, keys, PageDown, wheel | 0.97-1.02 | 0.99-1.01 | | core, medium: edit-char, edit-twin, deep-sel | 0.90-0.96 | 0.99-1.00 | | core, medium: open, body-2m, body-appends | 0.94-1.02 | 0.88-1.08 | | tty, keystroke to present p50 (5 gestures) | 0.94-1.11 | 0.90-1.02 | | tty, the core's part of it p50 | 0.95-1.04 | 0.86-0.99 | | tty, `yes \| head -c 32M` wall; 1M-line open | 0.96; 0.98 | 1.09; 1.02 | | GUI, keystroke to present p50 (present-bound, 30 ms) | 0.98-1.01 | 1.03-1.10 | | GUI, the core's part of it p50 | 0.84-1.01 | 1.01-1.19 | Every cell is inside its A/A; the widest, the tty's PageDown at 1.11 (runs 386-506 us against 410-451), is 0.94 best against best. What one view pays is a byte test in the pump, the step's sync and an edit (`views.several`), a store a key, an index for the live terminal or window, and a `from` never posted. A first A/B, of this line before the rebase onto phase 1's merge (its render still made the geometry afresh every frame), read the GUI's core at 0.89-1.16 with no A/A to set it against; it is not the one reported. Two views (not gated): every frame draws both, so a change costs two renders; a view switch copies ~60 fields and each pane's share, a few microseconds. Gates, on the line rebased onto main's screen-state terminals (mnlwlpvk): - `zig build replay -Dplatform=tty`: 113 snapshot scripts pass (the 110, views.snap, views-term.snap and views-pan.snap), their 131 tapes replay in all three replicas (393 runs, 29,436 digests); 20 monkey seeds at one view (60 runs, 14,004 digests) and 20 at three views (60 runs, 9,666 digests); no divergence, no panic. views.snap: 75 digests a replica, views-term.snap 35. core-test: 1117 of 1135, 18 skipped. - Unit tests (on qzsouslm): tty 1478 of 1498 pass, 20 skipped; GUI 1549 of 1560, 9 skipped and 2 failed, the same two that fail on phase 1's line (`tagline pointer mapping retains physical grips...` and `a tag caret past its row's end glides...`). The views' own: 8 (own cursor, scroll, mode, layout and focus; an edit and a register shared; a terminal sized by its last typist; views closing; the other views drawn leave the digest; a view keeps its place in a text another edits; a short view keeps a tall terminal's cursor in sight). views-pan.snap: a shell sized for the tall view, its prompt and last lines in the short one under an arrow, then typed into from the short one and sized for it. - test/fs.py passes (with a TMPDIR short enough for the 9P sockets' paths). test/daemon.py: 20 detach and attach cycles, then a second terminal on the attached session, both drawn, detached from inside and then closed outright, the first drawing on, and the kill and Adopt; `--gui`: 10 window cycles, then a second window on the open one, detached alone. - `pardes-monkey --views=3 --tape` found the one divergence fixed here (the drawn views' post_present). ## FreeBSD notes Groundwork for phase 5's main test, a client here and the session on the FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and runs there. The GUI is not built for it, since frontends stay local. The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is the host's root shell; port 2222 is a git jail and is never touched. Everything of ours lives in `/root/pardes-test`, nothing outside it. - **Building.** Everything, the C deps included, cross-compiles here: ```sh export PATH=$HOME/.local/bin:$PATH E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P' $E zig build -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/bin/pardes-freebsd-x86_64 $E zig build test-install -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files ``` The `.15.0` matters: Zig's default FreeBSD (14.0) links `libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs the test-build programs instead of running them. A test binary takes no filter when it runs, so running one test alone means a rebuild with `-Dtest-filter=` (and `--prefix ` to keep the full set). - **Deploying.** `-F /dev/null` is required here, because /etc/ssh is owned by nobody: ```sh S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519' zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all scp $S p.zst root@65.109.81.103:/root/pardes-test/bin/pardes.zst ssh $S root@65.109.81.103 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst' ``` The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh ` (a build, PARDES_* stripped, logged there), `deploy.sh [name]` and `bx.sh ''` (a command run there with env.sh sourced). `sshcheck.py` drives a session through `ssh -tt` from a local pty, and `fwd9p.py` times 9P through `ssh -L`. - **The box's layout.** Under `/root/pardes-test`: - `bin/`: pardes, and `python3` linked to python3.12; - `run/`: the runtime directory (sockets, the daemon's lock); - `home/` and `tmp/`; - `src/`: the tracked tree, which the tests need: they read docs/typ, and make their temporary directories in `.zig-cache/tmp` under the working directory; - `test-bin/test/` and `logs/`; - `env.sh`, which points every path pardes uses there (HOME carries the XDG defaults): ```sh T=/root/pardes-test export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color} ``` - **Running.** - A session: `ssh -t $S root@65.109.81.103 '. /root/pardes-test/env.sh; pardes notes.txt'`, and `pardes --attach=` the same way. - The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`. Some tests name `/tmp/pardes-*` outright and leave seven empty directories there (`pardes-arg-save` and the like): rmdir them after. - The e2e scripts, from `src/`: `python3 -B test/daemon.py /root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way. - Kill every process you start there by its pid, and check `ps -axww` afterwards. - **What FreeBSD does differently.** Each item was found on the box, and each is fixed where pardes met it unless it says otherwise: - A write to a terminal that is gone fails with ENXIO, not EIO. std does not expect that errno, and a Debug build traced it to stderr, which is the same dead terminal, recursing until the stack ran out: a session whose ssh dropped hung. Tracing is off on FreeBSD (main.zig). - A signal whose action is to be ignored is dropped when it is sent, sigwait or not, so WINCH never reached the session's watcher. A handler that does nothing keeps it (tty.zig). - waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64 bits. With Linux's numbers a command pane's exit was never seen. - There is no /proc. A shell's directory and program come from sysctl instead (kern.proc.cwd, kern.proc.pathname). - `ttyTaken` answers taken for any foreground group but the shell's, so a nested shell counts as a job. - `signalSession` stays Linux-only, so Kill leaves a command's `&` jobs running there. - File watching uses FreeBSD 15's inotify: Linux's events, with its own IN_CLOEXEC and IN_NONBLOCK. - `shm_open` refuses O_NONBLOCK; its names are not files, so there is no FIFO to fear. A `write` does not grow a shared memory object. - A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a stat by path now comes first. - A Unix socket whose listen queue is full refuses a connect with ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep that judged by connecting could remove a live session's socket while its queue overflowed. A session's liveness is now its lock: it holds `.lock` flocked from before it binds until after it unlinks (9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they unlink only under it. A connect judges only a socket with no lock, one of a pardes from before the locks. The daemon's own start was already decided by its flock. - **The tests there.** The box runs them as root, with no zls, fish or fonts installed. - main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no zls), and two tests that make a file or directory unreadable, which root reads anyway. - panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43 passed. The small programs all pass but fonts (no fonts). - Off Linux, the parts of tests whose refusing path is Linux's /proc/1 are skipped, and so is the builtins JSON check, since the committed lists are Linux builds' (Tty9p included). - test/daemon.py passes there at 20 cycles, its killed stub included (procctl's death signal). One earlier run failed at cycle 3, one process short in its count (most likely ps -e failing to read an environment). - test/fs.py runs past its `pardes -` +Pager checks now, as far as the SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty after the first frame, FreeBSD's pty fills before the session's frame and reset are written, and the session waits in `ttyout` (a binary from before these fixes too). Open. - **At 207 ms, for the replicas.** - In today's model every key is a round trip. Measured through `ssh -tt` (min, median and max): - a key typed into a file was on screen in 213, 229 and 346 ms; - a key typed into a shell pane in 218, 219 and 224 ms. That is one RTT and about 15 ms. - 9P over `ssh -L` of the session's socket costs one RTT a message: 670 ms to connect, 1035 ms for a file read (walk, open, read, read, clunk), 830 ms for a ctl write. A replica cannot fetch its state file by file. - The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L` cannot forward it (tried: EPROTOTYPE here, connect failed there), and no fd would cross anyway. The replication connection has to be a stream of its own, as the 9P socket is, which the daemon hands on locally. - A dropped connection is a hangup: the session detaches, and `pardes --attach` takes it back, as after a Detach. - sun_path is 104 bytes there, not 108. ## Phase 1 notes Built 2026-10-29 (`src/walk.zig`, `src/tape.zig`, the hooks across the core; `pardes-snap --tape`, `pardes-monkey --tape`, `zig build replay`). Decisions taken at the forks, with the ponytail lens: - **The tape is a tree recorded at the host boundary, both ways.** Each call the host makes into the core (`tape.Input`: update, serveFs, pump, perform, the shell's acknowledgements and failures, a Restore...) is recorded with its arguments; each call the core makes out -- a Host function, wrapped generically over the vtable's fields, or a syscall of its own (`tape.site`, which yields around it; every old `turn.yield()` is one) or a read of the clock or environment (`tape.value`) -- with its result. `done` and `ret` close each, so a 9P request served while a step is out lands inside that step's syscall: turn order is tape order. A replica drives the same functions and its host answers from the tape; one that makes a call the recording did not diverges on the spot, before any digest. `turn.yield` panics in a replica, so an untaped syscall is a stack trace. - **The replay is a mode of the same binary** (`PARDES_REPLAY= pardes`), not a harness of its own: an error travels as this build's number, and the code replayed is the code that recorded. Three replicas run one after another: two that draw (the first's memory held while the second runs, so it lies elsewhere) and one that never renders. The headless one steps over the presents it does not make but takes what the host told the core inside them (`acknowledge_panel_presentation`). - **Fields a host writes between calls** (`native_images`, `present_skipped`, `active`...) are found by a diff of a short list at every crossing, not by touching the frontends. - **The digest** is taken after every call the host made at the top (a pump, in the tty), as one per field of the core, its stream and each pane, its file and places, so a divergence names the field. `PARDES_RECORD_EVERY` digests after every input, nested ones too, to find the step one starts at. - **The walker needed a fourth annotation, `len`**: a fixed array written only as far as a length field says (`jumps`/`njumps`, a message's text) is undefined beyond it, 0xAA in a safe build and a fault where it holds a slice. Integers are hashed by value (a `u21`'s padding bits are not its value). - **View geometry** is `File.geometry`, a lazy cache keyed by a hash of what it is made of: revision, length, path, scroll, hscroll, the pane's size, tags and query, tables as far as read, context declarations, the output origin, and the pane's `view` (wrap, layered context, row metrics, rect height), which `sync` sets. A hundred readers a frame ask for it, so the key is one hash of those facts laid out as words (fed field by field it cost 4% of every frame). A step reading it (`wrapAt`, `wrapRow`, `ensureCursorVisible`, a body drag, `zz`) makes it on demand, and the render draws from what the step made: made afresh for every frame as well (to show an input the key missed only as a headless divergence), it cost a scroll a third walk of the rows, 4% of a PageDown. An input the key misses shows as a stale frame in a golden. - **Render wrote more than the census found**, all moved into the step: context declarations, the notice bands and their chips' columns (a click on a chip was routed by the last frame's), the header's caret reveal (`tagnav.golden` moved: after `g l B B` typed in one burst the bar stays scrolled, as a human typing it sees it), the tables' reading (a first look of 4096 lines, a count, wherever it is asked; the rest a frame's taped clock budget a tick), the tty filter palette and body chrome (derived). A view's paint-ahead stays the render's own prefetch: instead of queuing an effect it is asked of the host once the frame is presented (`Stream.askAhead`, in the pump), so a replica that draws nothing asks for none; query rows' paints count apart from it. (Asked inside the render, its worker's thread start cost a held `j` 10% in the live tty measurement.) - **Clock.** `advance`'s wakes are the recording session's (`tape.value(.wake, nextWake)`): a picture decoding or a fade asks a frame, and a replica that draws none must still take the session's ticks. The budgets of Places, Names and Tables are taped verdicts (`tape.over`), as are a message's time of day (TZ included), File.mtime and the pump's clock read (made whether or not it waits). A message's stamp and an unattended session's expiry of it read the clock through the tape too, not the session clock: that is the last pump's, and stamped by it a message written over 9P after an idle spell was as old as the spell and went at once (fs.py's `unattended_messages_expire`). - **A pointer seen through a sliding pane** is taped (`Pardes.pointerCell`): what was under it is the frame the user saw. - **The host's facts**: the session directory (getcwd, asked once at init), `$SHELL` and whether it is there, `$HOME` for `~`, a `Shell` that is not there, a missing interpreter, a directory refused above, a file deleted on disk (`File.deleted` is an input). The rest of the core's getenv (PARDES_DUMP and the XDG dirs in dump.zig, TMPDIR for kitty media, PARDES_TEST_CLOCK in the render) is the host's or authority-only. Per read rather than "captured once into session state": less code, and the same for a replica. - **Authority-only**: worker futures and their Io (a future's presence too), the 9P server's state (`fs`: fids, opens, the log; serveFs itself is replayed in turn order, 9p_io's own writes into `fs` are not), the store file every window of the user shares, which a replica keeps in memory, and Recent read from it. **Local**: the surface, presentation, frame stats, animations, the effect and event queues (drained by every pump before its digest). **Derived**: the geometry, notices, highlights, line index, tables, node trees, rail ticks, a compiled regex, an image's decoded pixels, PDF rasters. - **Terminals** replay from their taped pty bytes (they are `output` events) and are digested as their active screen, cursor and scroll numbers; their kitty images are left out until phase 5. - **Worker results**: the census's render-polled ones (image decode, PDF lanes) are derived state; the rest already arrive as events. - **Unit tests that mutate and render** without a step now step (`p.sync()`) first, as every host does; three of them had passed on the last frame's stale state: a prompt chip a body click had already closed, a `.center` landing measured on no wrap map at all (it centres by lines, so under wrap a line above can now push the cursor down where `ensureCursorVisible` takes the view back: that test is unwrapped now), and a context row count a scroll had not refreshed. The perf gate's `deep-sel` cell was one more: its view, placed from the last frame's map, stopped six lines short of the 2 MB line it selects in. In view, that line showed `File.rawAtDisplay` measuring a line's whole rest before walking to a column (20 ms a frame, Debug); it walks to the column only now, and the cell is the baseline's (91 us against 106). What proved wrong or missing in the design: - The census missed the host writes between calls, the pointer through the presentation, the render-time writes above, `Stream.start` (host code doing core work, made harmless by the futures being authority's), getcwd, `access(2)` for `$SHELL`, File.mtime's clock, and a deadlock in vaxis's queue (a `yes` stopped being read; fixed on this line, found by the new measurement). - `windowed`/`native_images` are watched host writes for now, not View state (phase 4); the semantics hash in the hello is not done (there is no hello in phase 1), so a tape from another build is not refused yet. - Deferred: the GUI's own host-to-core calls (font requests, opacity, `core.advance`/`render` are inputs, the rest not yet), PDF lanes and Handover yields (never reached by the gate; a replica would panic on them, which is the point), `Stream.runHere`'s yield (a host with no workers). The gate on the final line (on main vukknslx): `pardes-snap --tape`, 110/110 scripts, 128 tapes, 384 replica runs, 33,093 digests, no divergence; `pardes-monkey --tape`, 50 seeds of 70 steps, 150 replica runs, 37,800 digests, none; `zig build replay` (the 110 scripts and 20 seeds) clean; `snap` 110/110 with no golden changed but `tagnav`; unit-test tty, pane-test, 9p-io-test and fs-test pass, unit-test gui fails only its two known tests. Perf, base (main with perf --live and the queue fix) against this line, ReleaseFast, interleaved A B A ... A, medians of runs, the base's own spread as the A/A noise. The perf harness's gesture cells (render, keys, PageDown, wheel, edit, deep-sel) 0.97-1.03 at an A/A of 2-7%, and on the final line, measured on the shared perf cores, 0.91-0.97 at 10-31%. The live tty on a quiet machine: core time p50 0.96-0.97, keystroke-to-present p50 0.98-1.03 (A/A 1-23%), `yes | head -c 1G` 0.96, a million-line open 1.01. The live GUI's core p50 1.00-1.04 (A/A 2-18%), present-bound at the window's 30 ms. Two costs the A/B found are gone (the geometry key, the paint-ahead's thread start in the render). ## Tty throughput notes Found and fixed 2026-10-29 (`src/tty/tty.zig`, `waitInput`). - **What the 1 GB `yes` cell measures is mostly the kernel.** A pty's output processing (ONLCR, a `\n` made `\r\n`) costs the writer about 0.4 us a newline here, and `yes` writes one every two bytes. Every reader of a pty is capped by it: `script(1)` drains 100 MB of `yes` at 8.6 MB/s with OPOST on and at 650 MB/s with it off, a Python drain at 8.0 MB/s, and tmux at 5.4 MB/s. The tty pardes was at 5.7 to 6.1 MB/s, three quarters of that ceiling, before any change. The 179 s were the kernel's. - **Where pardes itself was slow:** with 200-byte lines (`yes` of a long word), the ceiling rises to 264 MB/s, tmux reaches 55 MB/s, and pardes managed 18.6 MB/s. A flat profile (perf at paranoid 2 allows no call graphs) put 98.7% of the samples in pardes: `Vaxis.render` 29%, `body_layer.renderBody` 17%, `Shell.present` 10%, memcpy 8%, ghostty's parse 1.5%. The tty drew a frame for every pty read, which is 4 KB, nearly all of them overwritten before anyone could see them (about 690k frames for the 1 GB cell). - **The fix: output is drawn once per frame tick at most** (16 ms), with whatever came in until then, as the GUI drains its queue once per vsync. Two things are drawn at once, as before: - an input's frame; - the focused pane's output within a second of an input (its echo; a remote shell's arrives an RTT later). Neither waits while it is undrawn, including when FrameAck held its frame for the terminal's answer. Snapshot runs (the virtual clock) are not paced. - **And a pane has two reads queued at most** (`readPty`): its reader waits for the loop to take one before it reads on. ghostty parses `yes` at about 9 MB/s, 0.45 ms a 4 KB read, slower than this machine's pty delivers it. So a flood filled the loop's queue (512 events): a key and its echo waited behind its reads, and which blocked writer a free slot woke was luck, which gave half-second peaks. Paced but unbounded, the wait was steady at about 20 reads, 9 ms for the key and 9 more for its echo. A frame dump lined up with each key's write showed it (PARDES_FRAMES and `flood.py` with FRAMES=). What a flood writes beyond two reads now waits in its pty; the loop could not have kept up with it anyway. - **Measured** (ReleaseFast, `.scratch/daemon/term/yes.py` through a real 120x40 pty, all interleaved, with measurements pinned to two physical cores and so four CPUs; medians of 3 rounds, MB/s): | | `yes`, 32 MiB | 200-byte lines, 256 MiB | |---|---|---| | base (before the rework) | 5.5 | 14.5 | | phase 1's tip | 4.8 | 15.0 | | pacing alone (the queue limit came after) | 4.6 | 117 | | tmux | 5.4 | 47.6 | | raw pty drain (Python) | 7.7 | 239 | | GUI, test mode | 0.9 | | Unpinned, before the pinning: 200-byte lines 18.6 to 247 MB/s (1 GB in 53.7 s and in 4.0 s), and `yes` 5.7 to 6.9 MB/s. The GUI could only be measured in its test mode (a hidden window), which downloads every frame from the GPU, so 0.9 MB/s says nothing of a real window. There is no headless compositor here, and a real window must not open on the user's session. In a later interleaved run, the change as committed against pacing alone gave 4.8 and 6.0 against 4.8 and 5.4 MB/s for `yes`, and 102 and 117 against 124 and 119 MB/s for long lines: the same within the noise. `pardes-perf --live`'s `yes` cell drew 2.3k frames for 32 MiB against 11k before, in the same time: it is kernel-bound. - **A held key while another pane floods** (`flood.py`: a key every 30 ms into `cat` in one pane of `-n 3`, timed from its write to the pty to its echo on screen, with pyte in the loop; `yes` in another pane): in one interleaved run of two rounds (p50, p90, worst, in ms): | | under the flood | idle | |---|---|---| | phase 1's tip | 3.0, 212 to 214, 445 to 470 | 2.2 to 2.3 | | pacing alone | 10.3 to 10.6, 19 to 20, 26 to 33 | 2.3 | | this change | 3.3 to 4.2, 4.3 to 5.6, 6.0 to 7.8 | 2.3 to 3.2 | The driver's own cost (pyte) is in every number. With 16 KiB queued a pane instead of two reads, the flood's p50 was 7.5 to 7.8 ms for no throughput gained, so two reads it is. - **Keystroke-to-present** (`pardes-perf-live --live`, base and this change interleaved, 5 and 4 runs; A/A between the base runs' halves): p50 against base is 0.92 for key-down, 0.89 key-right, 0.85 edit-char, 0.80 page-down and 0.93 wheel. The A/A was 0.73 to 0.92. The 32 MiB `yes` took 1.00 the time with 0.18 the frames, and the big open 1.10 (A/A 1.05). Two earlier runs of pacing alone, 3 and 5 rounds, were within their A/A too. - **Unrelated, found on the way:** `test/fs.py` failed `unattended_messages_expire` (its first `shown()`) on phase 1's tip as well; pmxpuolt on main (a message's stamp and expiry read the clock again) is that fix. fs.py also needs a short TMPDIR: under `.scratch/daemon/term/tmp`, `directory_panes`'s socket path is 108 bytes, one past the limit, and the session starts with no 9P. Two GUI unit tests fail on phase 1's tip as well (`tagline pointer mapping ...`, `a tag caret past its row's end ...`). A suite started as a shell's `&` job runs with SIGINT ignored, so four host_io tests whose Ctrl-C must stop a `sleep` fail there; started otherwise, they pass. ## Screen-state terminal notes Built 2026-10-29 (`src/screen_state.zig`), a pure encode and apply pair over byte buffers for the phase 5 stream; no transport yet. Only tests reference it, so the tty and GUI binaries carry none of it: no symbol of it is linked into the tty built from this tree. The local path's A/B, base against that tty, is the Tty throughput notes' keystroke-to-present. - **Both kinds of blob are ghostty's own snapshot records,** not a cell format of ours. The terminal-wide state (modes, colours and palette, title, pwd, tab stops, mouse modes, scrolling region) is a TERMINAL record. A row is the PAGE record of a one-row page the row is cloned into, which carries its styles, graphemes, hyperlinks, wide cells, wrap and prompt marks exactly. The replica decodes them with ghostty's decoders and writes a row with `Screen.clonePartialRowGrowCapacity`. - **A row's address is a hash of what it shows** (Wyhash of its flags, its width and its cells by value), not of its blob: ghostty keeps the source page's style ids when it clones a row, so equal rows give blobs that differ. The width is in the address, because a blank row is the same at every width. - **A delta** is a header (cursor, kitty keyboard flags, the active area's first row as an absolute row, the history's extent), the TERMINAL record's address, an address for each active row, an address for each row that scrolled off since the acknowledged state (the tail, up to a screen of them), and the blobs for the addresses the peer does not hold. An unchanged row costs 8 bytes. - **One delta per peer is in flight,** against the state it acknowledged, as FrameAck keeps one frame in flight for the tty. A slow peer skips the states in between. The peer holds exactly the blobs of the state it acknowledged, at most two screens of rows and the meta, so the authority knows what it holds without asking. At 207 ms this means at most about 5 deltas a second to the FreeBSD box; phase 6's predicted echo is meant to cover typing there. - **Absolute rows** come from a tracked pin at the last active top. The rows scrolled since are the distance to the new top; if the pin's page was pruned, the count jumps past every row the replica could hold. A resize renumbers (an epoch), and the peer starts afresh. Scrollback is fetched by absolute row (`encodeRows`, `applyRows`). - **The replica writes into a ghostty Terminal,** a terminal pane's own, so `src/terminal.zig` draws it unchanged. A test renders a replica's terminal pane and its authority's in two cores: the bodies are equal, cell for cell. - **Gates.** - The soak drives an authority with pty bytes and lands deltas at random, both skipping states and delaying a delta while the authority moves on. Resizes and fetches of holes also come at random. After each delta it checks the replica's screen against the authority's as encoded, and every history row the replica holds outside its holes against the authority's row at that number. - It passed with synthetic shell, TUI and `yes` streams at 4 seeds (in the unit tests, with a 64 KB scrollback so it prunes). It also passed with 197 recorded streams at 3 seeds each (the test that PARDES_SCREEN_STATE= turns on): the 192 pane outputs of the snapshot scripts' tapes, plus real vim, nvim, htop, less and btop sessions recorded through a pty. Its corpus and recorder are in `.scratch/daemon/term`. - **Numbers** (ReleaseFast: `PARDES_SCREEN_STATE= zig build core-test -Dplatform=tty -Doptimize=ReleaseFast -Dtest-filter=screen_state`, which runs under std.testing's checking allocator): - Encode, 200x60, every row new: p50 250 us, p99 487 us, 29.8 KB. - Encode, 200x60, nothing new: p50 71 us, p99 139 us, 534 bytes. That is the 60 rows hashed and the TERMINAL record encoded to hash it. - Under `yes` as fast as ghostty takes it in-process (9 to 12 MB/s), with a delta every 16 ms: 62 deltas a second, 63 KB/s on the wire, 1 KB a delta (60 rows and the tail's 60, all held). That is bounded by the frame rate, not the output. - Under varied text at 36 to 55 MB/s: 355 KB/s on the wire, 5.6 KB a delta. - The replica applies a `yes` delta in 2.4 ms, most of it pushing the rows scrolled past into its history (a row at a time). A delta of varied text applies in 0.24 ms under the C allocator and 0.95 ms under the checking one. - **What proved wrong in the design.** - "Scrollback fetched on demand" alone loses rows. A row that changed after the last delta and then scrolled off would sit stale in the replica's history. The tail fixes the rows that scrolled within a screen. Older ones, which the replica never saw or saw before they last changed, are holes: rows it must fetch before it shows them. - The address had to include the row's width, as above. The soak found this: a replica reused an 80-column blank row after a resize to 10. - ghostty narrows pages lazily, so an active row can be wider than its terminal: only the terminal's columns are hashed and encoded. - **Not done, or a gap.** - Kitty images in terminals. - pardes's own state derived from the stream is not carried: its `MarkedHandler` prompt phase, command-output pins and the replies it writes to the pty. ghostty's prompt marks on rows do cross. - The view's scroll offset is per client and is not sent. - A replica's core must not resize its pane's terminal itself; the authority sizes it. - The replica's history extent follows its own pruning under the authority's limit, so it can be a page off, and its oldest rows can be ones the authority has dropped. - A fetch sends every row as a blob (phase 6's cache will send addresses). No zstd. - A replica pushes a flood's rows into its history one at a time. - The integration into the stream and the cores (a replica pane's State fed by `apply` instead of `.output`) is the next piece.