# The daemon: one pardes per machine, many clients per session > I want to have a global pardes daemon ... holding file descriptors for > ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a > cpu+file server ... the main test: multiple clients to a single session, > and a client here and the session on freebsd and very deep and thorough > performance testing and optimizations to make it feel like its local and > snappy. Designed 2026-10-29 in discussion (the user, the coordinator, a survey agent). Nothing here is built yet. Paths are relative to the repo. ## Decided by the user - **A deterministic core, replicated.** The session is the authority: it orders every client's input and every host result, and only it performs effects. Each client runs a full core replica, applies its own input at once, and rolls back and replays when the order differs. Visible rollback is acceptable. - **Kakoune-style clients.** Each client has its own view (layout, focus, selections, scroll) over shared buffers and terminals. Registers are session-wide. A terminal shown at two sizes is sized by its last typist. - **Performance order.** First, the regular tty and GUI get no slower. Then the detached and remote clients reach that local performance, with the session on the FreeBSD box included. - **A host step may show up one RTT late, but is never wrong.** This covers Look, open and exec, as sam's hostlock does. - **Join is complete.** The joining client pays for it in time; the authority and the other clients do not. - **One long-lived daemon per user per machine.** It supervises one child process per session, postgres-style. Nothing is left dangling, and the core is not de-globalised. ## What today's seam is, and why it goes `src/detached/server.zig` is a third Host: it runs the core and the ptys in a single poll loop and sends cell frames to the frontends, at the smallest common grid across them. - **The frames are mostly not diffed.** Every layer's cells are re-sent on each frame (wire.zig:734, :755). - **Several things never cross the wire.** Images, PDF pixels, rail ticks and pane animations stay behind (surface.zig:483-509). - **Every keystroke is a round trip,** which no amount of optimisation can make feel local at 30-80 ms. - **Each remote 9P operation dials afresh,** costing about 6 RTTs (9p_io.zig:2361-2385). ## The design - **Processes.** - The daemon is subreaper, spawner, fd holder, registry and 9P root. - Every session is a child of the daemon: a GUI window, a tty session behind a stub, or a headless session. - A session is always an in-process authority with its own local view, so the regular path does not change. Replicas exist only for extra clients or remote ones. - **The tty stub.** - `pardes` in a terminal passes its terminal fd to the daemon (SCM_RIGHTS), along with cwd, env and argv. - The session child drives that fd directly. The stub forwards SIGWINCH and exits with the session's status. - Detach: the session drops the fd and the stub exits. No fork happens. - **GUI detach.** The window is destroyed and the process carries on headless. Closing a window that has no other client attached quits it. - **Shells survive a session crash.** After each `forkShell`, the session sends the daemon a dup of the pty master, so the shells do not get SIGHUP. Being subreaper only decides who reaps them. - **The recorder.** It is null when no client is attached. When one is, it appends what the authority consumed, in order: - every step's host tape (the results of the ~30 `turn.yield()` sites); - each `serveFs` mutation. Turn order is log order, so the local path changes by one null check. - **The protocol.** One ordered stream: events, host tapes, digests and terminal state. Framing, the hello and the Event codec are kept from wire.zig; the frames go. - **Terminals cross as authoritative screen state,** not as pty bytes: - metadata plus immutable row blobs, assigned in absolute terms; - scrollback fetched on demand. A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the authority, and the local path keeps reading ghostty directly. - **The reflective walker.** One comptime `@typeInfo` walk gives: - the digest; - the complete dump, which also serves join, Restore and Handover; - the rollback copy. Fields are annotated local, authority-only or derived. A hash of the type graph versions it. The hello carries a semantics hash (version, features, grammars), and a mismatched build is refused. - **Rollback.** - When the first speculative event goes pending, the replica copies what speculation can write: views, buffers, registers, counters and terminal metadata. Blobs are pinned rather than copied. - On a divergence it restores that copy, then replays the confirmed events and its own pending ones. - The authority never rolls back. - **Join.** The authority forks a writer child at a step boundary; the child streams the dump. The authority buffers the events after it for the joiner. - **Remote.** - `ssh -L` of the daemon's one socket. - The daemon hands a replication connection's socket to its session. - A remote daemon mounts as `/n/`. ## Determinism leaks (the survey's census) | Leak | Where | Fix | |---|---|---| | Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak | | 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order | | Host syscalls in a step | the ~30 `turn.yield()` sites | host tape | | Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done | | Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local | | Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ | | Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced | | getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state | | Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash | | ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals | ## Phases Each phase ships on its own and keeps `zig build perf --base` level (with an A/A run). All 110 snapshots stay green throughout. 1. **Determinism and the replay-divergence harness.** - Work: the recorder and tape; the leak fixes, the render one first; the semantics hash; the walker's digest. - Check: replay each recorded log into two cores at different addresses and one headless core. - Gate: zero divergences on the snapshot scripts and on N monkey seeds. 2. **A complete dump on the walker.** It replaces dump.zig for Restore and Handover. - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint. 3. **The daemon and the tty stub.** - Work: in-place detach and reattach; pty master dups; the frame wire deleted. - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves its shells adoptable; no stale sockets after N cycles. 4. **Views.** A second window on the same session, in one process. - Gate: logs with several views replay without divergence. 5. **Replicas.** - Work: speculation, rollback, screen-state terminals, join, socket handoff, `ssh -L`. - Gates: - a 30/80 ms delay relay keeps remote within noise of local; - the rollback rate stays low under the fuzzer; - a lagging client leaves the authority flat; - a join costs the authority under one frame. 6. **Remote amortisation.** A content cache by hash, prefetch, zstd, predicted echo in TTY mode, and session upgrade by re-exec. New measurements to add before phase 1 lands: keystroke-to-present in the tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file open. ## Still open (the coordinator's position in brackets) 1. Delete the frame wire at phase 3, leaving no remote attach until phase 5. [yes: it was never used daily] 2. The stream as its own framed protocol, or as a 9P file. [framed: one ordered stream with no per-read round trip; 9P stays for scripting and the relay] 3. Rollback by walker copy or by arena copy-on-write. [whichever measurement of the mutable state's size decides] 4. How the daemon starts: spawning itself or socket activation. [itself, by double fork; FreeBSD has no systemd. A user unit can be added if a terminal's scope ever kills it] 5. The join writer: a fork child, or a metadata copy serialised on a worker. [fork first, measure the page-table copy] ## Phase 3 notes Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`). Decisions taken at the forks, with the ponytail lens: - **Sessions are not the daemon's children; every one registers.** The design had the daemon spawn a terminal's session from the fd, cwd, env and argv the stub passes. That proved wrong here: - the user's terminals each run in a `9ns --mntgen` user and mount namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon spawned would run in whichever namespace the daemon was started from, its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits, umask and stdin (`pardes -`) would be the daemon's too, or have to be carried and re-execed; - startup would pay an IPC round trip and an exec of a 100+ MB binary; - the snapshot harness's idleness probe walks the launched process's tree, and a daemon-spawned session is outside it. So `pardes` in a terminal forks its session off itself before anything starts (`daemon.forkSession`): the child calls `setsid` and draws on the `/dev/tty` the parent opened; the parent stays as its stub. GUI windows and `--detach` sessions just register. The daemon spawns nothing, so it is no subreaper: a detached session is reparented to the system's reaper (systemd --user or init), which reaps it, as it reaps a crashed session's shells. Nothing dangles: the daemon lists every session (its connection) and holds every master, and exits a second after it holds neither. - **Starting the daemon.** The first session takes the flock, binds and listens itself and queues its own connection; `pardes --daemon` is then double-forked and exec'd onto that listener and lock (fds 3 and 4), by a terminal's stub after the session's fork, so no session waits for it: its hello sits queued until the daemon runs, and is not waited on (the daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s). With the spawn before the fork, a cold start measured +0.38 ms (+7.6%). The daemon's reader thread likewise starts after the first frame. Another starter waits on the lock; a stale socket is the lock holder's to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and the fds it carries. - **pty masters.** `forkShell` sends a dup; a host's close sends `close`; a session's end sends `quit` (its shells hang up); a master whose shell is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT handlers send `quit` too (one `send`, async-signal-safe), so only a session killed outright or crashed leaves shells behind. - **Adoption is a word, `Adopt`,** calling the host through a vtable entry (no Effect): the daemon hands over every orphaned master, and each becomes a pane through the `Tty` word, its spawn taking the pty instead of forking; a 1x1 size then the pane's makes it redraw. Adopting at startup was rejected: an `EDITOR='pardes --wait'` session would take the shells and hang them up when the editor exits. - **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and ends with the session's status (a forked stub `waitpid`s; an attaching one reads `x`). A HUP is the terminal's end: kept by a daemon, a session takes HUP on its signal thread (sigwait, beside WINCH) and detaches, as on EIO; HUP never ends a kept session, since the terminal's own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane (the terminal is raw, as before); a TSTP sent to the stub is ignored, since a stopped stub hands the terminal back under a session still drawing; a background start stops at a `tcsetattr` before the fork, as the old raw mode did. `--wait`, nested forwarding and `--tty` run before any of this and are unchanged. - **`--detach` is a session started detached:** the tty host with no terminal, the GUI with a hidden window closed at once. With the frames gone, `detached/server.zig` was a third host with nothing left to do that the other two could not. - **`Attach` (the word) is deleted:** it switched a session's own terminal to another session through the frames. - **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a session as before: in the terminal's own process, ended by its hangup, for a harness that kills by pid and closes the terminal. A GUI test window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`. - **Known gaps.** A forked session takes its stub's death, a SIGKILL too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and detaches; a `pardes --attach` stub is no parent of its session, and one killed with SIGKILL while its terminal stays open still leaves the session drawing there until the terminal closes or Detach. A reattached terminal keeps the first one's environment (TERM, COLORTERM); vaxis queries its capabilities again. A daemon killed outright leaves its sessions unregistered, and Detach then refuses. A daemon of an older protocol answers `version`, and a session then runs without one. A GUI started in a terminal's foreground stays that terminal's job. Measured (ReleaseFast, base main a7369739 against these changes, both live at once and their samples alternating A, B, A, B; A/A the base against itself; p50 ratio new/base): | | tty | A/A | GUI (test mode) | A/A | |---|---|---|---|---| | startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 | | startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | | | key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 | | key echoed by bash, to its frame | 1.016 | 1.005 | | | What is left of the tty's startup is the stub's fork (about 70 us), which the design cannot do without: the job the shell waits on must be able to end while the session runs on. Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout), and overran the stack, clobbering a neighbouring local in the forked child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one. ## FreeBSD notes Groundwork for phase 5's main test, a client here and the session on the FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and runs there. The GUI is not built for it, since frontends stay local. The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is the host's root shell; port 2222 is a git jail and is never touched. Everything of ours lives in `/root/pardes-test`, nothing outside it. - **Building.** Everything, the C deps included, cross-compiles here: ```sh export PATH=$HOME/.local/bin:$PATH E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P' $E zig build -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/bin/pardes-freebsd-x86_64 $E zig build test-install -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 # zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files ``` The `.15.0` matters: Zig's default FreeBSD (14.0) links `libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs the test-build programs instead of running them. A test binary takes no filter when it runs, so running one test alone means a rebuild with `-Dtest-filter=` (and `--prefix ` to keep the full set). - **Deploying.** `-F /dev/null` is required here, because /etc/ssh is owned by nobody: ```sh S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519' zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all scp $S p.zst root@65.109.81.103:/root/pardes-test/bin/pardes.zst ssh $S root@65.109.81.103 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst' ``` The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh ` (a build, PARDES_* stripped, logged there), `deploy.sh [name]` and `bx.sh ''` (a command run there with env.sh sourced). `sshcheck.py` drives a session through `ssh -tt` from a local pty, and `fwd9p.py` times 9P through `ssh -L`. - **The box's layout.** Under `/root/pardes-test`: - `bin/`: pardes, and `python3` linked to python3.12; - `run/`: the runtime directory (sockets, the daemon's lock); - `home/` and `tmp/`; - `src/`: the tracked tree, which the tests need: they read docs/typ, and make their temporary directories in `.zig-cache/tmp` under the working directory; - `test-bin/test/` and `logs/`; - `env.sh`, which points every path pardes uses there (HOME carries the XDG defaults): ```sh T=/root/pardes-test export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color} ``` - **Running.** - A session: `ssh -t $S root@65.109.81.103 '. /root/pardes-test/env.sh; pardes notes.txt'`, and `pardes --attach=` the same way. - The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`. Some tests name `/tmp/pardes-*` outright and leave seven empty directories there (`pardes-arg-save` and the like): rmdir them after. - The e2e scripts, from `src/`: `python3 -B test/daemon.py /root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way. - Kill every process you start there by its pid, and check `ps -axww` afterwards. - **What FreeBSD does differently.** Each item was found on the box, and each is fixed where pardes met it unless it says otherwise: - A write to a terminal that is gone fails with ENXIO, not EIO. std does not expect that errno, and a Debug build traced it to stderr, which is the same dead terminal, recursing until the stack ran out: a session whose ssh dropped hung. Tracing is off on FreeBSD (main.zig). - A signal whose action is to be ignored is dropped when it is sent, sigwait or not, so WINCH never reached the session's watcher. A handler that does nothing keeps it (tty.zig). - waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64 bits. With Linux's numbers a command pane's exit was never seen. - There is no /proc. A shell's directory and program come from sysctl instead (kern.proc.cwd, kern.proc.pathname). - `ttyTaken` answers taken for any foreground group but the shell's, so a nested shell counts as a job. - `signalSession` stays Linux-only, so Kill leaves a command's `&` jobs running there. - File watching uses FreeBSD 15's inotify: Linux's events, with its own IN_CLOEXEC and IN_NONBLOCK. - `shm_open` refuses O_NONBLOCK; its names are not files, so there is no FIFO to fear. A `write` does not grow a shared memory object. - A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a stat by path now comes first. - A Unix socket whose listen queue is full refuses a connect with ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep that judged by connecting could remove a live session's socket while its queue overflowed. A session's liveness is now its lock: it holds `.lock` flocked from before it binds until after it unlinks (9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they unlink only under it. A connect judges only a socket with no lock, one of a pardes from before the locks. The daemon's own start was already decided by its flock. - **The tests there.** The box runs them as root, with no zls, fish or fonts installed. - main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no zls), and two tests that make a file or directory unreadable, which root reads anyway. - panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43 passed. The small programs all pass but fonts (no fonts). - Off Linux, the parts of tests whose refusing path is Linux's /proc/1 are skipped, and so is the builtins JSON check, since the committed lists are Linux builds' (Tty9p included). - test/daemon.py passes there at 20 cycles, its killed stub included (procctl's death signal). One earlier run failed at cycle 3, one process short in its count (most likely ps -e failing to read an environment). - test/fs.py runs past its `pardes -` +Pager checks now, as far as the SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty after the first frame, FreeBSD's pty fills before the session's frame and reset are written, and the session waits in `ttyout` (a binary from before these fixes too). Open. - **At 207 ms, for the replicas.** - In today's model every key is a round trip. Measured through `ssh -tt` (min, median and max): - a key typed into a file was on screen in 213, 229 and 346 ms; - a key typed into a shell pane in 218, 219 and 224 ms. That is one RTT and about 15 ms. - 9P over `ssh -L` of the session's socket costs one RTT a message: 670 ms to connect, 1035 ms for a file read (walk, open, read, read, clunk), 830 ms for a ctl write. A replica cannot fetch its state file by file. - The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L` cannot forward it (tried: EPROTOTYPE here, connect failed there), and no fd would cross anyway. The replication connection has to be a stream of its own, as the 9P socket is, which the daemon hands on locally. - A dropped connection is a hangup: the session detaches, and `pardes --attach` takes it back, as after a Detach. - sun_path is 104 bytes there, not 108.