1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
|
# The daemon: one pardes per machine, many clients per session
> I want to have a global pardes daemon ... holding file descriptors for
> ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a
> cpu+file server ... the main test: multiple clients to a single session,
> and a client here and the session on freebsd and very deep and thorough
> performance testing and optimizations to make it feel like its local and
> snappy.
Designed 2026-10-29 in discussion (the user, the coordinator, a survey
agent). Nothing here is built yet. Paths are relative to the repo.
## Decided by the user
- **A deterministic core, replicated.** The session is the authority: it
orders every client's input and every host result, and only it performs
effects. Each client runs a full core replica, applies its own input at
once, and rolls back and replays when the order differs. Visible rollback
is acceptable.
- **Kakoune-style clients.** Each client has its own view (layout, focus,
selections, scroll) over shared buffers and terminals. Registers are
session-wide. A terminal shown at two sizes is sized by its last typist.
- **Performance order.** First, the regular tty and GUI get no slower.
Then the detached and remote clients reach that local performance, with
the session on the FreeBSD box included.
- **A host step may show up one RTT late, but is never wrong.** This
covers Look, open and exec, as sam's hostlock does.
- **Join is complete.** The joining client pays for it in time; the
authority and the other clients do not.
- **One long-lived daemon per user per machine.** It supervises one child
process per session, postgres-style. Nothing is left dangling, and the
core is not de-globalised.
## What today's seam is, and why it goes
`src/detached/server.zig` is a third Host: it runs the core and the ptys
in a single poll loop and sends cell frames to the frontends, at the
smallest common grid across them.
- **The frames are mostly not diffed.** Every layer's cells are re-sent on
each frame (wire.zig:734, :755).
- **Several things never cross the wire.** Images, PDF pixels, rail ticks
and pane animations stay behind (surface.zig:483-509).
- **Every keystroke is a round trip,** which no amount of optimisation can
make feel local at 30-80 ms.
- **Each remote 9P operation dials afresh,** costing about 6 RTTs
(9p_io.zig:2361-2385).
## The design
- **Processes.**
- The daemon is subreaper, spawner, fd holder, registry and 9P root.
- Every session is a child of the daemon: a GUI window, a tty session
behind a stub, or a headless session.
- A session is always an in-process authority with its own local view,
so the regular path does not change. Replicas exist only for extra
clients or remote ones.
- **The tty stub.**
- `pardes` in a terminal passes its terminal fd to the daemon
(SCM_RIGHTS), along with cwd, env and argv.
- The session child drives that fd directly. The stub forwards SIGWINCH
and exits with the session's status.
- Detach: the session drops the fd and the stub exits. No fork happens.
- **GUI detach.** The window is destroyed and the process carries on
headless. Closing a window that has no other client attached quits it.
- **Shells survive a session crash.** After each `forkShell`, the session
sends the daemon a dup of the pty master, so the shells do not get
SIGHUP. Being subreaper only decides who reaps them.
- **The recorder.** It is null when no client is attached. When one is,
it appends what the authority consumed, in order:
- every step's host tape (the results of the ~30 `turn.yield()` sites);
- each `serveFs` mutation.
Turn order is log order, so the local path changes by one null check.
- **The protocol.** One ordered stream: events, host tapes, digests and
terminal state. Framing, the hello and the Event codec are kept from
wire.zig; the frames go.
- **Terminals cross as authoritative screen state,** not as pty bytes:
- metadata plus immutable row blobs, assigned in absolute terms;
- scrollback fetched on demand.
A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the
authority, and the local path keeps reading ghostty directly.
- **The reflective walker.** One comptime `@typeInfo` walk gives:
- the digest;
- the complete dump, which also serves join, Restore and Handover;
- the rollback copy.
Fields are annotated local, authority-only or derived. A hash of the
type graph versions it. The hello carries a semantics hash (version,
features, grammars), and a mismatched build is refused.
- **Rollback.**
- When the first speculative event goes pending, the replica copies what
speculation can write: views, buffers, registers, counters and terminal
metadata. Blobs are pinned rather than copied.
- On a divergence it restores that copy, then replays the confirmed
events and its own pending ones.
- The authority never rolls back.
- **Join.** The authority forks a writer child at a step boundary; the
child streams the dump. The authority buffers the events after it for
the joiner.
- **Remote.**
- `ssh -L` of the daemon's one socket.
- The daemon hands a replication connection's socket to its session.
- A remote daemon mounts as `/n/<host>`.
## Determinism leaks (the survey's census)
| Leak | Where | Fix |
|---|---|---|
| Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak |
| 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order |
| Host syscalls in a step | the ~30 `turn.yield()` sites | host tape |
| Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done |
| Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local |
| Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ |
| Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced |
| getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state |
| Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash |
| ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals |
## Phases
Each phase ships on its own and keeps `zig build perf --base` level (with
an A/A run). All 110 snapshots stay green throughout.
1. **Determinism and the replay-divergence harness.**
- Work: the recorder and tape; the leak fixes, the render one first; the
semantics hash; the walker's digest.
- Check: replay each recorded log into two cores at different addresses
and one headless core.
- Gate: zero divergences on the snapshot scripts and on N monkey seeds.
2. **A complete dump on the walker.** It replaces dump.zig for Restore and
Handover.
- Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint.
3. **The daemon and the tty stub.**
- Work: in-place detach and reattach; pty master dups; the frame wire
deleted.
- Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves
its shells adoptable; no stale sockets after N cycles.
4. **Views.** A second window on the same session, in one process.
- Gate: logs with several views replay without divergence.
5. **Replicas.**
- Work: speculation, rollback, screen-state terminals, join, socket
handoff, `ssh -L`.
- Gates:
- a 30/80 ms delay relay keeps remote within noise of local;
- the rollback rate stays low under the fuzzer;
- a lagging client leaves the authority flat;
- a join costs the authority under one frame.
6. **Remote amortisation.** A content cache by hash, prefetch, zstd,
predicted echo in TTY mode, and session upgrade by re-exec.
New measurements to add before phase 1 lands: keystroke-to-present in the
tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file
open.
## Still open (the coordinator's position in brackets)
1. Delete the frame wire at phase 3, leaving no remote attach until
phase 5. [yes: it was never used daily]
2. The stream as its own framed protocol, or as a 9P file. [framed: one
ordered stream with no per-read round trip; 9P stays for scripting and
the relay]
3. Rollback by walker copy or by arena copy-on-write. [whichever
measurement of the mutable state's size decides]
4. How the daemon starts: spawning itself or socket activation. [itself,
by double fork; FreeBSD has no systemd. A user unit can be added if a
terminal's scope ever kills it]
5. The join writer: a fork child, or a metadata copy serialised on a
worker. [fork first, measure the page-table copy]
## Phase 3 notes
Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`).
Decisions taken at the forks, with the ponytail lens:
- **Sessions are not the daemon's children; every one registers.** The
design had the daemon spawn a terminal's session from the fd, cwd, env
and argv the stub passes. That proved wrong here:
- the user's terminals each run in a `9ns --mntgen` user and mount
namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon
spawned would run in whichever namespace the daemon was started from,
its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits,
umask and stdin (`pardes -`) would be the daemon's too, or have to be
carried and re-execed;
- startup would pay an IPC round trip and an exec of a 100+ MB binary;
- the snapshot harness's idleness probe walks the launched process's
tree, and a daemon-spawned session is outside it.
So `pardes` in a terminal forks its session off itself before anything
starts (`daemon.forkSession`): the child calls `setsid` and draws on the
`/dev/tty` the parent opened; the parent stays as its stub. GUI windows
and `--detach` sessions just register. The daemon spawns nothing, so it
is no subreaper: a detached session is reparented to the system's reaper
(systemd --user or init), which reaps it, as it reaps a crashed
session's shells. Nothing dangles: the daemon lists every session (its
connection) and holds every master, and exits a second after it holds
neither.
- **Starting the daemon.** The first session takes the flock, binds and
listens itself and queues its own connection; `pardes --daemon` is then
double-forked and exec'd onto that listener and lock (fds 3 and 4), by
a terminal's stub after the session's fork, so no session waits for it:
its hello sits queued until the daemon runs, and is not waited on (the
daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s).
With the spawn before the fork, a cold start measured +0.38 ms (+7.6%).
The daemon's reader thread likewise starts after the first frame.
Another starter waits on the lock; a stale socket is the lock holder's
to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and
the fds it carries.
- **pty masters.** `forkShell` sends a dup; a host's close sends `close`;
a session's end sends `quit` (its shells hang up); a master whose shell
is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT
handlers send `quit` too (one `send`, async-signal-safe), so only a
session killed outright or crashed leaves shells behind.
- **Adoption is a word, `Adopt`,** calling the host through a vtable entry
(no Effect): the daemon hands over every orphaned master, and each
becomes a pane through the `Tty` word, its spawn taking the pty instead
of forking; a 1x1 size then the pane's makes it redraw. Adopting at
startup was rejected: an `EDITOR='pardes --wait'` session would take the
shells and hang them up when the editor exits.
- **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and
ends with the session's status (a forked stub `waitpid`s; an attaching
one reads `x<code>`). A HUP is the terminal's end: kept by a daemon, a
session takes HUP on its signal thread (sigwait, beside WINCH) and
detaches, as on EIO; HUP never ends a kept session, since the terminal's
own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane
(the terminal is raw, as before); a TSTP sent to the stub is ignored,
since a stopped stub hands the terminal back under a session still
drawing; a background start stops at a `tcsetattr` before the fork, as
the old raw mode did. `--wait`, nested forwarding and `--tty` run before
any of this and are unchanged.
- **`--detach` is a session started detached:** the tty host with no
terminal, the GUI with a hidden window closed at once. With the frames
gone, `detached/server.zig` was a third host with nothing left to do
that the other two could not.
- **`Attach` (the word) is deleted:** it switched a session's own terminal
to another session through the frames.
- **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a
session as before: in the terminal's own process, ended by its hangup,
for a harness that kills by pid and closes the terminal. A GUI test
window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`.
- **Known gaps.** A forked session takes its stub's death, a SIGKILL
too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and
detaches; a `pardes --attach` stub is no parent of its session, and one
killed with SIGKILL while its terminal stays open still leaves the
session drawing there until the terminal closes or Detach. A
reattached terminal keeps the first one's environment (TERM, COLORTERM);
vaxis queries its capabilities again. A daemon killed outright leaves
its sessions unregistered, and Detach then refuses. A daemon of an older
protocol answers `version`, and a session then runs without one. A
GUI started in a terminal's foreground stays that terminal's job.
Measured (ReleaseFast, base main a7369739 against these changes, both
live at once and their samples alternating A, B, A, B; A/A the base
against itself; p50 ratio new/base):
| | tty | A/A | GUI (test mode) | A/A |
|---|---|---|---|---|
| startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 |
| startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | |
| key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 |
| key echoed by bash, to its frame | 1.016 | 1.005 | | |
What is left of the tty's startup is the stub's fork (about 70 us), which
the design cannot do without: the job the shell waits on must be able to
end while the session runs on.
Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct
termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout),
and overran the stack, clobbering a neighbouring local in the forked
child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one.
## FreeBSD notes
Groundwork for phase 5's main test, a client here and the session on the
FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and
runs there. The GUI is not built for it, since frontends stay local.
The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is
the host's root shell; port 2222 is a git jail and is never touched.
Everything of ours lives in `/root/pardes-test`, nothing outside it.
- **Building.** Everything, the C deps included, cross-compiles here:
```sh
export PATH=$HOME/.local/bin:$PATH
E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P'
$E zig build -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1
# zig-out/bin/pardes-freebsd-x86_64
$E zig build test-install -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1
# zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files
```
The `.15.0` matters: Zig's default FreeBSD (14.0) links
`libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs
the test-build programs instead of running them. A test binary takes
no filter when it runs, so running one test alone means a rebuild
with `-Dtest-filter=<name>` (and `--prefix <dir>` to keep the full set).
- **Deploying.** `-F /dev/null` is required here, because /etc/ssh is
owned by nobody:
```sh
S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519'
zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all
scp $S p.zst [email protected]:/root/pardes-test/bin/pardes.zst
ssh $S [email protected] 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst'
```
The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh <log>
<args>` (a build, PARDES_* stripped, logged there), `deploy.sh <file>
[name]` and `bx.sh '<command>'` (a command run there with env.sh
sourced). `sshcheck.py` drives a session through `ssh -tt` from a
local pty, and `fwd9p.py` times 9P through `ssh -L`.
- **The box's layout.** Under `/root/pardes-test`:
- `bin/`: pardes, and `python3` linked to python3.12;
- `run/`: the runtime directory (sockets, the daemon's lock);
- `home/` and `tmp/`;
- `src/`: the tracked tree, which the tests need: they read docs/typ,
and make their temporary directories in `.zig-cache/tmp` under the
working directory;
- `test-bin/test/` and `logs/`;
- `env.sh`, which points every path pardes uses there (HOME carries
the XDG defaults):
```sh
T=/root/pardes-test
export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp
export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color}
```
- **Running.**
- A session: `ssh -t $S [email protected] '. /root/pardes-test/env.sh;
pardes notes.txt'`, and `pardes --attach=<name>` the same way.
- The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`.
Some tests name `/tmp/pardes-*` outright and leave seven empty
directories there (`pardes-arg-save` and the like): rmdir them after.
- The e2e scripts, from `src/`: `python3 -B test/daemon.py
/root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way.
- Kill every process you start there by its pid, and check `ps -axww`
afterwards.
- **What FreeBSD does differently.** Each item was found on the box, and
each is fixed where pardes met it unless it says otherwise:
- A write to a terminal that is gone fails with ENXIO, not EIO. std
does not expect that errno, and a Debug build traced it to stderr,
which is the same dead terminal, recursing until the stack ran out:
a session whose ssh dropped hung. Tracing is off on FreeBSD
(main.zig).
- A signal whose action is to be ignored is dropped when it is sent,
sigwait or not, so WINCH never reached the session's watcher. A
handler that does nothing keeps it (tty.zig).
- waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64
bits. With Linux's numbers a command pane's exit was never seen.
- There is no /proc. A shell's directory and program come from sysctl
instead (kern.proc.cwd, kern.proc.pathname).
- `ttyTaken` answers taken for any foreground group but the
shell's, so a nested shell counts as a job.
- `signalSession` stays Linux-only, so Kill leaves a command's `&`
jobs running there.
- File watching uses FreeBSD 15's inotify: Linux's events, with its
own IN_CLOEXEC and IN_NONBLOCK.
- `shm_open` refuses O_NONBLOCK; its names are not files, so there is
no FIFO to fear. A `write` does not grow a shared memory object.
- A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a
stat by path now comes first.
- A Unix socket whose listen queue is full refuses a connect with
ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep
that judged by connecting could remove a live session's socket while
its queue overflowed. A session's liveness is now its lock: it holds
`<socket>.lock` flocked from before it binds until after it unlinks
(9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they
unlink only under it. A connect judges only a socket with no lock, one
of a pardes from before the locks. The daemon's own start was already
decided by its flock.
- **The tests there.** The box runs them as root, with no zls, fish or
fonts installed.
- main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no
zls), and two tests that make a file or directory unreadable, which
root reads anyway.
- panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43
passed. The small programs all pass but fonts (no fonts).
- Off Linux, the parts of tests whose refusing path is Linux's
/proc/1 are skipped, and so is the builtins JSON check, since the
committed lists are Linux builds' (Tty9p included).
- test/daemon.py passes there at 20 cycles, its killed stub included
(procctl's death signal). One earlier run failed at
cycle 3, one process short in its count (most likely ps -e failing to
read an environment).
- test/fs.py runs past its `pardes -` +Pager checks now, as far as the
SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty
after the first frame, FreeBSD's pty fills before the session's frame
and reset are written, and the session waits in `ttyout` (a binary
from before these fixes too). Open.
- **At 207 ms, for the replicas.**
- In today's model every key is a round trip. Measured through `ssh
-tt` (min, median and max):
- a key typed into a file was on screen in 213, 229 and 346 ms;
- a key typed into a shell pane in 218, 219 and 224 ms.
That is one RTT and about 15 ms.
- 9P over `ssh -L` of the session's socket costs one RTT a message:
670 ms to connect, 1035 ms for a file read (walk, open, read, read,
clunk), 830 ms for a ctl write. A replica cannot fetch its state
file by file.
- The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L`
cannot forward it (tried: EPROTOTYPE here, connect failed there), and
no fd would cross anyway. The replication connection has to be a
stream of its own, as the 9P socket is, which the daemon hands on
locally.
- A dropped connection is a hangup: the session detaches, and
`pardes --attach` takes it back, as after a Detach.
- sun_path is 104 bytes there, not 108.
|