summaryrefslogtreecommitdiff
path: root/docs/next-daemon.md
blob: d164eced87020faac931f4704e9a809729da6b15 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
# The daemon: one pardes per machine, many clients per session

> I want to have a global pardes daemon ... holding file descriptors for
> ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a
> cpu+file server ... the main test: multiple clients to a single session,
> and a client here and the session on freebsd and very deep and thorough
> performance testing and optimizations to make it feel like its local and
> snappy.

Designed 2026-10-29 in discussion (the user, the coordinator, a survey
agent). Nothing here is built yet. Paths are relative to the repo.

## Decided by the user

- **A deterministic core, replicated.** The session is the authority: it
  orders every client's input and every host result, and only it performs
  effects. Each client runs a full core replica, applies its own input at
  once, and rolls back and replays when the order differs. Visible rollback
  is acceptable.
- **Kakoune-style clients.** Each client has its own view (layout, focus,
  selections, scroll) over shared buffers and terminals. Registers are
  session-wide. A terminal shown at two sizes is sized by its last typist.
- **Performance order.** First, the regular tty and GUI get no slower.
  Then the detached and remote clients reach that local performance, with
  the session on the FreeBSD box included.
- **A host step may show up one RTT late, but is never wrong.** This
  covers Look, open and exec, as sam's hostlock does.
- **Join is complete.** The joining client pays for it in time; the
  authority and the other clients do not.
- **One long-lived daemon per user per machine.** It supervises one child
  process per session, postgres-style. Nothing is left dangling, and the
  core is not de-globalised.

## What today's seam is, and why it goes

`src/detached/server.zig` is a third Host: it runs the core and the ptys
in a single poll loop and sends cell frames to the frontends, at the
smallest common grid across them.

- **The frames are mostly not diffed.** Every layer's cells are re-sent on
  each frame (wire.zig:734, :755).
- **Several things never cross the wire.** Images, PDF pixels, rail ticks
  and pane animations stay behind (surface.zig:483-509).
- **Every keystroke is a round trip,** which no amount of optimisation can
  make feel local at 30-80 ms.
- **Each remote 9P operation dials afresh,** costing about 6 RTTs
  (9p_io.zig:2361-2385).

## The design

- **Processes.**
  - The daemon is subreaper, spawner, fd holder, registry and 9P root.
  - Every session is a child of the daemon: a GUI window, a tty session
    behind a stub, or a headless session.
  - A session is always an in-process authority with its own local view,
    so the regular path does not change. Replicas exist only for extra
    clients or remote ones.
- **The tty stub.**
  - `pardes` in a terminal passes its terminal fd to the daemon
    (SCM_RIGHTS), along with cwd, env and argv.
  - The session child drives that fd directly. The stub forwards SIGWINCH
    and exits with the session's status.
  - Detach: the session drops the fd and the stub exits. No fork happens.
- **GUI detach.** The window is destroyed and the process carries on
  headless. Closing a window that has no other client attached quits it.
- **Shells survive a session crash.** After each `forkShell`, the session
  sends the daemon a dup of the pty master, so the shells do not get
  SIGHUP. Being subreaper only decides who reaps them.
- **The recorder.** It is null when no client is attached. When one is,
  it appends what the authority consumed, in order:
  - every step's host tape (the results of the ~30 `turn.yield()` sites);
  - each `serveFs` mutation.

  Turn order is log order, so the local path changes by one null check.
- **The protocol.** One ordered stream: events, host tapes, digests and
  terminal state. Framing, the hello and the Event codec are kept from
  wire.zig; the frames go.
- **Terminals cross as authoritative screen state,** not as pty bytes:
  - metadata plus immutable row blobs, assigned in absolute terms;
  - scrollback fetched on demand.

  A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the
  authority, and the local path keeps reading ghostty directly.
- **The reflective walker.** One comptime `@typeInfo` walk gives:
  - the digest;
  - the complete dump, which also serves join, Restore and Handover;
  - the rollback copy.

  Fields are annotated local, authority-only or derived. A hash of the
  type graph versions it. The hello carries a semantics hash (version,
  features, grammars), and a mismatched build is refused.
- **Rollback.**
  - When the first speculative event goes pending, the replica copies what
    speculation can write: views, buffers, registers, counters and terminal
    metadata. Blobs are pinned rather than copied.
  - On a divergence it restores that copy, then replays the confirmed
    events and its own pending ones.
  - The authority never rolls back.
- **Join.** The authority forks a writer child at a step boundary; the
  child streams the dump. The authority buffers the events after it for
  the joiner.
- **Remote.**
  - `ssh -L` of the daemon's one socket.
  - The daemon hands a replication connection's socket to its session.
  - A remote daemon mounts as `/n/<host>`.

## Determinism leaks (the survey's census)

| Leak | Where | Fix |
|---|---|---|
| Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak |
| 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order |
| Host syscalls in a step | the ~30 `turn.yield()` sites | host tape |
| Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done |
| Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local |
| Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ |
| Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced |
| getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state |
| Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash |
| ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals |

## Phases

Each phase ships on its own and keeps `zig build perf --base` level (with
an A/A run). All 110 snapshots stay green throughout.

1. **Determinism and the replay-divergence harness.**
   - Work: the recorder and tape; the leak fixes, the render one first; the
     semantics hash; the walker's digest.
   - Check: replay each recorded log into two cores at different addresses
     and one headless core.
   - Gate: zero divergences on the snapshot scripts and on N monkey seeds.
2. **A complete dump on the walker.** It replaces dump.zig for Restore and
   Handover.
   - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint.
3. **The daemon and the tty stub.**
   - Work: in-place detach and reattach; pty master dups; the frame wire
     deleted.
   - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves
     its shells adoptable; no stale sockets after N cycles.
4. **Views.** A second window on the same session, in one process.
   - Gate: logs with several views replay without divergence.
5. **Replicas.**
   - Work: speculation, rollback, screen-state terminals, join, socket
     handoff, `ssh -L`.
   - Gates:
     - a 30/80 ms delay relay keeps remote within noise of local;
     - the rollback rate stays low under the fuzzer;
     - a lagging client leaves the authority flat;
     - a join costs the authority under one frame.
6. **Remote amortisation.** A content cache by hash, prefetch, zstd,
   predicted echo in TTY mode, and session upgrade by re-exec.

New measurements to add before phase 1 lands: keystroke-to-present in the
tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file
open.

## Still open (the coordinator's position in brackets)

1. Delete the frame wire at phase 3, leaving no remote attach until
   phase 5. [yes: it was never used daily]
2. The stream as its own framed protocol, or as a 9P file. [framed: one
   ordered stream with no per-read round trip; 9P stays for scripting and
   the relay]
3. Rollback by walker copy or by arena copy-on-write. [whichever
   measurement of the mutable state's size decides]
4. How the daemon starts: spawning itself or socket activation. [itself,
   by double fork; FreeBSD has no systemd. A user unit can be added if a
   terminal's scope ever kills it]
5. The join writer: a fork child, or a metadata copy serialised on a
   worker. [fork first, measure the page-table copy]

## Phase 3 notes

Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`).
Decisions taken at the forks, with the ponytail lens:

- **Sessions are not the daemon's children; every one registers.** The
  design had the daemon spawn a terminal's session from the fd, cwd, env
  and argv the stub passes. That proved wrong here:
  - the user's terminals each run in a `9ns --mntgen` user and mount
    namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon
    spawned would run in whichever namespace the daemon was started from,
    its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits,
    umask and stdin (`pardes -`) would be the daemon's too, or have to be
    carried and re-execed;
  - startup would pay an IPC round trip and an exec of a 100+ MB binary;
  - the snapshot harness's idleness probe walks the launched process's
    tree, and a daemon-spawned session is outside it.

  So `pardes` in a terminal forks its session off itself before anything
  starts (`daemon.forkSession`): the child calls `setsid` and draws on the
  `/dev/tty` the parent opened; the parent stays as its stub. GUI windows
  and `--detach` sessions just register. The daemon spawns nothing, so it
  is no subreaper: a detached session is reparented to the system's reaper
  (systemd --user or init), which reaps it, as it reaps a crashed
  session's shells. Nothing dangles: the daemon lists every session (its
  connection) and holds every master, and exits a second after it holds
  neither.
- **Starting the daemon.** The first session takes the flock, binds and
  listens itself and queues its own connection; `pardes --daemon` is then
  double-forked and exec'd onto that listener and lock (fds 3 and 4), by
  a terminal's stub after the session's fork, so no session waits for it:
  its hello sits queued until the daemon runs, and is not waited on (the
  daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s).
  With the spawn before the fork, a cold start measured +0.38 ms (+7.6%).
  The daemon's reader thread likewise starts after the first frame.
  Another starter waits on the lock; a stale socket is the lock holder's
  to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and
  the fds it carries.
- **pty masters.** `forkShell` sends a dup; a host's close sends `close`;
  a session's end sends `quit` (its shells hang up); a master whose shell
  is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT
  handlers send `quit` too (one `send`, async-signal-safe), so only a
  session killed outright or crashed leaves shells behind.
- **Adoption is a word, `Adopt`,** calling the host through a vtable entry
  (no Effect): the daemon hands over every orphaned master, and each
  becomes a pane through the `Tty` word, its spawn taking the pty instead
  of forking; a 1x1 size then the pane's makes it redraw. Adopting at
  startup was rejected: an `EDITOR='pardes --wait'` session would take the
  shells and hang them up when the editor exits.
- **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and
  ends with the session's status (a forked stub `waitpid`s; an attaching
  one reads `x<code>`). A HUP is the terminal's end: kept by a daemon, a
  session takes HUP on its signal thread (sigwait, beside WINCH) and
  detaches, as on EIO; HUP never ends a kept session, since the terminal's
  own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane
  (the terminal is raw, as before); a TSTP sent to the stub is ignored,
  since a stopped stub hands the terminal back under a session still
  drawing; a background start stops at a `tcsetattr` before the fork, as
  the old raw mode did. `--wait`, nested forwarding and `--tty` run before
  any of this and are unchanged.
- **`--detach` is a session started detached:** the tty host with no
  terminal, the GUI with a hidden window closed at once. With the frames
  gone, `detached/server.zig` was a third host with nothing left to do
  that the other two could not.
- **`Attach` (the word) is deleted:** it switched a session's own terminal
  to another session through the frames.
- **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a
  session as before: in the terminal's own process, ended by its hangup,
  for a harness that kills by pid and closes the terminal. A GUI test
  window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`.
- **Known gaps.** A stub killed with SIGKILL while its terminal stays open
  leaves the session drawing there until the terminal closes or Detach. A
  reattached terminal keeps the first one's environment (TERM, COLORTERM);
  vaxis queries its capabilities again. A daemon killed outright leaves
  its sessions unregistered, and Detach then refuses. A daemon of an older
  protocol answers `version`, and a session then runs without one. A
  GUI started in a terminal's foreground stays that terminal's job.

Measured (ReleaseFast, base main a7369739 against these changes, both
live at once and their samples alternating A, B, A, B; A/A the base
against itself; p50 ratio new/base):

| | tty | A/A | GUI (test mode) | A/A |
|---|---|---|---|---|
| startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 |
| startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | |
| key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 |
| key echoed by bash, to its frame | 1.016 | 1.005 | | |

What is left of the tty's startup is the stub's fork (about 70 us), which
the design cannot do without: the job the shell waits on must be able to
end while the session runs on.

Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct
termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout),
and overran the stack, clobbering a neighbouring local in the forked
child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one.