summaryrefslogtreecommitdiff
path: root/docs/next-daemon.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/next-daemon.md')
-rw-r--r--docs/next-daemon.md171
1 files changed, 171 insertions, 0 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
new file mode 100644
index 00000000..8bc9b38c
--- /dev/null
+++ b/docs/next-daemon.md
@@ -0,0 +1,171 @@
+# The daemon: one pardes per machine, many clients per session
+
+> I want to have a global pardes daemon ... holding file descriptors for
+> ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a
+> cpu+file server ... the main test: multiple clients to a single session,
+> and a client here and the session on freebsd and very deep and thorough
+> performance testing and optimizations to make it feel like its local and
+> snappy.
+
+Designed 2026-10-29 in discussion (the user, the coordinator, a survey
+agent). Nothing here is built yet. Paths are relative to the repo.
+
+## Decided by the user
+
+- **A deterministic core, replicated.** The session is the authority: it
+ orders every client's input and every host result, and only it performs
+ effects. Each client runs a full core replica, applies its own input at
+ once, and rolls back and replays when the order differs. Visible rollback
+ is acceptable.
+- **Kakoune-style clients.** Each client has its own view (layout, focus,
+ selections, scroll) over shared buffers and terminals. Registers are
+ session-wide. A terminal shown at two sizes is sized by its last typist.
+- **Performance order.** First, the regular tty and GUI get no slower.
+ Then the detached and remote clients reach that local performance, with
+ the session on the FreeBSD box included.
+- **A host step may show up one RTT late, but is never wrong.** This
+ covers Look, open and exec, as sam's hostlock does.
+- **Join is complete.** The joining client pays for it in time; the
+ authority and the other clients do not.
+- **One long-lived daemon per user per machine.** It supervises one child
+ process per session, postgres-style. Nothing is left dangling, and the
+ core is not de-globalised.
+
+## What today's seam is, and why it goes
+
+`src/detached/server.zig` is a third Host: it runs the core and the ptys
+in a single poll loop and sends cell frames to the frontends, at the
+smallest common grid across them.
+
+- **The frames are mostly not diffed.** Every layer's cells are re-sent on
+ each frame (wire.zig:734, :755).
+- **Several things never cross the wire.** Images, PDF pixels, rail ticks
+ and pane animations stay behind (surface.zig:483-509).
+- **Every keystroke is a round trip,** which no amount of optimisation can
+ make feel local at 30-80 ms.
+- **Each remote 9P operation dials afresh,** costing about 6 RTTs
+ (9p_io.zig:2361-2385).
+
+## The design
+
+- **Processes.**
+ - The daemon is subreaper, spawner, fd holder, registry and 9P root.
+ - Every session is a child of the daemon: a GUI window, a tty session
+ behind a stub, or a headless session.
+ - A session is always an in-process authority with its own local view,
+ so the regular path does not change. Replicas exist only for extra
+ clients or remote ones.
+- **The tty stub.**
+ - `pardes` in a terminal passes its terminal fd to the daemon
+ (SCM_RIGHTS), along with cwd, env and argv.
+ - The session child drives that fd directly. The stub forwards SIGWINCH
+ and exits with the session's status.
+ - Detach: the session drops the fd and the stub exits. No fork happens.
+- **GUI detach.** The window is destroyed and the process carries on
+ headless. Closing a window that has no other client attached quits it.
+- **Shells survive a session crash.** After each `forkShell`, the session
+ sends the daemon a dup of the pty master, so the shells do not get
+ SIGHUP. Being subreaper only decides who reaps them.
+- **The recorder.** It is null when no client is attached. When one is,
+ it appends what the authority consumed, in order:
+ - every step's host tape (the results of the ~30 `turn.yield()` sites);
+ - each `serveFs` mutation.
+
+ Turn order is log order, so the local path changes by one null check.
+- **The protocol.** One ordered stream: events, host tapes, digests and
+ terminal state. Framing, the hello and the Event codec are kept from
+ wire.zig; the frames go.
+- **Terminals cross as authoritative screen state,** not as pty bytes:
+ - metadata plus immutable row blobs, assigned in absolute terms;
+ - scrollback fetched on demand.
+
+ A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the
+ authority, and the local path keeps reading ghostty directly.
+- **The reflective walker.** One comptime `@typeInfo` walk gives:
+ - the digest;
+ - the complete dump, which also serves join, Restore and Handover;
+ - the rollback copy.
+
+ Fields are annotated local, authority-only or derived. A hash of the
+ type graph versions it. The hello carries a semantics hash (version,
+ features, grammars), and a mismatched build is refused.
+- **Rollback.**
+ - When the first speculative event goes pending, the replica copies what
+ speculation can write: views, buffers, registers, counters and terminal
+ metadata. Blobs are pinned rather than copied.
+ - On a divergence it restores that copy, then replays the confirmed
+ events and its own pending ones.
+ - The authority never rolls back.
+- **Join.** The authority forks a writer child at a step boundary; the
+ child streams the dump. The authority buffers the events after it for
+ the joiner.
+- **Remote.**
+ - `ssh -L` of the daemon's one socket.
+ - The daemon hands a replication connection's socket to its session.
+ - A remote daemon mounts as `/n/<host>`.
+
+## Determinism leaks (the survey's census)
+
+| Leak | Where | Fix |
+|---|---|---|
+| Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak |
+| 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order |
+| Host syscalls in a step | the ~30 `turn.yield()` sites | host tape |
+| Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done |
+| Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local |
+| Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ |
+| Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced |
+| getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state |
+| Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash |
+| ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals |
+
+## Phases
+
+Each phase ships on its own and keeps `zig build perf --base` level (with
+an A/A run). All 110 snapshots stay green throughout.
+
+1. **Determinism and the replay-divergence harness.**
+ - Work: the recorder and tape; the leak fixes, the render one first; the
+ semantics hash; the walker's digest.
+ - Check: replay each recorded log into two cores at different addresses
+ and one headless core.
+ - Gate: zero divergences on the snapshot scripts and on N monkey seeds.
+2. **A complete dump on the walker.** It replaces dump.zig for Restore and
+ Handover.
+ - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint.
+3. **The daemon and the tty stub.**
+ - Work: in-place detach and reattach; pty master dups; the frame wire
+ deleted.
+ - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves
+ its shells adoptable; no stale sockets after N cycles.
+4. **Views.** A second window on the same session, in one process.
+ - Gate: logs with several views replay without divergence.
+5. **Replicas.**
+ - Work: speculation, rollback, screen-state terminals, join, socket
+ handoff, `ssh -L`.
+ - Gates:
+ - a 30/80 ms delay relay keeps remote within noise of local;
+ - the rollback rate stays low under the fuzzer;
+ - a lagging client leaves the authority flat;
+ - a join costs the authority under one frame.
+6. **Remote amortisation.** A content cache by hash, prefetch, zstd,
+ predicted echo in TTY mode, and session upgrade by re-exec.
+
+New measurements to add before phase 1 lands: keystroke-to-present in the
+tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file
+open.
+
+## Still open (the coordinator's position in brackets)
+
+1. Delete the frame wire at phase 3, leaving no remote attach until
+ phase 5. [yes: it was never used daily]
+2. The stream as its own framed protocol, or as a 9P file. [framed: one
+ ordered stream with no per-read round trip; 9P stays for scripting and
+ the relay]
+3. Rollback by walker copy or by arena copy-on-write. [whichever
+ measurement of the mutable state's size decides]
+4. How the daemon starts: spawning itself or socket activation. [itself,
+ by double fork; FreeBSD has no systemd. A user unit can be added if a
+ terminal's scope ever kills it]
+5. The join writer: a fork child, or a metadata copy serialised on a
+ worker. [fork first, measure the page-table copy]