summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/next-daemon.md373
1 files changed, 372 insertions, 1 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
index 5039c81e..f6400770 100644
--- a/docs/next-daemon.md
+++ b/docs/next-daemon.md
@@ -1870,6 +1870,376 @@ Follow-ups (2026-10-29):
p99 0.27 ms (0.22 before, within the run's noise); none diverged, none
unequal, no delta wrong.
+## Phase 5 part B notes
+
+Built 2026-10-29 and 30: the stream over the replica socket (src/replica.zig
+`Peers`, and its client half, `dial` and `readStream`; detached/wire.zig
+version 15), the client process (`pardes --attach=<socket>:<name>` and
+`--attach=<host>:<name>`, tty.zig `remote`), the session's side of it in
+the tty host, rterm's screen-state terminals and the echo agent's
+predicted echo carried by it, a client's rejoin, and test/daemon.py's
+replica clients. Decisions taken at the forks, with the ponytail lens:
+
+- **A client's input is a unit of its own, settled by `step`.** On the
+ authority, at the step boundary after each pump (`Peers.boundary`), each
+ input that came is `toView(its view)`, `update(it)` and `step()`: the
+ middle of a pump (queued inputs and effects), with no wait, no clock
+ read and no frame. The client's speculation is the same three calls, so
+ the two tapes read alike and the fast path takes the unit as it came.
+ With the pump in its place (part A's harness) every speculation stopped
+ at the tty host's `wait_input` and read the clock in ns.
+- **The stream.** Framed as wire.zig frames everything, a tag and a u32le
+ length:
+ - the session's: 0x01 welcome (pid, name, the client's id and view, the
+ dump's length, the session's offset from UTC, its digest as dumped
+ and its build's semantics), 0x02 refused (words), 0x04 a piece of the
+ dump (1 MiB), 0x05 a unit (origin, seq, the authority's realtime and
+ monotonic clocks when it was made, its tape), 0x06 a terminal's
+ message (rterm's: a delta or rows), 0x07 a check's answer;
+ - the client's: its inputs as the existing event tags (their seq is the
+ n-th on the connection), 0x04 a terminal's message (rterm's: an
+ acknowledgement, rows asked, a reset), 0x05 check.
+
+ A unit carries one digest, at its end (`unit_digests`); under screen
+ state it is one word (`whole_digest`, PARDES_REPLICA_TRACE for one a
+ field). Blobs are hashed once and folded in as a word (main's
+ walk.zig), so a digest a unit is cheap. The draft's lengths-only digest
+ went with that.
+- **Terminals cross as screen state, in rterm's order.** The session's
+ Peers keep a `Screens`, a peer each. At each boundary: the rows the
+ pump's steps read that a client holds as holes (`fill`, then `filled`),
+ the pump's unit, then the deltas (`flush`). Before each client input's
+ step its deltas again, after it the rows it read, then its unit. A
+ joiner's terminals start afresh after its dump. The client applies a
+ delta or rows in stream order, acknowledges, and asks for the history
+ rows a frame shows that it lacks. Units carry no pty bytes.
+ - **A wait takes one step of a person's input, and none after a pty's
+ output** (tty.zig `split`), while clients replicate: a step reads the
+ terminals as the deltas after each unit bring them, and in a pump that
+ took a pty's output and then a key, the key would read on the session
+ what the client has only after the unit. The next is held for the
+ next wait. Only while a client is attached.
+ - **A step that reads a terminal's prompt after its output is taped**
+ (tape.zig `.prompt`): the first prompt's greeting (`ls`, after a pty
+ resize) and a command waiting for the prompt. Found at once by a
+ client opening a shell of its own: the session's layout after the
+ shell's first output saw the prompt, and the client's replay, its
+ emulator a delta behind, did not.
+ - A dump-loaded client's emulators are on the C heap, as
+ `terminal.create` makes a screen replica's (walk.zig `.alloc` of a
+ pane's `terminal`, dump.zig `terminal_gpa`).
+- **Predicted echo is the echo agent's** (src/echo.zig), wired as its
+ notes say: the client's core points at its replica's `echo`, a frame
+ judges it before drawing, and a wake comes a frame later while
+ predictions are out. PARDES_NO_ECHO turns it off.
+- **Join, at a step boundary, the step paying for the mutable state
+ alone.** The view is opened and the session dumped without its blobs
+ (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted
+ where they go) and without its terminals' emulators (their size: the
+ first delta brings each). The client's writer thread writes the
+ welcome, then the dump with the blobs put back where they go: what the
+ client reads is `dumpBytes`' dump byte for byte but for the terminals
+ (a test splices one of a session without any and compares), and the
+ client is unchanged. Each blob is pinned until it is
+ written (memory.zig `pin`: its free is put off; a blob is never written
+ in place and is freed raw). The texts the session never digested (it
+ records nothing until a client comes) would be hashed by the join's
+ first digest, 250 MB for a 100 MB file and its undo: they are hashed
+ first on a thread of their own (walk.zig `unhashed`, `makeWords`),
+ pinned meanwhile, and the join waits a boundary for them; a second
+ round joins whatever is left.
+- **A joiner checks its load.** The welcome carries the session's digest
+ as dumped; a client whose load differs says so and ends. The first unit
+ after a join says the host and its writes again (`tape.resync`), which a
+ core made from a dump has not. Found on the way: the dump's reader took
+ the first unit with the dump's last piece and dropped it (the client
+ diverged at its first digest about one join in three); what it read past
+ the dump is the stream's now (`Framer`).
+- **A diverged client joins again**, the same join as the first: its link
+ ended, the session dialled again, its dump loaded, its view's size sent,
+ in the same terminal, and a line over its frame until its next key says
+ why ("joined again: this client diverged from the session (...)"). It
+ gets a new view: the old one went with its link. PARDES_TEST_DIVERGE=<n>
+ puts a client's core off the session's (a digested counter) once quiet
+ after its n-th unit. A divergence shows only at a unit the client
+ replays (another's input, or its own it did not apply): the fast path
+ compares records, not digests.
+- **An Upgrade lets its clients go in words** (`Peers.refuseAll`, before
+ the dump): each is sent `refused` ("the session is upgrading to another
+ build of pardes") after what it was sent, for up to 200 ms, its view
+ closed. The client says it in its line and dials again, every 100 ms
+ for 20 s (the exec, the load, the daemon's own exec). A daemon's
+ refusal for another build, or a welcome whose semantics are not its
+ own (the daemon judged the hello by the hash it last heard, which an
+ upgrade under way may not have said yet), ends it with "session
+ upgraded to a newer pardes: upgrade this client too". "A newer pardes
+ is installed" is now a Host call the core makes in its focus step
+ (`newer_build`), so a session's clients take the line from the tape.
+- **The session's clock, guessed.** A unit carries the authority's clocks
+ when it was made; a speculation's clock reads answer the last unit's
+ reading plus the time since it came plus the client's round trip
+ (`tape.Guess`): a file's mtime (realtime seconds), a message's time of
+ day (in the session's zone, its offset in the welcome) and a message's
+ monotonic ms. Never the client's own wall clock.
+- **Mostly no rollback for the session's own pumps.** Every client input
+ woke the session's pump before its own unit, and that pump advanced the
+ clock: a foreign unit, a rollback a key. Two changes: a pump woken only
+ by a replica's post reads no new time (`mail_only`: the tty's `now`
+ answers the core's own), and a replica keeps what it applied over a
+ unit of another's that ends on the digest the last one ended on
+ (`neutral`; such a unit is replayed when the copy is next put back). A
+ unit that says what the host wrote, or which host it is, is never
+ neutral.
+- **The copy is taken while idle.** The client takes its rollback copy
+ after drawing, when nothing is queued (`Replica.ready`), and lets it go
+ when a unit comes first: a key no longer pays a megabyte's copy (memcpy
+ was a third of the client's profile). The epoch's and the copy's maps
+ hash an address by a multiply, not Wyhash (the other sixth).
+- **Each client a view, drawn by its own process.** `Pardes.undrawn` marks
+ a replica client's view: the session's pump and `drawOthers` skip it.
+ The client draws its own view between steps (`views.drawOne`), presenting
+ through the tty Shell's own `present`; what that writes in the core (a
+ frame skipped, one owed) is put back. A drawn parked view gives its
+ selections back to its slot (`drawParked`).
+- **The client is the tty host with its inputs fed elsewhere.** Shell
+ gains `remote`; `feed` sends a terminal's input to the replica engine
+ and the session instead of to the core; quit, hangup and the size are
+ the link's. Its replica core's allocator is the rollback epoch; images'
+ and PDFs' are the C heap's, whose workers allocate on threads of their
+ own. ponytail: a picture's pixels freed while a copy is held are not
+ put off.
+- **Every kind of input is speculated** but one into a terminal (rterm's
+ rule: its step reads the emulator), a view's own (a resize, the
+ pointer) too.
+- **`unattended` is a watched field**, and a session with a client is
+ attended: its messages go by keys, not the clock.
+- **Detach in a client ends that client**, a client's death too (its
+ link's end); the session and the others carry on. A Restore lets every
+ client go (they attach again).
+- **A lagging client never holds the session up.** Each client has a
+ writer thread and a queue the session appends to (units coalesce there
+ into one write); one whose queue passes 64 MiB is cut off. rterm's
+ deltas skip states within that (32 in flight).
+- **ssh: a terminal asked for, not -N.** `--attach=<host>:<name>` runs
+ `$PARDES_SSH -tt -q -L <here>:<there> <host> 'exec sleep ...'`: only an
+ interactive connection gets TCP_NODELAY from ssh and sshd. With -N,
+ Nagle held each key of a burst for the last one's ACK, and a shell's
+ echo through the box took two round trips (413 ms) where it takes one.
+ `$PARDES_REMOTE_RUNTIME` names the remote runtime directory when it is
+ not this one's path.
+- **`ssh -C` by default** when `$PARDES_SSH` is not set (it is then the
+ whole command). Through the box a 3.9 MB join took 7.8 to 7.9 s plain
+ and 5.4 to 5.8 s compressed. Of a join's time ssh's own forward is 2.0 s
+ (connect, key exchange, authentication, a pty) and opening a channel
+ 380 ms; a ControlMaster in the user's ssh config would skip the first.
+ A rejoin keeps its forward.
+
+Measured. ReleaseFast; timings alone on the perf cores (lock.sh), the
+box's through its own link (207 ms, 170 to 400 KB/s from it). The harness
+scripts are in `.scratch/daemon/remote/t2/`.
+
+- **The main test** (`e2e_box2.py`): a session on the FreeBSD box with a
+ 400-line file, two replica clients here through `ssh -L` each, and a
+ terminal on the box itself attached as a view (`pardes --attach` there
+ over `ssh -tt`). Each types into the file; A and B move and scroll; A
+ opens a shell, runs a command in it and exits it; B is hung up; A and
+ the box's terminal detach. A's and B's digests were the session's at
+ every quiet point (every check agreed), the session's text had
+ all three's words, the shell's answer was on A's screen 214 to 224 ms
+ after Enter, and nothing was left on the box once its idle daemon had
+ gone (a few seconds). The clients' first frames
+ came 3.4 to 4.0 s after they started (ssh's forward, below).
+- **Keystroke to present on a client, local-only actions** (base's
+ `pardes-perf-live --live`, its keys on a 50k-line file; the client of a
+ session detached on that file, through test/delay_relay.py at each RTT
+ and through the box's link; two rounds interleaved; p50 / p99, µs):
+
+ | an idle machine, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel |
+ |---|---|---|---|---|---|
+ | a local session, this line | 2070 / 2067 (2926) | 1507 / 1502 (1895) | 4744 / 4708 (7141) | 1997 / 2033 (5486) | 2098 / 2126 (2885) |
+ | a local session, base | 2059 / 2068 (2678) | 1517 / 1536 (2391) | 4755 / 4695 (7464) | 1916 / 1927 (5441) | 2100 / 2108 (2642) |
+ | client, direct | 1903 / 1917 (6813) | 1644 / 1650 (2136) | 4477 / 4413 (7912) | 1994 / 2244 (7405) | 1934 / 2063 (6793) |
+ | client, relay 30 ms | 1849 / 1838 (6910) | 1576 / 1557 (2695) | 4465 / 4300 (15560) | 1755 / 1712 (7682) | 1860 / 1757 (7163) |
+ | client, relay 80 ms | 1799 / 1753 (7281) | 1532 / 1501 (2034) | 3085 / 2654 (13705) | 1778 / 1743 (6386) | 2106 / 1870 (7692) |
+ | client, relay 207 ms | 1778 / 1814 (6721) | 1453 / 1478 (1887) | 4309 / 4333 (18092) | 4640 / 1745 (7562) | 1963 / 1978 (7315) |
+ | client, the box, ssh -C | 1773 (6947) | 1471 (2138) | 4441 (14018) | 3219 (7503) | 2163 (7692) |
+
+ The same harness while builds ran on the other cores (the line before
+ its last two changes, which do not touch a client's keys; one round):
+
+ | builds on the other cores, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel |
+ |---|---|---|---|---|---|
+ | a local session, this line | 464 (574) | 343 (417) | 1190 (1714) | 370 (1251) | 471 (638) |
+ | a local session, base | 463 (629) | 349 (499) | 1188 (1663) | 399 (2078) | 2096 (2591) |
+ | client, direct | 433 (1266) | 389 (441) | 1381 (2071) | 1183 (1400) | 451 (1360) |
+ | client, relay 30 ms | 413 (1243) | 364 (537) | 1332 (4083) | 1163 (1254) | 421 (1293) |
+ | client, relay 80 ms | 421 (1379) | 357 (500) | 1104 (1815) | 1143 (1359) | 1112 (1487) |
+ | client, relay 207 ms | 398 (1228) | 339 (444) | 1145 (6882) | 1151 (1263) | 615 (1469) |
+
+ - **A client's local-only key is as fast as a local session's at any
+ RTT, the box's link included**: its p50s are the local session's,
+ whatever the delay. Its p99 is 1 to 3x the local one's: the rollbacks
+ (another's unit before its pending inputs; p99 1.6 to 17 ms in the
+ client's own count) land between some keys and their frames.
+ - **The machine's state moves every cell together** by up to 4.5x: on
+ an idle machine the cores sleep deeply between keys (the `powersave`
+ governor), and with builds on the other cores they do not; in a run
+ made while one build ran, half the cells were in each state. Within
+ one run, base and this line's local session read alike, and the
+ clients alike with them.
+ - Page-down on a client costs 1.15 ms where a local one costs 0.37 when
+ the machine is awake: a client draws outside the pump, so the render's
+ paint-ahead (`Stream.askAhead`, asked of the host after a pump's
+ present) is never asked, and the rows a page-down brings are
+ highlighted in its frame.
+
+- **Terminal echo** (`echo2.py`: a key written to the client's terminal,
+ to the first output holding its glyph; 30 keys; p50 / p90, ms):
+
+ | | file | shell, predicted | shell, not predicted (PARDES_NO_ECHO) |
+ |---|---|---|---|
+ | a local session, base | 0.5 / 0.5 | | 0.8 / 0.8 |
+ | a local session, this line | 0.5 / 0.5 | | 0.8 / 0.8 |
+ | client, relay 30 ms | 0.6 / 0.6 | 0.8 / 0.9 | 31.4 / 31.4 |
+ | client, relay 80 ms | 0.5 / 0.5 | 0.5 / 0.8 | 81.4 / 81.6 |
+ | client, relay 207 ms | 0.5 / 0.5 | 0.5 / 0.5 | 208.4 / 208.6 |
+ | client, the box, ssh | 0.5 / 0.6 | 0.5 / 0.6 | 206.9 / 207.4 |
+ | client, the box, ssh -C | 0.5 / 0.5 | 0.5 / 0.6 | 215.8 / 216.9 |
+
+ Predicted, a key into a shell at a prompt shows as fast as a local one
+ (it is drawn by the client's own frame); not predicted, it is one round
+ trip and a millisecond and a half. Before this round (the draft's) the
+ echo through the box was 211 ms p50.
+- **The visible rollback rate.** A frame drawn after one of the client's
+ own inputs was recorded otherwise is counted as a rollback that may have
+ been seen (an upper bound: the frame shows the inputs after it too).
+ In the keys harness, of 669 inputs a run (two runs each): none at 0
+ and 30 ms; 2 and 3 (0.4%) at 80 ms; 11 and 13 (1.6 to 1.9%) at 207 ms,
+ of 25 and 29 mispredicted (a key the session recorded otherwise than
+ the client applied it). Every other rollback (another's unit before the
+ client's pending inputs) is put back and replayed before the next frame.
+- **Join cost on the session** (`join2.py`: three clients join a session
+ one after another, through the Unix socket; the session's worst
+ boundary, which holds the join, and the client's first frame):
+
+ | session | its dump | of it, made in the step | worst step, before | worst step, after | first frame |
+ |---|---|---|---|---|---|
+ | typical: 4.1 MB of files, 1 MB of scrollback, 6 panes | 8.1 MB | 113 KB | 12.4 ms | 1.6 ms | 22 to 46 ms |
+ | big: 38.6 MB of files, 8 MB of scrollback, 14 panes | 75.6 MB | 122 KB | 51.3 ms | 1.5 ms | 87 to 117 ms |
+ | a 100 MB file beside typical's (130 MB of files) | 254 MB | 114 KB | 172.5 ms | 1.6 ms | 247 to 288 ms |
+
+ Each join itself took 0.3 to 1.4 ms of a step. The dump is about twice
+ the files: their undo snapshots are blobs too.
+- **Join through the box** (`boxjoin.py`: a session there with five of the
+ tree's source files open, 3.5 MB, and a shell with 1 MB of scrollback;
+ a 3.9 MB dump; two joins each): the client's first frame 7.8 and 7.9 s
+ through plain ssh, 5.4 and 5.8 s through `ssh -C` (the default now); the
+ session's worst step 0.9 ms. A 16 KB session's first frame takes 3.4 to
+ 4.0 s of which ssh's forward is 2.0 s and its channel's opening 380 ms.
+- **The session's step while a client lags** (`lag2.py`: a shell printing
+ a line a millisecond for 12 s, client A typing a key every 30 ms, client
+ B stopped with SIGSTOP and cut off at a 256 KB lag; µs, p50 / p99 /
+ max): A's inputs 47 / 150 / 213 and the boundaries 66 / 208 / 681 with B
+ stopped (cut off: yes), against 60 / 134 / 309 and 69 / 276 / 4072 with
+ B running.
+- **Digests at quiescence**: every client of every test above ended on
+ the session's digest (PARDES_REPLICA_CHECK): the main test's, the
+ clients test's four (one rejoined after a forced divergence), and the
+ upgrade test's client that joined again.
+
+- **No local regression.** With no client attached the recorder costs an
+ atomic load a pump (`Peers.boundary`), a flag test an event (`split`)
+ and one a raw free (`memory.pinned`); a session's own core has no echo.
+ ReleaseFast, interleaved base new base new base on the perf cores, the
+ median of each side's runs, new/old, beside an A/A of the old:
+ - keystroke to present (base's `pardes-perf-live --live`, against base,
+ present / core):
+
+ | | new/base | A/A |
+ |---|---|---|
+ | key-down | 0.98 / 0.98 | 0.99 / 1.01 |
+ | key-right | 0.86 / 0.84 | 1.00 / 0.99 |
+ | edit-char | 0.76 / 0.86 | 1.01 / 1.00 |
+ | page-down | 0.81 / 0.81 | 0.91 / 0.93 |
+ | wheel | 0.89 / 0.88 | 0.96 / 1.00 |
+ | `yes \| head -c 32M` wall | 0.98 | 1.06 |
+ | a 1M-line open | 1.03 | 1.32 |
+
+ - the core in process (`pardes-perf --reps 3`, each tree's own build of
+ it), against main (ykrsossp): the keys, page-down and wheel 0.92 to
+ 1.08 over the small, medium and large files; the edits 0.91 to 1.05;
+ the opens 0.93 to 1.02; body-2m 1.02 to 1.26 (small's 1.26 alone); the
+ scrollback cells 0.93 to 1.05. The A/A read 0.79 to 1.46 (most within
+ 0.9 to 1.1). Against base the keys read 1.11 to 1.20 and the
+ scrollback renders 1.68 to 2.18: main's since base (this line is level
+ with main), and base's harness is an older perf.zig, so that pair
+ measures the harness too.
+
+The gates, once at the end (Debug unless said):
+- `zig build unit-test -Dplatform=tty` passes, its perf gate too;
+ `-Dplatform=gui` 1563 of 1575, 10 skipped, and the two that fail on main
+ already (`tagline pointer mapping...`, `a tag caret past its row's
+ end...`).
+- `zig build snap`: 113 of 113. `zig build replay`: the 113 scripts and 20
+ monkey seeds, none panicked or diverged. `zig build replicas`: 4 seeds
+ at each RTT, none diverged, none unequal, no delta wrong.
+- `test/daemon.py` (20 cycles, its replica clients' test included: four
+ clients, one through the relay, one rejoined after a forced divergence,
+ one a shell's, a terminal attached while a client's view was live),
+ `test/daemon.py --upgrade` (an unstamped build to a stamped one, its two
+ replica clients included) and `test/fs.py` (49 s) pass.
+- core-test's new tests: a join's dump with its blobs spliced back is the
+ session's dump byte for byte; a pinned blob's free is put off until its
+ last pin goes; and the replica tests (`-Dtest-filter=replica`).
+
+Found on the way:
+- **A terminal attached while a client's view is live drew nothing.**
+ The tty's attach took the live view's screen, and a client's view has
+ none in the session: it returned without a word, its stub waiting (the
+ main test's terminal on the box). It takes the session's own now; the
+ clients test attaches one while a client's view is live.
+- **The session's layout read a terminal's prompt after its output**
+ (above, `.prompt`): a client opening a shell diverged at its first
+ prompt.
+- **A divergence shows only at a unit the client replays.** The fast path
+ compares a speculation's records with the unit's, not digests, so a
+ client off the session's state that only types on stays off until a
+ unit it did not apply comes (another's input, or its own it waited
+ for). The forced-divergence test has another client type.
+- **Harness traps** (the scripts, not pardes): a terminal on the box
+ through `ssh -tt` from a pty must have its size before ssh starts, else
+ the remote pty is 0 by 0 and the session draws nothing in it; an Esc
+ and the next key written within the input parser's lone-Esc window read
+ as an Alt chord; a ReleaseFast build stamped with another commit has
+ another semantics hash than the box's of the same tree (`-Dstamp-commit
+ =false` for both); keys a test types into a client whose live pane is a
+ shell in tty mode go to the shell (end it with a hangup).
+
+Not done, and why:
+- **A GUI client**, and a window session serving replicas (refused in
+ words). The GUI host feeds its core from 52 places, each to route
+ through the replica as the tty's `feed` does; its frame loop is the
+ pump's (the client draws between steps, outside it); and its own
+ presentation (the window's surface, its animations) would have to come
+ out of the core's step as the tty's present does. A window client of a
+ tty session is the main test's other half; it is the next piece.
+- Paint-ahead on a client (above: page-down 1.15 ms where a local one is
+ 0.37): its draw would ask for it as a pump does, the worker's rows taken
+ in as the client's own events, outside the session's tape.
+- A rejoin gets a new view: the old one's layout and focus go with its
+ link. A rejoin that kept the view would take the view's number in the
+ hello.
+- A notice of the session's own while clients replicate goes through the
+ tape (the upgrade's line); a host that writes core state outside a step
+ (the GUI's notice) would diverge them, and the GUI serves none.
+- The join waits for the session's texts to be hashed once a session (its
+ first join, or after new texts): the joiner pays some milliseconds
+ more, never the session.
+- A held paste (`split`) is dropped if the session ends with it held.
+- A `--attach=<host>:<name>` client ended by a hangup leaves its forward's
+ socket file in $TMPDIR (its ssh goes with the terminal; the unlink is on
+ the normal way out only).
+
## Phase 6: upgrade notes
Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`,
@@ -2543,4 +2913,5 @@ Not done, and why:
harness checks in the rendered surface, and its build compiles and
passes the echo tests.
- The remote client (the other line) wires it: `initDump`'s core, the
- frame hook, its `moved`.
+ frame hook, its `moved`. (Done since: Phase 5 part B notes, with the
+ echo through the FreeBSD box's link.)