diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/next-daemon.md | 373 |
1 files changed, 372 insertions, 1 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md index 5039c81e..f6400770 100644 --- a/docs/next-daemon.md +++ b/docs/next-daemon.md @@ -1870,6 +1870,376 @@ Follow-ups (2026-10-29): p99 0.27 ms (0.22 before, within the run's noise); none diverged, none unequal, no delta wrong. +## Phase 5 part B notes + +Built 2026-10-29 and 30: the stream over the replica socket (src/replica.zig +`Peers`, and its client half, `dial` and `readStream`; detached/wire.zig +version 15), the client process (`pardes --attach=<socket>:<name>` and +`--attach=<host>:<name>`, tty.zig `remote`), the session's side of it in +the tty host, rterm's screen-state terminals and the echo agent's +predicted echo carried by it, a client's rejoin, and test/daemon.py's +replica clients. Decisions taken at the forks, with the ponytail lens: + +- **A client's input is a unit of its own, settled by `step`.** On the + authority, at the step boundary after each pump (`Peers.boundary`), each + input that came is `toView(its view)`, `update(it)` and `step()`: the + middle of a pump (queued inputs and effects), with no wait, no clock + read and no frame. The client's speculation is the same three calls, so + the two tapes read alike and the fast path takes the unit as it came. + With the pump in its place (part A's harness) every speculation stopped + at the tty host's `wait_input` and read the clock in ns. +- **The stream.** Framed as wire.zig frames everything, a tag and a u32le + length: + - the session's: 0x01 welcome (pid, name, the client's id and view, the + dump's length, the session's offset from UTC, its digest as dumped + and its build's semantics), 0x02 refused (words), 0x04 a piece of the + dump (1 MiB), 0x05 a unit (origin, seq, the authority's realtime and + monotonic clocks when it was made, its tape), 0x06 a terminal's + message (rterm's: a delta or rows), 0x07 a check's answer; + - the client's: its inputs as the existing event tags (their seq is the + n-th on the connection), 0x04 a terminal's message (rterm's: an + acknowledgement, rows asked, a reset), 0x05 check. + + A unit carries one digest, at its end (`unit_digests`); under screen + state it is one word (`whole_digest`, PARDES_REPLICA_TRACE for one a + field). Blobs are hashed once and folded in as a word (main's + walk.zig), so a digest a unit is cheap. The draft's lengths-only digest + went with that. +- **Terminals cross as screen state, in rterm's order.** The session's + Peers keep a `Screens`, a peer each. At each boundary: the rows the + pump's steps read that a client holds as holes (`fill`, then `filled`), + the pump's unit, then the deltas (`flush`). Before each client input's + step its deltas again, after it the rows it read, then its unit. A + joiner's terminals start afresh after its dump. The client applies a + delta or rows in stream order, acknowledges, and asks for the history + rows a frame shows that it lacks. Units carry no pty bytes. + - **A wait takes one step of a person's input, and none after a pty's + output** (tty.zig `split`), while clients replicate: a step reads the + terminals as the deltas after each unit bring them, and in a pump that + took a pty's output and then a key, the key would read on the session + what the client has only after the unit. The next is held for the + next wait. Only while a client is attached. + - **A step that reads a terminal's prompt after its output is taped** + (tape.zig `.prompt`): the first prompt's greeting (`ls`, after a pty + resize) and a command waiting for the prompt. Found at once by a + client opening a shell of its own: the session's layout after the + shell's first output saw the prompt, and the client's replay, its + emulator a delta behind, did not. + - A dump-loaded client's emulators are on the C heap, as + `terminal.create` makes a screen replica's (walk.zig `.alloc` of a + pane's `terminal`, dump.zig `terminal_gpa`). +- **Predicted echo is the echo agent's** (src/echo.zig), wired as its + notes say: the client's core points at its replica's `echo`, a frame + judges it before drawing, and a wake comes a frame later while + predictions are out. PARDES_NO_ECHO turns it off. +- **Join, at a step boundary, the step paying for the mutable state + alone.** The view is opened and the session dumped without its blobs + (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted + where they go) and without its terminals' emulators (their size: the + first delta brings each). The client's writer thread writes the + welcome, then the dump with the blobs put back where they go: what the + client reads is `dumpBytes`' dump byte for byte but for the terminals + (a test splices one of a session without any and compares), and the + client is unchanged. Each blob is pinned until it is + written (memory.zig `pin`: its free is put off; a blob is never written + in place and is freed raw). The texts the session never digested (it + records nothing until a client comes) would be hashed by the join's + first digest, 250 MB for a 100 MB file and its undo: they are hashed + first on a thread of their own (walk.zig `unhashed`, `makeWords`), + pinned meanwhile, and the join waits a boundary for them; a second + round joins whatever is left. +- **A joiner checks its load.** The welcome carries the session's digest + as dumped; a client whose load differs says so and ends. The first unit + after a join says the host and its writes again (`tape.resync`), which a + core made from a dump has not. Found on the way: the dump's reader took + the first unit with the dump's last piece and dropped it (the client + diverged at its first digest about one join in three); what it read past + the dump is the stream's now (`Framer`). +- **A diverged client joins again**, the same join as the first: its link + ended, the session dialled again, its dump loaded, its view's size sent, + in the same terminal, and a line over its frame until its next key says + why ("joined again: this client diverged from the session (...)"). It + gets a new view: the old one went with its link. PARDES_TEST_DIVERGE=<n> + puts a client's core off the session's (a digested counter) once quiet + after its n-th unit. A divergence shows only at a unit the client + replays (another's input, or its own it did not apply): the fast path + compares records, not digests. +- **An Upgrade lets its clients go in words** (`Peers.refuseAll`, before + the dump): each is sent `refused` ("the session is upgrading to another + build of pardes") after what it was sent, for up to 200 ms, its view + closed. The client says it in its line and dials again, every 100 ms + for 20 s (the exec, the load, the daemon's own exec). A daemon's + refusal for another build, or a welcome whose semantics are not its + own (the daemon judged the hello by the hash it last heard, which an + upgrade under way may not have said yet), ends it with "session + upgraded to a newer pardes: upgrade this client too". "A newer pardes + is installed" is now a Host call the core makes in its focus step + (`newer_build`), so a session's clients take the line from the tape. +- **The session's clock, guessed.** A unit carries the authority's clocks + when it was made; a speculation's clock reads answer the last unit's + reading plus the time since it came plus the client's round trip + (`tape.Guess`): a file's mtime (realtime seconds), a message's time of + day (in the session's zone, its offset in the welcome) and a message's + monotonic ms. Never the client's own wall clock. +- **Mostly no rollback for the session's own pumps.** Every client input + woke the session's pump before its own unit, and that pump advanced the + clock: a foreign unit, a rollback a key. Two changes: a pump woken only + by a replica's post reads no new time (`mail_only`: the tty's `now` + answers the core's own), and a replica keeps what it applied over a + unit of another's that ends on the digest the last one ended on + (`neutral`; such a unit is replayed when the copy is next put back). A + unit that says what the host wrote, or which host it is, is never + neutral. +- **The copy is taken while idle.** The client takes its rollback copy + after drawing, when nothing is queued (`Replica.ready`), and lets it go + when a unit comes first: a key no longer pays a megabyte's copy (memcpy + was a third of the client's profile). The epoch's and the copy's maps + hash an address by a multiply, not Wyhash (the other sixth). +- **Each client a view, drawn by its own process.** `Pardes.undrawn` marks + a replica client's view: the session's pump and `drawOthers` skip it. + The client draws its own view between steps (`views.drawOne`), presenting + through the tty Shell's own `present`; what that writes in the core (a + frame skipped, one owed) is put back. A drawn parked view gives its + selections back to its slot (`drawParked`). +- **The client is the tty host with its inputs fed elsewhere.** Shell + gains `remote`; `feed` sends a terminal's input to the replica engine + and the session instead of to the core; quit, hangup and the size are + the link's. Its replica core's allocator is the rollback epoch; images' + and PDFs' are the C heap's, whose workers allocate on threads of their + own. ponytail: a picture's pixels freed while a copy is held are not + put off. +- **Every kind of input is speculated** but one into a terminal (rterm's + rule: its step reads the emulator), a view's own (a resize, the + pointer) too. +- **`unattended` is a watched field**, and a session with a client is + attended: its messages go by keys, not the clock. +- **Detach in a client ends that client**, a client's death too (its + link's end); the session and the others carry on. A Restore lets every + client go (they attach again). +- **A lagging client never holds the session up.** Each client has a + writer thread and a queue the session appends to (units coalesce there + into one write); one whose queue passes 64 MiB is cut off. rterm's + deltas skip states within that (32 in flight). +- **ssh: a terminal asked for, not -N.** `--attach=<host>:<name>` runs + `$PARDES_SSH -tt -q -L <here>:<there> <host> 'exec sleep ...'`: only an + interactive connection gets TCP_NODELAY from ssh and sshd. With -N, + Nagle held each key of a burst for the last one's ACK, and a shell's + echo through the box took two round trips (413 ms) where it takes one. + `$PARDES_REMOTE_RUNTIME` names the remote runtime directory when it is + not this one's path. +- **`ssh -C` by default** when `$PARDES_SSH` is not set (it is then the + whole command). Through the box a 3.9 MB join took 7.8 to 7.9 s plain + and 5.4 to 5.8 s compressed. Of a join's time ssh's own forward is 2.0 s + (connect, key exchange, authentication, a pty) and opening a channel + 380 ms; a ControlMaster in the user's ssh config would skip the first. + A rejoin keeps its forward. + +Measured. ReleaseFast; timings alone on the perf cores (lock.sh), the +box's through its own link (207 ms, 170 to 400 KB/s from it). The harness +scripts are in `.scratch/daemon/remote/t2/`. + +- **The main test** (`e2e_box2.py`): a session on the FreeBSD box with a + 400-line file, two replica clients here through `ssh -L` each, and a + terminal on the box itself attached as a view (`pardes --attach` there + over `ssh -tt`). Each types into the file; A and B move and scroll; A + opens a shell, runs a command in it and exits it; B is hung up; A and + the box's terminal detach. A's and B's digests were the session's at + every quiet point (every check agreed), the session's text had + all three's words, the shell's answer was on A's screen 214 to 224 ms + after Enter, and nothing was left on the box once its idle daemon had + gone (a few seconds). The clients' first frames + came 3.4 to 4.0 s after they started (ssh's forward, below). +- **Keystroke to present on a client, local-only actions** (base's + `pardes-perf-live --live`, its keys on a 50k-line file; the client of a + session detached on that file, through test/delay_relay.py at each RTT + and through the box's link; two rounds interleaved; p50 / p99, µs): + + | an idle machine, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | + |---|---|---|---|---|---| + | a local session, this line | 2070 / 2067 (2926) | 1507 / 1502 (1895) | 4744 / 4708 (7141) | 1997 / 2033 (5486) | 2098 / 2126 (2885) | + | a local session, base | 2059 / 2068 (2678) | 1517 / 1536 (2391) | 4755 / 4695 (7464) | 1916 / 1927 (5441) | 2100 / 2108 (2642) | + | client, direct | 1903 / 1917 (6813) | 1644 / 1650 (2136) | 4477 / 4413 (7912) | 1994 / 2244 (7405) | 1934 / 2063 (6793) | + | client, relay 30 ms | 1849 / 1838 (6910) | 1576 / 1557 (2695) | 4465 / 4300 (15560) | 1755 / 1712 (7682) | 1860 / 1757 (7163) | + | client, relay 80 ms | 1799 / 1753 (7281) | 1532 / 1501 (2034) | 3085 / 2654 (13705) | 1778 / 1743 (6386) | 2106 / 1870 (7692) | + | client, relay 207 ms | 1778 / 1814 (6721) | 1453 / 1478 (1887) | 4309 / 4333 (18092) | 4640 / 1745 (7562) | 1963 / 1978 (7315) | + | client, the box, ssh -C | 1773 (6947) | 1471 (2138) | 4441 (14018) | 3219 (7503) | 2163 (7692) | + + The same harness while builds ran on the other cores (the line before + its last two changes, which do not touch a client's keys; one round): + + | builds on the other cores, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | + |---|---|---|---|---|---| + | a local session, this line | 464 (574) | 343 (417) | 1190 (1714) | 370 (1251) | 471 (638) | + | a local session, base | 463 (629) | 349 (499) | 1188 (1663) | 399 (2078) | 2096 (2591) | + | client, direct | 433 (1266) | 389 (441) | 1381 (2071) | 1183 (1400) | 451 (1360) | + | client, relay 30 ms | 413 (1243) | 364 (537) | 1332 (4083) | 1163 (1254) | 421 (1293) | + | client, relay 80 ms | 421 (1379) | 357 (500) | 1104 (1815) | 1143 (1359) | 1112 (1487) | + | client, relay 207 ms | 398 (1228) | 339 (444) | 1145 (6882) | 1151 (1263) | 615 (1469) | + + - **A client's local-only key is as fast as a local session's at any + RTT, the box's link included**: its p50s are the local session's, + whatever the delay. Its p99 is 1 to 3x the local one's: the rollbacks + (another's unit before its pending inputs; p99 1.6 to 17 ms in the + client's own count) land between some keys and their frames. + - **The machine's state moves every cell together** by up to 4.5x: on + an idle machine the cores sleep deeply between keys (the `powersave` + governor), and with builds on the other cores they do not; in a run + made while one build ran, half the cells were in each state. Within + one run, base and this line's local session read alike, and the + clients alike with them. + - Page-down on a client costs 1.15 ms where a local one costs 0.37 when + the machine is awake: a client draws outside the pump, so the render's + paint-ahead (`Stream.askAhead`, asked of the host after a pump's + present) is never asked, and the rows a page-down brings are + highlighted in its frame. + +- **Terminal echo** (`echo2.py`: a key written to the client's terminal, + to the first output holding its glyph; 30 keys; p50 / p90, ms): + + | | file | shell, predicted | shell, not predicted (PARDES_NO_ECHO) | + |---|---|---|---| + | a local session, base | 0.5 / 0.5 | | 0.8 / 0.8 | + | a local session, this line | 0.5 / 0.5 | | 0.8 / 0.8 | + | client, relay 30 ms | 0.6 / 0.6 | 0.8 / 0.9 | 31.4 / 31.4 | + | client, relay 80 ms | 0.5 / 0.5 | 0.5 / 0.8 | 81.4 / 81.6 | + | client, relay 207 ms | 0.5 / 0.5 | 0.5 / 0.5 | 208.4 / 208.6 | + | client, the box, ssh | 0.5 / 0.6 | 0.5 / 0.6 | 206.9 / 207.4 | + | client, the box, ssh -C | 0.5 / 0.5 | 0.5 / 0.6 | 215.8 / 216.9 | + + Predicted, a key into a shell at a prompt shows as fast as a local one + (it is drawn by the client's own frame); not predicted, it is one round + trip and a millisecond and a half. Before this round (the draft's) the + echo through the box was 211 ms p50. +- **The visible rollback rate.** A frame drawn after one of the client's + own inputs was recorded otherwise is counted as a rollback that may have + been seen (an upper bound: the frame shows the inputs after it too). + In the keys harness, of 669 inputs a run (two runs each): none at 0 + and 30 ms; 2 and 3 (0.4%) at 80 ms; 11 and 13 (1.6 to 1.9%) at 207 ms, + of 25 and 29 mispredicted (a key the session recorded otherwise than + the client applied it). Every other rollback (another's unit before the + client's pending inputs) is put back and replayed before the next frame. +- **Join cost on the session** (`join2.py`: three clients join a session + one after another, through the Unix socket; the session's worst + boundary, which holds the join, and the client's first frame): + + | session | its dump | of it, made in the step | worst step, before | worst step, after | first frame | + |---|---|---|---|---|---| + | typical: 4.1 MB of files, 1 MB of scrollback, 6 panes | 8.1 MB | 113 KB | 12.4 ms | 1.6 ms | 22 to 46 ms | + | big: 38.6 MB of files, 8 MB of scrollback, 14 panes | 75.6 MB | 122 KB | 51.3 ms | 1.5 ms | 87 to 117 ms | + | a 100 MB file beside typical's (130 MB of files) | 254 MB | 114 KB | 172.5 ms | 1.6 ms | 247 to 288 ms | + + Each join itself took 0.3 to 1.4 ms of a step. The dump is about twice + the files: their undo snapshots are blobs too. +- **Join through the box** (`boxjoin.py`: a session there with five of the + tree's source files open, 3.5 MB, and a shell with 1 MB of scrollback; + a 3.9 MB dump; two joins each): the client's first frame 7.8 and 7.9 s + through plain ssh, 5.4 and 5.8 s through `ssh -C` (the default now); the + session's worst step 0.9 ms. A 16 KB session's first frame takes 3.4 to + 4.0 s of which ssh's forward is 2.0 s and its channel's opening 380 ms. +- **The session's step while a client lags** (`lag2.py`: a shell printing + a line a millisecond for 12 s, client A typing a key every 30 ms, client + B stopped with SIGSTOP and cut off at a 256 KB lag; µs, p50 / p99 / + max): A's inputs 47 / 150 / 213 and the boundaries 66 / 208 / 681 with B + stopped (cut off: yes), against 60 / 134 / 309 and 69 / 276 / 4072 with + B running. +- **Digests at quiescence**: every client of every test above ended on + the session's digest (PARDES_REPLICA_CHECK): the main test's, the + clients test's four (one rejoined after a forced divergence), and the + upgrade test's client that joined again. + +- **No local regression.** With no client attached the recorder costs an + atomic load a pump (`Peers.boundary`), a flag test an event (`split`) + and one a raw free (`memory.pinned`); a session's own core has no echo. + ReleaseFast, interleaved base new base new base on the perf cores, the + median of each side's runs, new/old, beside an A/A of the old: + - keystroke to present (base's `pardes-perf-live --live`, against base, + present / core): + + | | new/base | A/A | + |---|---|---| + | key-down | 0.98 / 0.98 | 0.99 / 1.01 | + | key-right | 0.86 / 0.84 | 1.00 / 0.99 | + | edit-char | 0.76 / 0.86 | 1.01 / 1.00 | + | page-down | 0.81 / 0.81 | 0.91 / 0.93 | + | wheel | 0.89 / 0.88 | 0.96 / 1.00 | + | `yes \| head -c 32M` wall | 0.98 | 1.06 | + | a 1M-line open | 1.03 | 1.32 | + + - the core in process (`pardes-perf --reps 3`, each tree's own build of + it), against main (ykrsossp): the keys, page-down and wheel 0.92 to + 1.08 over the small, medium and large files; the edits 0.91 to 1.05; + the opens 0.93 to 1.02; body-2m 1.02 to 1.26 (small's 1.26 alone); the + scrollback cells 0.93 to 1.05. The A/A read 0.79 to 1.46 (most within + 0.9 to 1.1). Against base the keys read 1.11 to 1.20 and the + scrollback renders 1.68 to 2.18: main's since base (this line is level + with main), and base's harness is an older perf.zig, so that pair + measures the harness too. + +The gates, once at the end (Debug unless said): +- `zig build unit-test -Dplatform=tty` passes, its perf gate too; + `-Dplatform=gui` 1563 of 1575, 10 skipped, and the two that fail on main + already (`tagline pointer mapping...`, `a tag caret past its row's + end...`). +- `zig build snap`: 113 of 113. `zig build replay`: the 113 scripts and 20 + monkey seeds, none panicked or diverged. `zig build replicas`: 4 seeds + at each RTT, none diverged, none unequal, no delta wrong. +- `test/daemon.py` (20 cycles, its replica clients' test included: four + clients, one through the relay, one rejoined after a forced divergence, + one a shell's, a terminal attached while a client's view was live), + `test/daemon.py --upgrade` (an unstamped build to a stamped one, its two + replica clients included) and `test/fs.py` (49 s) pass. +- core-test's new tests: a join's dump with its blobs spliced back is the + session's dump byte for byte; a pinned blob's free is put off until its + last pin goes; and the replica tests (`-Dtest-filter=replica`). + +Found on the way: +- **A terminal attached while a client's view is live drew nothing.** + The tty's attach took the live view's screen, and a client's view has + none in the session: it returned without a word, its stub waiting (the + main test's terminal on the box). It takes the session's own now; the + clients test attaches one while a client's view is live. +- **The session's layout read a terminal's prompt after its output** + (above, `.prompt`): a client opening a shell diverged at its first + prompt. +- **A divergence shows only at a unit the client replays.** The fast path + compares a speculation's records with the unit's, not digests, so a + client off the session's state that only types on stays off until a + unit it did not apply comes (another's input, or its own it waited + for). The forced-divergence test has another client type. +- **Harness traps** (the scripts, not pardes): a terminal on the box + through `ssh -tt` from a pty must have its size before ssh starts, else + the remote pty is 0 by 0 and the session draws nothing in it; an Esc + and the next key written within the input parser's lone-Esc window read + as an Alt chord; a ReleaseFast build stamped with another commit has + another semantics hash than the box's of the same tree (`-Dstamp-commit + =false` for both); keys a test types into a client whose live pane is a + shell in tty mode go to the shell (end it with a hangup). + +Not done, and why: +- **A GUI client**, and a window session serving replicas (refused in + words). The GUI host feeds its core from 52 places, each to route + through the replica as the tty's `feed` does; its frame loop is the + pump's (the client draws between steps, outside it); and its own + presentation (the window's surface, its animations) would have to come + out of the core's step as the tty's present does. A window client of a + tty session is the main test's other half; it is the next piece. +- Paint-ahead on a client (above: page-down 1.15 ms where a local one is + 0.37): its draw would ask for it as a pump does, the worker's rows taken + in as the client's own events, outside the session's tape. +- A rejoin gets a new view: the old one's layout and focus go with its + link. A rejoin that kept the view would take the view's number in the + hello. +- A notice of the session's own while clients replicate goes through the + tape (the upgrade's line); a host that writes core state outside a step + (the GUI's notice) would diverge them, and the GUI serves none. +- The join waits for the session's texts to be hashed once a session (its + first join, or after new texts): the joiner pays some milliseconds + more, never the session. +- A held paste (`split`) is dropped if the session ends with it held. +- A `--attach=<host>:<name>` client ended by a hangup leaves its forward's + socket file in $TMPDIR (its ssh goes with the terminal; the unlink is on + the normal way out only). + ## Phase 6: upgrade notes Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`, @@ -2543,4 +2913,5 @@ Not done, and why: harness checks in the rendered surface, and its build compiles and passes the echo tests. - The remote client (the other line) wires it: `initDump`'s core, the - frame hook, its `moved`. + frame hook, its `moved`. (Done since: Phase 5 part B notes, with the + echo through the FreeBSD box's link.) |
