summaryrefslogtreecommitdiff
path: root/docs/next-daemon.md
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-10-30 16:04:16 -0300
committerGabriel Schneider <[email protected]>2026-10-30 17:01:06 -0300
commit12d15ce9c00c96f5f7aa5bf30102d7ebebf0f9b7 (patch)
treea7ff710c09a1e116772ca16d73c2c0642d48e094 /docs/next-daemon.md
parentc890a00a52af0e6318bc2cbef755312b9e8602d4 (diff)
downloadpardes-12d15ce9c00c96f5f7aa5bf30102d7ebebf0f9b7.tar.gz
pardes-12d15ce9c00c96f5f7aa5bf30102d7ebebf0f9b7.zip
docs/next-daemon.md: the remote bugs the user hit, fix2's: what a step read that only the authority holds (the store, Recent, the 9P server's readers and socket, the process's options) and the model for it (tape.authority), the rollback over the host's heap, units that ended with effects queued, the lowered copy's heap bytes, the live slot left aliasing; the tests, the gate, the soak twice and H17's own configuration, the copy before and after; H18 handed back with what was found (a resize leaves a replica's screen state stale for a round trip) and a fix to try
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Diffstat (limited to 'docs/next-daemon.md')
-rw-r--r--docs/next-daemon.md196
1 files changed, 196 insertions, 0 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
index 28449487..8ec76742 100644
--- a/docs/next-daemon.md
+++ b/docs/next-daemon.md
@@ -3653,6 +3653,19 @@ same scripts (`.scratch/daemon/final/final2.sh`).
## The remote bugs the user hit
+> due to the ping I know it won't be possible to have the same snappy
+> feeling as the local version, but these kinds of errors have nothing to
+> do with this limitation, you should do it much better, and measure it
+> too since I don't want to catch those myself
+
+The user's window client of the box blinked, lost its panes and jumped its
+cursor: each was a divergence, and the rejoin after it opened a fresh
+view. An adversarial soak of replica clients (`test/soak.py`, a session
+here or on the box, terminal and window clients through a jittered delay
+relay, a seeded monkey of a person's actions) found the causes; the fixers
+took them by root cause, each with a test that fails before its fix. Ids
+are the hunt's (`.scratch/daemon/hunt/findings.md`).
+
### fix3: what the replica stream carried in its build's own words
A unit of the replica stream is its session's tape, written by
@@ -3729,3 +3742,186 @@ Found by the variant hunt (V2, E), with real ReleaseSafe executables.
replay even by its own build, on main too: pane 0's message differs a
step after a failure is said (seven times in the first runs, none in 15
since). cross_replay.py leaves the setting out.
+
+### fix2: what a step read that only one side had, and units that ended undrained
+
+Each of these made every client of a session diverge at the same unit, so
+the session's step and its replicas' replay disagreed. None was a lag.
+
+- **A step read what only the authority holds** (H4, H8, H11, H12, H13, and
+ the variant hunt's list). The store's closed panes and Recent are the
+ authority's (`.authority`: every window of the user writes the store's
+ file); a replica keeps its own, which has no file and only what it
+ replayed since it joined. Steps made digested state of them:
+ - the Panes picker's closed rows and Recent's rows (H4: every client
+ diverged at the key that opened them; H13: a client on another machine
+ read its own store);
+ - `trackJump`, at every step, dropped a closed file's jumps when Recent
+ did not know the file (H11: once a pane had closed, every client that
+ joined diverged at its first unit, forever, in `jumps njumps jcur`);
+ - `noteClosed` looked in Recent before it asked the host whether the file
+ is there (`recentKeeps`, a taped stat), so the session skipped a
+ syscall the replica made (H12: every Del of a file crashed the client);
+ - a closed pane brought back (`Store.reopen`: a Look at `@pN`, a Panes
+ row), a jump to a closed file and the +Jumps rows (`recentByHash`);
+ - and, the same class: a Look or exec handed to a script reading a pane's
+ events (`pane.fs.readers`, the 9P server's), Tty9p's socket check, and
+ the session's config directory and startup file (`opts`, `.local`:
+ ThemeFile, DumpThemes, Config, DumpConfig, an init file saved).
+
+ **The model:** what only the authority holds stays the authority's, and a
+ step reads it through `tape.authority` (src/tape.zig): a `value` read,
+ its answer taped; a replay takes the answer, and a speculation that
+ reaches one waits for it, as at a host call (one RTT, never wrong). A
+ replica's own store and Recent are never read by a step. Rejected: the
+ store and Recent as session state (dumped, digested, copied), which
+ needs the store's file I/O on the tape call by call (it reads other
+ windows' records at every step that writes), for the same rows.
+ - `trackJump` no longer reads Recent: a closed file's jump stays, and the
+ jump to it drops that file's entries when Recent has forgotten it, as
+ when the file is gone.
+ - `noteClosed` asks the host first, on every side.
+ - The tty toggle, read on every key (which cannot wait a round trip), is
+ session state: `Pardes.tty_toggle`, from the options when the core is
+ made, dumped.
+- **A rollback put back pointers to freed memory** (H8, the segfault in the
+ Panes picker). The copy saves a struct's bytes field by field, the
+ `.authority` ones included, but walks only what the digest walks. The
+ store and Recent live on the host's heap (`recentGpa`), whose frees the
+ epoch never puts off: a speculated Del grew the closed list, the
+ rollback put the old header back, and the next listing read a freed
+ name. `store` and `recent` are never saved now (rollback.zig `unsaved`);
+ a step never reads a replica's own.
+- **A unit taped outside a step ended with effects queued** (H6, H10). A
+ leave is the session's `closeView`, whose sync sizes again a terminal the
+ closed view sized and queues its pty's resize; the session did it at its
+ next pump, in another unit. A replica that replayed the close kept the
+ effect queued, and its next idle copy asserted (H10: a client killed when
+ another left). A client joining at that boundary was dumped with it
+ queued, which a loaded core has none of, and diverged at the session's
+ next pump (H6: `the core called poll_frame where the tape has a host
+ (12)`). `closeView` and `openView` settle inside their own input now.
+- **The lowered copy of a searched text held heap bytes** (H5's and H17's
+ `places.hits|hay`). A smart-case search lowers the text a piece at a time
+ into `Places.hay`, made with `memory.unfilled`; the digest and a join's
+ dump take the whole slice, so the pieces not searched yet were whatever
+ each process's heap held. Zeroed where it is made: 20 µs for 1 MB,
+ 0.4 ms for 10 MB, 10 ms for 100 MB in one frame (ReleaseFast, page
+ faults included; lowering the same text is four times that, over
+ frames), the order of the copy an edit of that text already makes.
+- **A view drawn outside a step left the live slot aliasing** (H5's
+ hypothesis, in a path it did not name). `withParked` (a replica client
+ drawing or reading its own view while another is live) parked the live
+ view and copied it back without emptying its slot or its panes' shares
+ of selections, which the rollback copy walks. Not the cause of H5's
+ signatures, which were the jumps (Recent) and the places (the lowered
+ copy) above; the slot was saved and put back all along.
+
+Tests, each failing before its fix: src/rollback.zig ("a rollback leaves the
+store and Recent as a speculation left them", "a view drawn outside a step
+leaves the live view's slot holding nothing"), src/replica.zig ("a replica
+joined after a pane closed", with the variant hunt's Recent written by
+another window; "a replica's Del of a file its session's Recent knows"; "a
+client leaves, and one joins"), src/Places.zig ("a lowered copy's pieces not
+searched yet are the same in every process"). The hunt's repros
+(`panes_closed.py`, `del_file.py`, `recent_store.py` with the client's own
+XDG_STATE_HOME, `leave_resize.py`) agree on the fixed builds; on the
+installed 0.24 `panes_closed.py` still rejoins 16 times. `zig build
+replicas` now looks at files, closes panes, opens the Panes and Recent
+pickers and jumps.
+
+The gate, on fix2's five changes (`.scratch/daemon/fix2/gate.sh`):
+
+- **Debug:** `core-test` tty; `core-test` gui and `unit-test -Dplatform=gui`
+ fail only on main's two (`tagline pointer mapping...`, `a tag caret past
+ its row's end...`); `unit-test` tty, `snap` (113/113), `replay`,
+ `replicas`, `test/daemon.py` 20, `--far`, `--manage`, `--upgrade`,
+ `--window-clients`, `--gui` 5 and `test/fs.py` pass.
+- **ReleaseSafe:** `test/daemon.py` 20, `--far`, `--window-clients`,
+ `--gui` 5, `--upgrade` from Debug and `test/fs.py` pass.
+- **Rebased onto fix3's line** (myulmrpt; its `tape.results` now lists
+ what `tape.authority` returns): `core-test` and `unit-test` tty,
+ `snap`, `replicas`, `test/daemon.py` 20, `--far`, `--window-clients`,
+ `--gui` 5 and `test/fs.py` pass again in Debug; `test/daemon.py` 20,
+ `--window-clients`, `--gui` 5 and `test/fs.py` in ReleaseSafe.
+- **The copy** (`zig build rollback-bench`, ReleaseFast, main's
+ rollback.zig against fix2's): a typical session's 980.3 KB to 980.1 KB,
+ a big one's 1487.9 to 1487.8 (the Store's and Recent's own bytes; their
+ heap was never saved). Its take 0.045 and 0.063 ms, 0.075 and 0.077;
+ its restore 0.036 and 0.034 ms, 0.053 and 0.054: noise, one run each.
+
+The soak (`test/soak.py local --minutes 5 --rtts 0,30,80,207 --clients
+tty,win --trace`, ReleaseSafe, the mouse on, Upgrades, resizes, detaches
+and scripts among the moves), twice:
+
+| cell | client | min | inputs | links | visible | mispred | p50 / p99 ms | cut short by |
+|---|---|---|---|---|---|---|---|---|
+| 0 ms | tty | 1.4 | 330 | 2 | 0 | 0 | 1.5 / 1298 | H7 (session) |
+| 0 ms | window | 1.4 | 225 | 3 | 6 | 2 | 4.5 / 12.4 | H7 |
+| 30 ms | tty | 0.6 | 101 | 1 | 0 | 0 | 43.2 / 1283 | H7 |
+| 30 ms | window | 0.6 | 56 | 1 | 0 | 0 | 4.8 / 25.4 | H7 |
+| 80 ms | tty | 5.1 | 962 | 1 | 6 | 30 | 14.4 / 1570 | |
+| 80 ms | window | 5.1 | 911 | 3 | 63 | 35 | 5.9 / 117 | |
+| 80 ms | window, attached later | 0.8 | 35 | 0 | 0 | 0 | 2.0 / 3.6 | its replay panic (H9) |
+| 207 ms | tty | 0.8 | 98 | 1 | 0 | 0 | 42.3 / 1879 | H7 |
+| 207 ms | window | 0.8 | 84 | 1 | 0 | 0 | 6.3 / 18.7 | H7 |
+
+- Every divergence of that run was the pointer's: `pointer_tag_hit
+ look_hover_wait` (H3, fixer 1's: a tag hit made from the process's own
+ frame), which takes every client of the session at once, and a window
+ client's replay that met the `pointer` site where the tape had none (H9's
+ panic, the same class). None of this section's classes.
+- H7 (unassigned: `integer overflow` in ghostty's PageList under
+ `Screens.flush`, after a unit with a `pty_resize`) ended the session in
+ three of four cells; the first run's cells, of the same seeds, did not
+ meet it.
+- The first run (its table lost to the harness, which raised when a session
+ with unsaved text would not Exit; fix2's copy now ends it by pid and
+ counts a breach): 0 ms, three rejoins on H3 alone; 30 ms, one window
+ client's replay panic at the `pointer` site; 80 ms, three on H3 and three
+ of H18's (`jumps … pane N.body` at units with a `pty_resize`, every
+ client together).
+- H17's own configuration (30 ms, seed 22, 4 minutes, where both clients
+ had died at `n` on different places): no divergence in `places`. Its
+ rejoins were H3's (five) and H18's (two units, `jumps … pane 0.body`
+ and `pane 0.body pane 0.ovl`, each with a `pty_resize`).
+
+Not done, and found on the way:
+
+- **H18, handed back: a terminal's resize leaves a replica's screen state
+ stale for a round trip.** Two signatures, both on every client at one
+ unit (`to_view`, a key, a `step` with a `pty_resize`):
+ - `pane 2.ovl` (the hunt's seed 5 at 80 ms, deterministic): an edit copy
+ extended over a three-row terminal read history rows that both
+ replicas held as blank holes where the session read text, and the
+ session sent none before the unit (`Screens.fill`). The resize in that
+ unit was another pane's. An instrumented build showed the step read
+ the rows through the core's cache (`shellRows`), which marks a
+ terminal read only when it builds the rows, so `fill` skips it; and
+ with that fixed the unit still diverged.
+ - `jumps … pane N.body` (this table's 80 ms cell, H5's own signature).
+ - The window: a resize renumbers a terminal's rows (`Rows.epoch`); the
+ session's next delta is against the old base, a replica refuses it
+ (`Renumbered`) and is started afresh a round trip later, while
+ `noteSent` took it as applied and expects the replica's holes where
+ they were. A step that reads the terminal meanwhile, or in the step
+ that resized it after the reflow, reads another grid on each side.
+ - The likely fix, untried: a delta across a renumbering is a reset
+ (`encode` with no base when the base's epoch is another), so the peer
+ is afresh at once and `fill` knows its holes; `shellRows` marks a read
+ on a kept hit too; and, with screen-state replicas, the session
+ reflows at the cut of the step's unit, not inside it, so the step
+ reads one grid on both sides. A test needs Screens with two replicas
+ of different sizes and a short terminal with a history.
+- **Two edges of the same class, left:** a pipe or an Edit in flight when
+ a client joins (`Pipe.wait`, `.authority`, not dumped: the joiner drops
+ its answer), and a 9P batch held at a join (`fs.batch`). Both need a
+ join in the instant between a request and its answer.
+- **A Debug session with `PARDES_REPLICA_TRACE` is no soak:** its per-field
+ digests rehash every blob at every unit (`walk.rehash`, Debug's check),
+ a whole undo history of a big file a unit, and a boundary took 62 s.
+ Soak ReleaseSafe, or Debug without `--trace`.
+- **std's segfault handler panics while it symbolizes** (H8's S3:
+ `SelfInfo.Elf.findModule`, index out of bounds), so a crash record says
+ that and not the segfault. It is std's, not pardes's (`crash.zig` takes
+ panics only).