diff options
| author | Gabriel Schneider <[email protected]> | 2026-10-31 06:28:48 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-10-31 06:28:48 -0300 |
| commit | a479792a1ab2b593642d7f648194bfa23ce01dba (patch) | |
| tree | 96ddee7d879a9084daa07db43b0b7910b072a785 /docs | |
| parent | e179cc44fe5060c76a5b00c41e53f11209f5acd9 (diff) | |
| parent | 4c761893e5b3fdae422622cfa02104ab36633045 (diff) | |
| download | pardes-a479792a1ab2b593642d7f648194bfa23ce01dba.tar.gz pardes-a479792a1ab2b593642d7f648194bfa23ce01dba.zip | |
Merge the upstream daemon and replicated client/server refactor
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/helix-keys.md | 2 | ||||
| -rw-r--r-- | docs/next-daemon.md | 4546 | ||||
| -rw-r--r-- | docs/shared-libs.md | 1 | ||||
| -rw-r--r-- | docs/typ/building.typ | 79 | ||||
| -rw-r--r-- | docs/typ/builtins-gui.json | 5 | ||||
| -rw-r--r-- | docs/typ/builtins-tty.json | 5 | ||||
| -rw-r--r-- | docs/typ/cheatsheet.typ | 5 | ||||
| -rw-r--r-- | docs/typ/guide.typ | 27 | ||||
| -rw-r--r-- | docs/typ/reference.typ | 80 | ||||
| -rw-r--r-- | docs/web.md | 4 |
10 files changed, 4686 insertions, 68 deletions
diff --git a/docs/helix-keys.md b/docs/helix-keys.md index d47f3051..04aa60dd 100644 --- a/docs/helix-keys.md +++ b/docs/helix-keys.md @@ -122,7 +122,7 @@ language-backend queries, and the shell pipe. | `y` | yank each range's text into the register (`"<reg>` names one, else `"`), one value per range | line-yank vim-ism removed (phase 5); yank keeps selection AND cursor (helix). The system clipboard is `"+y` / `SPC y`, helix's own split: a `d` of one character never clobbers what the desktop holds | helix-verified | | `u` / `U` | undo / redo | restores the pre-edit selection (helix); snapshot granularity, `Alt-u`/`Alt-U` are the same two: helix walks a history tree by time with them, and pardes's history is linear, so earlier and later ARE undo and redo | helix-verified | | `p` (normal) | paste the register after each range: value i at range i, the last value repeated when there are fewer | helix `paste_impl`. `"+p` asks the system clipboard (`SPC p`); on a tty that read is OSC 52, which most terminals refuse: an honest no-op rather than a paste of the wrong text | helix-verified | -| `Esc` (body normal) | clear a pending modal prefix / exit select mode, keeping the selection, then run `Last`: hop to the pane you were in before this one, whichever kind it was, exactly like `SPC j j` — so held down it alternates between two panes, two files as readily as a file and its shell. A PDF pane is the ONE exception: there Esc is the document's own cancel (drop the mouse selection and the search overlay, stay where you are reading) and `Shift-Esc` is the hop out, while raw tty takes unmodified Esc as `Last` and sends Shift-Esc to the program (with `ProgramEsc on`, Esc goes to the program except at a detected shell prompt) | Pardes-specific focus binding layered on helix's cleanup. A leader path, tag, topbar or search owns Esc while it is active; raw tty forwards it | pardes-specific (cleanup helix-verified) | +| `Esc` (body normal) | clear a pending modal prefix / exit select mode, keeping the selection, then run `Last`: hop to the pane you were in before this one, whichever kind it was, exactly like `SPC j j` — so held down it alternates between two panes, two files as readily as a file and its shell. In a PDF pane Esc first drops the mouse selection and the search highlights when the page shows any, staying where you are reading, and the next Esc hops; `Shift-Esc` hops at once. Raw tty takes unmodified Esc as `Last` and sends Shift-Esc to the program (with `ProgramEsc on`, Esc goes to the program except at a detected shell prompt) | Pardes-specific focus binding layered on helix's cleanup. A leader path, tag, topbar or search owns Esc while it is active; raw tty forwards it | pardes-specific (cleanup helix-verified) | | `Esc` (insert) | back to normal mode, cursor right after the insertion (no vim left-step) | | helix-verified | | `Enter` (normal) | acme **look** chord: EXPLICIT selection, else file-ish word under cursor | pardes-specific, keep (helix normal-mode Enter unbound). Covers helix `gf`. Implicit motion residue falls back to the cursor word | pardes-specific | | `Tab` (normal) | acme **execute** chord | pardes-specific, keep; explicit-selection rule as Enter | pardes-specific | diff --git a/docs/next-daemon.md b/docs/next-daemon.md new file mode 100644 index 00000000..5537bf1a --- /dev/null +++ b/docs/next-daemon.md @@ -0,0 +1,4546 @@ +# The daemon: one pardes per machine, many clients per session + +> I want to have a global pardes daemon ... holding file descriptors for +> ptys, files, etc. ... like kakoune or emacs, or as in plan9 terms a +> cpu+file server ... the main test: multiple clients to a single session, +> and a client here and the session on freebsd and very deep and thorough +> performance testing and optimizations to make it feel like its local and +> snappy. + +Designed 2026-10-29 in discussion (the user, the coordinator, a survey +agent). Nothing here is built yet. Paths are relative to the repo. + +## Decided by the user + +- **A deterministic core, replicated.** The session is the authority: it + orders every client's input and every host result, and only it performs + effects. Each client runs a full core replica, applies its own input at + once, and rolls back and replays when the order differs. Visible rollback + is acceptable. +- **Kakoune-style clients.** Each client has its own view (layout, focus, + selections, scroll) over shared buffers and terminals. Registers are + session-wide. A terminal shown at two sizes is sized by its last typist. +- **Performance order.** First, the regular tty and GUI get no slower. + Then the detached and remote clients reach that local performance, with + the session on the FreeBSD box included. +- **A host step may show up one RTT late, but is never wrong.** This + covers Look, open and exec, as sam's hostlock does. +- **Join is complete.** The joining client pays for it in time; the + authority and the other clients do not. +- **One long-lived daemon per user per machine.** It supervises one child + process per session, postgres-style. Nothing is left dangling, and the + core is not de-globalised. + +## What today's seam is, and why it goes + +`src/detached/server.zig` is a third Host: it runs the core and the ptys +in a single poll loop and sends cell frames to the frontends, at the +smallest common grid across them. + +- **The frames are mostly not diffed.** Every layer's cells are re-sent on + each frame (wire.zig:734, :755). +- **Several things never cross the wire.** Images, PDF pixels, rail ticks + and pane animations stay behind (surface.zig:483-509). +- **Every keystroke is a round trip,** which no amount of optimisation can + make feel local at 30-80 ms. +- **Each remote 9P operation dials afresh,** costing about 6 RTTs + (9p_io.zig:2361-2385). + +## The design + +- **Processes.** + - The daemon is subreaper, spawner, fd holder, registry and 9P root. + - Every session is a child of the daemon: a GUI window, a tty session + behind a stub, or a headless session. + - A session is always an in-process authority with its own local view, + so the regular path does not change. Replicas exist only for extra + clients or remote ones. +- **The tty stub.** + - `pardes` in a terminal passes its terminal fd to the daemon + (SCM_RIGHTS), along with cwd, env and argv. + - The session child drives that fd directly. The stub forwards SIGWINCH + and exits with the session's status. + - Detach: the session drops the fd and the stub exits. No fork happens. +- **GUI detach.** The window is destroyed and the process carries on + headless. Closing a window that has no other client attached quits it. +- **Shells survive a session crash.** After each `forkShell`, the session + sends the daemon a dup of the pty master, so the shells do not get + SIGHUP. Being subreaper only decides who reaps them. +- **The recorder.** It is null when no client is attached. When one is, + it appends what the authority consumed, in order: + - every step's host tape (the results of the ~30 `turn.yield()` sites); + - each `serveFs` mutation. + + Turn order is log order, so the local path changes by one null check. +- **The protocol.** One ordered stream: events, host tapes, digests and + terminal state. Framing, the hello and the Event codec are kept from + wire.zig; the frames go. +- **Terminals cross as authoritative screen state,** not as pty bytes: + - metadata plus immutable row blobs, assigned in absolute terms; + - scrollback fetched on demand. + + A `cat` of 1 GB costs screens, not bytes. ghostty stays exact on the + authority, and the local path keeps reading ghostty directly. +- **The reflective walker.** One comptime `@typeInfo` walk gives: + - the digest; + - the complete dump, which also serves join, Restore and Handover; + - the rollback copy. + + Fields are annotated local, authority-only or derived. A hash of the + type graph versions it. The hello carries a semantics hash (version, + features, grammars), and a mismatched build is refused. +- **Rollback.** + - When the first speculative event goes pending, the replica copies what + speculation can write: views, buffers, registers, counters and terminal + metadata. Blobs are pinned rather than copied. + - On a divergence it restores that copy, then replays the confirmed + events and its own pending ones. + - The authority never rolls back. +- **Join.** The authority forks a writer child at a step boundary; the + child streams the dump. The authority buffers the events after it for + the joiner. +- **Remote.** + - `ssh -L` of the daemon's one socket. + - The daemon hands a replication connection's socket to its session. + - A remote daemon mounts as `/n/<host>`. + +## Determinism leaks (the survey's census) + +| Leak | Where | Fix | +|---|---|---| +| Render writes core state | body_layer.zig:181-187, File.zig:1923-1957 | view geometry in a lazy cache, render pure; the worst leak | +| 9P mutating between steps | pardes.zig:6060, pump :6016 | sequenced; turn order is log order | +| Host syscalls in a step | the ~30 `turn.yield()` sites | host tape | +| Wall-clock budgets | Names.zig:150, Places.zig:480, Tables.zig:519 | tape the work done | +| Ticks off the host clock | pardes.zig:6014/7850 | session ticks stamped, presentation ticks local | +| Clock and TZ in messages | Messages.zig:129-145 | event stamp, session TZ | +| Worker results polled in render | image.zig:1103, pdf_view.zig:1991, Stream.zig:482 | consumed at step boundaries, sequenced | +| getenv in the core (~30) | e.g. fs.zig:798 | env captured once into session state | +| Frontend kind / build flags | `windowed`/`native_images`; 223 comptime branches | moved into View; semantics hash | +| ghostty media reads, its pages outside the arenas | terminal.zig:629, PageList.zig:580 | authority-only under screen-state terminals | + +## Phases + +Each phase ships on its own and keeps `zig build perf --base` level (with +an A/A run). All 110 snapshots stay green throughout. + +1. **Determinism and the replay-divergence harness.** + - Work: the recorder and tape; the leak fixes, the render one first; the + semantics hash; the walker's digest. + - Check: replay each recorded log into two cores at different addresses + and one headless core. + - Gate: zero divergences on the snapshot scripts and on N monkey seeds. +2. **A complete dump on the walker.** It replaces dump.zig for Restore and + Handover. + - Gates: `digest(load(dump(x))) == digest(x)`, and dump is a fixpoint. +3. **The daemon and the tty stub.** + - Work: in-place detach and reattach; pty master dups; the frame wire + deleted. + - Gates: keystroke-to-present unchanged; a session `kill -9`'d leaves + its shells adoptable; no stale sockets after N cycles. +4. **Views.** A second window on the same session, in one process. + - Gate: logs with several views replay without divergence. +5. **Replicas.** + - Work: speculation, rollback, screen-state terminals, join, socket + handoff, `ssh -L`. + - Gates: + - a 30/80 ms delay relay keeps remote within noise of local; + - the rollback rate stays low under the fuzzer; + - a lagging client leaves the authority flat; + - a join costs the authority under one frame. +6. **Remote amortisation.** A content cache by hash, prefetch, zstd, + predicted echo in TTY mode, and session upgrade by re-exec. + +New measurements to add before phase 1 lands: keystroke-to-present in the +tty and the GUI, `yes | head -c 1G` through a real pty, and a big-file +open. + +## Still open (the coordinator's position in brackets) + +1. Delete the frame wire at phase 3, leaving no remote attach until + phase 5. [yes: it was never used daily] +2. The stream as its own framed protocol, or as a 9P file. [framed: one + ordered stream with no per-read round trip; 9P stays for scripting and + the relay] +3. Rollback by walker copy or by arena copy-on-write. [whichever + measurement of the mutable state's size decides] Decided: the walker + copy, about 100 KB and 0.05 ms once file texts are pinned (Phase 5 + groundwork notes). +4. How the daemon starts: spawning itself or socket activation. [itself, + by double fork; FreeBSD has no systemd. A user unit can be added if a + terminal's scope ever kills it] +5. The join writer: a fork child, or a metadata copy serialised on a + worker. [fork first, measure the page-table copy] Decided: the copy on + a worker. A fork of 1 GB stalls the authority 13 to 20 ms, and + copy-on-write taxes it for as long as the join streams (Phase 5 + groundwork notes). + +## Phase 3 notes + +Built 2026-10-29 (`src/daemon.zig`, the tty and GUI hosts, `main.zig`). +Decisions taken at the forks, with the ponytail lens: + +- **Sessions are not the daemon's children; every one registers.** The + design had the daemon spawn a terminal's session from the fd, cwd, env + and argv the stub passes. That proved wrong here: + - the user's terminals each run in a `9ns --mntgen` user and mount + namespace (`unshare(CLONE_NEWUSER|CLONE_NEWNS)`); a session the daemon + spawned would run in whichever namespace the daemon was started from, + its `/mnt/9p` served by a 9ns that may be gone. Its cgroup, rlimits, + umask and stdin (`pardes -`) would be the daemon's too, or have to be + carried and re-execed; + - startup would pay an IPC round trip and an exec of a 100+ MB binary; + - the snapshot harness's idleness probe walks the launched process's + tree, and a daemon-spawned session is outside it. + + So `pardes` in a terminal forks its session off itself before anything + starts (`daemon.forkSession`): the child calls `setsid` and draws on the + `/dev/tty` the parent opened; the parent stays as its stub. GUI windows + and `--detach` sessions just register. The daemon spawns nothing, so it + is no subreaper: a detached session is reparented to the system's reaper + (systemd --user or init), which reaps it, as it reaps a crashed + session's shells. Nothing dangles: the daemon lists every session (its + connection) and holds every master, and exits a second after it holds + neither. +- **Starting the daemon.** The first session takes the flock, binds and + listens itself and queues its own connection; `pardes --daemon` is then + double-forked and exec'd onto that listener and lock (fds 3 and 4), by + a terminal's stub after the session's fork, so no session waits for it: + its hello sits queued until the daemon runs, and is not waited on (the + daemon's `ok` is the reader thread's; Detach waits for it, up to 2 s). + With the spawn before the fork, a cold start measured +0.38 ms (+7.6%). + The daemon's reader thread likewise starts after the first frame. + Another starter waits on the lock; a stale socket is the lock holder's + to replace. Messages are `SOCK_SEQPACKET` datagrams, a line of words and + the fds it carries. +- **pty masters.** `forkShell` sends a dup; a host's close sends `close`; + a session's end sends `quit` (its shells hang up); a master whose shell + is gone (`POLLHUP`, polled with no events) is dropped. TERM, HUP and INT + handlers send `quit` too (one `send`, async-signal-safe), so only a + session killed outright or crashed leaves shells behind. +- **Adoption is a word, `Adopt`,** calling the host through a vtable entry + (no Effect): the daemon hands over every orphaned master, and each + becomes a pane through the `Tty` word, its spawn taking the pty instead + of forking; a 1x1 size then the pane's makes it redraw. Adopting at + startup was rejected: an `EDITOR='pardes --wait'` session would take the + shells and hang them up when the editor exits. +- **The stub.** It forwards WINCH, TERM, INT, QUIT and HUP by `kill`, and + ends with the session's status (a forked stub `waitpid`s; an attaching + one reads `x<code>`). A HUP is the terminal's end: kept by a daemon, a + session takes HUP on its signal thread (sigwait, beside WINCH) and + detaches, as on EIO; HUP never ends a kept session, since the terminal's + own EIO and HUP arrive in either order. Ctrl-Z stays a key for the pane + (the terminal is raw, as before); a TSTP sent to the stub is ignored, + since a stopped stub hands the terminal back under a session still + drawing; a background start stops at a `tcsetattr` before the fork, as + the old raw mode did. `--wait`, nested forwarding and `--tty` run before + any of this and are unchanged. +- **`--detach` is a session started detached:** the tty host with no + terminal, the GUI with a hidden window closed at once. With the frames + gone, `detached/server.zig` was a third host with nothing left to do + that the other two could not. +- **`Attach` (the word) is deleted:** it switched a session's own terminal + to another session through the frames. +- **Harnesses.** `PARDES_DAEMON=off` (the e2e harness sets it) runs a + session as before: in the terminal's own process, ended by its hangup, + for a harness that kills by pid and closes the terminal. A GUI test + window (`PARDES_TEST`) has none unless `PARDES_DAEMON=on`. +- **Known gaps.** A forked session takes its stub's death, a SIGKILL + too, as a HUP (PR_SET_PDEATHSIG; FreeBSD's PROC_PDEATHSIG_CTL) and + detaches; a `pardes --attach` stub is no parent of its session, and one + killed with SIGKILL while its terminal stays open still leaves the + session drawing there until the terminal closes or Detach. A + reattached terminal keeps the first one's environment (TERM, COLORTERM); + vaxis queries its capabilities again. A daemon killed outright leaves + its sessions unregistered, and Detach then refuses. A daemon of an older + protocol answers `version`, and a session then runs without one. A + GUI started in a terminal's foreground stays that terminal's job. + +Measured (ReleaseFast, base main a7369739 against these changes, both +live at once and their samples alternating A, B, A, B; A/A the base +against itself; p50 ratio new/base): + +| | tty | A/A | GUI (test mode) | A/A | +|---|---|---|---|---| +| startup, no daemon running | 1.017 (4.73 to 4.81 ms) | 0.998 | 0.982 (295 to 290 ms) | 0.999 | +| startup, daemon running | 1.014 (4.66 to 4.72 ms) | | 1.014 (noisy) | | +| key in a file, to its frame | 0.992 | 1.009 | 1.003 | 0.986 | +| key echoed by bash, to its frame | 1.016 | 1.005 | | | + +What is left of the tty's startup is the stub's fork (about 70 us), which +the design cannot do without: the job the shell waits on must be able to +end while the session runs on. + +Trap found on the way: glibc's `tcgetattr` writes its 60-byte `struct +termios` into Zig's `std.c.termios`, which is 44 (the kernel's layout), +and overran the stack, clobbering a neighbouring local in the forked +child; `std.posix.tcgetattr` (an ioctl on that layout) is the right one. + +## Phase 4 notes + +Built 2026-10-29 (`src/views.zig`, the tty and GUI hosts, `daemon.zig`, +the snapshot harness and the monkey). Decisions taken at the forks, with +the ponytail lens: + +- **A view is swapped in, not pointed at.** The ~1,900 references to the + core's layout, focus and pane state stay as they are: the core sees one + view, the live one, in the fields it always had. A view's share of the + session is a list of field names (`views.fields`, ~60) made into a struct + by one comptime `@Struct` (`Part`), and of each pane a struct beside it + (`PaneView`, in `Pane.pv`); going into view `k` copies the live one's out + (`park`) and `k`'s in (`unpark`). Copies, not swaps: a slot is stale + while its view is live and written whenever it goes out, so no two places + own a selection's heap. One view never parks: the single client pays + `views.several` (an AND and a compare) in the pump, the step's sync and an + edit, and one store a key (`typedInto`). Rejected: a `*View` the core + reads through (every reference rewritten, an indirection on every + access); a twin pane per view (Zerox exists only for files, and a + terminal's pty is its pane's slot). +- **Where each piece of state went.** + - Each view's own (`views.fields`): the screen size and cell metrics; + the columns, their weights, panes, serials, tags and the finder; rects; + `active`, `active_column` and the header focus; the jump list (kakoune's + is per client); the pointer, hover, drag, mouse grab, clicks and chords; + key prefixes (`"x` register name, leader, Ctrl-w, window pick); a + pending clipboard read; the presentation and the animations (lifts, + cursor glide); `windowed`, `native_images` and `native_image_patches`, + which the census had as watched host writes: a tty view on kitty and + one on a plain terminal differ. The 223 comptime platform branches stay + comptime: a session is one build, which the semantics hash says. + - Each pane's, each view's own (`PaneView`): `vweight`, `collapsed` and + `attached_to` (a layout's); `focus`; the body's mode, cursor, + selections, sticky column, pending normal command and find; a document's + scroll, `hscroll` and size; and what the step measured for that view + (`tag_rows`, `tag_top`, `tag_scroll`, `view`, `notices`), so a view drawn + without a step of its own draws its own geometry. + - The session's, every view's: the panes' text, undo, terminals and + their scroll (tmux shares a pane's copy mode too), tags (acme's are + per window; one tag per buffer is one text, its caret included), the + workspace tag, prompts, messages, registers (decided), the macro being + recorded, look-hover (one pointer at a time), mouse sweeps, query + panes, search state, settings and theme, the effect and input queues, + `needs_frame` (any change draws every view). A `.name` prompt (a + file's name drafted in its tag) is the view's that opened it + (`Pane.prompt_view`): only that view's focus leaving the pane drops it, + and its closing. +- **A terminal shown at two sizes** takes the size of the view that typed + into it last (`Pane.sized_by`, set by a key or a paste; a new pane's is + its maker's; a closed view's goes to the next to sync it). The others show + it at that size: `sync` resizes no pty for them, and `pane.cols/rows` + stay the terminal's own, so the terminal code's invariant (pane size is + the emulator's) holds. A view shorter than the terminal is passed down + its top rows as far as keeps its cursor's row in sight, as tmux shows a + bigger window in a smaller client (`Pane.pan`, terminal.panFor: per view, + made in the step and again when a view is drawn out of turn, read where + the body's rows start, `Pane.scroll` and the body walk), an arrow in its + gutter's first row saying rows are above; a shell's prompt, at the + bottom, is never out of sight. ponytail: a panned view's mouse reports to + the terminal's program and its kitty images are placed as unpanned. +- **Panes come and go under parked views.** A view keeps the serial of + each slot when it goes out; unparked, the ids whose serial changed leave + its layout and its keyboard moves to its first pane left. A pane focused + in a view that does not show it (a Look at a file another view opened, a + jump, a 9P focus) is placed in the view's active column (`views.show`, + kakoune's buffer shown in the window that asked). A text edited in one + view moves every other view's cursor and scroll with the splice, as a + Zerox twin's (`File.ParkedPlaces`); their selections go, as a twin's do. +- **Drawing the other views stays out of the digest.** The pump draws each + other view by unparking it, rendering and presenting, and restores the + live one exactly (`drawOthers`): no digested field moves, so the headless + replica, which draws nothing and skips it, agrees. What drawing a view + makes of its own (its animations, its presentation) is kept; what its + unpark normalised (a cursor clamped inside a changed text, a pruned + layout) is not, and is done again when it next goes live by an input, + in every replica alike. `parked` and `pv` are `.local`: a parked view is + checked when it is live again. Found by `pardes-monkey --views=3 --tape`: + a drawn view's `post_present` (acknowledgePanelPresentation, which moves + the pointer's hover) was applied by the headless replica to the live view; + other views take no post_present, their panels acknowledged when they + are live and drawn. +- **Inputs.** `toView`, `openView` and `closeView` are taped. A host goes + into a view before the input its client sent: the tty's terminal readers + post through a `Poster` that puts a `from` ahead of a terminal's input + whenever another terminal posted last (so one terminal posts none, and + coalesced motions never cross views); the GUI asks SDL which window an + event is for (`route`), which also takes a window's resize to its view + (the GUI sizes the live view from its window each frame). +- **The tty.** Each terminal is a `Screen` of its own: its vaxis, input + reader, frame answer (`FrameAck`, was global), kitty images and shm probe + (were global), glide, trail and flash, its stub and its Killed slot. The + primary's stub stays `daemon.stub`. A SIGWINCH asks every terminal its + size (attach stubs forward theirs; another view's terminal is no one's + controlling terminal); a SIGHUP asks each whether its terminal or stub's + pipe hung up, where it used to mean the one terminal. Detach, or a + terminal's end, with views left closes only that view and tells its stub + `d`; the session's end tells every stub. ponytail: a bracketed paste in + progress is the shell's, not a terminal's: keys from another terminal + during it join it. +- **The GUI.** One `Gui` (device, atlas, fonts, pipelines) and its + window's share swapped like the core's (`window_fields`: the SDL window + and its presenting, scroll and glide, scene and post chain, pointer, + blink, captures); `Shell.follow` keeps the Gui on the core's live view. + Closing a window while others stay closes its view; Restore closes the + others. ponytail: a second window's compositor hit test reads whichever + window's state is in the Gui (the workspace tag is in the same place in + every view). +- **The daemon.** `attach` names an attached session too: it takes another + view (a terminal, or a window). With no name, the detached sessions are + asked first, then the only session of that kind. +- **Harnesses, extended not new.** A snapshot script's `start ... views=N` + gives the app N-1 more ptys (PARDES_TEST_VIEWS, opened as attached ones + from the start), `view <k>` types into and captures terminal k, and every + wait reads all of them (a pty nobody reads would block the session's + frame); a `resize` there SIGWINCHes the app, as a stub would. + `pardes-monkey --views=N` switches among N terminals a quarter of its + steps, each with its own size; `zig build replay` runs its seeds at one + view and at three. The GUI's test feed takes `ESC]777;view;<k>BEL`, and + PARDES_TEST_VIEWS=N opens N more hidden windows, captured under view-k. +- **Not done, and why.** + - A view of the other kind (a terminal on a window session, a window on + a terminal's): each process links one frontend; the daemon still + matches kinds. + - 9P acts in the live view, the one whose client spoke last: no view is + named in a path yet (a script's focus or layout is that view's). + - Every change draws every view (`needs_frame` is the session's); a + view's own dirtiness waits for a measurement that wants it. + - Restore and `-l` keep one view, the live one; a dump has every view + (phase 2 notes), and Handover carries panes, which join the views of + the window they land in. +- **What proved wrong in the design.** "Views" in the design were a + struct the core reads through; the swap made them cheaper and kept every + reference. Its rollback copy (phase 5) is the same list of fields. The + design's view-local state assumed a step per view; the step's + measurements (`tag_rows`, `notices`, the pane's geometry key) had to + become per view too, or a view drawn after another's step drew the + other's tag heights. And the census's single "frontend kind" leak is not + per process but per client: kitty graphics are a terminal's. + +Measured: ReleaseFast, the base binaries from before the rework against +this line on phase 1's merge (vvmuwntp, before main's tty throughput +fix), interleaved base, new, base ... in one run on the perf cores, then +an A/A of the base against itself the same way; the ratio of the medians +of runs, new/base (4 runs of new, 5 of base; `--reps 25` for the core, +`--live` and `--live-gui` through the 32 MiB live driver): + +| | new/base | A/A | +|---|---|---| +| core, medium: render, keys, PageDown, wheel | 0.97-1.02 | 0.99-1.01 | +| core, medium: edit-char, edit-twin, deep-sel | 0.90-0.96 | 0.99-1.00 | +| core, medium: open, body-2m, body-appends | 0.94-1.02 | 0.88-1.08 | +| tty, keystroke to present p50 (5 gestures) | 0.94-1.11 | 0.90-1.02 | +| tty, the core's part of it p50 | 0.95-1.04 | 0.86-0.99 | +| tty, `yes \| head -c 32M` wall; 1M-line open | 0.96; 0.98 | 1.09; 1.02 | +| GUI, keystroke to present p50 (present-bound, 30 ms) | 0.98-1.01 | 1.03-1.10 | +| GUI, the core's part of it p50 | 0.84-1.01 | 1.01-1.19 | + +Every cell is inside its A/A; the widest, the tty's PageDown at 1.11 +(runs 386-506 us against 410-451), is 0.94 best against best. What one +view pays is a byte test in the pump, the step's sync and an edit +(`views.several`), a store a key, an index for the live terminal or +window, and a `from` never posted. A first A/B, of this line before the +rebase onto phase 1's merge (its render still made the geometry afresh +every frame), read the GUI's core at 0.89-1.16 with no A/A to set it +against; it is not the one reported. + +Two views (not gated): every frame draws both, so a change costs two +renders; a view switch copies ~60 fields and each pane's share, a few +microseconds. + +Gates, on the line rebased onto main's screen-state terminals (mnlwlpvk): + +- `zig build replay -Dplatform=tty`: 113 snapshot scripts pass (the 110, + views.snap, views-term.snap and views-pan.snap), their 131 tapes replay + in all three replicas (393 runs, 29,436 digests); 20 monkey seeds at one + view (60 runs, 14,004 digests) and 20 at three views (60 runs, 9,666 + digests); no divergence, no panic. views.snap: 75 digests a replica, + views-term.snap 35. core-test: 1117 of 1135, 18 skipped. +- Unit tests (on qzsouslm): tty 1478 of 1498 pass, 20 skipped; GUI 1549 of 1560, 9 + skipped and 2 failed, the same two that fail on phase 1's line (`tagline + pointer mapping retains physical grips...` and `a tag caret past its + row's end glides...`). The views' own: 8 (own cursor, scroll, mode, + layout and focus; an edit and a register shared; a terminal sized by its + last typist; views closing; the other views drawn leave the digest; a + view keeps its place in a text another edits; a short view keeps a tall + terminal's cursor in sight). views-pan.snap: a shell sized for the tall + view, its prompt and last lines in the short one under an arrow, then + typed into from the short one and sized for it. +- test/fs.py passes (with a TMPDIR short enough for the 9P sockets' + paths). test/daemon.py: 20 detach and attach cycles, then a second + terminal on the attached session, both drawn, detached from inside and + then closed outright, the first drawing on, and the kill and Adopt; + `--gui`: 10 window cycles, then a second window on the open one, + detached alone. +- `pardes-monkey --views=3 --tape` found the one divergence fixed here + (the drawn views' post_present). + +## FreeBSD notes + +Groundwork for phase 5's main test, a client here and the session on the +FreeBSD box (2026-10-29): the tty pardes is built here for FreeBSD and +runs there. The GUI is not built for it, since frontends stay local. + +The box runs FreeBSD 15.1 amd64 at 65.109.81.103, 207 ms away. Port 22 is +the host's root shell; port 2222 is a git jail and is never touched. +Everything of ours lives in `/root/pardes-test`, nothing outside it. + +- **Building.** Everything, the C deps included, cross-compiles here: + + ```sh + export PATH=$HOME/.local/bin:$PATH + E='env -u PARDES_PID -u PARDES_PANE -u PARDES_9P' + $E zig build -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 + # zig-out/bin/pardes-freebsd-x86_64 + $E zig build test-install -Dplatform=tty -Dtarget=x86_64-freebsd.15.0 >log 2>&1 + # zig-out/test/{main,panes,kitty,9p,fonts,...}, named for their root files + ``` + + The `.15.0` matters: Zig's default FreeBSD (14.0) links + `libutil.so.9`, and 15 ships only `.so.10`. `test-install` installs + the test-build programs instead of running them. A test binary takes + no filter when it runs, so running one test alone means a rebuild + with `-Dtest-filter=<name>` (and `--prefix <dir>` to keep the full set). +- **Deploying.** `-F /dev/null` is required here, because /etc/ssh is + owned by nobody: + + ```sh + S='-F /dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o UserKnownHostsFile=/home/goblin/.ssh/known_hosts -i /home/goblin/.ssh/id_ed25519' + zstd -q -f -T0 zig-out/bin/pardes-freebsd-x86_64 -o p.zst # 108 MB Debug; ~11 s in all + scp $S p.zst [email protected]:/root/pardes-test/bin/pardes.zst + ssh $S [email protected] 'cd /root/pardes-test/bin && zstd -q -d -f pardes.zst && rm pardes.zst' + ``` + + The same steps are scripts in `.scratch/daemon/bsd/`: `zb.sh <log> + <args>` (a build, PARDES_* stripped, logged there), `deploy.sh <file> + [name]` and `bx.sh '<command>'` (a command run there with env.sh + sourced). `sshcheck.py` drives a session through `ssh -tt` from a + local pty, and `fwd9p.py` times 9P through `ssh -L`. +- **The box's layout.** Under `/root/pardes-test`: + - `bin/`: pardes, and `python3` linked to python3.12; + - `run/`: the runtime directory (sockets, the daemon's lock); + - `home/` and `tmp/`; + - `src/`: the tracked tree, which the tests need: they read docs/typ, + and make their temporary directories in `.zig-cache/tmp` under the + working directory; + - `test-bin/test/` and `logs/`; + - `env.sh`, which points every path pardes uses there (HOME carries + the XDG defaults): + + ```sh + T=/root/pardes-test + export HOME=$T/home XDG_RUNTIME_DIR=$T/run TMPDIR=$T/tmp + export PATH=$T/bin:$PATH SHELL=/usr/local/bin/bash TERM=${TERM:-xterm-256color} + ``` +- **Running.** + - A session: `ssh -t $S [email protected] '. /root/pardes-test/env.sh; + pardes notes.txt'`, and `pardes --attach=<name>` the same way. + - The unit tests: `cd /root/pardes-test/src && ../test-bin/test/main`. + Some tests name `/tmp/pardes-*` outright and leave seven empty + directories there (`pardes-arg-save` and the like): rmdir them after. + - The e2e scripts, from `src/`: `python3 -B test/daemon.py + /root/pardes-test/bin/pardes 20`, and `test/fs.py` the same way. + - Kill every process you start there by its pid, and check `ps -axww` + afterwards. +- **What FreeBSD does differently.** Each item was found on the box, and + each is fixed where pardes met it unless it says otherwise: + - A write to a terminal that is gone fails with ENXIO, not EIO. std + does not expect that errno, and a Debug build traced it to stderr, + which is the same dead terminal, recursing until the stack ran out: + a session whose ssh dropped hung. Tracing is off on FreeBSD + (main.zig). + - A signal whose action is to be ignored is dropped when it is sent, + sigwait or not, so WINCH never reached the session's watcher. A + handler that does nothing keeps it (tty.zig). + - waitid's P_PID is 0, WEXITED 16 and WNOWAIT 8, and its id is 64 + bits. With Linux's numbers a command pane's exit was never seen. + - There is no /proc. A shell's directory and program come from sysctl + instead (kern.proc.cwd, kern.proc.pathname). + - `ttyTaken` answers taken for any foreground group but the + shell's, so a nested shell counts as a job. + - `signalSession` stays Linux-only, so Kill leaves a command's `&` + jobs running there. + - File watching uses FreeBSD 15's inotify: Linux's events, with its + own IN_CLOEXEC and IN_NONBLOCK. + - `shm_open` refuses O_NONBLOCK; its names are not files, so there is + no FIFO to fear. A `write` does not grow a shared memory object. + - A kitty `t=f` naming a FIFO hung the open everywhere but Linux; a + stat by path now comes first. + - A Unix socket whose listen queue is full refuses a connect with + ECONNREFUSED, exactly as a socket nobody listens on does, so a sweep + that judged by connecting could remove a live session's socket while + its queue overflowed. A session's liveness is now its lock: it holds + `<socket>.lock` flocked from before it binds until after it unlinks + (9p_io.zig `claim`), the sweeps and `taken` ask that lock, and they + unlink only under it. A connect judges only a socket with no lock, one + of a pardes from before the locks. The daemon's own start was already + decided by its flock. +- **The tests there.** The box runs them as root, with no zls, fish or + fonts installed. + - main: 1139 passed, 36 skipped, 3 failed: the LSP format retry (no + zls), and two tests that make a file or directory unreadable, which + root reads anyway. + - panes: 224 passed, 1 failed (`Shell fish`: no fish). kitty: 43 + passed. The small programs all pass but fonts (no fonts). + - Off Linux, the parts of tests whose refusing path is Linux's + /proc/1 are skipped, and so is the builtins JSON check, since the + committed lists are Linux builds' (Tty9p included). + - test/daemon.py passes there at 20 cycles, its killed stub included + (procctl's death signal). One earlier run failed at + cycle 3, one process short in its count (most likely ps -e failing to + read an environment). + - test/fs.py runs past its `pardes -` +Pager checks now, as far as the + SIGTERM and SIGHUP of a `--tty` stub: that test stops reading the pty + after the first frame, FreeBSD's pty fills before the session's frame + and reset are written, and the session waits in `ttyout` (a binary + from before these fixes too). Open. +- **At 207 ms, for the replicas.** + - In today's model every key is a round trip. Measured through `ssh + -tt` (min, median and max): + - a key typed into a file was on screen in 213, 229 and 346 ms; + - a key typed into a shell pane in 218, 219 and 224 ms. + + That is one RTT and about 15 ms. + - 9P over `ssh -L` of the session's socket costs one RTT a message: + 670 ms to connect, 1035 ms for a file read (walk, open, read, read, + clunk), 830 ms for a ctl write. A replica cannot fetch its state + file by file. + - The daemon's socket is SOCK_SEQPACKET and carries fds, so `ssh -L` + cannot forward it (tried: EPROTOTYPE here, connect failed there), and + no fd would cross anyway. The replication connection has to be a + stream of its own, as the 9P socket is, which the daemon hands on + locally. + - A dropped connection is a hangup: the session detaches, and + `pardes --attach` takes it back, as after a Detach. + - sun_path is 104 bytes there, not 108. + +## Phase 1 notes + +Built 2026-10-29 (`src/walk.zig`, `src/tape.zig`, the hooks across the +core; `pardes-snap --tape`, `pardes-monkey --tape`, `zig build replay`). +Decisions taken at the forks, with the ponytail lens: + +- **The tape is a tree recorded at the host boundary, both ways.** Each + call the host makes into the core (`tape.Input`: update, serveFs, pump, + perform, the shell's acknowledgements and failures, a Restore...) is + recorded with its arguments; each call the core makes out -- a Host + function, wrapped generically over the vtable's fields, or a syscall of + its own (`tape.site`, which yields around it; every old `turn.yield()` + is one) or a read of the clock or environment (`tape.value`) -- with its + result. `done` and `ret` close each, so a 9P request served while a + step is out lands inside that step's syscall: turn order is tape order. + A replica drives the same functions and its host answers from the tape; + one that makes a call the recording did not diverges on the spot, + before any digest. `turn.yield` panics in a replica, so an untaped + syscall is a stack trace. +- **The replay is a mode of the same binary** (`PARDES_REPLAY=<tape> + pardes`), not a harness of its own: an error travels as this build's + number, and the code replayed is the code that recorded. Three replicas + run one after another: two that draw (the first's memory held while the + second runs, so it lies elsewhere) and one that never renders. The + headless one steps over the presents it does not make but takes what + the host told the core inside them (`acknowledge_panel_presentation`). +- **Fields a host writes between calls** (`native_images`, + `present_skipped`, `active`...) are found by a diff of a short list at + every crossing, not by touching the frontends. +- **The digest** is taken after every call the host made at the top (a + pump, in the tty), as one per field of the core, its stream and each + pane, its file and places, so a divergence names the field. + `PARDES_RECORD_EVERY` digests after every input, nested ones too, to + find the step one starts at. +- **The walker needed a fourth annotation, `len`**: a fixed array written + only as far as a length field says (`jumps`/`njumps`, a message's text) + is undefined beyond it, 0xAA in a safe build and a fault where it holds + a slice. Integers are hashed by value (a `u21`'s padding bits are not + its value). +- **View geometry** is `File.geometry`, a lazy cache keyed by a hash of + what it is made of: revision, length, path, scroll, hscroll, the pane's + size, tags and query, tables as far as read, context declarations, the + output origin, and the pane's `view` (wrap, layered context, row + metrics, rect height), which `sync` sets. A hundred readers a frame ask + for it, so the key is one hash of those facts laid out as words (fed + field by field it cost 4% of every frame). A step reading it (`wrapAt`, + `wrapRow`, `ensureCursorVisible`, a body drag, `zz`) makes it on demand, + and the render draws from what the step made: made afresh for every + frame as well (to show an input the key missed only as a headless + divergence), it cost a scroll a third walk of the rows, 4% of a + PageDown. An input the key misses shows as a stale frame in a golden. +- **Render wrote more than the census found**, all moved into the step: + context declarations, the notice bands and their chips' columns (a + click on a chip was routed by the last frame's), the header's caret + reveal (`tagnav.golden` moved: after `g l B B` typed in one burst the bar + stays scrolled, as a human typing it sees it), the tables' reading (a + first look of 4096 lines, a count, wherever it is asked; the rest a + frame's taped clock budget a tick), the tty filter palette and body + chrome (derived). A view's paint-ahead stays the render's own prefetch: + instead of queuing an effect it is asked of the host once the frame is + presented (`Stream.askAhead`, in the pump), so a replica that draws + nothing asks for none; query rows' paints count apart from it. (Asked + inside the render, its worker's thread start cost a held `j` 10% in the + live tty measurement.) +- **Clock.** `advance`'s wakes are the recording session's + (`tape.value(.wake, nextWake)`): a picture decoding or a fade asks a + frame, and a replica that draws none must still take the session's + ticks. The budgets of Places, Names and Tables are taped verdicts + (`tape.over`), as are a message's time of day (TZ included), File.mtime + and the pump's clock read (made whether or not it waits). A message's + stamp and an unattended session's expiry of it read the clock through + the tape too, not the session clock: that is the last pump's, and + stamped by it a message written over 9P after an idle spell was as old + as the spell and went at once (fs.py's `unattended_messages_expire`). +- **A pointer seen through a sliding pane** is taped + (`Pardes.pointerCell`): what was under it is the frame the user saw. +- **The host's facts**: the session directory (getcwd, asked once at + init), `$SHELL` and whether it is there, `$HOME` for `~`, a `Shell` that + is not there, a missing interpreter, a directory refused above, a file + deleted on disk (`File.deleted` is an input). The rest of the core's + getenv (PARDES_DUMP and the XDG dirs in dump.zig, TMPDIR for kitty + media, PARDES_TEST_CLOCK in the render) is the host's or authority-only. + Per read rather than "captured once into session state": less code, and + the same for a replica. +- **Authority-only**: worker futures and their Io (a future's presence + too), the 9P server's state (`fs`: fids, opens, the log; serveFs itself + is replayed in turn order, 9p_io's own writes into `fs` are not), the + store file every window of the user shares, which a replica keeps in + memory, and Recent read from it. **Local**: the surface, presentation, + frame stats, animations, the effect and event queues (drained by every + pump before its digest). **Derived**: the geometry, notices, highlights, + line index, tables, node trees, rail ticks, a compiled regex, an image's + decoded pixels, PDF rasters. +- **Terminals** replay from their taped pty bytes (they are `output` + events) and are digested as their active screen, cursor and scroll + numbers; their kitty images are left out until phase 5. +- **Worker results**: the census's render-polled ones (image decode, PDF + lanes) are derived state; the rest already arrive as events. + +- **Unit tests that mutate and render** without a step now step + (`p.sync()`) first, as every host does; three of them had passed on the + last frame's stale state: a prompt chip a body click had already closed, + a `.center` landing measured on no wrap map at all (it centres by lines, + so under wrap a line above can now push the cursor down where + `ensureCursorVisible` takes the view back: that test is unwrapped now), + and a context row count a scroll had not refreshed. The perf gate's + `deep-sel` cell was one more: its view, placed from the last frame's + map, stopped six lines short of the 2 MB line it selects in. In view, + that line showed `File.rawAtDisplay` measuring a line's whole rest + before walking to a column (20 ms a frame, Debug); it walks to the + column only now, and the cell is the baseline's (91 us against 106). + +What proved wrong or missing in the design: +- The census missed the host writes between calls, the pointer through + the presentation, the render-time writes above, `Stream.start` (host + code doing core work, made harmless by the futures being authority's), + getcwd, `access(2)` for `$SHELL`, File.mtime's clock, and a deadlock in + vaxis's queue (a `yes` stopped being read; fixed on this line, found by + the new measurement). +- `windowed`/`native_images` are watched host writes for now, not View + state (phase 4); the semantics hash in the hello is not done (there is + no hello in phase 1), so a tape from another build is not refused yet. +- Deferred: the GUI's own host-to-core calls (font requests, opacity, + `core.advance`/`render` are inputs, the rest not yet), PDF lanes and + Handover yields (never reached by the gate; a replica would panic on + them, which is the point), `Stream.runHere`'s yield (a host with no + workers). + +The gate on the final line (on main vukknslx): `pardes-snap --tape`, +110/110 scripts, 128 tapes, 384 replica runs, 33,093 digests, no +divergence; `pardes-monkey --tape`, 50 seeds of 70 steps, 150 replica +runs, 37,800 digests, none; `zig build replay` (the 110 scripts and 20 +seeds) clean; `snap` 110/110 with no golden changed but `tagnav`; +unit-test tty, pane-test, 9p-io-test and fs-test pass, unit-test gui +fails only its two known tests. Perf, base (main with perf --live and +the queue fix) against this line, ReleaseFast, interleaved A B A ... A, +medians of runs, the base's own spread as the A/A noise. The perf +harness's gesture cells (render, keys, PageDown, wheel, edit, deep-sel) +0.97-1.03 at an A/A of 2-7%, and on the final line, measured on the +shared perf cores, 0.91-0.97 at 10-31%. The live tty on a quiet machine: +core time p50 0.96-0.97, keystroke-to-present p50 0.98-1.03 (A/A 1-23%), +`yes | head -c 1G` 0.96, a million-line open 1.01. The live GUI's core +p50 1.00-1.04 (A/A 2-18%), present-bound at the window's 30 ms. Two +costs the A/B found are gone (the geometry key, the paint-ahead's +thread start in the render). + +## Tty throughput notes + +Found and fixed 2026-10-29 (`src/tty/tty.zig`, `waitInput`). + +- **What the 1 GB `yes` cell measures is mostly the kernel.** A pty's + output processing (ONLCR, a `\n` made `\r\n`) costs the writer about + 0.4 us a newline here, and `yes` writes one every two bytes. Every reader + of a pty is capped by it: `script(1)` drains 100 MB of `yes` at 8.6 MB/s + with OPOST on and at 650 MB/s with it off, a Python drain at 8.0 MB/s, + and tmux at 5.4 MB/s. The tty pardes was at 5.7 to 6.1 MB/s, three + quarters of that ceiling, before any change. The 179 s were the kernel's. +- **Where pardes itself was slow:** with 200-byte lines (`yes` of a long + word), the ceiling rises to 264 MB/s, tmux reaches 55 MB/s, and pardes + managed 18.6 MB/s. A flat profile (perf at paranoid 2 allows no call + graphs) put 98.7% of the samples in pardes: `Vaxis.render` 29%, + `body_layer.renderBody` 17%, `Shell.present` 10%, memcpy 8%, + ghostty's parse 1.5%. The tty drew a frame for every pty read, which + is 4 KB, nearly all of them overwritten before anyone could see them + (about 690k frames for the 1 GB cell). +- **The fix: output is drawn once per frame tick at most** (16 ms), with + whatever came in until then, as the GUI drains its queue once per vsync. + Two things are drawn at once, as before: + - an input's frame; + - the focused pane's output within a second of an input (its echo; a + remote shell's arrives an RTT later). + Neither waits while it is undrawn, including when FrameAck held its + frame for the terminal's answer. Snapshot runs (the virtual clock) are + not paced. +- **And a pane has two reads queued at most** (`readPty`): its reader + waits for the loop to take one before it reads on. ghostty parses `yes` + at about 9 MB/s, 0.45 ms a 4 KB read, slower than this machine's pty + delivers it. So a flood filled the loop's queue (512 events): a key and + its echo waited behind its reads, and which blocked writer a free slot + woke was luck, which gave half-second peaks. Paced but unbounded, the + wait was steady at about 20 reads, 9 ms for the key and 9 more for its + echo. A frame dump lined up with each key's write showed it (PARDES_FRAMES + and `flood.py` with FRAMES=). What a flood writes beyond two reads now + waits in its pty; the loop could not have kept up with it anyway. +- **Measured** (ReleaseFast, `.scratch/daemon/term/yes.py` through a real + 120x40 pty, all interleaved, with measurements pinned to two physical + cores and so four CPUs; medians of 3 rounds, MB/s): + + | | `yes`, 32 MiB | 200-byte lines, 256 MiB | + |---|---|---| + | base (before the rework) | 5.5 | 14.5 | + | phase 1's tip | 4.8 | 15.0 | + | pacing alone (the queue limit came after) | 4.6 | 117 | + | tmux | 5.4 | 47.6 | + | raw pty drain (Python) | 7.7 | 239 | + | GUI, test mode | 0.9 | | + + Unpinned, before the pinning: 200-byte lines 18.6 to 247 MB/s (1 GB in + 53.7 s and in 4.0 s), and `yes` 5.7 to 6.9 MB/s. The GUI could only be + measured in its test mode (a hidden window), which downloads every frame + from the GPU, so 0.9 MB/s says nothing of a real window. There is no + headless compositor here, and a real window must not open on the user's + session. In a later interleaved run, the change as committed against + pacing alone gave 4.8 and 6.0 against 4.8 and 5.4 MB/s for `yes`, and + 102 and 117 against 124 and 119 MB/s for long lines: the same within + the noise. `pardes-perf --live`'s `yes` cell drew 2.3k frames for 32 MiB + against 11k before, in the same time: it is kernel-bound. +- **A held key while another pane floods** (`flood.py`: a key every 30 + ms into `cat` in one pane of `-n 3`, timed from its write to the pty to + its echo on screen, with pyte in the loop; `yes` in another pane): + in one interleaved run of two rounds (p50, p90, worst, in ms): + + | | under the flood | idle | + |---|---|---| + | phase 1's tip | 3.0, 212 to 214, 445 to 470 | 2.2 to 2.3 | + | pacing alone | 10.3 to 10.6, 19 to 20, 26 to 33 | 2.3 | + | this change | 3.3 to 4.2, 4.3 to 5.6, 6.0 to 7.8 | 2.3 to 3.2 | + + The driver's own cost (pyte) is in every number. With 16 KiB queued a + pane instead of two reads, the flood's p50 was 7.5 to 7.8 ms for no + throughput gained, so two reads it is. +- **Keystroke-to-present** (`pardes-perf-live --live`, base and this + change interleaved, 5 and 4 runs; A/A between the base runs' halves): + p50 against base is 0.92 for key-down, 0.89 key-right, 0.85 edit-char, + 0.80 page-down and 0.93 wheel. The A/A was 0.73 to 0.92. The 32 MiB + `yes` took 1.00 the time with 0.18 the frames, and the big open 1.10 + (A/A 1.05). Two earlier runs of pacing alone, 3 and 5 rounds, were + within their A/A too. +- **Unrelated, found on the way:** `test/fs.py` failed + `unattended_messages_expire` (its first `shown()`) on phase 1's tip as + well; pmxpuolt on main (a message's stamp and expiry read the clock + again) is that fix. fs.py also needs a short TMPDIR: under + `.scratch/daemon/term/tmp`, `directory_panes`'s socket path is 108 bytes, + one past the limit, and the session starts with no 9P. Two GUI unit tests + fail on phase 1's tip as well (`tagline pointer mapping ...`, `a tag caret + past its row's end ...`). A suite started as a shell's `&` job runs with + SIGINT ignored, so four host_io tests whose Ctrl-C must stop a `sleep` + fail there; started otherwise, they pass. + +## Screen-state terminal notes + +Built 2026-10-29 (`src/screen_state.zig`), a pure encode and apply pair +over byte buffers for the phase 5 stream; no transport yet. Only tests +reference it, so the tty and GUI binaries carry none of it: no symbol of +it is linked into the tty built from this tree. The local path's A/B, base +against that tty, is the Tty throughput notes' keystroke-to-present. + +- **Both kinds of blob are ghostty's own snapshot records,** not a cell + format of ours. The terminal-wide state (modes, colours and palette, + title, pwd, tab stops, mouse modes, scrolling region) is a TERMINAL + record. A row is the PAGE record of a one-row page the row is cloned + into, which carries its styles, graphemes, hyperlinks, wide cells, wrap + and prompt marks exactly. The replica decodes them with ghostty's + decoders and writes a row with `Screen.clonePartialRowGrowCapacity`. +- **A row's address is a hash of what it shows** (Wyhash of its flags, + its width and its cells by value), not of its blob: ghostty keeps the + source page's style ids when it clones a row, so equal rows give blobs + that differ. The width is in the address, because a blank row is the + same at every width. +- **A delta** is a header (cursor, kitty keyboard flags, the active + area's first row as an absolute row, the history's extent), the TERMINAL + record's address, an address for each active row, an address for each + row that scrolled off since the acknowledged state (the tail, up to a + screen of them), and the blobs for the addresses the peer does not hold. + An unchanged row costs 8 bytes. +- **One delta per peer is in flight,** against the state it acknowledged, + as FrameAck keeps one frame in flight for the tty. A slow peer skips the + states in between. The peer holds exactly the blobs of the state it + acknowledged, at most two screens of rows and the meta, so the authority + knows what it holds without asking. At 207 ms this means at most about 5 + deltas a second to the FreeBSD box; phase 6's predicted echo is meant to + cover typing there. +- **Absolute rows** come from a tracked pin at the last active top. The + rows scrolled since are the distance to the new top; if the pin's page + was pruned, the count jumps past every row the replica could hold. A + resize renumbers (an epoch), and the peer starts afresh. Scrollback is + fetched by absolute row (`encodeRows`, `applyRows`). +- **The replica writes into a ghostty Terminal,** a terminal pane's own, + so `src/terminal.zig` draws it unchanged. A test renders a replica's + terminal pane and its authority's in two cores: the bodies are equal, + cell for cell. +- **Gates.** + - The soak drives an authority with pty bytes and lands deltas at + random, both skipping states and delaying a delta while the authority + moves on. Resizes and fetches of holes also come at random. After each + delta it checks the replica's screen against the authority's as + encoded, and every history row the replica holds outside its holes + against the authority's row at that number. + - It passed with synthetic shell, TUI and `yes` streams at 4 seeds (in + the unit tests, with a 64 KB scrollback so it prunes). It also passed + with 197 recorded streams at 3 seeds each (the test that + PARDES_SCREEN_STATE=<dir> turns on): the 192 pane outputs of the snapshot scripts' + tapes, plus real vim, nvim, htop, less and btop sessions recorded + through a pty. Its corpus and recorder are in `.scratch/daemon/term`. +- **Numbers** (ReleaseFast: `PARDES_SCREEN_STATE=<dir> zig build core-test + -Dplatform=tty -Doptimize=ReleaseFast -Dtest-filter=screen_state`, which + runs under std.testing's checking allocator): + - Encode, 200x60, every row new: p50 250 us, p99 487 us, 29.8 KB. + - Encode, 200x60, nothing new: p50 71 us, p99 139 us, 534 bytes. That + is the 60 rows hashed and the TERMINAL record encoded to hash it. + - Under `yes` as fast as ghostty takes it in-process (9 to 12 MB/s), + with a delta every 16 ms: 62 deltas a second, 63 KB/s on the wire, + 1 KB a delta (60 rows and the tail's 60, all held). That is bounded by + the frame rate, not the output. + - Under varied text at 36 to 55 MB/s: 355 KB/s on the wire, 5.6 KB a + delta. + - The replica applies a `yes` delta in 2.4 ms, most of it pushing the + rows scrolled past into its history (a row at a time). A delta of + varied text applies in 0.24 ms under the C allocator and 0.95 ms + under the checking one. +- **What proved wrong in the design.** + - "Scrollback fetched on demand" alone loses rows. A row that changed + after the last delta and then scrolled off would sit stale in the + replica's history. The tail fixes the rows that scrolled within a + screen. Older ones, which the replica never saw or saw before they + last changed, are holes: rows it must fetch before it shows them. + - The address had to include the row's width, as above. The soak found + this: a replica reused an 80-column blank row after a resize to 10. + - ghostty narrows pages lazily, so an active row can be wider than its + terminal: only the terminal's columns are hashed and encoded. +- **Not done, or a gap.** + - Kitty images in terminals. + - pardes's own state derived from the stream is not carried: its + `MarkedHandler` prompt phase, command-output pins and the replies it + writes to the pty. ghostty's prompt marks on rows do cross. + - The view's scroll offset is per client and is not sent. + - A replica's core must not resize its pane's terminal itself; the + authority sizes it. + - The replica's history extent follows its own pruning under the + authority's limit, so it can be a page off, and its oldest rows can + be ones the authority has dropped. + - A fetch sends every row as a blob (phase 6's cache will send + addresses). No zstd. + - A replica pushes a flood's rows into its history one at a time. + - The integration into the stream and the cores (a replica pane's + State fed by `apply` instead of `.output`) is the next piece. + +## Phase 5 groundwork notes + +Built 2026-10-29: the delay relay (`test/delay_relay.py`), the replica +socket (`src/daemon.zig`, `src/detached/wire.zig`), and two measurements +for open points 3 and 5 (`zig build rollback-bench`, which runs +`test/rollback_bench.zig`, and `test/fork_stall.py`). Decisions taken at +the forks, with the ponytail lens: + +- **The delay relay is a Python script.** Each direction is a reader that + stamps every chunk with a due time and a writer that sends it then. The + due time is the chunk's arrival plus the delay. With a rate cap, it is + the moment the link has carried what came before, plus the delay. So + throughput is not serialised behind the delay, and the bytes stay in + order. Either side is a Unix or TCP socket. ponytail: its buffer is + unbounded, which makes a link with an infinite bandwidth-delay product; + on a real link, ssh's window is the bound. +- **Two sockets, where the design had "the daemon's one".** The + fd-carrying SEQPACKET socket stays local. Once the daemon holds its + lock, it binds `pardes-replica.sock` (SOCK_STREAM) beside it, and + unlinks it at exit. `ssh -L` forwards that one. +- **The daemon judges the hello, so a session never sees one it + refuses.** + - Every session's registration carries its semantics hash (daemon + protocol 2), and the daemon compares the client's with the session's. + - It refuses in words: another protocol; another build, naming both + hashes and pointing at `pardes --version` to compare them; or a + session nobody has. + - It reads exactly the hello's length (the header, then the payload), + so whatever the client sends after it is left for the session. + - Up to 16 hellos may be on their way at once. ponytail: a client that + never sends one holds its slot until it hangs up. +- **The welcome comes from the session**, with its pid and name, over the + connection it was handed. That tells the client the handoff happened. + The test checks that the session's fd count went up and the daemon's + did not. +- **The semantics hash** (`wire.semantics`) is computed at comptime and + covers: + - the protocol, the release and the commit; + - the platform and the build options; + - every dependency the manifest pins, by its content hash (build.zig's + `deps` option). The design said grammars, but ghostty and the + Unicode tables also make a core step differently, so it takes all of + them; + - the walker codec's assumptions (pointer width, byte order). + + It leaves out the OS. The main test is a Linux client of a FreeBSD + session, and what differs between those is the host's, which the tape + carries. The two builds of one tree hash alike (1087257bf166588a on + both), and the box's session welcomed the client from here. ponytail: + an unstamped build (Debug with `-Dplatform`, so every test build) has + no commit, so two dev trees with one release and the same options hash + alike. +- **The hello lost its grid**: a replica's view takes a resize like any + other input. A ping (client tag 0x03) and three server tags (welcome, + refused, pong) are new. The wire is version 14. +- **The session's half is a stub**: a thread per replica that sends the + welcome, answers pings, and ends at bye or a hangup. + +Measured. Timings ran alone through the perf lock, pinned to cores 6-7 +while builds ran niced on the others. The local A/B ran interleaved, with +an A/A run beside it. + +- **The relay's self-check** added this much over a direct connection, + at each delay (median RTT, then throughput through it): + + | one-way delay | wanted | added | throughput | + |---|---|---|---| + | 15 ms | 30 ms | 30.26 ms | 3.2 GB/s | + | 40 ms | 80 ms | 80.27 ms | 2.3 GB/s | + | 103.5 ms | 207 ms | 207.26 ms | 2.3 GB/s | + + Direct, the same pair gave 9.9 GB/s. The extra 0.13 ms each way is + the threads' wake-ups. A 20 MB/s cap held at 19.98 MB/s. +- **A replica's round trips.** Each row is a connect and hello, then 50 + pings, to a detached session of this build. The session's thread + answers each ping. + + | path | hello | ping min / median / p90 | + |---|---|---| + | direct (Unix socket) | 0.26 ms | 0.01 / 0.01 / 0.01 ms | + | relay, 30 ms RTT | 30.95 ms | 30.26 / 30.34 / 30.40 ms | + | relay, 80 ms RTT | 80.73 ms | 80.25 / 80.31 / 80.39 ms | + | relay, 207 ms RTT | 207.62 ms | 207.22 / 207.35 / 207.41 ms | + | FreeBSD box, `ssh -L` | 411 to 476 ms | 203.2 / 204.9 to 210.4 / 210 to 221 ms | + + The box's rows cover six runs over two passes. In the same passes, a + TCP connect to the box's port 22 took 203.8 to 205.3 ms at best and + 206.4 to 208.3 ms at the median. The relay rows are the second pass; + the first pass had p90s up to 5 ms worse. So a ping through `ssh -L` + and the daemon's handoff costs the RTT and nothing more measurable. + The hello costs two RTTs: + ssh confirms the channel before it forwards a byte. A refused hello + over the link came back in 412 ms, also two RTTs. That is the floor + phase 5 builds on. A replica connects once, so it pays the second RTT + once. +- **Open point 3, rollback: the walker copy.** `zig build + rollback-bench -Doptimize=ReleaseFast` builds two sessions in process: + - typical: a shell with 1 MB of scrollback, three 60 KB files and a + 3 MB one, 40 edits; + - big: 14 panes, an 18 MB file, four 3 MB files and eight small ones, + 8 MB of scrollback, 200 edits. + + For each, it measures the state the digest walk visits (what + speculation can write) and the share of it that is blobs. Blobs are a + file's text and its undo snapshots. `File.setContentSpan` puts a new + buffer in place of the old one at every edit, and a snapshot is a + copy, so both are immutable once made and a copy can pin them. The + walk is then timed again with them taken out. + + | session | walked | of it blobs | left to copy | walk | walk, blobs out | copy (walk + memcpy) | + |---|---|---|---|---|---|---| + | typical | 36.3 MB | 36.2 MB | 80 KB | 3.1 to 3.8 ms | 0.019 to 0.023 ms | 0.020 to 0.024 ms | + | big | 282.5 MB | 282.4 MB | 96 KB | 23 to 27 ms | 0.043 to 0.044 ms | 0.044 to 0.045 ms | + + A copy is about 0.1 to 0.3% of a 16.7 ms frame, so the walker wins: + arena copy-on-write would buy nothing. Three conditions come with it: + - a blob a copy holds must not be freed until the copy is let go, + whether by a refcount or by a free list drained at confirmation; + - caches derived from a blob (`line_starts`, highlights) are also made + anew at each edit, so they can be pinned with it, or remade on + demand, since the geometry is keyed by revision; + - a terminal's history is not in the walk, because under screen-state + terminals it is the authority's pinned rows. + + The walk with the blobs out is the stand-in for a copy walk: phase 2's + dump walk is the real one, and this step re-measures it when it lands. +- **Open point 5, the join writer: not a fork.** The authority's stall, + as median fork time, with 64 MB written by the authority while the + child lives: + + | written memory | Linux 7.2 (THP always) | FreeBSD 15.1 | + |---|---|---| + | 100 MB | 1.8 to 2.7 ms | 4.1 ms | + | 600 MB | 8.6 to 10.8 ms | 14.0 ms | + | 1 GB | 13.3 to 19.6 ms (max 24) | 20.4 ms (max 26) | + | 4 GB (one Linux pass) | 62 ms (4 KiB pages: 162) | 52 ms | + | then 64 MB written under the child | 26 to 32 ms | 19 to 27 ms | + + Linux is two passes on the pinned cores, beside the builds; the box was + idle. The big session forked in process at 431 MB RSS took 3.4 and + 8.3 ms. The user's windows hold 450 and 630 MB. + + The gate, "a join costs the authority under one frame", fails twice: + - fork itself, at 1 GB on both systems (and at 600 MB once a frame is + 8.3 ms, at 120 Hz); + - copy-on-write after it. Every page the authority writes while the + child streams costs a fault, about 1.3 µs per 4 KiB page. The child + lives as long as the dump takes to cross, which is seconds at + 207 ms. + + So the join is the same walker copy as rollback: tens of microseconds + at a step boundary, blobs pinned. A worker thread then serialises the + copy and streams the pinned blobs, which are immutable and so need no + lock, while the authority buffers the events after it for the joiner. + This is the coordinator's alternative, "a metadata copy serialised on + a worker". The authority pays the copy, and the blobs stay pinned + while they stream. + +- **No local change.** tty startup and keys, the phase 1 tip (a4b85cb7, + ReleaseFast) against this, both live at once with their samples + alternating; the p50 ratio new/base: + + | | three A/B runs | two A/A runs | + |---|---|---| + | startup | 1.040, 0.965, 1.024 | 1.053, 0.978 | + | key in a file, to its frame | 1.084, 1.025, 1.027 | 1.025, 1.013 | + | key echoed by bash, to its frame | 0.996, 0.972, 1.010 | 1.015, 0.998 | + + The daemon and the hello are off the hot path, and every ratio is + within the spread of the A/A runs beside it, whose second slot reads 1 + to 5% slow against the same binary. So there is no measurable change. + The absolute times are not comparable with phase 3's 4.73 ms: the + startup p50 was 6.5 to 26 ms here, on two cores shared by both live + sessions and the driver, with builds running on the others. + +What proved wrong in the design: +- "`ssh -L` of the daemon's one socket": there are two. The one that + carries fds cannot cross ssh, and no fd would cross anyway. +- "The authority forks a writer child": fork's page-table copy is over a + frame at 1 GB on both systems, and the copy-on-write tax lasts as long + as the join streams. Once blobs are pinned, a copy of what speculation + can write is about 100 KB, small enough for a join to take one too. +- "When the first speculative event goes pending, the replica copies + views, buffers, ...": a buffer's text never needs copying, since every + edit replaces it. Only its handle and the views are copied. +- "The hello carries a semantics hash (version, features, grammars)": it + hashes every dependency, and leaves the OS out on purpose. +- Pre-existing on the phase 1 tip, and not this line's: test/fs.py's + `mounted-startup` session hangs at startup (its main thread waits on a + futex), the same with the base binary, and still on main after phase 4 + (Phase 5 part A notes). A ReleaseFast `exit_answers` run loses its + Rclunk race about 1 time in 7 on base, and 1 in 40 here. + +## Phase 5 part A notes + +Built 2026-10-29, in one process and with no transport: the rollback copy +(`src/rollback.zig`), the replica engine (`src/replica.zig`), the tape's +speculation mode (`src/tape.zig`), and an in-process harness +(`test/replicas.zig`, `zig build replicas`). `zig build rollback-bench` +now times the real copy. Decisions taken at the forks, with the ponytail +lens: + +- **The copy saves bytes and puts them back where they were.** Its walk + is the digest's (src/walk.zig): views, buffers, registers, counters and + terminal metadata. It saves the bytes of every allocation the walk + reaches, and puts each back over its allocation. That works because the + replica core's allocator (`rollback.Epoch`) changes three things while + a copy is held: + - a free of anything older than the copy is put off; + - growing such an allocation in place is refused, so the core moves it; + - anything allocated meanwhile belongs to the epoch. + + So every pointer in the saved bytes still points at what it pointed at + when the copy was taken. Putting the copy back frees the epoch's + allocations and forgets the put-off frees. Letting the copy go makes + them. Nothing has to know who owns what. +- **Three blobs are pinned instead of copied** (4 KiB and up): a file's + text, its line index and its undo snapshots. An edit puts a new one in + place of the old (File.setContentSpan) and never writes one, so the + copy keeps a reference and the put-off free keeps it alive. A safe + build checks every pin is unchanged when the copy goes. +- **More than the digest's fields are saved.** Within a struct the copy + saves every field: the derived caches, and this process's own state too. + It skips the queues a step drains (asserted empty) and the frame + statistics, and cuts an array at its `len`. Both were measured wrong + first: + - a safe build's `free` writes 0xAA over what it frees, so a cache the + speculation freed came back as that (a line index full of 0xAA, then a + crash); + - this process's own fields (the frame arena, the surface) point at what + the epoch then frees. + + For the same reason, file texts and line indexes are now freed raw + everywhere (`memory.freeRaw`), as setContentSpan already did. +- **A speculation is checked against the authority's tape of the same + input.** The replica applies its own input as the authority will (an + update, then a pump) and records it with its own tape. If the + authority's unit for that input has the same records (inputs, host calls + and their answers, digests aside), the head goes: that is the fast path. + The digests are left out of the comparison because the starting states + are equal by induction. A digest checks every unit a replica replays. +- **Each client has its own view** (src/views.zig). The authority goes + into the client's view before its input (`toView`, which is taped), and + the replica's speculation does the same. So a unit records the switch + only when another view was live, alike on both sides. The views not + live (`parked`, each pane's `pv`) are this process's own fields, and + the copy saves them with the rest. +- **A host call or a syscall stops a speculation, but cannot stop it + midway.** Zig has no unwinding. The call is answered with a harmless + value (an error of its set, null, zeroes, or a union's first zeroable + case), the input runs to its end, and then it is undone: the copy is + put back and the earlier pending inputs applied again. That input, and + those after it, wait for the authority's answer. A read of the clock or + the environment (`tape.value`) is a guess made locally, which the + comparison checks. `watched` is answered false, as the authority's host + says too: a replica draws outside the step. +- **An emulator is the authority's.** Its writers in terminal.zig (feed, + resize, scroll, follow, create, deinit) stop a speculation before they + write. A safe build asserts that a clean speculation left every + terminal's digest as it was. Tree-sitter's trees live on the C heap, + which the epoch cannot see. While a copy is held, `syntax.held_trees` + puts off their frees and frees the epoch's own. +- **The view's events wait** (a resize, the pointer): phase 4. +- **Messages are bytes.** A client's input is a u16 client id, a u32 + sequence number, and the event as detached/wire.zig frames it. A unit is + the input's origin (0 for the authority's own: a pty's output, a setup), + its sequence number, and the authority's tape of it. +- **The harness** is a discrete-event simulation. One authority and three + replicas share one view, each client typing monkey.zig's mix (keys, + typing, pastes, a few commands, resizes and clicks) at 50-200 ms with + pauses. Each link takes half the RTT plus up to 10% of it in jitter, in + order. The session is the tty's own boot (a raw shell over a text pane) + with three files looked at. The authority's pty echoes what it is + written 2 ms later, which makes units of its own. + +Found on the way: phase 1's replay ran Debug, whose 0xAA fill made every +read of unwritten memory alike in two cores. In a release build they read +what the heap had, and replicas diverged at once, from: +- `col_panes`, digested whole and undefined past each column's panes (now + zeroed); +- `restore_buf`, the same (now zeroed); +- `terminal.State.reply_dropped`, never set, since `create` leaves fields + as the heap had them; +- `Text.sels` and `PendingPipe.ranges`, digested past their length (a + `len` now cuts a slice too). + +`PARDES_REPLICA_FILL=0xff` fills a replica's allocations, so that what is +left shows at once. Any replica is a release build, so these mattered. + +Measured on main with views (lyntonns). The ReleaseFast runs were alone +through the perf lock; the correctness sweeps ran beside builds. + +- **Correctness.** Three clients each typed 100 inputs per seed, at RTTs + of 30, 80 and 207 ms. No replica diverged, and every replica ended on + the authority's digest, in all three runs: + - Debug: 12 seeds, a view each; + - ReleaseFast with `PARDES_REPLICA_FILL=0xff`: 20 seeds, a view each; + - ReleaseFast, timed: 20 seeds, a view each and in one shared view. +- **What happened to the inputs, and what it cost** (ReleaseFast, a view + each; times in µs, p50 / p99): + + | RTT | rollbacks per input (mispredicted + another's) | speculative apply | copy (1.3 MB) | rollback | + |---|---|---|---|---| + | 30 ms | 31% (106 + 1745) | 35 / 183 | 75 / 198 | 342 / 818 | + | 80 ms | 76% (257 + 4285) | 54 / 168 | 94 / 447 | 455 / 1318 | + | 207 ms | 129% (423 + 7335) | 50 / 174 | 91 / 373 | 569 / 1561 | + + - Of 6000 inputs a run, 259 to 348 stopped at a host call and waited, + and 434 to 449 were the view's own (resizes and clicks). + - In one shared view, the copy is half the size (0.66 MB), the costs + are half (copy 37 / 152, rollback 184 to 259 at p50), and the + rollbacks are the same: 30%, 70% and 121%. + - The worst rollback, 1.6 ms, is a tenth of a 16.7 ms frame. A + speculative apply is 0.2 ms at worst. +- **The copy alone** (`rollback-bench`, ReleaseFast, single view): + + | session | saved | pinned | take | put back after an edit | + |---|---|---|---|---| + | typical (5 panes) | 980 KB | 36.6 MB | 0.044 ms | 0.036 ms | + | big (14 panes) | 1.49 MB | 286.5 MB | 0.071 ms | 0.051 ms | + + Letting a copy go costs nothing measurable. + +What proved wrong: +- "Views cut the conflicts": they cut what a rollback shows, not how + often one happens. Any unit of another client's, in any view, comes + before this replica's pending inputs in the authority's order, so the + copy goes back and the pending inputs are applied again. Each costs a + third of a millisecond, a tenth of a frame at worst. Telling which + units commute with which pending inputs is the next lever, not a + needed one. +- "A copy of about 100 KB": it is 1 to 1.5 MB. The core struct itself is + most of it (the presentation, the surface, the settings and the column + tags), plus the views' parked state, now that the copy saves all of a + struct and not only what a digest takes. It is still under 0.1 ms. +- "A misprediction is rare": every key that leaves a message stamps it + with the time of day (`HH:MM:SS`). So a key confirmed in another second + than it was applied is recorded differently: 2 to 7% of inputs here. In + the harness both cores share one clock, so this is the floor. Across + machines an offset between the clocks raises it. Part B should guess + the authority's clock (its last confirmed reading, plus the time since + locally), not read its own. +- views.zig left a pane's slot for the live view unwritten, and a slot + holding the live body's selections. A pane closed before that view + parked again freed whatever its slot held. This is fixed in its own + change, before this one. Every replica switches views on every input, + and found it at once. + +Not done, and why: +- No transport: part B. +- The view's events (a resize, a click) still wait for the authority. + With views they could be applied at once. +- A pty write stops a speculation, so a shell's echo is shown late. + Phase 6 predicts the echo. +- The authority's own units (a shell's echo) roll back a replica that has + pending inputs. +- The epoch allocator is single-threaded. A replica core's workers + (images, search) must allocate elsewhere, or hold a lock, in part B. +- A replica keeps a timing sample for every input it applies (the + harness reads them). A long-lived one keeps only counters. +- test/fs.py's `mounted-startup` still hangs on main (lyntonns). The same + hang shows with main's own binary in a workspace of its own, so it is + not this line's. + +## Phase 2 notes + +Built 2026-10-29 (`src/walk.zig` Dump/Load, `src/dump.zig`, terminal.zig's +dump hooks; `pardes-snap --dump`, `pardes-monkey --dump`, and both in +`zig build replay`). Decisions taken at the forks, with the ponytail lens: + +- **The dump is the digest's preimage.** It walks exactly what the digest + walks (local, derived and authority fields stay out), so `digest(load( + dump(x))) == digest(x)` is the definition of complete, and the gate. The + dump is `magic`, a table of every type the roots' walk reaches (each + struct's and union's fields by name with their types' places, each + enum's names and values, each number's width), then the values as the + tape lays them out but for an enum or union tag by its place in the + table, and a pointer or slice met again written as a reference to the + first. The table is a comptime graph of the roots: a session's is 324 + types in 23 KB, of a fresh session's 96 KB dump. +- **Crossing builds: by name, and positional when it need not be.** A + table byte-identical to this build's is read field after field, no name + asked (the join's same-build path: no separate strict format). Any other + is matched by name: a field added takes its default, one removed is + passed over, an enum value by its name (Builtin's ordinals shift with + every builtin added), numbers widened or narrowed by value, arrays as + far as they fit, one that came to hold a `len` read either way. A field + whose kind changed takes its default and is said; with no default the + value it is in is refused, and an element of an array of optional + pointers (the session's panes) is then left out with its place in the + layout (`dropMissing`), never more. What was let go is the session's + first message after the load. A pane whose terminal another build could + not read is a fresh terminal (`ensureKind`). +- **Roots, not wrappers.** A session's dump is five roots: the core, + read *into* a core `Pardes.create` made (its local, derived and + authority fields stay create's: no defaults to invent for an arena or a + host), its 9P mounts (state the user made, inside the 9P server's + authority `fs`), which file panes' files were on disk when it was made + (one gone by the load is said deleted, so Del asks: its text is nowhere + else; one never on disk comes back as it was), the theme's name, and + the views not live. A pane's dump (the store's, a handover's) is a Pane + alone. +- **Every view is in the dump** (decided after phase 4 merged). The live + view is the core's own fields, so `view` and `views_open` were dumped + already, and a session dumped with several views loaded into a core + that said so with no parked slot: its first view switch, close or + drawing of the others unwrapped a null. Leaving them out would have + needed `load` to fold them anyway, and the join (phase 5) needs them: + a replica applies the other clients' `toView`. So `views.dumped` writes + each open view but the live one (the live one's slot is stale by + design): its share of the session but what drawing keeps + (`Pardes.replica.local`, computed from `views.fields`: its panels as + presented, its animations), its slots' serials, and its share of every + pane (`pv_known` says which shares own their selections, not which + exist: a parked view has one of every pane). `views.undumped` parks + them again in `load`, last, so nothing after it rolls back what it + moved; a pane a view has no share of gets the live view's, a view the + core says open that the dump lacks is closed, a dump with no views root + (before this) has one view. They stay out of the digest as before + (parked views are checked when live again); the gate is the re-dump + byte for byte, now also on the three-view monkey seeds. A session + *revived* for one client (Restore, `-l`) is one view, the live one as + view 0 (`views.only`, what the hosts already assumed), and a pane + carried or reopened takes the view it lands in (`sized_by`, + `prompt_view`). Cost: the views root is empty for one view; with + several, a share is about 100 bytes a pane a view. +- **Aliasing by reference.** Zerox twins share their text and undo again, + an `inherited` cwd points at its pane again. Keyed by address, length + and type, which is also what made twins dump their text once. +- **Terminals are ghostty 1.4's snapshot** (`terminal.State.dumpWrite`): + both screens, history, styles, links, and the kitty images with their + pixels and placements (the known gap in test/kitty.zig closed), plus a + walk dump of pardes's own parts (unsent replies, the view, the OSC 133 + command cycle, its output pins as screen points). The snapshot is at + the ground state: a sequence the program had begun at the dump's moment + is lost (ponytail: the stream would have to track its continuation, a + cost on every pty read). ghostty calls its format a work in progress: + a ghostty bump that changes it makes a terminal another build cannot + read, which degrades as above. +- **Clean files' text is in the dump.** A join cannot read the + authority's disk, and the session is what it holds. A file changed on + disk since comes back as the watch a load emits finds it (as on open: a + clean pane reloads, an unsaved one is marked changed), not by a read in + the load. The store keeps a closed pane's record under 8 MiB by a slim + form (no terminal history, no undo, a clean file's text read again on + reopen), as it kept terminals without history before. +- **Load and revive are two halves.** `load` makes the core and nothing + else (what the gate checks, and what a replica takes); `revivePane` and + `reviveSession` give it to a host: a new shell under each terminal's + history (alternate screen left, keys, mouse and paste modes reset, the + banner; the old shell's REPL, Tty+ word and sent command dropped; a + command pane finished), watches, PDFs opened again (MuPDF's document is + derived: `Document.handle` is optional now), Recent and the store, the + ThemeFile asked for again, the startup config. The screen stays the + dump's until the host resizes: setting this process's `opts` size first + reflowed twice and lost where a terminal's view was. +- **`opts` is local** (how this process was started: allocators, socket + names, store path), and so are `dump_out` and the effect and input + queues' counters (the queues were local, their heads and lengths were + walked: a loaded core read a garbage effect). Pipe's runs, an Edit's, + Handover's dump in flight and its carrier, and the host's fallback file + table are authority: host errands, not state. +- **Restore keeps serials.** Renumbering would rewrite every serial the + session holds (attachments, query owners, places, messages, LSP and + callstack requests); Restore hangs up every connection anyway. The next + serials are past the old session's; `restored`/`restoredcol` records + are gone. A Restore over an older session keeps the host's facts (cell + pixels, row metrics, native images) and the higher of the request + sequences. +- **A failed load leaves nothing**: the loader records each allocation it + made and frees them all (`rollback`), and a `dumpFree` for a custom + type's insides. ghostty's snapshot decode itself leaks when an + allocation inside it fails: the OOM Restore test runs without a + terminal. +- **Handover** dumps `Carried` (the kind, a column's tag, each pane's + dump, weights, owners) behind `pardes-carried\n`; a new window's `-l` + of it is an empty core that adopts the panes, its settings this + process's (the old carried dump brought the source window's theme and + settings along). Panes carried now keep their undo. +- **No v1 reader.** A ZON dump is refused before anything is replaced + (`not a pardes dump: it does not begin as one`); a ZON record in the + store is an old build's session's, which no new one lists (below). + Dumps are `.dump` files now. + +What proved wrong or missing in the design: +- The walker's list rule (exactly `items` and `capacity`) never matched in + a safe build: Zig 0.17's ArrayList has a `pointer_stability` lock there, + so phase 1's digest hashed every list's capacity and a dump wrote it. + Both take a list's items now (`walk.isList`). +- Merged onto phase 5 part A (sqsnvoux): a `len` may cut a slice now (a + text's room for its ranges, made whole and used `nsel` deep). The dump + writes such a slice whole, room and all, and a load gives it the room + it had; only one of plain numbers may be cut so (every bit pattern a + value past the cut), else no build. +- Phase 4's other views lost their selections whenever they were drawn: + `unpark` hands a slot's selections to the body (main's fix for a double + free) and `drawOthers` never gave them back, so a drawing session took + them from every view not live while a headless replica kept them. The + three-view monkey seeds with the headless replay found it (seeds 5 and + 35 of 50 diverged at the next key typed at several cursors); + `drawOthers` gives them back after each frame. +- "pty masters passed as fds where Handover passes them today": Handover + passes none (it refuses terminals: their shells are the window's + children), and phase 3 made Detach happen in place, so no dump carries + a master. A loaded terminal gets a new shell; the daemon's Adopt is the + way back to the old ones. +- There are no crash dumps: crash.zig records the panic message only. + None was added: a panic handler would walk state that may be what broke + and allocate inside the handler. The path that exists is Dump, rebuild, + Restore, which the by-name load is for (a session read by name is a + unit test over the whole type graph). +- Two behaviours of the old dump were its approximations, and goldens + moved to the real session: a terminal's view comes back where it was + (load.snap's old view was not the dumped one), its prompt rows stay + hidden in text mode (OSC 133 marks are in the snapshot), its Filter, a + pane's message and typed text over its history are kept, and a query + pane comes back a live query, not plain text. +- The web shell had not built on main for a while (u64 counts on a + 32-bit target, threads, stb_image and the tape's host calls). It builds + again and loads a native build's dump by name: `-Ddump=` embeds + test/web-snapshots/touch.dump, made by tools/web_fixture.py, in place + of the hand-written ZON fixture. A field this build compiled out (void + on wasm) is passed over, and the theme goes by its name, a fourth + session root, since the theme table differs between builds. web-snap's + touch check passes; its drift was the test's (13 tag-layer fields, + helix's retuned colours, and a pane resized by its grip, there being no + rule between panes to drag). The web shell's own `chromeTarget` still + calls an upper pane's last row a handle, which the mouse code no longer + resizes on. +- Not done: the website's replay (pardes-website/replays/00) still runs + its July wasm; the dump it was built from is gone. `node --test + test/web_driver_test.mjs` fails "CDP closes a stalled connection after + its deadline" every time, a test of the harness's CDP client, not of + pardes. A dump is read whole, not decoded as it arrives (the join can + stream it; decoding as it arrives needs a reader over a stream, not a + slice). Native byte order: every host pardes runs on is little-endian. + +Upgrading from a build before phase 2, a user loses: +- A ZON dump: Restore and `-l` refuse it, the session they are in + unchanged. A pane carried between an old window and a new one stays + where it was (a failed handover). +- Nothing in the store. Its framing and records are as they were; only a + closed pane's dump inside its record changed. A closed pane is listed + only by the session that closed it (a session's id is new each + process), so a new build never lists, opens or reports an old build's + records, and an old window still running brings back its own. Both + builds' compactions keep the records of every running session and drop + those of sessions that ended, as before. + +Checked on a copy of the user's real store (14.4 MB: Recent's 234 paths, +140 ZON records of 8 sessions, three of them running), the copy under the +workspace's scratch, the real one never opened: a new build starts on it +with no error or message, /recent lists the same 200 entries as the base +build's on the same copy, Panes shows none of the old records, a pane +closed and brought back comes back. A compaction forced by a record cut +short (as a writer that died leaves one) gives the same file in both +builds but for each one's own records: the running sessions' 72 records +byte for byte, Recent folded to the same 200, the ended sessions' 68 +dropped. The base build then runs on a store the new one wrote, with its +walker records in it, without an error. One cost moved: a closed small +pane's record is 14.5 KB, not 1.5 KB, most of it the type table each +self-describing dump carries (ponytail: bounded by 64 a session and the +compaction; a table once per store generation if it matters). + +The gates on the line rebased onto main with views and phase 5 part A +(sqsnvoux). `zig build replay -Dreplay-seeds=50`, with `--dump` in all +three parts: the 113 scripts (132 sessions, 29,430 digests over their +first, second and headless replicas), 50 monkey seeds of 70 steps +(37,722 digests) and 50 at three views (29,148), no divergence, and +every session's dump at its end loaded to its digest (all 232, field by +field) and dumped again byte for byte (14.4, 5.4 and 6.7 MB of dumps). +The snapshots pass with five goldens moved (above). `zig build replicas +-- --seeds 12` (Debug, three clients, 30/80/207 ms): none diverged or +unequal. unit-test tty passes; unit-test gui fails only phase 1's two +known tests; test/fs.py and test/daemon.py (20 cycles) pass; the web +shell builds and its touch check passes. The cross-build check is +walk.zig's test of two hand-written versions of a struct (a field added, +one removed, one renamed, enums renumbered, a width changed, a pane whose +required field changed kind left out), both ways, and dump.zig's of a real +session read by name. The real-store check (above) holds on this line. + +Measured (ReleaseFast, the base binaries from before the rework against +this line, interleaved A B A ... A on the perf cores, medians, the base's +own spread as the A/A): the perf harness's core cells 0.92-0.99 of base +(p90 0.80-1.03) at an A/A of 15-70%. A session of 16 small files, a +100 MB file and a terminal with `seq 1 3000000` of scrollback, through +9P in a real pty: + +| | base | new | A/A | +|---|---|---|---| +| Dump, the big file clean | 8.8 ms, 0.2 MB | 83 ms, 105 MB | 21% | +| Restore of it | 69 ms | 105 ms | 4% | +| Dump, the big file unsaved | 514 ms, 140 MB | 191 ms, 210 MB | 1% | +| Restore of it | 1845 ms | 310 ms | 0% | +| a closed pane kept: the terminal | 131 ms | 143 ms | 4% | +| the 100 MB file | 433 ms | 127 ms | 3% | +| a small file, after the 100 MB one | 9.6 ms | 17 ms | 44% | +| a small file, alone | 1.0 ms | 1.0 ms | 7% | + +A clean file's text is in the dump now (what makes it complete), and an +unsaved one's undo (the 100 MB file's one edit keeps the text before it); +binary, not base64 in ZON, so the rest is faster. The small close after +the big one is not the small pane's: it is the kernel freeing the 100 MB +pane's memory on the 9P connection's thread (panes are freed two syncs +after they close), 208 MB against the base's 108, since the restored +unsaved file's undo holds a copy of its text (found with the access +tracer, perf and schedstat, .scratch/daemon/trace). A small close alone +is level (three after two others: 1.0 ms both; an isolated one, median +of ten: base 0.77 ms, new 0.92). Of the 0.15 ms left, two thirds were +outside the walk and are gone: savePane kept the whole 88 KB Pane aside +and put it back, and built a 41 KB empty undo history when it was not +slim; it saves only the fields it changes now. The type table is made +once a process (it was made at every store record: 123 µs to 33 µs a +small pane's record in Debug). ponytail: Store.close's CRC32 over the +record is most of what is left, and a record repeats its 14.6 KB type +table; a table once per store file would shrink both, a change to the +store's format left until a close's cost matters. The live keystroke +cells were not run (perf rule: `--live` only for a change to terminal +output); a dump is off the hot path. + +## Phase 5 part B terminal notes + +Built 2026-10-29: screen-state terminals wired into the replicas +(`src/screen_state.zig`, `src/replica.zig`, `src/tape.zig`, +`src/terminal.zig`; `zig build replicas`). Transport-free: bytes in, bytes +out, proven in part A's in-process harness; the remote agent's stream +carries them. Decisions taken at the forks, with the ponytail lens: + +- **What crosses.** The authority's units, as in part A, but a pty's + output is recorded without its bytes. Then, per peer, its terminals' + messages. Each one is u8 pane, u32 the pane's serial (a slot's next pane + is another), u8 kind, and the kind's bytes: + - to a replica (the remote's 0x06): 1, a delta; 2, rows (`encodeRows`); + - to the authority (0x04): 1, an acknowledgement (u32 seq); 2, a request + for rows (u64 lo, u64 hi, absolute); 3, a delta refused: start this + terminal afresh. +- **Where they go in the stream.** Each replica steps every unit on its + own core, and a step in a terminal pane reads the emulator (rows, + cursor, modes, marks). So before the authority steps any input that is + not a pty's output, and once a frame, it sends a peer its own steps since + the last unit (`Authority.pending`, below) and then what the peer is owed + of the terminals (`Screens.flush`); after a step that read a terminal's + history, before its unit, the rows the peer lacks (`Screens.fill`). A + replica takes them in that order between units, and a step it replays + reads the terminal its authority's step read. Nothing goes after an + output step: a flood is paced by the frame rate, not by pty reads. +- **The authority's own steps are batched.** One unit a pty read was + 2000 units a second under `yes`: each rolled back every replica that had + pending input, and each carried a digest of one word a field, about + 12 KB (28 MB/s a peer). `Authority.output` steps into an open batch + without a digest; `pending` sends the batch as one unit with one digest, + before any other input is stepped and once a frame. Under screen state a + digest is one word (the harness's trace mode keeps the fields, to name + one). Units under the flood went from 28 MB/s to 0.1 MB/s a peer. +- **Several deltas in flight, against the version last sent.** Each state + sent is a version of its terminal (`Track`), made once for every peer: a + hash of each active row, the TERMINAL record, and what pardes keeps of + the stream (the OSC 133 marks, `terminal.marksOf`). A peer's delta is + against the version it was last sent. The stream is ordered and loses + nothing, so the peer has that one by the time it applies the next, and + the replica holds exactly that version's blobs. Up to 32 are in flight (13 + at 207 ms and 60 Hz). A peer at the bound gets nothing until an + acknowledgement frees room, then the newest version, the ones between + skipped. Peers at the same base get the same bytes (one encode for all). + A delta against a base the replica does not have (one lost, or it started + afresh) is refused, and its acknowledgement asks for a reset. + - Rejected: mosh's base at the last *acknowledged* state with a + throwaway number. On a stream that loses nothing it only makes deltas + bigger and the replica keep more. +- **The replica's terminal is the authority's, to the row.** + - Its history is exactly as long as the authority's: no byte limit of its + own, trimmed to the authority's count at each delta. Row numbers and + the scrollbar then say the same on both sides, which steps read. + - It takes the authority's viewport, its cursor's semantic content (a + prompt's input, which `promptInputReady` reads) and its marks. + - Its emulator is written by `apply` and fetched rows alone. A replica's + core never ingests, resizes (the authority sizes it; a resize's own + core work, its pointer selections cleared, still runs) or replies to + its program; its pty writes are off the tape on both sides. + - No digest covers an emulator under screen state (`State.digest`); the + harness compares screens instead. +- **What an ingest tells pardes is taped.** Over a terminal, pardes keeps + positions: the text-mode cursor, selections, the edit copy, the pointer's + rows. An ingest moves them when it prunes or erases history or clears the + screen. The authority records what it told them (each tracked row's row + after it and whether it was erased, the clear, whether the pages lived: + `Site.positions`); a replica, which ingests nothing, takes that, and the + rest of the logic runs alike on both. +- **History a step reads is sent before its unit.** Scrolled past more than + a screen at once between deltas, rows are holes on the replica, blank + until fetched. A step in text mode reads a terminal's whole grid, and + the replica's replay of that step must read what the authority's read, + which a fetch an RTT later cannot give. The authority keeps each peer's + holes as the replica does (`Peer.holes`, from what it sent), and after a + step that read a terminal's rows (`State.read`, set outside the render) + sends each peer those it lacks, before the step's unit (`Screens.fill`). + A history row never changes, so each goes once. Scrolled into view, holes + are blanked and asked for (`Replica.screenRequest`); the answer fills + them an RTT later. +- **A flood costs the replica a counter, not its rows.** When a delta + scrolls more than the replica's screen and history hold, nothing it + holds is in the new state: its rows stay where they are, renumbered, the + active ones and the tail written from the delta, the rest holes. Growing + and trimming a history's rows one by one was the 2.4 ms a `yes` delta + took. +- **Speculation stays off terminals.** An input whose live pane is a + terminal is not speculated: its step reads the emulator, the last + delta's here and the pty's newer one there. It waits for the authority + (a key in tty mode already did, at its pty write). Phase 6 predicts the + echo. +- **Rollback never brings a terminal's past back.** + - The rollback copy never held an emulator (State is skipped), and a + replica's emulators, their State included, are allocated on the C heap, + outside its rollback epoch (`terminal.emulator_gpa`, `State.gpa`). + Deltas write them while a copy is held, and nothing of them is a copy's + to save, keep or put back. Without that, a rollback freed a delta's + pages (the test below crashes). A first fix, the epoch paused while a + delta was written, was not enough: the copy also saved emulator memory + reached through a stale pointer (below), and put its old bytes back. + - The core's cache of a terminal's rows is marked stale after a delta, + dropped when a copy is taken and emptied when one is put back: it is a + grid's worth of text, and the copy's would be of an older grid (without + the mark, the test reads it). + +Found on the way: +- **A view's slot that is live is stale, and the copy follows it.** views.zig + keeps a parked view's share as copies, stale while that view is live + (a column tag's text, `Text.own`, freed by the live view since). The + rollback copy walks every slot: a stale pointer that lands on a live + allocation saves it, and a rollback puts its old bytes back. Part A was + safe while every free under a held copy was put off, which holds again + now that a delta's writes are outside the epoch. A ReleaseFast harness + run found it (heap corruption after 20 seeds), a safe build's check that + named what was freed under the copy pinned it. ponytail: the copy still + follows the stale slots; skipping the live view's slot diverged (part of + it is read), and is the next fix in views.zig, not here. +- A pane's `cwd` inherited from a terminal's pane is a pointer, and the + walker follows it: a digest of a pane next to a shell digested the + shell's emulator. It mattered when a digest was taken outside the + authority's tape (`pending`'s first version). +- `resizeGrid` does core work before it resizes (the pointer selections + of a narrowed terminal): skipping it whole on a replica diverged. + +Measured. Correctness ran beside builds; the timings alone on the perf +cores (ReleaseFast). + +- **Correctness.** + - The soak (unit tests, and the 197 recorded streams at 3 seeds): several + deltas in flight, landing late and in order, one in 50 lost (refused, + then a reset), random resizes and fetches. After every delta the + replica's screen, history extent and viewport are the authority's as + encoded, and every history row it holds outside its holes is the + authority's. None failed, Debug or ReleaseFast. + - `zig build replicas`: three clients typing in their own views over + 30, 80 and 207 ms links while the shell runs a TUI, a shell's session + with clears, then 8 MB of `yes` read 4 KB at a time. In ReleaseFast, + 20 seeds at each RTT, there was no divergence, every replica's digest + and every terminal at quiescence was the authority's, and each of the + 23,508 deltas applied matched the authority's screen at its version. + The same held in Debug: 4 seeds at each RTT, and 20 at 30 and at + 80 ms; and in ReleaseFast with every replica allocation filled with + 0xff (`PARDES_REPLICA_FILL`), 20 seeds at each RTT. +- **The terminals in the harness** (ReleaseFast, 20 seeds, three clients; + a peer's numbers while the flood runs): + + | | 30 ms | 80 ms | 207 ms | + |---|---|---|---| + | deltas a replica applied a second | 76.3 | 75.7 | 71.7 | + | delta bytes a second | 10.3 KB | 10.5 KB | 10.0 KB | + | unit bytes a second | 101.6 KB | 102.0 KB | 101.5 KB | + | encode a frame, every peer, p50 / p99 | 15 / 36 us | 16 / 42 us | 16 / 31 us | + | the same, one peer | 13 / 28 us | 14 / 30 us | 15 / 30 us | + | apply a delta, p50 / p99 | 37 / 494 us | 35 / 442 us | 34 / 400 us | + | rows fetched on demand, all runs | 1103 | 649 | 786 | + | rows sent before a step that read them, all runs | 18.0 MB | 18.5 MB | 10.0 MB | + + One delta in flight would have been one an RTT: 4.8 a second at 207 ms. + The rate is the frame's (60) and the inputs' (a delta before each). +- **The bench** (`PARDES_SCREEN_STATE=<dir> zig build core-test + -Doptimize=ReleaseFast -Dtest-filter=screen_state`, 200x60, the floods + under the C allocator): + - encode a version, every row new: p50 273 us, p99 394 us, 29.8 KB; + nothing new: 71 / 95 us, 549 bytes; + - `yes` at 19-20 MB/s: 63 deltas a second, 65 KB/s a peer (1038 bytes a + delta), encode a frame 28 us at one peer and 29 at three, apply p50 65 + to 72 us and p99 202 to 265 us (it was 2.4 ms); + - varied text at 59-64 MB/s: 356 KB/s a peer, encode a frame 52 / 54 us, + apply p50 468 to 493 us. +- **Part A's costs, with terminals** (the harness, three clients): + speculative apply p50 28 to 42 us; copy p50 95 to 151 us (1.3 to 1.5 + MB); rollback p50 331 to 442 us, p99 1.6 ms at 80 and 207 ms. At 30 ms + the copy's p99 is 7.8 ms and a rollback's 17 ms: a register holding a + terminal's whole grid (`%` then `y` in a shell's text, 2 MB after the + flood) is copied whole by every speculation. Registers could be pinned + as file texts are. (Done since: the follow-ups below.) +- **No local regression.** `pardes-perf-live --live`, A and the new tty + interleaved A B A ... A (4 runs of the new one, 5 of A) on the perf cores; + the median of the runs' p50s, new/A, present and core: + + | | against main (kuovmsxw) | against base | A/A (base) | + |---|---|---|---| + | key-down | 1.02 / 1.02 | 0.94 / 0.94 | 1.00 / 1.01 | + | key-right | 1.02 / 1.03 | 1.00 / 0.95 | 0.99 / 1.01 | + | edit-char | 1.01 / 1.00 | 0.99 / 0.99 | 1.00 / 1.00 | + | page-down | 0.92 / 0.95 | 1.22 / 1.05 | 1.00 / 0.99 | + | wheel | 0.83 / 0.85 | 1.13 / 1.10 | 1.00 / 1.00 | + | `yes \| head -c 32M` wall | 0.92 | 1.12 | 0.94 | + | a 1M-line open | 1.03 | 1.14 | 0.97 | + + Against main, this line's own A/B, every cell's runs overlap main's. + Page-down and wheel against the base read 1.21 and 1.12 in an earlier + run too, and 0.92 and 0.83 against main: main's since the base, not + this line's. What the local path gained: a counter a pty read, a resize + and a scroll; a flag a frame and a store when rows are built; and, for a + terminal with positions over it, a taped value (a null check) and two + stack arrays. + +Not done, and why: +- Kitty images in terminals still do not cross. +- Units under a flood still cost about 0.1 MB/s a peer: an output step is + about 45 bytes of tape (its input, the pump, a host call), 2000 a second. + Phase 6's zstd takes most of it (each step's records are alike); or a + host could feed a pane's queued reads to the core as one. +- The first step in text mode after a flood sends its terminal's whole + history of holes, up to a few MB, ahead of its unit (ponytail: a step + reads every row, its motion surface). +- A 9P run waiting on a replica's marks (`noteMarks`) reads them from the + last delta, not from the output step; its writes are the authority's, but + `sent_command` could differ. The output pins (`out_start`, `out_end`) + stay the authority's. +- Text reads outside `buildRows` (a pointer selection's copy in tty mode, + 9P's screen reads) do not ask for the rows; 9P's are the authority's + anyway. +- **A hazard for part A** (fixed since, below): the rollback copy followed + a view's stale slot. +- Echo prediction (phase 6): a key into a terminal waits a round trip. +- A replica steps every output step of the authority's (bytes-less), about + 2000 a second under a flood, each a core step. +- The GUI was not measured: its host is unchanged, and the core's terminal + path is the tty's. +- The other agent wires it in: the tty and GUI hosts, `Screens` per + attached peer, and a replica frontend drawing after `screen`. + +Follow-ups (2026-10-29): +- **A live view's slot keeps no pointer.** When a view goes live (`to`, + `close`, `only`, and `drawOthers` putting the live one back), its parked + share is emptied (views.zig `emptied`): it held the pointers it handed + the live fields (a column tag's text, a chord's argument, the + presentation's) until it parked again, and the copy, walking every + slot, saved and put back whatever memory had since taken their place. + A pane's share holds no pointer but its selections, which `unpark` + already hands over. A test frees a tag's text under the stale slot, + hands its memory to another allocation, writes it under a held copy and + puts the copy back: it fails without the fix. +- **Registers and location rows are pinned, not copied.** A register's + values and joined text, and a file's location rows (with the ones an + edit dropped and a query's saved), are replaced whole and never written, + so the copy pins them as it does a file's text, freed raw. Rows pin + whole with what each owns (its path, its colours: `pinnedDeep`). In the + harness at 30 ms the big copies were not the register but a file's + location rows, 5 to 6 MB after the flood. ReleaseFast, 20 seeds, three + clients, p99: copy 5.1 ms to 0.22 ms at 30 ms (0.19 to 0.20 ms at 80 and + 207); rollback 14.9 to 9.5 ms at 30 ms (1.3 to 1.4 ms at 80 and 207). + What is left of a rollback's tail is the replay's digests: half the + harness's CPU is Wyhash over the whole core at every replayed unit, the + pinned blobs included (part A's digest). Hashing a pinned blob once, + while it lives, would take most of it. +- **The live perf harness ends what it starts.** Every process a live run + starts inherits its private runtime directory: after each cell, and in + the cleanup every path takes, the harness sends TERM, then KILL, to each + whose environment names it (`liveSweep`). With a wrapper that leaves a + daemon-kept detached session (as the remote agent's client-live.sh + does), the old harness left three sessions and their daemon a run; this + one leaves none. The gate's baselines are recorded again from base (main + and the new perf.zig), and .scratch/daemon/base/pardes-perf-live rebuilt + from it. +- **A blob's digest is a word, made once.** The walker's digest, taken at + every unit a replica is sent and every unit it replays, hashed every blob + whole each time: a 100 MB file was 5 ms a digest, two digests a unit, on + the user's own session. A blob (a field annotated `.blob`: a file's text + and line index, an undo snapshot, a register's values and joined text, a + file's location rows, the ones an edit dropped, a query's saved) is now + hashed once, its word kept by its address (walk.zig), and the digest + folds that word in. The rollback copy pins what the same annotation + names (`pinned` and `pinnedDeep` are gone). + - **Exact while every free drops the word:** every such blob is freed + raw, through `memory.freeRaw` or `freeUnfilled` (the pins needed raw + frees), and those drop its word, so an address used again is hashed + again. The first ReleaseFast harness run found one that did not: an + edit freed the old text with `rawFree` itself, the next text of the + same length came back at the same address, and the old word was folded + in (162 of 180 replica runs diverged, in `file.content`, named with + PARDES_REPLICA_TRACE); it goes through `freeUnfilled` now. A replica's + blobs born under a copy are never digested (a speculation records + none), so the epoch's own frees leave no word behind. A register's + values array was the one written in place; `put` now makes a new one. + ponytail: the word is kept by address, so a new free path that skips + those two misses it; a word carried in the struct beside its blob if + that bites again. + - **Written in place, or freed past its word:** a Debug build hashes a + blob's bytes at every digest and folds those in, and panics where the + word kept at that address and length is not their hash: a blob written + in place (which the pins forbid too) or freed without dropping its + word. The tests, the replicas harness and replay run Debug and go + through it (and pass); walk.zig's test writes a blob in place and sees + its kept word is no longer its hash. A release build (ReleaseSafe, the + user's own, and ReleaseFast) takes the word and never panics on a stale + one: at worst it shows as a digest divergence. Not recovered yet: a + replica that diverges records why (`Replica.diverged`) and takes no + more units; nothing resets it or joins it again. That is the remote + client's to add (a fresh join from the authority's dump), not a crash. + - Measured (ReleaseFast, alone on the perf cores; PARDES_DIGEST_BENCH=100 + zig build core-test -Dtest-filter="big file open"): with a 100 MB file + open, a unit of a key into it (two digests) went from p50 10.45 ms, + p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the + file's word), and a digest from 5.18 ms to 10 us. The harness (20 seeds + x 120 inputs x 3 clients, ReleaseFast): rollback p99 at 30 ms went from + 9.5 to 2.6 ms (1.5 and 1.4 ms at 80 and 207 ms, about as before), copy + p99 0.27 ms (0.22 before, within the run's noise); none diverged, none + unequal, no delta wrong. + +## Phase 5 part B notes + +Built 2026-10-29 and 30: the stream over the replica socket (src/replica.zig +`Peers`, and its client half, `dial` and `readStream`; detached/wire.zig +version 15), the client process (`pardes --attach=<socket>:<name>` and +`--attach=<host>:<name>`, tty.zig `remote`), the session's side of it in +the tty host, rterm's screen-state terminals and the echo agent's +predicted echo carried by it, a client's rejoin, and test/daemon.py's +replica clients. Decisions taken at the forks, with the ponytail lens: + +- **A client's input is a unit of its own, settled by `step`.** On the + authority, at the step boundary after each pump (`Peers.boundary`), each + input that came is `toView(its view)`, `update(it)` and `step()`: the + middle of a pump (queued inputs and effects), with no wait, no clock + read and no frame. The client's speculation is the same three calls, so + the two tapes read alike and the fast path takes the unit as it came. + With the pump in its place (part A's harness) every speculation stopped + at the tty host's `wait_input` and read the clock in ns. +- **The stream.** Framed as wire.zig frames everything, a tag and a u32le + length: + - the session's: 0x01 welcome (pid, name, the client's id and view, the + dump's length, the session's offset from UTC, its digest as dumped + and its build's semantics), 0x02 refused (words), 0x04 a piece of the + dump (1 MiB), 0x05 a unit (origin, seq, the authority's realtime and + monotonic clocks when it was made, its tape), 0x06 a terminal's + message (rterm's: a delta or rows), 0x07 a check's answer; + - the client's: its inputs as the existing event tags (their seq is the + n-th on the connection), 0x04 a terminal's message (rterm's: an + acknowledgement, rows asked, a reset), 0x05 check. + + A unit carries one digest, at its end (`unit_digests`); under screen + state it is one word (`whole_digest`, PARDES_REPLICA_TRACE for one a + field). Blobs are hashed once and folded in as a word (main's + walk.zig), so a digest a unit is cheap. The draft's lengths-only digest + went with that. +- **Terminals cross as screen state, in rterm's order.** The session's + Peers keep a `Screens`, a peer each. At each boundary: the rows the + pump's steps read that a client holds as holes (`fill`, then `filled`), + the pump's unit, then the deltas (`flush`). Before each client input's + step its deltas again, after it the rows it read, then its unit. A + joiner's terminals start afresh after its dump. The client applies a + delta or rows in stream order, acknowledges, and asks for the history + rows a frame shows that it lacks. Units carry no pty bytes. + - **A wait takes one step of a person's input, and none after a pty's + output** (tty.zig `split`), while clients replicate: a step reads the + terminals as the deltas after each unit bring them, and in a pump that + took a pty's output and then a key, the key would read on the session + what the client has only after the unit. The next is held for the + next wait. Only while a client is attached. + - **A step that reads a terminal's prompt after its output is taped** + (tape.zig `.prompt`): the first prompt's greeting (`ls`, after a pty + resize) and a command waiting for the prompt. Found at once by a + client opening a shell of its own: the session's layout after the + shell's first output saw the prompt, and the client's replay, its + emulator a delta behind, did not. + - A dump-loaded client's emulators are on the C heap, as + `terminal.create` makes a screen replica's (walk.zig `.alloc` of a + pane's `terminal`, dump.zig `terminal_gpa`). +- **Predicted echo is the echo agent's** (src/echo.zig), wired as its + notes say: the client's core points at its replica's `echo`, a frame + judges it before drawing, and a wake comes a frame later while + predictions are out. PARDES_NO_ECHO turns it off. +- **Join, at a step boundary, the step paying for the mutable state + alone.** The view is opened and the session dumped without its blobs + (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted + where they go) and without its terminals' emulators (their size: the + first delta brings each). The client's writer thread writes the + welcome, then the dump with the blobs put back where they go: what the + client reads is `dumpBytes`' dump byte for byte but for the terminals + (a test splices one of a session without any and compares), and the + client is unchanged. Each blob is pinned until it is + written (memory.zig `pin`: its free is put off; a blob is never written + in place and is freed raw). The texts the session never digested (it + records nothing until a client comes) would be hashed by the join's + first digest, 250 MB for a 100 MB file and its undo: they are hashed + first on a thread of their own (walk.zig `unhashed`, `makeWords`), + pinned meanwhile, and the join waits a boundary for them; a second + round joins whatever is left. +- **A joiner checks its load.** The welcome carries the session's digest + as dumped; a client whose load differs says so and ends. The first unit + after a join says the host and its writes again (`tape.resync`), which a + core made from a dump has not. Found on the way: the dump's reader took + the first unit with the dump's last piece and dropped it (the client + diverged at its first digest about one join in three); what it read past + the dump is the stream's now (`Framer`). +- **A diverged client joins again**, the same join as the first: its link + ended, the session dialled again, its dump loaded, its view's size sent, + in the same terminal, and a line over its frame until its next key says + why ("joined again: this client diverged from the session (...)"). It + gets a new view: the old one went with its link. PARDES_TEST_DIVERGE=<n> + puts a client's core off the session's (a digested counter) once quiet + after its n-th unit. A divergence shows only at a unit the client + replays (another's input, or its own it did not apply): the fast path + compares records, not digests. +- **An Upgrade lets its clients go in words** (`Peers.refuseAll`, before + the dump): each is sent `refused` ("the session is upgrading to another + build of pardes") after what it was sent, for up to 200 ms, its view + closed. The client says it in its line and dials again, every 100 ms + for 20 s (the exec, the load, the daemon's own exec). A daemon's + refusal for another build, or a welcome whose semantics are not its + own (the daemon judged the hello by the hash it last heard, which an + upgrade under way may not have said yet), ends it with "session + upgraded to a newer pardes: upgrade this client too". "A newer pardes + is installed" is now a Host call the core makes in its focus step + (`newer_build`), so a session's clients take the line from the tape. +- **The session's clock, guessed.** A unit carries the authority's clocks + when it was made; a speculation's clock reads answer the last unit's + reading plus the time since it came plus the client's round trip + (`tape.Guess`): a file's mtime (realtime seconds), a message's time of + day (in the session's zone, its offset in the welcome) and a message's + monotonic ms. Never the client's own wall clock. +- **Mostly no rollback for the session's own pumps.** Every client input + woke the session's pump before its own unit, and that pump advanced the + clock: a foreign unit, a rollback a key. Two changes: a pump woken only + by a replica's post reads no new time (`mail_only`: the tty's `now` + answers the core's own), and a replica keeps what it applied over a + unit of another's that ends on the digest the last one ended on + (`neutral`; such a unit is replayed when the copy is next put back). A + unit that says what the host wrote, or which host it is, is never + neutral. +- **The copy is taken while idle.** The client takes its rollback copy + after drawing, when nothing is queued (`Replica.ready`), and lets it go + when a unit comes first: a key no longer pays a megabyte's copy (memcpy + was a third of the client's profile). The epoch's and the copy's maps + hash an address by a multiply, not Wyhash (the other sixth). +- **Each client a view, drawn by its own process.** `Pardes.undrawn` marks + a replica client's view: the session's pump and `drawOthers` skip it. + The client draws its own view between steps (`views.drawOne`), presenting + through the tty Shell's own `present`; what that writes in the core (a + frame skipped, one owed) is put back. A drawn parked view gives its + selections back to its slot (`drawParked`). +- **The client is the tty host with its inputs fed elsewhere.** Shell + gains `remote`; `feed` sends a terminal's input to the replica engine + and the session instead of to the core; quit, hangup and the size are + the link's. Its replica core's allocator is the rollback epoch; images' + and PDFs' are the C heap's, whose workers allocate on threads of their + own. ponytail: a picture's pixels freed while a copy is held are not + put off. +- **Every kind of input is speculated** but one into a terminal (rterm's + rule: its step reads the emulator), a view's own (a resize, the + pointer) too. +- **`unattended` is a watched field**, and a session with a client is + attended: its messages go by keys, not the clock. +- **Detach in a client ends that client**, a client's death too (its + link's end); the session and the others carry on. A Restore lets every + client go (they attach again). +- **A lagging client never holds the session up.** Each client has a + writer thread and a queue the session appends to (units coalesce there + into one write); one whose queue passes 64 MiB is cut off. rterm's + deltas skip states within that (32 in flight). +- **ssh: a terminal asked for, not -N.** `--attach=<host>:<name>` runs + `$PARDES_SSH -tt -q -L <here>:<there> <host> 'exec sleep ...'`: only an + interactive connection gets TCP_NODELAY from ssh and sshd. With -N, + Nagle held each key of a burst for the last one's ACK, and a shell's + echo through the box took two round trips (413 ms) where it takes one. + `$PARDES_REMOTE_RUNTIME` names the remote runtime directory when it is + not this one's path. +- **`ssh -C` by default** when `$PARDES_SSH` is not set (it is then the + whole command). Through the box a 3.9 MB join took 7.8 to 7.9 s plain + and 5.4 to 5.8 s compressed. Of a join's time ssh's own forward is 2.0 s + (connect, key exchange, authentication, a pty) and opening a channel + 380 ms; a ControlMaster in the user's ssh config would skip the first. + A rejoin keeps its forward. + +Measured. ReleaseFast; timings alone on the perf cores (lock.sh), the +box's through its own link (207 ms, 170 to 400 KB/s from it). The harness +scripts are in `.scratch/daemon/remote/t2/`. + +- **The main test** (`e2e_box2.py`): a session on the FreeBSD box with a + 400-line file, two replica clients here through `ssh -L` each, and a + terminal on the box itself attached as a view (`pardes --attach` there + over `ssh -tt`). Each types into the file; A and B move and scroll; A + opens a shell, runs a command in it and exits it; B is hung up; A and + the box's terminal detach. A's and B's digests were the session's at + every quiet point (every check agreed), the session's text had + all three's words, the shell's answer was on A's screen 214 to 224 ms + after Enter, and nothing was left on the box once its idle daemon had + gone (a few seconds). The clients' first frames + came 3.4 to 4.0 s after they started (ssh's forward, below). +- **Keystroke to present on a client, local-only actions** (base's + `pardes-perf-live --live`, its keys on a 50k-line file; the client of a + session detached on that file, through test/delay_relay.py at each RTT + and through the box's link; two rounds interleaved; p50 / p99, µs): + + | an idle machine, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | + |---|---|---|---|---|---| + | a local session, this line | 2070 / 2067 (2926) | 1507 / 1502 (1895) | 4744 / 4708 (7141) | 1997 / 2033 (5486) | 2098 / 2126 (2885) | + | a local session, base | 2059 / 2068 (2678) | 1517 / 1536 (2391) | 4755 / 4695 (7464) | 1916 / 1927 (5441) | 2100 / 2108 (2642) | + | client, direct | 1903 / 1917 (6813) | 1644 / 1650 (2136) | 4477 / 4413 (7912) | 1994 / 2244 (7405) | 1934 / 2063 (6793) | + | client, relay 30 ms | 1849 / 1838 (6910) | 1576 / 1557 (2695) | 4465 / 4300 (15560) | 1755 / 1712 (7682) | 1860 / 1757 (7163) | + | client, relay 80 ms | 1799 / 1753 (7281) | 1532 / 1501 (2034) | 3085 / 2654 (13705) | 1778 / 1743 (6386) | 2106 / 1870 (7692) | + | client, relay 207 ms | 1778 / 1814 (6721) | 1453 / 1478 (1887) | 4309 / 4333 (18092) | 4640 / 1745 (7562) | 1963 / 1978 (7315) | + | client, the box, ssh -C | 1773 (6947) | 1471 (2138) | 4441 (14018) | 3219 (7503) | 2163 (7692) | + + The same harness while builds ran on the other cores (the line before + its last two changes, which do not touch a client's keys; one round): + + | builds on the other cores, p50 (p99), µs | key-down | key-right | edit-char | page-down | wheel | + |---|---|---|---|---|---| + | a local session, this line | 464 (574) | 343 (417) | 1190 (1714) | 370 (1251) | 471 (638) | + | a local session, base | 463 (629) | 349 (499) | 1188 (1663) | 399 (2078) | 2096 (2591) | + | client, direct | 433 (1266) | 389 (441) | 1381 (2071) | 1183 (1400) | 451 (1360) | + | client, relay 30 ms | 413 (1243) | 364 (537) | 1332 (4083) | 1163 (1254) | 421 (1293) | + | client, relay 80 ms | 421 (1379) | 357 (500) | 1104 (1815) | 1143 (1359) | 1112 (1487) | + | client, relay 207 ms | 398 (1228) | 339 (444) | 1145 (6882) | 1151 (1263) | 615 (1469) | + + - **A client's local-only key is as fast as a local session's at any + RTT, the box's link included**: its p50s are the local session's, + whatever the delay. Its p99 is 1 to 3x the local one's: the rollbacks + (another's unit before its pending inputs; p99 1.6 to 17 ms in the + client's own count) land between some keys and their frames. + - **The machine's state moves every cell together** by up to 4.5x: on + an idle machine the cores sleep deeply between keys (the `powersave` + governor), and with builds on the other cores they do not; in a run + made while one build ran, half the cells were in each state. Within + one run, base and this line's local session read alike, and the + clients alike with them. + - Page-down on a client costs 1.15 ms where a local one costs 0.37 when + the machine is awake: a client draws outside the pump, so the render's + paint-ahead (`Stream.askAhead`, asked of the host after a pump's + present) is never asked, and the rows a page-down brings are + highlighted in its frame. + +- **Terminal echo** (`echo2.py`: a key written to the client's terminal, + to the first output holding its glyph; 30 keys; p50 / p90, ms): + + | | file | shell, predicted | shell, not predicted (PARDES_NO_ECHO) | + |---|---|---|---| + | a local session, base | 0.5 / 0.5 | | 0.8 / 0.8 | + | a local session, this line | 0.5 / 0.5 | | 0.8 / 0.8 | + | client, relay 30 ms | 0.6 / 0.6 | 0.8 / 0.9 | 31.4 / 31.4 | + | client, relay 80 ms | 0.5 / 0.5 | 0.5 / 0.8 | 81.4 / 81.6 | + | client, relay 207 ms | 0.5 / 0.5 | 0.5 / 0.5 | 208.4 / 208.6 | + | client, the box, ssh | 0.5 / 0.6 | 0.5 / 0.6 | 206.9 / 207.4 | + | client, the box, ssh -C | 0.5 / 0.5 | 0.5 / 0.6 | 215.8 / 216.9 | + + Predicted, a key into a shell at a prompt shows as fast as a local one + (it is drawn by the client's own frame); not predicted, it is one round + trip and a millisecond and a half. Before this round (the draft's) the + echo through the box was 211 ms p50. +- **The visible rollback rate.** A frame drawn after one of the client's + own inputs was recorded otherwise is counted as a rollback that may have + been seen (an upper bound: the frame shows the inputs after it too). + In the keys harness, of 669 inputs a run (two runs each): none at 0 + and 30 ms; 2 and 3 (0.4%) at 80 ms; 11 and 13 (1.6 to 1.9%) at 207 ms, + of 25 and 29 mispredicted (a key the session recorded otherwise than + the client applied it). Every other rollback (another's unit before the + client's pending inputs) is put back and replayed before the next frame. +- **Join cost on the session** (`join2.py`: three clients join a session + one after another, through the Unix socket; the session's worst + boundary, which holds the join, and the client's first frame): + + | session | its dump | of it, made in the step | worst step, before | worst step, after | first frame | + |---|---|---|---|---|---| + | typical: 4.1 MB of files, 1 MB of scrollback, 6 panes | 8.1 MB | 113 KB | 12.4 ms | 1.6 ms | 22 to 46 ms | + | big: 38.6 MB of files, 8 MB of scrollback, 14 panes | 75.6 MB | 122 KB | 51.3 ms | 1.5 ms | 87 to 117 ms | + | a 100 MB file beside typical's (130 MB of files) | 254 MB | 114 KB | 172.5 ms | 1.6 ms | 247 to 288 ms | + + Each join itself took 0.3 to 1.4 ms of a step. The dump is about twice + the files: their undo snapshots are blobs too. +- **Join through the box** (`boxjoin.py`: a session there with five of the + tree's source files open, 3.5 MB, and a shell with 1 MB of scrollback; + a 3.9 MB dump; two joins each): the client's first frame 7.8 and 7.9 s + through plain ssh, 5.4 and 5.8 s through `ssh -C` (the default now); the + session's worst step 0.9 ms. A 16 KB session's first frame takes 3.4 to + 4.0 s of which ssh's forward is 2.0 s and its channel's opening 380 ms. +- **The session's step while a client lags** (`lag2.py`: a shell printing + a line a millisecond for 12 s, client A typing a key every 30 ms, client + B stopped with SIGSTOP and cut off at a 256 KB lag; µs, p50 / p99 / + max): A's inputs 47 / 150 / 213 and the boundaries 66 / 208 / 681 with B + stopped (cut off: yes), against 60 / 134 / 309 and 69 / 276 / 4072 with + B running. +- **Digests at quiescence**: every client of every test above ended on + the session's digest (PARDES_REPLICA_CHECK): the main test's, the + clients test's four (one rejoined after a forced divergence), and the + upgrade test's client that joined again. + +- **No local regression.** With no client attached the recorder costs an + atomic load a pump (`Peers.boundary`), a flag test an event (`split`) + and one a raw free (`memory.pinned`); a session's own core has no echo. + ReleaseFast, interleaved base new base new base on the perf cores, the + median of each side's runs, new/old, beside an A/A of the old: + - keystroke to present (base's `pardes-perf-live --live`, against base, + present / core): + + | | new/base | A/A | + |---|---|---| + | key-down | 0.98 / 0.98 | 0.99 / 1.01 | + | key-right | 0.86 / 0.84 | 1.00 / 0.99 | + | edit-char | 0.76 / 0.86 | 1.01 / 1.00 | + | page-down | 0.81 / 0.81 | 0.91 / 0.93 | + | wheel | 0.89 / 0.88 | 0.96 / 1.00 | + | `yes \| head -c 32M` wall | 0.98 | 1.06 | + | a 1M-line open | 1.03 | 1.32 | + + - the core in process (`pardes-perf --reps 3`, each tree's own build of + it), against main (ykrsossp): the keys, page-down and wheel 0.92 to + 1.08 over the small, medium and large files; the edits 0.91 to 1.05; + the opens 0.93 to 1.02; body-2m 1.02 to 1.26 (small's 1.26 alone); the + scrollback cells 0.93 to 1.05. The A/A read 0.79 to 1.46 (most within + 0.9 to 1.1). Against base the keys read 1.11 to 1.20 and the + scrollback renders 1.68 to 2.18, but that pair was wrong: both of + this line's core benchmarks were built at the default `-Doptimize` + (their `--json` identity says `core_optimize` debug and + `terminal_optimize` safe; the driver alone is ReleaseFast) and base's + is ReleaseFast throughout. Built ReleaseFast, main's file gestures are + level with base, and a terminal's frame is faster than base's since + `terminal.cellAt` (the final gate's notes, at the end). + +The gates, once at the end (Debug unless said): +- `zig build unit-test -Dplatform=tty` passes, its perf gate too; + `-Dplatform=gui` 1563 of 1575, 10 skipped, and the two that fail on main + already (`tagline pointer mapping...`, `a tag caret past its row's + end...`). +- `zig build snap`: 113 of 113. `zig build replay`: the 113 scripts and 20 + monkey seeds, none panicked or diverged. `zig build replicas`: 4 seeds + at each RTT, none diverged, none unequal, no delta wrong. +- `test/daemon.py` (20 cycles, its replica clients' test included: four + clients, one through the relay, one rejoined after a forced divergence, + one a shell's, a terminal attached while a client's view was live), + `test/daemon.py --upgrade` (an unstamped build to a stamped one, its two + replica clients included) and `test/fs.py` (49 s) pass. +- core-test's new tests: a join's dump with its blobs spliced back is the + session's dump byte for byte; a pinned blob's free is put off until its + last pin goes; and the replica tests (`-Dtest-filter=replica`). + +Found on the way: +- **A terminal attached while a client's view is live drew nothing.** + The tty's attach took the live view's screen, and a client's view has + none in the session: it returned without a word, its stub waiting (the + main test's terminal on the box). It takes the session's own now; the + clients test attaches one while a client's view is live. +- **The session's layout read a terminal's prompt after its output** + (above, `.prompt`): a client opening a shell diverged at its first + prompt. +- **A divergence shows only at a unit the client replays.** The fast path + compares a speculation's records with the unit's, not digests, so a + client off the session's state that only types on stays off until a + unit it did not apply comes (another's input, or its own it waited + for). The forced-divergence test has another client type. +- **Harness traps** (the scripts, not pardes): a terminal on the box + through `ssh -tt` from a pty must have its size before ssh starts, else + the remote pty is 0 by 0 and the session draws nothing in it; an Esc + and the next key written within the input parser's lone-Esc window read + as an Alt chord; a ReleaseFast build stamped with another commit has + another semantics hash than the box's of the same tree (`-Dstamp-commit + =false` for both); keys a test types into a client whose live pane is a + shell in tty mode go to the shell (end it with a hangup). + +Not done, and why: +- **A GUI client**, and a window session serving replicas (refused in + words). The GUI host feeds its core from 52 places, each to route + through the replica as the tty's `feed` does; its frame loop is the + pump's (the client draws between steps, outside it); and its own + presentation (the window's surface, its animations) would have to come + out of the core's step as the tty's present does. A window client of a + tty session is the main test's other half; it is the next piece. +- Paint-ahead on a client (above: page-down 1.15 ms where a local one is + 0.37): its draw would ask for it as a pump does, the worker's rows taken + in as the client's own events, outside the session's tape. +- A rejoin gets a new view: the old one's layout and focus go with its + link. A rejoin that kept the view would take the view's number in the + hello. +- A notice of the session's own while clients replicate goes through the + tape (the upgrade's line); a host that writes core state outside a step + (the GUI's notice) would diverge them, and the GUI serves none. +- The join waits for the session's texts to be hashed once a session (its + first join, or after new texts): the joiner pays some milliseconds + more, never the session. +- A held paste (`split`) is dropped if the session ends with it held. +- A `--attach=<host>:<name>` client ended by a hangup leaves its forward's + socket file in $TMPDIR (its ssh goes with the terminal; the unlink is on + the normal way out only). + +## Phase 6: upgrade notes + +Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`, +`main.zig`, `dump.resumeFromDump`; `test/daemon.py --upgrade`). A running +session, and the daemon, move onto the pardes installed since they +started, in place: text, undo, views and layout, terminals with their +running shells, and their terminals, windows and daemon link. Decisions +taken at the forks, with the ponytail lens: + +- **The trigger is the word, `Upgrade`** (`SPC s U`, any tag, `/ctl` and + `/pane/N/exec` over 9P). It moves the daemon too: the session sends it + `upgrade` just before its own exec, so one word moves both, and no word + of the daemon's own was needed. Nothing upgrades by itself. A focus-in + (one comes back to the terminal or window from rebuilding elsewhere) + stats the start path, and a file there that is not the one the process + started from is said once, in a line: `a newer pardes is installed: + Upgrade (SPC s U) moves this session onto it`. Rejected: a watch on the + binary's directory (an inotify entry and an event source for a line), + and upgrading on its own (a build half-written, or one the user means + to test first, would take every session). +- **Which binary: the path this process was started from**, read once + just after its first frame (`std.process.executablePath`, a readlink, + and a stat for the notice: off a startup's way, and no build is + installed within a frame of a start), exec'd again at the upgrade. + `install(1)` and `zig build` put a new file at the path, renamed over + the running one, which `/proc/self/exe` then names `(deleted)`. The + record carries the path + exec'd, so a process that fell back to the old binary (whose own exe is + that deleted file) upgrades from the same path next time. Not argv[0] + and PATH: a session's PATH is the one it started with, and the readlink + is what the daemon's spawn already used. +- **Asked first, `<path> --version`**: run as it is, it must exit 0 with a + `pardes ` line within 10 s. A file not there, not a pardes, or one that + dies at start is found before anything is let go, and said ("Upgrade: + <path> exited 3 at --version; this session stays as it is"). 9P is + served while it runs (the turn rested), so a client's clunk after its + `Upgrade` write, and anything else, is answered; after it, nothing is. +- **At a step boundary**: after a pump, the pty readers are canceled (a + canceled tty reader posts an end of file, passed over while upgrading), + the terminals' input readers stopped, the loop's queue applied, and the + core settled (`Pardes.settle`: queued inputs, then effects). In the GUI + the queue is drained, the readers canceled, and drained again (a reader + waiting for room gets it). Bytes read are never lost; keys typed during + the exec wait in the terminal's kernel buffer for the new process. +- **The dump goes to the DumpDir, `pardes-upgrade-<pid>.dump`**, not a + memfd and not the runtime directory: a refused load leaves it where a + relative `Restore` finds it, on a disk and not a tmpfs (a big file's + session dumps 100 MB and up). Never $PARDES_DUMP, which is a Dump's. A + successful load deletes it. +- **The record is one argument**, `--upgraded=<lines>`, ahead of the + arguments the session was started with, which configure it again + (`--9p=`, `--tty-toggle`, `--geometry`, `--nested`); its file, stdin, + directory, `--detach` check, forwarding and daemon fork are skipped. A + line a word: `link`, `stub`, `old` (fds), `window`, `pty <pane> <pid> + <fd>`, `tty <view> <fd> <stub> <primary> <termios hex>`, `exe`, + `dump`, and `refused` in a fallback; a word another build adds is + passed over. Every fd it names is close-on-exec but across the one exec + (`Record.closeOnExec`), and the new process sets them back at once, so + no shell it forks inherits a master or a terminal. +- **The pid stays**: the shells stay the session's children, the stub + keeps waiting on it, the 9P name of a session named by its pid is the + same, and PDEATHSIG (set again) still means the stub's death. +- **Terminals.** A kept terminal pane is not revived (`resumeFromDump` + with `kept`): no reset, no banner, no new shell; ghostty's state is the + dump's, the program's own, and its pty keeps its size (no 1x1 nudge: + the screen is already right). The host puts the master back in the + pane's slot and reads on (a command pane's exit watched again). A + terminal pane with no live master (its shell gone) is revived as a + Restore revives it. +- **The tty's terminals stay raw, their cooked termios in the record**: + put back to cooked between the two processes, a key would echo and a + ^C (forwarded by the stub) could kill the new process before its + handlers. What is a stack is unwound: the kitty keyboard flags are + popped (the new process pushes them once its query is answered), its + kitty images deleted and its shm names unlinked (the pid is the same, + and so would the names be). The alternate screen, mouse and paste modes + stay on; the new process sets them again and draws everything, one + full frame over the old one in place. The old image's prompt rc files + (the running shells have read them) are swept by the new one, which + makes its own: at its sweep, any of its pid are the old image's. They + stay for an exec that failed. +- **Views come back whole.** `resumeFromDump` keeps every view the dump + has (no `views.only`) and does not run the startup config again (the + session's settings are the dump's). Each terminal's view is in its + `tty` line: the primary one is `run`'s, the others are entered as + `--attach` enters one, with their stubs; a view that lost its terminal + closes. The GUI opens a window for each open view, the live one first + (`openAnotherWindow(k)`): its windows are the one flash, as decided. +- **9P is cut and comes back.** What the connections had to send is let + go first (`Listener.flushOut`, the 200 ms the quit's path already + had), then the exec closes the listener and its lock, and the new + process claims the name again. ponytail: a dial during the exec and the + load is refused (one round of a client's retry); passing the listener + through as well would keep it queued. +- **Replicas are told in words.** Each connection a session serves gets + a `refused` frame ("the session is upgrading to another build of + pardes: connect again with that build") and a shutdown before the exec. + Their hello's semantics hash would not match the new build anyway, and + the session registers its new hash on the same link (a second `hello`, + which the daemon takes as an update). The reconnect, and a local + client of the same path re-exec'ing itself, are phase 5B's: there is no + replica client yet. +- **The daemon re-execs the same way**: its own start path, asked + `--version` first, the listener and lock (fds 3 and 4), the replicas' + listener, every session's link and every held master kept, its + registry on its command line (`--daemon --upgraded=` with a `c` line a + connection, its slot, kind, pid, state, semantics and name, and an `h` + line a master and its owner's slot); a replica whose hello is still on + its way is let go and dials again. Sessions never see it: their links + are the same sockets. +- **Failure never loses a shell.** + 1. The binary does not answer `--version`: nothing changes, said. + 2. The exec fails: the fds close-on-exec again, the readers and the + terminals' keys start again, the kitty flags are pushed back, the + dump deleted, said. + 3. The new build cannot load the dump: it execs the old binary again, + from `/proc/self/exe` opened before the exec (`old`, Linux), with + `refused` set; that one loads its own dump and says "Upgrade: the new + build could not load this session (<why>); it goes on as <version>". + 4. No old binary to go back to (FreeBSD has no /proc by default), or it + refuses too: a fresh session adopts every kept master as a terminal + pane (the Adopt path) and says "Upgrade: no build could load the + session (<why>); its shells are here, and Restore <dump> brings the + rest back". The dump stays. + 5. A crash after the exec is the crash Phase 3 covers: the daemon's + dups keep the shells for an Adopt, and the dump is in the DumpDir. + `PARDES_TEST_UPGRADE_REFUSE=1` makes a start refuse the load as such a + build would (the first start only), `=all` the fallback's start too: + test/daemon.py installs a script that sets it and execs the new build. + +Found on the way: +- A fresh session's first pane has no directory until its own shell's + spawn is performed, so the `Tty` word that adopts a shell refused ("Tty: + : no such directory") and the shell was hung up with "no room": the + kept shells are taken after the startup effects, and those they emit + performed then. +- A test reconnecting after the word could reach the old process, still + asking `--version` (and serving 9P meanwhile): the test waits for the + exec (the session's exe, or on FreeBSD its new semantics in the + daemon's registry) before it reconnects. +- An `Adopt` (or a `Detach`) served in a session's first pump, before its + first frame, is refused as if no daemon kept it: the reader that takes + the daemon's `ok` starts after that frame, and `kept()` waits 2 s for + it. The test's next session waits for its first frame. Pre-existing, + left as it is. + +Gates (Debug, `A` this line unstamped, `B` the same with `-Dstamp-commit`, +a trivial visible change: `pardes 0.24 (<commit>)` and another semantics +hash; both installed in turn at a private prefix by a rename, as +`zig build` and install(1) do): +- `python3 -B test/daemon.py --upgrade A B`, under a private runtime dir. + A tty session with an unsaved edit, a shell running `sleep 1000; echo + done-after-upgrade` and a second terminal (a second view), and a + replica of A's build connected: + - B installed, a focus-in says the newer build; `Upgrade` written to + `/ctl`: the session's 9P answers again 123 ms later (81 to 83 ms on + the box). The session's pid is the same, `/proc/<pid>/exe` is B (not + deleted), and so is the daemon's, the same pid. + - The edit is there, the pane dirty, and `Undo` takes it back. + - The shell's pid is the same, still the session's child, its `sleep` + running; killed, the shell goes on and `done-after-upgrade` shows. + - Both terminals are drawn again (each its own view); the stubs wait on. + - The replica of A's build is told `the session is upgrading to + another build...`; a hello of B's build is welcomed by the daemon, + one of A's refused (the daemon lists the session as B). + - The fds level: the session's as before but the replica's, the + daemon's the same; the session's prompt files as many as before, the + new image's. + - A refusing build (a script that sets `PARDES_TEST_UPGRADE_REFUSE=1` + and execs B) installed and asked: the session goes back to the binary + it ran, its exe that inode again, the shell and text as they were. + - One that refuses everywhere (`=all`): a fresh session, the shell a + pane of it (`echo $$` there is its pid), the dump left in the DumpDir. + - The session killed outright: the daemon, upgraded three times, still + holds the shell's master, and a new session adopts it. + - Nothing left: no process, no socket. Passed 3 runs of 3 (and every + run since). +- `--upgrade --gui A-gui B-gui`: a window session in test mode (hidden + windows, captured) with a second window from `pardes-gui --attach`, the + same edit and job: after `Upgrade` the pid is the same and the exe B's, + the daemon B, the edit and undo there, the shell and its job the + session's, both windows' captures written again, B's hello welcomed; + the session ends with nothing left. Passed 5 runs of 5. +- FreeBSD 15.1 (the box, `pardes-ua` and `pardes-ub` cross-built from + this line): the same tty test but the fallback to a kept copy (there is + none). The new build is told by its semantics in the daemon's registry, + and the daemon's re-exec by `--upgraded=` on its command line; the fds + level by procstat. 81 to 83 ms from the word to 9P. Passed 9 runs of 10 + (the last two on this line's final builds); the one failure was the + test's first-frame race above, fixed, and it passed after. +- The existing gates: `test/daemon.py` (20 cycles) and `--gui` (10) + pass; `test/fs.py` passes (49 s); `zig build snap` 113 of 113; `zig + build replay` passes (the 113 scripts, 20 seeds at one view and 20 at + three, their dumps loaded to their digests); unit-test tty 1479 of + 1499, 20 skipped; unit-test gui 1550 of 1561, 9 skipped, and the two + known failures (`tagline pointer mapping...`, `a tag caret past its + row's end...`). core-test's new tests: the record read back as written + (and refused when cut), and an upgraded dump keeping its views, a + running terminal's alternate screen and no new shell, and settings the + startup config would change. + +Measured. The upgrade itself, Debug, a 300 MB binary: from the word +written to `/ctl` to the session's 9P answering again, 123 to 130 ms, +one run beside builds 299 ms (the `--version` ask, the drain, a dump of +a small session, the exec and the load); on the box, 81 to 83 ms with +its 125 MB binary. + +No local change (ReleaseFast tty, main kuovmsxw against this line, both +live at once, their samples alternating, through the perf lock; lat.py's +startup is a spawn with no daemon running to its first frame's end): + +| | A/B | A/A | +|---|---|---| +| startup, 100 a build: p50 | 0.944 | 1.010 | +| the same: mean | 0.919 | 1.061 | +| the same: median of the fast mode (and its share) | 0.964 (54 to 52) | 0.996 (69 to 73) | +| startup, 40 a build, two runs: p50 | 1.001, 1.266 | 0.998, 1.010 | +| key in a file, to its frame, p50, two runs | 0.997, 1.002 | 0.958, 0.990 | +| key echoed by bash, to its frame, p50, two runs | 0.997, 0.991 | 1.004, 0.999 | + +The startup times have two modes on this machine, about 4.5 and about 12 +ms, and the share in each moves between runs and between the builds of +an A/A alike, so a p50 that falls between them jumps (the 1.266). Within +a mode the builds are the same. A first A/B, made while the path was +still read at the start (a readlink and a stat before the first frame, +some microseconds), read p50 1.04 to 1.13 against an A/A of 0.98 to 1.00 +on runs that overlapped builds; the read was moved after the first frame +anyway, so a start does nothing it did not. + +Not done, and why: +- Remote replicas reconnect on their own, and a local client of the same + path re-execs itself first: phase 5B's (no replica client exists yet). + What they get today is the `refused` frame above, then the hangup. +- The 9P listener is not kept across the exec: a dial in that window (a + hundred milliseconds or so) is refused. Clients that dial afresh (the + hosts' Client) retry; a mounted session sees its connection end. +- On FreeBSD a build that cannot load the dump starts afresh (shells + adopted, dump kept): there is no `/proc/self/exe` to go back to by + default. Keeping an fd of the binary open from the start would do it, + at an fd a session. +- A crash of the new build after it starts is a crash: the shells wait in + the daemon for an Adopt and the dump is in the DumpDir, but nothing + restarts the session on its own (the stub, which runs the old binary, + could). +- An escape sequence a program had half written when the dump was made is + lost (the snapshot is at ground state, Phase 2 notes). +- Language servers and `|` pipes running at the upgrade are let go (their + pipes close at the exec; they end), and the shells of panes closed just + before it, still waiting to be reaped, are reaped only when the session + ends. A daemon note (a terminal attaching, adopted shells) read by the + old process's thread but not yet taken by its loop is lost; the stub of + such a terminal sees its pipe end. +- A detached session's upgrade (no terminal) was checked by hand + (scratch script), not in test/daemon.py. +- GUI windows come back at the size and place the compositor gives them. +- A daemon protocol bump across an upgrade is not handled: the sessions + of the older build keep talking theirs. +- A session name with a newline would break the daemon's registry line. +- An upgraded process records no tape (a recording ends at the upgrade). + +## Phase 6: 9P round trips notes + +Built 2026-10-30 (`src/9p_io.zig` Client, `src/fs.zig`, the Mount word in +`src/builtins.zig`, a paragraph of `docs/typ/reference.typ`). pardes's own +9P client is what mounts, the Mount word, the forwarding CLI, `--wait` +and Handover use; scripting clients (9p(1), 9ns, test/ninep.py) are not +changed here. Decisions at the forks, with the ponytail lens: + +- **The server already takes a connection's requests in order; nothing + there changes.** cloud9's engine has one job slot: it starts the next + frame only once the request before it is answered. What parks leaves + that slot, and the frames after it go ahead: a read with nothing yet, + and, while the editor is out in a syscall, a write or a truncating + open (`tree.needsQuiet`; an OTRUNC open is a setattr first). So a walk + and the open of its fid can be sent together (a walk never parks), and + so can an open and its first read (an OREAD open never parks). Two + things cannot: + - a Twrite behind a truncating Topen that parks is refused `bad use of + fid` (not open yet); + - a Tclunk behind a parked Twrite interrupts it. + + The client therefore sends a write or read that came back `bad use of + fid` again after its open's answer: that refusal is the engine's, made + before the backend, so nothing was done. It sends its Tclunk only after + the last answer, and does not wait for the Rclunk. The park test ("a + change waits while the editor is out mid-step") takes the resend path. +- **One flight per operation.** The flight is `[Tversion Tattach] Twalk + [Twalk] Tstat Topen Tread` for a read, or `... Topen Twrite` for a + write, each request with its own tag (`Flight`, `Session.land`). Past 16 + names, the second walk goes from the fid to itself, so no fid is in + between. +- **Version and attach pipelined by a poke.** cloud9's client submits + nothing while a Tversion is out. `greet` sets its `versioning` and + `msize` around the submits, taking the asked msize until the Rversion + lands. Only a pardes is dialled here, and every pardes offers + `ninep.msize`. A write in that first flight is kept under + `min_msize - 23`. The fix belongs in cloud9's client (taking requests + while it versions). It is not made here, since this round pushes + nothing (`// ponytail:` at `greet`). +- **The end of a read.** 9P has no end of file but an empty read. + read(5) says a reply "may be less than the requested amount", and a + directory's next offset must be the last plus what it returned, so + neither the end nor a directory's next read can be sent ahead. The + Tstat in the flight gives a file's length: a read that comes back short + at that length is the end. pardes reports real lengths for bodies, + `/os` files, ctl, tag and the like. A file of no stated length, or a + directory, costs one round trip more: index, layout, screen and log are + generated whole by each read and say 0 by design (tree.zig `topSize`). + An end flag would be a protocol extension, which is not worth one round + trip on generated files. +- **Read ahead.** With the length known, the Treads for the rest go at + once, 14 at a time (the 16-tag window, less a clunk still out and a + spare). A 1.3 MB file takes 3 round trips where it took 22. A file of + no stated length is not read ahead (`// ponytail:`). +- **Reads are timed by their progress.** Each Rread gives the rest the + 2 s budget again, as each Twrite already did. Found on the way: on main, + every read of one operation shared one 2 s budget, so a remote file of + 1.3 MB failed to open (`dial failed: timed out`) at 80 and 207 ms. On + the box without `-C`, the 77 KB `src/fs.zig` failed too. +- **A mount keeps its connection** (`kept`), as Plan 9's mount keeps its + channel: an operation through it is the flight alone, with no dial, + version or attach. + - There are eight slots, keyed by dial. A session is taken out of the + table while it is in use, so two users of one dial each get their + own. + - Unmount forgets it (`Client.forget`). + - Before reuse, a poll takes what arrived while it waited (a Rclunk + owed); an EOF, or anything else, means the peer let go, and a new + connection is dialled. + - A read on a kept connection that hangs up all the same is asked once + more on a new one. A write is not, since it may have been done. + - Only `readMount`, `writeMount` and `probeMount` (the Mount word) + keep one, since each costs the peer one of its 16 connections while + kept. The CLI, `--wait` and Handover dial afresh as before, with + their version, attach and walk pipelined. +- **No qid or stat cache.** The trace shows that a Look through a mount + is one operation with one walk. With walk, stat, open and read in one + flight, a cache would save bytes but no round trip, and a cached stat + is wrong once the peer changes the file. +- **msize and iounit stay.** They are already 64 KiB, with iounit = + msize - 24, so a body of up to 65,525 bytes is one Tread. A larger + msize would cost each of a session's 16 connections five buffers of + it. test/ninep.py asks 8192 by its own choice, so its body reads of + more than that cost it more messages. +- **`/n/<host>`.** The daemon serves no 9P yet (phase 3), so there is no + remote daemon to mount. A remote session mounts today with two + commands, now in reference.typ: `ssh -C -N -L /tmp/box.sock:<its + socket> host`, then `Mount box /tmp/box.sock`. A relay in the daemon + that lists a host's sessions under one mount waits for the daemon to + serve 9P. `-C` is there because the box's link is slow in that + direction (below). + +Measured. The harness is `.scratch/daemon/ninep/rt.py`. It runs a peer +session and a local one that Mounts it through a relay. The relay holds +each chunk a fixed delay after it arrives, as test/delay_relay.py does, +and it also splits the stream into 9P frames, counting the T-messages +and the rounds (runs of T-messages with no answer delivered between +them). `box.sh` runs the same harness against a detached session on the +box through `ssh -L`. Builds are ReleaseFast (main ymwmwrvx against this +line), 3 reps each, medians. + +The RTTs column counts the round trips the client waited on. pardes's +own client sends its last Tclunk without waiting, before and after; the +python client waits on every message. The box columns are `ssh -L` at +207 ms, plain and with `ssh -C`; "fails" means `dial failed: timed out`. + +| operation (pardes's client) | msgs | RTTs | 30 ms | 80 ms | 207 ms | box | box `-C` | +|---|---|---|---|---|---|---|---| +| connect: `Mount` (version, attach, root read) | 5 → 7 | 4 → 1 | 124 → 33 | 324 → 81 | 831 → 208 | 1032 → 426 | 1024 → 410 | +| file read: Look `/n/peer/index` (no length) | 7 → 6 | 6 → 2 | 185 → 61 | 486 → 161 | 1247 → 415 | 1441 → 426 | 1434 → 409 | +| directory listing: Look `/n/peer/pane/` | 7 → 6 | 6 → 2 | 185 → 62 | 483 → 161 | 1247 → 415 | 1437 → 427 | 1430 → 410 | +| ctl write: Save to `/n/peer/pane/1/ctl` | 6 → 4 | 5 → 1 | 155 → 32 | 402 → 81 | 1039 → 208 | 1237 → 214 | 1233 → 206 | +| pane body read: Look `/n/peer/pane/1/body` (30 KB) | 7 → 5 | 6 → 1 | 186 → 33 | 483 → 81 | 1247 → 208 | 1843 → 641 | 1432 → 206 | +| a remote file open: Look `/n/peer/os/.../fs.zig` (77 KB) | 8 → 6 | 7 → 2 | 213 → 64 | 563 → 162 | 1454 → 416 | fails → 1300 | 1844 → 617 | +| a remote file open, 1.3 MB | 27 → 25 | 26 → 3 | 807 → 103 | fails → 244 | fails → 625 | fails → 5334 | fails → 1273 | + +| the python client (unchanged, for scale) | msgs, RTTs | 207 ms | box | +|---|---|---|---| +| connect | 2 | 415 | 614 to 635 (with ssh's channel open) | +| read `/index`, list `/pane` | 5 | 1037 | 1023 to 1067 | +| ctl write | 4 | 829 | 818 to 852 | +| body read, 30 KB | 5 | 1037 | 1024 (`-C`) to 1283 (plain) | + +- **Each operation is one round trip.** A directory, or a file of no + stated length, takes two. The first operation through a new `ssh -L` + forward pays one more, the channel opening (the Mount, 426 ms on the + box). +- **On the box, bytes cost more than the 9P rounds.** The link from the + box to here carries 170 to 400 KB/s through ssh. A 1 MiB `dd` takes + about 6 s and 4 MiB about 10 s, where 4 MiB up takes 2.4 s. A 64 KiB + answer arrives 1 to 2 RTTs after the request, so the 30 KB body reads + in 641 ms plain and 206 ms with `-C`. The 1.3 MB file's Rreads come in + at about one a round trip (the timeline is in `rt.py`'s `TIMELINE=1`). + `blast.py` measured all 21 of its Treads in flight at once at 1.9 to + 3.0 s, against 5.3 s for this client's 14 and then 7. A sliding window + instead of batches would buy that, but only for large files on this + link (not done). Compression helps every size: `-C` is in the + documented recipe, and the replica stream's zstd (phase 6) will face + the same link. + +No local regression. Local 9P has no delay. lab.py runs the base and +the new build at once with their samples alternating, 1000 each, under +the perf lock, and gives the p50 ratio new/base. The server is not +changed, so its rows are level. The mount rows are faster because they +dial nothing. + +| local, p50 | base | new | A/B | A/A | +|---|---|---|---|---| +| python client: read a 9 KB body (server path) | 88 µs | 89 µs | 1.004 | 1.000 | +| python client: ctl write `clean` (server path) | 80 µs | 81 µs | 1.011 | 0.992 | +| Look `/n/peer/pane/1/body` through a local mount | 329 µs | 255 µs | 0.776 | 1.009 | +| Save to `/n/peer/pane/1/ctl` through a local mount | 214 µs | 141 µs | 0.660 | 1.000 | + +The gates: +- test/fs.py, `zig build 9p-io-test`, unit-test tty, the snapshots and + test/daemon.py (20 cycles) pass. +- New: `a mount's connection is kept between operations, and one the + peer hung up is dialled again` (a write longer than a message on a + kept connection, a miss, a hangup, a redial and a forget). It also + passes on the box (FreeBSD test build). + +Not done, and why: +- **cloud9's client should take requests while it versions;** the poke + in `greet` stands in for that until cloud9 is pushed and re-pinned. + 9ns (cloud9's) still waits one round trip per message. +- **A write longer than one message** sends its chunks after the first + one at a time: a Save of a 1 MB file through a mount is 16 round + trips. Pipelined Twrites could run in the wrong order once one parks; + that does not matter for `/os` (pwrite at an offset) but would for + some pane files. +- **A file of no stated length** is not read ahead, and its end (like a + directory's) costs a round trip more. +- **The CLI's `ask`** (`pardes file` from a pane, `pardes -`) and + `--wait` pipeline only version, attach and walk. Their open, writes + and reads go one at a time, on a connection of their own. +- **A peer's Restore cuts its clients up to 200 ms after it starts.** A + request on a kept connection in that window is refused `Input/output + error`, where a new dial would already have been the replacement's. +- **Scripting from here to a remote session** through test/ninep.py or + 9p(1) is still one round trip per message (1 s for a read at 207 ms). + A python helper for remote scripting could pipeline the same way; + nobody scripts that way yet. + +## Phase 6: predicted echo notes + +Built 2026-10-30 (`src/echo.zig`; hooks in `src/replica.zig`; a field on +the core and one call at the end of `body_layer.renderBody`; `zig build +replicas -- --echo`). Transport-free like part B: proven in part A's +in-process harness, carried by the remote client. A key typed into a +terminal pane goes to the authority, and its echo comes back as a delta a +round trip later: 207 ms a character in a shell on the FreeBSD box. The +replica now draws it at once, mosh's way, and the authority's screen +confirms or takes it back. Decisions at the forks, with the ponytail lens: + +- **Presentation, the replica's own.** The overlay is `Replica.echo`; the + core points at it (`Pardes.echo`, `.local`, and in rollback.zig's + `unsaved`, so the copy neither saves nor puts it back). No digest, dump + or tape sees it, and the core's emulator stays the authority's. Its + only write into what is drawn is `Echo.paint`, at the end of + `renderBody` for a terminal pane: the cells predicted and the cursor + after them, over the body as drawn. A terminal's body is painted + straight onto the grid (a body layer is a file's with context rows), + and the tty and the GUI both draw the grid's cells and cursor, so both + get it. A session's own core has `echo` null: one null test a terminal + pane a frame, and no overlay. +- **What is predicted:** a printable ASCII character at the cursor, + Backspace, Left and Right, typed into the view's terminal in tty mode + (keys going to its program), on the primary screen, at the live bottom, + with the cursor shown. Never for a chord, Enter, Tab, Esc, Up and Down, + a paste, or anything on the alternate screen (vim never sees one). More + narrowly, so as never to draw what the line editor would not: + - a character only over a blank cell or the same character (typing + along a fish suggestion); a line edited in its middle, or a + suggestion left, is the shell's to redraw; + - not in the last column (wrap is the program's); + - Backspace only at the end of the line, over a prompt's input or a + predicted character; Left over the same; Right over typed text, a + prompt's input in the colour of the cell before it (past the text's + end fish's Right takes its suggestion). + Each prediction is a cell (an absolute row, the authority's numbering + that the delta's header carries) and the cursor after it; a newer key's + cell replaces an older one's, as mosh's overlay does. +- **Judged once the authority has the key and its echo has had time.** + Each unit of the replica's own says which of its keys the authority + stepped (`Echo.confirmed`). A stream that loses nothing keeps the order + the authority sent in, and it sends what it owes of the terminals + before it steps an input, so a delta that arrives after that unit was + made after the key was written. 50 ms after the unit (`grace_ns`, + mosh's ECHO_TIMEOUT: the shell's echo, then the authority's next frame) + each applied delta judges it: a cell the screen shows goes (confirmed), + one it does not is a misprediction and every prediction goes. A + predicted space (Backspace's too) is judged neither way, as mosh's + blanks are; the cursor after the newest key is judged with it. With no + delta at all (nothing changed: no echo), a frame judges it 250 ms + later (`stale_ns`). +- **A key that is not predicted** (Enter, Tab, a chord, or one the rules + above refuse) is the next to wait for: once the authority has it, every + prediction made before it goes, judged neither way (the deltas from + then show that key's doing, which nothing predicted). The keys typed + behind it wait (`deferred`) until its grace has passed; then those the + authority does not have yet are predicted, in order, over the screen + that shows its doing (`catchUp`). One the authority has whose grace is + not over puts that off a little. So the characters typed straight on + after Tab are drawn where the completion left the cursor once it shows + (a round trip and 50 ms after Tab), the rest at once, and never where + the cursor was before it. + Found by the harness: the first version predicted nothing until the + wait was over, and every key typed in that time became a new wait, so a + word typed after Tab or Enter at 207 ms was all late (p90 242 ms). An + input that is not a key into this terminal (a click, a command) has no + deferral: where the keys after it go is not known here until the + authority has it. +- **Shown where the echo is vouched for.** At a shell's prompt, pardes's + OSC 133 marks (phase input: a B and no C since) and the cursor's own + semantic content say its line editor echoes: drawn at once. Elsewhere + (`cat`, a shell without pardes's marks) only once an echo was confirmed + there since the last Enter, Tab or chord (`trusted`), so the first + character of each line waits a round trip. A key before the last one + not predicted vouches for nothing after it: the harness found the + characters of `sudo ls`, confirmed after its Enter, vouching for the + password prompt that Enter brought up. After a misprediction that was + drawn, nothing is drawn until an echo is confirmed again (`doubt`). A + password prompt or a program that ignores what is typed confirms none, + so nothing typed there is ever drawn. +- **Underlined off a prompt only.** At a prompt the marks vouch for the + echo, and an underline trailing every character for a round trip at + 207 ms is noise; off one the trust is mosh's inference, and the + underline says so. After a misprediction nothing is drawn at all, which + mosh's adaptive underline approximates. +- **No protocol change.** No echo acknowledgement crosses (mosh's + `echo_ack`): the replica's own units stand for it, and the grace for its + 50 ms. No termios flag crosses either: a program's echo is inferred from + confirmations, as mosh does. + +The hook for the remote client (its `Replica` and stream as they are): +- `Replica.init(..., screens = true)` points its core at its echo + (`core.echo = &r.echo`). A replica made another way (the remote line's + `initDump`) does that itself once its core exists. +- Per key: nothing more. `r.input(ev)` predicts after its settle, in + `r.view`. +- Per unit: nothing more. `r.confirm(unit)` tells it which of its keys the + authority has. +- Per delta: nothing more. `r.screen(msg)` judges after each delta it + applies. +- Per frame, before drawing: `if (r.echoFrame()) dirty = true;` judges + what is stale, predicts the keys deferred whose wait is over, and says + whether the overlay changed since the last call; `r.echo.changed` says + the same between frames (after `r.input` and `r.screen`), for a client + that draws only when its state moved (the remote line's `moved`). +- Draw its own view (`core.view == r.view`): the overlay is drawn in no + other, and over a terminal in tty mode only. +- The clock is the monotonic one (`FrameStats.nowNs`) unless `r.echo.now` + is set (the harness's virtual clock). + +Measured. + +- **The harness** (`zig build replicas -Dplatform=tty -- --echo`): one + authority and one replica over part A's delayed, jittered link, the + authority's pty a shell played in the harness (`FakeShell`): a line + editor at a prompt with pardes's marks and Tab completion, and the + programs the corpus runs in it. Each key's echo comes 2 ms after the + authority writes it. The client types a corpus of 222 keys at a + typist's pace (60 to 180 ms a key) and in bursts (12 to 60 ms): lines at + a prompt, Backspace, Left and Right, Tab typed straight on, `sudo` and + its password, vim, a program that ignores what is typed, `cat` (the + kernel's echo, no marks), and a line typed before the last one's prompt. + Each client frame (60 Hz, its own phase) is what its screen shows: its + core rendered, the overlay where it is drawn. Ground truth is the + authority's screen right after each key's echo. +- **Key to its character on the client's screen** (the simulation's clock; + 20 seeds, 3060 keys the shell echoed at each RTT, ms, without → with): + + | | 30 ms | 80 ms | 207 ms | + |---|---|---|---| + | p50 | 49.8 → 8.7 | 102.3 → 8.7 | 239.0 → 8.9 | + | p90 | 59.0 → 15.2 | 111.8 → 15.4 | 247.9 → 15.8 | + | max | 66.9 → 65.3 | 121.5 → 114.5 | 261.7 → 251.6 | + + With it, a key waits for the client's next frame, 0 to 16.7 ms. The tty + host draws an input's frame at once, so a real client's is the frame's + cost. The tail is by design: at 207 ms the p90 of the characters typed + straight on after Tab is 38 ms, of a line typed ahead of its prompt + 48 ms, and of `cat`'s 179 ms (its first characters of a line wait for an + echo to be confirmed); every other part's is 14 to 15 ms. +- **Accuracy** (the same runs, 4440 keys at each RTT): 3718 to 3720 + predicted (84%); 714 to 722 not, by the rules (each Enter, Tab, Ctrl-C, + Ctrl-D and Esc, everything typed in vim, and the characters behind a + key not predicted that the screen showed by the time it was over). Of + those predicted, 3238 to 3240 were what the authority's screen showed + right after the echo; the other 480, at each RTT, are exactly the + password's and the ignoring program's 24 a seed, and none was drawn. + Wrong characters drawn: none, at every RTT (0 keys, 0 frames). Frames a + password's or the ignoring program's key was drawn: 0. vim's keys + predicted: 0. The overlay judged 97 to 132 cells against the screen as + mispredictions, all of them undrawn ones (the password's, the ignoring + program's), and no cursor. Every frame drawn with a prediction shown + (13,083 to 21,009 a cell) had each predicted cell where it was + predicted in the rendered surface. +- **Costs** (ReleaseFast): `paint` p50 50 to 63 ns, p99 86 to 155 ns; the + replica's render with a prediction shown p50 54 µs; a key's `typed` is + a few array scans. +- **No divergence**: the replicas harness (ReleaseFast, 20 seeds x 120 + inputs x 3 clients): none diverged, none unequal, 23,547 deltas checked + and none wrong; its deltas a second (76.3, 75.7, 71.7) are part B's to + the decimal. +- **No local regression.** A session's own core draws no overlay + (`echo` is null: a unit test says the authority's core has none). + `pardes-perf-live --live` (base's driver), main (vtzpkxwu) and this + line interleaved A B A ... A on the perf cores (3 runs of the new one, 4 + of main), ReleaseFast; the median of the runs' p50s, new/main: + + | | new/main | A/A (main) | + |---|---|---| + | key-down, present / core | 0.99 / 0.99 | 1.00 / 0.99 | + | key-right | 0.99 / 1.00 | 1.00 / 1.01 | + | edit-char | 0.97 / 0.98 | 0.99 / 0.98 | + | page-down | 0.85 / 0.92 | 0.82 / 0.85 | + | wheel | 0.97 / 0.99 | 0.98 / 0.98 | + | `yes \| head -c 32M` wall | 1.11 | 1.04 | + | a 1M-line open | 0.93 | 1.01 | + + Every cell's runs overlap main's. The `yes` cell is the kernel's (Tty + throughput notes): main's own runs read 4.5 to 7.1 s in the A/B and 5.8 + to 7.6 s in the A/A, this line's 5.0 to 7.2 s. What the local path + gained: a null test a terminal pane a frame, and 8 bytes of core. + +The gates: +- Unit tests (`zig build core-test -Dtest-filter=echo`): keys typed at a + prompt drawn at once in the rendered surface, unflagged, the cursor past + them, then confirmed by the echo, Backspace too, and the replica's digest + the authority's; a password prompt draws nothing typed and lets it go + when stale, and vim's alternate screen predicts nothing; a misprediction + takes every prediction back and hides the next until an echo is + confirmed, and after Tab the keys behind it are drawn where the + completion left the cursor, or not at all if the screen already shows + them. +- `zig build replicas -- --echo` fails on any divergence, unequal end, + wrong delta, drawn password or ignored key, predicted vim key, or cell + rendered otherwise than predicted; it passes at 20 seeds, Debug and + ReleaseFast. `zig build replicas` passes at 20 seeds, ReleaseFast (and 4 + in Debug). +- Unit tests tty (`zig build unit-test -Dplatform=tty`, its perf gate + too), the full `core-test`, and the 113 snapshot scripts pass; the GUI + build's echo tests pass (`core-test -Dplatform=gui`). + +Not done, and why: +- ASCII only: a width table would let the rest of Unicode in (`kindOf`). +- A line edited in its middle (insert, Backspace), Delete, Home, End, the + readline chords, word moves and a paste are not predicted: the line + editor redraws what is right of the cursor, which a prediction would + have to guess. Each waits a round trip, and the keys typed straight on + behind it are drawn when it is over. +- A key typed after an input that is not a key into this terminal (a + click, a command) and that the replica could not apply itself waits for + the authority, and so do the keys after it until a pause: where they go + is not known here until the authority has it. +- The echo is inferred, not told. No termios flag and no echo + acknowledgement cross. A password prompt is safe because nothing typed + there is ever confirmed. A shell at its prompt that does not echo a key + (zsh's vi command mode) has it drawn until its grace passes, about a + round trip and 50 ms, and then nothing more until an echo is confirmed; + so does a program that turns echo off mid-line in `cat`'s plain echo. +- An echo later than about 30 ms on the authority's host (a loaded box) + makes a drawn character go when judged and come back with the echo: a + flicker, never a wrong character. The grace could follow the measured + echo delay. +- A shell without pardes's marks (fish's own marks, a shell started + without the integration) is trusted only by confirmation, so each line's + first character waits a round trip, as in `cat`. +- The GUI was not run: it draws the grid's cells and cursor that the + harness checks in the rendered surface, and its build compiles and + passes the echo tests. +- The remote client (the other line) wires it: `initDump`'s core, the + frame hook, its `moved`. (Done since: Phase 5 part B notes, with the + echo through the FreeBSD box's link.) + +## The final gate notes + +Run 2026-10-30 on main with every phase merged (rqzuvput, then the three +fixes below), in the default workspace. Scripts and logs are in +`.scratch/daemon/final/`. + +Found and fixed on the way: +- **A terminal's paint walked the page list once a cell** (uvprzmxw). + `recolorAnsi` read each cell through `pages.getCell(viewport x, y)`, + which walks from the viewport's top down to the row. The main build no + longer inlined it. `terminal.cellAt` reads the cell from the row's own + pin instead. Phase 5B's "regression against base" was something else: + its core benchmark was a Debug core beside base's ReleaseFast one (the + `--json` identity says which). +- **glibc's `tcgetattr` overran Zig's termios** (qoprvkln). glibc's struct + is 60 bytes and Zig 0.17's `std.c.termios`/`std.posix.termios` is the + kernel's 44, so every call wrote 16 bytes past its caller's struct + (vaxis's `makeRaw` too). The ReleaseSafe tty trapped at its first + terminal (`BADF => unreachable`); Debug and ReleaseFast survived on the + stack's luck. On linux-gnu, main.zig exports `tcgetattr`/`tcsetattr` + as the kernel's TCGETS/TCSETS* ioctls, which touch only the 36 bytes + both layouts share. FreeBSD's libc struct is 44 bytes, so nothing + overruns there (checked on the box). +- **Restore of a session with a command pane failed in ReleaseSafe** + (xquktpqo): "end of dump". The walker loaded an optional by setting it + to an undefined payload and reading into `.?`. A `?[]u8`'s null is its + address, so the optional itself was undefined, and ReleaseSafe's + optimizer took it as null. `Pane.command` read one byte long and every + field after it fell out of step. Pointer-like payloads now load through + a local. Found by tracing (type.field, offset) from the loader in a + Debug and a ReleaseSafe build and diffing for the first divergence. + +The last two were invisible to every Debug suite. The gate therefore runs +the installed (ReleaseSafe) binaries through the suites that start a real +pardes. + +Gates, Debug unless said: +- `core-test`, `unit-test -Dplatform=tty` (its perf gate too), `snap` + (113/113), `replay`, `replicas` (no divergence, no wrong delta at + 30/80/207 ms), `9p-io-test`, `fs-test`, `test/daemon.py` (20 cycles), + `test/daemon.py --upgrade`, `test/fs.py`: all pass. +- `unit-test -Dplatform=gui`: 1563 of 1575, 10 skipped, and the two that + failed on main already. +- ReleaseSafe, the binaries installed: a tty in a pty with the daemon on + and off, `test/daemon.py` 20 cycles, `test/fs.py`, `snap -Doptimize= + ReleaseSafe` (113/113), `--upgrade` Debug to ReleaseSafe and back, the + GUI 5 s offscreen: all pass. +- The FreeBSD main test (ReleaseSafe on both sides, `e2e_box.py`): a + session on the box, two replica clients here and a terminal there, all + three typing. Every digest agreed (72 checks), first frames at 3.8 and + 3.9 s, a shell's answer 223 ms after Enter, 1 misprediction and none + visible. + +No local regression, ReleaseFast, interleaved base/new on the perf cores, +new/base of the medians: +- The core in process, on an idle machine (`--only medium|small --reps + 25`, three rounds): every cell 0.91 to 1.02. Run beside builds, the + big-buffer cells (`body-2m`, `body-appends`, `open`) spread 1.6x run to + run on base alone, so that run's 1.10 to 1.16 there is noise. The idle + rerun is the measure. +- A terminal's frame (the scrollback cells, beside builds): render 0.86 + to 0.90, output 0.89 to 0.91, edit-char 0.87 to 0.91. +- Startup: boot 0.93 to 1.00, first frame 0.94 to 0.96, peak memory the + same. +- Live tty keystroke-to-present: level within each CPU state. With the + machine busy, key-down is 459/469 µs against 452/465 and page-down + 370/378 against 371. Idle, it is 1553/2060 against 2033 and 1939/2102 + against 1921 (the `powersave` governor). The medians across both states + read 0.33 to 1.70 and mean nothing. +- Live GUI: present 32 to 38 ms on both (the offscreen software present), + core 220 to 560 µs on both. + +Installed 2026-10-30 05:27 into ~/.local/bin: pardes and pardes-gui +0.24 (bbcebf80c45e), ReleaseSafe. The previous ones (0.24 caffd9c1f874, +2026-10-29 08:57) are `*.bak-pre-daemon`, and pardes-v9fs is unchanged. + +## Phase 6: content cache and zstd notes + +Built 2026-10-30 (`src/replica.zig`: `Peer.farJoin`, the `Framer`'s +envelopes, `Have` and `Cache`; detached/wire.zig version 16; `walk.blobs`; +tty.zig `remote`; main.zig `remoteAttach`; libzstd in build.zig). A far +client, one through ssh, is sent a join's blobs by hash and only those it +holds nowhere, and everything the session sends it after its welcome goes +in zstd. Decisions at the forks, with the ponytail lens: + +- **Far or near, the client says.** A client dialling through ssh + (`--attach=<host>:<name>`) is far; one on this machine's own socket is + near and joins as before, byte for byte: there a hash and a compression + cost more than the bytes they save (a 100 MB file's first join takes + 1.5 s far against 0.25 s near, the table below). + `PARDES_REPLICA_FAR=1` makes a socket client far (the tests, the local + measurements), `=0` an ssh one near. The daemon reads no further than the + hello, so the word is the client's first message after it (`far`), in + the same write: the session's writer reads it before anything after the + welcome, with no round trip. +- **The protocol: the session lists, the client asks.** After the welcome + (which now says how many blobs the dump holds) a far client's writer + sends the list (`blobs`: each blob's place in the state, its length and + its BLAKE3 hash, 48 bytes), then the state with the blobs cut out (its + pieces), then flushes and waits for the client's `want` (a bit a blob), + then sends the wanted blobs, in order. The client resolves the list as + soon as it is whole and answers while the state is still on its way + (a first join pays that round trip: not done, below). A blob listed + twice (an undo step that changed nothing) is asked for once and copied. + The client says what it lacks, not what it holds: its cache can hold + thousands of blobs, the list of a join a few hundred. The step never + waits for any of it: the list is hashed on the session's writer thread + for that client, from the blobs part B pins for it. ponytail: hashed + again at each far join (BLAKE3, 3.6 GB/s); a session's later joins could + keep the hashes beside its words. +- **BLAKE3 from Zig's std, 256 bits.** A collision or a planted file must + never become wrong content, so the client hashes what it reads from disk + again before it uses it, and a file of another length or content (cut + short, corrupted, planted, longer) is a miss: deleted, and the blob is + asked for. What comes over the link is not hashed (the session said its + hash); the cache file it is written to is checked when it is read. +- **Where a far client holds blobs.** In order: the core it is leaving + (a rejoin after a divergence or an Upgrade: the old core is kept until + the new dump is whole, its blobs looked at by length and each hashed + once when one of its length is asked for), then the cache on disk. + ponytail: a scan of the old core's blobs a lookup, fine to some thousands + of blobs. +- **The cache: `$XDG_CACHE_HOME/pardes/blobs` (else `~/.cache/...`), a + file a blob named by its hash in hex, the directory 0700.** Not the + runtime directory: the blobs are worth keeping across a logout and a + reboot (a client started tomorrow) and safe to lose, and they can be + hundreds of megabytes, which the runtime directory (a tmpfs, often + capped at a tenth of memory, emptied at logout) is not for. A file is + written aside and renamed into place, so a second client never reads half + of one. A far client writes what it was sent on a thread after its load + (the first frame waits for none of it), and at its end, the blobs of its + core the cache lacks: what it made since it joined (an edit's text, its + undo snapshots), so a client started again is not sent them. A hangup + now ends a remote client the normal way, for that (and its ssh forward's + socket is unlinked: part B's last note). +- **Bounded: 1 GiB, `PARDES_BLOB_CACHE_MB` to change it; the least recently + used go first.** A hit makes its file's time now; before a store, the + directory is scanned and the oldest files deleted until what comes fits. + A join bigger than the cap stores what fits. ponytail: the scan and + sort at each store, fine to tens of thousands of files, where an index + would take over. +- **zstd, decided by its numbers (below), on the whole stream after the + welcome, not the join alone.** The session's writer puts every byte it + sends a far client through one compressor and writes envelopes + (`zstd`), each the flush of at most 1 MiB of input, so a reader unpacks + one whole into a bounded buffer; the client's `Framer` unpacks them and + hands their messages on as if they had come bare, so the dial and the + stream's reader are unchanged. The join is one frame at level 6 over an + 8 MiB window: an undo snapshot is a near copy of its text, a file's + length before or after it, and zlib's 32 KiB window never sees the copy. + What follows (units, terminal deltas) is a frame at level 3 over 1 MiB. + The client's direction is not compressed. With all of that, `ssh -C` + only spends both ends' time, and is no longer in the default command. + ponytail: a big paste through a slow link goes as it is; it would be an + envelope of its own. +- **libzstd 1.5.7, vendored as the upstream tarball** (one copy; nothing + else in the build bundles one; Zig's std has a decompressor but no + compressor). Its compressor and decompressor only, single-threaded, + without its x86 assembly, the block compressors of levels 3 and 6 alone + (`ZSTD_EXCLUDE_*`), its unused functions dropped at link; every context + made by `ZSTD_create*_advanced` with a `ZSTD_customMem` over c_heap, so + a test's allocations are its testing allocator's. + +The zstd decision, measured. A join's stream as a client reads it, taped +by a tee relay (`.scratch/daemon/amort/h/capture.py`): `real` is part B's +box session made here (pardes.zig, host_io.zig, terminal.zig, look.zig and +3 MB of the tree's sources joined, a shell with 1 MB of scrollback, 4.0 MB +of stream), `edited` the same after three edits to each text (each text's +undo snapshots: 15.8 MB), `typical` and `big` join2.py's synthetic files +(their lines differ by a counter, so everything compresses far better than +source does). Compressed whole with the CLIs, here (one core); the box's +CPU takes 0.03 s, 0.11 s and 1.04 s for real at -3, -9 and -19: + +| stream | raw | gzip -6 (ssh -C's) | zstd -3 | zstd -3, 8 MiB window | zstd -6, 8 MiB | zstd -9 | zstd -19 | +|---|---|---|---|---|---|---|---| +| real | 4.04 MB | 1.00 MB, 91 ms | 1.02 MB, 19 ms | 0.95 MB, 23 ms | 0.85 MB, 49 ms | 0.82 MB, 58 ms | 0.72 MB, 1005 ms | +| edited | 15.82 MB | 3.97 MB, 336 ms | 3.36 MB, 63 ms | 0.96 MB, 30 ms | 0.86 MB, 61 ms | 0.85 MB, 87 ms | 0.73 MB, 967 ms | +| typical | 8.33 MB | 1.22 MB, 98 ms | 0.72 MB, 19 ms | | | 0.35 MB, 39 ms | 0.20 MB, 723 ms | +| big | 77.4 MB | 11.17 MB, 768 ms | 6.94 MB, 136 ms | 4.10 MB, 131 ms | 3.71 MB, 336 ms | 4.80 MB, 467 ms | 1.82 MB, 22.4 s | + +The box's link (each payload `cat` from the box through ssh, the time +from its first byte here to its last, the median of three interleaved): + +| payload | through `ssh -C` | zstd -3, plain ssh | zstd -9, plain ssh | zstd -19, plain ssh | +|---|---|---|---|---| +| real (4.04 MB raw, 10.0 s through plain ssh) | 4.32 s | 4.82 s | 4.06 s | 4.14 s | +| edited (15.82 MB raw) | 8.90 s | 7.03 s | 4.51 s | 3.87 s | + +- **Without undo history zstd buys nothing** that the link shows: 0.82 MB + against zlib's 1.00 MB, 4.06 s against 4.32 s, within the runs' spread + (3.9 to 5.0 s). The link is not the bottleneck its 170 to 400 KB/s + suggests for a payload this size: a megabyte or less takes about 4 s + whatever it is, TCP's window growing over a 207 ms round trip. +- **With it, zstd halves the transfer** (8.90 s to 4.51 s): an undo + snapshot is a near copy of its text, which zstd's window of megabytes + matches and zlib's 32 KiB does not, and a session one works in for an + hour holds up to 256 of them a file. The window decides it, not the + level: at -3 with 8 MiB, edited is 0.96 MB where -3's own 2 MiB leaves + 3.36 MB (the 3 MB file's snapshots are 3 MB apart). +- **So zstd, at level 6 over 8 MiB**: within 4% of -9's size at about + two thirds of its time; -19 saves a little more and costs a second of the session's CPU + every 4 MB (22 s for big). The cache does not replace it: a first join + has nothing cached, and an edit made while no client was attached is new. + +The benefit, measured (`h/joincache.py` here, `h/boxcache.py` through the +box, both on `h/cachejoin.py`; ReleaseFast here and ReleaseSafe for the +box's pair, timings alone on the perf cores). Four joins of one session, +each client with its own terminal and check file, one cache between them: +the first (the cache empty); a second client while the first is attached; +that second one put off the session's state (PARDES_TEST_DIVERGE), which +joins again from the core it leaves; and one started after both ended. +The first frame is from the client's start, ssh's forward included (2.0 s +of it on the box, part B); a rejoin keeps its forward, and its time is its +dial and load (PARDES_REPLICA_STATS `join_ms`), its frame drawn after. + +- **Through the box** (ms; `real` 3.9 MB of files, a 3.9 MB dump; `edited` + the same after three edits a text, a 15.8 MB dump; main is the final + gate's ReleaseSafe pair, bbcebf80, through its default `ssh -C`): + + | session | build and ssh | first join | second client | rejoin, dial and load | started again | + |---|---|---|---|---|---| + | real | main, `ssh -C` | 6535 | 7166 | a whole join | 6952 | + | real | this line, `ssh` (the default) | 7069 | 2484 | 434 | 2619 | + | real | this line, `ssh -C` | 7623 | 2449 | 430 | 2409 | + | edited | main, `ssh -C` | 11931 | 11257 | a whole join | 10921 | + | edited | this line, `ssh` (the default) | 7079 | 2442 | 452 | 2482 | + | edited | this line, `ssh -C` | 6824 | 2412 | 460 | 2437 | + + - A client that holds the blobs is drawn 2.4 to 2.6 s after it starts + where main took 7.0 to 11.9 s: past ssh's 2.0 s, 0.4 to 0.5 s, + whatever the session's size. Its link carries the state (113 KB + before zstd) and the list. + - A first join of a session with undo history: 7.1 s where main took + 11.9 s (zstd's window). Without one, level with main (7.1 s against + 6.5 s, within the link's spread): zstd's 15% fewer bytes are worth + about the round trip the list's answer costs. + - `ssh -C` over zstd: 7.6 and 6.8 s against 7.1 and 7.1 for the first + join, the same for the others; noise either way, and dropped. + - The session's step while each joined (its stats, real): 0.3 to 1.2 ms + on the box, main's 0.3 to 0.9 ms. + +- **Here, the socket** (ms; join2.py's shapes: typical 4.1 MB of files + and 1 MB of scrollback, big 38.6 MB and 8 MB, huge a 100 MB file beside + typical's; three edits a text; near as before, far with + PARDES_REPLICA_FAR=1): + + | session | client | first join | second client | rejoin, dial and load | started again | the session's worst step during the joins | + |---|---|---|---|---|---|---| + | typical | near | 22 | 22 | 9 | 22 | 0.6 | + | typical | far | 42 | 32 | 17 | 32 | 0.5 | + | big | near | 104 | 103 | 66 | 83 | 1.2 | + | big | far | 517 | 153 | 127 | 153 | 0.8 | + | huge | near | 254 | 244 | 170 | 285 | 0.7 | + | huge | far | 1514 | 517 | 450 | 506 | 2.3 | + + - Near is part B's join unchanged (22 to 46, 87 to 117 and 247 to 288 + ms then), and the default on this machine's socket: far there costs + more than the bytes it saves (the session's hashing and compression on + a first join, the client's read and hash of what it holds after). + - A join costs the session's step at most 2.3 ms with a 100 MB file + open, far or near (the gate: under 16.7 ms). + +Gates (Debug unless said; scripts and logs in `.scratch/daemon/amort/`): +- **Correctness.** `test/daemon.py --far` (in its main run too): far + clients of a detached session with two files (one 1.3 MB) and edits; + the first sent every blob (1104 KB), a second client on this machine + from disk (3 of 4, the fourth the text the first had just made), its + rejoin after a forced divergence from the core it left (4 of 4), one + started after both ended from disk (6 of 6: what they made, kept at their + end), one after every cache file was corrupted or cut in half by hand + sent them all again (5470 KB), and two with a 1 MiB cap held by older + files: those went first and the cache stayed under its cap. Every + client's digest was the session's at every quiet point (11 to 17 checks + a run, none differed), Debug and ReleaseSafe. Through the box, every + check of the four joins above agreed (28 to 41 a run). `zig build + replicas`: 4 seeds at 30, 80 and 207 ms, none diverged, none unequal, no + delta wrong. `test/daemon.py`'s replica tests pass, near and far. +- **New tests** (core-test, `-Dtest-filter=blob`): a cached blob is hashed + again as it is read, and one cut short, corrupted, planted or longer is + a miss and goes; the cache stays under its cap, the least recently used + going first (a read makes a file the newest; a file not a blob's counts + and goes by its time); a far join over a socket pair, the session's + writer and reader threads against the client's dial, is sent each blob + once: again with them on disk, or in the core it leaves, the state + alone, and a cached file gone bad sends that blob again; a near join's + dump is the session's byte for byte. + +- **Suites**, once at the end: `core-test` (its run beside the box + measurements had the 9P failure above; again, all pass), `unit-test + -Dplatform=tty` (its perf gate, and shared-libs-check, which now refuses + a second zstd too), `snap` 113 of 113, `replay` (113 scripts, 20 monkey + seeds, none panicked or diverged), `replicas`, `9p-io-test`, `fs-test`, + `test/daemon.py` 20 cycles (its far clients included), `--upgrade` + (unstamped Debug to stamped), `test/fs.py` (49 s): pass. + `unit-test -Dplatform=gui`: 1568 of 1580, 10 skipped, and the two that + fail on main (`tagline pointer mapping...`, `a tag caret past its row's + end...`). +- **ReleaseSafe** (`zig build -Doptimize=ReleaseSafe -Dhome-install=false`, + the binaries the box measurements ran): a tty in a pty with the daemon on + and off, `test/daemon.py` 20 cycles, `test/daemon.py --far`, `test/fs.py` + (39 s), `snap -Doptimize=ReleaseSafe` (113 of 113) and the cache tests + (`core-test -Doptimize=ReleaseSafe -Dtest-filter=blob`): pass. +- **No local regression.** ReleaseFast, interleaved base new base new base + new base on the perf cores (`core_optimize` fast on both sides), the + median of each side's runs, new/base, beside base's own A/A; builds ran + on the other cores throughout, so every cell is in that state: + - the core in process (`pardes-perf --reps 15`): the opens 0.90 to 0.99, + render 0.94 to 0.99, the keys, page-down and wheel 0.80 to 1.03, the + edits 0.90 to 1.05, body-2m 0.99 to 1.04, body-appends 0.92 to 1.07 + (its A/A 0.99 to 1.30), deep-sel 0.89 to 0.97 (and longline's 364 ms + on base, 0.1 ms on main already); a terminal's cells + (sb-64k to sb-8m) 0.87 to 1.01. The A/A elsewhere 0.94 to 1.05. + - live tty keystroke to present (base's `pardes-perf-live --live`, + present p50 / core p50): key-down 1.01 / 0.99, key-right 1.01 / 0.95, + edit-char 0.97 / 0.98, page-down 0.99 / 0.92, wheel 1.08 / 1.04 (A/A + 1.05 to 1.16); a 1M-line open 1.06 (A/A 0.92); `yes | head -c 32M` + 1.22 (A/A 1.13), its runs 4.1 to 7.3 s on this line and 5.4 to 6.7 on + base, as main's 7.0 to 7.3 beside base's 6.0 to 7.3 were in the final + gate. A session's own path is untouched by this change: a session with + no far client hashes and compresses nothing, and a near one is sent + its dump as before. + +Found on the way: +- **PARDES_REPLICA_CHECK keeps a session's units coming.** With two clients + checking, an idle session sends a unit about every 300 ms on main too + (`h/idle.py`: 27 checks in 8 s, near or far, main's build alike): a + check wakes the session's pump, whose unit makes the client's state + "changed", and the client asks again once quiet. Only the test knob + does it; a harness that expects a divergence to wait for a key it sends + finds the client joined again before (the measurements above wait for + the first link's stats line instead). +- **A rejoin's "joined again" line can be gone before it is drawn**: it + lasts until the next key, and keys typed during a long rejoin (a 100 MB + file's) come first. As designed; the harnesses do not look for it. +- **Harness traps.** A terminal's frame request split across two reads + went unanswered (test/daemon.py's `Term` now keeps the last bytes of a + read); keys written right after a first frame raced the terminal's + answer to it, an answer behind an Esc in one read being an Alt chord + (the far test waits 0.3 s, and writes the Esc on its own); a `pkill -f` + whose pattern was in the shell's own argv killed the shell. +- The core-test run of the suites had one failure, `9p_io`'s "a change + waits while the editor is out mid-step", with the box measurements and + builds running; alone three times, and the whole core-test again, it + passed. + +Not done, and why: +- **A first join pays the list's round trip** (207 ms on the box): the + wanted blobs wait for the client's answer, and the state (113 KB before + zstd) no longer hides it. A client with nothing cached could say so in + its `far` word and be sent everything at once; it is a first join on a + new machine only. +- **A blob is hashed again at every far join** on the session's writer + thread (a 254 MB dump: 70 ms there). The hashes could be kept beside + the blob words. +- **What a client sends is not compressed** (its keys, and a paste): with + `-C` gone, a megabyte's paste crosses the box's link raw. +- **A client killed outright keeps on disk only what it was sent**: what + it made since (its edits' texts and undo) is written at its end, a + hangup's included, but not on a kill -9; the next join gets those again, + in zstd. +- **The cache is per user, unencrypted, 0700**: a blob of a file the user + could read elsewhere, kept until evicted or the cache is cleared. + +## Phase 6: GUI replica notes + +Built 2026-10-30 (`src/gui/gui.zig` `remote` and the window session's +`Peers`, `src/replica.zig` `Link`, `src/views.zig` `within`, the +`builtins.zig` registry's `here`, `Event.frontend`, wire 16; +`test/daemon.py --window-clients`). A window is now a replica client, of a +terminal's session or a window's, and a window's session serves replicas of +either kind. Decisions at the forks, with the ponytail lens: + +- **The window client is the tty's, with a window for a terminal.** + `pardes-gui --attach=<socket>:<name>` and `--attach=<host>:<name>` (ssh, + main.zig `remoteAttach`, now the frontend's own `remote`) make the window + as a session's is (`runNative`) and run `remoteSession` instead of + `localSession`: the dial, the dump's replica checked against the + session's digest, a reader thread, the rejoin after a divergence or an + Upgrade's `refused`, the end in words. What is frontend-free in the tty's + `Remote` moved to `replica.Link` and both use it (`input`, `take`, + `drawn`, `quietCheck`, `rejoins`, `redial`, the stats and checks); the + tty's keeps its draw. `pardes-gui --attach` and `--attach=<name>` (no + colon) re-open a local session's window through the daemon as before. +- **The window's 52 writes into its core are one funnel, read over the + client's own view.** The window feeds its core through `Input.post` + (dispatch, the wheel, the touch and gamepad paths, the timedemo, the + test feed), and a handful of frame chores call `core.update` themselves + (its size, the pointer under a frame that moved, the smooth scroll). In + a client `Input.post` appends to the client's list (`Client.post`, texts copied) and nothing writes the + core. Mapping a pointer reads the core's view fields (its rects, its + presentation, its row metrics), and another client's input may have made + the replica's live view another's: so the window's work runs inside + `Replica.within` (`views.within`: the live view parked, the client's + unparked, put back after exactly as it was, as `drawOne` draws), and its + inputs are applied after, one `Link.input` each. With one client the + live view is its own and `within` is a call. +- **No pump in the client.** Its core moves only by its own inputs and the + session's units; it draws its view between steps (`Replica.draw` through + the window's own `present`), puts back what presenting writes in the + core (a skipped image, a frame owed) and records its frames as the tty + client does. The window's per-frame host chores that write the core stay + out of a client: opacity, font requests and their answers, a scene + effect the GPU refused, user shader files (their compiles' messages). + ponytail: a client's window keeps its own font, opacity and shader + files; the built-in scene effects follow the session's settings. +- **The wheel in a client moves a row a turn.** The window's glide and a + touchpad's fraction run on the core's clock, which in a client is the + session's and moves only with its units. ponytail: whole rows, no glide; + the local window's is unchanged. +- **A client says its frontend first** (`Event.frontend`: windowed, native + images and patches; wire 16, client tag 0x20): the view the session opens + for a joiner copies the live view's, and a window's view of a terminal's + session would have drawn pictures as cells and the window's words would + have been refused there. A terminal's client says it is no window, and + draws pictures as cells (ponytail: a kitty-graphics terminal client was + drawing none before, its shm never probed). +- **The terminal's build and the window's step alike.** The semantics hash + named the platform, so a window could not join a terminal's session (the + FreeBSD box has terminals only) nor a terminal a window's. Their cores + differ in the step only by the window's own words (Fonts, Pet, Font, + TaglineSize, WindowOpacity, Ligatures, Lift and its family, the scene + effects, their leader paths `SPC t f` and `SPC t r`) and the Settings + report. Three ways: refuse cross-build joins (the box and a window's + clients impossible); accept known divergences (Help's index, `SPC t f` + and Settings would each make the other build's clients join again); or + give both builds every word and let the *view's frontend* decide + (kakoune's way: each client its own). The last: a desktop build's + registry is the window's (`builtins.capabilities`), and + `builtins.registry.here` says whether the live view takes a word (a + window's own words in a window's view alone, `Pardes.windowed`, which a + window's build starts true and a client says). `runBuiltinFrom` refuses + the rest in the words the terminal said before ("Lift is GUI-only, + invalid here"), and the leader paths, Help's index, `/commands`, the root + ctl's settings and the Settings report list the view's own. The render + keeps the build's own frontend (`builtins.frontend`: the tty draws no + lift). Local behaviour is unchanged: the 113 snapshots, which hold the + tty's Help index and leader groups, pass as they were. The hash names a + family (`desktop`) for the two; the tty binary carries fonts.zig and the + window's builtins (a few KB). +- **A window's session takes replicas at its step boundary,** as the tty's + does: `Peers` in the window's Shell, its wake an SDL event, `boundary` + after each pump, a Restore and an Upgrade letting clients go + (`dropAll`, `refuseAll`), Detach in a client's view ending that client. + What the tty's session does with clients, the window's does too: one + person's input a wait (`split`: the next SDL input held for the next + wait, so a pump is one unit of it), a pump woken only by a client's post + reads no new time (`mail_only`), and its last window closed with clients + attached is a Detach, not the session's end. +- **A client's view is no window here.** The window's Shell kept the live + view's window in the Gui (`follow`); a client's view live (its input the + last) has none, so `follow` keeps the window it holds (`windowed`, a mask + of the views that are windows), and what the window does to its own view + makes that view live first (`windowView`: an SDL event of the window, + `Input.post`, the frame chores when one is due: its size, its scroll, a + font). Found by the first test with a window typing beside a client: the + window's keys went into the client's view. +- **What the window writes into the core outside a step is taped.** The + window's font (observed, requested, taken, acknowledged, refused), its + opacity's answer and a refused scene effect are core calls now + (`tape.Input`), so a window session's clients replay them; the rest of + its writes were taped already (sizes, pointers, panel acknowledgements, + messages). ponytail: an adopted shell's pane (`Adopt`) and a terminal + reader's failure still write the core directly; neither is taped. +- **The far path is the tty's** (amort's line, merged under this one): + through ssh the window client is far as the tty's is + (`PARDES_REPLICA_FAR` either way), its join's blobs found by hash in the + core it leaves or the cache on disk, its stream in zstd, what it made + kept at its end (`replica.Have`, the same calls in `remoteSession` as in + tty.zig's `remote`, through `Link.load` and `Link.redial`). One wire + version, 16, carries amort's far word (the first message after the + hello) and the frontend (a client's first input). +- **A check once a quiet while, not once a wake.** The tty's client asked + the session's digest on its first wake after a change; a window session + in test mode pumps a frame a tick, a unit each, so no wake ever came + first. A client now asks once it is quiet, 300 ms after its last input + and its last check (`quietCheck`); the answer is in stream order, so it + holds with units flowing. + +Found on the way: +- **A pty/run's answer read the replica's emulator** (vnolzlvq). A run's end + (`ninep/pty.zig finish`) took the command's output off the terminal in + the step; a screen-state replica's emulator is its deltas', which a + lagging client's are states behind, and replaying the end read pins into + pages it never had: ghostty's `leftWrap` overflowed (a window client + stopped while a shell flooded the session aborted 3 runs in 8). The + output is taped now (`.prompt`, the same site as a prompt read after + output); none in 6 since. +- **A client cut off while lagging could end in silence** (ussmwtkk): its + own write found the link shut before its reader did, which ended the + loop as a quit, exit 0, no words. A failed write is the session's hangup. +- **The rejoin's drain** (the coordinator's ymzywopz, on main): the window + client joins its reader before it drains, so it never had that race. +- **A second's turn is a misprediction** (ssruwvql): an edit stamps its + file with the clock's second, which a replica guesses (its last unit's + time, the time since and the link's round trip); a key the session takes + in another second than the guess goes the slow way, replayed. Right, and + rare live (the round trip's jitter against a second); but a test's + replica has no round trip yet and its authority takes the keys after all + are typed, and `another client's input first` counted exact fast + confirmations: under load in a Debug window build (a slower step) they + straddled a second in 11 runs of 40. The two tests that count them start + early in a second now: 0 of 30. +- **Harness traps:** a window's test feed is a pipe, whose lone ESC waits + for the next byte (the parser cannot tell it from a sequence's start), so + the tests send Escape as `CSI 27 u`; a terminal client's Esc is resolved + by the 50 ms wait for more input, which the test terminal's answers to + frames (`CSI 0 n`) can keep refilling while other clients' units arrive + (main's `normal_again`); a window session in test mode starts no daemon + unless PARDES_DAEMON says so. + +Measured. ReleaseFast, unstamped (one semantics here and on the box), the +timings alone on the perf cores (lock.sh), the windows offscreen (SDL's +offscreen driver, software present); the harness scripts are in +`.scratch/daemon/guirep/live/`. + +- **Keystroke to present on a window client, local-only actions** (base's + `pardes-perf-live --live-gui`, its timedemo's keys on a 50k-line file, + through `gui-live.sh`: a local window, or a window client of a terminal's + session detached on that file, through test/delay_relay.py at each RTT + and through the box's link; two rounds; p50 (p99), ms): + + | | key-down | key-right | edit-char | page-down | wheel | + |---|---|---|---|---|---| + | a local window, this line | 31.1 (40.3) / 31.5 (45.5) | 34.8 (44.2) / 35.3 (41.0) | 35.9 (44.4) / 36.5 (46.2) | 37.0 (50.0) / 37.8 (50.3) | 36.5 (47.3) / 36.9 (52.8) | + | a local window, base | 29.0 (35.9) / 31.5 (37.5) | 35.1 (45.9) / 35.3 (42.2) | 36.2 (42.2) / 36.5 (45.1) | 37.2 (48.1) / 37.5 (52.5) | 36.5 (50.0) / 36.8 (44.2) | + | client, direct | 31.3 (40.4) / 28.6 (36.2) | 35.1 (47.3) / 35.1 (54.4) | 36.7 (41.1) / 35.8 (44.0) | 36.0 (46.4) / 35.4 (40.8) | 35.8 (45.1) / 35.2 (45.8) | + | client, relay 30 ms | 29.9 (54.6) / 29.9 (42.7) | 34.9 (45.8) / 36.4 (53.6) | 35.7 (54.0) / 35.8 (45.8) | 34.9 (52.1) / 35.1 (47.3) | 35.1 (44.0) / 35.1 (42.6) | + | client, relay 80 ms | 32.7 (39.5) / 30.5 (36.6) | 38.1 (46.6) / 34.9 (44.0) | 39.1 (57.4) / 35.6 (46.1) | 37.7 (56.5) / 35.1 (43.9) | 38.0 (46.7) / 34.8 (44.5) | + | client, relay 207 ms | 31.1 (35.9) / 30.6 (36.2) | 35.6 (46.3) / 34.6 (59.9) | 35.9 (45.4) / 35.5 (55.3) | 27.9 (56.6) / 34.9 (47.8) | 36.9 (55.3) / 34.6 (50.8) | + | client, the box (far, zstd) | 33.7 (48.1) / 28.3 (43.9) | 31.1 (43.4) / 30.2 (46.4) | 30.9 (44.6) / 29.4 (39.3) | 33.8 (51.0) / 29.6 (48.4) | 28.7 (49.9) / 30.1 (43.2) | + + - **A window client's local-only action is as fast as a local window's + at every RTT, the box's included.** Both are the offscreen software + present's (30 to 38 ms, the final gate's GUI numbers): what the harness + can tell apart is the core's share, the client's render 0.4 to 0.5 ms + p50 (0.1 to 0.9 through the box) against a local window's update and + render 0.24 to 0.6 ms. The client's render runs on its rollback epoch's + allocator, which a local core's does not. + - **Visible rollbacks** (a frame drawn after an input the session + recorded otherwise, an upper bound; of 664 inputs a run): none direct + and at 30 ms, 10 and 0 at 80 ms, 0 and 9 at 207 ms, 15 to 24 through + the box (2.3 to 3.6%). Every other rollback (the session's own pumps' + units before the client's pending inputs: 536 to 610 a run here, 268 + to 295 through the box) is put back and replayed before the next + frame, p99 2.6 to 8.9 ms. +- **Terminal echo** (`echo-gui.py`: a key read off the window's test feed, + to the end of the GPU frame that shows it, PARDES_TEST_LATENCY's notes; + 40 keys; p50 / p90, ms): + + | | file (insert) | shell, predicted | shell, not predicted (PARDES_NO_ECHO) | + |---|---|---|---| + | a local window | 2.8 / 4.6 | | | + | client, direct | 2.5 / 4.5 | 4.3 / 9.7 | 25.5 / 29.7 | + | client, relay 30 ms | 2.6 / 4.7 | 5.1 / 11.0 | 60.4 / 64.8 | + | client, relay 80 ms | 3.1 / 4.6 | 4.7 / 4.9 | 101.9 / 121.9 | + | client, relay 207 ms | 2.7 / 8.2 | 4.4 / 10.0 | 236.8 / 248.9 | + | client, the box (far, zstd) | 4.2 / 5.2 | 4.2 / 4.8 | 242.5 / 243.3 | + + (Through the box, the harness's ssh had `-C` beside the stream's zstd; + a predicted key never waits for the link.) Every key of the shell's was + predicted and confirmed (40 of 40, at each RTT): a key at a prompt shows + in the window's next frame, as a key into a file does, the 0.5 ms of the + tty client's being a terminal's write where a window's is a frame (its + software present's 2.5 ms). Not predicted, it is a round trip and the + session's paced output. +- **Through the box, its join** (far, amort's path): the main test's 400 + lines, 4 blobs, 51 KB sent, dial and load 0.63 s (ReleaseSafe 0.99 s), + its first frame about 2.6 s after it started (ssh's forward is 2.0 s); + its rejoin after a forced divergence from the core it left, 4 blobs of + 4, nothing sent, 0.42 s. The live harness's 3.1 MB file: 3126 KB, 2.3 to + 2.4 s. +- **The session's step while a window client lags** (`lag-gui.py`: a shell + printing a line a millisecond for 12 s, a terminal's client typing a key + every 30 ms, the window stopped with SIGSTOP and cut off at a 256 KB lag; + µs, p50 / p99 / max; ReleaseSafe): the terminal client's inputs 30 / 75 / + 289, the boundaries 49 / 148 / 764, the window cut off (`cut 1`), and let + go on it said "let this client go" and ended. + +Gates (Debug unless said; scripts and logs in `.scratch/daemon/guirep/`): +- **Correctness.** `test/daemon.py --window-clients` (new): window clients + of a terminal's session at 0, 30, 80 and 207 ms beside a terminal's + client, every check of every client the session's digest (28 a run, + none differed); one put off the session's state joined again, and again + across an Upgrade (agreed 7 times), and detached; its far client's first + join 9 blobs, 57 KB, its rejoins 9 of 9 and 10 of 10 from the core it + left, nothing sent; a window's session served a terminal's client and a + window's (agreed 9 and 7 times); nothing left behind. Debug and + ReleaseSafe. `zig build replicas` at 30, 80 and 207 ms: none diverged, + 1545, 1590 and 1512 deltas checked, none wrong. Through the box + (`live/e2e_box_gui.py`: a session on the box, a window client and a + terminal's client from here, a terminal on the box attached beside them, + a file's edits, scrolls and moves, a shell's command, a second window + far, forced off the session's state and joined again): every check + agreed (the window 128, the terminal's 132; ReleaseSafe 131 and 134), + none mispredicted, the second window's first join 4 blobs 51 KB, its + rejoin 4 of 4 from the core it left, nothing sent; nothing left on the + box. +- **The window's own reopen** (`test/daemon.py --gui`, 5 detach and attach + cycles, a second window detached alone): pass, Debug and ReleaseSafe + (SDL's offscreen driver). +- **Suites**, once at the end (before the tests' fix and the rebase below): + `core-test -Dplatform=tty`, `unit-test -Dplatform=tty` (its perf gate), + `snap` 113 of 113, `replay` (113 scripts, 20 monkey seeds, none panicked + or diverged), `replicas`, `9p-io-test`, `fs-test`, `test/daemon.py` 20 + cycles, `--window-clients`, `--upgrade` (unstamped Debug to stamped), + `test/fs.py` (49 s): pass. + `unit-test -Dplatform=gui`: 1568 of 1580, 10 skipped, and the two that + fail on main (`tagline pointer mapping...`, `a tag caret past its row's + end...`); `core-test -Dplatform=gui`: the caret one, and in two of its + full runs a replica test's count of fast confirmations (below, ssruwvql; + after it, the core-test runs pass but for the caret one). + After the rebase onto main's `--ps` (prvoxvzk): `core-test` tty and gui, + `test/daemon.py` 20 cycles (its `manage` included), `--window-clients` + (its `--ps` checks: a terminal's session with window clients listed + alone as `(tty, detached)`, a window's serving clients as `(gui, + attached)`), `--gui` 5 cycles: pass, `core-test -Dplatform=gui` but + for the caret one. +- **ReleaseSafe** (`-Doptimize=ReleaseSafe -Dhome-install=false`, the + binaries the box's second run used): `--version`, a tty in a pty with the + daemon on and off, a window session offscreen for 5 s, `test/daemon.py` + 20 cycles, `--window-clients`, `--gui`, the lag test (`live/lag-gui.py`, + above), `test/fs.py` (38 s), `snap -Doptimize=ReleaseSafe` 113 of 113: + pass. Again after the rebase, built afresh: `test/daemon.py` 20 cycles, + `--window-clients` (its `--ps` checks), `--manage`, `--gui` 5 cycles, + `test/fs.py` (38 s): pass. +- **No local regression.** ReleaseFast, interleaved base new base new base + on the perf cores (`core_optimize` fast on both sides; base is + `.scratch/daemon/base`'s), the median of each side's runs, new/base, + beside base's own A/A; the new binaries are this line before its rebase + onto main's last two (a test's rest and `--ps`, neither on these paths): + - the core in process (`pardes-perf --reps 15`, idle cores, the second + of two runs): the opens 0.98 to 1.03, render 0.96 to 0.98, the keys, + page-down and wheel 0.95 to 1.02 (longline's 0.79 to 0.87, main's + already), the edits 0.90 to 1.03, body-2m 0.95 to 1.02, body-appends 0.95 to 1.02, + deep-sel 0.92 to 0.99 (longline's 360 ms on base, 0.1 ms on main + already), a terminal's cells (sb-64k to sb-8m) 0.88 to 1.02. The first + run's two outliers did not come again: body-appends on the board's + fixture 1.29 (A/A 0.96; then 1.02 alone, A/A 0.99, and 1.06 with the + other cores busy) and resize-cols at 1M lines 1.10 (then 1.02). + - live tty keystroke to present (base's `pardes-perf-live --live`, the + other cores kept busy by spinners at nice 19, as idle cores let the + powersave governor put the package to sleep between keys: base's runs + read 347 to 1651 µs; present p50 / core p50): key-down 1.03 / 1.03, + key-right 1.00 / 0.94, edit-char 1.00 / 0.99, page-down 1.01 / 0.95, + wheel 1.05 / 1.05 (A/A 1.00 to 1.02; base's runs 389 to 469 µs, this + line's 444 to 480), a 1M-line open 1.01, `yes | head -c 32M` 0.63 (A/A + 1.03). + - live window keystroke to present (`--live-gui`, offscreen; present / + core): idle 0.93 to 0.98 / 0.94 to 1.03 (A/A 0.95 to 1.05), busy 0.90 + to 0.97 / 0.98 to 1.03 (A/A 0.85 to 0.97 / 0.97 to 1.10). + +Not done, and why: +- **A window client paints nothing ahead.** A key whose frame the session + owes (an image's, a PDF's lane) waits for its unit; a local window's + paint-ahead is the core's, which a client's moves only by units. +- **No glide in a client, and a touchpad's fractions round to rows** (the + core's clock is the session's there, above). +- **The notice a client gets is in its title and its log, not its frame** + (the session's words: joined again, let go, refused). The tty's draws + them on its notice band, which the window's frame has no slot for + outside the core. +- **A client's window keeps its own font, opacity and shader files**: the + session's `Font`, `Fonts` and `WindowOpacity` move the session's own + window, and a client's follows them only through its core (the scene + effects do; the font does not). `Fonts` reads the machine's font + directories untaped, so its list is the session machine's. +- **An EffectCode's sources differ by backend** (Vulkan or GL): a client of + another build's window reads the session's. +- **A terminal's client draws a window session's pictures as cells**, never + through kitty graphics (no shm across machines). +- **Adopt and a terminal reader's failure write the core untaped** (above): + a client replaying them would diverge and join again. +- **A rejoin opens a new view**: the old one's place, selection and scroll + are the session's to keep, and it lets the old view go. +- **A hit test reads the live view**, which `within` makes the client's for + its own events; a frame chore outside one (a pointer's hover after a + unit moved the frame) reads it as the unit left it. +- **A window client has no SIGHUP to quit on**: its ssh's end is the link's, + read as a hangup. +- **One window a client**: a session's second window through one client + would be a second client. + +## The final gate, again, on everything merged + +Run 2026-10-30 on main lprqtsom (the content cache and zstd, the GUI +replica client, `--ps`/`--kill`, the four flaky tests' fixes) with the +same scripts (`.scratch/daemon/final/final2.sh`). + +- **Debug:** + - `core-test` tty passes. `core-test` gui and `unit-test -Dplatform=gui` + fail only on the two tests main already failed, 1568 of 1580 with 10 + skipped. + - `unit-test` tty, `snap` (113/113), `replay`, `replicas`, `9p-io-test`, + `fs-test`, `test/daemon.py` (20 cycles, `--manage` and `--far` + included), `--upgrade`, `--window-clients`, `--gui` (5 cycles) and + `test/fs.py` pass. +- **ReleaseSafe, the binaries installed:** + - a tty started in a pty, daemon on and off; + - `test/daemon.py` 20 cycles, `--window-clients`, `--gui` 5, `--upgrade` + from Debug, `test/fs.py`, `snap -Doptimize=ReleaseSafe` (113/113). + + All pass. +- **The FreeBSD main test:** ReleaseSafe at both ends. Two replica clients + here and a terminal on the box. Every one of 130 checks agreed, with 0 + mispredicted. First frames came 2.5 and 2.6 s after start (3.8 s before + the content cache and zstd), and a shell answered 224 ms after Enter. +- **The A/B against base:** ReleaseFast, interleaved, idle machine, + `core_optimize` fast on both sides. + - The core: medium 0.89 to 1.11, small 0.93 to 1.08. The edit cells + were the high end. Rechecked on the medium file with 40 reps and four + rounds, edit-twin reads 0.99, edit-char 1.00, key-down 0.99 and render + 0.95: the noise this cell's A/A showed before (0.79 to 1.46). + - Live tty keystroke to present: each run within base's own spread + (key-down 1769 to 2071 µs against 1586 to 2007). + - Live window: present 32.7 to 46.9 ms against 32.3 to 48.6, core 227 + to 568 µs against 229 to 581. +- **Found on the way, not fixed:** once, a window client of a session on + the box diverged at every rejoin, each time at the first unit's digest + (tape byte 119), for the 25 s it ran. The session was a directory and two + shells. + - Not reproduced since: not by the same launch, not by four session + shapes, and not by the session run with per-field digests + (PARDES_REPLICA_TRACE, which never diverged). + - The single-word digest is `walk.digest` over the whole core. The + per-field parts cover the core's fields, its stream and each pane's + Pane, File.State and Places. So the word covers something the parts do + not. + - The next step is a client that, when a word differs, asks the session + for that unit's parts, so the field is named the first time. + - The client heals by rejoining, at the cost of a redraw each time. + +## The remote bugs the user hit + +> due to the ping I know it won't be possible to have the same snappy +> feeling as the local version, but these kinds of errors have nothing to +> do with this limitation, you should do it much better, and measure it +> too since I don't want to catch those myself + +The user's window client of the box blinked, lost its panes and jumped its +cursor: each was a divergence, and the rejoin after it opened a fresh +view. An adversarial soak of replica clients (`test/soak.py`, a session +here or on the box, terminal and window clients through a jittered delay +relay, a seeded monkey of a person's actions) found the causes; the fixers +took them by root cause, each with a test that fails before its fix. Ids +are the hunt's (`.scratch/daemon/hunt/findings.md`). + +### fix3: what the replica stream carried in its build's own words + +A unit of the replica stream is its session's tape, written by +`walk.encode`, which said "same build only": a window client (pardes-gui) +of a terminal's session, or of the box's, read another build's bytes. +Found by the variant hunt (V2, E), with real ReleaseSafe executables. + +- **An error crossed as its number** (`@intFromError`, read back with + `@errorFromInt`). The numbers are a build's own: 451 of 505 shared names + are numbered otherwise in the Linux window's build than in the terminal's, + 244 between the Linux and the FreeBSD terminal's (FileNotFound is 62 in + both terminals and 60 in the window, whose 62 is NoDefaultValue). A + terminal's tape replayed by pardes-gui panicked at its first failed read + (`unexpected error code, found error.NonLocalPath`); a replica read the + wrong error, and diverged. + - Fix: an error is taped by its name. A set decodes it among its own + names; a name this build lacks is `error.NoSuchError`, a divergence, + never another error. `anyerror` keeps no list of names, so the tape's + five (a host's reports: `shell_failed`, `save_failed`, ...) read back + as one of `tape.Failure`, the union of the sets a host reports from. + ponytail: kept by hand; one outside it panics a Debug build as it is + taped, and is a release build's divergence. +- **A stat crossed as `std.Io.File.Stat`** (the `os_stat` site, /os over + 9P), whose `nlink` is 4 bytes on Linux and 8 on FreeBSD and `permissions` + 4 and 2: a Linux client read a FreeBSD session's stat out of step. Fix: + the site returns `fs.OsStat` (kind, size, mtime), the same everywhere. +- **The guard:** `tape.shape`, what every type the tape carries is laid + out as (each field, element and tag, an enum's names and numbers; an + error as an error), goes into `wire.semantics`. Builds whose tapes differ + in shape refuse each other's replicas in words at the hello, as builds of + other options do; a site whose result `tape.results` does not list is a + compile error. The sweep that set it (every type the tape reaches, from + the Linux terminal, the Linux window and the FreeBSD terminal builds): the + stat was the only one laid out otherwise. All three builds now hash to + one semantics; with the stat taped as before, the Linux and the FreeBSD + build hash to two. No wire version for it: every build's semantics + changes with it, so an older build is refused in words. +- **A search's answer was the build mode's (E).** regexp.zig's step budget + was 4M in Debug and 32M in a release build, and semantics names no mode: + `\s*(\w+)\s*=` over 200 letters was TooSlow in a Debug replica and no + match in its ReleaseSafe session. Fix: 32M in every mode (a Debug build's + worst search takes about two seconds). +- **Found by the cross-build replay, the same class:** a core's first view + was a window's by the build's default (`windowed = platform == .gui`), so + a window's build replaying a terminal's session started otherwise (its + init file's window settings applied); now an option (`Options.windowed`), + taped with the rest. And a window's font words (`observe_font` and the + three beside it) were no-ops in a terminal's build, so a terminal's + replica of a window's session diverged at the window's first font; every + hosted build takes them now. +- **Tests:** walk.zig's "an error is taped by its name" and regexp.zig's + "a search's answer is every build mode's" (from the variant hunt; it + fails in Debug before). `test/cross_replay.py <recorder> <replayer>...`: + a session that meets the host's failures (a Look of a fifo and of a + directory, a Save into a file, a Restore of no dump, a stat and a read + under /os), recorded by one build and replayed by the others, and + `--record` and `--replay` alone for the box. Another frontend's tape must + agree in the replica that draws nothing, as a replica client draws none of + its session's frames: the two that draw meet the other build's drawing, + which owes frames for animations of its own (on 132 snapshot tapes, a + window's build agrees in all three on 13). `zig build + replay -Dreplay-with=<pardes-gui>` replays every snapshot script's tape + with the window's build as well. + - Before: pardes-gui panicked replaying a terminal's tape, and live, a + window client (main's ReleaseSafe) of a session on the box (main's) + panicked the moment the session looked at a fifo: `unexpected error + code, found error.NonLocalPath`. + - After: every direction agrees, tty to gui and back, the FreeBSD build's + tape replayed here by both Linux builds and theirs on the box; live, a + window and a terminal client here of a session on the box agree after + the same failures. +- **The second round (H19, the linger leak, H15, the web build):** + - **A Linux client could not join a FreeBSD session holding a + +LocationsConfig (H19):** `the session's dump loaded as another + session`. An enum was digested by its number, and `registry.Builtin`, + made at compile time, has Tty9p in a Linux build only, so the pane's + origin (`Output.State.from = .{ .cmd = .LocationsConfig }`) was another + number in each. Found by one dump loaded in both builds, its digests + printed field by field. Fix: an enum is digested by its name's word (a + fixed-width hash, looked up by number), as the dump already reads it by + name; one numbered by hand stays its number. Before, main's window + client of main's session on the box could not join; after, a terminal + and a window client here join a FreeBSD session holding a +Search, a + directory, a renamed file and a +LocationsConfig, and agree. + cross_replay.py's session holds the last two (a FreeBSD tape replayed + here diverged at `pane 2.file.output`). + - **A detached session's messages went in the host's wait** (tty + waitInput, gui waitDetached), untaped, so a replay, and every replica + of a session nobody attached, kept a message its session had let go; + with `MessageLinger 0` (test/daemon.py's sessions) every tape whose + session said a failure diverged at pane 0's message. They go in the + pump now, after its wait, the clock read taped. cross_replay.py's + session says `MessageLinger 0` again. + - **Adopt and a window's failed terminal reader wrote the core between + steps (H15):** the Tty word run and the greeting cleared straight on + the core, and a pane's mode set; every client of a session that took in + a dead session's shells joined again. Now inputs (`adoptShell`, + `readerFailed`); replica.zig's test fails with the old writes. + - **`zig build web` failed** on main at terminal.zig:601 (`c_allocator` + with no libc), and at walk.zig's blob lists behind it: both are a + hosted build's alone now. The web build passes `touch.mjs`; the web + driver's own CDP-deadline test fails as it did, untouched. + - **A Grep or Find a 9P write runs** searches on the loop + (Stream.runHere, so that the write's answer says whether it found + anything), and its reads were no tape site: every replica of a session + a script greps in joined again (`a syscall that is no tape site, made + replaying`). The search is a site now (`search`), its batches what a + replica takes, as it takes a worker's (Event.search). cross_replay.py's + session greps and finds; a terminal client of a session a script greps + in agrees. + - **A +Search's rows were recoloured in place** (`Query.recolor`, as a + paint lands): the rows are a blob, whose word a digest keeps by address + and a rollback copy pins, so a release session kept the word of their + colours before; an edit moved them into `dropped_marks` with it. A + replica that hashed them at another time diverged there (fixer 1's soak, + `pane 5.file.dropped_marks`), and a client loading the session's dump + hashed the colours it carried: "loaded as another session". They are + made again now (`locations.cloneRows`). `dropped_marks` stays in the + digest: an undo back to its text reads it. Stream's test fails in Debug + before (`written in place`). + +### fix2: what a step read that only one side had, and units that ended undrained + +Each of these made every client of a session diverge at the same unit, so +the session's step and its replicas' replay disagreed. None was a lag. + +- **A step read what only the authority holds** (H4, H8, H11, H12, H13, and + the variant hunt's list). The store's closed panes and Recent are the + authority's (`.authority`: every window of the user writes the store's + file); a replica keeps its own, which has no file and only what it + replayed since it joined. Steps made digested state of them: + - the Panes picker's closed rows and Recent's rows (H4: every client + diverged at the key that opened them; H13: a client on another machine + read its own store); + - `trackJump`, at every step, dropped a closed file's jumps when Recent + did not know the file (H11: once a pane had closed, every client that + joined diverged at its first unit, forever, in `jumps njumps jcur`); + - `noteClosed` looked in Recent before it asked the host whether the file + is there (`recentKeeps`, a taped stat), so the session skipped a + syscall the replica made (H12: every Del of a file crashed the client); + - a closed pane brought back (`Store.reopen`: a Look at `@pN`, a Panes + row), a jump to a closed file and the +Jumps rows (`recentByHash`); + - and, the same class: a Look or exec handed to a script reading a pane's + events (`pane.fs.readers`, the 9P server's), Tty9p's socket check, and + the session's config directory and startup file (`opts`, `.local`: + ThemeFile, DumpThemes, Config, DumpConfig, an init file saved). + + **The model:** what only the authority holds stays the authority's, and a + step reads it through `tape.authority` (src/tape.zig): a `value` read, + its answer taped; a replay takes the answer, and a speculation that + reaches one waits for it, as at a host call (one RTT, never wrong). A + replica's own store and Recent are never read by a step. Rejected: the + store and Recent as session state (dumped, digested, copied), which + needs the store's file I/O on the tape call by call (it reads other + windows' records at every step that writes), for the same rows. + - `trackJump` no longer reads Recent: a closed file's jump stays, and the + jump to it drops that file's entries when Recent has forgotten it, as + when the file is gone. + - `noteClosed` asks the host first, on every side. + - The tty toggle, read on every key (which cannot wait a round trip), is + session state: `Pardes.tty_toggle`, from the options when the core is + made, dumped. +- **A rollback put back pointers to freed memory** (H8, the segfault in the + Panes picker). The copy saves a struct's bytes field by field, the + `.authority` ones included, but walks only what the digest walks. The + store and Recent live on the host's heap (`recentGpa`), whose frees the + epoch never puts off: a speculated Del grew the closed list, the + rollback put the old header back, and the next listing read a freed + name. `store` and `recent` are never saved now (rollback.zig `unsaved`); + a step never reads a replica's own. +- **A unit taped outside a step ended with effects queued** (H6, H10). A + leave is the session's `closeView`, whose sync sizes again a terminal the + closed view sized and queues its pty's resize; the session did it at its + next pump, in another unit. A replica that replayed the close kept the + effect queued, and its next idle copy asserted (H10: a client killed when + another left). A client joining at that boundary was dumped with it + queued, which a loaded core has none of, and diverged at the session's + next pump (H6: `the core called poll_frame where the tape has a host + (12)`). `closeView` and `openView` settle inside their own input now. +- **The lowered copy of a searched text held heap bytes** (H5's and H17's + `places.hits|hay`). A smart-case search lowers the text a piece at a time + into `Places.hay`, made with `memory.unfilled`; the digest and a join's + dump take the whole slice, so the pieces not searched yet were whatever + each process's heap held. Zeroed where it is made: 20 µs for 1 MB, + 0.4 ms for 10 MB, 10 ms for 100 MB in one frame (ReleaseFast, page + faults included; lowering the same text is four times that, over + frames), the order of the copy an edit of that text already makes. +- **A view drawn outside a step left the live slot aliasing** (H5's + hypothesis, in a path it did not name). `withParked` (a replica client + drawing or reading its own view while another is live) parked the live + view and copied it back without emptying its slot or its panes' shares + of selections, which the rollback copy walks. Not the cause of H5's + signatures, which were the jumps (Recent) and the places (the lowered + copy) above; the slot was saved and put back all along. + +- **A step that resized a terminal read rows its replicas could not be + sent** (H18). A client's first key into a terminal sizes it to its view + (`typedInto`, `sizes`); one that also read the shell's history in that + step (an edit copy over its text, `ovl`) read rows the replicas held as + holes. The session's `sync` reflowed its emulator inside the step, so + the rows `Screens.fill` sent before the unit were the reflowed grid's, + in the resize's new numbering (`Rows.epoch`); a replica replaying the + step on the grid its last delta made drops rows of another numbering + and read blanks. Every client diverged at that unit (`pane N.ovl`, and + `jumps … pane N.body` where the rest of the step read the reflowed + grid; `registers pane 2.body pane 2.ovl` in fixer 1's soak, a `d` in + a terminal's text whose step resized it). Seen with an instrumented + build on the hunt's seed 5 at 80 ms: the replica's `rows pane 2 epoch + 69/68`. + - The fix is fixer 1's (`terminal.resizeGrid` notes `resize_to` on an + authority with screen-state replicas, made after the unit in + `Peers.unit` and at `Peers.stop`), found on both lines at once; fix2's + was the same and is dropped. With it, `terminal.create` must make + `resize_to` null: it sets every field by hand, and the heap's bytes + there reflowed an emulator to 43690x43690 in a Debug run (0xAA). + - Found on the way: `shellRows` marked a terminal read only when it + built its rows, not when it handed out the kept ones, so a client that + came since (its history all holes) was sent none of what a step read. + - Not a stale round trip, as first supposed: a delta across a + renumbering is a reset already. The hunt's H7 fix (fixer 1's + `Rows.update`) is under this one. + - The gate on fix2's version of it (main mqkxnsqu, fixer 1's line): Debug `core-test` + and `unit-test` tty, `snap`, `replay`, `replicas` (none diverged or + unequal at 30, 80 and 207 ms), `test/daemon.py` 20, `--far`, + `--manage`, `--upgrade`, `--window-clients`, `--gui` 5, `test/fs.py`; + gui `core-test` and `unit-test` on main's two; ReleaseSafe + `test/daemon.py` 20, `--far`, `--window-clients`, `--gui` 5, + `--upgrade`, `test/fs.py`. Once, `test/daemon.py` 20 failed in its + far clients' end: a session asked to Exit twice closed its 9P link + before the second write's clunk was answered, an `EOFError` the test + takes for no `OSError`; twice again it passed. + +Tests, each failing before its fix: src/rollback.zig ("a rollback leaves the +store and Recent as a speculation left them", "a view drawn outside a step +leaves the live view's slot holding nothing"), src/replica.zig ("a replica +joined after a pane closed", with the variant hunt's Recent written by +another window; "a replica's Del of a file its session's Recent knows"; "a +client leaves, and one joins"), src/Places.zig ("a lowered copy's pieces not +searched yet are the same in every process"; src/replica.zig's "a step +that reads a terminal's rows as the core keeps them"). +The hunt's repros +(`panes_closed.py`, `del_file.py`, `recent_store.py` with the client's own +XDG_STATE_HOME, `leave_resize.py`) agree on the fixed builds; on the +installed 0.24 `panes_closed.py` still rejoins 16 times. `zig build +replicas` now looks at files, closes panes, opens the Panes and Recent +pickers and jumps. + +The gate, on fix2's five changes (`.scratch/daemon/fix2/gate.sh`): + +- **Debug:** `core-test` tty; `core-test` gui and `unit-test -Dplatform=gui` + fail only on main's two (`tagline pointer mapping...`, `a tag caret past + its row's end...`); `unit-test` tty, `snap` (113/113), `replay`, + `replicas`, `test/daemon.py` 20, `--far`, `--manage`, `--upgrade`, + `--window-clients`, `--gui` 5 and `test/fs.py` pass. +- **ReleaseSafe:** `test/daemon.py` 20, `--far`, `--window-clients`, + `--gui` 5, `--upgrade` from Debug and `test/fs.py` pass. +- **Rebased onto fix3's line** (myulmrpt; its `tape.results` now lists + what `tape.authority` returns): `core-test` and `unit-test` tty, + `snap`, `replicas`, `test/daemon.py` 20, `--far`, `--window-clients`, + `--gui` 5 and `test/fs.py` pass again in Debug; `test/daemon.py` 20, + `--window-clients`, `--gui` 5 and `test/fs.py` in ReleaseSafe. +- **The copy** (`zig build rollback-bench`, ReleaseFast, main's + rollback.zig against fix2's): a typical session's 980.3 KB to 980.1 KB, + a big one's 1487.9 to 1487.8 (the Store's and Recent's own bytes; their + heap was never saved). Its take 0.045 and 0.063 ms, 0.075 and 0.077; + its restore 0.036 and 0.034 ms, 0.053 and 0.054: noise, one run each. + +The soak (`test/soak.py local --minutes 5 --rtts 0,30,80,207 --clients +tty,win --trace`, ReleaseSafe, the mouse on, Upgrades, resizes, detaches +and scripts among the moves), twice: + +| cell | client | min | inputs | links | visible | mispred | p50 / p99 ms | cut short by | +|---|---|---|---|---|---|---|---|---| +| 0 ms | tty | 1.4 | 330 | 2 | 0 | 0 | 1.5 / 1298 | H7 (session) | +| 0 ms | window | 1.4 | 225 | 3 | 6 | 2 | 4.5 / 12.4 | H7 | +| 30 ms | tty | 0.6 | 101 | 1 | 0 | 0 | 43.2 / 1283 | H7 | +| 30 ms | window | 0.6 | 56 | 1 | 0 | 0 | 4.8 / 25.4 | H7 | +| 80 ms | tty | 5.1 | 962 | 1 | 6 | 30 | 14.4 / 1570 | | +| 80 ms | window | 5.1 | 911 | 3 | 63 | 35 | 5.9 / 117 | | +| 80 ms | window, attached later | 0.8 | 35 | 0 | 0 | 0 | 2.0 / 3.6 | its replay panic (H9) | +| 207 ms | tty | 0.8 | 98 | 1 | 0 | 0 | 42.3 / 1879 | H7 | +| 207 ms | window | 0.8 | 84 | 1 | 0 | 0 | 6.3 / 18.7 | H7 | + +- Every divergence of that run was the pointer's: `pointer_tag_hit + look_hover_wait` (H3, fixer 1's: a tag hit made from the process's own + frame), which takes every client of the session at once, and a window + client's replay that met the `pointer` site where the tape had none (H9's + panic, the same class). None of this section's classes. +- H7 (unassigned: `integer overflow` in ghostty's PageList under + `Screens.flush`, after a unit with a `pty_resize`) ended the session in + three of four cells; the first run's cells, of the same seeds, did not + meet it. +- The first run (its table lost to the harness, which raised when a session + with unsaved text would not Exit; fix2's copy now ends it by pid and + counts a breach): 0 ms, three rejoins on H3 alone; 30 ms, one window + client's replay panic at the `pointer` site; 80 ms, three on H3 and three + of H18's (`jumps … pane N.body` at units with a `pty_resize`, every + client together). +- H17's own configuration (30 ms, seed 22, 4 minutes, where both clients + had died at `n` on different places): no divergence in `places`. Its + rejoins were H3's (five) and H18's (two units, `jumps … pane 0.body` + and `pane 0.body pane 0.ovl`, each with a `pty_resize`). + +Not done, and found on the way: + +- **Two edges of the same class, left:** a pipe or an Edit in flight when + a client joins (`Pipe.wait`, `.authority`, not dumped: the joiner drops + its answer), and a 9P batch held at a join (`fs.batch`). Both need a + join in the instant between a request and its answer. +- **A Debug session with `PARDES_REPLICA_TRACE` is no soak:** its per-field + digests rehash every blob at every unit (`walk.rehash`, Debug's check), + a whole undo history of a big file a unit, and a boundary took 62 s. + Soak ReleaseSafe, or Debug without `--trace`. +- **std's segfault handler panics while it symbolizes** (H8's S3: + `SelfInfo.Elf.findModule`, index out of bounds), so a crash record says + that and not the segfault. It is std's, not pardes's (`crash.zig` takes + panics only). + +### fix: the session's abort, the window's rejoins, a client's end + +The user's window client of the box's session (`pardes-gui [email protected]:freebsd`, 2026-10-30): the cursor not keeping +up, the window blinking, panes closing just after they opened, then a +freeze. The first three were the client joining again, 15 times in its +25 s; the freeze was the session's abort at about its 16th far join, and +the client's 20 s of redials after it. An adversarial hunt (a soak harness +and a variant sweep) found the rest below. Each fix has a test that failed +before it. + +- **The session aborted after 15 far joins (H2).** `c_heap: more distinct + allocators than it can name`, from zstd's context on a peer's writer + thread. The core's table held 64 copies of one allocator, `ptr` 0 and + c_allocator's vtable. `zstd.gpa` is `std.heap.c_allocator`, whose `ptr` + is undefined, and nothing outside a test writes it: a release build made + it a constant and folded `known.ptr == current.ptr` to false, so every + malloc named a new allocator (about four a join) and the 65th panicked. A + client's decompressor (one a join) would have aborted after about 30 + rejoins; the web shell's heap over `wasm_allocator` trapped the same way. + - Fix: c_heap reads `current` through a volatile pointer, as its bytes + are, so no heap compares a value the compiler may take for anything. + - Tests: c_heap's own, which unit-test now runs built ReleaseSafe too (it + panicked at the 65th malloc); `test/daemon.py --far` makes 200 far joins + of one session, each read whole, and checks its fds, threads and RSS. + The installed 0.24 aborted with c_heap's panic within that run. Now, + ReleaseSafe: 200 joins in 1.7 s, fds and threads level (8, 8), RSS +108 + KB; on the box (FreeBSD ReleaseSafe, through ssh): 200 in 185 s, fds 13 + and threads 11 level, RSS 48964 to 49168 KB. +- **A window's pointer over a tag diverged it at once (H3, the user's + `tape byte 119`), and a click there aborted it (H9).** A step resolved a + frontend's pointer hit again on `p.surface`, this process's last frame, + which is `.local`: the session never draws a client's view, so the same + motion gave the client a tag hit and the session none (`pointer_tag_hit`, + `look_hover_wait`). Body hits did the same through `reprojectBodyHit` and + a sweep's clamp (the variant hunt's F). + - Fix: a hit carries the place it was made on (`Layer.Place`: the layer's + viewport, columns, rows and compact rows) and the step resolves it from + that, never from a frame; a sweep keeps its body's place to clamp onto. + The wire carries the place and a tag hit's line: version 17. + - Tests: `--window-clients` sweeps the pointer over the tags and the body + and clicks there, at 0 and 207 ms (before: joined again, `tape byte + 119`); mouse.zig's two tests from the variant hunt. The hunter's + repros: 15 rejoins in 3 s of hovering a tag before, none after; clicks, + drags and B3 in a window aborted it before, none after. +- **A replay that met a site its tape lacked panicked (H9, H12).** Fix: a + site or value the replay does not find is a divergence and returns what a + speculation gets (`poisonOf`): any later leak costs a rejoin, never the + process. Test: tape.zig's. +- **A view closing left a terminal's resize queued (H10, H6).** `closeView` + syncs, which sizes a terminal the closed view sized and queues its + `resize_pty`, with no step after it to drain it: a replica's next copy + asserted, and a speculation over the queue lost the effect when it rolled + back, so the next pump's replay found no `pty_resize` where the tape had + one. With the user's config (a font of their own sizes their view) every + rejoin diverged at its first keys. + - Fix: `closeView` settles in its own unit (fix2's, above, which took + the place of a drain here), and so does `leaveView` below; a copy is + refused (`error.Queued`), not asserted, over a queue, and the input + waits for the authority. + - Tests: replica.zig's in-process one (two clients at two sizes, one + leaves; the other copies, types and agrees). The user's config with a + random mix of keys and pointer: 18 rejoins in a minute before, none in + four minutes after. +- **A reflow overflowed the session's screen state (H7).** A resize to + fewer columns puts more rows above the last active top than ever scrolled + past it, so `top - history` underflowed where a peer starts afresh; a + window client's font made such resizes. Fix: `Rows.update` keeps `top` + at or past the history's length (a resize renumbers anyway). Test: + screen_state.zig's, 200 long lines narrowed. +- **A rejoin dropped the client's layout.** Its new view copied the live + one, the session's own as it was before the client came. Fix: a session's + last client, while nobody else watches the session, leaves it its layout + (`leaveView`: columns, focus, scrolls, cursors and size), which a rejoin + (an Upgrade's, a divergence's) copies back. Test: views.zig's. +- **A client's end.** One refused at a redial exited from inside itself and + left its ssh and `/tmp/pardes-ssh-<pid>.sock`; one rejoining after a + divergence redialled a gone session for 20 s; a session that panicked + kept its clients' links open for the seconds its trace took. + - Fixes: the ssh is ended at `atexit` and dies with its client + (PDEATHSIG); a redial waits only after an Upgrade's `refused` and else + says "could not join again"; the panic and segfault handlers shut every + client's link first. + - Tests: `--far` attaches through an ssh stand-in, refused and detached, + and finds both gone; `test/daemon.py` kills a session under a replica + client, which ends within 2 s, in words; replica.zig's redial test. +- **A far join's leave held the session's step** while the writer hashed + the rest of its blobs (the hunter's review, unconfirmed). Fix: the writer + stops at the next blob once it is let go. ponytail: one blob's hash (about + 30 ms for 100 MB) is the ceiling. +- **Kept as diagnostics:** the trace's unit descriptions give a key's and a + pointer's words, and a confirmed speculation's state is checked against + its unit's digest, which names the input that diverged rather than a later + unit replayed. +- **The user's symptoms, rechecked** on the box (ReleaseSafe at both ends, a + window here with the user's own config through ssh, a monkey of keys, + pointer, wheel, clicks and shell commands, 2 minutes), on this line over + fix2's and fix3's: + - Rejoins: none in 760 actions, 20 checks all agreed. (Before fix2's + line, one in 799, at a `.kind` site its tape lacked, which had aborted + the client.) Panes no longer close: a rejoin keeps the layout anyway. + - The freeze: the session no longer aborts, and a client of one that does + ends at once, saying so. + - The blinking and the cursor: 4 visible rollbacks in the 2 minutes (4 + inputs of 2375 mispredicted; 45 and 32 of 2369 before fix2's line). + Each is a clock guess: an edit's file time a second off across a + second's turn, or the millisecond clock the steps of Ctrl-W, Ctrl-O and + Ctrl-U read. Not fixed: a guess of the session's clock is the design's + ceiling there; reading a coarser clock in those steps is the next step. +- **Two leaks the soak showed after that (its local80 and local207 cells):** + - A client typing into a terminal at another size than its last typist + sized it in that key's update (sync), and the session's emulator + reflowed at once while its replicas' wait for the delta. The rows the + step read that a replica held as holes then went to it in the new + grid's numbering, which it cannot take, so its replay read them blank + (`jumps`, `pane 1.body` at a `b`). Fix: an authority with screen-state + replicas makes the size after the step's unit (`resize_to`, in + `Peers.unit`); the step reads the grid it had on both sides, and the + next flush brings the new one; `create` sets `resize_to` null, as it + sets every field by hand (fixer 2 found a Debug session's terminals + asked to be 43690 by 43690, the heap's 0xAA). Tests: replica.zig's, + which reads history in the resizing step in the session's order of + flush, fill and unit; terminal.zig's, a terminal made from 0xAA memory. + - A client that joined had its panes' places loaded without their regex + (`re` is derived: in no dump and no copy), took a search for a list's + and found nothing an edit made, so it diverged at the next edit and at + every rejoin after (`pane 3.places.hits`; `pane 17.cwd` too, which + inherits pane 2's and hashes it whole). Fix: the regex is made again + from the pattern when a set holds none. Test: Places.zig's, a dump + loaded and the same edit made in both. + - Not confirmed: "panes [5, 8] left its view while still open" in local0 + came at 104 s, before any client of that cell had left (the rejoins' + joins were at units 2028 and 2029, near its end), so neither + `leaveView` nor a closed view's settle moved them. The soak compares + pane ids (/index, /layout), and a slot is taken again by the next pane + made: two panes the terminal's view showed, closed, and the window's + Looks just before (Enter on a row) opening panes into those slots, in + the window's view only as a view's new pane goes, reads the same. A + serial a pane in /layout would tell the two apart. +- **Gates, on this line over main npzpposw (fix2's and fix3's), again; + then over wxpkrpwo, the builds, `core-test` tty, `replicas`, + `test/daemon.py` 20 and `--window-clients` in Debug and ReleaseSafe and + `test/fs.py` ReleaseSafe, all passing:** Debug `core-test` tty + and gui, `unit-test` tty and gui (only the two GUI tests main already + failed: the tagline pointer mapping and the tag caret), `snap` 113/113, + `replay`, `replicas`, `test/daemon.py` 20 cycles, `--far`, + `--window-clients`, `--manage`, `--gui`, `--upgrade` and `test/fs.py`; + then `test/daemon.py` 20, `--far`, `--window-clients`, `--manage`, `--gui` + 5, `--upgrade` from Debug and `test/fs.py` against ReleaseSafe builds. All + pass. + +## Phase 6: instant interaction notes + +Built 2026-10-30 (`src/replica.zig`, `src/tape.zig`, `src/terminal.zig`, +`src/echo.zig`, `src/detached/wire.zig`'s `stamp`, one line in +`src/pardes.zig`; PARDES_TEST_LATENCY's `I` and `U` lines in +`src/gui/gui.zig`). The user: "make sure to use the mosh +tricks so that user interaction to type/scroll/move the cursor feels +instant specially when they don't require a request going to the daemon", +and "measure it too since I don't want to catch those myself". The +measure is test/soak.py's `instant` mode (the hunter's, per action class +against a local session's own frontend) and a bench of my own that says +what stopped each input (`.scratch/daemon/instant/harness/classes.py`). +Decisions at the forks, with the ponytail lens: + +- **What stopped an input is counted by name.** A speculation that + meets a host call, a site or a poison records which (`Tape.poisoned_by`), + and the client's PARDES_REPLICA_STATS line ends `stops name:n,...` + (PARDES_REPLICA_TRACE says it with the input). Every number below that + says why an input waited comes from it. +- **pardes's own steps over a terminal are speculated (H14).** Until now + every input of a client whose keyboard was in a terminal waited a round + trip (`a terminal has the keyboard`): normal-mode keys over its history, + the wheel, Ctrl-u. The blanket poison goes. What reaches the program + (a key in tty mode, a paste, a mouse report) is a pty write, a host + call, and stops as before; the predicted echo covers it. The rest reads + the emulator and may move its view: + - **Its view moves here.** A screen-state replica's speculation may + scroll a terminal (`terminal.viewOnly`: `scrollGrid`, `followOutput`); + nothing else of an emulator. The rollback copy holds no emulator, so + the replica keeps each terminal's view as the copy was taken + (`Replica.views`), a delta sets it the authority's, and a rollback + puts it back, between the units it replays in the order the deltas + came among them (a delta's view comes after the units it shows: a + neutral unit of another's that scrolled, replayed over it, scrolled + twice; the harness's per-delta check found it). + - **Applied again over each delta.** What it did depends on the + emulator, here the last delta's and there what the pty said since. + The stream sends a client its terminals before the authority steps + any input and the rows a step read before its unit, so a speculation + over a terminal (`Pending.terminal`: it read a terminal's rows, + moved a view, or a terminal had the keyboard) is put back and applied + again over each delta or rows that come while it waits + (`overTerminals`): when its unit comes it read the emulator the + authority's step read. + - **No digest in a speculation.** A first version kept a terminal + speculation's digest to check its unit's against; but a blob born + under the copy and digested keeps its word past the epoch's free of + it (src/walk.zig), and a later blob at that address read as it (the + harness's Debug blob check, seed 11 at 207 ms). Gone: it is confirmed + by its records, as any speculation is. + - **Never kept to replay later.** A unit confirmed on the fast path + with more applied on top is kept in `since`, to replay at the next + rollback, over the deltas that came after it. One over a terminal is + taken at once instead, over the emulator its step read, and those on + top applied again. + - Closing a terminal stops (`retirePane`: its pty's end is the + authority's); the blanket poison had covered it, and the in-process + harness's host has no `close_pty` to stop at. + - Found on the way, others' to fix and fixed in main since: with more + of a terminal's steps speculated, the replicas harness met H18 more + often (a client typing at another size than the last typist: the + session's emulator reflowed inside the step, the rows sent for the + replicas' holes came renumbered, their replay read blanks; seeds 10, + 11 and 17 at 207 ms, `pane 0.ovl`, `pane 0.sel_snap`). Main met it too + (seed 53 of 60 at 207 ms). Fixer 1's `resize_to` and fixer 2's `read` + through the rows cache (wwkwwlzt, tysmmuzx) fix it: on them, 20 seeds + at each RTT, none diverged. +- **A pane closes in a speculation.** `deinitEmulator` asked the authority + before it looked for an emulator, so the reap after any close (a file's + pane, a picker's, a preview's) stopped the input after it: 4 of the + finder's 6 stops. +- **The authority's yes or no is guessed (mosh's way).** Fixer 2's + `tape.authority` makes a step that reads what only the authority holds + wait for it: whether a script reads a pane's events, which every Look + and click's command asks, put every Look a round trip late. A bool + answer is now guessed in a speculation from the authority's last answer + to the same question (`Tape.answers`, filled as units replay), recorded + as the answer, so its unit confirms the guess or rolls it back. Lists + (the closed panes the Panes picker shows, Recent) change with what the + user does and are still asked: opening the Panes picker waits a round + trip (its typing does not). +- **An input's time is its client's.** A speculation guessed the + session's clocks (`Tape.guess`: the last unit's reading, the time since, + a round trip), and a step that read one recorded the guess; the + authority read its own, so a message's millisecond (Ctrl-W's, a jump's) + always and an edit's second across a turn sometimes recorded otherwise: + a rollback each, which a window counts as seen (fixer 1: 1 to 8 a + minute, window clients worst). Now the input carries the guess (wire + 18's `stamp` frame before its event; `Pending.guess`, the same at every + reapply), and the authority's step reads it as its clocks (`Tape.stamp`, + `stampOf`; within a second of its own, else its own), so the two record + the same time. A message's linger is then off by the guess's error, tens + of milliseconds. +- **The echo's lost predictions wait for their keys.** Found by measuring + at 207 ms: the shell's typing was predicted (24 of 26) and drawn almost + nowhere, key to output p50 65 ms. A drop with keys still in flight (a + delta that moved the screen's top, here the pty's resize at the first + key; or a misprediction) restarted the predictions at the screen's + cursor, where those keys' echo was about to land: every key after it was + judged wrong (5 of 6), and once wrong, nothing was drawn until an echo + was confirmed, which none could be. Now the keys typed after a loss wait + for the newest key in flight to show (`lose`, the `deferred` path keys + behind Tab take), then are predicted over it. +- **mosh's rules, checked** (its source is in ~/05-genizah/mosh now, + src/frontend/terminaloverlay.cc `new_user_byte`): a printable character + (inserted, the rest of the row shifted right), Backspace (the rest + shifted left), Left and Right; CR moves the predicted cursor to the next + line, tentative; everything else (Home, End, Delete, Up, Down, any other + CSI or control) makes it tentative, nothing predicted. pardes predicts + the same less the mid-line insert and delete (a line editor redraws what + is right of the cursor; fish's suggestions mispredict a shift) and CR. + Nothing mosh does is missing that would raise the measured share: at a + prompt every letter is predicted. + +Measured. Before is main at uxwrlzlw with the counting (`base2`), after +is this line on mqkxnsqu (`new2`; fixer 1's line between them changes +none of what is measured here, and the line has since moved onto +vplsqtyz); both ReleaseFast, unstamped, on the perf cores, the box's +session the same build cross-compiled. The hunter's `instant` mode +(test/soak.py, its xvmytsym copy pointed at my box directories): a +class's 25 inputs a frame or more apart in one client, against a local +session's own frontend measured in the same run. `at once` is the share +that neither stopped nor showed later than half a round trip (a window's +two frames). + +- **A terminal's history, by keys (Ctrl-b, k, Ctrl-u) and by the wheel** + (tty; at once, then key to output p99 ms; local p99 0.6 to 1.2 ms): + + | | 0 ms | 30 ms | 80 ms | 207 ms | the box | + |---|---|---|---|---|---| + | term_scroll_keys | 0% 15.2 → 100% 1.1 | 0% 54.3 → 100% 1.5 | 0% 101.2 → 100% 0.9 | 0% 232.2 → 100% 0.9 | 4% 428.6 → 100% 1.4 | + | term_scroll_wheel | 0% 1.5 → 100% 0.6 | 0% 48.6 → 100% 0.6 | 0% 99.1 → 100% 0.6 | 0% 226.3 → 100% 0.6 | 0% 207.7 → 100% 0.6 | + + The window client's, the same at once: 0% (4% on the box) → 100% at + every cell. (A window's p99 in this mode is the first frame after the + read, which a window draws whether or not the input waited: its at + once is the measure.) +- **Shell typing at a prompt** (tty, p99 ms; every letter predicted + before and after, 25 of 25): 0.8 → 1.4 at 0 ms, 1.5 → 1.1 at 30, 0.8 → + 0.9 at 80, 49.0 → 27.7 at 207, 210.6 → 18.0 on the box. My own bench's + shell class at 207 ms (lines of `echo` and three words, Backspace, + Enter): p50 65 ms (main, ReleaseFast) → 1.8 ms (this line, Debug), the + echo judged wrong 5 of 6 → 0. +- **Every file class stays at once** (type, move by keys and by click, + scroll by keys and the wheel, select, the picker's typing): 100% before + and after at every cell and on the box, tty p99 0.5 to 1.5 ms after + (a local session's 0.6 to 0.9; its wheel 48 ms, the glide). +- The gate: `instant: pass` at 0, 30, 80, 207 ms and through the box + (before: 4 to 6 breaches a cell, each a terminal class's, 28 in all). +- What stopped inputs, by name (my bench, 207 ms, before; `stops` in + PARDES_REPLICA_STATS): the shell's keys `pty_write` (their echo is + predicted), a terminal's history `scroll a terminal` 23 and `a terminal + has the keyboard` 3 of 27, the finder `close a terminal` 4 and + `preview` 2, a Look the authority's `readsEvents` 11 of 12 and + `resolve` (the first open of a file) 1, a Save `kind` (and then its + write), the Panes picker's opening the authority's closed panes; typing + in a file, moving, scrolling a file, selecting: none. After: the + shell's `pty_write`, the finder's `preview`, a Look's first `resolve`, + a Save's `kind`, the picker's opening. +- **The soak's blink** (test/soak.py `local --minutes 2.5`, its gate, + a tty and a window client a cell at 0, 30, 80 and 207 ms, far): + before (`base2`) visible rollbacks 0.0 to 1.0 a minute, the window at + 80 ms 4.2 (13 in 3.1 min, a breach), mispredicted inputs 1 to 13 a + client, and that line's crashes and rejoins (fixer 1's since). Fixer 1's + own build, with them fixed: windows 2.2 to 2.7 a minute, ttys 0 to 1.3, + clock guesses all. The final line (this on xymsrmso): 0 visible, 0 + mispredicted, 0 rejoins at every cell, tty and window; `soak: pass`. +- **The replicas harness** (ReleaseFast, 20 seeds x 120 inputs x 3 + clients): inputs that stopped 731/649/790 at 30/80/207 ms → 453/390/398; + confirmed on the fast path 6249/6246/5953 → 6695/6755/6753; mispredicted + 220/305/458 → 15/15/21; none diverged, none unequal (before, main at + uxwrlzlw: seed 9 at 207 ms unequal at quiescence, as main at vplsqtyz + still was). +- **The final line** (this on xymsrmso, fixer 1's line): the gate at + 207 ms passes again, every class at once in both clients (tty p99 1.7 + to 4.4 ms, a local session's 0.5 to 1.4 ms; shell typing 25 of 25 + predicted, p99 28 ms); and through the box, every class at once in + both clients (tty p99 0.5 to 2.4 ms, a local session's 1.6 to 3.6; + shell typing p99 1.7 ms, 210.6 before). + +The gates (on the final line): Debug `core-test` tty, `unit-test` tty, +`snap` 113/113, `replay` (20 seeds), `replicas` (4 seeds) and `--echo`, +`test/daemon.py` 20 cycles, `--far`, `--window-clients`, `--manage`, +`--gui`, `--upgrade`, `test/fs.py`: all pass; `core-test` gui and +`unit-test` gui fail only main's two (the tag caret, the tagline pointer +mapping). ReleaseSafe (`zig build -Doptimize=ReleaseSafe`): +`test/daemon.py` 20 cycles, `--far`, `--window-clients`, `--manage` and +`test/fs.py` pass. The new unit tests: a terminal's history +by keys at once with a delta while they wait (replica.zig); keys still on +their way when their predictions go (echo.zig); an edit's second across a +turn confirmed on the fast path (replica.zig, which fails without the +stamp). + +Not done, and why: +- Opening the Panes picker and Recent wait a round trip (their lists are + the authority's, and change with what the user does); the finder's + preview reads the file it previews (a host call); a Save waits for its + `kind` site and then its write; the first Look of a pane waits once (no + answer to guess from yet). Each is one round trip, never wrong. +- The first key typed into a shell scrolled back waits a round trip: + typing snaps the view to the bottom, but the key is a pty write and its + speculation, the snap included, is undone; the echo predicts only at the + live bottom. +- Under the in-flight bound (32 deltas) a step can read a terminal state + the client was not sent: replayed, its unit diverges (a rejoin), as + another client's always could; a speculation of one is confirmed by its + records, so it is caught at the next digest a replay checks (fixer 1's + check when nothing is left on top), not at once. +- A window counts every misprediction followed by a present as seen, + whatever it drew (a tty only one that wrote to its terminal): an upper + bound, now 0 in the soak. +- A terminal's history is not fetched ahead of its view: a scroll back + (at once now) into rows a flood skipped shows them blank for a round + trip. Asking for two screens around the view diverged a replica in the + harness (seed 10 at 30 ms) before H18's fix, which may have been H18 + met in the other order of events it made; dropped, not tried since. +- The window's present is its frame (a software present in the test, up + to a frame): a prediction is drawn in the same pass as its key, so it + cannot come sooner than the window's own frame. + +After the line, the hunt's H21 (a client's normal end aborted): +- **A closed terminal brought back put its emulator on the rollback + epoch.** Both clients still alive at a box soak's end died at their + hangup with `double free or corruption (!prev)` in `Epoch.deinit`; a + local soak at 207 ms did it in 2.5 min, `free(): invalid pointer` in a + terminal's screen replica's teardown at a rejoin. A terminal closed and + brought back (Panes, `Look @p<n>`) is loaded by `dump.loadPane`, which + named no `.terminal` allocator, so on a screen-state replica its + emulator was made with the core's, the epoch's, against `State.gpa`'s + rule. Its screen replica's blobs (`apply`, with `State.gpa`) were the + epoch's then, freed with the C heap's at the end behind its back, and + freed again by the epoch; a delta under a held copy made them born in + it, and a rollback freed them, and the emulator's pages, from under it. + - Fix: `terminal.emulatorGpa`, `create`'s rule, is `loadPane`'s + `.terminal` too. + - Test: replica.zig's closed terminal brought back on a screen-state + replica, its State not the epoch's (failed before). The local soak at + 207 ms, ReleaseSafe (before, on lppumkvm, both its clients aborted): + 2.5 min, both end 0; 4 min at seed 32, four clients, none aborts (its + one breach, a window's panes gone from its view, is not this one). + +**A rejoin's view is its own again** (2026-10-30; `src/views.zig`'s +`away`, `Pardes.awayView`/`rejoinView`, wire 19). Every rejoin gave the +client a new view, a copy of the session's: a divergence's, and above all +an Upgrade's, which every client makes after each install (the hunt's +80 ms cell: the terminal client lost `+Search` and `+New` from its view; +likely the user's "some panes just close randomly right after I open +them"). `leaveView` covered only a session's last client, while no one +else watched it. +- **A view kept for its client.** An Upgrade's clients + (`Peers.refuseAll`) and one whose bye says it joins again (a + divergence's: `bye.again`) leave their view open as it is, kept for them + (`views_away`, a session field: digested, and dumped, so the new build + has it). The client names the view it had in its next hello + (`Hello.view`; the daemon hands it on with the connection, + `replica <k>`), and its join takes that view back (`rejoinView`, settled + as `openView` is) rather than a new one. Both are taped inputs. Any other + end closes the view as before. +- **The live view is never one.** One kept going live sends the session + into another, or into a copy of its own. `views.count` counts the views + someone looks at, so the last of those stays, a tty's Detach is "one of + several" only beside another, and `closeKeeping` never writes into a kept + one. Drawing skips them, and an Upgrade's new build opens no window for + one and closes none for want of a terminal (`resumeTerminals`, + `resumeWindows`). +- **Bounded by room.** A kept view goes when a join finds every view taken + (`openView`, the lowest first): a client that never comes back (one of + the old build, told to upgrade too) leaves one. ponytail: no timer; until + then it costs a parked slot, and a terminal it sized last keeps that size + until another view types into it. ponytail: a divergence's redial that + reached the session before its old link's end takes a new view. +- **Tests.** views.zig's: two clients' views through an Upgrade's dump, + each back in its own, and a join with no room lets one go. replica.zig's: + a divergence's rejoin, its pane in its view again, and the other client + agrees. `test/daemon.py --upgrade`: two replica clients each open a pane + in their own view, and after the Upgrade's rejoin each one's `/layout` + holds its own pane and not the other's; and the plain run's client D, + put off now at its 100th unit, opens a pane in its own view first, and + after its divergence's rejoin its view holds it and its keys go into it. + main's binaries fail both (`('its own view again', 4, 5, ...)`, `('client + D in its own view again', 2, ...)`). +- **Not this one:** the H21 soak's "panes [...] left its view while still + open, with no rejoin since the last quiet point" (local 207 ms, seed 32, + a window client with one link all through) is no rejoin's. +- **Gates** (on zponxort): Debug `core-test` tty, `replicas`, `replay`, + `test/daemon.py` 20 cycles, `--upgrade` (against a `-Dstamp-commit` + build), `--window-clients` and `--gui`; ReleaseSafe `--upgrade` (from + the Debug build) and `--window-clients`: all pass. + +After the line, the box soak's divergences and its clean quit: +- **The session that quit at its 1093rd input was asked to.** Seed 32's + window typed `Ctrl-W h`, then `imike quebec `. Window mode stays open + until Enter, Esc, `:` or a label, so `i`, `m` and `i` went nowhere. `k` + at a column's top went to its tag, and there `e SPC q` ran Exit, with + nothing unsaved. The soak now leaves window mode with Enter. +- **A FreeBSD session's +Help had one row fewer than its Linux + clients'.** The box soak's rejoins after a divergence were + `pane 4.file.content`, and `.splice` over a +Help made again. Each was + at a `?` after SPC, which runs Help. Help's rows are the build's words, + and Tty9p was a Linux build's alone (`enabled` asked the OS), so a + Linux replica made a `SPC n 9 Tty9p` row its session never had. H19 + made the enum's digest the same in every build; the set of words was + still not. + - Fix: Tty9p is a word of every hosted build. Whether the host is + Linux is the authority's answer (`tape.authority`), so a replica of + a FreeBSD session refuses Tty9p as its session does. + - Test: test/cross_replay.py's session runs Help. Main's FreeBSD tape, + replayed by a Linux ReleaseSafe build, diverged at it (`pane + 5.file.content`); with the fix, 30 digests agree in Debug and in + ReleaseSafe. +- **A Debug replica of a release session diverged at every Grep's query + pane** (`pane 2.query`: cross_replay.py's ReleaseSafe tape replayed by + Debug). `Names.query_buf` was digested whole, and past `query_len` it + is undefined: 0xAA in Debug, something else in a release build. Fix: a + replica `len` on it. The same tape now agrees in Debug (30 digests). +- **The hunt's `pane 2.file.scroll` (seed 41) went with the Help fix.** + On main (zponxort) it came again at the same unit: tty207-1's `SPC ?` + (1/596, 1/597) made a +Help, a window client joined, and tty207-1's `%` + (1/713) diverged that window at `pane 2.file.scroll`. With the fix, the + same keys and units pass with no rejoin, on lppumkvm and on zznqlptu. + Not traced: which field the joined window got wrong, and why the window + that replayed the Help unit itself did not diverge there. +- **Box** (traced, ReleaseSafe at both ends): seed 32 (win,tty), 3 min. + On main: four rejoins in three clients. The traces put three of them at + the two Helps; the fourth, in the window that typed the first Help, + gave no reason. With the fix: no rejoin. Seed 41 (tty,win), 3 min, the same before and + after (one rejoin on main, none with the fix). +- **Gates** (on zznqlptu): Debug `core-test` tty, `replicas` (none + diverged or unequal), `test/daemon.py` 20 cycles and + `--window-clients`, cross_replay.py with tty and window recorders, and + the FreeBSD tape replayed by Debug tty, ReleaseSafe tty and Debug gui; + ReleaseSafe `test/daemon.py` 20 cycles: all pass. + +After the line, a window that joined late: +- **It diverged at the next key that moved a pane of another view** + (`pane 23.file.scroll`, the box soak's seed 7: win207-3 at its join + 466 s in; the same field in the two windows that joined at 16 and 17 + min of an earlier run, not kept). A pane's `view` (`Pane.SessionView`: Wrap, the context rows' + style, the body's and the tag's row heights, its rect's height) is + written by `sync`, and was derived: never dumped. A key's move reads it + before its step syncs (`halfPageMove`, `ensureCursorVisible`, through + `File.geometry`), and a core loaded from a dump had the default, Wrap + on and no row heights. The tty had typed `SPC t w` at 243 s, so the + session's lines no longer wrapped and the joiner's did; its first + stepped unit after the dump was the tty's Ctrl-D (1/2018, after + `open_view` and a pump), and it scrolled further. A window there all + along had made `view` in its own steps and agreed. A parked view's + share (`PaneView.view`) was read the same way at a `to_view`, before + the step syncs. Window row heights with TreeContext's context rows + diverged it too, with Wrap on. + - Fix: `view` is digested and dumped, in a pane and in a parked view's + share. A dump of an older build has none and takes the default, as + before. + - Test: replica.zig's "a client joined from a dump moves through another + view's pane as the session does, its lines not wrapped" (Wrap off, the + cursor near the view's bottom, a join, Ctrl-D). Main fails it at + `pane 0.file.scroll`. A throwaway exploration (120 seeds: two views, + window row heights, TreeContext, Wrap, keys from either view after a + join) diverged 20 times on main (`file.scroll`, `hscroll`), none with + the fix. + - ponytail: `notices` and `pan` stay derived. `unpark` makes `pan` + again; a pane's notices are read by a click on its bands. +- **Not this one:** the box run with the fix (seed 7, 21 min, seven + clients) showed no `scroll` divergence in the four windows that joined + late and no `view` difference anywhere, but win207-4 diverged once at its + own key in a query pane (`pane 21.file.location_rows`, unit 4/447: the + session's tape had the same host answers), and after its rejoin a + speculated Enter panicked in `Query.land` (`f.location_rows[c + 1]`, + len 0). Both left open. +- **Gates** (on vtoozmwn): Debug `core-test` tty, `replay`, `replicas` + (none diverged or unequal), `test/daemon.py` 20 cycles and + `--window-clients`; ReleaseSafe `test/daemon.py` 20 cycles and + `--window-clients`: all pass. The box soak, ReleaseSafe at both ends: + 0 rejoins in six of seven clients, the seventh's the query pane's above. + +After it, the query pane's two: +- **A speculated Enter panicked in `Query.land`** (`f.location_rows[c + + 1]`, `index 1, len 0`: win207-4 after its rejoin). An edit of a frozen + query pane's rows drops them with the text they were of + (`setContentSpan`'s dropped marks), while its items and its current row + stay, and a later search in the pane keeps them so in its `saved` for an + Esc. `land` indexed the rows by the current item all the same; every + other reader of them asks first. In the soak it was line 1 of a live pane + (another view's SPC b b and Esc, and the window's own keys, in another + order than the session took them), which the session never ran: the + input panicked before it was sent. A session's own keys reach it too: + SPC b b, Enter, then `jxd` on the pane's rows and Enter. + - Fix: a current item without its row is none: the keyboard to the + owner, as with no current row. + - Test: Query.zig's "Enter on a frozen query pane's rows after its text + was edited: no row to land on, the keyboard to its owner". Main panics + in it. +- **A window joined from a dump made a `/` query's rows plain where its + session had coloured them** (`pane 21.file.location_rows`, win207-4's + `/r`, unit 4/447: the session's tape had the same host answers). A hit + row's colours are the owner's highlights where a frame painted them, + else the locations cache's: both derived, made by what the process has + drawn and painted, and a joiner has neither. + - Fix: a row's colours are in no digest (Row's own `digest`); a dump + still keeps them. They are how a row looks, not where it points. + - Test: replica.zig's "a client joined from a dump makes a `/` query's + rows as the session does, though only the session has drawn its + owner". Main fails it at `pane 1.file.location_rows`. +- **Gates** (on nopmlypk): Debug `core-test` tty, `replay`, `replicas` + (none diverged or unequal), `test/daemon.py` 20 cycles and + `--window-clients`; ReleaseSafe `test/daemon.py` 20 cycles and + `--window-clients`; the box soak, seed 7, ReleaseSafe at both ends, 21 + min and six clients (late windows and a late terminal among them): no + rejoin, no differing check, no panic. All pass. diff --git a/docs/shared-libs.md b/docs/shared-libs.md index 79f3cdb9..ce0c7d3f 100644 --- a/docs/shared-libs.md +++ b/docs/shared-libs.md @@ -49,6 +49,7 @@ and gui columns are the two binaries. | Regex | mvzr | pardes | 1 → 1 | 17 KiB | Zig | | Parsing | tree-sitter | pardes | 1 → 1 | | `ts_set_allocator` → c_heap | | Window, input, GPU | SDL 3.4.18 | gui | 1 → 1 | 1009 KiB | `SDL_SetMemoryFunctions` | +| Compress and decompress a far replica's stream (2026-10-30) | zstd 1.5.7 (the upstream tarball; build.zig compiles its lib/: no assembly, no threads, the dfast and lazy block compressors alone) | src/replica.zig | 0 → 1 | 380 KiB tty (516 with every block compressor) | `ZSTD_customMem` → c_heap | Text section: tty 61,369,343 → 61,294,855 bytes (**−74,488**); gui 67,336,392 → 67,182,000 bytes (**−154,392**). diff --git a/docs/typ/building.typ b/docs/typ/building.typ index d551c251..d8c089f6 100644 --- a/docs/typ/building.typ +++ b/docs/typ/building.typ @@ -1,6 +1,6 @@ // Building pardes, for contributors: platforms, options, tests, the // release gates and sending patches, then how the pieces fit: the core -// and its shells, the threads, detached sessions, the 9P engine, the +// and its shells, the threads, the daemon and its sessions, the 9P engine, the // listeners and mounts. #import "style.typ": key, keys, btn, chord, word, tag, addr, file, cmd, doc, pairs @@ -19,7 +19,7 @@ builds MuPDF and is on by default (`-Dmupdf=false` drops it). 9P over QUIC [`zig build`], [the terminal shell and the SDL window together, *installed into `~/.local/bin`* (`pardes`, `pardes-gui`, and `pardes-v9fs`, the #word("Tty9p") helper)], [`zig build -Dplatform=tty`], [the terminal shell alone], [`zig build -Dplatform=gui`], [the SDL3 window alone], - [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<dump.zon>`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)], + [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<file>`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)], [`zig build -Dplatform=macos`], [an AppKit and CoreText app over a static `libpardes.a` (`docs/macos.md`)], [`zig build -Dplatform=esp32p4`], [a freestanding riscv32 editor object for the ESP32-P4, with a 384 KiB heap], ) @@ -31,7 +31,8 @@ leaves that copy out. Zig 0.17 keeps the prefix from the build script, so so always pass `-Dplatform` while developing. Test, benchmark and run steps build what they need without installing. `pardes --version` prints `pardes <version>` (from `build.zig.zon`), plus the commit for release builds and the `~/.local` -install; `pardes --help` lists every flag. +install, and on a second line the build's semantics hash, which a replica +must share with its session; `pardes --help` lists every flag. The ESP32-P4 firmware is linked in the sibling `../05-zig-p4` toolchain, which needs ESP-IDF register headers: after building the editor object @@ -58,7 +59,7 @@ there; its namespace is in `src/esp32p4_gpio.zig`. [`-Dprebuilt-shaders`], [bool: embed the committed SPIR-V; on for a bare `zig build`, off with `-Dplatform`; `zig build shaders` refreshes it with glslc], [`-Dtracy`], [path to a Tracy checkout; off], [`-Dmacos-identity`], [codesigning identity for `pardes.app`; `-` (ad hoc)], - [`-Ddump`], [a `dump.zon` to embed in the web shell], + [`-Ddump`], [a Dump to embed in the web shell], [`-Dtest-filter`], [run only tests whose name contains it; a filter that matches nothing fails], [`-Dtest-rebuild`], [bool: fresh Zig test compilation], [`-Dhelix-harness`], [reference executable for live differential tests; `HX_HARNESS`, else `hx-harness` on `PATH`], @@ -177,8 +178,9 @@ Code map: `src/panes.zig` and the pane kinds it names (`src/File.zig`, `src/Terminal.zig`, ...), `src/layout.zig`, `src/fs.zig` (host access, mounts and resolution) and `src/pardes.zig` (input), with one file per thing beside them (`edit.zig`, `normal.zig`, `look.zig`, `exec.zig`, -`mouse.zig`, ...). `src/ninep/` is the control tree, `src/detached/` the -wire, the detached core and the frontend client, `src/lsp/` the +`mouse.zig`, ...). `src/ninep/` is the control tree, `src/daemon.zig` the +per-user daemon and a terminal's stub, `src/detached/wire.zig` the input +wire a remote client will speak, `src/lsp/` the language-server client; `test/` holds harnesses, goldens and helix cases, `build/` the snapshot suite's build step. @@ -197,26 +199,55 @@ reading the live core, and a request id, the pane's serial and (for an edit) the file's revision reject a stale reply. A process that never calls `turn.start` (the tests, the ESP32-P4, the browser) has no second thread. -== Detached sessions +== The daemon and detached sessions -One poll loop in `src/detached/server.zig` owns core mutation, frontend -connections, pty I/O and file watches. The frontend socket is -`pardes-detached-<name>.sock` beside the session's 9P socket; a second -session cannot take a live name. Up to 32 frontends attach; frames go to -every one, while clipboard reads, browser opens and #word("Detach") go to -the frontend that asked (else the first attached). Frames are full grids -or changes against what each frontend last received, never queued: a -frontend with unsent bytes skips that frame, and one that stops draining -past 1 MiB of control backlog is closed, its peers untouched. Frontends -never spawn shells, write session files or watch files. #word("Attach") -connects first and swaps second: only after the handshake does a shell -give up its own core. #word("Restore") builds the replacement core before touching -the current one. +One `pardes --daemon` per user (`src/daemon.zig`) listens on +`pardes-daemon.sock` in the runtime directory, its `.lock` flocked for its +life; the first session that finds none starts it (a double fork and an +exec) and it exits a second after it holds nothing. It is nobody's parent: +a session registers on one connection, sends it a dup of every pty master +it forks (so a session killed outright never hangs its shells up, and +#word("Adopt") takes them into another), says whether a terminal or a +window shows it, and is handed a terminal (`pardes --attach`) or a window +request (`pardes-gui --attach`) through it. Messages are `SOCK_SEQPACKET` +datagrams: a line of words and the fds it carries. -`src/detached/wire.zig` is the versioned protocol (version 10): fixed-width -little-endian fields, a 5-byte header (tag, then a u32 length), payloads up -to 16 MiB and grids up to 512 by 128. A comptime hash of `pardes.Chrome` -forces a version bump when that struct changes. +`pardes` in a terminal forks its session off before anything starts: the +session, in a session of its own (`setsid`), draws on the terminal it +opened, and the process the shell waits on stays as its stub, passing it +`SIGWINCH`, `SIGTERM`, `SIGINT` and `SIGQUIT` and ending with its status. +#word("Detach"), or the terminal's end (EIO, a hangup), puts +the terminal back and lets it go; the stub ends (exit 0) and the session +runs on with nothing drawn, until `pardes --attach` hands it another +terminal. A window's #word("Detach") closes the window and keeps the +process; `pardes-gui --attach` opens one again. `PARDES_DAEMON=off` (the +snapshot harness) runs a session as before the daemon: in the terminal's +own process, ended by its hangup. + +A replica (a client running its own copy of the core, phase 5 of +`docs/next-daemon.md`) connects to `pardes-replica.sock`, a stream socket +beside it that `ssh -L` can forward. `src/detached/wire.zig` is its +protocol (version 14): fixed-width little-endian fields, a 5-byte header +(tag, then a u32 length), the hello, a ping, the input events. The hello +names a session and carries the client's semantics hash (the protocol, +release, commit, platform, build options and every dependency's hash); the +daemon refuses another build's, or a session nobody has, in words, and +hands an accepted connection to its session (`SCM_RIGHTS`), which welcomes +it and from then on is the one it talks to. Today the session answers +pings and nothing more. + +#word("Upgrade") (`src/upgrade.zig`) execs the path the session was started +from, at a step boundary, after asking it `--version`: the session's dump +goes to the DumpDir (`pardes-upgrade-<pid>.dump`), and its pty masters, +terminals, stubs and daemon link stay open across the exec, named in a +record on the new command line (`--upgraded=`) ahead of the arguments it +was started with. The pid stays, so the shells stay its children; the new +process loads the dump with every view and takes the fds back. One that +cannot load it execs the old binary again (kept open from +`/proc/self/exe`), or with none starts afresh with the shells adopted and +the dump left for a Restore. The daemon re-execs the same way, its +registry on its command line. `python3 -B test/daemon.py --upgrade <old> +<new>` (and `--upgrade --gui`) is the end-to-end test. = 9P diff --git a/docs/typ/builtins-gui.json b/docs/typ/builtins-gui.json index db1031a0..e731b75e 100644 --- a/docs/typ/builtins-gui.json +++ b/docs/typ/builtins-gui.json @@ -1,6 +1,6 @@ [ + {"name": "Adopt", "leader": null, "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Take in the shells of a session that died holding them, each as a terminal pane here: the pardes daemon kept their terminals open."}, {"name": "Ascii", "leader": "SPC t a", "also": [], "arg": true, "scope": "pane", "setting": false, "choices": null, "doc": "add the printable ASCII bitmaps to the matcher's glyph set"}, - {"name": "Attach", "leader": "SPC s a", "also": [], "arg": true, "scope": "root", "setting": false, "choices": null, "doc": "Attach this terminal to a detached pardes session by name."}, {"name": "Back", "leader": "SPC j o", "also": ["C-o"], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Ctrl-o: one step back into the history."}, {"name": "Bloom", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "on, off, 0-3", "doc": "The brightest ink glows a little: a level, 0 (off) through 3."}, {"name": "BootShell", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "keep, replace", "doc": "A document dropped where a lone untouched shell stands: keep the shell, or replace it."}, @@ -29,7 +29,7 @@ {"name": "DelAbove", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close the pane, giving its rows to the pane above it."}, {"name": "DelBelow", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close the pane, giving its rows to the pane below it."}, {"name": "Delcol", "leader": "SPC c d", "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close this column and its panes, refusing once over unsaved text."}, - {"name": "Detach", "leader": "SPC s D", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Detach this pardes from its terminal, leaving the session running."}, + {"name": "Detach", "leader": "SPC s D", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Detach this pardes from its terminal or window, leaving the session running: `pardes --attach` (or `pardes-gui --attach`) takes it back."}, {"name": "Diagnostics", "leader": "SPC l d", "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "List the file's diagnostics from the language server."}, {"name": "Diff", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Stack a diff's hunks as a diff viewer shows them, each under its place and the declarations around it, in a +Diff; on a +Diff, show the diff itself as text."}, {"name": "DiffLook", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "hunks, text", "doc": "A .diff or .patch looked at: hunks, a +Diff stacking its hunks, or text, the file itself; Diff goes from one to the other."}, @@ -152,6 +152,7 @@ {"name": "Undo", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "acme's Undo and Redo: step the pane's body back or forward through its edits, from a tag, a pane ctl or exec."}, {"name": "Unmount", "leader": null, "also": [], "arg": true, "scope": "root", "setting": false, "choices": null, "doc": "Unmount the 9P tree mounted under /n/<name>."}, {"name": "Up", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Move the keyboard to the pane above, and from a column's top pane to its tag."}, + {"name": "Upgrade", "leader": "SPC s U", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Move this session onto the pardes installed since it started, in place: its text, undo, views, terminals with their running shells, and its terminals and windows come along."}, {"name": "Verbose", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "on, off", "doc": "A builtin says its own name on the message row as it runs, on or off."}, {"name": "Vignette", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "on, off, 0-3", "doc": "The window's corners fall into shade: a level, 0 (off) through 3."}, {"name": "Window", "leader": "SPC w", "also": ["C-w"], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Window mode: h/j/k/l move an outline between the panes; Enter or Esc focuses the outlined pane, `:` its tag, the letter shown on a pane that one; d is Del on it (then k or j between two), c Collapse."}, diff --git a/docs/typ/builtins-tty.json b/docs/typ/builtins-tty.json index 6bdca598..fe81b3fa 100644 --- a/docs/typ/builtins-tty.json +++ b/docs/typ/builtins-tty.json @@ -1,6 +1,6 @@ [ + {"name": "Adopt", "leader": null, "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Take in the shells of a session that died holding them, each as a terminal pane here: the pardes daemon kept their terminals open."}, {"name": "Ascii", "leader": "SPC t a", "also": [], "arg": true, "scope": "pane", "setting": false, "choices": null, "doc": "add the printable ASCII bitmaps to the matcher's glyph set"}, - {"name": "Attach", "leader": "SPC s a", "also": [], "arg": true, "scope": "root", "setting": false, "choices": null, "doc": "Attach this terminal to a detached pardes session by name."}, {"name": "Back", "leader": "SPC j o", "also": ["C-o"], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Ctrl-o: one step back into the history."}, {"name": "BootShell", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "keep, replace", "doc": "A document dropped where a lone untouched shell stands: keep the shell, or replace it."}, {"name": "Callees", "leader": "SPC l C", "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "List the functions the function under the cursor calls."}, @@ -26,7 +26,7 @@ {"name": "DelAbove", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close the pane, giving its rows to the pane above it."}, {"name": "DelBelow", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close the pane, giving its rows to the pane below it."}, {"name": "Delcol", "leader": "SPC c d", "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Close this column and its panes, refusing once over unsaved text."}, - {"name": "Detach", "leader": "SPC s D", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Detach this pardes from its terminal, leaving the session running."}, + {"name": "Detach", "leader": "SPC s D", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Detach this pardes from its terminal or window, leaving the session running: `pardes --attach` (or `pardes-gui --attach`) takes it back."}, {"name": "Diagnostics", "leader": "SPC l d", "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "List the file's diagnostics from the language server."}, {"name": "Diff", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Stack a diff's hunks as a diff viewer shows them, each under its place and the declarations around it, in a +Diff; on a +Diff, show the diff itself as text."}, {"name": "DiffLook", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "hunks, text", "doc": "A .diff or .patch looked at: hunks, a +Diff stacking its hunks, or text, the file itself; Diff goes from one to the other."}, @@ -135,6 +135,7 @@ {"name": "Undo", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "acme's Undo and Redo: step the pane's body back or forward through its edits, from a tag, a pane ctl or exec."}, {"name": "Unmount", "leader": null, "also": [], "arg": true, "scope": "root", "setting": false, "choices": null, "doc": "Unmount the 9P tree mounted under /n/<name>."}, {"name": "Up", "leader": null, "also": [], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Move the keyboard to the pane above, and from a column's top pane to its tag."}, + {"name": "Upgrade", "leader": "SPC s U", "also": [], "arg": false, "scope": "root", "setting": false, "choices": null, "doc": "Move this session onto the pardes installed since it started, in place: its text, undo, views, terminals with their running shells, and its terminals and windows come along."}, {"name": "Verbose", "leader": null, "also": [], "arg": true, "scope": "root", "setting": true, "choices": "on, off", "doc": "A builtin says its own name on the message row as it runs, on or off."}, {"name": "Window", "leader": "SPC w", "also": ["C-w"], "arg": false, "scope": "pane", "setting": false, "choices": null, "doc": "Window mode: h/j/k/l move an outline between the panes; Enter or Esc focuses the outlined pane, `:` its tag, the letter shown on a pane that one; d is Del on it (then k or j between two), c Collapse."}, {"name": "Wrap", "leader": "SPC t w", "also": [], "arg": true, "scope": "root", "setting": true, "choices": "on, off", "doc": "Long lines wrap onto the next row, on or off."}, diff --git a/docs/typ/cheatsheet.typ b/docs/typ/cheatsheet.typ index 43162f52..9dc9c473 100644 --- a/docs/typ/cheatsheet.typ +++ b/docs/typ/cheatsheet.typ @@ -72,7 +72,7 @@ Helix-style: motions *select*, then an edit acts on the selection. Mode box: bla == Keys: panes #pairs( [#key("Ctrl-w")], [window mode: #keys("h", "j", "k", "l") move the outline, #key("d") deletes it (then #key("k")/#key("j") between two), #key("c") collapses it, #key("Enter")/#key("Esc") or a pane's letter focuses (not in insert or raw `$`)], - [#key("Esc")], [back to the previous pane (insert: to normal)], + [#key("Esc")], [back to the previous pane (insert: to normal; a PDF: its selection and search highlights go first)], [#key("Shift-Esc")], [back to the previous pane from any mode but raw, where it is the program's Esc; needs the kitty keyboard protocol or the GUI, elsewhere a plain Esc], [#keys("Ctrl-o", "Ctrl-i")], [jump history back, forward; this file first. #key("Ctrl-i") is #key("Tab") without the kitty protocol: #key("SPC j i")], [#key("Alt-n")], [new terminal below], @@ -105,7 +105,6 @@ Helix-style: motions *select*, then an edit acts on the selection. Mode box: bla key("SPC c p"), word("Placement"), key("SPC s d"), word("Dump"), key("SPC s r"), word("Restore"), - key("SPC s a"), word("Attach"), key("SPC s D"), word("Detach"), key("SPC h t"), word("Tutor"), key("SPC h m"), word("Messages"), @@ -124,7 +123,7 @@ Helix-style: motions *select*, then an edit acts on the selection. Mode box: bla - A query pane under the pane: line 1 the query, its first word the source: `/pat`, `Find name` (fuzzy), `Grep text` (literal), `Recent`, `Panes`, `Symbols`, `All`. #keys("Ctrl-n", "Ctrl-p") step, #key("Enter") lands and keeps the rows, #key("Esc") puts things back. #word("Help"), #word("Tutor"). == Sessions -`pardes --detach=work`; `--attach=work` or #word("Attach") `work`; #word("Detach") leaves it running. #doc("detached") +`pardes --detach=work`; `--attach=work` (`pardes-gui --attach=work`) takes it back; #word("Detach") leaves it running; #word("Adopt") takes in a killed session's shells. #doc("detached") == Config #word("Config") (#key("SPC f c")): `~/.config/pardes/init`, a builtin a line (#word("Theme") `atelier`). #word("DumpConfig"): every setting. #word("Dump") / #word("Restore"): the workspace. #doc("config") diff --git a/docs/typ/guide.typ b/docs/typ/guide.typ index 58e5d907..56fe3ebd 100644 --- a/docs/typ/guide.typ +++ b/docs/typ/guide.typ @@ -43,7 +43,7 @@ steps raw, normal, insert. [#mode-box(" ") normal], [#key("Esc"): back to the previous pane (#word("Last")), unlike helix. #key("Shift-Esc"): the same.], [#mode-box("^") insert], [#key("Esc"): back to normal. #key("Shift-Esc"): out of insert and back to the previous pane.], [#mode-box("$") raw], [#key("Esc"): back to the previous pane. #key("Shift-Esc"): an Esc to the program. Either way the terminal stays raw. #word("ProgramEsc") `on` swaps them back: #key("Esc") goes to the program, except at a shell prompt with nothing typed on it, and #key("Shift-Esc") always goes back.], - [a PDF], [#key("Esc"): clears the selection and the search highlights. #key("Shift-Esc"): back to the previous pane.], + [a PDF], [#key("Esc"): clears the selection and the search highlights, and with none shown goes back to the previous pane. #key("Shift-Esc"): back at once.], ) In acme, Esc selects the text typed since the last click; here it leaves @@ -343,12 +343,29 @@ cheatsheet (#doc("cheatsheet")) say so. ``` pardes --detach=work & a session with no screen of its own pardes --attach=work show it here +pardes-gui --attach=work a window session's window again +pardes --ps every session the daemon keeps, and its shells +pardes --kill=work end it and its shells (--kill-all: every one) ``` -The session owns the panes, shells and files; frontends come and go, and -all of them see the same screen. #word("Attach") `work` (#key("SPC s a")) -switches this window to it; #word("Detach") (#key("SPC s D")) leaves it -running. +#word("Detach") (#key("SPC s D")) lets the terminal or the window go and +leaves the session running, its panes, shells and files kept; so does a +terminal that closes. `pardes --attach` takes it back in any terminal, +`pardes-gui --attach` a window session's window: a terminal session takes +a terminal, a window session a window. Attached already, it takes another +view: each terminal or window has its own columns, focus, cursors and +scroll over the same panes, texts and registers, and a terminal shown in +two takes the size of the one that last typed into it. #word("Detach") in +one lets only that one go. A session is named by `--9p=` or +`--detach=`, else its pid. A session killed outright leaves its shells +running; #word("Adopt") in another takes them in as terminal panes, what +they showed gone. One `pardes --daemon` per user keeps all of it, and +leaves once nothing is kept. After a rebuild, #word("Upgrade") +(#key("SPC s U")) moves a running session onto the pardes now installed +where it was started from, in place: its text and undo, its views, its +terminals with their running shells, and its terminals and windows come +along, and the daemon moves too. A build that cannot read the session +hands it back to the one that ran it. = Config <config> diff --git a/docs/typ/reference.typ b/docs/typ/reference.typ index b83b4a83..4de57d64 100644 --- a/docs/typ/reference.typ +++ b/docs/typ/reference.typ @@ -338,7 +338,13 @@ nothing answers (`dial failed: no answer`, `timed out`, `hung up`); with no dial it is `wrong #args`, and any other `x!y` `bad dial address`, both EINVAL. There are eight named mounts; `os` and `self` are reserved. #word("Unmount") refuses a mount a pane, a working directory or a pending -#word("Save") still uses. Mounts are dumped. +#word("Save") still uses. Mounts are dumped. A mount keeps its connection, +and sends each walk, open and first read or write together: an operation +through it costs one round trip (a directory, or a file of no stated +length, one more to see its end). A session on another machine mounts +through a forwarded socket: `ssh -C -N -L /tmp/box.sock:<its socket> host`, +then #word("Mount") `box /tmp/box.sock` (`-C`: a link far away carries +fewer bytes in each round trip). A session may open its own tree through a mount (a #word("Look") of `$m/pane/2/body` from the editor serving `$m`): requests are answered on the connection's @@ -368,27 +374,25 @@ Writes take settings and session builtins (`scope = .session` in #word("Delcol") and a pane's `get` refuse the same way with their own word, except that #word("Delcol") opens no `+Unsaved` pane: it names the panes only in its `unsaved` records and its notice. -- #word("Dump") writes `pardes-<date>-<time>.zon` (UTC) in #word("DumpDir"), +- #word("Dump") writes `pardes-<date>-<time>.dump` (UTC) in #word("DumpDir"), logs `dump <path>`, and adds a #word("Restore") `<path>` word naming it to the workspace tag (the last dump's, replacing an earlier one's). #word("Restore") `[path]` replaces every pane (bare: the last dump this session wrote). The #word("Restore") write is answered, then *every connection is hung up*: dial again, and restart a 9ns mount. The new log has a `new` per - pane, `restore <path>`, then `restored <old> <new>` per pane and - `restoredcol <old> <new>` per column. A dump keeps an unsaved pane's - text and the hash of the file it was read against; a clean file pane's - text is read from disk at #word("Restore"), so a file changed since comes back as - it is now. An unsaved one whose file changed comes back with its own - text, marked changed on disk, and its first #word("Save") asks before - overwriting. A PDF's fit and tint are kept too. A #word("Restore") that fails says why in - the parser's words, a ZON error with its line (`line 3: expected ','`), - so the errno a mount gives depends on them (usually EIO): read the `err` - record for why. A #word("ThemeFile") the dump names that fails to load changes + pane, then `restore <path>`. Its panes and columns keep the serials they + had when dumped; one made after takes a serial past every one the old + session handed out. A dump keeps every pane's text and undo: a clean + file changed on disk since comes back as it is now, an unsaved one with + its own text, marked changed on disk, and its first #word("Save") asks + before overwriting. A #word("Restore") of a file that is no dump fails + before anything is replaced (`not a pardes dump: it is cut short`), with + EIO: read the `err` record for why. A #word("ThemeFile") the dump names that fails to load changes nothing. - #word("Kill") `[word...]` (above), #word("Mount") `name dial`, #word("Unmount") `name`, #word("Theme") `x`. -- `size C R` sizes a `--detach` session no frontend is attached to (160x50 - until then): from 20x6 to 4096x4096, else `invalid size`; refused while a - frontend owns the size, and when a column would lose its panes' minimum +- `size C R` sizes a session no terminal or window is attached to (160x50 + for a `--detach` one until then): from 20x6 to 4096x4096, else `invalid + size`; refused while a terminal or window owns the size, and when a column would lose its panes' minimum rows (`size: too small for the panes, each its tag and 2 rows`). A bad `size` is refused quoting the value it got (`"5 2"`). @@ -454,8 +458,7 @@ turning it off: #word("PanelSlide"), #word("PanelZoom"), #word("PanelDissolve"), #word("PanelAscii"), #word("PanelVertical"), #word("PanelEdges"), #word("PanelFall"), #word("PanelWave"), #word("PanelCurtain"), #word("PanelScramble"), #word("PanelType"); all -start off, and the web shell has none. Scene passes (the SDL window, and -one attached to a detached session), each at a level 0-3 (`on` is 2), all +start off, and the web shell has none. Scene passes (the SDL window), each at a level 0-3 (`on` is 2), all off: #word("Crt"), #word("Bloom"), #word("Vignette"), #word("Grain"). #word("Shader") `<file.glsl>` adds a Shadertoy file written for ghostty to the chain (#word("Shader") `off` removes it; it recompiles when saved); @@ -891,7 +894,6 @@ just before the #word("Restore") are not lost. [`changed <serial> [reloaded|deleted]`], [its file changed on disk (bare: under unsaved edits)], [`unsaved <serial> <name>`], [a pane an #word("Exit"), #word("Restore"), #word("Del") or #word("Delcol") refused over, before the `err`], [`dump <path>`, `restore <path>`], [a #word("Dump") written; a #word("Restore"), after its panes' `new`s], - [`restored <old> <new>`, `restoredcol <old> <new>`], [serial maps after a #word("Restore")], ) The serial is the pane the line ran at (the pane with the keyboard for the root's @@ -1010,12 +1012,28 @@ to its end before the write is answered, and one that finds nothing fails, another; a removed line's `-` goes to the line now standing where it was. git's `a/` `b/` (and `c/ i/ w/ o/`) prefixes are dropped; a `--no-prefix` diff's paths are kept. -- *Sessions.* Bare `--detach` names the session after its pid; bare - `--attach` needs exactly one session; a second `--detach=NAME` while NAME - runs (a plain session is named by its pid) says so in words and exits 1. Frontends share the smallest common size. With - none attached, `size C R` sets the screen, messages clear by the clock, - and a pty reports its size in pixels at the last frontend's cell size - (8×16 before any), as CSI 14, 16 and 18 t do. +- *Sessions.* `pardes --ps` prints what the daemon keeps: each session + (its pid, terminal or window, attached or detached, name) with the pids + and commands of the shells it holds, then the shells no session holds, + which #word("Adopt") takes. `--kill=NAME` ends that session and its + shells (a TERM, then a KILL 2 s later; unsaved changes are lost), + `--kill-all` every session and every kept shell; with no daemon running + each says so and exits 1. Bare `--detach` names the session after its pid; bare + `--attach` needs exactly one detached session of its kind (a terminal's + for `pardes`, a window's for `pardes-gui`), or with none detached the + only session of that kind; a second `--detach=NAME` + while NAME runs (a plain session is named by its pid) says so in words + and exits 1. A session attached already takes another view, up to four: + each terminal or window its own layout, focus, cursors, selections, + modes, scroll and jumps, over the panes, texts, tags, terminals, + messages and registers they share. A terminal pane shown in two takes the + size of the view that typed into it last; a shorter view shows the + rows down to its cursor, an arrow in the gutter when rows are above. #word("Detach"), or a terminal closing, lets that view go and + leaves the others. With none + attached, `size C R` sets the screen, messages clear by the clock, and a + pty reports its size in pixels at the last cell size (8×16 before any), + as CSI 14, 16 and 18 t do. Attaching from another machine is gone until + the replicated clients (docs/next-daemon.md, phase 5). - *Config.* The startup file is `$XDG_CONFIG_HOME/pardes/init` when that is absolute, else `~/.config/pardes/init` (macOS: `~/Library/Application Support/pardes/init`). A line that fails is a @@ -1023,11 +1041,15 @@ to its end before the write is answered, and one that finds nothing fails, setting only the other frontend has (#word("Font") in a terminal) is skipped; text that is no builtin is not run; saving the init file in a pane applies its settings again and says a bad line as startup does - (`err - init file line N: why`), its commands not run again. A dump keeps panes, columns, tags, - selections, the theme and changed settings, but not a picture or a PDF - that is on disk (#word("Restore") reads it from there); a terminal - comes back with its last MiB of output and a new shell in its old - directory; undo history and REPL bindings are not kept. A crash appends + (`err - init file line N: why`), its commands not run again. A dump keeps the whole + session: panes with their text, undo, selections and places, terminals + with their screens, history and kitty images, pictures, registers, + columns, tags, the theme and settings. A terminal comes back with a new + shell in its old directory under its history (not the REPL it ran), a + PDF read from its file. + A dump made by another build loads: a field added since takes its + default, and what cannot be read (a pane whose kind changed) is left out + and said in a notice. A crash appends two lines (build, time, platform, pid; the panic message) to `crashes` beside `init`. diff --git a/docs/web.md b/docs/web.md index 0a9783c9..b9d03b13 100644 --- a/docs/web.md +++ b/docs/web.md @@ -57,7 +57,7 @@ zig build web \ -Dplatform=web \ -Dtarget=wasm32-freestanding \ -Dtree-sitter=zig \ - -Ddump=test/web-snapshots/touch.dump.zon + -Ddump=test/web-snapshots/touch.dump ``` The output is in `zig-out/web`. Serve that directory over HTTP; browsers do not @@ -133,7 +133,7 @@ zig build web-harness \ -Dplatform=web \ -Dtarget=wasm32-freestanding \ -Dtree-sitter=zig \ - -Ddump=test/web-snapshots/touch.dump.zon + -Ddump=test/web-snapshots/touch.dump # Real headless Chrome, DOM rendering, and browser touch input. zig build web-e2e |
