summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-10-29 22:56:07 -0300
committerGabriel Schneider <[email protected]>2026-10-29 23:37:35 -0300
commitbbb37c129126e9675d3ab6f7730083dba2d2becd (patch)
treea1d0eb564a902c2fca8b9148ca771b812cc5eb3d
parent370913ae34c69b7958911c6146612c77275f6ebf (diff)
downloadpardes-bbb37c129126e9675d3ab6f7730083dba2d2becd.tar.gz
pardes-bbb37c129126e9675d3ab6f7730083dba2d2becd.zip
docs/next-daemon.md: phase 6's upgrade by re-exec: the decisions at its forks, what was found, the gates here and on the FreeBSD box, the startup A/B, and what is not done
Co-Authored-By: Claude Opus 5.5 <[email protected]>
-rw-r--r--docs/next-daemon.md260
1 files changed, 260 insertions, 0 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
index c7cb719d..00f66cf8 100644
--- a/docs/next-daemon.md
+++ b/docs/next-daemon.md
@@ -1795,3 +1795,263 @@ Not done, and why:
path is the tty's.
- The other agent wires it in: the tty and GUI hosts, `Screens` per
attached peer, and a replica frontend drawing after `screen`.
+
+## Phase 6: upgrade notes
+
+Built 2026-10-29 (`src/upgrade.zig`, the tty and GUI hosts, `daemon.zig`,
+`main.zig`, `dump.resumeFromDump`; `test/daemon.py --upgrade`). A running
+session, and the daemon, move onto the pardes installed since they
+started, in place: text, undo, views and layout, terminals with their
+running shells, and their terminals, windows and daemon link. Decisions
+taken at the forks, with the ponytail lens:
+
+- **The trigger is the word, `Upgrade`** (`SPC s U`, any tag, `/ctl` and
+ `/pane/N/exec` over 9P). It moves the daemon too: the session sends it
+ `upgrade` just before its own exec, so one word moves both, and no word
+ of the daemon's own was needed. Nothing upgrades by itself. A focus-in
+ (one comes back to the terminal or window from rebuilding elsewhere)
+ stats the start path, and a file there that is not the one the process
+ started from is said once, in a line: `a newer pardes is installed:
+ Upgrade (SPC s U) moves this session onto it`. Rejected: a watch on the
+ binary's directory (an inotify entry and an event source for a line),
+ and upgrading on its own (a build half-written, or one the user means
+ to test first, would take every session).
+- **Which binary: the path this process was started from**, read once
+ just after its first frame (`std.process.executablePath`, a readlink,
+ and a stat for the notice: off a startup's way, and no build is
+ installed within a frame of a start), exec'd again at the upgrade.
+ `install(1)` and `zig build` put a new file at the path, renamed over
+ the running one, which `/proc/self/exe` then names `(deleted)`. The
+ record carries the path
+ exec'd, so a process that fell back to the old binary (whose own exe is
+ that deleted file) upgrades from the same path next time. Not argv[0]
+ and PATH: a session's PATH is the one it started with, and the readlink
+ is what the daemon's spawn already used.
+- **Asked first, `<path> --version`**: run as it is, it must exit 0 with a
+ `pardes ` line within 10 s. A file not there, not a pardes, or one that
+ dies at start is found before anything is let go, and said ("Upgrade:
+ <path> exited 3 at --version; this session stays as it is"). 9P is
+ served while it runs (the turn rested), so a client's clunk after its
+ `Upgrade` write, and anything else, is answered; after it, nothing is.
+- **At a step boundary**: after a pump, the pty readers are canceled (a
+ canceled tty reader posts an end of file, passed over while upgrading),
+ the terminals' input readers stopped, the loop's queue applied, and the
+ core settled (`Pardes.settle`: queued inputs, then effects). In the GUI
+ the queue is drained, the readers canceled, and drained again (a reader
+ waiting for room gets it). Bytes read are never lost; keys typed during
+ the exec wait in the terminal's kernel buffer for the new process.
+- **The dump goes to the DumpDir, `pardes-upgrade-<pid>.dump`**, not a
+ memfd and not the runtime directory: a refused load leaves it where a
+ relative `Restore` finds it, on a disk and not a tmpfs (a big file's
+ session dumps 100 MB and up). Never $PARDES_DUMP, which is a Dump's. A
+ successful load deletes it.
+- **The record is one argument**, `--upgraded=<lines>`, ahead of the
+ arguments the session was started with, which configure it again
+ (`--9p=`, `--tty-toggle`, `--geometry`, `--nested`); its file, stdin,
+ directory, `--detach` check, forwarding and daemon fork are skipped. A
+ line a word: `link`, `stub`, `old` (fds), `window`, `pty <pane> <pid>
+ <fd>`, `tty <view> <fd> <stub> <primary> <termios hex>`, `exe`,
+ `dump`, and `refused` in a fallback; a word another build adds is
+ passed over. Every fd it names is close-on-exec but across the one exec
+ (`Record.closeOnExec`), and the new process sets them back at once, so
+ no shell it forks inherits a master or a terminal.
+- **The pid stays**: the shells stay the session's children, the stub
+ keeps waiting on it, the 9P name of a session named by its pid is the
+ same, and PDEATHSIG (set again) still means the stub's death.
+- **Terminals.** A kept terminal pane is not revived (`resumeFromDump`
+ with `kept`): no reset, no banner, no new shell; ghostty's state is the
+ dump's, the program's own, and its pty keeps its size (no 1x1 nudge:
+ the screen is already right). The host puts the master back in the
+ pane's slot and reads on (a command pane's exit watched again). A
+ terminal pane with no live master (its shell gone) is revived as a
+ Restore revives it.
+- **The tty's terminals stay raw, their cooked termios in the record**:
+ put back to cooked between the two processes, a key would echo and a
+ ^C (forwarded by the stub) could kill the new process before its
+ handlers. What is a stack is unwound: the kitty keyboard flags are
+ popped (the new process pushes them once its query is answered), its
+ kitty images deleted and its shm names unlinked (the pid is the same,
+ and so would the names be). The alternate screen, mouse and paste modes
+ stay on; the new process sets them again and draws everything, one
+ full frame over the old one in place. The old image's prompt rc files
+ (the running shells have read them) are swept by the new one, which
+ makes its own: at its sweep, any of its pid are the old image's. They
+ stay for an exec that failed.
+- **Views come back whole.** `resumeFromDump` keeps every view the dump
+ has (no `views.only`) and does not run the startup config again (the
+ session's settings are the dump's). Each terminal's view is in its
+ `tty` line: the primary one is `run`'s, the others are entered as
+ `--attach` enters one, with their stubs; a view that lost its terminal
+ closes. The GUI opens a window for each open view, the live one first
+ (`openAnotherWindow(k)`): its windows are the one flash, as decided.
+- **9P is cut and comes back.** What the connections had to send is let
+ go first (`Listener.flushOut`, the 200 ms the quit's path already
+ had), then the exec closes the listener and its lock, and the new
+ process claims the name again. ponytail: a dial during the exec and the
+ load is refused (one round of a client's retry); passing the listener
+ through as well would keep it queued.
+- **Replicas are told in words.** Each connection a session serves gets
+ a `refused` frame ("the session is upgrading to another build of
+ pardes: connect again with that build") and a shutdown before the exec.
+ Their hello's semantics hash would not match the new build anyway, and
+ the session registers its new hash on the same link (a second `hello`,
+ which the daemon takes as an update). The reconnect, and a local
+ client of the same path re-exec'ing itself, are phase 5B's: there is no
+ replica client yet.
+- **The daemon re-execs the same way**: its own start path, asked
+ `--version` first, the listener and lock (fds 3 and 4), the replicas'
+ listener, every session's link and every held master kept, its
+ registry on its command line (`--daemon --upgraded=` with a `c` line a
+ connection, its slot, kind, pid, state, semantics and name, and an `h`
+ line a master and its owner's slot); a replica whose hello is still on
+ its way is let go and dials again. Sessions never see it: their links
+ are the same sockets.
+- **Failure never loses a shell.**
+ 1. The binary does not answer `--version`: nothing changes, said.
+ 2. The exec fails: the fds close-on-exec again, the readers and the
+ terminals' keys start again, the kitty flags are pushed back, the
+ dump deleted, said.
+ 3. The new build cannot load the dump: it execs the old binary again,
+ from `/proc/self/exe` opened before the exec (`old`, Linux), with
+ `refused` set; that one loads its own dump and says "Upgrade: the new
+ build could not load this session (<why>); it goes on as <version>".
+ 4. No old binary to go back to (FreeBSD has no /proc by default), or it
+ refuses too: a fresh session adopts every kept master as a terminal
+ pane (the Adopt path) and says "Upgrade: no build could load the
+ session (<why>); its shells are here, and Restore <dump> brings the
+ rest back". The dump stays.
+ 5. A crash after the exec is the crash Phase 3 covers: the daemon's
+ dups keep the shells for an Adopt, and the dump is in the DumpDir.
+ `PARDES_TEST_UPGRADE_REFUSE=1` makes a start refuse the load as such a
+ build would (the first start only), `=all` the fallback's start too:
+ test/daemon.py installs a script that sets it and execs the new build.
+
+Found on the way:
+- A fresh session's first pane has no directory until its own shell's
+ spawn is performed, so the `Tty` word that adopts a shell refused ("Tty:
+ : no such directory") and the shell was hung up with "no room": the
+ kept shells are taken after the startup effects, and those they emit
+ performed then.
+- A test reconnecting after the word could reach the old process, still
+ asking `--version` (and serving 9P meanwhile): the test waits for the
+ exec (the session's exe, or on FreeBSD its new semantics in the
+ daemon's registry) before it reconnects.
+- An `Adopt` (or a `Detach`) served in a session's first pump, before its
+ first frame, is refused as if no daemon kept it: the reader that takes
+ the daemon's `ok` starts after that frame, and `kept()` waits 2 s for
+ it. The test's next session waits for its first frame. Pre-existing,
+ left as it is.
+
+Gates (Debug, `A` this line unstamped, `B` the same with `-Dstamp-commit`,
+a trivial visible change: `pardes 0.24 (<commit>)` and another semantics
+hash; both installed in turn at a private prefix by a rename, as
+`zig build` and install(1) do):
+- `python3 -B test/daemon.py --upgrade A B`, under a private runtime dir.
+ A tty session with an unsaved edit, a shell running `sleep 1000; echo
+ done-after-upgrade` and a second terminal (a second view), and a
+ replica of A's build connected:
+ - B installed, a focus-in says the newer build; `Upgrade` written to
+ `/ctl`: the session's 9P answers again 123 ms later (81 to 83 ms on
+ the box). The session's pid is the same, `/proc/<pid>/exe` is B (not
+ deleted), and so is the daemon's, the same pid.
+ - The edit is there, the pane dirty, and `Undo` takes it back.
+ - The shell's pid is the same, still the session's child, its `sleep`
+ running; killed, the shell goes on and `done-after-upgrade` shows.
+ - Both terminals are drawn again (each its own view); the stubs wait on.
+ - The replica of A's build is told `the session is upgrading to
+ another build...`; a hello of B's build is welcomed by the daemon,
+ one of A's refused (the daemon lists the session as B).
+ - The fds level: the session's as before but the replica's, the
+ daemon's the same; the session's prompt files as many as before, the
+ new image's.
+ - A refusing build (a script that sets `PARDES_TEST_UPGRADE_REFUSE=1`
+ and execs B) installed and asked: the session goes back to the binary
+ it ran, its exe that inode again, the shell and text as they were.
+ - One that refuses everywhere (`=all`): a fresh session, the shell a
+ pane of it (`echo $$` there is its pid), the dump left in the DumpDir.
+ - The session killed outright: the daemon, upgraded three times, still
+ holds the shell's master, and a new session adopts it.
+ - Nothing left: no process, no socket. Passed 3 runs of 3 (and every
+ run since).
+- `--upgrade --gui A-gui B-gui`: a window session in test mode (hidden
+ windows, captured) with a second window from `pardes-gui --attach`, the
+ same edit and job: after `Upgrade` the pid is the same and the exe B's,
+ the daemon B, the edit and undo there, the shell and its job the
+ session's, both windows' captures written again, B's hello welcomed;
+ the session ends with nothing left. Passed 5 runs of 5.
+- FreeBSD 15.1 (the box, `pardes-ua` and `pardes-ub` cross-built from
+ this line): the same tty test but the fallback to a kept copy (there is
+ none). The new build is told by its semantics in the daemon's registry,
+ and the daemon's re-exec by `--upgraded=` on its command line; the fds
+ level by procstat. 81 to 83 ms from the word to 9P. Passed 9 runs of 10
+ (the last two on this line's final builds); the one failure was the
+ test's first-frame race above, fixed, and it passed after.
+- The existing gates: `test/daemon.py` (20 cycles) and `--gui` (10)
+ pass; `test/fs.py` passes (49 s); `zig build snap` 113 of 113; `zig
+ build replay` passes (the 113 scripts, 20 seeds at one view and 20 at
+ three, their dumps loaded to their digests); unit-test tty 1479 of
+ 1499, 20 skipped; unit-test gui 1550 of 1561, 9 skipped, and the two
+ known failures (`tagline pointer mapping...`, `a tag caret past its
+ row's end...`). core-test's new tests: the record read back as written
+ (and refused when cut), and an upgraded dump keeping its views, a
+ running terminal's alternate screen and no new shell, and settings the
+ startup config would change.
+
+Measured. The upgrade itself, Debug, a 300 MB binary: from the word
+written to `/ctl` to the session's 9P answering again, 123 to 130 ms,
+one run beside builds 299 ms (the `--version` ask, the drain, a dump of
+a small session, the exec and the load); on the box, 81 to 83 ms with
+its 125 MB binary.
+
+No local change (ReleaseFast tty, main kuovmsxw against this line, both
+live at once, their samples alternating, through the perf lock; lat.py's
+startup is a spawn with no daemon running to its first frame's end):
+
+| | A/B | A/A |
+|---|---|---|
+| startup, 100 a build: p50 | 0.944 | 1.010 |
+| the same: mean | 0.919 | 1.061 |
+| the same: median of the fast mode (and its share) | 0.964 (54 to 52) | 0.996 (69 to 73) |
+| startup, 40 a build, two runs: p50 | 1.001, 1.266 | 0.998, 1.010 |
+| key in a file, to its frame, p50, two runs | 0.997, 1.002 | 0.958, 0.990 |
+| key echoed by bash, to its frame, p50, two runs | 0.997, 0.991 | 1.004, 0.999 |
+
+The startup times have two modes on this machine, about 4.5 and about 12
+ms, and the share in each moves between runs and between the builds of
+an A/A alike, so a p50 that falls between them jumps (the 1.266). Within
+a mode the builds are the same. A first A/B, made while the path was
+still read at the start (a readlink and a stat before the first frame,
+some microseconds), read p50 1.04 to 1.13 against an A/A of 0.98 to 1.00
+on runs that overlapped builds; the read was moved after the first frame
+anyway, so a start does nothing it did not.
+
+Not done, and why:
+- Remote replicas reconnect on their own, and a local client of the same
+ path re-execs itself first: phase 5B's (no replica client exists yet).
+ What they get today is the `refused` frame above, then the hangup.
+- The 9P listener is not kept across the exec: a dial in that window (a
+ hundred milliseconds or so) is refused. Clients that dial afresh (the
+ hosts' Client) retry; a mounted session sees its connection end.
+- On FreeBSD a build that cannot load the dump starts afresh (shells
+ adopted, dump kept): there is no `/proc/self/exe` to go back to by
+ default. Keeping an fd of the binary open from the start would do it,
+ at an fd a session.
+- A crash of the new build after it starts is a crash: the shells wait in
+ the daemon for an Adopt and the dump is in the DumpDir, but nothing
+ restarts the session on its own (the stub, which runs the old binary,
+ could).
+- An escape sequence a program had half written when the dump was made is
+ lost (the snapshot is at ground state, Phase 2 notes).
+- Language servers and `|` pipes running at the upgrade are let go (their
+ pipes close at the exec; they end), and the shells of panes closed just
+ before it, still waiting to be reaped, are reaped only when the session
+ ends. A daemon note (a terminal attaching, adopted shells) read by the
+ old process's thread but not yet taken by its loop is lost; the stub of
+ such a terminal sees its pipe end.
+- A detached session's upgrade (no terminal) was checked by hand
+ (scratch script), not in test/daemon.py.
+- GUI windows come back at the size and place the compositor gives them.
+- A daemon protocol bump across an upgrade is not handled: the sessions
+ of the older build keep talking theirs.
+- A session name with a newline would break the daemon's registry line.
+- An upgraded process records no tape (a recording ends at the upgrade).