summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-10-30 00:01:29 -0300
committerGabriel Schneider <[email protected]>2026-10-30 00:57:20 -0300
commit965a78eafacbffb7cd25104181c81fe5871102c1 (patch)
treecba60dec9594dd119900b017b137e02e56f60aa9
parent79efd33ba0964796d4dc4653b68435755b35f834 (diff)
downloadpardes-965a78eafacbffb7cd25104181c81fe5871102c1.tar.gz
pardes-965a78eafacbffb7cd25104181c81fe5871102c1.zip
docs/next-daemon.md: phase 6's 9P round trips: the decisions at its forks (the server's order kept, what a park lets go ahead and how the client meets it, the flight, the end of a read, read ahead, kept connections, no stat cache, /n/<host> as ssh -L and Mount), messages, round trips and times before and after at 30/80/207 ms and on the FreeBSD box (plain and ssh -C), what the box's link costs beyond the round trips, the local A/B, the gates, and what is not done
Co-Authored-By: Claude Opus 5.5 <[email protected]>
-rw-r--r--docs/next-daemon.md183
1 files changed, 183 insertions, 0 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
index 681a2ed4..0fe72b76 100644
--- a/docs/next-daemon.md
+++ b/docs/next-daemon.md
@@ -2129,3 +2129,186 @@ Not done, and why:
of the older build keep talking theirs.
- A session name with a newline would break the daemon's registry line.
- An upgraded process records no tape (a recording ends at the upgrade).
+
+## Phase 6: 9P round trips notes
+
+Built 2026-10-30 (`src/9p_io.zig` Client, `src/fs.zig`, the Mount word in
+`src/builtins.zig`, a paragraph of `docs/typ/reference.typ`). pardes's own
+9P client is what mounts, the Mount word, the forwarding CLI, `--wait`
+and Handover use; scripting clients (9p(1), 9ns, test/ninep.py) are not
+changed here. Decisions at the forks, with the ponytail lens:
+
+- **The server already takes a connection's requests in order; nothing
+ there changes.** cloud9's engine has one job slot: it starts the next
+ frame only once the request before it is answered. What parks leaves
+ that slot, and the frames after it go ahead: a read with nothing yet,
+ and, while the editor is out in a syscall, a write or a truncating
+ open (`tree.needsQuiet`; an OTRUNC open is a setattr first). So a walk
+ and the open of its fid can be sent together (a walk never parks), and
+ so can an open and its first read (an OREAD open never parks). Two
+ things cannot:
+ - a Twrite behind a truncating Topen that parks is refused `bad use of
+ fid` (not open yet);
+ - a Tclunk behind a parked Twrite interrupts it.
+
+ The client therefore sends a write or read that came back `bad use of
+ fid` again after its open's answer: that refusal is the engine's, made
+ before the backend, so nothing was done. It sends its Tclunk only after
+ the last answer, and does not wait for the Rclunk. The park test ("a
+ change waits while the editor is out mid-step") takes the resend path.
+- **One flight per operation.** The flight is `[Tversion Tattach] Twalk
+ [Twalk] Tstat Topen Tread` for a read, or `... Topen Twrite` for a
+ write, each request with its own tag (`Flight`, `Session.land`). Past 16
+ names, the second walk goes from the fid to itself, so no fid is in
+ between.
+- **Version and attach pipelined by a poke.** cloud9's client submits
+ nothing while a Tversion is out. `greet` sets its `versioning` and
+ `msize` around the submits, taking the asked msize until the Rversion
+ lands. Only a pardes is dialled here, and every pardes offers
+ `ninep.msize`. A write in that first flight is kept under
+ `min_msize - 23`. The fix belongs in cloud9's client (taking requests
+ while it versions). It is not made here, since this round pushes
+ nothing (`// ponytail:` at `greet`).
+- **The end of a read.** 9P has no end of file but an empty read.
+ read(5) says a reply "may be less than the requested amount", and a
+ directory's next offset must be the last plus what it returned, so
+ neither the end nor a directory's next read can be sent ahead. The
+ Tstat in the flight gives a file's length: a read that comes back short
+ at that length is the end. pardes reports real lengths for bodies,
+ `/os` files, ctl, tag and the like. A file of no stated length, or a
+ directory, costs one round trip more: index, layout, screen and log are
+ generated whole by each read and say 0 by design (tree.zig `topSize`).
+ An end flag would be a protocol extension, which is not worth one round
+ trip on generated files.
+- **Read ahead.** With the length known, the Treads for the rest go at
+ once, 14 at a time (the 16-tag window, less a clunk still out and a
+ spare). A 1.3 MB file takes 3 round trips where it took 22. A file of
+ no stated length is not read ahead (`// ponytail:`).
+- **Reads are timed by their progress.** Each Rread gives the rest the
+ 2 s budget again, as each Twrite already did. Found on the way: on main,
+ every read of one operation shared one 2 s budget, so a remote file of
+ 1.3 MB failed to open (`dial failed: timed out`) at 80 and 207 ms. On
+ the box without `-C`, the 77 KB `src/fs.zig` failed too.
+- **A mount keeps its connection** (`kept`), as Plan 9's mount keeps its
+ channel: an operation through it is the flight alone, with no dial,
+ version or attach.
+ - There are eight slots, keyed by dial. A session is taken out of the
+ table while it is in use, so two users of one dial each get their
+ own.
+ - Unmount forgets it (`Client.forget`).
+ - Before reuse, a poll takes what arrived while it waited (a Rclunk
+ owed); an EOF, or anything else, means the peer let go, and a new
+ connection is dialled.
+ - A read on a kept connection that hangs up all the same is asked once
+ more on a new one. A write is not, since it may have been done.
+ - Only `readMount`, `writeMount` and `probeMount` (the Mount word)
+ keep one, since each costs the peer one of its 16 connections while
+ kept. The CLI, `--wait` and Handover dial afresh as before, with
+ their version, attach and walk pipelined.
+- **No qid or stat cache.** The trace shows that a Look through a mount
+ is one operation with one walk. With walk, stat, open and read in one
+ flight, a cache would save bytes but no round trip, and a cached stat
+ is wrong once the peer changes the file.
+- **msize and iounit stay.** They are already 64 KiB, with iounit =
+ msize - 24, so a body of up to 65,525 bytes is one Tread. A larger
+ msize would cost each of a session's 16 connections five buffers of
+ it. test/ninep.py asks 8192 by its own choice, so its body reads of
+ more than that cost it more messages.
+- **`/n/<host>`.** The daemon serves no 9P yet (phase 3), so there is no
+ remote daemon to mount. A remote session mounts today with two
+ commands, now in reference.typ: `ssh -C -N -L /tmp/box.sock:<its
+ socket> host`, then `Mount box /tmp/box.sock`. A relay in the daemon
+ that lists a host's sessions under one mount waits for the daemon to
+ serve 9P. `-C` is there because the box's link is slow in that
+ direction (below).
+
+Measured. The harness is `.scratch/daemon/ninep/rt.py`. It runs a peer
+session and a local one that Mounts it through a relay. The relay holds
+each chunk a fixed delay after it arrives, as test/delay_relay.py does,
+and it also splits the stream into 9P frames, counting the T-messages
+and the rounds (runs of T-messages with no answer delivered between
+them). `box.sh` runs the same harness against a detached session on the
+box through `ssh -L`. Builds are ReleaseFast (main ymwmwrvx against this
+line), 3 reps each, medians.
+
+The RTTs column counts the round trips the client waited on. pardes's
+own client sends its last Tclunk without waiting, before and after; the
+python client waits on every message. The box columns are `ssh -L` at
+207 ms, plain and with `ssh -C`; "fails" means `dial failed: timed out`.
+
+| operation (pardes's client) | msgs | RTTs | 30 ms | 80 ms | 207 ms | box | box `-C` |
+|---|---|---|---|---|---|---|---|
+| connect: `Mount` (version, attach, root read) | 5 → 7 | 4 → 1 | 124 → 33 | 324 → 81 | 831 → 208 | 1032 → 426 | 1024 → 410 |
+| file read: Look `/n/peer/index` (no length) | 7 → 6 | 6 → 2 | 185 → 61 | 486 → 161 | 1247 → 415 | 1441 → 426 | 1434 → 409 |
+| directory listing: Look `/n/peer/pane/` | 7 → 6 | 6 → 2 | 185 → 62 | 483 → 161 | 1247 → 415 | 1437 → 427 | 1430 → 410 |
+| ctl write: Save to `/n/peer/pane/1/ctl` | 6 → 4 | 5 → 1 | 155 → 32 | 402 → 81 | 1039 → 208 | 1237 → 214 | 1233 → 206 |
+| pane body read: Look `/n/peer/pane/1/body` (30 KB) | 7 → 5 | 6 → 1 | 186 → 33 | 483 → 81 | 1247 → 208 | 1843 → 641 | 1432 → 206 |
+| a remote file open: Look `/n/peer/os/.../fs.zig` (77 KB) | 8 → 6 | 7 → 2 | 213 → 64 | 563 → 162 | 1454 → 416 | fails → 1300 | 1844 → 617 |
+| a remote file open, 1.3 MB | 27 → 25 | 26 → 3 | 807 → 103 | fails → 244 | fails → 625 | fails → 5334 | fails → 1273 |
+
+| the python client (unchanged, for scale) | msgs, RTTs | 207 ms | box |
+|---|---|---|---|
+| connect | 2 | 415 | 614 to 635 (with ssh's channel open) |
+| read `/index`, list `/pane` | 5 | 1037 | 1023 to 1067 |
+| ctl write | 4 | 829 | 818 to 852 |
+| body read, 30 KB | 5 | 1037 | 1024 (`-C`) to 1283 (plain) |
+
+- **Each operation is one round trip.** A directory, or a file of no
+ stated length, takes two. The first operation through a new `ssh -L`
+ forward pays one more, the channel opening (the Mount, 426 ms on the
+ box).
+- **On the box, bytes cost more than the 9P rounds.** The link from the
+ box to here carries 170 to 400 KB/s through ssh. A 1 MiB `dd` takes
+ about 6 s and 4 MiB about 10 s, where 4 MiB up takes 2.4 s. A 64 KiB
+ answer arrives 1 to 2 RTTs after the request, so the 30 KB body reads
+ in 641 ms plain and 206 ms with `-C`. The 1.3 MB file's Rreads come in
+ at about one a round trip (the timeline is in `rt.py`'s `TIMELINE=1`).
+ `blast.py` measured all 21 of its Treads in flight at once at 1.9 to
+ 3.0 s, against 5.3 s for this client's 14 and then 7. A sliding window
+ instead of batches would buy that, but only for large files on this
+ link (not done). Compression helps every size: `-C` is in the
+ documented recipe, and the replica stream's zstd (phase 6) will face
+ the same link.
+
+No local regression. Local 9P has no delay. lab.py runs the base and
+the new build at once with their samples alternating, 1000 each, under
+the perf lock, and gives the p50 ratio new/base. The server is not
+changed, so its rows are level. The mount rows are faster because they
+dial nothing.
+
+| local, p50 | base | new | A/B | A/A |
+|---|---|---|---|---|
+| python client: read a 9 KB body (server path) | 88 µs | 89 µs | 1.004 | 1.000 |
+| python client: ctl write `clean` (server path) | 80 µs | 81 µs | 1.011 | 0.992 |
+| Look `/n/peer/pane/1/body` through a local mount | 329 µs | 255 µs | 0.776 | 1.009 |
+| Save to `/n/peer/pane/1/ctl` through a local mount | 214 µs | 141 µs | 0.660 | 1.000 |
+
+The gates:
+- test/fs.py, `zig build 9p-io-test`, unit-test tty, the snapshots and
+ test/daemon.py (20 cycles) pass.
+- New: `a mount's connection is kept between operations, and one the
+ peer hung up is dialled again` (a write longer than a message on a
+ kept connection, a miss, a hangup, a redial and a forget). It also
+ passes on the box (FreeBSD test build).
+
+Not done, and why:
+- **cloud9's client should take requests while it versions;** the poke
+ in `greet` stands in for that until cloud9 is pushed and re-pinned.
+ 9ns (cloud9's) still waits one round trip per message.
+- **A write longer than one message** sends its chunks after the first
+ one at a time: a Save of a 1 MB file through a mount is 16 round
+ trips. Pipelined Twrites could run in the wrong order once one parks;
+ that does not matter for `/os` (pwrite at an offset) but would for
+ some pane files.
+- **A file of no stated length** is not read ahead, and its end (like a
+ directory's) costs a round trip more.
+- **The CLI's `ask`** (`pardes file` from a pane, `pardes -`) and
+ `--wait` pipeline only version, attach and walk. Their open, writes
+ and reads go one at a time, on a connection of their own.
+- **A peer's Restore cuts its clients up to 200 ms after it starts.** A
+ request on a kept connection in that window is refused `Input/output
+ error`, where a new dial would already have been the replacement's.
+- **Scripting from here to a remote session** through test/ninep.py or
+ 9p(1) is still one round trip per message (1 s for a read at 207 ms).
+ A python helper for remote scripting could pipeline the same way;
+ nobody scripts that way yet.