diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/9p.typ | 702 | ||||
| -rw-r--r-- | docs/cloud9.md | 46 | ||||
| -rw-r--r-- | docs/design.typ | 2071 | ||||
| -rw-r--r-- | docs/detached.md | 49 | ||||
| -rw-r--r-- | docs/ideas.typ | 2 | ||||
| -rw-r--r-- | docs/lsp-evaluation.md | 6 | ||||
| -rw-r--r-- | docs/refs.yml | 82 | ||||
| -rw-r--r-- | docs/registry.typ | 1209 | ||||
| -rw-r--r-- | docs/typ/building.typ | 293 | ||||
| -rw-r--r-- | docs/v9fs.md | 56 |
10 files changed, 297 insertions, 4219 deletions
diff --git a/docs/9p.typ b/docs/9p.typ deleted file mode 100644 index 50cec965..00000000 --- a/docs/9p.typ +++ /dev/null @@ -1,702 +0,0 @@ -// Second draft. The first is superseded; what changed and why is recorded -// entry by entry in `docs/registry.typ`, and every §-to-entry link below is -// live. Build: typst compile docs/9p.typ docs/9p.pdf -#let app = "PaRDeS" - -#set page( - paper: "us-letter", - margin: (x: 1.5in, y: 1.2in), - numbering: "1", - number-align: center, -) -#set text(font: "Libertinus Serif", size: 11pt, lang: "en") -#set par(justify: true, leading: 0.62em, first-line-indent: 1.5em) -#show raw: set text(font: "DejaVu Sans Mono", size: 8.5pt) -#show raw.where(block: true): block.with(inset: (left: 1.5em, y: 0.4em), width: 100%) -#set table(stroke: none, inset: 0.35em) - -#set heading(numbering: "1.") -#show heading: it => { - set text(size: 11pt, weight: "bold") - v(1.1em, weak: true) - block(counter(heading).display() + h(0.6em) + it.body) - v(0.4em, weak: true) -} -#show heading.where(level: 1): it => { - set text(size: 11pt, weight: "bold") - v(1.4em, weak: true) - block(counter(heading).display() + h(0.6em) + it.body) - v(0.5em, weak: true) -} - -/// A registry cross-reference. Every contestable claim in this note is an -/// entry over there, with its evidence and its losing arguments attached. -#let r(id) = text(size: 9pt)[#raw(id)] - -#align(center)[ - #v(0.5em) - #text(size: 15pt)[A File Interface for an Editor and Terminal Multiplexer] - #v(0.9em) - #text(size: 10.5pt, style: "italic")[architecture draft, second revision] - #v(0.4em) - #text(size: 10pt)[#datetime.today().display("[year]-[month]-[day]")] -] - -#v(1.2em) - -*Historical proposal, superseded by `docs/fs.md`.* Pardes now serves 9P by -default over Unix sockets, with optional TCP and QUIC. FUSE support and the -old examples have been removed. The arguments and source references below -describe the earlier implementation, not the current interface. - -#block(inset: (x: 2.5em))[ - #set text(size: 10pt) - #set par(first-line-indent: 0em) - #text(weight: "bold")[ABSTRACT] #h(0.8em) - #app is an editor with attached terminals. It runs as a daemon; clients attach - to it. It already serves acme's control filesystem, over FUSE, on Linux only. - This note adds 9P as that filesystem's second transport, and as a carrier for - the display protocol it already has. The program becomes a file server that - needs no kernel, so the tree reaches macOS, a browser tab, a microcontroller - and another machine, none of which FUSE can reach. The same program is also a - 9P client, which is how it edits a file on a board it cannot mount. We - describe the interface, what it costs — measured against two shipping - implementations rather than estimated — and, at greater length, the parts we - do not build. The first draft of this note proposed replacing FUSE and - replacing the wire protocol. Both are wrong, for reasons that turn out to be - cheap to state and expensive to discover. -] - -= What this is for - -One sentence, because every other section is downstream of it. - -*PaRDeS has no way to ask anything a question.* It can ask the kernel, and that -is all. Two running instances can do exactly two things to each other: shout — -`nested.sendLook` formats one line, connects, writes, returns, and never reads, -and its own test asserts that nothing comes back — or *become*, which is what -`Attach` does, and `Attach` is not a connection but a replacement: on a -successful handshake the frontend reaps its pane shells, closes its watches, -unmounts its filesystem, deinitialises the core, and turns into a thin client of -somebody else's. Nothing in the tree reads `PARDES_FS`. `Event` has no variant -for a peer. The 21-method host vtable has no peer method. Reading another -session's text is possible only by shelling out to `cat` a path derived from a -pid you must already know, on Linux, if that session was started with `--fs` — -and impossible against a detached session, which serves no filesystem at all. - -9P is a request/response protocol that crosses machines. That property is the -one thing none of the three private mechanisms can be extended to have, and -every item on the wanted list is the same feature wearing different clothes: -scripting from macOS, driving a session over a network, reading the board's -memory, chaining one instance to another without destroying either. All of it -is *being able to ask, and get an answer back*. - -That is the justification, and it is the only one. In particular it is not -simplification: §12 puts the honest figure at between 750 and 1,050 lines of -net growth, and the measurement is in the registry, not in a hope. What does -fall is the number of ideas — four framings become two, three schemes for -"which instance?" become two, two version negotiations become one, and six -concepts currently duplicated across 799 lines become one copy each. Fewer -ideas, more lines. Anyone selling this as "simpler" has not counted. - -= What changed, and why this draft exists - -The first draft argued that 9P should replace two things: the FUSE mount, and -the private protocol on the daemon's socket. Neither survives contact with the -sources. - -It cannot replace the mount. Linux grades mount privilege by a per-filesystem -flag, and 9P and FUSE are on opposite sides of it. `mount_capable` falls back to -`capable(CAP_SYS_ADMIN)` — root in the *initial* namespace — for any filesystem -that does not set `FS_USERNS_MOUNT`; FUSE sets it, v9fs does not. So -`mount -t 9p` costs real root, a user namespace does not help, and `trans=fd` -does not help because the syscall is checked before the transport is consulted. -The 281 lines in `src/fuse.zig` that fork a setuid helper and receive a -descriptor over `SCM_RIGHTS` are not overhead; they buy an unprivileged -pathname, which 9P cannot buy at any price on Linux. The usual escape, `9pfuse`, -is FUSE again in someone else's process — which is exactly what `ad`, the one -editor that already took this route, ships #cite(<ad>). #r("9P-13") - -It cannot replace the wire either, because the wire's payload is a compressed -cell grid and 9P has no server-initiated message. But the interesting half of -that finding is that 9P *can carry* the wire unchanged, and that this is how -remote display has actually shipped for thirty years #cite(<devdraw>). #r("9P-12") - -What is left is better than what was proposed, and smaller. - -= Two layers - -The first draft's mistakes are almost all one layer's property asserted about -the other, so the layers are worth naming before anything else. - -*Layer 1 is the control tree.* Request and response, nine operations, a pure -transaction over the core: `handle(p, req) Reply` in `src/acmefs.zig`, whose ABI -already records that "FUSE opcodes, 9P messages and a unit test all reduce to -these". It is transport-neutral today and unit-tested with no transport at all. - -*Layer 2 is the display protocol.* Nineteen message tags in -`src/detached/wire.zig` carrying run-length-encoded cell grids and input, one -message per frame, pushed by the server to N frontends that share one screen. - -They share a machine and nothing else. The tree is not private and not -undocumented — it is acme(4) #cite(<acme>). The wire is both, and deliberately: -it is a codec, and its cost is measured at seven lines per new fact, which is -cheap for what it does. - -9P is Layer 1's second transport and Layer 2's carrier. It is never a -re-encoding of either. #r("9P-1") - -= The interface - -A session is a tree. Windows are numbered directories under it. - -``` -/ - ctl session control; read lists windows - index one line per window: id, tag, flags - new/ walking here creates a window - 7/ - ctl commands in; id and state out - tag the tag line - body the text - addr an address - data read and write at addr - event the interesting one; see below - errors writes appear in the error window - pty/ - data the stream - ctl winsize, raw, cooked, sig, exec - status dimensions, mode, exit status -``` - -This is acme's tree plus `pty/`, minus the plan9 compatibility stubs. Four -conventions matter, and the first draft got three of them wrong. - -*Offsets are honoured everywhere they mean something.* `body`, `tag`, `addr`, -`ctl`, `index` and `rdsel` are seekable files and are read at the offset the -message carries. acme does the same, and its entire byte-to-rune cache exists to -serve a non-zero offset on `body`. Only `data` and `event` ignore offsets, and -only because they are positioned by `addr` and by the queue respectively. All -writes ignore offsets: a write to `body` appends, which acme(4) specifies. A -server that ignored offsets on `body` would make `cat`, `wc`, `diff` and `tail` -return the first chunk forever, which is the opposite of the whole point. - -*Directory reads carry a cookie, not an index.* 9P requires a directory read at -offset zero or at exactly the byte offset where the previous read ended, and the -reply must contain whole `stat` entries; anything else is `Ebadoffset`. The core -counts entries, so the transport keeps a per-fid byte cursor and the one entry -that did not fit. This is the only genuine offset discontinuity between FUSE and -9P, and it is about forty lines. #r("9P-5") - -*The address belongs to the window.* The first draft claimed per-fid and -attributed it to acme; acme keeps `Range addr` in `struct Window`, #app keeps it -per pane, and `ad` keeps it per buffer. Per-fid is a real improvement — two -scripts could then address one window without colliding — but it is a proposal, -not a restatement, and it is the only thing in this note that asks the core to -grow state. It is argued on its own merits or not at all. #r("9P-6") - -*`event` is opened by a client, not by a fid.* One controller per window is -right; enforcing it per fid is not, because a client that opens the file twice -is not two controllers. `ad` reaches the same conclusion and scopes `QTEXCL` to -a connection. #r("9P-7") - -= Events - -A window handles its own mouse and keyboard until a program opens `event`. From -then on it ships them out: origin, action, the character range, the text. The -program reads a record, decides, and writes it back to accept it; or handles it -itself and stays silent. - -This is the whole extension mechanism and it already works — `examples/acmefs/` -has four programs that use it, one of which puts words #app has never heard of -into a tag and makes them execute. There are no callbacks and no plugin API. A -program that wants to redefine what the right button does opens a file and reads -it. - -A read blocks. The core does not: it answers `again` — nothing consumed, ask me -later — and the transport holds the request. That inversion is what lets a -single-threaded core serve a filesystem at all, and it is worth stating plainly -because the prior art does not have it. `ad` spends three to four threads per -connection and then serialises all of them behind one mutex and a blocking -round trip into the editor thread. #app answers a blocked `event` read in -twenty-one nanoseconds and allocates nothing. - -= Terminals - -A pty is a file interface wearing the wrong clothes. Everything one wants to do -to it is an `ioctl`, and 9P has none in any dialect. They become writes to -`pty/ctl`: - -#v(0.3em) -#align(center)[ - #set text(size: 9.5pt) - #table( - columns: 2, - align: left, - column-gutter: 2em, - row-gutter: 0.3em, - [`TIOCSWINSZ`], [`winsize 80 24`], - [`TCSETS`], [`raw`, `cooked`, `echo off`], - [`kill`], [`sig INT`], - [`TIOCGWINSZ`], [read `status`], - [spawn], [`exec /bin/sh`], - ) -] -#v(0.3em) - -`pty/data` is the stream. A write is input to the process; a read blocks until -there is output. On exit a read returns zero bytes rather than an error, because -clients already know what end of file means. - -Two things about this section that the first draft did not say. It has *nothing -to do with 9P*: `push_spawn` and `push_pty_resize` are effects the core already -has, so `exec` and `winsize` are existing capabilities acquiring a name, and -`sig INT` is the only new one. Build it in the FUSE tree first and the 9P -transport inherits it. And it has *no prior art anywhere*: acme has no pty -files, `ad` has no terminal surface at all. Being first is a reason to keep it -to three files and stop. #r("9P-8") - -Every keystroke in raw mode is a round trip. On a local socket this does not -matter. Over a slow link it matters a great deal, which is why Plan 9's terminal -programs do line editing locally and send whole lines. Raw mode should be -entered deliberately and left promptly. - -= Layering: how 9P and the wire meet - -Six arrangements were costed. The finding that decides it is that the cost is -almost the same in all of them: a base-9P2000 codec, a dispatcher onto the -existing nine operations, and a fid table come to roughly 970 lines that appear -in every option. Layering moves about 150 lines either way. So this is not a -cost question. It is a question of reach. - -#v(0.3em) -#align(center)[ - #set text(size: 9pt) - #table( - columns: (auto, auto, auto, 1fr), - align: (left, right, right, left), - stroke: (y: 0.4pt), - table.header([], [msgs/frame], [msgs/key], [what decides it]), - [O1 9P replaces all], [2], [4], [64 park slots needed against 32; a round trip per frame], - [O2 two listeners], [1], [2], [no routing origin for the 9P descriptor], - [O3 9P inside the wire], [1], [2], [*works over the board's UART unchanged*], - [O4 wire inside 9P], [2], [4], [+34 B on a 56 B diff, or +61%], - [O5 side by side], [1], [2], [cheapest, but cannot reach the board], - [O6 frontend serves], [2], [4], [core would write and match tags inside `update`], - ) -] -#v(0.3em) - -That table is superseded, and the correction is the most important thing in -this note. It was written believing 9P has no way to push a frame, so every -arrangement that carried frames over 9P paid a round trip. The premise is -wrong — not because 9P can push, but because *the pushing end should be the -client.* - -Plan 9 has shipped this for thirty years and it needs no invention. `drawterm` -dials out to a cpu server, writes a shell script, and then calls -`exportfs(fd, fd)`: the dialer becomes the *server* on the socket it dialed. -The script the remote runs is `mount -nc /fd/0 /mnt/term`, and the remote is -the *client*. When the remote application draws, the bytes cross as the -application's `Twrite` to `/dev/draw/N/data`, and the terminal — the machine -with the screen — executes them. One descriptor, roles fixed per side, data -flowing both ways, no server-initiated message anywhere. - -So the rule is one role per connection, not one role per message: - -#v(0.3em) -#align(center)[ - #set text(size: 9.5pt) - #table( - columns: (auto, auto, 1fr), - align: (left, left, left), - column-gutter: 1.2em, - row-gutter: 0.3em, - [*display*], [frontend serves], [core `Twrite`s frames to `screen`, blocking-`Tread`s `input`], - [*session*], [core serves], [scripts and other instances walk `7/body`, `event`, `ctl`], - [*devices*], [board serves], [core reads `mem/`, `gpio/`, `prof`], - ) -] -#v(0.3em) - -Three connections, one role each; the program contains both halves and uses -whichever the link calls for. Do *not* build a link that carries both roles at -once. 9P permits it — T-messages are even and R-messages odd, so a stream is -self-demuxing — but nothing has ever done it, and it buys a `Tversion` ordering -hazard in which each side must answer the peer's version while awaiting its -own. - -Two numbers make the display path credible. Chunking is a non-issue: payload -per message is 131,072 bytes at Linux's default, so every real frame — 20 B, a -56 B keystroke diff, a 6,298 B full board frame, a 27 KB desktop frame — is one -`Twrite`, and the 1.7 MB worst case is thirteen. And the core need not block: -tags are allocated per outstanding request with no in-order reply requirement, -which is why `exportfs` can answer out of order at all. Where `exportfs` needs -a slave process per blocked request, PaRDeS needs none — `Status.again` and the -park table are the same thing done single-threaded. - -Copy the file discipline rather than inventing one: `/dev/draw/N/data` is a -write-only batched binary command stream with an exclusive `ctl`, and -`/dev/mouse` is an exclusive single-reader file whose read blocks until -something happens. That is `screen` and `input` already designed, and it is the -shape `event` already has. - -Frames are never re-represented. Whatever carries them carries `encodeFrame` -bytes; the existing five-byte length prefix already makes such a message a legal -9P payload. The counter-example is worth keeping in view: Plan 9's `/dev/screen` -is an offset-addressed raster a client polls, with no change notification, which -is precisely why nobody ever ran a remote rio through it. - -One scope limit, measured and not negotiable. This applies to *remote* displays -only. Turning the local host vtable into a tree costs every shell more than it -saves — tty and gui about +204 lines each, macOS +164, and the web shell +142 -lines of Zig plus roughly 500 of JavaScript, because its `present` today writes -a flat cell array that JavaScript reads straight out of wasm memory and 9P would -make that an RLE stream needing a decoder and a codec. The board is worse in -kind: its host is the same process behind a function pointer, so a message pair -per frame is overhead against a direct call. `host.VTable` is already the -abstraction — the design document says the surface *is* the abstraction and -refuses a layer over the shells — so a 9P host is one more implementation of -that vtable, chosen when the display is on the other end of a wire, and never -for the window in front of you. #r("9P-19") #r("9P-23") - -= The client half - -The same program is a 9P client. This is not symmetry for its own sake: it is -how one instance shows another instance's windows, and how it edits a file on a -machine it cannot mount. - -The client half needs no mount, and after §1 that is no longer one advantage -among several — it is the whole of what 9P buys that we cannot otherwise have. -To edit a file on the board, #app walks to `board/fs/etc/config`, opens, reads, -edits in a buffer, writes, and clunks. There is no path on the local machine, no -kernel involvement, and no privilege. That works on macOS, which has no 9P in -the kernel and no control filesystem at all today; it works in a browser tab, -which has no filesystem at all; it works on the board, which has no sockets; and -it works when the operator is not root, which on Linux is now the interesting -case. - -A mount is still worth offering, because `grep`, `make` and the compiler take -filenames and a filename is the one thing a client library cannot produce. But -on Linux that mount is `src/fuse.zig`, which we keep, or it is `9pfuse`, which -is `src/fuse.zig` written by someone else. It is not `mount -t 9p` unless the -operator is root and chooses to be. - -= The board - -The microcontroller is the case that motivated this note, and both the first -draft and its review were wrong about it in opposite directions. - -The first draft said the board runs "a 9P server and nothing else". It runs the -whole editor, as firmware, with two of twenty-one host methods filled. The -review said a 9P server would not fit, quoting nine kilobytes of free heap. That -number is two refactors stale: since the shadow grids were reduced to one cell, -the heap reports around 336 KB free and the binding constraint moved to the -240 KiB of low memory that holds `.bss`. #r("FIX-2") - -Costed from real components, a 9P server on this board is: - -#v(0.3em) -#align(center)[ - #set text(size: 9.5pt) - #table( - columns: (auto, auto, 1fr), - align: (left, right, left), - column-gutter: 1.2em, - row-gutter: 0.3em, - [two `msize` buffers], [8,192 B], [u9fs uses three; a minimal server needs two], - [fid table, fixed array], [512 B], [32 entries × 16 B, linear scan ≈0.4 µs], - [codec scratch], [128 B], [one `stat`, `ERRMAX`], - [#text(weight: "bold")[RAM total]], [#text(weight: "bold")[8,832 B]], [2.5% of free heap], - [flash, 9P-only image], [≈39 KiB], [2.5% of a 1.5 MB partition], - ) -] -#v(0.3em) - -The 4,096-byte floor that sizes those buffers is imposed by the Linux kernel and -by nothing else; Plan 9, plan9port and #app's own client all accept a -512-byte `msize`, which would halve them. And a 4,096-byte reassembly buffer -already exists on this exact serial line, with the property measured: that much -in a single write arrives intact. - -So it fits, easily. What does not work is having it both ways: the editor owns -the one UART bidirectionally and the board's header exposes no second one, so a -9P server on the P4 is a *second firmware image*, not a second role for this -one. The build already expresses that shape — the on-die test suite is a -separate executable with its own image and flash steps in fifty-four lines, and -it links no editor object. The board is an editor or a filesystem, and saying so -is better than implying both. #r("9P-11") - -Before quoting any latency for it, raise the console to 921600 baud. It is one -divider write on the existing crystal, the routine already exists, and it takes -a warm read of a small file from 10.8 ms to 1.35 ms. #r("BOARD-1") - -The tree such an image would serve is not invented. The board already exposes -its whole address space, its pin header and one pad toggle — as *acme words a -human types into a tag*, capped at four kilobytes per command because of the -console, with the answer landing in an output pane. Nothing about it is -machine-readable and nothing is remote. `mem/`, `gpio/pinout` and `prof` are -backed by functions that exist today; four more files need one new exported -symbol each; four have no implementation at all. A 9P tree is that capability -with names instead of verbs, and the inventory above is the honest scope. - -= What we do not build - -The temptation in this design is to keep going. Each of the following was -considered and declined. - -*A window system.* #app draws in one window. Panes are ours; surfaces are not. -Becoming a Wayland compositor would buy the ability to put someone else's -program in a pane, and cost input handling, buffer management, output hotplug, -scaling, clipboard, and a permanent maintenance obligation against a moving -protocol. We do not want someone else's program in a pane. - -*A namespace.* Per-process namespaces, union directories, `bind`, and walks that -cross mount points are what make Plan 9's implementation large #cite(<plan9>). -The distinction is worth defending, because the first union directory brings the -rest. - -*An aggregate.* The first draft's longest technical section described a prefix -router that re-exports attached instances, and its own final section concluded -the aggregate is not worth building before a second machine exists. That verdict -is right and also covers the client half of it. It also understates the work: a -proxied `Twalk` cannot be answered until the remote `Rwalk` arrives, and the -core may not block, so it is not a fid map — it is per-tag continuations, tag -remapping, flush forwarding and fid invalidation on connection death, in a -daemon that is one `poll(2)` with no worker pool. Deferred with a named -precondition rather than described as easy. #r("9P-10") - -*A file server.* Where a host's files must be exported, we run `u9fs` or `diod` -and attach as a client #cite(<u9fs>). Real filesystems are where the difficulty -in 9P actually lives: stable qids, `..` clamped at the export root, symlinks -that escape, identity mapping, and the full `wstat` surface. Our own trees are -synthetic, so we invent every file and there are no cases we did not choose. -Note that `u9fs` serves its requests from a strictly serial loop and its flush -handler is a `break`, so it is a reference for the tree and not for §10. - -*Authentication.* Not now, and not by us. The Unix socket is protected by the -permissions on the socket. Where a network is involved, the connection is -tunnelled. Linux's client never sends `Tauth` at all, and Plan 9's `mount` only -does so without `-N`, so a server that refuses it is not an obstacle to either. -Real authentication is `Tauth` or TLS, and it can be added without changing the -tree. - -= Protocol notes - -We serve base 9P2000 #cite(<intro5>). Plan 9 mounts it natively, plan9port's -`9p` speaks it, and Linux mounts it with `version=9p2000` #cite(<v9fs>). The -following are the traps, in the order they will bite. Each is cheap once known -and each has been shipped wrongly by someone. - -*Set `qid.version` to zero on every file.* This is the 9P equivalent of FUSE's -`FOPEN_DIRECT_IO` and it is server-side, not advice to the operator: Linux's -client disables both read and write caching for any file whose qid version is -zero, whatever the cache mode, unless `ignoreqv` is passed explicitly. A -synthetic tree of live editor state has no business being cached, and this is -how we say so. The first draft's "mount with `cache=none`" was unnecessary; its -claim that the default message size is small was simply wrong, since the default -is 128 KiB and the cap is 1 MB. #r("9P-3") - -*Clamp every reply to the client's `count`.* An `Rread` longer than the count -asked for is a hard `EIO` in the Linux client, not a truncation. This has no -FUSE analogue, which is why nobody looks for it, and `ad` gets it wrong on -exactly the file we care about. The existing rule that a read too small for one -event record is refused rather than split is what makes the clamp safe on -`event`. #r("9P-17") - -*Decide the error ABI deliberately.* `Rerror` in base 9P2000 is a string, and -Linux recovers an errno by exact match against a fixed table; a miss is not -`EIO` but `ESERVERFAULT`, which userspace prints as "Unknown error 526". `ad` -emits nineteen prose strings and not one of them is in that table. Either emit -the exact strings Linux knows, or serve 9P2000.u and send the number. The second -also restores `statfs`, which base 9P2000 does not have and which the core's -operation set includes. #r("9P-4") - -*`Tflush` is a park-table lookup.* The common failure is not omitting it — it is -implementing the message and still hanging, which is what `ad` does: correct -`Rflush` ordering in thirty-nine lines, and a filesystem hook that defaults to -doing nothing. #app is structurally better placed than either reference, because -the thirty-two-slot park table is already keyed per outstanding request and -already answers `FUSE_INTERRUPT` the same way. Implement flush against the -table, not against a callback, and the bug is unrepresentable. #r("9P-16") - -*Serve requests concurrently.* Several tags may be outstanding on one connection -and `event` reads block by design. This does not require threads: it requires -that a blocked request be parked rather than waited on, which is what the core's -`again` already means. - -= Cost, and the order to build in - -The honest figure is 1,600 to 1,900 lines of server plus around 500 of tests. -This is measured, not estimated: `ad`'s 9P2000 server core is 2,321 lines -(codec 704, stat and mode 437, session and fid and flush 615, loop and handlers -474) and `u9fs` is 1,838 with an 805-line codec. Two independent -implementations agree on codec ≈750 and server logic ≈1,700. Roughly 970 of -those lines are identical under every layering, so none of the estimate is at -risk from §6. #r("9P-15") - -What comes off against it has now been counted too, and it is less than hoped. -Six concepts are genuinely duplicated across the existing mechanisms — -endpoint-path derivation in six copies, directory create-and-vet in six, stale -sweeping in three, bind/listen/accept in three, "which instance?" in three, and -version negotiation in two — 799 lines in total, of which about 330 collapses -to one copy. Add `nested.zig`'s socket half and its socket tests, 518 more, and -the deletion is *≈850 lines*. What cannot be deleted is ≈7,700: `src/fuse.zig` -by §1, `wire.zig` because frames stay frames, the daemon's host half, the -158-line ancestor walk that answers a question 9P has no message for, and every -kernel interface that was never a protocol choice — inotify, the pty, the -subprocess pipes, `mkstemp`. - -*Net: between 750 and 1,050 lines of growth.* It breaks the house rule that a -change leaves the file it touches no longer than it found it, §1 forbids -retiring `src/fuse.zig` in exchange, and there is no arithmetic that rescues -it. The growth is justified by reach and by §0's reply, or it is not justified. - -One cost is still unpriced and should be before any code is written. `wire.zig` -chooses every protocol tag in one file so that reordering a core enum is a -compile error, and a test asserts that exactly eleven client tags exist and that -the other 250 bytes are refused, so a frontend built before a change cannot -forge a pane's output into a session built after it. 9P's codec validates 9P; it -cannot validate that a byte string is a legal keystroke. An exhaustive switch -checked by the compiler becomes a runtime string parse with nothing to bind to. -The likely answer is to generate the `ctl` grammar from the same enum so the -compile error survives as a parser generator, but nobody has tried it. -#r("9P-20") #r("9P-21") - -The order matters more than the total. Each step is stated with what it is, -where it lands, what becomes possible that was not, and how you know it worked. -Steps 1 to 3 are independent of each other and of 9P; step 4 needs step 2; -step 6 needs step 5. Any prefix of this list is a reasonable place to stop. - -== Step 1 — let a detached session be scripted - -A daemon started with `--detach=work` holds the core, and today it serves no -control filesystem at all: `src/detached/server.zig:566-570` records that it has -no `push_fs_reply` "because this process mounted no /dev/fuse". It simply never -calls `fs_service.start`. Nothing prevents it. - -The change is a `Source.fuse` variant in the poll union, an `fs` field, the -`fs_service.start` call, `fsReply` copied verbatim from `src/tty/tty.zig`, one -vtable entry, a pollfd and its dispatch arm, and the `drain` call. About sixteen -lines. It is *better* in the daemon than in the desktop hosts: those need a -background thread to notice a request, and the daemon's own `poll(2)` already -watches everything, so it can watch `/dev/fuse` too and the thread disappears. - -After: `pardes --detach=work` and then `ls $XDG_RUNTIME_DIR/pardes/<pid>/` works, -and every script in `examples/acmefs/` drives a daemon it previously could not -see. Proof is those four programs and the existing filesystem snapshots, run -against a detached session instead of a local one. #r("9P-14") - -== Step 2 — name the transport seam - -`fs_service.drain` and `step` take a `*fuse.Fs`, but they only ever call three -methods on it: `retry()`, `next()` and `reply()`. Replace the concrete pointer -with a context pointer and a three-function vtable, at the three call sites in -`tty.zig` and `gui.zig`. Forty lines changed, none added, `fuse.Fs` is the first -implementor, and nothing observable changes. - -This is the whole preparation for a second transport, and it is worth doing on -its own merits: if 9P is never written, the seam is documented in code rather -than in a comment. #r("9P-2") - -== Step 3 — make terminals scriptable - -Add `pty/data`, `pty/ctl` and `pty/status` to each pane's directory in -`src/acmefs.zig`, with `ctl` taking `winsize 80 24`, `raw`, `cooked`, -`echo off`, `sig INT` and `exec /bin/sh`. - -Today a script can write into a terminal that already exists and read its -rendered scrollback, and that is all — it cannot start one, resize one, or -signal one. Two of those are free: `push_spawn` and `push_pty_resize` are -already effects the core emits (`src/host.zig:80-82`), so `exec` and `winsize` -are existing capabilities acquiring a name. Only `sig INT` is new work; there is -no `kill` anywhere in `host_io.zig`. - -This has nothing to do with 9P. It lands in the FUSE tree, works immediately, -and any later transport inherits it. Keep it to three files: acme has no pty -files and neither does `ad`, so there is no prior art to be wrong about, which -is a reason for restraint rather than ambition. #r("9P-8") - -== Step 4 — serve the existing tree over 9P - -A new `src/ninep.zig`: base 9P2000 over a unix socket, serving exactly the tree -`src/acmefs.zig` already defines. No client, no aggregation, no `aname`, and no -new files — step 3 already added the only ones wanted. - -Implement `Tversion`, `Tattach`, `Twalk`, `Topen`, `Tread`, `Twrite`, `Tclunk`, -`Tstat`, `Twstat` and `Tflush`, and answer `Rerror` to `Tauth`, `Tcreate` and -`Tremove`. Lift the park table and its `retry`/`take`/`freeSlot`/`findSlot` -helpers out of `src/fuse.zig` unchanged — `Status.again` means the same thing to -both transports. What is genuinely new is the codec, the fid table, a per-fid -directory cursor, and `stat` marshalling. Every trap in §11 applies here and -nowhere else. - -After: macOS and the browser have a control filesystem for the first time, and a -script on another machine can drive a session over TCP. On Linux the FUSE mount -stays, because `mount -t 9p` needs root and `fusermount3` does not — the two are -complementary, not competing. Proof is the existing snapshots run through the -9P transport unchanged, and `examples/acmefs/` run through `9pfuse`. #r("9P-15") - -== Step 5 — become a 9P client - -The other half, and the one that moved up the list, because it is the mechanism -for everything the note is for. A server lets pardes be *talked to*; a client is -how pardes *asks*. - -It is a state machine over a byte stream — walk, open, read, write, clunk — with -no kernel, no mount and no privilege, and the protocol layer is freestanding-safe: -no allocator, no threads, no sockets, so it runs on the board's UART and in the -browser as readily as on a socket. - -After: `Look board/fs/etc/config` opens a file that lives on the microcontroller, -in a pane, with no mount anywhere. And `Attach` gains the sibling it has always -needed — one that *connects* instead of replacing. Today `Attach` deinitialises -the local core and turns the process into a thin frontend; the new word leaves -both cores alive and lets each walk the other's tree. That is the "chaining" -that is currently impossible. #r("9P-22") - -== Step 6 — let a remote display serve the core - -A frontend exports `screen`, `input`, `snarf` and `ctl`; the core dials it and -becomes its client, writing frames to `screen` and taking a blocking read on -`input`. This is `drawterm` exactly, and the file discipline should be copied -rather than invented: a write-only batched command stream with an exclusive -`ctl`, and an exclusive single-reader input file whose read blocks. - -The frame bytes are `encodeFrame` output verbatim — the codec does not change, -only what carries it. *Remote displays only.* The window in front of you keeps -the direct vtable call, because a tree costs every local shell more than it -saves. - -What this actually unlocks: the ESP32-P4 stops being a shrunken pardes — an -809 KB image running a subset of the editor in 384 KB of RAM — and becomes a -terminal for a full core running on the workstation. Roughly 39 KB of firmware, -and it gains language servers, tree-sitter and PDF because those now run -somewhere that can afford them. Raise the console to 921600 baud first, or every -measurement will be eight times worse than it needs to be. #r("9P-19") #r("BOARD-1") - -= Status - -The tree, the event protocol, the pty control language and the layering are -settled enough to implement, and the registry holds the arguments that settled -them. Three questions remain genuinely open: whether the address becomes a -per-fid property and the core grows state to hold it #r("9P-6"); which error ABI -we adopt #r("9P-4"); and whether the parked second core on the board could own a -9P server so the board need not choose between being an editor and being a -filesystem #r("9P-11"). - -Steps one through three are worth doing whatever the answer to the rest is, -which is the best property this plan has. - -#v(1.5em) -#line(length: 30%, stroke: 0.5pt) -#v(0.5em) - -#set text(size: 9.5pt) -#set par(first-line-indent: 0em) - -#bibliography( - title: none, - full: false, - ("refs.yml"), -) diff --git a/docs/cloud9.md b/docs/cloud9.md deleted file mode 100644 index f993750f..00000000 --- a/docs/cloud9.md +++ /dev/null @@ -1,46 +0,0 @@ -# cloud9 - -The `cloud9` package owns the 9P2000 wire format, client and server -connections, the file-server engine and the Unix/TCP/QUIC transports. -`build.zig.zon` pins a commit from `git.sr.ht/~gbrls/cloud9`, fetched into -`zig-pkg/` like any dependency. Re-pin with -`zig fetch --save=cloud9 git+https://git.sr.ht/~gbrls/cloud9#<commit>`, or -use `.cloud9 = .{ .path = "../cloud9" }` while editing both. - -- The engine (fids, jobs, parking, flush, hangup) is cloud9's `fs.Server`; - the control tree in `src/ninep/` is its backend. `src/9p.zig` names the - editor's and the board's `fs.Options` (the editor's: msize 65536, 256 fids, 128 held reads a connection). -- Unix and TCP listeners run on cloud9's `serve.Runner` (`std.Io`: an accept - task per listener, a reader and a writer task per connection, 16 - connections). Requests are answered on the connection's task, which takes - the editor's turn (`pardes.turn`) while the editor waits for input or is - out in a syscall. A request that would change a pane while the editor is - mid-step parks (`Status.again`) and is retried when the editor rests. A - read with nothing to answer yet is held by the core and answered through - the ticket `Conn.hold` gave it, only while that park still waits. -- QUIC (`src/9p_quic.zig`, ALPN `pardes-9p`) still runs on the editor's - poll loop in `src/9p_io.zig`, since cloud9's QUIC adapter is - nonblocking-descriptor based. - -Tests: `zig build 9p-test` (engine configurations), `zig build 9p-io-test --Dquic=true` (transports and client). cloud9's own `zig build test`, -`transport-test`, `quic-test -Dquic=true`, `fuzz` and `differential` cover -the shared code. - -## The posted-9P registry - -`$XDG_RUNTIME_DIR/9p` is this machine's `/srv`: servers post themselves -there by name, and `9ns --mntgen` mounts the whole registry (default -`/mnt/9p`). A pardes whose socket is in the runtime directory posts -`$XDG_RUNTIME_DIR/9p/pardes/<name>`, a symlink to its socket (one directory -per program, as zmx posts its sessions); a socket that fell back to -`~/.local/state/pardes` is not posted. Stopping unposts the entry if it is -still ours. Posting first sweeps the group: an entry that is a symlink whose -socket refuses a connect (ECONNREFUSED) is removed with its socket; anything -else counts as live. - -pardes does not dial the registry: `9ns` is the client, and `--mount` dials -resolve as always (a bare name is a pardes session, anything with a slash a -path). pardes binds its own socket instead of posting through `cloud9.post`, -because `post` takes only flat names and pardes posts into a group -directory. A reader of the registry must `stat` through the symlink. diff --git a/docs/design.typ b/docs/design.typ deleted file mode 100644 index 80c8bb49..00000000 --- a/docs/design.typ +++ /dev/null @@ -1,2071 +0,0 @@ -// docs/design.pdf is a retained test fixture, not regenerated by normal builds. -#import "@preview/cetz:0.5.1" - -#set page(paper: "a4", margin: (x: 1.5cm, y: 1.8cm), columns: 2, numbering: "1") -#set text(font: "New Computer Modern", size: 8.8pt) -#set par(justify: true, leading: 0.55em) -#set heading(numbering: "1.1") -#show heading: set block(above: 1.15em, below: 0.6em) -#show heading.where(level: 1): set text(size: 10.2pt) -#show heading.where(level: 2): set text(size: 9.2pt) -#show heading.where(level: 3): set text(size: 8.8pt, style: "italic") - -#show raw.where(block: true): it => block( - width: 100%, - fill: luma(246), - inset: (x: 0.5em, y: 0.45em), - radius: 1pt, - breakable: true, - text(size: 7.2pt, it), -) -#show raw.where(block: false): set text(size: 8.1pt) - -#show figure.caption: set text(size: 7.6pt) -#set figure(gap: 0.55em) -#set table(stroke: 0.4pt, inset: 0.35em) - -#let wide(caption: none, body) = place( - top, - scope: "parent", - float: true, - clearance: 1.2em, - figure(body, caption: caption), -) - -#let diagram(body) = [ - #show raw.where(block: false): set text(size: 5.9pt) - #body -] - -#place(top + center, scope: "parent", float: true, clearance: 1.5em)[ - #text(16pt)[*Pardes: A Text Environment*] - - #text(8.8pt)[Architecture and implementation of the rewrite] -] - -#place(top + center, scope: "parent", float: true, clearance: 1.4em)[ - #block(width: 82%, inset: (x: 0pt))[ - #set par(justify: true, leading: 0.52em) - #text(8.2pt)[ - *Abstract.* Pardes is a text environment in the acme tradition: columns of - panes, each pane a tag line plus a body, the body a live terminal, a file, - an image or a PDF. It is one program with five thin shells — a terminal - over libvaxis, an SDL3 window, a browser tab driven by vanilla JavaScript, - an AppKit application, and an ESP32-P4 microcontroller — where the - prototype had three parallel implementations of one editor. Two seams - carry that. Outward, the core is a state machine over plain values: - `Event` in, `Surface` and a queue of `Effect` out, and nothing that cannot - be said in those types exists in pardes. Downward, `Host.VTable` is - optional function pointers, with in-process fallbacks, - so the zero-method host is both the test harness and the browser. A - session can also be a daemon: the core keeps the ptys and the disk and N - frontends carry only a screen, a keyboard and a clipboard. - ] - ] -] - -= What it is - -== The acme inheritance - -Columns of panes; each pane is a tag line plus a body; the body is a live -terminal, a file, an image, or a PDF. The mouse carries meaning — left selects, -middle executes, right looks. Everything on screen is text and all text is -equally alive, whether the shell printed it or the user typed it. - -Two acme mechanisms are reproduced rather than reinterpreted. A *dump* is the -session as data: `pardes -l state.zon` on every platform reconstructs the panes -— terminals by replaying their raw VT streams into fresh emulators, files, -images and PDFs from their bytes. Editable tag tails are stored separately from -their dynamic live prefixes, and image records retain PETSCII, palette, and -ASCII renderer choices. A PDF rides the dump's `image` kind, carrying its path, -its bytes and the page it was on. The reader validates weights, scroll ranges, -pane references, and bounded tag tails before constructing anything; invalid -base64 fails instead of silently becoming empty content. A PDF record whose path -cannot be opened — or a build without MuPDF — falls back to an ordinary file -pane with those exact embedded bytes and its original editable tail. - -That is what collapses the prototype's *two* applications into one. Its browser -build was a separate 800-line read-only replay viewer; here loading another -instance's dump is a first-class feature of the one application, and the web -shell is that application with an embedded dump and `spawn` left unanswered. -Same core, no viewer fork. - -The other acme mechanism is a control filesystem, `src/fs.zig` -(@fs). - -== One core, five shells - -Pardes is a *library*, in the way ghostty-vt is a library: you feed it bytes and -events, and you read state out of it. Every platform owns its own event loop and -its own renderer; the core owns everything the user would recognize as pardes. - -*The core* owns layout, panes, modes, selection, click semantics, themes, and the -text of the UI. Pure state machine: `update(event)` mutates, `render(arena)` -returns the surface. No rendering, no event loop, and no IO that has an effect -for it — but "no syscalls" was never true and is less true now: `look` reads a -file a Look opened, walks a directory for Find and reads every candidate for -Grep, `fonts` walks the font directories, and MuPDF opens a `.pdf`. Those are the -ones that are cheaper done in place than round-tripped through an effect and -back; everything with a lifetime — a pty, a window, the clipboard — is still -asked for. - -*A shell* is one MODULE per platform: tty is one file, gui adds the CRT and -gamepad files plus a C font loader and eight GLSL shaders, and web, macOS and the -board each add a host language or a host repository. It owns the event loop, -translates native input into core events, renders the core's surface, and -performs the core's requested effects — spawn a shell, write a pty, open a link. - -The five, and what each one actually is: the terminal (libvaxis); a native SDL3 -window (the steamdeck); the browser (a freestanding wasm core driven by vanilla -JavaScript and rendered as HTML/CSS); a native macOS app (an AppKit and CoreText -shell over a static `libpardes.a`); and an ESP32-P4 microcontroller, a -freestanding riscv32 *object* that the sibling `05-zig-p4` toolchain links beside its own -`_start`, linker script and UART driver (`src/esp32p4.zig` header; the -`pardes-esp32p4` object `build.zig` emits). `pardes.Platform` is -`enum { tty, gui, web, macos, esp32p4 }`. - -Native shells share `host_io.Shell`'s OSC 133 startup snippets, but not their -files. Each host owns a private `mkstemp` pair for its lifetime, writes and -closes both before the first fork, passes those unpredictable paths directly -in child argv, and unlinks them at teardown. Concurrent tty, SDL and macOS -launches therefore cannot truncate, source, or replace one another's startup -files. - -== A shell need not share the process - -`pardes --detach[=name]` runs the core with no terminal of its own and -`pardes --attach[=name]` makes a frontend of it over a unix socket, so one -session can carry a terminal and an SDL window at the same time and outlive -both. `main.nativeMain` dispatches `--detach` before it switches on the -platform, because it is not a shell: the tty and gui builds can both be asked -for one. The two flags together are refused. Local and detached sessions both -serve 9P by default. Both flags take `=name` and never a separate word, so -`pardes --attach README` opens `README` in a fresh session instead of attaching -to one called `README`. See @detached and `docs/detached.md`. - -= The seam <seam> - -#wide(caption: [The core/shell seam. `Event` is the only way in; `Surface` and a -queue of `Effect` are the only ways out. `Host.VTable` is how an `Effect` -reaches a resource, and every method a host leaves null is answered by -`host_io.Fallback` inside the same process.])[ - #diagram[ - #cetz.canvas(length: 0.995cm, { - import cetz.draw: * - set-style(stroke: 0.4pt, mark: (fill: black, scale: 0.35)) - - // ---- the core ---- - rect((6.0, 0.9), (11.6, 4.3), name: "core") - content((8.8, 4.02), text(7.6pt)[*the core* --- `src/pardes.zig`]) - line((6.0, 3.78), (11.6, 3.78)) - content((8.8, 3.42), text(6.2pt)[`update(ev)` --- one dispatch, mutates]) - content((8.8, 2.92), text(6.2pt)[`render(arena)` #sym.arrow.r `*Surface`]) - content((8.8, 2.42), text(6.2pt)[`emit(e)` #sym.arrow.r fixed ring, `limits.effect_cap`]) - content((8.8, 1.92), text(6.2pt)[`nextEffect()` #sym.arrow.r `perform(e)`]) - content((8.8, 1.32), text(6.2pt)[single-threaded by construction]) - - // ---- native input ---- - rect((0, 3.0), (4.3, 4.3), name: "in") - content((2.15, 3.98), text(7.0pt)[*native input*]) - content((2.15, 3.52), text(6.0pt)[termios+vaxis / SDL / DOM /]) - content((2.15, 3.19), text(6.0pt)[AppKit / a byte sink on a UART]) - line("in.east", (6.0, 3.65), mark: (end: "stealth")) - content((5.15, 3.92), text(6.4pt)[`Event`]) - content((5.15, 3.5), text(5.6pt)[16 arms]) - - // ---- surface out ---- - rect((13.3, 3.0), (18.0, 4.3), name: "paint") - content((15.65, 3.98), text(7.0pt)[*paint the grid*]) - content((15.65, 3.52), text(6.0pt)[vaxis cell-by-cell / glyph atlas /]) - content((15.65, 3.19), text(6.0pt)[DOM cells / CoreText / ANSI]) - line((11.6, 3.65), "paint.west", mark: (end: "stealth")) - content((12.5, 3.92), text(6.4pt)[`Surface`]) - content((12.5, 3.5), text(5.6pt)[cells, cursor,]) - content((12.5, 3.16), text(5.6pt)[images, tracks]) - - // ---- effect out ---- - rect((13.3, 1.1), (18.0, 2.5), name: "perf") - content((15.65, 2.2), text(7.0pt)[*perform*]) - content((15.65, 1.78), text(6.0pt)[fork a pty, write a path, mark a]) - content((15.65, 1.45), text(6.0pt)[directory, put text on a clipboard]) - line((11.6, 1.8), "perf.west", mark: (end: "stealth")) - content((12.45, 2.05), text(6.4pt)[`Effect`]) - content((12.45, 1.62), text(5.6pt)[18 arms]) - - // ---- the vtable ---- - rect((6.0, -1.0), (11.6, 0.4), name: "vt") - content((8.8, 0.14), text(7.0pt)[`Host.VTable` --- optional callbacks]) - content((8.8, -0.32), text(6.0pt)[one host owns each core]) - content((8.8, -0.68), text(6.0pt)[null callbacks use core fallbacks]) - line((8.8, 0.9), (8.8, 0.4), mark: (end: "stealth")) - line("vt.east", (13.3, 1.5), mark: (end: "stealth")) - - // ---- fallback ---- - rect((0, -1.0), (4.3, 0.9), name: "fb") - content((2.15, 0.62), text(7.0pt)[`host_io.Fallback`]) - content((2.15, 0.18), text(6.0pt)[every null method, answered here:]) - content((2.15, -0.18), text(6.0pt)[a virtual filesystem over the]) - content((2.15, -0.52), text(6.0pt)[embedded source, a virtual]) - content((2.15, -0.84), text(6.0pt)[clipboard, silent ptys]) - line("vt.west", "fb.east", mark: (end: "stealth")) - - // ---- answers loop back. Routed through the one corridor that is free of - // every box (x between `fb`/`in` at 4.3 and the core column at 6.0) and - // below the lowest box edge, so the canvas stays inside 0..18 and cannot - // bleed into the page margins. - line((15.65, 1.1), (15.65, -1.6), (5.72, -1.6), (5.72, 3.58), - mark: (end: "stealth"), stroke: (dash: "dashed")) - content((9.0, -1.94), text(5.8pt)[an answer returns as an ordinary `Event`: `paste`, `lsp_resp`, `pipe_resp`, `file_changed`]) - }) - ] -] - -The boundary is three plain data types and one struct of function pointers. - -== `Event`: what comes in - -Sixteen arms (`pardes.Event`), in declaration order: -`key`, `mouse`, `resize`, `output`, `eof`, `lsp_resp`, `pipe_resp`, -`file_changed`, `paste`, `command`, `pdf_scroll`, `pinch`, `touch_scroll`, -`pointer_leave`, `tick`, `fs_req`. - -`key` carries typed `text`. `mouse` carries press/release/motion/drag; button -left, middle, right and the four wheel directions; a cell position; and the -`ctrl` flag, because Ctrl-left-click is goto-definition. `resize` carries cols, -rows, and — in a MuPDF build — the pixel size of one cell. `output` is bytes a -pty produced, tagged with the pane id. `command` is one builtin line arriving -from another process over the nested socket. `pointer_leave` exists because an -out-of-range motion must clamp onto the last grid cell and a pointer that has -left the drawable area must not: otherwise leaving the window previews whatever -is under the final cell. `fs_req` is one filesystem request from a process that -opened a file under the control mount, and its answer leaves as an -`Effect.fs_reply` in the same update. - -Four of the arms are answers to something the core asked for; @effect -names the asks. - -Touch policy belongs to the shell: web maps a one-finger tap to right-button -LOOK and a drag past its tap slop to natural scrolling; native SDL keeps its -two-finger gestures. A joystick cursor is a motion plus buttons. The shells do -this translation and nothing else with input: one event vocabulary, five -dialects translated at the door. - -`postEvent` is a value queue of 64 and asserts what cannot survive the trip -(`Pardes.postEvent`): `output`, `paste`, `lsp_resp`, `pipe_resp`, -`file_changed`, `command` and `fs_req` all carry a borrowed slice, so they are -`unreachable` there and must be handed to `update` directly inside the host's -borrow window. That is also what keeps the pty read path copy-free. - -== `Surface`: what goes out - -`cols`, `rows`, a grid of cells, a cursor, pixel attachments, and a bounded list -of plain panel-transition tracks (`pardes.Surface`). This is the -*canonical interface*: it is literally what the tty shell hands to vaxis, cell -by cell. SDL rasterizes the same grid through a glyph atlas; the browser reads a -packed copy and patches native DOM cells. If it cannot be expressed in the -surface, it does not exist in pardes. - -Pixel images ride along as a list of PLACEMENTS rather than one per pane — a PDF -pane contributes every page its viewport intersects, and pages can be -arbitrarily short. The SDL shells blit RGBA; the tty shell uses kitty graphics -when available and falls back to the petscii matcher. Transition tracks identify -their pane by slot and serial and carry only phase, effect, frame, and from/to -cell boxes. Canonical layout is committed immediately; tracks are finite -presentation data. - -== `Effect`: what is asked for <effect> - -Eighteen arms (`pardes.Effect`). The core never performs IO for any of -them; it asks. - -```zig -pub const Effect = union(enum) { - spawn: struct { pane: u8, cwd: Buf(256) }, - write: struct { pane: u8, bytes: Buf(64) }, - resize_pty: struct { - pane: u8, cols: u16, rows: u16 }, - open_link: Buf(256), - save_file: struct { pane: u8 }, - save_text: struct { - pane: u8, serial: u32, path: Buf(256) }, - write_dump, - set_clipboard, - read_clipboard, - lsp: struct { id: u32, kind: lsp.Kind, - pane: u8, offset: u32, arg: Buf(128) }, - pipe: struct { id: u32 }, - watch: struct { pane: u8, on: bool }, - theme_file: struct { - generation: u32, on: bool }, - dump_themes: struct { pane: u8 }, - fs_reply: filesystem.Reply, - attach: struct { - pane: u8, name: Buf(attach_name_max) }, - detach: struct { pane: u8 }, - quit, -}; -// doc comments elided; see the file -``` - -Every arm is a fixed-size value, and that is the constraint the widths record. -Unbounded content is never in the union: `save_file` names a pane and the shell -reads the bytes off the core, and `save_text` carries the *path* — bounded -exactly like a spawn's cwd, so two saves armed in one batch cannot cross — while -the bytes are read at drain time. `save_text` also carries the pane's `serial`, -so a slot freed and reused before the drain writes nothing rather than another -pane's text to that path. `attach_name_max` is 256 for the same reason `spawn`'s -cwd is, and reusing that width is why the arm costs the ring nothing: -`save_text` is still the widest member (`pardes.attach_name_max`). - -Four asks have an answer coming back. `read_clipboard` returns an ordinary -`paste` event — or nothing at all when the shell cannot read the clipboard, which -is most terminals, since they refuse the OSC 52 read. `lsp` returns `lsp_resp`. -`pipe` returns `pipe_resp`. `watch` returns `file_changed` whenever the shell -notices a text file or PDF moved under it. - -`theme_file` carries only a generation because the path lives in the core's fixed -request buffer, which keeps an already-large ring compact. `fs_reply` carries no -bytes either: `payload` says where they live — a staging buffer in the core, or a -range of a pane's live text — and `fsPayload` resolves it during the drain, so a -megabyte read costs one `writev` and no copy. - -`attach` and `detach` are the two arms no host method performs directly. `attach` -must not tear anything down (@swap); `detach` is served only by a -detached core's host, and the null case is the point rather than an oversight — -a local tty or SDL shell has no session to leave, so `perform` reports that on -the pane's message row instead of quietly quitting something (the `.detach` arm of -`Pardes.perform`). - -The watch path, in full, because it is the ask with the most machinery behind -it. The tty and SDL hosts and the detached daemon implement it in -`src/file_watch.zig`, with Linux inotify or a macOS kqueue behind one -mark/reconcile transaction; the native macOS host watches with debounced -DispatchSources instead, then restats the exact path under a pane-generation -guard. All three macOS watchers mark the same two things for the same reason: -the file catches in-place writes while the parent directory follows rename-over -saves, and the file mark is re-armed once such a save has moved the inode. -Text snapshots are filtered by their content hash; -PDFs use bounded inode/size/time identity and commit it only when equal stats -bracket a successful transactional MuPDF reopen (`file_watch.Generation`). A -mismatched transaction gets one bounded self-retry. This catches rename-over -saves without reading a large PDF merely to notice it changed, or spinning on a -malformed one. A PDF response retains its reading position and pane settings. -Web has no filesystem watcher, and nothing in the core waits for one. - -== The effect ring - -```zig -pub fn emit(p: *Pardes, e: Effect) void { - if (p.effects_len == p.effects.len) return; - const tail = (p.effects_head + - p.effects_len) % p.effects.len; - p.effects[tail] = e; - p.effects_len += 1; -} - -pub fn nextEffect(p: *Pardes) ?Effect { - if (p.effects_len == 0) { - p.effects_head = 0; - return null; - } - const e = p.effects[p.effects_head]; - p.effects_head = - (p.effects_head + 1) % p.effects.len; - p.effects_len -= 1; - return e; -} -``` - -`emit` REFUSES when full and never evicts (`Pardes.emit`). Byte -order is the reason: a `.write` dropped from the middle of a run would reorder a -pty's input, and a `.write` dropped from the tail merely truncates it. - -Capacity is 4096 on every hosted platform and 128 on the board -(`limits.effect_cap`). No capacity can deadlock the drain, because `pump` empties -the ring on every iteration with an unconditional `while (nextEffect())` — -including effects `perform` itself queues — so the only question a capacity -answers is how large a single-pump *burst* may be. The one producer that can -burst is `emitWrite`, which chunks arbitrary bytes into 64-byte `.write` effects -for a pty; everything else queues O(1) effects per event, and the input queue -holds at most 64 events per pump, so 128 leaves two effects per queued event. A -build with no terminal panes has no pty to write to at all. - -== `Host.VTable`: who serves the core - -`host_io.Host` holds a context pointer and optional callbacks. macOS enters -its native shell through the separate `Runtime` C ABI. - -```zig -pub const VTable = struct { - /// The ONLY place the process may sleep. - wait_input: ?*const fn ( - ctx: ?*anyopaque, - timeout_ms: u32, - ) void = null, - present: ?*const fn ( - ctx: ?*anyopaque, - surface: *const pardes.Surface, - ) void = null, - // ... - tty_taken: ?*const fn ( - ctx: ?*anyopaque, - pane: u8, - ) bool = null, - // ... -}; -``` - -A null method is not an error: the core substitutes a default backed by ordinary -data structures in the same process (`host_io.Fallback`). A `Save` lands in a real -file under the tty host and in `Fallback.files` under a host that never wrote a -filesystem method, and every path above that behaves identically. Two -consequences were taken on purpose. The zero-method host IS the test harness: a -`Host{}` is a complete, deterministic, in-process pardes with a virtual -filesystem, a virtual clipboard and silent ptys. And `Fallback` lives on the -`Pardes` instance rather than on the host, so cores keep independent state. - -The fallback filesystem is not empty. It is pardes's own source, embedded -(`src/fs.zig`), with `files` holding only what this session WROTE, -so a Save shadows the built-in copy and reading it back returns the edit. That is -what makes a host with no file methods a usable pardes rather than one staring -at an empty buffer. - -What is deliberately NOT in the vtable: whether a capability EXISTS in this -build. That stays comptime and stays next to the code it shapes -(`pardes.platform`, `pardes.hosted`, `pardes.pdf_enabled`, -`pardes.can_attach`, `pardes.terminal_panes`, `builtins.capabilities`, -`PdfSlot`). A vtable cannot make a field zero-sized or a builtin absent from an -enum. Comptime decides what a build HAS; the vtable decides who SERVES it at -runtime. - -`pardes.can_attach` is the sharpest of those, because it exists to close a gap -the coarser gate left open. `Attach` and `Detach` were gated on `pardes.hosted`, -and macOS is hosted: it has a unix socket and it compiles `detached/`. What it -does not do is POLL. `takeAttach` has to be a poll rather than a host method -precisely because attaching replaces the core the call is running inside -(@swap), and `src/macos.zig` never calls it — so there the word -parsed, queued an effect, stored a request in `attach_buf`, and then did nothing -at all, for ever, silently. `can_attach` admits exactly the tty and gui -platforms, which is exactly the pair `main.zig` accepts `--attach` for: the same -question asked at the command line instead of in a tag. A capability that a -build cannot serve should not be a word that build offers. - -=== Callback ownership - -Each core has one host. The detached host explicitly broadcasts shared state -and routes clipboard reads and link opening to the originating frontend. -There is no name-based fan-out layer. - -== The frame, as a frontend writes it - -```zig -pub fn pump(p: *Pardes, h: Host) !void { - p.host = h; - const v = h.vtable; - if (v.wait_input) |f| - f(h.ctx, if (p.animationActive()) - animation.frame_ms else 0); - while (p.nextQueued()) |ev| p.update(ev); - while (p.nextEffect()) |e| p.perform(e); - // A quitting frame has already freed - // what it would draw. - if (p.quit) return; - if (v.poll_frame) |f| f(h.ctx); - _ = p.frame_arena.reset(.retain_capacity); - const surface = try p.render( - p.frame_arena.allocator()); - if (v.present) |f| f(h.ctx, surface); - if (v.post_present) |f| f(h.ctx); - // ... -} -``` - -`Pardes.pump`, eighteen lines in the source, and the order of them is the -architecture. Input first, in whichever host owns the sleep; -then every queued event, to completion; then every effect, to completion, -including effects `perform` queued; then one arena reset and one render; then -present, then post-present. - -Animation time is not spent in here. `wait_input` was told how long it may -sleep, and a display clock wakes faster than that on input, so only the host -knows when a real frame interval has passed. Each spends it by handing back one -`.tick`. - -`post_present` is split from `present` because it must observe a frame -the user has actually seen: panel-presentation acknowledgement and pointer -refresh both depend on that, and a hook that ran before the pixels landed would -acknowledge a frame that was never shown. - -== Platform divergence is comptime - -There is one deliberately platform-divergent file by design: `look.zig` holds -path and `:line` resolution, URL detection, and the per-platform outcomes. The -divergence is a comptime switch on `pardes.platform`, used the way the stdlib -switches on `os.tag`, so every platform's behaviour sits in the same screenful. - -```zig -const platform_has_fs = !pardes.isolated and - switch (pardes.platform) { - .tty, .gui, .macos => true, - // The browser's filesystem is the embedded - // source archive; the P4 firmware's is - // whatever the serial host answers for, - // through the Host vtable — never a path - // this process opens. - .web, .esp32p4 => false, - }; -``` - -`look.platform_has_fs`. An isolated build has no filesystem *by construction* — -the option is comptime, so every libc path below is dead code the compiler -removes rather than a branch that could be taken by accident. - -The core's one `pointerOperand` primitive (in `exec.zig`) owns click-word expansion and is shared -verbatim by right-click and the delayed hover preview; that policy stays beside -input because it also observes live pane selections and wrapped grid coordinates. - -Each pane kind keeps its storage and operations together in its own file -(`File.zig`, `terminal.zig`, `Output.zig`, `mini.zig`, `pdf_view.zig`, and the -image pane in `image.zig`), reached through `panes.zig` beside `Pane`. -`layout.zig` owns placement and presentation state. `pardes.zig` handles input -and cross-pane state directly, without a pane vtable; the rest of the editor -is split by thing as acme is: `Text.zig`, `edit.zig`, `normal.zig`, `tagline.zig`, -`look.zig`, `exec.zig`, `mouse.zig`, `Messages.zig`, `Pipe.zig`, `colors.zig`, -`surface.zig`, `body_layer.zig`, `Layer.zig` and `dump.zig`. Integration fixtures live -in `test/panes.zig`, `test/output.zig`, and `test/pdf.zig`. - -= State - -== One struct, fixed where it can be - -The whole state is one struct. Panes themselves are heap-allocated on demand and -their contents (file bytes, the yank register, PDF rasters, tree-sitter state) -grow with what you open; everything else is sized at init. - -```zig -Pardes - ncol + col_weight[6], col_panes[6][16], col_n[6] - panes: [16]?*Pane - active + drag + config.Runtime - rects: [16]layout.Rect - presentation: layout.Presentation - effects: [limits.effect_cap]Effect + head/len - in_q: [64]Event + head/len - fallback: host_io.Fallback - fs: fs.Namespace - -Pane - serial + mode + vweight - terminal: ?*panes.Terminal.State - file: ?panes.File.State - image: ?panes.Image.State - pdf: PdfSlot - cwd: none | inherited(*Pane) | owned([]u8) - tag_tail + prompt + cursor + selections - ovl: ?panes.Terminal.EditBuffer - -Terminal.State = VT + stream + replay + reply -File.State = path + bytes + line index + syntax + undo -Image.State = decoded pixels + render cache -Pdf.State = document + layout + raster cache + search -``` - -`MAX_PANES` is 16 and `MAX_COLS` is 6 (`pardes.MAX_PANES`, `pardes.MAX_COLS`). Sixteen panes -is also why the jump stack's depth is what it is: the depth that matters is -"more visits than you can hold in your head", not vim's hundred. - -== Cells - -```zig -pub const CellStyle = struct { - fg: Color = .default, - bg: Color = .default, - bold: bool = false, - dim: bool = false, - italic: bool = false, - blink: bool = false, - reverse: bool = false, - invisible: bool = false, - strikethrough: bool = false, - ul: enum { off, single, double, - curly, dotted, dashed } = .off, - font_role: FontRole = .body, -}; - -/// One surface cell. `default = true` means -/// "never painted this frame": the shell renders -/// it as the terminal's default cell (vaxis clear -/// semantics). -pub const Cell = struct { - text: [7]u8 = @splat(' '), - len: u8 = 1, - style: CellStyle = .{}, - default: bool = true, - - pub fn grapheme(c: *const Cell) []const u8 { - return c.text[0..c.len]; - } - // ... -}; -``` - -`pardes.CellStyle` and `pardes.Cell`. Seven bytes of `text` because a cell holds a -complete grapheme cluster and not a codepoint: keeping the cluster is what makes -combining marks visible and keeps ZWJ, modifier, flag and Indic sequences in the -same screen cell that cursor and edit maths treat as one unit. `len` bounds it, -and `default` distinguishes "a space was painted here" from "nothing was". - -That distinction is load-bearing twice over. `visuallyEqual` compares only what -a shell can present — bytes past `len` are scratch left by earlier graphemes and -must never manufacture a diff, and an unpainted default cell has no visible -style or text either. And `printableAscii` returns `' '` for a default cell but -`null` for a painted multi-byte one, which is what admits a cell to the ASCII -transition walk (@diff). - -== Columns and panes are arithmetic - -Layout is arithmetic, not objects: columns are weights over the width, panes are -weights over the column. - -#figure( - placement: auto, - caption: [The layout model. Each pane is a narrow gutter strip (move box and - scrollbar) plus a tag row plus a body. `col_weight` is 64-bit fixed point with - 32 fractional bits, so every possible column split divides an initial weight - exactly; `vweight` is an `f32` over its own column. `col_terms[c][k]` is the - pane in slot position `k` of column `c`, and `rects[id]` is where that pane - landed this frame.], - diagram[ - #cetz.canvas(length: 1cm, { - import cetz.draw: * - set-style(stroke: 0.35pt, mark: (fill: black, scale: 0.3)) - let w = 7.6 - let h = 4.5 - - rect((0, 0), (w, h)) - line((0, h - 0.28), (w, h - 0.28)) - content((w / 2, h - 0.14), text(4.9pt, style: "italic")[topbar, execute-only]) - - line((2.8, 0), (2.8, h - 0.28)) - line((5.2, 0), (5.2, h - 0.28)) - line((2.8, 1.8), (5.2, 1.8)) - - let pane(x0, y0, x1, y1, kind) = { - line((x0 + 0.26, y0), (x0 + 0.26, y1 - 0.26)) - line((x0, y1 - 0.26), (x1, y1 - 0.26)) - content(((x0 + x1) / 2, y1 - 0.13), text(4.9pt, style: "italic")[tag: #kind]) - } - pane(0, 0, 2.8, h - 0.28, [file]) - pane(2.8, 1.8, 5.2, h - 0.28, [terminal]) - pane(2.8, 0, 5.2, 1.8, [output]) - pane(5.2, 0, w, h - 0.28, [PDF]) - content((1.45, 2.3), text(4.9pt, style: "italic")[body]) - content((6.4, 2.0), text(4.9pt, style: "italic")[body]) - - line((0.04, -0.3), (2.76, -0.3), mark: (start: "stealth", end: "stealth")) - content((1.4, -0.58), text(5.4pt)[`col_weight[0]`]) - line((2.84, -0.3), (5.16, -0.3), mark: (start: "stealth", end: "stealth")) - content((4.0, -0.58), text(5.4pt)[`col_weight[1]`]) - line((5.24, -0.3), (w - 0.04, -0.3), mark: (start: "stealth", end: "stealth")) - content((6.4, -0.58), text(5.4pt)[`col_weight[2]`]) - - line((5.0, 1.86), (5.0, h - 0.34), stroke: (dash: "dotted"), - mark: (start: "stealth", end: "stealth")) - content((4.2, 3.0), text(5.2pt)[`vweight`]) - line((5.0, 0.06), (5.0, 1.74), stroke: (dash: "dotted"), - mark: (start: "stealth", end: "stealth")) - content((4.2, 0.9), text(5.2pt)[`vweight`]) - }) - ], -) - -Horizontal layout weights are fixed-point integers and geometry rounds -cumulative boundaries. Splitting a column replaces only its weight $W$ by -$A + B = W$ at the same position. Therefore every boundary outside the source -column is bit-identical before and after the split, including at awkward -non-dyadic screen widths; only the source and new column can receive movement -tracks. Vertical splits apply the corresponding rule to the source pane's -weight. - -`splitBelow` shrinks only the source pane; `absorbVWeight` gives a dying pane's -weight to one sibling; an emptied column hands its width to a neighbour. -Minimal motion is the invariant: an operation on one pane may not move panes it -does not touch. - -== Runtime settings are one table - -User-settable runtime choices are one plain `config.Runtime`: booleans, -theme index, owned bounded shell/font strings, requested/effective font facts, -one panel-transition enum, and scene-effect booleans -(`config.Runtime`). A compile-time `settings` array -(`config.Runtime.settings`) generates each setting builtin and the rows of the -single `Config` query. It has no callbacks and no parallel query registry to -drift from it. - -Requested and effective are separate fields on purpose. Resolving a shell name -to an executable, or a font name to a face, belongs to the native host; the core -retains what the host actually chose and marks a changed request `pending` until -the next spawn or the next atlas acknowledges it. A rejected request therefore -stays queryable without claiming it is on screen. - -= Modes and selections - -== Three modes and a boolean - -`Pane.mode` is `enum { normal, insert, tty }` (`pardes.Mode`). - -*normal* is the helix motion model. *insert* is click-and-type: terminals get -splice runs (shift right, never overwrite, anchored to an absolute row), files -get real edits. *tty* is raw pty forwarding — a Ctrl-key toggle, default Ctrl-b, -`--tty-toggle` — where the mouse is still usable, entry does a -`promptClickMove`, and prompts become visible again (they are hidden in the -other two modes via OSC 133). - -Select is not a fourth mode. `v` sets `Pane.select`, a bool on top of normal, -which makes motions extend from a fixed anchor instead of replacing the range; -`pane.mode` stays `.normal`, and insert/tty transitions drop it -(`Pane.select`). It is *displayed* as a fourth mode name, "select", -because that is what the user is in — but nothing in the dispatch branches on a -fourth mode. - -Since the helix motion model landed, a traversal motion SELECTS the range it -crossed. That is why there is no verb+noun grammar and why `i` after `w` types -at the selection's start. - -== One primary range and up to 63 more - -`MAX_SELS` is 64 (`pardes.MAX_SELS`). helix's `Selection` is a list of -ranges plus a primary index; pardes keeps the PRIMARY exactly where it has -always been — `cur_row`/`cur_col` plus `vsel` — and the other ranges in -`sels: [MAX_SELS - 1]SelRange`, document-ordered and disjoint -(`Pane.sels`, `Pane.nsel`). - -That split is the whole design. Every motion, operator, renderer and mouse path -still reads one selection, so with `nsel == 0` not a byte of behaviour moves, and -the differential and snapshot suites keep proving it. The extra ranges are driven -by replaying the single-selection key handler once per range (`replaySels`). - -`s`/`S` — select and split by regex — snapshot the selection they were armed on -in `sel_snap` and re-derive the preview FROM that snapshot on every keystroke, -rather than from the previous preview. This is what helix's `regex_prompt` does, -it is what makes typing a pattern one character at a time land on the same answer -as pasting it whole, and it is what makes Esc a plain restore with nothing else -to undo. The snapshot also records whether the range was a *user-intent* -selection, so restoring cannot silently promote motion residue into something -the acme chords will act on (`Pane.sel_snap`). - -== Three buttons, three meanings - -The mouse selections are `[3]Sel`, one per button, block-shaped, in text-area -coordinates where `r` counts from the tag row (`pardes.Sel`). Each -has three states, `none`, `dragging` and `done`, which is what lets a kept left -selection stay highlighted after the drag and still be distinguishable from one -in progress. - -Left selects and pins the cursor. Middle executes: with no drag it expands to a -file-ish word, then runs a builtin or sends the text to the shell, and it does -not focus. Right looks. A theme owns the SELECTION — `sel_bg`/`sel_fg` are one -pair per theme, and the three per-button tints and the dimmed extra cursors are -mixed off it, so what stays fixed is the distinction between buttons and not the -colours. - -`Drag` is a `union(enum)` with six arms (`mouse.Drag`): `none`, -`border_v`, `border_h`, `move`, `column_move`, `select`. `border_v` carries an optional -`corner`: when the press lands on a cell that is both a column's vertical border -and one of the two adjoining columns' own horizontal borders, the one drag moves -*both* boundaries — never three, and when both columns happen to be split at the -grabbed row the left one wins, so the gesture that existed before is bit-for-bit -unchanged. - -== The tag is a text - -A tag is a `Text` (`Text.zig`), as a body is: acme's `Text`, one per -`what` — the body, the pane's tag, the prompt line, a column's tag and the -workspace tag. Each holds its own cursor, selections, mode and undo, so tags -are edited by the same `edit.zig` and `normal.zig` code as bodies, and the one -key they do not share is `:`, which moves between a pane's body and its tag. -A tag holds any number of lines; a pane's tag takes a row per line up to -`MAX_TAG_ROWS`, each drawn as a tag layer of its own. - -A pane's tag shows a live prefix (path, dirty marker, PDF page) before the text -it owns, and that prefix is computed at every read, never stored -(`tagline.tagPrefix`). Everything sees the whole line as shown, prefix ++ -text: render, the mouse, Look, Exec, the 9P `tag` file and the keyboard, -whose motions reach the path as acme's do. The prefix is read-only: an edit -installs only what follows it, and one that would change it is refused -(`edit.setEditText`, `Text.refused`). Because the prefix is computed, sync -moves the tag's positions with it when its length changes (`Pane.tag_lead`). -Editing the path is a separate draft (`Pane.prompt = .name`) committed by -Enter, since a buffer's name is not text it owns; a click or a key typed -into it starts one. - -The tag's text is bounded by `limits.max_tag_tail`: the 9P `tag` file and a -dump reader refuse input that does not fit rather than truncating it. - -= Diffing and presenting a frame - -== Eleven transitions - -`layout.zig` is backend-neutral data and math: the transition -vocabulary, easing, exact endpoint progress, stable per-cell noise, and a POD -track. `Transition` has twelve members counting `off` -(`layout.Transition`), with explicit numeric values because they -cross both GUI shader ABIs — GLSL receives the enum in an instance `uvec4` and -the Core Image kernel receives it as a float, so spelling the numbers keeps a -source reorder from changing pixels. - -#table( - columns: (auto, auto, auto, 1fr), - align: (left, left, left, left), - table.header([*id*], [*name*], [*frames*], [*composed by*]), - [1], [`slide`], [12], [backend shader / tty grid], - [2], [`zoom`], [14], [backend shader / tty grid], - [3], [`dissolve`], [10], [backend shader / tty grid], - [4], [`ascii`], [13], [core], - [5], [`vertical`], [12], [backend, lifecycle only], - [6], [`edges`], [12], [core], - [7], [`fall`], [14], [core], - [8], [`wave`], [14], [core], - [9], [`curtain`], [12], [core], - [10], [`scramble`], [12], [core], - [11], [`typewriter`], [14], [core], -) - -The split in the last column is the design. `composedByCore` is true for every -*character* effect: the core writes the finished glyphs into the published -`Surface`, so no backend owns a byte walk, a stagger, or a noise threshold, and -every renderer presents the same byte at a given frame. The geometry effects -(`slide`, `zoom`, `vertical`) and `dissolve` are the ones a backend evaluates, -because they are transforms over rectangles rather than choices about characters. - -Easing follows from that too. A sweep and a typewriter are constant-rate by -definition, so `curtain` and `typewriter` are `linear`: easing their head would -make the pass visibly hesitate mid-pane. Character walks and per-cell locks read -best with a slow start, a fast middle and a slow settle, so `ascii`, `fall` and -`scramble` are `smoother`. - -The core detects opening and moving rectangles when it commits layout and -publishes only active tracks. A separate dense closing-track list is -presentation-only state for a pane whose functional lifetime has already ended, -which is why it needs no pane owner and no serial guard. Pointer input inverts -the presented slide/zoom/vertical rectangle back to the canonical grid, lets -unchanged dissolve and ASCII cells through immediately, and rejects closing -pixels, so pixels and gestures cannot disagree during a transition. - -== The semantic cell diff <diff> - -```zig -pub const PanelCellDiff = union(enum) { - unchanged, - visual, - ascii: AsciiDiff, - - pub fn between( - old: *const Cell, - new: *const Cell, - ) PanelCellDiff { - if (old.visuallyEqual(new)) return .unchanged; - if (AsciiDiff.between(old, new)) |diff| - return .{ .ascii = diff }; - return .visual; - } - - // ... -}; -``` - -`pardes.PanelCellDiff`; the elided member is `changed`, which is -`diff != .unchanged` and is what `Surface.panelCellChanged` calls. The core -retains the last successfully presented canonical grid and this typed old/new -classification, and it is the *semantics* that matter: unused grapheme bytes do -not manufacture a change, and a style-only or multi-byte change is `.visual` and -passes straight through. - -An `.ascii` diff is a `{ from: u8, to: u8 }` pair, and the core composes it by -incrementing or decrementing the printable byte. Short walks move one value per -frame; a longer walk is crossed by eased character skips and finishes within -`ascii_max_movement_frames`, which is 12 -(`layout.ascii_max_movement_frames`). Frame -zero is the exact old byte and the endpoint is exact, so an intermediate frame is -always valid UTF-8. `Track.frame_count` carries the core-computed duration for -these data-dependent effects — zero selects the effect preset, and the ASCII -composer fills it from the longest eased byte walk in the pane's diff. - -== What each backend does with it - -TTY copies that `Surface` grid into a compositor scratch grid, clears slide and -zoom destinations, then paints moving, opening, and closing panels in order. -Cleared geometry uses the theme page colour when it is explicit and the host -terminal default only for transparent themes, so a light theme cannot flash a -dark gap. Slide and zoom change the copied rectangle; dissolve changes only diff -cells from their old value to their new value; vertical raises only an opening or -frozen closing pane inside its own clip. - -Every effect's last active sample is its exact canonical endpoint, so the -compositor returns the source surface unchanged when no track is -non-canonical — keeping the real cursor and attachments in that sample rather -than suppressing them for one frame that is otherwise pixel-identical -(`panel_compositor.compose`). Kitty placements cannot be resampled -through the character-grid transform, so a moving pane's attachment is omitted -while its geometry moves and placed again on the last active sample. - -SDL supplies old/new glyph data, diff flags, final/presented boxes, and effect -parameters to the glyph and native-image shaders. Pixel attachments bypass ASCII -because they have no character byte. macOS passes the same records across its -plain C ABI and composites old/new panel images in Metal and Core Image. The -scene `Crt` is one full-window pass in each native GUI; the SDL GUI's post -chain also runs Shadertoy files (`Shader`). - -DOM web is a separate platform, not a shader GUI: retaining selectable HTML and -CSS is more important than duplicating the renderer in canvas, so it exposes -neither effect family. - -`EffectCode <effect>` lists the current backend's build-embedded source paths -under `/virtual`. Look opens each full file without a source checkout. -Shared implementations share paths. - -== The ASCII fast paths - -Three loops in this codebase run once per character over text that is almost -always ASCII, and each asks two or three general functions for what arithmetic -already knows. The guard that makes elision *correct* rather than merely fast is -the same in all three, and it is a statement about Unicode: an ASCII base joins a -following combining mark, ZWJ or spacing mark into ONE cluster, and every scalar -that can do that is non-ASCII. So a printable ASCII byte followed by another -ASCII byte, or by nothing, is a complete grapheme cluster one column wide. - -```zig -// ASCII FAST PATH. Printable ASCII is one byte, -// one cell, one column, and the general path -// below reaches that answer through a UTF-8 -// length, a decode, a freshly constructed -// grapheme iterator, a slice validation and a -// width lookup - per character. -{ - const b = text[i]; - if (b >= 0x20 and b < 0x7f and - (i + 1 == text.len or text[i + 1] < 0x80)) - { - s.set(col, y, text[i .. i + 1], style); - i += 1; - col += 1; - continue; - } -} -``` - -`Surface.print`. Its own comment records the reason -it exists: this function was 26% of a keystroke when profiled in the ESP32-P4's -configuration — 40×12, no tree-sitter — which was the largest single item there. -`\t`, `\r`, the C0 controls and DEL are excluded by the range test and keep their -existing handling. - -The second is `panes.File.fitEnd` (`panes.File.fitEnd`), which decides -where a soft-wrapped row breaks. It asks `modal.nextGrapheme` and -`graphemeDisplayWidth` once per character, and a 640-column line asks 640 times. - -The third is `modal.nextGrapheme` itself -(`modal.nextGrapheme`), and it is where the argument above needs its one -correction. GB3 is the single UAX #29 rule that joins two ASCII scalars: a CR -takes a following LF into the same cluster. The two range-tested loops never see -it, because `0x20..0x7e` excludes CR — but `nextGrapheme`'s fast path admits -every byte below `0x80`, so it excludes CRLF by name. It did not always, and -`graphemeStart` did: the two then disagreed about a CRLF file by exactly one -byte, a head stepped onto the offset between CR and LF, and `graphemeStart` -repaired it back onto the CR. Everything else ASCII is still O(1). - -Because `fitEnd` decides where text lands on screen, a fast path that is off by -one column *moves text*. Both of its tests therefore pin it to the general walk -it replaces rather than to a transcribed expectation: one sweeps every byte -below 0x80 against a set of neighbours at every width and start offset -(*the ASCII run in fitEnd survives an exhaustive byte sweep*), and the other runs a hand-written case list — -including `"abc\u{00e9}def"` for a hand-over mid-run, a wide glyph a width -boundary can land inside, `"a\u{0301}bc"` for a cluster the fast path must not -split, and `"e\u{0301}x"` for an ASCII byte followed by a continuation byte — -through both routes (*the ASCII run in fitEnd cuts where the grapheme walk -would*). `Surface.print` has the same -arrangement: its reference implementation is `print` with the fast-path block -deleted and nothing else changed (the test *the ASCII fast path in surface -print paints what the general arm paints*). - -= Detached sessions <detached> - -#wide(caption: [A detached session. The daemon owns the core and every -machine-local resource; a frontend owns a screen, a keyboard and a clipboard. -Counts and tag numbers from `wire.ClientTag` and `wire.ServerTag`; the routing rules -from `src/detached/server.zig:34-59`.])[ - #diagram[ - #cetz.canvas(length: 1cm, { - import cetz.draw: * - set-style(stroke: 0.4pt, mark: (fill: black, scale: 0.35)) - - // ---- the daemon ---- - rect((0, 2.0), (6.4, 5.5), name: "d") - content((3.2, 5.24), text(7.4pt)[*the daemon* --- `pardes --detach[=name]`]) - line((0, 5.02), (6.4, 5.02)) - content((3.2, 4.74), text(6.0pt)[one `Pardes`; `update` is called here]) - content((3.2, 4.38), text(6.0pt)[every pty master (`host_io.forkShell`)]) - content((3.2, 4.02), text(6.0pt)[every file (`host_io.writeFileBytes`)]) - content((3.2, 3.66), text(6.0pt)[the inotify fd (`file_watch.zig`)]) - content((3.2, 3.30), text(6.0pt)[the theme dir and the acme mount]) - content((3.2, 2.94), text(6.0pt)[16 of 21 `VTable` methods implemented]) - content((3.2, 2.58), text(6.0pt)[ONE `poll(2)` --- the only sleep]) - content((3.2, 2.26), text(5.6pt, style: "italic")[pane shells outlive every frontend]) - - // ---- the socket, drawn in segments so the labels sit in the gaps ---- - line((9.3, 2.0), (9.3, 3.02), stroke: (dash: "dashed")) - line((9.3, 3.46), (9.3, 4.72), stroke: (dash: "dashed")) - line((9.3, 5.16), (9.3, 5.6), stroke: (dash: "dashed")) - content((9.3, 5.78), text(6.2pt)[`AF_UNIX`]) - - // ---- the two directions ---- - line((12.2, 4.7), (6.4, 4.7), mark: (end: "stealth")) - content((9.3, 4.94), text(6.2pt)[`ClientTag` --- 11 tags]) - line((6.4, 3.0), (12.2, 3.0), mark: (end: "stealth")) - content((9.3, 3.24), text(6.2pt)[`ServerTag` --- 8 tags]) - content((9.3, 2.48), text(5.6pt, style: "italic")[a frame per pump, per client;]) - content((9.3, 2.18), text(5.6pt, style: "italic")[never queued, only skipped]) - - // ---- frontends ---- - rect((12.2, 4.35), (18.0, 5.5)) - content((15.1, 5.24), text(6.8pt)[*frontend* --- a terminal]) - content((15.1, 4.86), text(5.8pt)[`--attach`; vaxis paints `grid`/`cursor`]) - content((15.1, 4.52), text(5.8pt)[screen + keyboard + clipboard + a link]) - - rect((12.2, 3.1), (18.0, 4.15)) - content((15.1, 3.90), text(6.8pt)[*frontend* --- an SDL window]) - content((15.1, 3.52), text(5.8pt)[same protocol, same session, same screen]) - content((15.1, 3.22), text(5.8pt)[`screen -x`, not N sessions]) - - rect((12.2, 1.85), (18.0, 2.9)) - content((15.1, 2.68), text(6.8pt)[#sym.dots.v up to `max_clients` = 32]) - content((15.1, 2.32), text(5.8pt)[the 33rd gets a `refuse .full`, not a backlog]) - content((15.1, 2.02), text(5.6pt, style: "italic")[NO FORK HAPPENS IN A FRONTEND]) - - // ---- the message tables, full width ---- - rect((0, -2.8), (18.0, 1.5)) - content((9.0, 1.24), text(7.0pt)[*every message, and which way it crosses*]) - line((0, 1.02), (18.0, 1.02)) - let row(y, body) = content((0.3, y), body, anchor: "west") - row(0.72, text(5.6pt)[frontend #sym.arrow.r core (11): `hello` `bye` `key` `mouse` `resize` `paste` `command` `pdf_scroll` `pinch` `touch_scroll` `pointer_leave`]) - row(0.32, text(5.6pt)[core #sym.arrow.r frontend (8): `welcome` `refuse` `frame` `quit` `detach` | `set_clipboard` `read_clipboard` `open_link`]) - row(-0.16, text(5.6pt)[BROADCAST --- every screen must show the same thing: `frame`, `set_clipboard`]) - row(-0.54, text(5.6pt)[ORIGIN ELSE PRIMARY --- the answer belongs to the human who acted: `read_clipboard`, `open_link`, `detach`]) - row(-0.92, text(5.6pt)[`read_clipboard`'s answer is not a reply message: it comes back as an ordinary `Event.paste`]) - row(-1.40, text(5.6pt)[the gaps `0x13`..`0x17`, `0x1e` were `output` `eof` `lsp_resp` `pipe_resp` `file_changed` `tick`: deleted, not renumbered,]) - row(-1.78, text(5.6pt)[when the daemon took the disk --- a decodable `output` let an attached peer forge a pane's text]) - row(-2.16, text(5.6pt)[never on the wire: `wait_input` (it IS the poll loop), the two informationless frame pushes, the four `pull_`s]) - row(-2.56, text(5.6pt)[that answer their own caller, and `fs_reply` --- with N frontends, N#sym.minus 1 would get an answer they never asked for]) - }) - ] -] - -== The daemon owns the machine - -The `Pardes` instance lives in the detached process. A frontend owns a terminal -(or a window, or a serial panel) and a socket, and nothing else: it sends the -input it collects and draws the frames it is sent. One core per session, N -frontends attached to it, all looking at the same screen — `screen -x`, not N -sessions. - -The argument for putting every machine-local effect in the daemon is one -sentence: a unix socket means the core and its frontends are on the SAME machine, -so there is no question of whose disk or whose process table is meant, and given -that, the process that must hold them is the long-lived one. A shell forked by a -frontend dies with that frontend, and a session whose whole promise is outliving -the frontend attached to it cannot keep its panes that way. So the daemon forks -the pane shells, writes the files, marks the directories, and drains the pty -masters in its own `poll(2)`. The pane shells outlive every frontend: attach, -detach, kill the terminal, attach from another one, and the build that was -running in pane 3 is still running and has been scrolling into the core the whole -time. - -The detached core also serves the default 9P socket. Its event loop drains -filesystem transactions alongside frontend and terminal input. - -Nothing blocks indefinitely, and that property is what a detached session is -*for*. Every descriptor is non-blocking; the single `poll(2)` is the only place -the process sleeps; and every queue that could grow without bound has a ceiling -with a stated answer for reaching it. Frames in particular are NOT queued: a -client with bytes still owed to the kernel is skipped for this frame and its -mirror is left alone, so the next frame it does get is a diff against what it -actually has. A slow frontend therefore sees fewer, larger frames instead of a -growing queue, and coalescing costs no byte surgery. What is left in a client's -out-queue is control messages, capped by `out_backlog` and checked *before* an -append, so a single oversized message still goes out whole and what gets refused -is a client that has stopped draining: it is closed, and its peers are untouched. - -== The wire - -```zig -pub const ClientTag = enum(u8) { - hello = 0x01, - bye = 0x02, - - key = 0x10, - mouse = 0x11, - resize = 0x12, - paste = 0x18, - command = 0x19, - pdf_scroll = 0x1a, - pinch = 0x1b, - touch_scroll = 0x1c, - pointer_leave = 0x1d, -}; - -pub const ServerTag = enum(u8) { - welcome = 0x01, - refuse = 0x02, - frame = 0x03, - quit = 0x04, - detach = 0x05, - - set_clipboard = 0x10, - read_clipboard = 0x11, - open_link = 0x12, -}; -``` - -`wire.ClientTag` and `wire.ServerTag`. Eleven tags in, eight out, and the shape of both -lists is the argument. - -Every `ClientTag` is something a human did: a handshake, a goodbye, and what a -keyboard, a mouse, a trackpad or a window manager produces. Six numbers are -missing from the input run — `0x13`..`0x17` and `0x1e` — and the gaps are left -rather than tidied away, because renumbering is a change every deployed frontend -feels. They were `output`, `eof`, `lsp_resp`, `pipe_resp`, `file_changed` and -`tick`: the machine-local host's own reports, which stopped being a frontend's -business when the daemon took the disk and the process table. Leaving them -decodable was not merely dead weight. `server.zig`'s `apply` routes any decoded -non-resize event straight into `core.update`, so an attached peer could forge a -pane's output, forge an `eof` for a shell that was still running — and unlike the -daemon's own `paneEof` the wire path never called `closePty`, so the master -stayed open and the shell was orphaned for the life of the session — or replace a -pane's text with bytes the next `Save` would write to disk. - -On the way out there are only three effects left, and which three is the whole -design: what a process nobody is looking at genuinely cannot do is put something -on THIS human's clipboard, read it back, and open a link in front of the person -who clicked it. The eight machine-local pushes — `spawn`, `pty_write`, -`pty_resize`, `write_file`, `write_dump`, `watch_file`, `watch_theme`, -`dump_themes` — were routed to ONE frontend precisely because each has one real -resource behind it, and every one of them is now performed in the daemon. Two -frontends can no longer fork two shells for pane 3 or race each other writing one -path, because neither of them writes anything. - -`detach` sits in the session range beside `quit` rather than among the three -effects, because it is not an effect the session performs on the world: it is one -frontend being told it is done. - -Host polling, presentation, process queries and worker dispatch stay local to -the session owner. They are not frontend wire messages. The owner also serves -9P: each filesystem reply returns to its requesting connection, not to attached -frontends. `Event.fs_req` therefore has no `ClientTag`. - -=== The codec is architecture- and build-neutral - -The frontend on the other end may be riscv32-freestanding while the core is -`x86_64` linux, so: every integer is an explicit width, little -endian, and no `usize` reaches the wire; no native struct is ever blitted, because -`@bitCast` of a Zig struct puts this compiler's field order and padding on a -socket; every union and every enum gets a tag chosen in this file and never -`@intFromEnum` of a core type, with exhaustive mapping switches, so adding a -variant to `Event` is a compile error here rather than a silent protocol -redefinition; every variable-length payload carries an explicit length prefix and -`max_payload` bounds the lot; a bool is one byte, 0 or 1, and any other value is a -decode error rather than "nonzero is true"; floats travel as their IEEE-754 -binary32 bit pattern inside an explicit `u32`. - -The codec is build-neutral for the same reason: `Event.resize.cell_pixels` exists only when -native PDF placement is compiled in, and a frontend must not have to have been -built with the core's options, so it is ALWAYS on the wire and dropped on arrival -by a build with nowhere to put it. - -`version` is a `u16` checked on connect and refused loudly, because two builds of -pardes are routinely on one machine — `zig build` replaces the binary under a -running session — and a frontend decoding another version's frame layout would -paint garbage and blame the terminal. `ClientTag` is exhaustive: both ends are -pardes, and an unknown tag is not a valid message in this protocol version. - -`max_payload` is 16 MiB, derived rather than chosen: a full frame of the largest -grid this protocol admits (512×128) at a worst case of one run per cell is -512×128×(6+20) = 1.6 MiB, and one paste is already capped at 4 MiB by the tty -frontend (`wire.max_payload`). - -== Connect first, swap second <swap> - -```zig -if (core.takeAttach()) |req| { - var attempt = detached_client.attempt( - gpa, req.name, core.screen_w, core.screen_h); - switch (attempt) { - .greeted => |client| { - attached.* = client; - break :frames; - }, - else => { - var mbuf: [256]u8 = undefined; - core.setMessage( - req.pane, - attemptEnd(&attempt, req.name) - .row(&mbuf), - ); - }, - } -} -``` - -`tty.localSession`, and the `takeAttach` block in `src/gui/gui.zig` is the same -shape. -CONNECTING IS NOT BEING ATTACHED, which is why this asks for a *greeted* client -and not for a socket: `Client.open` writes a hello and returns, and every way a -session says no — `refuse .version` for a session built from other bytes, -`.full`, `.quitting`, or a plain `quit` from one that ended in the same round — -arrives *after* a successful `connect(2)`. A swap that trusted the connect would -already have SIGKILLed every pane shell, unmounted the filesystem and freed every -undo history by the time it decoded the refusal. - -So `attached` is set only with the welcome in hand, and until it is, nothing has -been touched: a failed `Attach` costs one message row and leaves every pane, -every shell and every undo history where it was. The teardown that follows is -the function's own defers, reached by leaving its scope. - -The ordering is enforced in three places at once. The builtin only asks -(`builtins.Attach`). `Effect.attach` is unpacked into `attach_req` and -`attach_buf` by `perform`, and what the frontend acts on is what `takeAttach` -hands back AFTER the drain, not the effect value, which dies in the loop that -read it (`drainForAttach`). And `takeAttach` is consumed from the shell's -OUTER loop, beside `takeRestore` and for the same reason: both END this core, and -nothing running inside `pump` may destroy the core it is running in -(`Pardes.attach_req`). The unit test *Attach asks for a session and tears -nothing down* asserts -exactly that — after `Attach` and a full drain, `p.quit` is false and `p.panes[0]` -is still there. - -Because a failed connect is cheap, `Attach` is the one word in the session group -that is safe to press by accident, and it is the one that gets a leader chord: -`SPC s a`. `Detach` takes `SPC s D`, a capital because `sd` has been `Dump`'s -since before there was anything to detach from. Both entries sit behind -`if (pardes.can_attach)` in `src/config.zig`, and that gate is not decoration: -the leader table may only name a builtin that EXISTS, so on macOS — hosted, but -no poller — the two words are compiled out and naming them would be a compile -error. Which is the good outcome, and the reason the predicate exists. - -`Detach` is not `Attach` backwards, and the asymmetry is deliberate. Turning a -live local session into a daemon needs `setsid` and a fork; a word that pretended -to would hand you a session that dies with the window it was typed in. Run -locally, `Detach` therefore reports rather than acts. - -== `host_io.zig`: the machine-local half - -`forkShell`, `writeFileBytes` and `writeFd`, and it is now the only copy of them: -`tty.zig`, `detached/server.zig`, `gui/gui.zig` and `macos.zig` all fork and -write through it (`src/host_io.zig`, module header). - -They did not always, and what the four copies had in common is the better -argument for the file existing than "it is shared" is: ALL FOUR were missing -`FD_CLOEXEC` on the pty master. `/dev/ptmx` is opened by `forkpty` with no -`O_CLOEXEC` and there is no flag argument to ask for one, so in every shell -pardes has ever shipped, a program in one pane could read and write another -pane's terminal. The silent half is worse and compounds: closing a master is the -only thing that hangs its shell up, and a master a later shell still holds open is -not closed, so a pane delete or a respawn left an orphaned shell that never -exited — never reaped, eventually blocked writing into a pty nobody reads — and -each orphan pinned every earlier pane's master in turn. The startup drain forks -pane 0 and then pane 1, so the arrangement existed from boot. - -`nested.setCloexec(master)` after the fork fixes it for all four callers at once, -and the file states the window that leaves rather than papering over it: `fcntl` -after `fork` is not atomic, so a thread that forks and execs between the two -syscalls inherits the master anyway. In the detached daemon there is no such -thread — it is single-threaded by construction, which is what putting the pty -masters in its own `poll(2)` bought. Closing the window in the threaded shells -means replacing `forkpty` with `posix_openpt(O_CLOEXEC)` / `grantpt` / -`unlockpt` / fork / `setsid`. - -`forkShell` takes the core it is forking on behalf of and nothing about -terminals: no vaxis, no `Loop`, no reader thread. Who drains the master is the -caller's business, and the callers answer differently on purpose — the tty, gui -and macOS shells hand it to a worker that posts into their event loop; the daemon -adds it to the one `poll(2)` it already runs and makes its own copy non-blocking -in order to. - -= The board - -== An object, not a module - -`-Dplatform=esp32p4` emits ONE freestanding riscv32 object exporting the C ABI in -`src/esp32p4.zig`; the sibling `05-zig-p4` toolchain links it beside its own `_start`, its -generated linker script, and its UART driver (the `pardes-esp32p4` object -`build.zig` emits). Not an -executable, because the entry point is over there. Not a library, because -`addLibrary` bundles a `compiler_rt` the firmware already has. - -Historically, it was a module exposed through `build.zig.zon`. A dependency in the OTHER direction was built and -reverted: nesting this package's roughly 30-package graph under `zig-p4`'s broke -every build in that repo, not just the firmware one. `std/Build.zig:2091` -exceeded its -1000-branch comptime quota through ghostty's `SharedDeps.zig:874` `lazyImport`, -seven cached tree-sitter versions failed to compile because their `build.zig` -uses APIs removed in 0.16, and the fetch materialised 2.6 GB across 42,736 files -into a repo whose entire claim is that Zig is its only dependency. This direction -was possible because `zig_p4` declared no dependencies of its own. The current -editor build no longer imports that package; the firmware toolchain is separate. - -The C ABI carries terminal bytes. After building the object here, `zig build --Dpardes` in `../05-zig-p4` links `src/esp32p4/app.zig` into the firmware image. -That toolchain needs ESP-IDF register headers; the local object build does not. -The standalone GPIO 9P image instead uses `zig build --Dapp=../02-pardes-code/src/esp32p4_9p.zig` there. It does not link the editor: -`src/esp32p4_gpio.zig` holds its fixed namespace and `src/esp32p4_9p.zig` drives -the UART protocol loop. - -The object is also the compile probe. Rooted at `src/esp32p4.zig` it drags the -whole core through the riscv32 backend by actually calling it, so `llvm-size` on -the result is a real number to hold against the board's 1.5 MiB factory -partition. - -Where the terminal is: on the host. The board writes ANSI and reads ANSI, and the -emulator at the far end of the serial line does the font rendering and answers -this program's own capability queries. That is why vaxis works here unmodified — -`Vaxis.render`, `queryTerminalSend` and `enableDetectedFeatures` all take a bare -`*std.Io.Writer`, so the transport is a parameter, while `vaxis.Tty` and -`vaxis.Loop` are termios/ioctl/SIGWINCH bound and are not used. Window size -arrives as DEC mode 2048 in-band resize reports, parsed by `vaxis.Parser` like -any other input, because firmware has no `TIOCGWINSZ`. - -== The memory budget is one table - -`src/memory.zig`'s `limits` contains the board-shaped capacities. These numbers used -to be nine `platform == .esp32p4` tests scattered across nine files, each one a -separate place to forget — and they are not nine decisions. They are ONE -decision, how much memory this build is allowed to spend, taken nine times where -no reader could see the total. - -The original budget table below is historical; `memory.limits` is authoritative. - -```zig -/// `board` is the ESP32-P4 firmware's budget: -/// a 384 KiB heap and a 240 KiB chunk of L2MEM -/// shared between `.bss`, `.data` and the stack. -const board = config.platform == .esp32p4; -/// No OS means no address space to reserve -/// megabytes out of, whatever the platform is -/// called. -const reduced_target = - builtin.os.tag == .freestanding; - -pub const board_heap_bytes = 384 * KiB; -pub const effect_cap = if (board) 128 else 4096; -pub const wrap_rows = if (board) 128 else 256; -pub const cwd_buf_cap = if (board) 0 else 1024; -pub const undo_max = if (board) 16 else 256; -pub const max_tag_tail: usize = - if (board) 512 else 4096; -pub const host_path_cap: usize = - if (board) 0 else 4095; -pub const embedded_sources = !board; -pub const hexdump_row_bytes: u32 = - if (board) 8 else 16; -// ... `arena` below; see the wide listing -``` - -Two booleans derive all of it, and nothing outside this file tests the platform -for a capacity again. Every cap says what it is measured against, and -`board_heap_bytes` is the number the others are measured against: the 384 KiB -chunk of L2MEM at `0x4FF40000`. It is unconditional and not profile-derived, -because it is a fact about the silicon rather than a budget this build chose — a -desktop build that wants to know what the board affords is asking exactly that -question, which is what the memory tests in `pardes.zig` do with it. - -Each cap is a different *kind* of shrink, which is why they are not one scale -factor. - -- `cwd_buf_cap` and `host_path_cap` go to *zero*, and zero is a type: `Text(0)` - is a zero-sized field whose `set` refuses every non-empty path, so the three - producers — the resolved shell, the picked font file, the watched theme file — - report failure instead of storing 12 KiB nothing can fill. This is the - `PdfSlot` rule applied to a capacity: the board has no filesystem, no processes - to spawn a shell for and no font picker. -- `undo_max` and `wrap_rows` shrink *gracefully*. `pushHistory` evicts and frees - the oldest once full, so the smaller ring loses the deepest undo steps and - nothing else. `wrapWidth` reads the array's own length and refuses to wrap a - pane taller than it, so a taller pane renders unwrapped rather than getting a - truncated map. -- `effect_cap` changes *behaviour*, and the file says so: `emit` has always - refused rather than evicted once full, so on the board a burst larger than 128 - effects now drops its tail where 4096 would have held it. That is reachable - only through `emitWrite`, i.e. only if a pty ever appears on this platform. -- `embedded_sources` is a capacity spelled as rodata. The allowlist is empty on - the board, because the table is about 0.95 MiB against a 1.5 MiB partition. - The API is unchanged — `all` is a zero-length array and `find` answers null — so - every caller compiles identically and simply finds nothing embedded. -- `hexdump_row_bytes` is the one that is about the *display* rather than memory. - `hexdump -C`'s sixteen needs 79 columns; the P4 drives 56 of which seven go to - the line-number gutter, so a sixteen-byte row wraps onto a second display line - and the columns stop lining up, which is the entire value of the layout. Eight - fits in 46 and keeps every property that matters. - -#wide(caption: [`limits.arena`. Three tiers, because the address space -differs by four orders of magnitude. The board's tier is deliberately ALL -FALLBACK: every buffer here is a `StackFallbackAllocator`'s static, which lands -in `.bss`, and on the P4 `.bss`, `.data` and the stack share one 240 KiB chunk of -L2MEM while the heap is a separate 384 KiB chunk. A megabyte-shaped reservation -would not fit, and every byte that did fit would be taken from the stack's -neighbourhood to duplicate memory the heap already has. Zero is legal and always -spills, which is exactly what an arena for a compiled-out subsystem should do.])[ -```zig -pub const arena = struct { - pub const pardes = if (board) 4 * KiB else if (reduced_target) 8 * MiB else 32 * MiB; - pub const frame = if (board) 4 * KiB else if (reduced_target) 4 * MiB else 16 * MiB; - // ... - pub const tree_sitter = if (board) 0 else if (reduced_target) 4 * MiB else 16 * MiB; - pub const image = if (board) 0 else if (reduced_target) 64 * KiB else 32 * MiB; - pub const pdf = if (board) 0 else if (reduced_target or !config.mupdf) 64 * KiB else 64 * MiB; -}; -``` -] - -What does NOT belong in this table is capability switches. `terminal_panes`, -`builtins.Board.enabled`, `hosted` and `font_picker` answer "does this build have -the thing at all", which is a question about the platform and not about a budget, -so they stay next to the thing they gate. - -That division is also why a build option selecting the board's budget on a -desktop cannot work, and the file records the attempt. `-Dmem-profile=board` was -meant to let a native test runner compile the board's capacities and boot the -core under them. The dominant term in a boot is `@sizeOf(Pane)`, which carries -the ghostty-vt `Terminal` — 1.1 MiB of it — and what removes that is -`pardes.terminal_panes`, a CAPABILITY keyed on the platform rather than a -capacity in this table. So the option shrank the rings and left the boot six -times over budget, producing a configuration nothing was designed for: -`zig build unit-test -Dmem-profile=board` deadlocked in a futex rather than -failing, because a hosted build with the board's effect ring silently drops -effects a hosted test is waiting on. What DOES test the board's memory pressure -natively is in `pardes.zig`: the grid-scaled cost and the allocation-failure -sweep, both platform-independent and both running on the ordinary build. - -== What the board does not have - -`terminal_panes` is false there and nowhere else (`pardes.terminal_panes`), and it -is a platform gate and deliberately not one derived from the target: `web` is -freestanding too and KEEPS the emulator, because the browser shell renders a -replayed dump. False means ghostty-vt is not in the module graph at all — -`build.zig` never even asks for the dependency — which removes about 400 KiB of -flash, a `PageList` of RAM spent parsing input that cannot arrive, and a pile of -freestanding root hooks (`os.PATH_MAX`, `os.heap.page_allocator`, a cwd handle) -that the core itself does not want. - -Tree-sitter is refused outright: `-Dplatform=esp32p4` with anything but -`-Dtree-sitter=disabled` fails the build, because the grammars' parse tables are -megabytes against a 1.5 MiB partition (`build.zig:298`). MuPDF is refused for the -same reason (`build.zig:297`). - -The board gains four words nothing else has, all gated on -`builtins.Board.enabled == (platform == .esp32p4)`: `Peek`, `Poke`, `Hexdump` and -`Gpio`. Each takes an address or a pin, so none can have a leader path — a key -path names a builtin and can never carry an operand. `Gpio` is the one that goes -through the host seam (@seam) rather than reaching the registers directly, and -`gpio_toggle`'s comment says why: driving a pad correctly is not one -register. It is the IO MUX function select, the GPIO matrix output route, the -pad's drive and input-buffer bits, and the output enable, keyed by a per-pin -table. The firmware already owns that code and checks it against ESP-IDF's own -headers on the die; a second copy in the core would be a second copy nobody -tests. - -`builtins.Board` makes the target the *witness* rather than the gate: `enabled` -is keyed on the platform, and a `comptime` block then refuses to compile if that -platform is hosted, is not freestanding, or is wasm — because whatever else -`esp32p4` means, it has to still be a machine whose addresses are the bus's -(`builtins.Board.enabled`). - -= The control filesystem <fs> - -== Namespace and transactions - -`src/fs.zig` owns Look resolution and the editor's file interface. An ordinary -Look checks the OS first, then the virtual tree. `/n/os` and `/n/self` select -those mounts explicitly; `/virtual` names the embedded and self-reflecting tree. -Named remote mounts live under `/n/<name>` and retain their identity through Save. - -Every native session opens a 9P2000 Unix socket. The wire root exposes `os` and -`self`, without the editor's `/n` prefix. `self/pane/<serial>` contains `body`, -`tag`, `ctl`, `addr`, `data`, `event`, and selection files. Offsets are UTF-8 -bytes. `self/screen` freezes rendered cells and styles for the lifetime of an open. -See `docs/fs.md` for the public paths and commands. - -`src/9p.zig` implements the protocol without OS dependencies; `src/9p_io.zig` -owns native sockets and the client. Requests enter through `Event.fs_req`, and -`Effect.fs_reply` carries replies. A pending event read returns `Status.again`; -the native listener owns waiting and retries. Filesystem mutation runs on the -same thread as editing. - -== Nested Look - -Pane shells inherit `PARDES_PID` (the editor's process id), `PARDES_9P` and -`PARDES_PANE` (the pane serial). The first answers whether the shell is inside -a pardes at all, the other two answer how to reach it, and they are separate -because a session whose listener never came up still owns its children. A child -launch that finds a live `PARDES_PID` and no way to reach it says so instead of -starting a second editor. Otherwise it resolves its OS-relative argument in the -child's working directory, then writes `look <path>` to the parent's -`pane/<serial>/ctl`. Explicit `/virtual` and `/n` paths resolve in the parent. -The ordinary filesystem update performs layout and drains host effects. - -`--nested` starts a separate editor and withholds `PARDES_PID` from its direct -pane shells, which is the whole of the opt-out. Its 9P socket remains available -for control and plugins, so those shells still get `PARDES_9P` and -`PARDES_PANE`. There is no executable-name discovery or separate Look listener. - -Detached frontends use `pardes-detached-<name>.sock` in the same runtime -directory. The shared Unix socket conventions live in `src/9p_io.zig`. - -= Build - -#wide(caption: [The build graph. One `root_mod` per invocation, rooted at -whichever file that platform's host enters through; up to three more compilations -of the same graph beside it; one `pardes_config` per distinct *frontend*, which -is the only thing two of them are allowed to disagree about. Line numbers are -`build.zig`.])[ - #diagram[ - #cetz.canvas(length: 1cm, { - import cetz.draw: * - set-style(stroke: 0.4pt, mark: (fill: black, scale: 0.35)) - let row(y, body) = content((0.3, y), body, anchor: "west") - - // ---- tier 1: inputs, full width ---- - rect((0, 6.76), (18.0, 9.5)) - content((9.0, 9.24), text(7.0pt)[*inputs, generated or read at configure time*]) - line((0, 9.02), (18.0, 9.02)) - row(8.72, text(5.8pt)[`highlights.scm` from 29 grammars #sym.arrow.r one options module]) - row(8.34, text(5.8pt)[`vendor/themes/*.toml` (214 helix) and `*.json` (11 zed) #sym.arrow.r 225 generated theme modules plus a `list.zig`, straight into the build cache]) - row(7.96, text(5.8pt)[working-tree `.zig` sources #sym.arrow.r the web shell's read-only source archive]) - row(7.58, text(5.8pt)[8 GLSL shaders #sym.arrow.r SPIR-V through `glslc` --- or `-Dprebuilt-shaders` embeds the committed `.spv` + `.glsl` pair, so `EffectCode` cannot lie]) - row(7.24, text(5.8pt)[`@import("build.zig.zon")`, UNTYPED (`:8`) #sym.arrow.r `zon.version` (`:1920`) and a `comptime` check that `zls_version` matches the pinned sha (`:36-46`)]) - row(6.90, text(5.8pt)[`gitCommit(b)` (`:1855-1867`) #sym.arrow.r `?[]const u8`, null when there is no repository, no `git`, or nothing to say]) - - // ---- tier 2: modules and the options module ---- - rect((0, 3.8), (8.7, 6.6)) - content((4.35, 6.34), text(7.0pt)[*modules that compile* `src/pardes.zig`]) - line((0, 6.12), (8.7, 6.12)) - content((4.35, 5.84), text(5.8pt)[`root_mod` (`:306`) --- its root is the file this]) - content((4.35, 5.50), text(5.8pt)[platform's host enters through:]) - content((4.35, 5.14), text(5.6pt)[`main.zig` (tty, gui) | `web.zig` | `macos.zig` | `esp32p4.zig`]) - content((4.35, 4.74), text(5.8pt)[`hx_core_mod` --- a headless second core for `hxdiff`]) - content((4.35, 4.38), text(5.8pt)[`isolated_mod` --- one comptime bool: no filesystem]) - content((4.35, 4.02), text(5.8pt)[`gui_mod` --- the same `main.zig` at `platform = .gui`]) - - rect((9.3, 3.8), (18.0, 6.6)) - content((13.65, 6.34), text(7.0pt)[`pardes_config` --- ONE PER DISTINCT FRONTEND]) - line((9.3, 6.12), (18.0, 6.12)) - content((13.65, 5.84), text(5.8pt)[`platform`, `mupdf`, tree-sitter tier, `zls_version`,]) - content((13.65, 5.50), text(5.8pt)[`esp32p4_cols`/`rows`, `theme_animation`, `zig_lib_dir`,]) - content((13.65, 5.14), text(5.8pt)[`version` (from the manifest), `commit` (from `git`)]) - content((13.65, 4.70), text(5.8pt)[`ShellConfig` (`:1874-1890`) holds every field that is a]) - content((13.65, 4.34), text(5.8pt)[property of the BUILD, so the two modules a default build]) - content((13.65, 4.00), text(5.8pt)[cannot drift apart in any field but `platform`]) - - line((4.35, 6.76), (4.35, 6.6), mark: (end: "stealth")) - line((13.65, 6.76), (13.65, 6.6), mark: (end: "stealth")) - line((9.3, 5.2), (8.7, 5.2), mark: (end: "stealth")) - - // ---- tier 3: outputs, full width ---- - rect((0, 0), (18.0, 3.3)) - content((9.0, 3.04), text(7.0pt)[*what one invocation emits*]) - line((0, 2.82), (18.0, 2.82)) - let out(y, name, what) = { - content((0.3, y), text(6.0pt)[#name], anchor: "west") - content((4.9, y), text(5.8pt)[#what], anchor: "west") - } - out(2.52, [`pardes`], [`-Dplatform=tty`, the default --- vaxis over a real terminal]) - out(2.12, [`pardes-gui`], [`-Dplatform=gui` --- SDL3, a FreeType atlas, SPIR-V]) - out(1.72, [`pardes.wasm`], [`-Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<dump.zon>`, plus a vanilla DOM shell]) - out(1.32, [`libpardes.a`], [`-Dplatform=macos`, then the `macos-app` step #sym.arrow.r `pardes.app` (Darwin host)]) - out(0.92, [`pardes-esp32p4.o`], [`-Dplatform=esp32p4`; the sibling `05-zig-p4` toolchain builds the firmware image]) - row(0.46, text(5.6pt)[a bare `zig build` emits the FIRST TWO together, because those two are what installing pardes means; every other spelling is one invocation each]) - row(0.18, text(5.6pt)[beside them, from the same graph: `pardes-isolate` (tty only --- the filesystem is not compiled in) and `hxdiff`'s headless core]) - - line((4.35, 3.8), (4.35, 3.3), mark: (end: "stealth")) - }) - ] -] - -== One primary shell, sometimes two - -`-Dplatform` names ONE shell. Absent, the build makes BOTH native shells — the -tty cli and the SDL gui — because those two together are what installing pardes -means, and asking for them one at a time is two invocations a person has to -remember are two (`also_gui` in `build.zig`). Everything else derives from `platform`, -the PRIMARY shell: the one rooted at `root_mod`, the one `unit-test` runs, and -the one the snapshot and harness suites drive. `also_gui` adds the second beside -it and changes nothing about the first. - -A bare `zig build` with an untouched prefix also redirects the install prefix, on -two conditions and the second is not the obvious one: no shell was NAMED -(`-Dplatform=web` keeps writing `zig-out/web`, which its docs name), and nothing -else has already said where to install — no `DESTDIR`, no `--prefix`, no -`--prefix-*dir`. It cannot depend on which *step* was asked for, because -`build.zig` cannot know that: `build_runner` keeps the step names in a local and -resolves them after `build()` returns. That is why the dev binaries install to -`<prefix>/dev` rather than `<prefix>/bin` — `zig build perf` redirects the prefix -too, and a 200 MB Debug benchmark must not land on a PATH. - -The other three platforms are separate invocations: -`-Dplatform=gui` for `pardes-gui`; -`-Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<dump.zon>` for `pardes.wasm` -plus a vanilla DOM shell; `-Dplatform=macos` plus the `macos-app` step for -`pardes.app` wrapped around a static `libpardes.a` on a Darwin host; and -`-Dplatform=esp32p4` for the freestanding object. Any other spelling of the -combination fails on purpose rather than building something silently wrong — the -web target, the web dump, MuPDF on a freestanding platform, and tree-sitter on -the board each have a named refusal (`build.zig:292-298`). The esp32p4 target is -in fact *forced* to `esp32p4_target` rather than taken from `-Dtarget`, so a -`-Dtarget` cannot discard its CPU features, and a wrong one is still an error -rather than silently overridden. - -Because `platform` is comptime and `pardes.platform` is read from -`pardes_config`, two frontends in one build need two options modules — the gui -shell `@cImport`s an SDL the cli must not link. `ShellConfig` gathers everything -`pardes_config` carries that is a property of the BUILD rather than of one -frontend into one value, so the two modules a default build makes cannot drift -apart in any field but the one that is supposed to differ -(`ShellConfig`). `hxdiff`'s headless core and `pardes-isolate` share the -primary shell's, being the same frontend. - -`pardes-isolate` is a third compilation of the same graph whose only difference -is one comptime bool. It cannot be the same binary with a flag, because the point -is that the isolated build does not CONTAIN the filesystem: `look.zig`'s libc -paths compile away, so no runtime mistake can reach a disk that a `--flag` build -still links. Only the tty platform has one — gui adds a GPU it would still need, -and web and macOS are libraries whose host owns `main()`. - -== Generated inputs - -Native dependencies are ghostty, vaxis, uucode (shared config), zstbi, SDL (a -pinned fork, lazy), FreeType, HarfBuzz (the SDL shell's text shaping, lazy, -built against that FreeType), MuPDF (`-Dmupdf`, on by default everywhere but the -web and the board, lazy), ZLS, mvzr (the regex engine behind `s`/`S`), -zig-tree-sitter with 29 grammars for 28 languages (markdown takes two, block and -inline) — all pinned through `zig fetch` and wired in `build.zig`. The board's -`05-zig-p4` firmware toolchain is a separate sibling checkout. - -The 9P library is the one pinned dependency that is not third-party: `cloud9` -lives at `git.sr.ht/~gbrls/cloud9`, pushed as `[email protected]:~gbrls/cloud9` and -fetched from that commit's HTTPS URL, since zig has no `git+ssh` support. - -Generated during the build: `highlights.scm` into an options module; the vendored -helix and zed theme sources into the generated half of the theme ring; -working-tree `.zig` sources into the web shell's read-only archive; and eight -GLSL shaders into SPIR-V. - -The themes go straight into the build cache and are handed over as a module, -rather than written back into `src/`. The output then has no freshness problem to -own — zig re-runs the generator only when a vendored file changes, and a cached -run leaves the compile's inputs byte-identical, so `zig build` with nothing -touched still does nothing — there is nothing to gitignore, and no stale `.zig` -can survive a deleted source. The INPUT list is read from the directory rather -than written out, so adding a theme is dropping a file in, and each file goes in -as a content-hashed argument, which is what makes that new file re-run the step -and nothing else. The generator is compiled in Debug on purpose: it runs for -about 40 ms, and every core module imports what it generates, so its compile is the -first link of every cold build; ReleaseSafe cost 16 s of compile to save 47 ms of -run time, and the files it writes are byte-identical either way -(`theme_gen` in `build.zig`). - -The shaders are the only build input wanting a tool a stock machine lacks -(`glslc`), so they are also the only one whose output is committed: -`zig build shaders` refreshes each paired `shaders/prebuilt/*.spv` binary and -`.glsl` source snapshot together. `-Dprebuilt-shaders` embeds that exact pair -rather than shelling out, so `EffectCode` cannot describe different shader text -from the binary on screen; this is also what lets a gui build need nothing but a -C toolchain. - -The tutor and the embedded font are plain `@embedFile`s, not codegen. Debug builds -are incremental for the seconds-loop; release builds are the product. - -== Versioning - -There is one version string in the tree and it is in the manifest. - -```zig -// build.zig:8 -const zon = @import("build.zig.zon"); -// ...in shellOptions, build.zig:1915-1923: -o.addOption([]const u8, "version", zon.version); -o.addOption(?[]const u8, "commit", cfg.commit); -``` - -The `@import` is UNTYPED on purpose, and that is what makes it possible: -annotating its type would demand an exact field match and reject -`.dependencies`, `.paths` and the rest, which is the failure the hand-synced -literal that used to sit there was working around. - -```zig -// src/pardes.zig -pub const version = @import("pardes_config").version; -pub const commit: ?[]const u8 = - @import("pardes_config").commit; - -// src/main.zig -const version_text = if (pardes.commit) |c| - "pardes " ++ pardes.version ++ " (" ++ c ++ ")\n" -else - "pardes " ++ pardes.version ++ "\n"; -``` - -Both halves are comptime, so `version_text` is one string in rodata and -`pardes --version` is one write (`main.version_text`). - -`gitCommit(b)` runs -`git -C <build root> rev-parse --short=12 HEAD` at CONFIGURE -time, so the binary carries a string rather than the ability to shell out. That -is the whole point: a `--version` that runs `git` itself reports the tree it -happens to be standing in rather than the one it was built from, and on the board -there is no `git` to run and no process to run it with. The `-C` is not -decoration either — `zig build` may be run from anywhere, and a `git` resolved -against the cwd would cheerfully answer about a different repository. - -Absence is not an error and must not be. A release tarball has no `.git`, a -container may have no `git` binary, and a source drop is not a repository; every -way of having no answer lands on the same `null` and the frontends print the -version alone. It is deliberately NOT `--dirty`: marking a dirty tree would cost -a worktree stat on every configure and — the real cost — would change -`pardes_config` on every file edit. Every module in the build imports that -options module, so a dirty marker means editing one line rebuilds the world. The -commit alone changes only when a commit does. - -The manifest is read at comptime twice, and the second read is what keeps the -one string this build cannot fold into the manifest honest. `zls_version` -(`zls_version`) has to be spelled beside `.dependencies.zls.url`, because the -semver half of it — `0.16.1-dev` — exists nowhere in the manifest, and ZLS's own -`build.zig` needs it: that file otherwise shells out to `git describe`, which -fails on a fetched package with no `.git`. So the duplication is unavoidable and -a `comptime` block beside it makes the two AGREE instead: it takes the -short commit after the `+`, takes the sha after the `#` in -`zon.dependencies.zls.url`, and `@compileError`s unless the pinned sha starts -with it. A `.zon` bump that forgets the line beside it is now a build error -rather than a `SPC l i` naming an analyser nobody linked. - -= Testing: the old program is the oracle - -Before the rewrite compiled, the prototype got a harness (`test/snapshot.zig`, -the `zig build snap` step): it forks either binary in a pty, feeds it an *event -script* (one line per input: keys, SGR mouse, resizes, sync points), and captures -the rendered grid — text, cursor, and per-cell style runs — through its own -ghostty terminal. Goldens are generated from the old binary -(`zig build snap -- --update`); the new binary must reproduce them byte for byte. -Eighteen scripts covered the checklist in Appendix A at the rewrite; ninety-five -cover it and everything since (`test/snapshots/` holds 95 `.snap`/`.golden` -pairs). All pass. - -Determinism pins: fixed workdir paths (they appear in tags), a controlled `$HOME` -with `PS1='$ '`, `LC_ALL=C`, and a grid-stability sync primitive instead of -timing guesses. - -Two of those pins were doing damage rather than work. - -== Deltas, not screens - -A capture used to restate the whole screen, so 82% of golden lines were a copy of -the line above (`test/snapshot.zig:59`), every golden carried the topbar and a tag -row, and adding one builtin word to a tagline rewrote 78 of them — 3,758 lines of -diff for a change no test was about. A capture is now a *delta* against the -previous capture of the same kind in the same script: the first is the whole -screen, the rest are only the rows that changed. The corpus went from 16,700 lines -to 5,722 and the same one-word edit now moves 337 (`src/CHANGELOG.md:9-13`). A -capture whose only change is the cursor is the empty delta its script always -meant. - -== A click names a word - -A click used to name a screen column, which is a coordinate into that same -chrome. When `Newtty`, `Joincol` and `Changelog` were added, seven scripts began -clicking the word next door. `snapshot.zig`'s own note enumerates six of them: -`tutor.snap` clicked `Grep` where it meant `Tutor`, `exec.snap`, `respawn.snap` -and `tagalign.snap` clicked `Newtty` where they meant `Del`, `find.snap` clicked -`Joincol` for `Find`, and `windowops.snap` clicked `Tutor` for `Debug` — and -`--update` blessed all of it, so 256 golden lines were green while asserting the -opposite of their script's first line (`test/snapshot.zig:817-825`). The seventh -is the changelog's: `tagbottomimage.snap` clicked blank space 176 columns from -the `Del` whose effect it asserted (`src/CHANGELOG.md:9-11`). A click may now -name the word (`press middle @Del 2`, `@Del#2` for the second pane on a row, -`@Save-2` for a column beside one), so the word is either there to be clicked or -the script fails. - -== Regeneration verifies itself - -Regeneration was the other half of that failure: `--update` captured once, -serially, and wrote whatever it saw, which is how six wrong clicks became -goldens. It now captures in parallel at the widest probe settings the harness -has and then runs the ordinary verify pass over what it wrote, so a capture that -does not reproduce is reported instead of committed — 77 s to 41 s, and the retry -machinery serial update never had (`src/CHANGELOG.md:14-16`). - -== Deviations kept after review - -Three deliberate deviations surfaced by the oracle and kept. - -The greeting `ls` waits for the exact OSC 133 B input mark after the real resize; -the prototype raced bash's startup and won only by allocator luck. Shells without -prompt integration omit that cosmetic greeting rather than guessing. Commands -which create a fresh shell are owned by its terminal pane until the host reports -the actual prompt capability, then wait for the same mark when it exists. - -Dump files compare with base64 pty history elided: it encodes prompt-redraw -micro-timing, not state, and the cleaned text fields are the contract. - -Typed insert runs do not survive a dump replay. They are an overlay, not pty -bytes, and the prototype's replay viewer had the same semantics. - -== The shell suites - -Shell correctness is no longer out of scope, and that is the biggest change to -this section since the rewrite. `web-snap` and `web-e2e` drive headless Chromium -over CDP with real DOM pointer and touch events and diff `test/web-snapshots/`; -`image-harness` and `pdf-harness` snapshot native PIXEL output through kitty -graphics and SDL; `macos-e2e` is an offscreen AppKit snapshot suite over its own -seven scripts (`test/macos-snapshots/`: boot, cwd, drop, font, keys, rotate, -trackpad). In the sibling `05-zig-p4` toolchain, `zig build selftest` flashes and -runs `src/esp32p4/selftest.zig` on real hardware. The local freestanding object -and the standalone GPIO 9P image can both be compiled without a board attached. - -Two suites are differential rather than golden: `hxdiff` compares the core's -motion and operator results against helix case by case, and `hxparity` compares -editing a file against editing the same text in a pty — which is what the -per-pane ghostty-vt `Terminal` buys. - -What remains untested by a snapshot is the last hop — that vaxis diffs correctly -onto a real terminal — plus each shell's inline unit tests (the FreeType atlas -raster, the trackpad and rotation maths, the gamepad replay). - -= Style - -TigerStyle, plus house rules proven in the prototype: imperative and flat; one -big `update` dispatch, not handler objects; no one-line helpers — inline the -four-line scan; assert invariants at entry (`assert(vsum > 0)`); static -allocation at init, arenas per frame. - -The rewrite used source size as a pressure toward direct code, and that remains -useful when a refactor deletes duplicate policy or state. A checked-in line-count -inventory does not: it goes stale whenever a pane kind, backend, or generated -asset moves. Measure the current tree when making that comparison; keep this -document about ownership and invariants that should survive the next edit. - -Two things that number is not. It is not one program's worth of growth — the -macOS, web and firmware shells and the PDF and language work are several products -sharing a core. And it is not licence: the rule that survives is the local one, -that a change should leave the file it touches no longer than it found it. - -= What is deliberately absent - -No render abstraction over the shells: the surface *is* the abstraction. No -plugin system, because the control filesystem is the extension point -(@fs) and needs no API of its own. No async runtime in the core — the -shells may thread, the core is single-threaded by construction. - -"No config files" held until the startup file (`docs/config.md`) arrived, and that -is the narrowest thing the phrase could still cover: a list of builtin COMMANDS -run before the first frame — no schema, no new vocabulary, and no key remapping. -The keymap is `src/config.zig`, compiled in, where a wrong binding is a compile -error rather than a silent no-op. - -#heading(numbering: none)[Appendix A: feature parity checklist] - -From the prototype survey; every line is covered by at least one of the event -scripts in `test/snapshots/` — the original eighteen (boot, tty, edit, scroll, -modal, look-file, look-dir, exec, tag, theme, tutor, windowops, dump, load, -ttyonly, syntax, fileedit, images), and seventy-seven more added since for -everything below that the prototype never had. - -*Layout.* Columns by weight (≤6), panes by vweight (16 panes in total, not -per column); global topbar; -per-pane gutter (move box + scrollbar) and tag row; `splitBelow` shrinks only -the source (cursor row kept visible); `Newcol` takes width only from the source -column and cannot resize any unrelated column; dying pane's weight absorbed by one -sibling; emptied column hands width to a neighbor; border-drag resize on a -pane's own trailing edge (v and h), hover shows `╎`/`╌` glyph-only hints; -corner grab moves exactly two boundaries; -move-drag via the gutter box with preview; Alt-n new shell below, Alt-c move -pane to new column; Ctrl-w h/j/k/l directional focus; body-normal Esc hops to -the pane you were in before this one, alternating between two (the same -builtin as SPC j j); -`--tty` single-pane mode. - -*Mouse.* Left: select (block, stays highlighted after drag, pins cursor, -enters normal), click clears, tag-row click enters tag edit, scrollbar -click scrolls up-to-row (right button: down), border/move drags. Middle: -execute — no-drag expands to file-ish word (alnum `.-+/:@_~`); builtin or send -to shell; does not focus. Right: look — peel `:NNN`, resolve against the -clicked pane's dir (`/proc/pid/cwd`, or a file's dirname) and, if that fails, -against every other live pane's, most-recently-focused first (the jump stack -backwards, duplicate dirs skipped; absolute words try one dir and stop) so a -relative name is openable from any window that can see it; dir → shell + `ls` -in source column (dedup by cwd), file → file pane at line (dedup by path, -first doc opens left column and may evict a lone pristine shell; later docs -split below the existing doc — an output pane, `+Search`/`+Help`, is not a -doc for either half of that rule: it never claims a column, it splits below -whatever pane asked for it, and nothing splits from it), image ext → image -pane, `.pdf` → PDF pane. Ctrl+left is goto-definition, the one chord borrowed -from every editor with a language backend. Two spellings the word expansion -has beyond a path: `` @`ls -la` `` is taken WHOLE and runs as a command rather -than opening as a file, and `@p7:10:5` addresses a live pane by number for the -things — terminals, output buffers — that have no path to name. -Middle+left chord: kept -left selection appended as trailing CLI argument. A stationary pointer gets a -delayed, theme-derived highlight of the exact side-effect-free selection that -Look would expand; it neither focuses nor installs that selection, and pointer -leave/input/content invalidation cancels it. Wheel: scroll hovered pane, batched. - -*Modes.* normal: helix motions (`h j k l w b e W B E 0 gl ^`, `f F t T` and -`Alt-.`, counts, `gg ge gh gs gl g| G`, `Ctrl-d/u/f/b`, `zt zz zb zj zk`), -insert entries (`i a I A o O`), selections (`v` extend — displayed as a fourth -mode name, "select" — `x`/`X`/`Alt-x`, `%`, `;`/`Alt-;`, `_`), MULTIPLE CURSORS -up to 64 (`C`/`Alt-C` copy, `s`/`S` select and split by regex with a live -preview, `Alt-s` split on newline, `,`/`Alt-,` keep and remove the primary, -`)`/`(` rotate, `Alt--`/`Alt-_` merge), operators (`d c y p P R u U`, `Alt-d` -delete-noyank, `J`, `>`/`<`, `~`/`` ` ``/`` Alt-` ``, `Ctrl-a`/`Ctrl-x`, -`Ctrl-c` comment-toggle), textobjects and surrounds under `m` -(`mm mi ma ms mr md`), the `]`/`[` pairs (`]p ]d ]D ]<space>`), `/` with `n`/`N`, -`|` to filter the selection through a command, `:` for the tag as a command -line, and Enter=look Tab=execute at cursor. Since the helix motion model -landed, a traversal motion SELECTS the range it crossed — which is why there is -no verb+noun grammar and why `i` after `w` types at the selection's start. insert: -click-and-type; terminals get splice runs (shift right, never overwrite; -absolute-row anchored), files get real edits. tty: raw pty forwarding -(Ctrl-key toggle, default Ctrl-b, `--tty-toggle`), mouse still usable, -promptClickMove on entry, prompts visible (hidden in the other modes via -OSC 133). `y` fills the yank register and `p` pastes it; neither touches the -system clipboard, which is helix's five words — `SPC y/Y/p/P/R`, out via -`set_clipboard` and back via `read_clipboard` — and nothing else, so a delete -cannot clobber what the desktop was holding. A paste from an outer terminal -arrives bracketed, as one `paste` event. - -*Tag.* Compact name/status prefix + editable command tail. File names support -staged edits: Enter commits a new buffer save target, Escape cancels, and no -disk rename or write happens until explicit Save. Terminal cwd and image/PDF -status stay generated. Save leads the tail of every pane holding text of its own: -`Save Tty Collapse Del` for a file or an output buffer, -`Tty Save Mode Filter Collapse Del` for a terminal, and `Tty Collapse Del` -for an image. PDFs use `Tty PdfSections PdfTint Collapse Del`, without tint status text. -The word that closes the thing is last on every tagline, so overshooting the -click before it cannot destroy anything. -`Collapse` toggles a pane between its tagline alone and its expanded height; -hidden body contents and running terminals are retained. -An unsaved file has `*` after its name; an image tag -reports -`img petscii:<on|off> palette:<commodore|terminal> ascii:<on|off> <path>` -before the ordinary tail (its renderer toggles are builtins under `SPC t -p/l/a`). Topbar: -`Newcol Joincol Find Grep Help Changelog Tutor Dump NextColor Debug Kill` — -editable by left click, with keyboard command navigation and Exec/Look gestures. -An additional editable tag in each column supplies local New, Tty, Find, -Grep and Joincol commands; it is always shown, as acme's column tag is. -Colors and Crt left it for their leader paths. - -*Panes.* Terminal: ghostty-vt, 16 MiB scrollback, OSC 133 prompt semantics, -OSC 7 cwd, DSR/DA/kitty-query replies (write_pty + device_attributes — the -nushell/helix regression), a default-on pane-local Filter which uses ghostty-vt's -theme-derived 256-colour generation and keys rendered cell foreground/background -truecolour and OSC overrides through that palette without mutating emulator state, -greeting `ls`, auto-follow output unless -navigating. File: line-number gutter (fixed width), tree-sitter highlights -(c/cpp/zig minimal tier; 26 grammars full tier; re-highlight on edit, -visible-range first), Save, open-at-line, undo/redo. Image: zstbi decode, -kitty graphics when available, petscii matcher fallback (C64/terminal -palettes, ascii glyph set toggle). PDF (`-Dmupdf`, native default on): MuPDF -rendering as one continuous page strip, real text search, mouse text -selection, the document outline into `+PdfSections`, fit-width/fit-height and -a themed duotone tint — the last three named in the pane's own live tag. -Embedded PDF links use the ordinary right-click Look gesture. Hovering one -shows an accent highlight and a pointing hand in graphical frontends; dragging -still selects text. Internal links reveal their page and anchor. When the -annotation's visible label resolves to a different Look location, a `+Links` -output lists the visible destination and the embedded destination for you to -choose with Look. Internal destinations in that list use `document.pdf:page`. -Labels that are not locations follow the embedded link directly; identical -destinations open once. Links use the same supported URL and file-location -rules as Look. -Tutor: embedded text as file pane. - -*Language.* The native host snapshots each request and runs it outside the UI -loop. Every language's server, Zig's zls included, is a child process the -JSON-RPC client speaks to. Results become output rows or edits, applied only while -the request's pane and revision still match. Web and board builds have no -language backend. See `docs/lsp.md` for configuration and commands. - -*Chrome.* One theme per `.zig` file, folded into the ring at comptime: ours -first — helix (default; near-black page, chrome one grey-ramp step off it, -colors lifted from the helix editor's own theme), dark, and the acme-light one -named simply `acme` — then -everything `tools/gen_themes.zig` exports at build time out of the vendored -helix `.toml` and zed `.json` sources in `vendor/themes`, which is all 214 helix -ships plus zed's 11, sorted by name. Names are unique by construction: helix's -own acme is vendored as `acme_helix.toml` so it cannot collide with ours, and -every zed theme takes a `_zed` suffix for the same reason. helix and dark leave -a child's ANSI palette native; acme and the zed exports resolve it onto -the page so shell output stays readable on a light one. A theme owns the page, -the tag bar, the gutter, the move box and the SELECTION: `sel_bg`/`sel_fg` are -one pair per theme, and the three per-button tints and the dimmed extra cursors -are mixed off it, so what stays fixed is the distinction between buttons and not -the colours. `NextColor` browses the ring one -step at a time — at 228 it is no longer how you REACH one — -`Theme <name>` jumps to one and `Themes` (`SPC t t`) lists them all into an -output buffer whose rows are those very commands — execute a row (Tab, middle -click) and the theme goes on; n/N select such a row WHOLE, since a command -line holds no place to pick out of it — the third grain of that motion, the -other two being one stop per ROW in a results list (the location at its head, -never the matched text after it) and every look-able word in free text. Native -shells also accept `ThemeFile <path>`: one complete ZON `Theme`, loaded at -runtime and, where document watches are available, watched with the same -parent-directory/rename-over semantics. `DumpThemes` materializes the compiled -ring under `<config>/themes/builtin/`, providing the schema and a copyable -starting point without adding inheritance or a second theme vocabulary. -Colors toggles -all recolor passes; Debug stats overlay; Dump writes state ZON. Eleven panel -transitions and three scene bits are settings, one builtin each, generated from -`config.Runtime.settings`. - -*Session.* `--detach[=name]` runs a core with no terminal; `--attach[=name]` -makes a thin frontend over a unix socket; `Attach [name]` (`SPC s a`) hands a -running frontend's screen to a detached core, connecting before it swaps; -`Detach` (`SPC s D`) leaves a session that carries on. Up to 32 frontends on one -session, all showing the same screen; the pane shells belong to the daemon and -outlive every frontend. The default 9P socket serves the control tree; a nested -`pardes <file>` hands its argument to the outer session over the per-pid socket. -`pardes --version` prints the manifest version and the commit it was configured -from. - -*Web (replay viewer parity).* Embedded dump; focus, border/move drags, wheel, -scrollbar, Colors/NextColor, and link-LOOK opening a new -tab (new — the prototype has no web link handling). One-finger touch is -deliberately complete by itself: tap = LOOK; drag past a small slop = natural -scroll, with no LOOK on release. A build-generated read-only archive lets LOOK -open the current contents of tracked or new/nonignored Pardes `.zig` files despite the web -shell having no host filesystem. The published launcher dump is captured from -a running Pardes TTY after -`git ls-files --cached --others --exclude-standard -- '*.zig' | sort`, so its -complete terminal listing is the same tracked-plus-new/nonignored set the user -can open. The freestanding module carries no host -libc, SDL, or WebGL; Tree-sitter's C runtime and the selected parsers link into -the module through a tiny local ABI shim (Zig is the compact web default). A body gesture retains -tap-LOOK/drag-scroll, but finger-down on -a tagline or a one-cell-tolerant pane separator latches to a left-mouse gesture -for its lifetime, keeping layout drags out of the scroll heuristic. Touch -input translation and scroll/tap state live entirely in the JavaScript shell. -The native SDL shell uses a FreeType light-hinted grayscale atlas over the SDL GPU API (SPIR-V), -with words shaped by HarfBuzz so a font's ligatures span their cells -(`Ligatures off` draws every cell as its own glyph). -The browser exposes each cell as selectable, inspectable text and applies the -surface styles with CSS. The browser `.snap` harness drives real Chromium touch -input and reads both text and per-cell styles from the DOM renderer's packed -surface. Joystick cursor for the steamdeck is *new* scope — the prototype ships -no gamepad input. - -*Board (ESP32-P4).* A 56×14 grid by default (`-Desp32p4-cols`/`-rows`), 384 KiB -of heap, no ptys, no tree-sitter, no MuPDF, no filesystem of its own, and no -embedded source table. vaxis unmodified over a byte sink the firmware supplies; -DEC mode 2048 for resizes. `Peek`, `Poke`, `Hexdump` (eight bytes per row) and -`Gpio` exist only here, and `Gpio` goes through the host vtable because the pad -sequence belongs to the firmware that already tests it against ESP-IDF's headers -on the die. diff --git a/docs/detached.md b/docs/detached.md deleted file mode 100644 index e010677c..00000000 --- a/docs/detached.md +++ /dev/null @@ -1,49 +0,0 @@ -# Detached sessions - -A detached session owns the editor core, pane shells, files, undo history -and layout. TTY and SDL frontends join and leave without ending it. - -```sh -pardes --detach=work & -pardes --attach=work -pardes-gui --attach=work -``` - -Bare `--detach` names the session after its pid; bare `--attach` needs -exactly one listening session. The detached process runs in the foreground -unless the shell backgrounds it. Inside an editor, `Attach work` (`SPC s a`) -switches this window to that session and `Detach` (`SPC s D`) closes only -this frontend. - -With no frontend attached, `/ctl`'s `size C R` sets the screen (160x50 until -then; [fs.md](fs.md#the-root-ctl)), and messages clear after `MessageLinger` -on the clock, since no key will come to dismiss them. - -## Files and transport - -The frontend socket is `pardes-detached-<name>.sock` under -`$XDG_RUNTIME_DIR`, else `~/.local/state/pardes`, beside the session's 9P -socket `pardes-9p-<name>.sock`. A second session cannot take a live name. -TCP and QUIC listeners, mounts and the control filesystem belong to the -session, not its frontends. - -## Ownership - -One poll loop owns core mutation, frontend connections, PTY I/O and file -watches; LSP and selection-pipe workers post completions through a bounded -mailbox. Restore builds the replacement core before changing the current -one, then sends existing frontends a fresh frame. Frontends provide input -and presentation only: clipboard writes are broadcast; clipboard reads, -browser opens and Detach go to the frontend that asked (else the first -attached). Frontends never spawn shells, write session files or watch files. - -## Wire and tests - -`src/detached/wire.zig` owns the versioned frontend protocol (version 8). -Frames are full grids or changes against each frontend's last frame; a new -attachment gets a full grid. Queues are bounded, so a lagging frontend -cannot hold up the session. - -`zig build unit-test` covers the wire, ownership, real frontend connections -and Restore; `zig build fs-test` drives detached sessions over 9P; the -snapshot suites exercise attach, detach and shared screens. diff --git a/docs/ideas.typ b/docs/ideas.typ index 8c83ab55..dfd5447a 100644 --- a/docs/ideas.typ +++ b/docs/ideas.typ @@ -1,5 +1,5 @@ // STATUS: live scratch notes, deliberately unfiled — nothing compiles this file -// (no `build.zig` step, and `docs/design.typ` does not include it); kept because +// (no `build.zig` step or other document includes it); kept because // the first two ideas are still open. The ideas themselves were last touched in // change `xmzpwklp` (2026-08-01); this header and the LANDED annotation below // are 2026-08-26 and changed no idea. diff --git a/docs/lsp-evaluation.md b/docs/lsp-evaluation.md index 184223e8..6aa58409 100644 --- a/docs/lsp-evaluation.md +++ b/docs/lsp-evaluation.md @@ -16,7 +16,7 @@ three independent implementations, one function each. C landed; see > `hxparity` **561** (`cases.jsonl` + the 80 editing extras in > `parity.jsonl`). The one number that IS kept current is the grammar count in > the Recommendation, because that is an argument about today rather than a -> measurement of then. For what the harness does TODAY, read `docs/lsp.md`. +> measurement of then. For what the harness does TODAY, read `docs/typ/building.typ` and the guide's Keys section. | | **A · stdlib** | **B · client** | **C · in-process** | |---|---|---|---| @@ -106,7 +106,7 @@ is spelled a second time in `build.zig` as `const zls_version = "0.16.1-dev+3e0d0820"`, for ZLS's own `-Dversion-string`; the semver half exists nowhere in the manifest, so the duplication cannot be removed, but a `comptime` block beside the constant `@compileError`s unless the two name the -same commit. See "Which ZLS, and which stdlib" in `docs/lsp.md`. +same commit. See `docs/typ/building.typ`. A and B were not deleted, only un-worktree'd. They remain whole commits: @@ -125,7 +125,7 @@ later, or if the ZLS coupling ever needs backing out. > status sink narrating server state onto the transient message row. B's > transport bones (socketpair, deadline-bounded writes, MSG_NOSIGNAL, > believe-the-server encoding) survive in it. See "The protocol client" in -> docs/lsp.md. +> docs/typ/building.typ. ## Recommendation (as written before the decision) diff --git a/docs/refs.yml b/docs/refs.yml deleted file mode 100644 index 47e49a69..00000000 --- a/docs/refs.yml +++ /dev/null @@ -1,82 +0,0 @@ -acme: - type: Article - title: "Acme: A User Interface for Programmers" - author: Pike, Rob - date: 1994 - parent: - type: Proceedings - title: Proceedings of the Winter 1994 USENIX Conference - page-range: 223-234 - -plan9: - type: Article - title: Plan 9 from Bell Labs - author: - - Pike, Rob - - Presotto, Dave - - Dorward, Sean - - Flandrena, Bob - - Thompson, Ken - - Trickey, Howard - - Winterbottom, Phil - date: 1995 - parent: - type: Periodical - title: Computing Systems - volume: 8 - issue: 3 - page-range: 221-254 - -netorg: - type: Article - title: The Organization of Networks in Plan 9 - author: - - Presotto, Dave - - Winterbottom, Phil - date: 1993 - parent: - type: Proceedings - title: Proceedings of the Winter 1993 USENIX Conference - page-range: 271-280 - -intro5: - type: Reference - title: intro(5) — introduction to the Plan 9 File Protocol, 9P - date: 2002 - parent: - type: Book - title: Plan 9 Programmer's Manual, Volume 1 - publisher: Bell Labs - -v9fs: - type: Article - title: "v9fs: A Plan 9 Resource Sharing Filesystem for Linux" - author: - - Van Hensbergen, Eric - - Minnich, Ron - date: 2005 - parent: - type: Proceedings - title: Proceedings of the 2005 USENIX Annual Technical Conference, FREENIX Track - -devdraw: - type: Reference - title: draw(3) — screen graphics - date: 2002 - parent: - type: Book - title: Plan 9 Programmer's Manual, Volume 1 - publisher: Bell Labs - -ad: - type: Repository - title: "ad: an adaptable text editor" - author: Ellis, Innes - url: https://github.com/sminez/ad - note: 9P server implementation in crates/ninep - -u9fs: - type: Repository - title: "u9fs: serve 9P from Unix" - publisher: Bell Labs - url: https://github.com/plan9foundation/u9fs diff --git a/docs/registry.typ b/docs/registry.typ deleted file mode 100644 index 4d386f1c..00000000 --- a/docs/registry.typ +++ /dev/null @@ -1,1209 +0,0 @@ -// THE DESIGN REGISTRY — the one place where an idea, a refactor or an open -// question lives between "someone said it" and "the tree does it". -// -// WHY A FILE AND NOT AN ISSUE TRACKER. Every claim in here cites `file:line` -// against this tree, and a tracker cannot be grepped from an editor pane, does -// not diff, and is not there when the network is not. `docs/design.typ` says -// what pardes IS; this file says what is being argued about. When an argument -// settles, its conclusion moves into `design.typ` (or into the code) and the -// entry here becomes `landed` or `rejected` — a tombstone with the reasoning -// still attached, because the expensive part of a decision is the part that -// says why the other option lost. -// -// NO PACKAGES. `design.typ` pins cetz because diagrams are genuinely painful to -// hand-roll; this file has none, so it depends on nothing and builds offline -// forever: -// -// typst compile docs/registry.typ docs/registry.pdf -// -// HOW TO ADD TO IT. Append an `#entry`. Argue inside it with `#note`. Cite with -// `#ev`. Never delete a note — flip the entry's status and let the losing -// argument stand. The dashboard on page one is generated from the entries, so -// there is no index to keep in sync. - -#set page(paper: "a4", margin: (x: 2.2cm, y: 2cm), numbering: "1") -#set text(font: "New Computer Modern", size: 9.6pt) -#set par(justify: true, leading: 0.58em) -#set heading(numbering: none) -#show heading: set block(above: 1.4em, below: 0.7em) -#show heading.where(level: 1): set text(size: 12pt) -#show heading.where(level: 2): set text(size: 10.4pt) - -#show raw.where(block: true): it => block( - width: 100%, - fill: luma(246), - inset: (x: 0.5em, y: 0.45em), - radius: 1pt, - breakable: true, - text(size: 7.8pt, it), -) -#show raw.where(block: false): set text(size: 8.8pt) -#set table(stroke: 0.4pt, inset: 0.4em) - -// --------------------------------------------------------------------------- -// machinery -// --------------------------------------------------------------------------- - -/// The lifecycle. `open` is a question nobody has taken; `investigating` has an -/// agent or a human on it; `decided` has an answer but no code; `landed` and -/// `rejected` are terminal and keep their argument; `deferred` is "correct, but -/// not until X exists" and must say what X is; `blocked` waits on someone else. -#let states = ( - open: (rgb("#8a6d3b"), "OPEN"), - investigating: (rgb("#31708f"), "LOOKING"), - decided: (rgb("#2f6f4f"), "DECIDED"), - landed: (rgb("#3c763d"), "LANDED"), - rejected: (rgb("#a94442"), "REJECTED"), - deferred: (rgb("#6f5499"), "DEFERRED"), - blocked: (rgb("#777777"), "BLOCKED"), -) - -#let chip(state) = { - let (c, label) = states.at(state) - box( - fill: c, - inset: (x: 4.5pt, y: 2.2pt), - outset: (y: 1.5pt), - radius: 2pt, - text(size: 6.4pt, fill: white, weight: "bold", tracking: 0.4pt, label), - ) -} - -/// One argument. `id` is stable forever — notes and code comments cite it, so a -/// renumber is a lie in every file that referenced the old number. -#let entry(id, title, state: "open", tags: (), body) = { - [#metadata((id: id, title: title, state: state, tags: tags)) <reg>] - block(above: 1.5em, below: 0.5em, breakable: true)[ - #block( - width: 100%, - fill: luma(242), - inset: (x: 0.6em, y: 0.45em), - radius: 2pt, - stroke: (left: 2pt + states.at(state).at(0)), - )[ - #text(weight: "bold", size: 9.6pt)[#raw(id) #h(0.5em) #title] - #h(1fr) - #chip(state) - #if tags.len() > 0 [ - #linebreak() - #text(size: 7.4pt, fill: luma(90))[#tags.join(" · ")] - ] - ] - #block(inset: (x: 0.2em, top: 0.5em))[#body] - ] -} - -/// A comment. The thread IS the design discussion; keep them in time order and -/// never edit one in place — reply to it instead. -#let note(who, date, body) = block( - width: 100%, - inset: (left: 0.9em, y: 0.35em), - stroke: (left: 1.6pt + luma(205)), - { - text(size: 7.6pt, weight: "bold", fill: luma(70))[#who] - text(size: 7.6pt, fill: luma(140))[ · #date] - linebreak() - set text(size: 9.2pt) - body - }, -) - -/// Evidence. A claim with a path is a fact; a claim without one is an opinion, -/// and this makes the difference visible at a glance. -#let ev(where, body) = block( - width: 100%, - inset: (left: 0.9em, y: 0.25em), - { - text(size: 8.2pt, fill: rgb("#2f6f4f"))[▸ ] - raw(where) - text(size: 9.2pt)[ — #body] - }, -) - -/// A question with no owner yet. Rendered so it can be found by eye when -/// scanning for something to pick up. -#let q(body) = block( - width: 100%, - inset: (x: 0.7em, y: 0.4em), - fill: rgb("#fdf6e3"), - radius: 2pt, - { text(size: 7.6pt, weight: "bold", fill: rgb("#8a6d3b"))[OPEN QUESTION]; linebreak(); body }, -) - -/// What the entry concluded. One per entry at most, and only when the state is -/// `decided`, `landed` or `rejected`. -#let verdict(body) = block( - width: 100%, - inset: (x: 0.7em, y: 0.4em), - fill: rgb("#eef5ef"), - radius: 2pt, - stroke: 0.4pt + rgb("#2f6f4f"), - { text(size: 7.6pt, weight: "bold", fill: rgb("#2f6f4f"))[VERDICT]; linebreak(); body }, -) - -// --------------------------------------------------------------------------- - -#align(center)[ - #text(size: 15pt, weight: "bold")[The Design Registry] - #v(0.3em) - #text(size: 9pt, style: "italic")[open arguments, their evidence, and how they were settled] - #v(0.2em) - #text(size: 8.5pt)[#datetime.today().display("[year]-[month]-[day]")] -] - -#v(0.8em) - -*Historical decision record.* Entries retain their original source references -and states. For the current filesystem and transport interface, use -`docs/fs.md`; FUSE and the proof-of-concept examples are no longer supported. - -#context { - let es = query(<reg>).map(e => e.value) - let order = ("open", "investigating", "blocked", "decided", "deferred", "landed", "rejected") - let counts = order - .map(s => (s, es.filter(e => e.state == s).len())) - .filter(p => p.at(1) > 0) - .map(p => [#chip(p.at(0)) #text(size: 8.4pt)[#p.at(1)]]) - align(center, counts.join(h(0.9em))) - v(0.6em) - table( - columns: (auto, 1fr, auto), - align: (left + horizon, left + horizon, right + horizon), - table.header( - text(size: 8pt, weight: "bold")[ID], - text(size: 8pt, weight: "bold")[Argument], - text(size: 8pt, weight: "bold")[State], - ), - ..es - .sorted(key: e => order.position(s => s == e.state) * 1000) - .map(e => (raw(e.id), text(size: 8.8pt)[#e.title], chip(e.state))) - .flatten() - ) -} - -#pagebreak() - -= 9P - -Born from the draft note `docs/9p.typ`. The owner's instinct: replace FUSE with -9P, replace the private unix-socket wire with 9P too, and put a small 9P server -on the ESP32-P4 that pardes talks to as a client. The instinct is recorded here -entry by entry so it can be argued with in pieces rather than accepted or -rejected whole. - -#entry("9P-1", "Where does 9P go? Two layers, and 9P is the outward face of one of them", state: "decided", tags: ("architecture", "umbrella"))[ - The umbrella. The draft conflated two things that share a socket and nothing - else. - - #ev("src/acmefs.zig:63-78")[LAYER 1, the acme control tree: request/response, nine operations, and an ABI that already says "FUSE opcodes, 9P messages and a unit test all reduce to these".] - #ev("src/detached/wire.zig:187-232")[LAYER 2, the detached wire: 19 tags carrying RLE cell frames and input, server-push, N frontends on one screen. None of them is a file.] - - #verdict[ - 9P is a TRANSPORT FOR LAYER 1 and a CARRIER FOR LAYER 2 — never a - re-encoding of either. - - Layer 1 gets 9P as a second transport beside FUSE, because the ABI was - built for it and because FUSE cannot leave the machine (`9P-13`). - - Layer 2 keeps `encodeFrame` byte for byte. 9P may carry those bytes - (`9P-12`), but the moment a frame is re-expressed as a file's contents the - design has lost the thing that makes it fast, and there is a thirty-year - demonstration of both answers (`9P-12`, devdraw against `/dev/screen`). - ] - - #note("review", "2026-08-27")[ - The reason this is worth stating as its own entry: nearly every mistake in - the draft is one layer's property asserted about the other. "Offsets are - meaningless" is true of Layer 2 and false of Layer 1. "The protocol is - private and undocumented" is true of Layer 2 and false of Layer 1, which is - acme(4). "Adding a feature means adding a message" is true of Layer 2 — - measured at 7 lines per fact (`src/detached/wire.zig`, `set_clipboard` - occurring 8 times) — and false of Layer 1, where a feature is a file. - ] -] - -#entry("9P-2", "Make the transport seam explicit before writing any 9P", state: "decided", tags: ("refactor", "cheap"))[ - `fs_service` already touches its transport through exactly three methods. - Turning `*fuse.Fs` into a ctx+vtable at three call sites is a no-behaviour - change that makes a second transport possible without deciding anything else. - - #ev("src/fs_service.zig:158-200")[`drain`/`step` call only `retry()`, `next()` and `reply()`.] - #ev("src/tty/tty.zig:1216")[call site 1.] - #ev("src/gui/gui.zig:3738")[call site 2 — and `:3841` for the headless grid harness.] - - #verdict[Do it first, independently of every other entry. ≈40 lines changed, 0 added, `fuse.Fs` is the first implementor. If 9P is never built, this costs nothing and documents the seam.] -] - -#entry("9P-3", "Direct I/O is `qid.version = 0`, not `cache=none`", state: "decided", tags: ("protocol", "correction"))[ - Under FUSE the server asserts `FOPEN_DIRECT_IO` per open and the client - cannot argue. The draft assumed 9P gives this up and compensated with advice - ("mount with `cache=none`"). It does not have to: Linux's client disables - both read and write caching for any file whose qid version is zero. - - #ev("linux/fs/9p/fid.h:52-53")[`(fid->qid.version == 0) && !(s_flags & V9FS_IGNORE_QV)` sets `P9L_DIRECT` — "no read or write cache".] - #ev("linux/fs/9p/v9fs.h:82")[`CACHE_NONE = 0` is the default anyway.] - #ev("linux/fs/9p/v9fs.c:93")[`ignoreqv` is the only opt-out, and it is explicit.] - - #verdict[A synthetic tree reports `qid.vers = 0` on every file. Server-enforced, near-parity with `FOPEN_DIRECT_IO`, and the mount advice in the draft becomes unnecessary. Plan 9 needs nothing: its cache is opt-in via `mount -c`.] -] - -#entry("9P-4", "The error ABI: 9P2000 Rerror is a string", state: "open", tags: ("protocol", "ABI"))[ - The core answers with numbers. 9P2000 answers with prose, and the Linux - client turns prose back into a number by exact string match. A miss is not - `EIO`; it is 526, which userspace prints as "Unknown error 526". - - #ev("src/acmefs.zig:152-165")[`E.PERM..E.NOSYS`, nine numeric values.] - #ev("linux/net/9p/error.c:224-243")[`p9_errstr2errno` hash lookup; on a miss `errno = ESERVERFAULT`.] - #ev("principia-softwarica/editors/acme/xfid.c:19-24")[acme's own strings — `Ebadctl`, `Ebadaddr`, `Ebadevent` — are all misses in that table.] - - Three ways out, and they are not equally good. - - / A: #[Emit Linux `strerror` text verbatim so all nine round-trip. Cheap (≈20 lines), and makes English kernel strings this project's error ABI.] - / B: #[Serve 9P2000.u, whose `Rerror` carries a numeric errno alongside the string. Costs a second dialect in the codec; buys exact errnos and keeps a human-readable string for Plan 9 clients.] - / C: #[Emit acme's strings and accept 526 on Linux. Faithful to acme, hostile to `mount -t 9p`.] - - #q[Which? Note that B also answers `9P-5`'s `statfs` gap for free, and that the draft deferred `.u`/`.L` without noticing either.] - - #note("prior-art", "2026-08-27")[ - `ad` — a Rust acme-like editor that already ships this — chose option C - without noticing. It emits nineteen lowercase prose strings: `"unknown - fid"`, `"permission denied"`, `"file not open"`, `"exclusive file already - open"`, `"invalid offset for read on directory"` - (`~/05-genizah/ad/crates/ninep/src/sansio/server.rs:25-43`). *None* of - them is in Linux's table, so under `mount -t 9p` every single error `ad` - can produce arrives as 526. It also serves `SUPPORTED_VERSION = "9P2000"` - and nothing else (`:46`), so there is no `.u` escape hatch in place. - Real implementations fall into this; it is not a theoretical trap. - ] -] - -#entry("9P-5", "Directory reads need per-fid state the core does not have", state: "decided", tags: ("protocol", "cost"))[ - This is the real offset discontinuity, and the draft missed it while - inventing a false one about `body`. - - 9P requires a directory read at offset 0 or at exactly the byte offset where - the previous read ended, and the reply must contain whole `Dir` entries. - `acmefs` treats the offset as an *entry index* and re-stages the whole - listing each call, which is right for FUSE and wrong here. - - #ev("principia-softwarica/lib_networking/lib9p/srv.c:473")[a dir read whose offset is neither 0 nor `fid->diroffset` is answered `Ebadoffset`.] - #ev("u9fs/u9fs.c:60-64")[the reference server keeps `diroffset`, a cached `dirent` and `direof` per fid.] - #ev("src/acmefs.zig:972")[`var skip = req.off;` — an entry index.] - - #verdict[Transport-side, not core-side: the 9P transport keeps a per-fid byte cursor plus the one entry that did not fit, and calls the existing `readdir` with the entry index it has counted. `acmefs.zig` is untouched. Budget ≈40 lines.] - - #note("prior-art", "2026-08-27")[ - `ad` has no cookie at all: `read_dir` returns the entire `Vec<Stat>` on - every call and the server re-serialises all of it and byte-skips the offset - (`ninep/src/sync/server.rs:203`, `sansio/server.rs:377-401`). Its - `FidMeta` is `{qid, mode}` — no dir state (`:700-703`). So it is O(N) per - read, O(N²) per directory, and `E_INVALID_OFFSET` fires only when the - offset lands mid-entry (`:388-390`), which means an arbitrary offset on an - entry boundary is silently accepted and a changing directory tears. - The 40-line budget above buys correctness `ad` does not have. - ] - - #note("built", "2026-08-27")[ - Built, and the carry-over entry turned out to be unnecessary. u9fs needs one - because `readdir(3)` has already consumed the entry it could not fit; - `acmefs` re-stages the whole listing from an index on every call and says - why (`src/acmefs.zig:1013-1016`), so an entry that does not fit is simply - not counted and the next read asks for it by index. The server keeps a - per-fid PAIR — a byte cursor for the client's rule and an entry index for - the core's — advanced together. That removes ≈300 bytes per fid and a class - of staleness bug. The ≈40-line budget held. - ] -] - -#entry("9P-6", "Is `addr` per-fid or per-window?", state: "open", tags: ("semantics", "divergence"))[ - The draft claimed per-fid and attributed it to acme. acme does the opposite, - and so does pardes today. - - #ev("principia-softwarica/editors/acme/dat.h:239")[`Range addr;` is a field of `struct Window`; every use in `xfid.c` is `w->addr`.] - #ev("src/acmefs.zig:1027-1031")[pardes copied that, deliberately, and records why: there is no fid table because FUSE puts the nodeid on every request.] - - Per-fid genuinely is better — two scripts can address one window without - colliding — but it is a divergence from acme, not a restatement of it, and it - is the one place in the whole draft that asks the *core* to grow state. Under - FUSE there is no fid to hang it on at all. - - #q[Take the divergence and pay for it (per-fid `addr` lives in the 9P transport, and the FUSE transport keeps one per mount), or keep acme's race and document it?] - - #note("prior-art", "2026-08-27")[ - Third independent source against the draft: `ad`'s address is per-BUFFER, - keyed by buffer id (`Req::SetBufferAddr{id, addr}`, - `ad/src/fsys/message.rs:77-80`). acme per-window, pardes per-pane, `ad` - per-buffer. Nobody has ever shipped it per-fid. That is not proof it is - wrong — it is proof it is a proposal, and it should be argued as one. - ] -] - -#entry("9P-7", "One controller per window, or many?", state: "open", tags: ("semantics", "divergence"))[ - The draft says `event` is exclusive-use. acme does not do this, and pardes - currently allows any number of readers. - - #ev("principia-softwarica/editors/acme/xfid.c:603-609")[acme's exclusivity is an advisory `lock` ctl verb setting `w->ctlfid`, cleared on clunk.] - #ev("principia-softwarica/editors/acme/fsys.c:537-563")[`fsysopen` checks permission bits only; a second `event` reader is not refused.] - #ev("src/acmefs.zig")[`pf.readers +|= 1` on open — a count, and the count is what suppresses button actions.] - - `DMEXCL` would make a second open fail with `EAGAIN` under Linux. That is a - tightening with a real cost: two cooperating scripts on one window stop - working, and nothing in `examples/acmefs/` was written expecting it. - - #q[Leave it a count (status quo, acme-compatible), or make it exclusive and lose multi-reader?] - - #note("prior-art", "2026-08-27")[ - `ad` DOES do what the draft describes: `event` is `FileType::EXCLUSIVE`, - i.e. QTEXCL (`ninep/src/sansio/protocol.rs:503`, added in commit - `678fbf5`). Note how it is scoped, though — enforcement is per *ClientId*, - not per fid (`sansio/server.rs:762-764`), so one client may hold two fids - on `event` and two clients may not share one window. That is a third - position between "a count" and "one fid", and it is probably the right one: - it stops two unrelated scripts fighting without breaking a script that - opens the file twice. - ] -] - -#entry("9P-8", "`pty/` files — the best idea in the draft, and unrelated to 9P", state: "open", tags: ("feature", "transport-independent"))[ - A script today can write a terminal pane's `body` (it becomes a pty write) - and read its rendered scrollback. It cannot spawn a terminal, resize one, or - signal one. The draft's `pty/{data,ctl,status}` fixes that, and none of it - needs 9P — it is `ctl` verbs and two new `PaneFile` variants. - - #ev("src/host.zig:80-82")[`push_spawn` and `push_pty_resize` already exist as effects, so `exec` and `winsize` are two existing effects with a name.] - #ev("src/acmefs.zig")[the `Verb` table has no pty verb; `PaneFile` has no pty entry.] - - `sig INT` is the only genuinely new capability — there is no `kill` anywhere - in `host_io.zig`. - - #verdict[Not blocked on anything. Build it in the FUSE tree now; a 9P transport inherits it for free. Sequencing it *after* 9P would be putting the transport before the feature.] - - #note("prior-art", "2026-08-27")[ - Worth knowing before building it: there is *no prior art anywhere*. acme - has no pty files; `ad` has none either — its tree is - `{ctl, minibuffer, scratch, log, buffers/…}` and contains no terminal - surface at all (`ad/src/fsys/mod.rs:17-32`). So `pty/` is a genuinely new - interface, which cuts both ways: nobody has made these mistakes for us, and - nobody's scripts already expect a particular spelling. Being first is a - reason to keep it small — `data`, `ctl`, `status`, and no more. - ] -] - -#entry("9P-9", "Naming sessions: `aname` or a top-level directory", state: "open", tags: ("protocol", "naming"))[ - Verified: `aname` works, and the draft's mount line is valid verbatim. - - #ev("linux/fs/9p/v9fs.c:72-90")[`aname` parses to `Opt_remotename`; `version=9p2000` selects `p9_proto_legacy`.] - #ev("linux/fs/9p/vfs_super.c:340")[`port` defaults to 564, so the draft's command line needs no `port=`.] - #ev("u9fs/u9fs.c:409-420")[u9fs uses `aname` to pick a tree, so there is precedent for exactly this use.] - - Against it: pardes has no multi-session concept in the core at all today — - `--detach=work` names a *socket*, not a tree. A top-level directory needs no - protocol feature and works with clients that ignore `aname`. - - #q[Is this question even live before `9P-12` settles? A per-session socket already names a session; `aname` matters only if one listener serves many.] - - #note("prior-art", "2026-08-27")[ - `ad` does not use `aname` either: `Server::new` installs a single anonymous - root `""` (`ninep/src/sansio/server.rs:91-95`), and multi-session is the - SOCKET name, `ad-<pid>`, with discovery through a `list_open_sessions` - helper (`ad/src/fsys/mod.rs:158-159`). That is exactly pardes's - `--detach=work` shape. Two implementations independently reaching for a - named socket over `aname` is evidence about which one people actually - build. - ] -] - -#entry("9P-10", "Aggregation: the prefix router", state: "deferred", tags: ("architecture", "premature"))[ - The draft's longest technical section, and its own §11 concludes the - aggregate is not worth building before a second machine exists. That verdict - is correct and also covers the client half. - - What the section understates: a proxied `Twalk` cannot be answered until the - remote `Rwalk` arrives, so "the fid table maps our fid to a pair" is not the - entire proxy — it needs per-tag continuations, remote↔local tag remapping, - `Tflush` forwarding and fid invalidation on connection death. - - #ev("src/detached/wire.zig:64-70")[a round trip inside `update` is the one thing the transport must never do.] - #ev("src/detached/server.zig:238-241")[the daemon is one `poll(2)` over 50 slots, and `:565-569` records that it has no worker pool.] - - #verdict[Deferred until a second machine exists AND `9P-12` has settled, because the proxy's shape depends entirely on which layer 9P occupies. Reopen with a named use case, not with an architecture.] - - #note("unblocked", "2026-08-27")[ - *Reopen this.* The deferral rested on one objection and steps 4 and 5 - dissolved it without meaning to. - - The objection was that a proxied `Twalk` cannot be answered until the remote - `Rwalk` arrives, so the router needs per-tag continuations, tag remapping, - flush forwarding and fid invalidation — machinery a core that must not block - has nowhere to put. Three of those four now exist as shipped primitives: - - #ev("src/9p.zig")[`Server.retry()` re-offers a parked request oldest-first and `reply()` RE-PARKS it when the answer is still `.again`. The park table is therefore a continuation store that already survives across frames, and it is the one the FUSE mount has used all along.] - #ev("src/9p.zig")[`Client` is sans-io: `submit()` hands back a tag and never waits, `take()` returns a completed operation or null. Its tag table is indexed BY the tag, so attributing an out-of-order reply is one bounds check — which is the remapping the objection was about.] - - So a proxy is now a loop, not a subsystem: `Server.next()` gives a request, - `Client.submit()` forwards it, the answer is `.again`, and each frame - `Server.retry()` offers it back until `Client.take()` completes and the real - reply goes out. Nothing blocks, nothing is added to the core, and the - daemon's poll set grows by one descriptor per upstream. - - What is still genuinely missing is fid invalidation on a connection that - dies mid-walk, and a decision about whether `Tflush` forwards or is answered - locally. Both are small and neither is architectural. - - Re-cost before building: the "several hundred lines" the draft claimed was - wrong in the other direction too. Measure it against the ≈200 lines this - loop looks like, and against `src/fs9_client.zig`'s 622, which already does - the connect-and-pump half. - ] -] - -#entry("9P-11", "A 9P server on the ESP32-P4 — real, cheap, and a second firmware image", state: "decided", tags: ("board", "motivating-case"))[ - The owner's motivating case, and the entry that changed the most under - measurement. Both the draft and the first review were wrong about it, in - opposite directions. - - The draft was wrong about what the board IS: it describes a machine running - "a 9P server and nothing else", and today the P4 runs the whole editor - (`src/esp32p4.zig:264-300`, 2 of 21 vtable methods at `:930-934`). - - The review was wrong about what the board CAN AFFORD, because it quoted a - stale number. - - #ev("src/esp32p4.zig:470-473")[the "9,128 bytes free at 80×24" comment predates `direct_emit` (`:870`), which sized vaxis's two shadow grids to one cell.] - #ev("05-zig-p4/experiments/report.typ:848-849,877-880")[measured after that change: "the heap now reports 336 KB free at every geometry tried, including ones that used to fail outright… the heap has 336 KB spare while `.bss` runs out". The binding resource is the 240 KiB low L2MEM, not the 384 KiB heap.] - - A 9P server's RAM, costed from real components: two msize buffers at 4,096 - (u9fs uses three — `rxbuf`, `txbuf`, `databuf`, `u9fs.c:1814-1816` — a - minimal server needs two) = 8,192 B; a FIXED-ARRAY fid table of 32 entries × - 16 B (`fid`, `qid.path`, `mode`, `diroffset`) = 512 B; codec scratch with - `P9_ERRMAX` = 128 B. *Total 8,832 B, or 2.5% of free heap.* - - #ev("src/esp32p4/input_rescue.zig:52")[and a 4,096-byte reassembly buffer already exists on this exact UART, measured: "4,096 bytes in a single write arrive intact, and past that the loss is counted rather than silent".] - #ev("linux/net/9p/client.c:840-843,908-910")[the 4,096 floor is imposed by the LINUX KERNEL and by nothing else. Plan 9's devmnt, plan9port's `9p` and pardes's own client accept a 512-byte msize, which halves the buffers to 1 KiB.] - - Flash: the editor image is 809,536 B of a 1,536,000 B partition - (`report.typ:375-376`), leaving 726,464 B. A 9P-only image is ≈23,870 B of - platform plus ≈15 KiB of server ≈ *39 KiB, 2.5% of the partition*. - - #ev("build.zig:1182-1235")[a second board image is ALREADY expressible: `esp32p4-test` builds its own executable, `ImageStep`, `FlashStep` and run step in 54 lines, and deliberately links no pardes object. That is the shape.] - #ev("src/acmefs.zig")[and the semantics layer already compiles for riscv32: `llvm-nm` finds 21,548 B across 17 `acmefs.*` symbols in `zig-out/pardes-esp32p4.o`.] - - #verdict[ - Buildable, and much cheaper than anyone assumed. But it is a SECOND - FIRMWARE IMAGE, not a second role for this one: the editor owns UART0 - bidirectionally (`esp32p4.zig:611`, `uart.zig:43-44,122-130`) and JP1 - exposes no second P4 UART (`board9p.Header`). The board is either - an editor or a filesystem at any one time. Say that plainly rather than - implying both. - ] - - #note("review", "2026-08-27")[ - The reframing the draft misses: the board already exposes `Peek`, `Poke`, - `Hexdump` and `Gpio` as acme words (`src/builtins.zig`, `Board` namespace). - All four cap at 4,096 bytes - per command and the cap's stated reason is the 115200 console. So the whole - 2³² address space is already reachable — by *typing a word into a tag*, - with the answer landing in an output pane. Nothing is machine-readable and - nothing is remote. A 9P tree is that same capability with names instead of - verbs, and `mem/`, `gpio/pinout` and `prof` are backed by functions that - exist today (`builtins.Board.readWord`, `builtins.Board.gpio`, - `pardes_esp32p4_frame_prof` at `esp32p4.zig:985`, already exported). - Four more files need one new C-ABI extern each; four have no - implementation at all. That inventory belongs in the note, not a wishlist. - ] - - #q[Should the parked second RISC-V core own the 9P server? `report.typ:552-563` says it needs four register writes plus a trampoline, shares one L1 D-cache so a lock-free ring needs only fences, and that giving core 1 the UART "eliminates the silent input loss". That is the one arrangement where the board serves 9P *and* keeps the editor. Uncosted.] - - #note("built", "2026-08-27")[ - *The RAM figure above is wrong and the built one is 21,776 B, not 8,832.* - Measured from the real structs: `Fid` is 64 B × 32 = 2,048, `Slot` is 208 B - × 32 = 6,656, and the whole `Server` is 9,488 B before buffers; a 4,096 - msize adds `in` 4,096 + `out` 8,192. - - Three reasons, all of them things the estimate did not know. `out` is TWO - msize — one message being written, one being built — which is what makes - every reply infallible and removes "can I write yet" from the whole file. A - fid entry is 64 B and not 16, because `Rstat` carries a NAME that a node id - does not, plus the open handle and the two-coordinate cursor. And the - estimate did not cost the park table at all, which is 6,656 B of the total. - - It still fits with room: 6.5% of the board's ≈336 KB free heap, and about - 11 KB in total at the 512-byte msize Plan 9 accepts. A test bounds `Fid` and - `Slot` so that a change to either shows up as a diff in the board's budget - rather than as a surprise on the die. - ] -] - -#entry("BOARD-1", "Raise UART0 to 921600 before quoting any board latency", state: "decided", tags: ("board", "cheap", "prerequisite"))[ - Every board 9P latency figure is eight times worse than it needs to be, for - no reason but that the bootloader left the divider alone. - - #ev("src/esp32p4/uart.zig:35-38")[the firmware never programs the divider; 115200 is inherited.] - #ev("05-zig-p4/src/hal/uart.zig:321-333")[`setBaudrate` and `divider` already exist and `reset()`'s refusal of instance 0 does not apply to them.] - #ev("05-zig-p4/experiments/report.typ:540-544")[921600 is one `UART_CLKDIV_SYNC` write on the existing 40 MHz XTAL — int 43, frag 6, +0.064% error. 2 Mbaud is representable but this CH340 is unreliable there, corroborated by the flasher at `build.zig:1136-1138`.] - - #verdict[ - One register write, 86.8 µs → 10.85 µs per byte. A warm - `cat /mnt/board/gpio/2/value` goes from 10.8 ms to 1.35 ms; a 4 KiB `Tread` - from 358 ms to 45 ms. Do it first, and re-measure everything after. - The hazard the same files record: writing the console UART's divider is - adjacent to what bricks the board, and the host must be reopened in step. - ] -] -#entry("9P-12", "Should 9P replace the private wire protocol? Half of it, and not the half you would guess", state: "decided", tags: ("architecture", "the-real-question"))[ - Six layerings were costed. The finding that settles it is that layering is - not a cost decision at all. - - #ev("src/detached/wire.zig")[1705 lines = 1092 code + 613 tests, of which only 303 are FRAME and 789 are TRANSPORT: bounds, tags, message structs, framing, codecs.] - #ev("src/detached/wire.zig:1401,1412")[a 56×14 full frame is 6,298 B; a one-keystroke diff is 56 B. Ratio over 100.] - #ev("src/detached/server.zig:1102-1110")[skip-slow is three lines: `if (c.out.items.len != 0) continue;` — skip the frame and leave the mirror alone, so the next frame diffs against what the client really has.] - - A base-9P2000 codec (≈450), a dispatcher onto `acmefs.Op` (≈400) and a fid - table (≈120) come to ≈970 lines that are IDENTICAL in all six layerings. - Layering moves ±150. So the question is reach and semantics, not lines. - - *Does 9P subsume the wire's job?* The transport half, yes, with shipped - precedent: clipboard becomes `snarf` (`rio/fsys.c:61`), input becomes - `mouse`/`cons` — and rio's `mouse` is exclusive-open with a blocking read and - a flush (`rio/xfid.c:167-173,638-660`), which is the `event` idiom exactly. - The frame half, no: 9P has no server-initiated message. - - *Is "the frame is a file" a category error?* Only if you mean push. Two - shipped counter-forms exist and they disagree with each other, which is the - most useful thing in this entry: - - #ev("plan9/rio/fsys.c:53, lens.c:281-282")[`/dev/screen` is an offset-addressed raster you POLL — `lens` seeks to a scanline and repaints on mouse events because it never learns the screen changed. Real, and precisely why nobody runs a remote rio through it.] - #ev("drawterm-9front/kern/devdraw.c:1574, include/draw.h:55")[`/dev/draw/N/data` is a PRIVATE BINARY COMMAND STREAM batched into 8,000-byte buffers and flushed as one `Twrite`. Real, and how remote display has actually shipped for thirty years.] - #ev("drawterm-9front/cpu.c:149,191,251,359")[and it inverts direction: the DISPLAY exports its devices and the APPLICATION mounts them. The frame travels as a client's `Twrite`, never as a server's push.] - - pardes's codec is already devdraw's shape. `wire.zig`'s 5-byte length prefix - (`:145`, `framed()` `:564`) makes an `encodeFrame` message a legal 9P payload - with zero new framing. - - #verdict[ - *Superseded in part by `9P-19`.* The reasoning below stands; the conclusion - moved. O3's tunnel was chosen because it was the only option that reached - the board without sockets. `9P-19` shows the tunnel is unnecessary: the - link simply carries 9P, with the FRONTEND as the server and the core as the - client, and the frame travels as the client's `Twrite` to `screen`. That is - O1 done the way it has actually shipped for thirty years, and it dissolves - the objection recorded here. - - What survives unchanged: *O4* (wire inside 9P) costs +34 B per frame, +61% - on a 56-byte diff. *O5* (a standalone listener) remains correct and is - independently worth building — it is the scripting face and it composes - with anything. - - What was wrongly rejected: *O6*. "The frontend serves, the core is a - client" was rejected here for making the core write and match tags inside - `update` and for foreclosing the freestanding targets. Both are wrong. The - core need not wait for `Rwrite` — 9P permits many outstanding tags - (`linux/net/9p/client.c:194-199`, `plan9/devmnt.c:783-800`) with no - in-order reply requirement — and the freestanding targets have a byte - stream even though they have no sockets, which is all 9P asks for. - O6 is the destination. See `9P-19`. - ] - - #note("review", "2026-08-27")[ - One honest caveat against O3: `exportfs` carries 9P and nothing else — its - framing is 9P's own 4-byte size prefix (`exportfs.c:434,476`) — so - multiplexing a second protocol beside 9P on one link has *no* Plan 9 - precedent. It is not hard, but we would be first, and "the transport - becomes someone else's problem" stops being true for that link. - ] -] - -#entry("9P-13", "9P cannot replace FUSE on Linux; it can only join it", state: "decided", tags: ("portability", "privilege", "decisive"))[ - The strongest argument for 9P was that it needs no kernel and no privileged - mount helper: 281 non-test lines of `fuse.zig` exist only to obtain a mount - an unprivileged user is not allowed to make. - - #ev("src/fuse.zig:497-703")[207 lines: `_FUSE_COMMFD`, socketpair, `CMSG_*`/`SCM_RIGHTS`, fork/execve/waitpid of the setuid `fusermount3`, environ rebuild.] - #ev("src/fuse.zig:752-773")[22 more for the helper's unmount, `:933-978` 46 for the mount call, `:1718-1723` 6 for `clearCloexec`. 281 total, measured.] - #ev("src/fuse.zig:64")[and all of it is Linux-only, so macOS (14/21 vtable) and web (6/21) have no control filesystem at all.] - - That argument is dead. The kernel grades mount privilege by a per-filesystem - flag, and the two filesystems are on opposite sides of it. - - #ev("linux/fs/super.c:694-700")[`mount_capable`: without `FS_USERNS_MOUNT` the check is `capable(CAP_SYS_ADMIN)` — and `capable()` is `ns_capable(&init_user_ns, …)`, i.e. root in the INITIAL namespace, not the caller's.] - #ev("linux/fs/9p/vfs_super.c:362")[`.fs_flags = FS_RENAME_DOES_D_MOVE` — v9fs does not set `FS_USERNS_MOUNT`.] - #ev("linux/fs/fuse/inode.c:2004")[`.fs_flags = FS_HAS_SUBTYPE | FS_USERNS_MOUNT | FS_ALLOW_IDMAP` — FUSE does.] - - So `mount -t 9p` costs real root, unconditionally: a fresh - `unshare(CLONE_NEWUSER|CLONE_NEWNS)` does not help, because `mount_capable` - falls back to the init namespace for exactly the filesystems that lack the - flag. `trans=fd` does not help either — it lets an unprivileged process own - the connected socket, but `mount(2)` is checked before the transport is ever - consulted (`linux/fs/namespace.c:3838`). And `9pfuse`, the usual escape, - is FUSE: it needs `fusermount` in `PATH` and reimports the whole setuid dance - in someone else's process. - - #verdict[ - The 281 lines are not overhead. They buy an *unprivileged pathname*, which - is a capability 9P has no way to provide on Linux. `fuse.zig` stays. - - This does not weaken 9P; it relocates it. The two are complementary faces - of one `acmefs.handle`: FUSE is the PATHNAME face — local, unprivileged, - Linux, for `grep` and `make`. 9P is the NETWORK AND IPC face — remote, - unprivileged, every platform, for pardes's own client, for the board, for - the daemon, and for anyone with root who wants `mount -t 9p`. - - Everything downstream follows from this split, and `9P-2` is what makes it - cost nothing. - ] - - #note("review", "2026-08-27")[ - Worth being precise about what survived. The draft's §6 has two halves and - only one of them died. "The client half needs no mount… no kernel - involvement, no privilege" is TRUE and remains the best paragraph in the - document. "The mount is for other people's programs… and it is sufficient" - is where the root requirement lands, and it is not sufficient — on Linux - that mount is strictly more privileged than the one we already have. - ] -] - -#pagebreak() - -= Corrections - -Small, certain, and independent of every argument above. - -#entry("FIX-1", "`acmefs.zig` is wrong about 9P truncation", state: "decided", tags: ("comment", "one-line"))[ - The comment justifying `setattr` says 9P has no truncate-on-open and that - nothing in acme answers a `Twstat` carrying a length. Both halves are wrong. - - #ev("src/acmefs.zig:1097-1100")[the claim.] - #ev("u9fs/plan9.h:150")[`#define OTRUNC 16` — base 9P2000, used at `u9fs.c:1578` and `:1682`.] - #ev("u9fs/u9fs.c:1047")[`Twstat` with a length calls `truncate(2)`; that is how 9P truncates.] - - #verdict[Rewrite the comment. The real reason `setattr` exists is that Linux strips `O_TRUNC` from the OPEN when `FUSE_ATOMIC_O_TRUNC` is not negotiated, which is a FUSE fact and stands on its own without the false claim about 9P. Under a 9P transport, `> body` arrives as `Topen` with `OTRUNC` or as `Twstat`, and maps onto the same `Req.truncate`.] -] - -#entry("DOC-1", "`design.typ` says \"No 9P\"", state: "open", tags: ("docs", "consistency"))[ - The project has already recorded a decision against this, under *What is - deliberately absent*, and the draft neither cites nor rebuts it. - - #ev("docs/design.typ:1969-1971")[«No 9P. The library boundary is exactly where acme put the file server, and the FUSE mount is already that server with a Linux transport instead of a 9P one; a sixth shell could serve `Surface` and `Event` over 9P without touching the core.»] - - Read closely, that paragraph is not hostile — it says *not built*, and its - second clause proposes something stronger than the draft does: serving - `Surface` and `Event`, i.e. `9P-12`'s option 1, which the project apparently - considered plausible enough to write down. - - #q[When `9P-1` settles, this paragraph is rewritten rather than deleted — it should say which layer 9P occupies and why the other one was left alone.] -] - -#entry("9P-14", "Do the daemon's filesystem first, with 16 lines and no 9P", state: "decided", tags: ("sequencing", "cheap", "do-first"))[ - The strongest *stated* motive for putting 9P in the daemon is that a - detached session has no control filesystem. That is true, and it is not a - protocol problem. - - #ev("src/detached/server.zig:563-570")[«no `push_fs_reply`, because this process mounted no /dev/fuse». The daemon simply never calls `fs_service.start`.] - - The fix, counted: a `Source.fuse` variant (1 line), an `fs` field (1), - `fs_service.start` (1), `fsReply` copied verbatim from `tty.zig:1455-1458` - (4), a vtable entry (1), a pollfd and dispatch arm (6), `drain` (2). - *≈16 lines.* And it is *better* there than on the desktop hosts: the - daemon's own `poll(2)` covers `/dev/fuse` directly, so it needs no wake - thread at all (`src/fs_service.zig:127-131`). - - #verdict[ - Build this before deciding anything else. It costs sixteen lines, it - delivers the feature people actually want, and it removes a motive from the - 9P argument so that argument is decided on its merits. If it turns out to - be all anyone needed, that is a good outcome, not a wasted one. - ] -] - -#entry("9P-15", "The honest cost of a 9P stack is ≈1,600-1,900 lines, not ≈770", state: "decided", tags: ("cost", "estimate"))[ - Two independent implementations were measured rather than guessed, and they - agree closely. - - #ev("~/05-genizah/ad/crates/ninep")[12,296 lines total; the irreducible 9P2000 SERVER core is 2,321 non-blank non-comment: codec 704, `Stat`/`Perm`/`Mode`/`WStat` 437, session+fid+flush 615, loop+handlers 474. Fid table alone is 57 lines; the server loop 141; the flush path ≈58; directory reads 50; tests 3,669, i.e. 31%.] - #ev("~/05-genizah/u9fs/u9fs.c")[1,838 lines — NOT the 6,149 the first review quoted, which counted a whole plan9-libc substitute. Its codec, `convM2S` + `convS2M`, is 805.] - - Codec ≈700-800 and server logic ≈1,500-1,900, from both. The earlier - ≈770-line figure was the size of one component. - - #verdict[ - Budget ≈1,600-1,900 core lines plus ≈500 of tests, and say so out loud, - because it *breaks the house rule* (`docs/design.typ:1957-1960`: a change - leaves the file it touches no longer than it found it) unless something is - retired in exchange. `9P-13` says `fuse.zig` cannot be the thing retired. - So this is net growth, and it has to be justified by reach — the board, the - network, macOS, web — not by simplification. - ] - - #note("review", "2026-08-27")[ - Two mitigations that are real. First, ≈970 of those lines are identical in - every layering (`9P-12`), so none of it is at risk from the layering - decision. Second, 296 of ninep's 704 codec lines are macro-generated - (`protocol.rs:809-1154`); Zig `comptime` over an exhaustive message enum - should compress that at least as well, and the message set is the one part - of 9P that never changes. Against that: `ad` needed `UnsafeCell` plus - `unsafe impl Send/Sync` to get zero-allocation parsing - (`protocol.rs:67-76`), which Zig gives for free — so the *hard* part of - their codec is not a cost we inherit. - ] -] - -#entry("9P-16", "Tflush is a park-table lookup, and this is where we beat the prior art", state: "decided", tags: ("protocol", "advantage"))[ - The draft says `Tflush` is the most common omission in hand-written 9P - servers. It understates the problem: the common failure is having it and - still hanging. - - #ev("~/05-genizah/ad/crates/ninep/src/sansio/server.rs:767-819")[`ad` gets the ORDERING right in 39 lines — an Rflush for an in-flight tag is chained and sent only after the original replies, matching `lib9p/srv.c:241-266`.] - #ev("~/05-genizah/ad/crates/ninep/src/sync/server.rs:163-164")[and then `Serve9p::flush` defaults to a no-op, which `ad` never overrides. A client flushing a blocked `event` read gets no Rflush until an unrelated editor event happens to arrive. Interrupts and unmounts hang exactly as the draft warns, *despite* Tflush being "implemented".] - - #verdict[ - pardes is structurally better placed than either reference. The 32-slot - park table (`src/fuse.zig:789`) is already keyed per outstanding request, so - `Tflush` is a lookup, a reply to the original, and a reply to the flush — - the same path `FUSE_INTERRUPT` already takes, which is tested - (`src/fuse.zig`: "interrupt answers the original with EINTR and drops it"). - Implement it against the table, not against a filesystem callback, and the - class of bug `ad` shipped is unrepresentable. - ] -] - -#entry("9P-17", "Clamp Rread to the client's count, or Linux hard-fails the read", state: "decided", tags: ("protocol", "correctness", "trap"))[ - A trap with no analogue in FUSE, which is why nobody looks for it. - - #ev("linux/net/9p/client.c:1475-1479")[`if (rsize < received) { pr_err("bogus RREAD count"); *err = -EIO; }` — an Rread longer than the `count` asked for is a HARD ERROR, not a truncation.] - #ev("~/05-genizah/ad/crates/ninep/src/fsys/event.rs:116-122")[`ad` clamps replies to `msize` (`protocol.rs:1011-1018`) but never to `count`, and its `event` file concatenates every drained event and returns the lot. Under a kernel mount that is `-EIO`.] - - #verdict[ - Every reply path clamps to `min(count, msize - 11)`. This interacts with a - decision already made the other way: `acmefs` refuses a read smaller than - one event record with `EINVAL`, on the grounds that half a record silently - desynchronises a client (`docs/acme-fs.md`). That rule stays and is now - load-bearing for a second reason — it is what makes "clamp to count" - safe on `event`, because a record is either whole or refused. - ] -] - -#entry("9P-18", "The spec-bug checklist, taken from someone else's git history", state: "open", tags: ("testing", "gift"))[ - `ad` shipped and then fixed each of these. They are the test list for - `src/ninep.zig`, in roughly the order they will bite. - - #ev("~/05-genizah/ad/crates/ninep/src/sansio/server.rs:335-338")[the scar they left in the source: «Spec: first element failure must be Rerror, not Rwalk with zero qids».] - - Partial walks (`09334ce`); `Tcreate` of `".."` (`bcb05d9`); write-then-remove - (`1c8ae96`); create permission masking (`3ccc461`); open with truncate - (`f0c71be`); remove-on-close (`6471cd3`); the root qid's name being `"/"` - (`64f2f4b`); the default `aname` (`b3a2c8b`, `ce7ebc6`); exclusive files - (`135190c`); clearing the fid cache on clunk (`113b053`); fid open state - (`22041ab`); flush tracking (`7170e6a`, `5a6b5bf`). - - #q[Most of these are `Tcreate`/`Tremove` bugs, and a synthetic tree could simply answer `Rerror` to both — the draft's §9 already argues our trees are invented and have no cases we did not choose. Does `new/` need real `Tcreate`, or is walk-to-create enough as it is under FUSE?] - - #note("built", "2026-08-27")[ - *Answered, and most of the list is moot.* `new/` creates a pane on WALK - (`src/acmefs.zig:901-919`), so the capability exists and is not spelled - `Tcreate`. The server answers `Rerror` to both `Tcreate` and `Tremove`, - which takes the create-permission-masking, create-`..`, remove-on-close and - write-then-remove bugs off the table entirely — four of `ad`'s twelve. - `Tremove` still clunks the fid first, because remove(5) says the fid is - invalid even if the remove fails. - - Of the rest, the partial-walk rule and the flush tracking are implemented - and tested by name; the root qid's name is `"/"`; the fid cache is cleared - on clunk and the release the core is owed is issued there. - ] -] - -#entry("FIX-2", "The board's free-heap comment is two refactors out of date", state: "decided", tags: ("comment", "board", "one-line"))[ - #ev("src/esp32p4.zig:470-473")[says 9,128 bytes free at 80×24. That was measured before `direct_emit` (`:870`) sized vaxis's two shadow grids to one cell.] - #ev("05-zig-p4/experiments/report.typ:848-849,877-880")[the heap now reports ≈336 KB free, and `.bss` in the 240 KiB low L2MEM is the binding constraint instead.] - - #verdict[ - Fix the comment and name the new constraint, because the stale number was - load-bearing in an architecture review — it is the reason a 9P server on - the board looked impossible when it costs 2.5% of free heap. A stale - measurement in a comment is worse than none: it gets quoted. - ] - - #q[The `report.typ` figure of "336 KB at every geometry tried" cannot be literally true across the 62,480-byte swing between 56×14 and 80×24 at 55 B/cell. One boot and one `MARK PARDES_HEAP` line closes it.] -] - -#pagebreak() - -= Direction - -The four entries below were written after the owner rejected the first -synthesis as directionless and stated the goal himself: *"make it overall -simpler, and more transparent to scripting and integrating over the network, -and that could unlock some interesting things like chaining a pardes to -another — right now these kinds of things are complex and ad hoc."* - -Three of those four are achievable. One is not, and `9P-20` says which. - -#entry("9P-19", "Bidirectional 9P is one role per CONNECTION, not one role per message", state: "decided", tags: ("protocol", "mechanism", "correction"))[ - The owner's objection to the first review: *"you said the server can't - initiate a message, but since the daemon and the app both could have a client - and a server, they could communicate bidirectionally."* He is right, and the - mechanism is better than either of us described. - - The tempting reading is a double-role link: both ends run a server and a - client on one descriptor, demuxing on the message-type parity, which 9P makes - possible for free. - - #ev("u9fs/fcall.h")[`Tversion = 100, Rversion, Tauth = 102, Rauth, …` — T is even, R is odd, so a receiver can always tell a request it must serve from a reply it is owed.] - - That reading is available and *nobody has ever built it*. - - #ev("drawterm-9front/exportfs/io.c:132-163")[`io()` is a pure server loop: read a T-message, dispatch through `fcalls[type]`, reply. It never emits a message that is not a reply. Plan 9's `exportfs` is the same (`exportfs.c:510-515`), u9fs is the same, and v9fs and `devmnt` are pure clients. No implementation anywhere demuxes two roles on one descriptor.] - - What ships instead is better, because it needs no invention at all: - - #ev("drawterm-9front/cpu.c:119-192")[drawterm DIALS OUT to the cpu server, writes a shell script, and then calls `exportfs(fd, fd)` — *the dialer becomes the server on the socket it dialed*.] - #ev("principia-softwarica/networking/misc/cpu.c:448,459-463")[the script the remote runs is `mount -nc /fd/0 /mnt/term`, then it opens `/mnt/term/dev/cons` as its stdin and stdout. The remote is the CLIENT.] - #ev("drawterm-9front/kern/devdraw.c:1323-1326")[so when the remote application draws, the bytes cross as *the application's `Twrite`* to `/dev/draw/N/data`, and the terminal — the machine with the screen — executes them as a server.] - - One descriptor, roles fixed per side, data flowing both ways. There is no - server push because none is needed: *the frame producer is the client.* - - #verdict[ - The rule for pardes, and it is the whole architecture in three lines: - - / display: #[the FRONTEND is the server (it owns the screen), the CORE is the client. The core `Twrite`s frames to `screen` and blocking-`Tread`s `input`. This is `cpu` verbatim.] - / session: #[the CORE is the server (it owns the windows), scripts and other instances are clients.] - / devices: #[the BOARD is the server (it owns the memory and the pads), the core is the client.] - - Three connections, one role each, and the program contains both halves. Do - NOT build a double-role single-descriptor link: no precedent, a `Tversion` - ordering hazard where each side must answer the peer's version while - awaiting its own, and `trans=fd` wants separate read and write descriptors - anyway. - ] - - #note("mechanism", "2026-08-27")[ - Two numbers that make the display path credible, both of which the first - review got wrong by assuming a push. Chunking is a non-issue: payload per - message is `msize − P9_IOHDRSZ` = 131,072 B at Linux's default - (`client.h:23`, `9p.h:364`), so every real frame — 20 B, a 56 B keystroke - diff, a 6,298 B full board frame, a 27 KB desktop frame — is ONE `Twrite`, - and even the 1,703,936 B worst case is 13. And the core need not block: - tags are allocated per outstanding request with no in-order reply - requirement (`client.c:194-199`, `devmnt.c:783-800`, and `exportfs` - deliberately answers out of order via slave procs, `exportsrv.c:408-520`). - pardes needs no slave procs — `Status.again` plus the park table is the - same thing done single-threaded. - ] - - #note("mechanism", "2026-08-27")[ - Copy the file discipline rather than inventing one. `/dev/draw/N/data` is a - write-only batched binary command stream (`devdraw.c:1323-1326`) with an - exclusive `ctl` (`:1047-1051`); `/dev/mouse` is an exclusive single-reader - file whose read blocks until something happens (`devmouse.c:96-98,150-158`). - That is exactly `screen` and `input`, already designed, already debugged, - and already the shape `event` has in `acmefs`. - ] -] - -#entry("9P-20", "\"Simpler\" is false in lines and true in concepts — argue reach instead", state: "decided", tags: ("cost", "honesty", "framing"))[ - The owner's first criterion was that 9P make the codebase *overall simpler*. - Measured, it does not. This entry exists so that nobody argues otherwise - later, including us. - - The baseline: *nine* IPC mechanisms, *four* framings, *three* discovery - schemes, *two* version-negotiation schemes and one mechanism with none. - - #ev("src/nested.zig")[743 lines to deliver ONE line. By region: 255 transport, 158 ancestor discovery, 43 `sendLook`, 232 tests, 30 header — and the FEATURE is ≈12 lines (`:301-306` format, `:507` filter). Ratio feature-to-transport ≈ 1:24.] - #ev("src/detached/wire.zig")[contains ZERO syscalls — no `socket`, `bind`, `accept`, `poll`, `read` or `write` anywhere. It is a pure codec into caller buffers, so calling 789 of its lines "transport" was wrong: they are the definition of what may be said. 9P replaces its 5-byte length prefix with a 4-byte one.] - - Six concepts are genuinely duplicated across the mechanisms, and the - duplication is real: endpoint-path derivation 82 lines across 6 copies; - directory create-and-vet 105 across 6; stale sweeping 112 across 3; - bind/listen/accept 213 across 3; "which instance?" 231 across 3; version - negotiation 56 across 2. *799 lines, of which ≈330 is recoverable.* - - #ev("src/fs_service.zig:37-55")[the comment there is right on the facts and wrong on the conclusion: `parentFrom` and `nested.socketDir` differ ONLY by appending `/pardes` under `$XDG_RUNTIME_DIR` and coincide verbatim under the HOME fallback. And there are two liveness oracles for one question — `kill(0)`/ESRCH at `nested.zig:401` and `fuse.zig:742`, `connect`/ECONNREFUSED at `server.zig:1940-1942`.] - - #verdict[ - Could delete ≈850 lines (`nested.zig`'s transport and its socket tests, - 518; the net collapse of duplicated plumbing, ≈330). Could not delete - ≈7,700 — `fuse.zig` 2,709 (`9P-13`), `wire.zig` 1,705, the daemon's host - half, the 158-line ancestor walk that answers a question 9P has no message - for, and every kernel interface that was never a protocol choice (inotify, - pty, pipes, `mkstemp`). - - Against a 1,600-1,900-line 9P stack (`9P-15`): *net growth of +750 to - +1,050 lines.* - - So: strike "simpler" from every argument. What IS true, and is worth - saying, is that the CONCEPT count falls — four framings to two, three - discovery schemes to two, two version schemes to one, six duplicated - concepts to one copy each. Fewer ideas, more lines. Say exactly that. - ] - - #note("contra", "2026-08-27")[ - The strongest form of the objection, and it should stay on the record: the - three mechanisms do not share a problem. `nested`'s format is twelve lines - *because* it reuses a language pardes already speaks — the file says so at - `:14-16`, "no serialization, nothing to version". `wire`'s codec is 1,167 - lines because it carries a 1.6 MiB worst-case RLE grid at one message per - frame. `acmefs` is 2,324 because it is acme's semantics. The general thing - costs ≈970 lines before saying anything specific to pardes, and each of the - three still needs its specific part afterwards. - ] -] - -#entry("9P-21", "The lost compile-time invariant, which nothing else prices", state: "open", tags: ("cost", "correctness", "unpriced"))[ - The best objection found against the whole direction, and no other entry - costs it. - - #ev("src/detached/wire.zig:23-27")[«every union and every enum gets a tag chosen HERE … so reordering `Event` or `CellStyle.ul` cannot silently redefine the protocol. The mapping switches are exhaustive: adding a variant to the core is a compile error in this file, which is the point of them.»] - #ev("src/detached/wire.zig:1199-1265")[and a test enumerates the eleven legal client tags by name under the rule "A HUMAN DID IT", asserts the enum holds exactly those, and asserts the other 250 tag bytes answer `BadTag` — so a frontend built before a change cannot forge a pane's output into a session built after it.] - - Under a `ctl`-file grammar, input arrives as `Twrite` payload bytes. 9P's - codec validates 9P; it cannot validate that a byte string is a legal `key`. - An exhaustive switch checked by the compiler and a 250-byte refusal test - become a runtime string parse with nothing to bind to. - - #ev("src/host.zig:44-52")[a related loss: fourteen of twenty-one vtable methods are `push_` and return nothing, because "a push with an answer would have N answers and no way to pick one". 9P answers every T with an R, so each acquires a tag, a reply, and a matching obligation.] - - #q[Three candidate answers, none costed yet. (a) Keep the typed wire for input and use 9P only for control and text — accepts two protocols forever. (b) Make the `ctl` grammar generated from the same enum, so the exhaustive switch survives as a parser generator and adding a variant is still a compile error. (c) Accept the loss and buy it back with a fuzz test over the grammar. (b) looks best and nobody has tried it.] -] - -#entry("9P-22", "The real gap: pardes cannot ask another pardes anything", state: "decided", tags: ("direction", "motivating-case"))[ - This is the entry the whole file was missing, and it is the owner's own - criterion stated precisely. - - Two instances of pardes can do exactly two things to each other today. - - / Shout: #[`nested.sendLook` formats `Look <path>`, connects, writes, returns `true`, and NEVER READS (`src/nested.zig:294-330`). The receiver filters for the one legal verb and closes (`:466-510`). The test at `:652-702` exercises the whole protocol and asserts that nothing comes back.] - / Become: #[`Attach` is not a connection, it is a replacement. Only on a successful handshake does the frontend reap its pane shells, close its watches, unmount `--fs`, *deinit the core*, and enter the thin loop (`src/detached/client.zig`, `docs/detached.md:207-213`). The local core is destroyed, not linked.] - - Everything else is out of reach. Nothing in `src/` reads `PARDES_FS`; `Event` - has no variant for a peer; `Host.VTable`'s 21 methods have no peer method. - Reading another session's text is possible only by shelling out — - `Exec cat /run/user/1000/pardes/<other-pid>/1/body` — which is Linux-only, - opt-in behind `--fs`, requires already knowing the pid, and is mediated - entirely by `/bin/sh`. And it fails outright against a detached session, - which serves no filesystem at all (`src/detached/server.zig:566-570`). - - #verdict[ - *The direction is a reply.* - - pardes has no request/response channel to anything that is not the kernel. - `nested` is one-way by construction. The detached wire forbids a round trip - inside `update` by design (`wire.zig:64-70`). `acmefs` has request/response - and cannot leave the machine (`fuse.zig:64`, `9P-13`). - - 9P is a request/response protocol that crosses machines, and that single - property is what the other three cannot be extended to have. Every good - thing on the list — scripting from macOS, driving a session over a network, - reading the board's memory, chaining one instance to another without - destroying either — is the same feature: *being able to ask, and get an - answer back.* - - That is the justification. Not simplification (`9P-20` forbids it), not - elegance, and not symmetry. - ] - - #note("direction", "2026-08-27")[ - The measure of success, so this can be checked rather than admired: after - the work, `Attach` should have a sibling that CONNECTS instead of - replacing — two live cores, each able to walk the other's tree — and the - twelve-line `Look` sentence should be a `Twrite` to the other instance's - `new/` on a connection that can also answer. If those two things are not - true at the end, the protocol was adopted for its own sake. - ] -] - -#entry("9P-23", "The host vtable stays; 9P is one implementation of it, for remote hosts only", state: "decided", tags: ("architecture", "scope-limit"))[ - The grand version of the direction was that `Host.VTable` becomes a tree the - core mounts, so that the downward seam and the outward seam are the same kind - of thing. Measured, that is wrong for every LOCAL host and right only for - remote ones. - - #ev("src/host.zig:52-140")[the seam is 21 methods: 15 `push_` and 6 `pull_`, with the prefix compiler-enforced (`isPull`, `:265-274`) — a `push_` returning non-void does not build.] - - The blocking objection turns out to be nearly empty. Of the six `pull_` - methods, three are ALREADY asynchronous — `pull_read_clipboard`, `pull_lsp` - and `pull_pipe` return `void` and their answers arrive as `Event.paste`, - `.lsp_resp`, `.pipe_resp` (`src/pardes.zig:6896-6907`). `pull_wait_input` - cannot become an event because it is how events arrive. That leaves two: - `pull_gpio_toggle`, whose answer is only used to set a message row - (`builtins.Board.gpio`), so deferring it a frame is invisible; and - `pull_tty_taken`, which is reached from `takesCommandLine` - (`pardes.zig:6465-6469`) at four call sites, two of which loop over all 16 - panes — 16 probes per call, collapsing to ONE read if `proc` is a single file - of sixteen lines rather than `proc/N/status`. - - So the core could be a 9P client without blocking. It should not be, locally: - - #ev("per-host measurement")[bodies replaced by an estimated 9P server, per host: tty 345 → ≈225 but *+204* with the tree and `ctl` grammars; gui *+204*; detached server *+182*; macos *+164*; web *+142 Zig plus ≈500 lines of JavaScript*; board *+59* and 8,832 B of RAM. *No host gets simpler.*] - #ev("src/web.zig:356-391")[and web gets actively worse than the count shows: `present` today writes a flat `WebCell` array that JavaScript reads *directly out of wasm memory* across four `extern` declarations (`:330-340`). Under 9P that zero-copy array becomes an RLE stream JS must decode, and wasm32 has no sockets, so JS must carry a 9P codec too.] - #ev("src/esp32p4.zig:127")[the board is worse still in kind: its "host" is the same process behind a C-ABI function pointer, so a `Twrite`/`Rwrite` pair per frame is pure protocol overhead against a direct call.] - - #verdict[ - `Host.VTable` is already the abstraction — `docs/design.typ:1964-1965` says - the surface IS the abstraction and refuses a render layer over the shells. - A tree would be that layer. - - So: a 9P host is *one more implementation of the existing vtable*, chosen - when the display is on the other end of a wire, exactly as `9P-2` makes a - 9P transport one more implementation of the filesystem seam. Local hosts - keep the direct call and pay nothing. The two seams stay two seams; what - they gain is a second backend each. - - That also settles `9P-19`'s scope: the core is a 9P client of its display - only when the display is REMOTE — a detached frontend, or a board acting as - a terminal for a full core elsewhere. Never for the window in front of you. - ] - - #note("correction", "2026-08-27")[ - Fact correction for anything quoting the earlier tally: `tty` and `gui` - fill *19* of 21, not 20 — `src/detached/server.zig:558-559` already says - "NINETEEN each". The 20 figure was wrong by one and is superseded here. - ] -] - -#entry("9P-24", "A dead descriptor left in a poll set is a whole core, forever", state: "landed", tags: ("bug", "lesson"))[ - Found by an adversarial pass over step 1 after it was committed and after the - happy path had been demonstrated with the project's own example clients. It is - recorded because the shape recurs, not because the fix was hard. - - Step 1 put `/dev/fuse` in the daemon's `poll(2)` set whenever the mount - existed, and gave `Source.fuse` an empty arm on the grounds that being in the - set was the whole point. - - #ev("linux/fs/fuse/dev.c")[`fuse_dev_poll` answers `EPOLLERR` once the connection is gone — and POSIX reports `POLLERR` whatever the `events` mask asked for, so an empty arm cannot decline it.] - #ev("src/fuse.zig")[`pollLoop`, the mount's own poll thread, has carried `if (revents & (ERR|HUP|NVAL) != 0) return;` all along. That is why the tty and SDL shells never showed this and the daemon did: they never poll the descriptor themselves.] - - So an external `fusermount3 -u`, a sysfs abort, or systemd taking - `/run/user/$UID` away at final logout — *exactly the moment a detached session - is supposed to keep running* — made `poll(2)` return instantly and forever. - Measured on the committed change: 0 CPU ticks over 10 s idle, then 1000 ticks - over the next 10 s. After the fix, 0 ticks over 8 s in the same scenario, with - the process alive and in state `S`. - - #verdict[ - Gate the insertion on `!f.dead` and let the arm consume `POLLERR`/`POLLHUP`/ - `POLLNVAL` by marking the mount dead. Both, not either: the gate is what - ends the spin, and the arm is what saves the one spinning round before - `Fs.next`'s first failed read would have set the flag anyway. - - The general rule, which is the reason this entry exists: *a descriptor - added to a shared poll set needs an error arm even when it needs no data - arm.* An empty arm is a decision about `POLLIN` only, and the kernel does - not ask permission before reporting `POLLERR`. - ] - - #note("review", "2026-08-27")[ - Worth noticing how it was caught. The feature was demonstrated working — - `pardesctl panes`, `new`, `send`, `body`, `del` against a live daemon — and - the bug was nowhere near the happy path. It took an adversary told to - *assume the happy path works and look elsewhere*, who then went and measured - `/proc/<pid>/stat` before and after an unmount. Four of that pass's other - seven findings were false comments rather than false code, which in this - codebase is the same severity: the comments are how the next change is made. - ] -] - -#pagebreak() - -= After the build - -Five steps shipped, and the shape of what is left is not the shape the note -predicted. These entries are written from the tree as built, not from the plan. - -#entry("9P-25", "The tree behind the server is pluggable, and that was not planned", state: "decided", tags: ("architecture", "windfall"))[ - `Server` is `pub fn Server(comptime fs: type) type`, duck-typed on exactly - `fs.Req`, `fs.Reply` and `fs.Reply.Attr`. That shape was chosen for a boring - reason — importing `acmefs.zig` drags `pardes.zig` into `zig test` — and it - turned out to be the most useful thing in the file. - - #ev("src/acmefs.zig")[filesystem one: the acme control tree, 9 ops.] - #ev("src/board9p.zig")[filesystem two: 867 lines that re-declare the same `Op`, `Status`, `Req`, `Reply` and `handle`, and are served by the same `Server` with no translation layer at all.] - - So "serve X over 9P" is no longer a protocol question. It is: write a - `handle()` over nine operations, and get a wire, a fid table, directory - cursors, `Tflush`, error strings and both freestanding targets for free. - - #verdict[ - Treat `Server(fs)` as the extension point it accidentally became, and say so - where someone will look. Candidates that are now cheap and were never on a - list: the LSP surface as a tree, a session dump as a tree, the config as a - tree. None of them needs a line of 9P. - - The discipline that keeps this from becoming a plugin system: nine - operations and no tenth. A filesystem that wants a tenth wants an API. - ] -] - -#entry("9P-26", "What is cheap now, ranked, and step 6 is not first", state: "open", tags: ("sequencing", "next"))[ - Step 6 — a remote display serving `screen` and `input` while the core is its - client — is cheaper than it was, because its one hard prerequisite is done: - `Client` exists, is 152 bytes, and blocks nowhere. And `board9p` proved that - writing a second filesystem behind `Server(fs)` is a day's work, which is - exactly what a `host` tree would be. - - But four things are now cheaper than step 6 AND serve the stated goals more - directly. Ranked by value over cost: - - / A serial transport for the client: #[the board image exists and speaks 9P on UART0; `src/fs9_client.zig` speaks unix sockets. One transport away from the motivating case being real, and it is the smallest item here.] - / TCP: #[`fs9_service` and `fs9_client` are `AF_UNIX` only. "Integrating over the network" was one of the four stated goals and it is currently a socket family, not a design problem.] - / macOS and the browser: #[step 4's entire portability argument — that a 9P server needs no kernel — is UNEXERCISED. Nobody has run `9pfuse` against the socket on macOS, and the web build has no client. This is the payoff that justified the growth in `9P-20`, and it is closer to zero code than anything else on the list.] - / Aggregation: #[unblocked, see `9P-10`'s note. Serves "chaining a pardes to another", which is the goal `9P-22` named as the whole point.] - - #q[Step 6 also changed SHAPE, and the note should be rewritten before it is built. It was sold as "the board stops being a shrunken pardes and becomes a terminal for a full core". What got built is the INVERSE: the board is a 9P server of its own devices and the desktop is the client. Both are useful and they are different images. Which one is step 6 — and is a board that shows a remote core's screen worth an image, now that a board that exposes its pins is already flashed?] - - #note("scope", "2026-08-27")[ - Unchanged by any of this: `9P-23`'s measurement. Local hosts keep the direct - vtable call, because a tree costs tty and gui about +204 lines each and the - web shell +142 Zig plus ≈500 JavaScript. Step 6 is remote displays only, and - the moment it is argued for the window in front of you, that number is the - answer. - ] -] - -#entry("9P-27", "What three adversaries found in steps 3, 4 and 5", state: "landed", tags: ("bug", "lesson", "review"))[ - Steps 1 and 2 were reviewed and the pass found a 100%-CPU spin in the smallest - change of the chain (`9P-24`). Steps 3, 4 and 5 — 1,005, 9,158 and 3,535 line - diffs — were verified on the happy path and shipped unreviewed. Three agents, - one per step, told to assume the happy path works and look elsewhere. - - Eight defects. Six fixed, five reproduced with measurements before and after. - - / Remote crash of the whole daemon: #[`Server.push` takes nothing once `startFrame` gives up on the framing, and `fs9_service.fill` asserted it took everything. One `size[4]` of zero plus one later byte reached `unreachable` — every pane, every frontend and the FUSE mount gone. The same stuck buffer separately made `room == 0` return without reading while poll reported ready forever: *99.7% of a core*, in one `write(2)`, from any process with the uid.] - / A 177-second freeze of the editor: #[`fs9_client.connect` ran `connect(2)` on a still-BLOCKING socket, before `setNonblock` and before the deadline existed. On AF_UNIX a full accept backlog waits in `unix_wait_for_peer` for `sk_sndtimeo`, which is forever, and the core is single-threaded. Measured at *177.3 s*, ending only because the peer was killed. Now 2.03 s, the budget.] - / A 64 KiB pty read wiping the queue: #[`queue_cap` is 65536 and a record must fit in `queue_cap - 4`, but every host reads a pty master with a 64 KiB buffer and a single read really returns 65536 on Linux. The eviction loop then emptied the queue and dropped the new record too, silently. Deterministic, not a race.] - / Four silent sockets denying `--fs9` forever: #[nothing took a slot back, and there are four. `version(5)` requires `Tversion` first and `msize == 0` already meant "has not versioned", so the frontend transport's own five-second rule applied unchanged.] - / EMFILE spinning a core: #[`accept` treated every failure as EAGAIN, but EMFILE leaves the connection in the backlog and poll is level triggered. *99.8% of one core.*] - / `max_fids = 32` refuting the step's own acceptance clause: #[a mounting client keeps a fid per cached inode, and `docs/9p.typ` §12.4 makes `9pfuse` the proof. At 32, `find` produced *57 consecutive `Rerror`s*. Now 256 for a host and `board_fids` 32 for the board.] - - #verdict[ - Three of the six are the SAME defect as `9P-24`: a descriptor left in a - level-triggered poll set with no error arm, or a wait with no deadline. That - is now four instances across five steps, every one of them a whole core or - a frozen editor, and every one written by someone who had just read the - previous one. - - So make it a rule rather than a lesson: *every descriptor this project adds - to a poll set needs an error arm, and every wait needs a deadline set before - the thing it is waiting on can begin.* Both are checkable by reading, and - both were missed by authors who knew the rule. - ] - - #note("method", "2026-08-27")[ - Two observations about the reviewing, worth more than the bugs. - - The instruction that worked was *"assume the happy path works — it has been - demonstrated live — and look everywhere else."* Every finding came from - somewhere the demonstration could not reach: a peer that stalls, a mount - that is torn down, a buffer at exactly its limit. The demonstrations were - all real and all passed, and none of them would ever have found any of this. - - And the reviewers were told to *measure*. Every serious finding arrived with - `/proc/<pid>/stat` before and after, or a wall-clock figure, or a count of - consecutive errors. A report saying "this could spin" would have been argued - with; "998 jiffies per 10 s against a 0-jiffy baseline, here is the script" - could not be. The cost of asking for that was a reviewer that took forty - minutes instead of ten. - ] - - #q[Two findings are NOT fixed and should be. A parked `pty/data` read whose client is gone keeps consuming the queue destructively — measured as 34% of a pane's output going nowhere on a long-running daemon, with the trigger not isolated in 23 attempts. And a `Tclunk` does not sweep park slots on the fid it retires, so a `close(2)` with a read in flight strands the tag forever; 32 of those and the connection can never serve a blocking read again. Both are reachable by ordinary client behaviour.] -] diff --git a/docs/typ/building.typ b/docs/typ/building.typ new file mode 100644 index 00000000..85f2dfbb --- /dev/null +++ b/docs/typ/building.typ @@ -0,0 +1,293 @@ +// Building pardes, for contributors: platforms, options, tests and the +// release gates, then how the pieces fit: the core and its shells, the +// threads, detached sessions, the 9P engine, the listeners and mounts. +#import "style.typ": key, keys, btn, chord, word, tag, addr, file, cmd, doc, pairs + += Platforms + +Zig *0.16.0* (`build.zig.zon` pins `minimum_zig_version`). Dependencies are +fetched and pinned by the manifest; the terminal build needs no system +package. The SDL shell builds SDL3 and FreeType from source. PDF support +builds MuPDF and is on by default (`-Dmupdf=false` drops it). 9P over QUIC +(`-Dquic=true`) uses system OpenSSL 3.6+ and pkg-config. + +#pairs( + [`zig build`], [the terminal shell and the SDL window together, *installed into `~/.local/bin`* (`pardes`, `pardes-gui`, and `pardes-v9fs`, the #word("Tty9p") helper)], + [`zig build -Dplatform=tty`], [the terminal shell alone], + [`zig build -Dplatform=gui`], [the SDL3 window alone], + [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<dump.zon>`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)], + [`zig build -Dplatform=macos`], [an AppKit and CoreText app over a static `libpardes.a` (`docs/macos.md`)], + [`zig build -Dplatform=esp32p4`], [a freestanding riscv32 editor object for the ESP32-P4, with a 384 KiB heap], +) + +A bare `zig build` installs into `~/.local`; `--prefix <dir>` installs +elsewhere, except `--prefix zig-out`, which counts as no prefix. With +`-Dplatform` the default prefix is `zig-out`, so always pass `-Dplatform` +while developing. Test, benchmark and run steps build what they need +without installing. `pardes --version` prints `pardes <version>` (from +`build.zig.zon`), plus the commit for release builds and the `~/.local` +install; `pardes --help` lists every flag. + +The ESP32-P4 firmware is linked in the sibling `../05-zig-p4` toolchain, +which needs ESP-IDF register headers: after building the editor object +here, `zig build -Dpardes` there links `src/esp32p4/app.zig`. The separate +GPIO 9P image is `zig build -Dapp=../02-pardes-code/src/esp32p4_9p.zig` +there; its namespace is in `src/esp32p4_gpio.zig`. + +== Build options + +`zig build --help` lists the options for the selected platform. + +#pairs( + [`-Dplatform`], [`tty`, `gui`, `web`, `macos`, `esp32p4`; absent: the tty and SDL shells together, installed into `~/.local`], + [`-Dstatic`], [bool, `false`], + [`-Dquic`], [bool, `false`: 9P over QUIC with system OpenSSL 3.6+], + [`-Dmupdf`], [bool; on natively, off for web and esp32p4], + [`-Djpx`], [bool, `true`: JPEG 2000, and with it scanned PDFs], + [`-Dtree-sitter`], [`disabled`, `zig`, `minimal`, `full`; `full` natively, `zig` for web, `disabled` for esp32p4], + [`-Dembed-sources`], [bool, `false`: serve the sources under #file("/src")], + [`-Dstamp-commit`], [bool: the git commit in `--version` and crash records; on for release builds and the `~/.local` install], + [`-Dtheme-animation`], [bool; on except for esp32p4], + [`-Dworkspace-tag`], [bool: draw the workspace tag row; on except for macOS, whose menu bar carries it], + [`-Dprebuilt-shaders`], [bool: embed the committed SPIR-V; on for a bare `zig build`, off with `-Dplatform`; `zig build shaders` refreshes it with glslc], + [`-Dtracy`], [path to a Tracy checkout; off], + [`-Dmacos-identity`], [codesigning identity for `pardes.app`; `-` (ad hoc)], + [`-Ddump`], [a `dump.zon` to embed in the web shell], + [`-Dtest-filter`], [run only tests whose name contains it; a filter that matches nothing fails], + [`-Dtest-rebuild`], [bool: fresh Zig test compilation], + [`-Dhelix-harness`], [reference executable for live differential tests; `HX_HARNESS`, else `hx-harness` on `PATH`], + [`-Desp32p4-cols`, `-Desp32p4-rows`], [the board's grid, 56 by 14], +) + += Tests + +``` +zig build unit-test module and shell unit tests (the perf gate runs with it) +zig build test-build compile the unit-test programs without running them +zig build core-test core tests without native shell tests +zig build pane-test pane, output, PDF and namespace integration tests +zig build syntax-test tree-sitter tests without building the editor +zig build syntax deterministic per-byte highlighting snapshots +zig build fs-test real sessions and mounts over 9P (with a short 9P monkey) +zig build monkey-9p random 9P operations checking the documented rules +zig build agent-session-test interactive session driver checks over 9P +zig build 9p-test freestanding protocol tests +zig build 9p-io-test native 9P transports and client +zig build quic-test -Dquic=true optional QUIC transport tests +zig build v9fs-test a real kernel mount (needs sudo -v; fails, not skips, without it) +zig build snap scripted input traces against frozen golden grids +zig build monkey random snapshot scripts hunting panics (not a gate) +zig build hxdiff differential suite against helix's own behaviour +zig build hxparity file-pane vs pty-pane editing parity +zig build hxgolf every helix-golf example, step by step, against helix +zig build perf-gate a gesture on the 50k-line file over 3x its baseline fails +zig build mupdf-check compile, link, render and search docs/design.pdf +zig build web-snap browser highlighting and touch interactions +zig build web-e2e Chrome-driven DOM end-to-end suite +zig build cheatsheet render the cheatsheet PDF (needs typst) +``` + +- `-Dtest-filter=<text>` applies to every unit-test binary. A failed test's + printed trace can be stale: run it alone with the filter. +- `snap -- --record=DIR` writes snapshots without touching goldens, + `snap -- --update` replaces goldens (re-record one script by name, after + reading its diff, never all), `snap -- --no-retry` makes the first + failure decisive. Snapshot scripts can use `snap9p` to capture core cells + through 9P. +- `zig build monkey -Dplatform=tty -- <seeds> [steps] [--from=N] [--out=DIR] [--keep]` writes one random script per seed and keeps any + that panics; a seed always makes the same script. Each crash found gets a + fix and a regression script in `test/snapshots/`. +- `zig build hxdiff -- --strict cases.jsonl reference.jsonl [waivers.jsonl]` runs custom differential cases. `docs/helix-keys.md` + tracks helix parity key by key and names the reference helix build; + `docs/selections.md` is the selection model under normal mode. +- Benchmarks (`perf`, `pdf-bench`, `pdf-scroll-bench`, + `pdf-sections-bench`, `lspbench`, `fs-bench`) take `-- --json`; measure + with `-Doptimize=ReleaseFast`. `perf -- --base old.json` refuses reports + whose build metadata differs. +- `zig build history -- run 'ancestors(@, 2)' DIR -- zig build unit-test` + records a command's output and runtimes across revisions; `history -- compare a.json b.json 1.20` fails past that runtime ratio. +- `python3 -B test/agent_session.py <pardes> --ready 'text' --min-rows N -- command args` drives an interactive command in a private shell and checks + it through 9P. + +== Release gates + +A release passes all of these first: `unit-test` with `-Dplatform=tty` and +with `-Dplatform=gui`, `core-test`, `fs-test`, `snap`, the GUI goldens +(`python3 -B test/gui_golden.py <a Debug pardes-gui>`, a hidden window), +`web`, and the ReleaseSafe tty and ReleaseFast gui builds. The committed +`docs/typ/cheatsheet-a4.pdf` is rendered again when the docs change. + += The core and its shells + +One core, five shells (tty, gui, web, macos, esp32p4). The core owns +editing, layout, rendering and the control filesystem; a shell turns native +input into `pardes.Event`, presents `pardes.Surface`, and performs host +effects such as spawning processes. + +- The core is a state machine: input arrives as an `Event` through + `update`; output leaves as a `Surface` from `render(arena)` and as a ring + of `Effect` values. Effects are fixed-size values with no lifetime ties + into the core; unbounded content is read off the core when an effect is + drained (`save_text` carries a path and the pane's serial, so a reused + slot writes nothing). `emit` refuses when the ring is full and never + evicts. +- Pty bytes leave as 64-byte `.write` effects; overflow parks in a per-pane + buffer that `nextEffect` drains in order. `postEvent` is a 64-entry value + queue for events that borrow no slices. +- `pump` runs: wait for input (the host owns the sleep), flush paused 9P + write batches, drain queued events, drain effects, settle the turn, then + render and present only when a frame is needed and someone is looking. +- `host_io.Host` is a context pointer and a vtable of optional callbacks; a + null method is not an error, and `Host{}` is a complete in-process pardes + that the tests use. Comptime decides what a build has (`pardes.platform`, + `hosted`, `can_attach`, `terminal_panes`, `pdf_enabled`); the vtable + decides who serves it. +- The root module is chosen by platform: `main.zig` (tty, gui), + `web.zig`, `macos.zig`, `esp32p4.zig`. The web and macOS shells are + libraries whose host owns `main()`. +- `memory.limits` is the one home for capacities that differ on the board + (panes 16, columns 6, selections 64, the tag's 512 bytes). Each core + allocator is a thread-safe stack-fallback allocator, under a + DebugAllocator in Debug builds. + +Code map: `src/panes.zig` and the pane kinds it names (`src/File.zig`, +`src/Terminal.zig`, ...), `src/layout.zig`, `src/fs.zig` (host access, +mounts and resolution) and `src/pardes.zig` (input), with one file per +thing beside them (`edit.zig`, `normal.zig`, `look.zig`, `exec.zig`, +`mouse.zig`, ...). `src/ninep/` is the control tree, `src/detached/` the +wire, the detached core and the frontend client, `src/lsp/` the +language-server client; `test/` holds harnesses, goldens and helix cases, +`build/` the snapshot suite's build step. + +== Threads + +The core is single-threaded under a turn mutex, `pardes.turn`. The editor +thread holds the turn and lets go of it while it waits for input and while +it is out in a host syscall mid-step; a 9P connection task takes the turn +in those gaps to answer a request. While a step is out, a request that +would change a pane parks (`Status.again`) and is retried when the editor +rests. 9P requests are not events: they enter through `Pardes.serveFs`, and +only one that changes a pane costs a frame. Language-server and +selection-pipe workers post completions through a bounded mailbox; a +`host_io.Lsp.Job` owns copies of the source, path and arguments, never +reading the live core, and a request id, the pane's serial and (for an +edit) the file's revision reject a stale reply. A process that never calls +`turn.start` (the tests, the board, the browser) has no second thread. + +== Detached sessions + +One poll loop in `src/detached/server.zig` owns core mutation, frontend +connections, pty I/O and file watches. The frontend socket is +`pardes-detached-<name>.sock` beside the session's 9P socket; a second +session cannot take a live name. Up to 32 frontends attach; frames go to +every one, while clipboard reads, browser opens and #word("Detach") go to +the frontend that asked (else the first attached). Frames are full grids +or changes against what each frontend last received, never queued: a +frontend with unsent bytes skips that frame, and one that stops draining +past 1 MiB of control backlog is closed, its peers untouched. Frontends +never spawn shells, write session files or watch files. #word("Attach") +connects first and swaps second: only after the handshake does a shell +give up its own core. Restore builds the replacement core before touching +the current one. + +`src/detached/wire.zig` is the versioned protocol (version 10): fixed-width +little-endian fields, a 5-byte header (tag, then a u32 length), payloads up +to 16 MiB and grids up to 512 by 128. A comptime hash of `pardes.Chrome` +forces a version bump when that struct changes. + += 9P + +The wire format, the client and server connections, the file-server engine +and the Unix, TCP and QUIC transports are the `cloud9` package +(`git.sr.ht/~gbrls/cloud9`), pinned in `build.zig.zon` and fetched into +`zig-pkg/`. Re-pin with +`zig fetch --save=cloud9 git+https://git.sr.ht/~gbrls/cloud9#<commit>`, or +use `.cloud9 = .{ .path = "../cloud9" }` while editing both. cloud9's own +`zig build test`, `transport-test`, `quic-test -Dquic=true`, `fuzz` and +`differential` cover the shared code. + +- The engine (`fs.Server`) owns fids, permissions, directory reads, + flushes and what a hangup releases; it allocates nothing and makes no OS + calls. The control tree in `src/ninep/` is its backend: `tree.zig` + (nodes, dispatch), `pane.zig`, `ctl.zig`, `cols.zig`, `addr.zig`, + `pty.zig`, `events.zig` (event and log), `screen.zig`, `sources.zig`. + `src/9p.zig` names the editor's and the board's engine settings: msize + 65536, 256 fids, 128 held reads a connection, names up to 255 bytes; the + board has 32 fids. +- A read, write, open, clunk, remove or truncating wstat can park; a walk, + attach, stat, create or rename cannot. Every Rread is clamped to the + count and the msize; Tflush answers the original request first. +- Unix and TCP listeners run on cloud9's `serve.Runner`: an accept task per + listener, a reader and a writer task per connection, 16 connections. QUIC + (`src/9p_quic.zig`) still runs on the editor's poll loop in + `src/9p_io.zig`. +- A body write takes only whole UTF-8 sequences and answers a short count + for the rest. Consecutive writes from one open at the end of the text are + held and applied as one edit, flushed by any other request, the open's + release, 64 MiB, or a 20 ms pause in the editor's step. + +== Writes through a mount <writes-through-a-mount> + +A mount cuts a big write into pieces of at most one message (msize 65536, +less the header), and each command line runs once it is whole. A write +that does not fill its message is whole, so its last line runs even +without a newline (`printf Save > exec`), unless it is a multiple of 4096 +bytes: that is where a writer's buffer (stdio, a mount's page cache) +filled and cut a line, so its tail waits for the next write or the close. +A line held to the close (such a tail, or an `Edit` block never ended) +runs there, and its failure is only in the log, as its `err` record: the +close reports no error, and the write that sent it had already succeeded. +So a script that needs a line's result ends the write with a newline. + +== Listeners <listeners> + +`--9p-tcp='tcp!127.0.0.1!5640'` adds TCP; `--9p-quic='quic!127.0.0.1!5641'` +adds QUIC (built with `-Dquic=true`; ALPN `pardes-9p`, an ephemeral TLS +identity, no peer verification). Addresses are numeric IPv4 or IPv6; port +0 picks one; #file("/listeners") reads them back. Every connection has full +session access, #file("/os") included, and TCP is unencrypted: use +loopback. Unix and TCP share 16 connection slots; a 17th client's Tversion +gets `too many connections` (and the log +`err - 9p: too many connections (N turned away)`). QUIC has 16 of its own. +plan9port and v9fs need a userspace bridge for QUIC. + +`$XDG_RUNTIME_DIR/9p` is the machine's `/srv`: servers post themselves +there by name, and `9ns --mntgen` mounts the whole registry. A pardes whose +socket is in the runtime directory posts +`$XDG_RUNTIME_DIR/9p/pardes/<name>`, a symlink to its socket, and unposts +it on a clean stop if it is still its own; one whose socket fell back to +`~/.local/state/pardes` posts nothing. Posting first sweeps the group: a +symlink whose socket refuses a connect is removed with its socket. pardes +binds its own socket rather than going through `cloud9.post`, which takes +only flat names. A reader of the registry must `stat` through the symlink. + +== Kernel mounts and Tty9p + +For Linux v9fs use `version=9p2000,cache=none,access=any`, `trans=unix` +(or `trans=tcp` with `port=`), `uname`, `dfltuid` and `dfltgid` for the +local user, and an empty `aname`. Linux follows `O_TRUNC` with a `Twstat` +of zero length and an `mtime` hint; pardes takes the truncation and drops +the hint. + +#word("Tty9p") starts the normal shell and queues a quoted helper command, +which bash and fish run at their first prompt as a foreground job, so +sudo has the terminal. The unprivileged launcher makes a private temporary +mountpoint and runs `sudo -E`; the elevated `pardes-v9fs` helper makes a +private mount namespace, mounts the session's socket with +`trans=unix,version=9p2000,cache=none,access=any,nosuid,nodev,noexec`, +drops every root id and capability, and runs the shell as the user (with +the caller's `PATH` again). The namespace and the mount go with its last +process. Nothing setuid and no passwordless sudo rule is installed; the +helper takes explicit paths and a command and is no restricted broker, so +never grant it passwordless sudo. The host finds the helper beside its own +executable, or at `PARDES_V9FS_HELPER`. Code: `src/linux/v9fs.zig`; +`zig build v9fs-terminal-test` and `v9fs-driver-test` need no privileges. + += Other notes + +`docs/` keeps the platform and design notes beside this book: +`web.md` and `macos.md` (the browser and macOS shells), `effects.md` and +`render-pipeline.md` (visual effects), `helix-keys.md` and `selections.md` (helix parity), +`lsp-evaluation.md`, `ui-review.md`, `divergences.md` (bookmarks off +`main`) and `open-questions.md`. `next-steps.txt` is a wishlist and +`transactions.txt` records one open structural gap against helix. diff --git a/docs/v9fs.md b/docs/v9fs.md deleted file mode 100644 index 26460658..00000000 --- a/docs/v9fs.md +++ /dev/null @@ -1,56 +0,0 @@ -# Tty9p: a terminal with a kernel 9P mount - -On Linux, `Tty9p` (`SPC n 9`) opens a terminal below this pane with the -session mounted through the kernel's v9fs. The pane asks for your sudo -password, mounts, and starts your shell as your normal user (with your -supplementary groups). The shell gets `PARDES_MOUNT`, the mountpoint: - -```sh -cat "$PARDES_MOUNT/index" -cat "$PARDES_MOUNT/pane/$PARDES_PANE/body" -echo 'Msg hello' > "$PARDES_MOUNT/exec" -n=$(cat "$PARDES_MOUNT/pane/new") -``` - -The files are those of [fs.md](fs.md). The mount lives in that pane's -private mount namespace: other panes and the editor do not see it, so a -Look at a path under `$PARDES_MOUNT` opens nothing. Each `Tty9p` makes its -own mount and takes one of the session's 16 Unix/TCP connection slots. It -works in TTY, SDL and detached sessions (the session host starts the shell, -not an attached frontend). Dump/Restore does not remake the mount. - -## Setup - -The build installs `pardes-v9fs` beside `pardes`; the host looks for it -beside its own executable, or at `PARDES_V9FS_HELPER` (an absolute path). -A running editor keeps the code it started with. - -Linux needs `9p` and its Unix transport (`9pnet_fd`); mounting needs -`CAP_SYS_ADMIN`, which sudo supplies. The launcher runs `sudo -E` (local -policy must allow it) and restores the caller's `PATH` after dropping -privileges. Nothing setuid, no passwordless sudo rule and no FUSE is -installed. The helper takes explicit mount paths and a command; it is not a -restricted privilege broker, so do not grant it passwordless sudo. - -## How it works - -`Tty9p` starts the normal shell and queues a quoted helper command, which -bash and fish run once their prompt is ready, as a foreground job, so sudo -uses the terminal. A failed or cancelled authentication returns to that -shell, as does exiting the mounted one. The unprivileged launcher makes a -private temporary mountpoint and runs sudo; the elevated helper makes a -private mount namespace, mounts the session's socket with -`trans=unix,version=9p2000,cache=none,access=any,nosuid,nodev,noexec`, drops -every root id and capability, and executes the shell. The namespace, and -the mount, go when its last process exits. Code: `src/linux/v9fs.zig`. - -Linux follows `O_TRUNC` with a `Twstat` of zero length and an `mtime` hint; -pardes takes the truncation and drops the hint. - -## Tests - -```sh -zig build v9fs-terminal-test -Dplatform=tty # builtin, launcher, cleanup; a sudo stand-in, no mount -zig build v9fs-driver-test -Dplatform=tty # the probe launcher, no privileges -sudo -v; zig build v9fs-test -Dplatform=tty # a real kernel mount (sudo -n); fails, not skips, without it -``` |