From 60367d8fe23f6af98ec28e3cf6c2094dfe332df0 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Sun, 6 Sep 2026 18:11:36 -0300 Subject: Refactor panes and filesystem; replace FUSE with 9P Consolidate pane, layout, memory and host code. Serve 9P by default over Unix sockets, with runtime mounts and optional TCP/QUIC transports. Remove FUSE and obsolete proof-of-concept examples. Fix highlighting and terminal-history performance, expand differential and stress-test infrastructure, sort navigation results while preserving the next occurrence, add syntax-colored Braille minimaps, remove SPC-k, and document 9P interaction as a repository skill. --- docs/9p.typ | 5 + docs/acme-fs.md | 431 ------------------------------------ docs/config.md | 55 ++--- docs/design.typ | 424 +++++++++++++----------------------- docs/detached.md | 359 ++++-------------------------- docs/fs.md | 89 ++++++++ docs/helix-keys.md | 26 +-- docs/lsp.md | 623 +++++++---------------------------------------------- docs/macos.md | 82 ++----- docs/registry.typ | 14 +- docs/web.md | 16 +- 11 files changed, 425 insertions(+), 1699 deletions(-) delete mode 100644 docs/acme-fs.md create mode 100644 docs/fs.md (limited to 'docs') diff --git a/docs/9p.typ b/docs/9p.typ index 82131b9d..50cec965 100644 --- a/docs/9p.typ +++ b/docs/9p.typ @@ -44,6 +44,11 @@ #v(1.2em) +*Historical proposal, superseded by `docs/fs.md`.* Pardes now serves 9P by +default over Unix sockets, with optional TCP and QUIC. FUSE support and the +old examples have been removed. The arguments and source references below +describe the earlier implementation, not the current interface. + #block(inset: (x: 2.5em))[ #set text(size: 10pt) #set par(first-line-indent: 0em) diff --git a/docs/acme-fs.md b/docs/acme-fs.md deleted file mode 100644 index 25cbcc2d..00000000 --- a/docs/acme-fs.md +++ /dev/null @@ -1,431 +0,0 @@ -# The control filesystem — pardes against acme - -`pardes --fs` serves plan9 [acme(4)](https://man.cat-v.org/plan_9/4/acme)'s control -filesystem over Linux FUSE: a directory per pane, named by the pane's serial and -holding `addr`, `body`, `ctl`, `data`, `errors`, `event`, `rdsel`, `tag`, `wrsel` -and `xdata`, plus a `pty/` directory on a terminal pane, plus `index`, `cons` -and `new/` at the root (`PaneFile` and -`TopFile` in `src/acmefs.zig`). A program that opens those files IS an editor -extension — no plugin API, no embedded interpreter, no rebuild. -`examples/acmefs/` has four of them: `clock.py`, `eventlog`, `life.py` and -`pardesctl`. - -This document is the **comparison report**: what acme does, what pardes does, -why they differ, and which one is simpler. acme's C is at -`/home/goblin/05-genizah/principia-softwarica/editors/acme` — the -principia-softwarica tree and not plan9port, which matters because the two -differ in what they serve; every `file:line` below is that checkout's. Every -claim cites both sides. Where acme is better, it says so. - -| | acme | pardes | -|---|---|---| -| transport | 9P over a pipe, own implementation | raw `/dev/fuse`, own codec (`src/fuse.zig`) | -| concurrency | 1 server thread + **one thread per in-flight request** | none in the core; one `poll()` thread in the transport | -| blocking read | park an `Xfid` in `w->eventx`, wake from `winevent` | `Status.again`, re-asked by the transport | -| offsets | runes | bytes, grapheme-clamped | -| node ids | `QID/WIN/FILE` shift macros | `packed struct(u64) { file: u4, serial: u60 }` | -| errors | 9P error strings (`Ebadctl`, `Edel`, ...) | errno | -| event queue | `realloc` per record, unbounded | length-framed `ArrayList`, capped, drop-oldest | -| `ctl` write | applies the good prefix, then reports 0 consumed | validate-all then apply-all | - -## 1. The service: threads-and-channels against one transaction - -acme runs a dedicated process for the wire (`proccreate(fsysproc)`, `fsys.c:136`) -whose loop reads a 9P message, borrows an `Xfid` from a pool, and dispatches -through the `fcall[x->type]` function table (`fsysproc`, `fsys.c:140-193`; the -table itself at `fsys.c:41`, the dispatch at `fsys.c:190`). Directory reads and -stats are answered inline; anything that touches a window is handed to that -`Xfid`'s own thread — -`sendp(x->c, xfidread)` (`fsys.c:660`) — and `xfidallocthread` creates **one -thread per `Xfid`** on first use (`acme.c:718-744`), each parked in -`for(;;){ f = recvp(x->c); (*f)(x); ... }` (`xfidctl`, `xfid.c:42-55`). -Serialisation is by `QLock`: one on the row (`dat.h:313`), one per window plus an -owner byte (`dat.h:228` and `dat.h:250`, taken together in `winlock1`, -`wind.c:131-136`), one on the mount table (`struct Mnt`, `fsys.c:96`, taken at -`fsys.c:201`). - -pardes has none of that. A request is a value, an answer is a value, and the -whole filesystem is one function: - -```zig -pub fn handle(p: *Pardes, req: Req) Reply // src/acmefs.zig -``` - -It arrives as an ordinary `Event.fs_req` and leaves as an ordinary -`Effect.fs_reply` (`src/pardes.zig`), so the transport is the queue every other -host↔core message already uses, and the core keeps the single-threaded model it -had. All the concurrency lives in `src/fuse.zig`: one thread that `poll()`s the -fd and wakes the loop, and a park table for requests the core answered with -"not yet". The thread never touches core state, never parses a request, and -never writes a reply — the same discipline `src/file_watch.zig`'s inotify thread -already followed. - -**Simpler: pardes, by a lot.** No channels, no locks, no thread per request, no -fid bookkeeping, and the semantics are unit-testable with no scheduler and no -FUSE anywhere near them (`src/acmefs.zig` has 21 such tests). - -**What acme buys, honestly:** isolation. Its request threads mean a slow read -cannot stall the editor. In pardes `handle` runs on the loop thread, so a -pathological request — reading the body of a 100 MB file, a `ctl get` that -re-reads a huge file from disk — is a frame the user waits for. The measured -numbers say this is theoretical rather than practical, but it is a real property -of the design and the reason acme's complexity exists. - -Every measurement in this document is one run of `zig build fs-bench --Doptimize=ReleaseFast` on an i7-11700. Each row is `handle` called directly — -no FUSE, no thread — and its figure is the MEAN over the row's reps: 100 000, -except 10 000 for the 1 MiB read, 200 for the append and 2000 per keystroke row. - -| row | per request | allocations, whole row | -|---|---|---| -| `getattr` on a 1 MiB body | 19 ns | 0 | -| `lookup ctl` | 39 ns | 0 | -| `read body`, 4 KiB | 25 ns | 0 | -| `read body`, 1 MiB | 25 ns | 0 | -| `read ctl` | 404 ns | 0 | -| `read index` | 631 ns | 0 | -| `readdir` of the root | 39 ns | 0 | -| `read event` on an empty queue (`Status.again`) | 21 ns | 0 | -| `write body`, 1 KiB appended to a body growing from 1 MiB | 3.36 ms | 600 | - -Two things in that table are the point of having it. The rows that FORMAT — -`ctl` and `index`, which `bufPrint` a line of `%11d` fields — cost about twenty -times a row that hands back a slice, and are still well under a microsecond. And -a 1 MiB `body` read costs exactly what a 4 KiB one does, because it is -zero-copy: `Payload.region` is a window onto the pane's live text. Every read -row allocates nothing at all, which is a property the benchmark exists to check -rather than a pleasing number — a non-zero count there would mean a read had -stopped answering out of the live text or out of the staging buffer that is -cleared and never freed. The one row that allocates is the append, at 600 -allocations across 200 writes, and that is the core's whole-body swap plus its -undo snapshot rather than anything this filesystem does. - -## 2. Blocking reads: a parked thread against a returned value - -acme's `event` read blocks: `xfideventread` (`xfid.c:994-1025`) stores its `Xfid` -in `w->eventx`, unlocks the window and sleeps on a channel (`xfid.c:1008-1010`); -`winevent` (`wind.c:543-569`) appends the record and wakes it; `windelete` -(`wind.c:217-225`) wakes it with no data so it can answer "window shut down" -(`xfid.c:1005`); and `xfidflush` (`xfid.c:58-87`) exists solely to cancel a -parked reader — it walks every column and every window looking for the tag, -because a blocked 9P read cannot otherwise be interrupted. - -pardes returns `Status.again` — "nothing consumed, ask me again" — and that is -the entire blocking primitive. The core keeps no waiter, no channel, no cancel -path. The transport parks the kernel's request (`max_slots = 32`, `src/fuse.zig`) -and re-offers it once per frame, oldest first, with a per-round flag so a -permanently blocked reader cannot starve the others (`retry`). A `FUSE_INTERRUPT` -answers the original with `-EINTR`, which is what keeps a SIGKILLed reader out of -permanent uninterruptible sleep: after a fatal signal `fuse_dev`'s final -`wait_event` is not killable, so the process survives its own kill until the -server replies. Three tests pin that mechanism — "again holds the request, retry -offers it back once per round", "interrupt answers the original with EINTR and -drops it", and "a full table answers EAGAIN and keeps the descriptor flowing", -whose comment records that the tempting alternative, gating the read on a free -slot, wedges a real mount: the INTERRUPT that would free a slot is never read -either. - -Two deliberate differences: - -- acme hands back up to `count` bytes and keeps the remainder, so a small read - can split a record (`xfid.c:1015-1017`). pardes refuses a read smaller than one - record with `EINVAL`: half a record is unparseable and silently desynchronises - a client. -- acme's parked thread survives the client's death (it stays blocked until the - window produces an event). pardes has nothing to leak — the kernel drops the - request. - -**Simpler: pardes.** **acme buys** an unbounded number of blocked readers; ours -are bounded by the park table because each one is a held kernel request. - -## 3. Node identity - -acme packs a qid with macros: `QID(w,q) ((w<<8)|(q))`, `WIN(q)`, `FILE(q)` -(`dat.h:463-465`) — 24 bits of window id, 8 of file id, no validation. A window -that dies while a client holds a file open is detected structurally, by every -handler remembering to check `if(w->col == nil) respond(..., Edel)` -(`xfid.c:422-425` in `xfidwrite`, and a dozen more; the string is at -`xfid.c:19`). - -pardes uses the type system: - -```zig -pub const Node = packed struct(u64) { file: u4 = 0, serial: u60 = 0 }; -``` - -`Node.target(node)` is the ONE place that validates, returning a tagged union of -"top-level file" or "pane file", so no handler re-decodes and none can forget. -`serial` is the pane's monotonic identity, never reused, so a stale path can go -dead but can never come to mean a different pane — and `State.forget`, called -from `deinitPane`, drops the pane's filesystem state at the moment it dies, -which is also what stops a dead script's reader count from suppressing button -actions forever. - -**Simpler: pardes.** The packed struct is the same bits with the shifts checked, -the sentinel-terminated `Dirtab` tables become enums with `name()`/`mode()` -methods, and `Edel`-by-convention becomes `ENOENT` by construction. - -## 4. Addressing: runes against bytes - -acme's document is `Rune*`, so every read converts. `xfidutfread` -(`xfid.c:875`) keeps a byte-to-rune cache per window — `w->utflastqid`, -`utflastboff`, `utflastq` (`xfid.c:891-897`) — and, when it misses, scans from -the beginning, carrying the comment `/* BUG: stupid code: scan from beginning */` -(`xfid.c:895`). The address language lives in `addr.c`: `number()` -(`addr.c:52`), `regexp()` (`addr.c:119`), `address()` (`addr.c:150`), with -failure reported through out-parameters (`int *evalp`, `uint *qp`) and patterns -grown one rune at a time. - -pardes is byte-addressed end to end (selections, look spots, LSP offsets), so -**every offset in this filesystem is a byte offset**, clamped to grapheme -boundaries — the one deliberate incompatibility with acme(4), stated in -`src/acmefs.zig`'s header and in `examples/README.md`. For ASCII, which is what -scripts compute with, the two agree. The address parser is the same left-to-right -state machine as `addr.c` with the C removed: the expression is a slice, the -cursor is a field, "did not evaluate" is `?Range`, and `limit=addr` is an -optional rather than a sentinel `-1`. - -**Simpler: pardes** — the entire rune↔byte layer and its cache do not exist. -**acme buys** rune semantics at every boundary, which is what its own manual -promises; ours promises bytes. - -## 5. Events, and the inversion that makes this a plugin API - -The record is the same on both sides, byte for byte: origin char, type char, four -blank-separated decimals, the text, a newline. acme builds it in two halves — -`text.c:380` formats `"%c%d %d 0 %d %.*S\n"` and `winevent` prepends the owner -byte at `wind.c:561` — and pardes builds it in one, `formatRecord` -(`src/acmefs.zig`). - -The rule that matters is the inversion: **while a script holds a pane's `event` -file open, buttons 2 and 3 in that pane belong to the script.** acme spells it -`if(!external && t->w!=nil && t->w->nopen[QWevent]>0){ winevent(...); return; }` -(`exec.c:150`, `look.c:35`); pardes spells it as the return value of -`noteAction` — "true means the core must not perform it" — checked in -`dispatchPointerBuiltin`. That is how `examples/acmefs/life.py` puts -`Step Run Stop Clear Random` in a tag pardes has never heard of and makes them -work. `test/snapshots/acmefs-event.snap` drives the A/B proof and -`acmefs-event.golden` records it: with a reader attached, a middle click on the -word `Newcol` changes nothing on screen and delivers `MX0 6 1 6 Newcol` -(`acmefs-event.golden:9`); with the reader gone, the same click opens a column. - -Differences worth knowing: - -- **Write-back takes four fields only** — `origin type q0 q1\n`, type in `xXlL` - — exactly as `xfideventwrite` demands (`xfid.c:791-872`: it reads the origin - byte, the type char, two `strtoul`s and a mandatory newline, and its `switch` - takes `x`, `X`, `l`, `L` and sends everything else to `Rescue`, which is - `err = Ebadevent`). There is no text field, so a client that wants to run text - not already on screen appends it to the tag and execs that range; - `examples/acmefs/pardesctl exec` does precisely this — `printf ' %s' "$text" - >>$d/tag` and then `printf 'Mx%d %d\n' "$q0" "$q1" >>$d/event` — and it is how - acme clients have always done it. -- pardes validates a whole batch of records before performing any of them; acme - performs them as it parses, which `writeEvent`'s own comment in - `src/acmefs.zig` names as the reason: a malformed batch is otherwise - half-applied and unrepeatable. -- acme records the origin byte the RECORD claimed — `w->owner = *p++;` with - `/* disgusting */` beside it (`xfid.c:812`) — and stamps it onto every record - it later produces (`wind.c:561`). pardes sets `State.origin` once per update - from the event kind (`K` keyboard, `M` mouse, `E` a write to body or tag - through this filesystem, `F` an action through one of its other files), - parses the character on the way in and drops it. **acme is arguably better - here**: its owner byte is per-record provenance and a writer can re-attribute - an action. Against that, a record saying where it came from is worth nothing - when the sender picks the answer, which is why pardes does not read it. - -## 6. Reporting edits: known ranges against a diff - -acme reports from the two functions that make edits, which already know their -range: `textinsert` emits `I` with `q0, q0+n` and the inserted runes -(`text.c:377-382`), `textdelete` emits `D` with `q0, q1` (`text.c:482`). - -pardes has no such pair — every edit lands in one place as a whole new buffer -(`file_pane.setContent`) — so the range is recovered by diffing there: -`acmefs.diffSpan` skips the common prefix and suffix in 64-byte chunks through -`std.mem.eql`, which lowers to vectorised compares, and `noteReplace` emits the -deletion then the insertion, the same two records in the same order. The -vectorising is not premature: its own comment records that the byte-at-a-time -loop it replaced cost 2.4x per keystroke on a 40 KB body. The whole path is -behind `p.fs.scripted(id)` — that pane's reader count, not the session-wide -`listeners` total — so an editor nobody is scripting pays one branch. Measured -cost of a keystroke on a 32 KiB body: 32.9 µs with no listener against 34.1 µs -with one, **+3.9%**. - -**acme is better here in principle** — a known range beats a scan — and it pays -for it by routing every mutation through a pair of functions that carry ranges -everywhere. pardes's single funnel is worth more than the scan costs. - -Undo grouping is acme's `mark`/`nomark` on both sides: acme bumps a global -sequence number and merges an `elog` of edits (`elog.c`; `if(w->nomark == FALSE) -{ seq++; filemark(t->file); }` at `xfid.c:501-504`); pardes suppresses the -per-write `pushUndo` snapshot. Same verb, same effect on the user's `u`, much -less machinery — and the reason it matters is measurable: the benchmark's append -row is 3.36 ms per 1 KiB write against a body around a megabyte, almost all of -it the whole-body swap and that snapshot, so a script writing a batch should say -`nomark` first. - -## 7. `ctl` - -Both print the same five `%11d` fields — id, tag length, body length, isdir, -dirty (`winctlprint`, `wind.c:534-535`) — and pardes adds acme's three extras -(`wind.c:537-538`) with the one honest substitution: width and tab in **cells**, -because pardes is a character grid where acme has pixels (`Dx(w->body.r)`). - -The verb parsers differ in two ways that matter: - -- acme matches verbs by **prefix** with `strncmp` and advances by the matched - length (`xfid.c:602-767`), so the table order is load-bearing: `delete` - (`xfid.c:697`) must precede `del` (`xfid.c:701`), `nomark` (`xfid.c:738`) - `mark` (`xfid.c:742`), `nomenu` `menu`, `noscroll` `scroll`. pardes matches a - whole token through `std.meta.stringToEnum`, which makes that class of bug - unrepresentable. -- acme applies verbs as it parses, so a bad verb leaves the good prefix applied - — the mutations already made stand — and then reports **zero** bytes consumed: - `err = Ebadctl` (`xfid.c:769`) falls through to `if(err) n = 0; fc.count = n;` - (`xfid.c:780-782`), and `xfideventwrite` repeats it verbatim at - `xfid.c:863-865`. So the client learns that it failed but not where, and the - editor has already been half-changed. pardes validates every verb first and - then applies them: a short count on a Linux `write(2)` is not read by anybody - as "the rest failed", and a half-applied batch is unrepeatable. **This is not - a trade**; the atomic answer is simply the better one. - -Verbs pardes cannot honour are refused loudly with a reason — -`refused_verbs` is `dump`, `dumpdir`, `font`, `lock`, `menu`, `nomenu`, -`unlock` — rather than silently accepted. - -## 8. Errors - -acme answers with strings: `Edel` "deleted window", `Ebadctl` "ill-formed control -message", `Ebadaddr` "bad address syntax", `Eaddr` "address out of range", -`Ebadevent` "bad event syntax" (`xfid.c:19-24`), handed through -`respond(x, &fc, err)`. pardes answers with an errno, because that is the only -channel FUSE has: the client would never see the string. Two acme errors that -differ in wording collapse to `EINVAL` here, which is a real loss of -diagnostics — the message row and `PARDES_LOG` carry the detail instead. - -## 9. Memory and bounds - -acme grows `w->events` with `realloc` and never caps it (`wind.c:560`), and -re-allocates the remainder on every partial read — `w->events = -estrdup(w->events+n); free(b);` (`xfid.c:1022-1023`). A client that stops reading -grows that buffer until `emalloc` fails and acme aborts. - -pardes's queue is length-framed (records contain newlines, so a length is the -only way to hand one back whole), capped at 64 KiB per pane (`queue_cap`), and -drops the oldest record when full: an editor must not stall or grow without bound -because a script stopped reading, and a reader that far behind can re-read `body` -and resynchronise. Formatted answers go into one staging buffer that is cleared -and never freed, which is why every read in the benchmark reports zero -allocations. **acme's unbounded buffer is a flaw, not a feature.** - -## 10. C-isms Zig removed - -Ranked by what they cost when they go wrong: - -1. **Threads and channels standing in for a state machine** — one thread per - in-flight request (`acme.c:718-744`, `xfid.c:42-55`) → a `Status.again` return - value and a park table in the transport. -2. **Macro-packed qids** — `QID/WIN/FILE` (`dat.h:463-465`) → `packed - struct(u64)` with one validating constructor. -3. **`Rune*` plus a byte-to-rune cache** with a scan-from-zero fallback - (`xfid.c:891-897`) → byte slices clamped to grapheme boundaries. -4. **`strtoul` pointer walking with `goto Rescue`** (`xfid.c:810-838`) → a slice - reader returning `?u32`. -5. **`longjmp`-ish `error()`** that aborts the process (`util.c:50-55`) → an - error union and a `Reply` value. -6. **Manual `realloc` growth** (`wind.c:560`) → `ArrayList` with retained - capacity. -7. **Sentinel-terminated tables** (`dirtab`, `fsys.c:62-74`; `dirtabw`, - `fsys.c:76-91`) → exhaustive enums, so adding a file to the tree does not - compile until every switch has an answer for it. -8. **`sprint` into fixed buffers** (`wind.c:534`) → `bufPrint` returning an - error. -9. **Ownership by convention** — `fbufalloc`/`fbuffree` pairs the caller must - match (`fns.h:5-6`) → `defer`, plus two explicit borrow windows - (`Payload.staged`, `Payload.region`) documented at the seam. -10. **Prefix-matched command tables** whose order is load-bearing - (`xfid.c:602-767`) → whole-token enum lookup. - -One property comes along with the transport rather than with either design: a -9P `Twrite` IS a message, so acme never sees a fragment, while a POSIX client -can call `write(2)` with one byte. A `ctl` verb, an `addr` expression and an -`event` record must therefore each arrive in a single write here, and a -fragment is EINVAL rather than state kept in the editor waiting for the rest. -Every ordinary client already does this (stdio buffers; `echo`, `dd` and -`os.write` are one call each), and the alternative — a per-pane line buffer — -would trade a clear error for a half-applied verb that never completes. - -## What is not served, and why - -`acme`, `consctl`, `draw`, `editout`, `label` — acme's root `dirtab` -(`fsys.c:62-74`) keeps them for rio and for its own `Edit` language, neither of -which pardes has, and `editout` appears in the per-window `dirtabw` -(`fsys.c:76-91`) for the same reason. Everything else in `dirtabw` is here: -`addr`, `body`, `ctl`, `data`, `errors`, `event`, `rdsel`, `tag`, `wrsel`, -`xdata`, and `index`, `cons` and `new/` at the root. `log` is NOT — and this is -the one entry that is not a decision about acme, because this acme's `dirtab` -has no `log` either; it is a plan9port addition. No example needed it, and a -script that wants to notice panes it did not open reads `index`, which is what -acme gives it. - -## What is served that acme does not have: `pty/` - -One directory, three files, and **no prior art anywhere**: acme has no terminals -and `ad` — the other editor that serves a control filesystem over 9P — has no -terminal surface at all (its tree is `{ctl, minibuffer, scratch, log, -buffers/…}`). So there is nobody's mistakes to learn from and nobody's scripts -to keep compatible, which is the argument for keeping it to three files and -stopping. - -A pty is a file interface wearing the wrong clothes: everything one wants to do -to it is an `ioctl`, and neither 9P nor FUSE has one. They become writes. - -| ioctl | here | -|---|---| -| `TIOCSWINSZ` | `winsize 80 24` → `pty/ctl` | -| `kill` | `sig INT` → `pty/ctl` | -| spawn | `exec` → `pty/ctl` | -| `TIOCGWINSZ` | read `pty/status` | -| `read`/`write` | `pty/data` | - -Two of the three verbs are effects the core already had — `push_spawn` and -`push_pty_resize` — so `exec` and `winsize` are existing capabilities acquiring -a name. `sig` is the one new host capability in the whole directory -(`push_pty_signal`, and there was no `kill` anywhere in `host_io.zig` before -it); it targets the tty's foreground process group rather than the shell's pid, -because an interactive shell ignores SIGINT while it waits for a job. - -The directory is **absent** on a pane that is not a terminal, rather than -present and refusing, so `test -d /pty` is how a script asks what kind of -pane it has. `pty/data`'s read is the only new state: the core keeps no raw pty -bytes anywhere (they go into the emulator grid, which is a rendering and cannot -be turned back into a stream), so they are queued as they arrive — gated on a -reader count exactly as `event` is, so a pane nobody is reading costs one -branch and no memory, and capped drop-oldest by the same `queue_cap`. Unlike -`event`, a read smaller than one arrival is SERVED and the remainder kept: raw -bytes have no record framing to split down the middle. - -Three things it deliberately does not have. There is no **exit status**, -because the core does not track one: a shell's death arrives as `Event.eof`, -whose whole handler is `removePane`, so by the time anyone could read a status -the directory is gone. There is no **`raw`/`cooked`**, because the termios -belongs to the program on the far side of the pty and it never reports one. And -`exec` takes **no argv**, because `Effect.spawn` carries a pane and a cwd and -has nowhere to put one — so `exec /bin/sh` is EINVAL rather than an argument -accepted and silently ignored. - -`Node.file` is a `u4`. These four variants (`pty`, `pty_ctl`, `pty_status`, -`pty_data`) take it to fifteen of sixteen used: **one value is left**, and the -next file added to a pane's directory needs a wider field and therefore a new -node-id layout. That is also why `pty/` is a directory rather than three more -names beside `body` — a subdirectory costs one value and buys a namespace, so -`ctl` and `data` did not have to be spelled twice. - -## Reading order - -`src/acmefs.zig` (semantics; start at its header), `src/fuse.zig` (the wire), -`src/fs_service.zig` (mount lifecycle), `examples/README.md` (the client's view), -`test/snapshots/acmefs.snap` and `acmefs-event.snap` (the drivers, i.e. what is -exercised end to end) beside their `.golden` files (what was observed), -`zig build fs-bench` (what it costs). diff --git a/docs/config.md b/docs/config.md index 448ac763..f895e599 100644 --- a/docs/config.md +++ b/docs/config.md @@ -12,9 +12,9 @@ main command file is named `init`: On the two unixes `XDG_CONFIG_HOME` counts only when it is ABSOLUTE, as the XDG base-directory specification requires; an empty or relative value falls -back to the home-directory form (`user_config.xdgBase`, and the test beside +back to the home-directory form (`config.User.path`, and the test beside it). Windows never consults it. An `init` that does not fit the `max_bytes` -read limit — 1 MiB, `src/user_config.zig` — or that cannot be read at all is +read limit — 1 MiB, `config.User` in `src/config.zig` — or that cannot be read at all is treated as no file: `load` takes the `readFileAlloc` error and keeps going. The path still resolves, because "nothing is there yet" is the answer `Config` exists to give. There is one case with no path at all: a native launch with no @@ -29,7 +29,7 @@ and size, tagline scale, panel transition, scene effects, hover delay, platform, native-image support, and (on SDL) whether the executable uses live-built shaders or the paired prebuilt shader snapshot. Platform-dependent rows say `unsupported` instead of looking like an off or -empty supported setting. The four fields of `runtime_config.Capabilities` gate +empty supported setting. The four fields of `config.Runtime.Capabilities` gate them and are stated once as plain data in `builtins.capabilities`: `font_picker` is the SDL GUI and native macOS only, `scene_shaders` the same two, `panel_transitions` every @@ -47,7 +47,7 @@ effective (last spawn)` is the executable the native host really chose after installation lookup and fallback. A changed request remains pending until a terminal is spawned, because the core does not resolve native executables. -The mutable global values live together in the plain `runtime_config.State` record. +The mutable global values live together in the plain `config.Runtime` record. One plain capability record gates the setting registry, leader table, `EffectCode`, and report; the compile-time setting table generates both setter builtins and their `Config` rows. Exhaustive checks require every table-backed @@ -75,12 +75,13 @@ Wrap A line matches a builtin whose name takes NO argument only as that whole word: `Kill` runs, `Kill something` does not. A builtin that takes one (`takes_arg` in `src/builtins.zig`, or a `settings` row whose action is -`shell`, `theme`, `font` or `tagline_size` in `src/runtime_config.zig`) takes +`shell`, `theme`, `font` or `tagline_size` in `config.Runtime.settings`) takes everything after the name as the argument. On a native build that is `Theme`, `ThemeFile`, `Font`, `TaglineSize`, `Shell`, `Save`, `Restore`, `Attach`, -`Find`, `Grep`, `Rename`, `WsSymbols`, `Look`, `Exec`, `Msg` and `EffectCode`. +`Mount`, `Unmount`, `Find`, `Grep`, `Rename`, `WsSymbols`, `Look`, `Exec`, +`Msg` and `EffectCode`. (`Peek`, `Poke`, `Hexdump` and `Gpio` take one too, but they exist only where -`board_memory.enabled` holds, and that build has no config file.) +`builtins.Board.enabled` holds, and that build has no config file.) `Theme ` wants one of the 228 names in the ring. Do not derive the spelling — read it off `ThemeSel` (`SPC t t`), which lists every one as the @@ -325,18 +326,12 @@ pass is bypassed. CRT works in linear light with restrained scanlines, mask, bloom, curvature, and noise rather than remapping the theme to a strong fixed palette; Ripple and Glitch primarily perturb sample coordinates. -`EffectCode ` opens the build-embedded effect math, host -paint/submission path, and backend shader/grid sources, for example -`EffectCode PanelAscii` or, in a GUI build, `EffectCode Crt`. TTY exposes it -for its grid transitions; native GUI builds -expose it for transitions and scene shaders. It is absent on web, where no -effect argument could succeed. Shared -passes are shown as shared source segments rather than manufactured per-effect -copies. The command works from an installed binary and does not need the source -checkout beside it. SDL output also labels its shader provenance. An ordinary -build prints the live GLSL that `glslc` compiled for that executable; -`-Dprebuilt-shaders` prints the tracked GLSL snapshot paired with the committed -SPIR-V instead and labels those segments with their `shaders/prebuilt/` paths. +`EffectCode PanelAscii` or `EffectCode Crt` lists the current backend's +build-embedded source paths under `/virtual`. Look opens each full file; +no checkout is needed. TTY exposes grid transitions, native GUI builds also +expose scene shaders, and web has neither. Shared implementations share paths. +SDL reports whether GLSL was compiled during this build or came from the +`-Dprebuilt-shaders` snapshot paired with the committed SPIR-V. `zig build shaders` refreshes both files of every pair together, so editing live GLSL without that explicit refresh changes neither half of a prebuilt executable. @@ -361,15 +356,15 @@ pub const look_preview_delay_frames: ?u16 = null; ## Build-time configuration -Everything above is chosen at runtime or in `src/config.zig`. The build itself -takes these, and this is the whole list — every `b.option` in `build.zig`, -besides `-Dtarget` and `-Doptimize` from `standardTargetOptions` and -`standardOptimizeOption`: +Runtime settings live in `src/config.zig`. Build options are listed below; +`zig build --help` lists the options available for the selected platform, +including the standard `-Dtarget` and `-Doptimize` options. | option | values | default | |---|---|---| | `-Dplatform` | `tty`, `gui`, `web`, `macos`, `esp32p4` | absent builds the tty cli and the SDL gui together | | `-Dstatic` | bool | `false` | +| `-Dquic` | bool; 9P over QUIC using system OpenSSL 3.6+ | `false` | | `-Dmupdf` | bool | on for a native target, off for web and esp32p4 | | `-Djpx` | bool | `true` — JPEG 2000, and with it scanned PDFs | | `-Dtree-sitter` | `disabled`, `zig`, `minimal`, `full` | `full` natively, `zig` for web, `disabled` for esp32p4 | @@ -379,17 +374,13 @@ besides `-Dtarget` and `-Doptimize` from `standardTargetOptions` and | `-Dmacos-identity` | codesigning identity for `pardes.app` | `-` (ad-hoc) | | `-Ddump` | a `dump.zon` to embed in the web shell | none | | `-Dtest-filter` | substring; run only tests whose name contains it | none | +| `-Dtest-rebuild` | bool; force fresh Zig test compilation, retaining cached C dependencies | `false` | +| `-Dhelix-harness` | native reference executable for live differential tests | `HX_HARNESS`, otherwise `hx-harness` on PATH | | `-Desp32p4-cols` | u16, the board's grid width in cells | `56` | | `-Desp32p4-rows` | u16, the board's grid height in cells | `14` | -| `-Desp32p4-cpu-mhz` | u16: `90`, `180` or `360` | `90`, the bootloader default | -| `-Desp32p4-port` | serial port the board is wired to | `/dev/ttyUSB0` | -| `-Desp32p4-prof` | bool; per-phase cycle counts for every frame | `false` | -| `-Desp32p4-firmware` | bool; also build the flashable image and its board steps | `false` | - -The five `-Desp32p4-*` options that are not `-Desp32p4-firmware` are registered -unconditionally rather than inside the `-Desp32p4-firmware` block that consumes -them, so `zig build --help` lists them and passing one without the flag is not -an "unknown option" error. + +The local board build emits an object. Firmware clock, serial port and profiling +options belong to the sibling `05-zig-p4` toolchain's build. Two build inputs reach the running binary as ordinary values rather than as behaviour. `build.zig` reads `.version` from `build.zig.zon` through an untyped diff --git a/docs/design.typ b/docs/design.typ index 9a92f7ee..8a5b9381 100644 --- a/docs/design.typ +++ b/docs/design.typ @@ -1,9 +1,4 @@ -// Diagrams come from cetz, which is the one thing here that is genuinely -// painful to hand-roll. Pinned to the version in the local package cache so -// this document builds offline; `typst compile docs/design.typ docs/design.pdf` -// must never need the network, because the PDF it produces is a test fixture -// (src/pdf.zig and src/pdf_pane*.zig open it, and `zig build mupdf-check` -// renders and searches it). +// docs/design.pdf is a retained test fixture, not regenerated by normal builds. #import "@preview/cetz:0.5.1" #set page(paper: "a4", margin: (x: 1.5cm, y: 1.8cm), columns: 2, numbering: "1") @@ -15,10 +10,6 @@ #show heading.where(level: 2): set text(size: 9.2pt) #show heading.where(level: 3): set text(size: 8.8pt, style: "italic") -// Listings are styled NATIVELY rather than through a package. codly is the -// obvious choice and the cached 1.2.0 does not compile under typst 0.15 — it -// still calls the pre-0.13 `pattern` — and a document that cannot be rebuilt is -// worse than one with plainer listings. Six lines buy independence from that. #show raw.where(block: true): it => block( width: 100%, fill: luma(246), @@ -33,9 +24,6 @@ #set figure(gap: 0.55em) #set table(stroke: 0.4pt, inset: 0.35em) -/// A figure that spans BOTH columns, for the diagrams and listings that will -/// not survive being folded into 8 cm. Floats to the top of a page, which is -/// where a reader expects a wide figure to be. #let wide(caption: none, body) = place( top, scope: "parent", @@ -44,9 +32,6 @@ figure(body, caption: caption), ) -/// Diagram labels are code more often than they are prose, and the inline-raw -/// rule above sizes for body text. Every canvas below is wrapped in this so a -/// symbol name inside a box does not outgrow the box. #let diagram(body) = [ #show raw.where(block: false): set text(size: 5.9pt) #body @@ -70,8 +55,8 @@ prototype had three parallel implementations of one editor. Two seams carry that. Outward, the core is a state machine over plain values: `Event` in, `Surface` and a queue of `Effect` out, and nothing that cannot - be said in those types exists in pardes. Downward, `host.VTable` is - twenty-one optional function pointers, every null one answered in-process, + be said in those types exists in pardes. Downward, `Host.VTable` is + optional function pointers, with in-process fallbacks, so the zero-method host is both the test harness and the browser. A session can also be a daemon: the core keeps the ptys and the disk and N frontends carry only a screen, a keyboard and a clipboard. @@ -106,7 +91,7 @@ instance's dump is a first-class feature of the one application, and the web shell is that application with an embedded dump and `spawn` left unanswered. Same core, no viewer fork. -The other acme mechanism is a control filesystem, `src/acmefs.zig` +The other acme mechanism is a control filesystem, `src/fs.zig` (@fs). == One core, five shells @@ -135,12 +120,12 @@ The five, and what each one actually is: the terminal (libvaxis); a native SDL3 window (the steamdeck); the browser (a freestanding wasm core driven by vanilla JavaScript and rendered as HTML/CSS); a native macOS app (an AppKit and CoreText shell over a static `libpardes.a`); and an ESP32-P4 microcontroller, a -freestanding riscv32 *object* that the `zig_p4` package links beside its own +freestanding riscv32 *object* that the sibling `05-zig-p4` toolchain links beside its own `_start`, linker script and UART driver (`src/esp32p4.zig` header; the `pardes-esp32p4` object `build.zig` emits). `pardes.Platform` is `enum { tty, gui, web, macos, esp32p4 }`. -Native shells share `shell_bin`'s OSC 133 startup snippets, but not their +Native shells share `host_io.Shell`'s OSC 133 startup snippets, but not their files. Each host owns a private `mkstemp` pair for its lifetime, writes and closes both before the first fork, passes those unpredictable paths directly in child argv, and unlinks them at teardown. Concurrent tty, SDL and macOS @@ -154,19 +139,17 @@ files. session can carry a terminal and an SDL window at the same time and outlive both. `main.nativeMain` dispatches `--detach` before it switches on the platform, because it is not a shell: the tty and gui builds can both be asked -for one. The two flags together are refused there rather than resolved by -declaration order, and so is `--fs` beside either of them — only a LOCAL session -mounts the control filesystem, so `--fs --detach` used to be parsed, stored and -served by nobody. Both flags take `=name` and never a separate word, so +for one. The two flags together are refused. Local and detached sessions both +serve 9P by default. Both flags take `=name` and never a separate word, so `pardes --attach README` opens `README` in a fresh session instead of attaching to one called `README`. See @detached and `docs/detached.md`. = The seam #wide(caption: [The core/shell seam. `Event` is the only way in; `Surface` and a -queue of `Effect` are the only ways out. `host.VTable` is how an `Effect` +queue of `Effect` are the only ways out. `Host.VTable` is how an `Effect` reaches a resource, and every method a host leaves null is answered by -`host.Fallback` inside the same process.])[ +`host_io.Fallback` inside the same process.])[ #diagram[ #cetz.canvas(length: 0.995cm, { import cetz.draw: * @@ -212,15 +195,15 @@ reaches a resource, and every method a host leaves null is answered by // ---- the vtable ---- rect((6.0, -1.0), (11.6, 0.4), name: "vt") - content((8.8, 0.14), text(7.0pt)[`host.VTable` --- 21 optional methods]) - content((8.8, -0.32), text(6.0pt)[`push_` #sym.arrow.r every host, returns nothing]) - content((8.8, -0.68), text(6.0pt)[`pull_` #sym.arrow.r exactly one host answers]) + content((8.8, 0.14), text(7.0pt)[`Host.VTable` --- optional callbacks]) + content((8.8, -0.32), text(6.0pt)[one host owns each core]) + content((8.8, -0.68), text(6.0pt)[null callbacks use core fallbacks]) line((8.8, 0.9), (8.8, 0.4), mark: (end: "stealth")) line("vt.east", (13.3, 1.5), mark: (end: "stealth")) // ---- fallback ---- rect((0, -1.0), (4.3, 0.9), name: "fb") - content((2.15, 0.62), text(7.0pt)[`host.Fallback`]) + content((2.15, 0.62), text(7.0pt)[`host_io.Fallback`]) content((2.15, 0.18), text(6.0pt)[every null method, answered here:]) content((2.15, -0.18), text(6.0pt)[a virtual filesystem over the]) content((2.15, -0.52), text(6.0pt)[embedded source, a virtual]) @@ -316,7 +299,7 @@ pub const Effect = union(enum) { theme_file: struct { generation: u32, on: bool }, dump_themes: struct { pane: u8 }, - fs_reply: acmefs.Reply, + fs_reply: filesystem.Reply, attach: struct { pane: u8, name: Buf(attach_name_max) }, detach: struct { pane: u8 }, @@ -408,27 +391,24 @@ for a pty; everything else queues O(1) effects per event, and the input queue holds at most 64 events per pump, so 128 leaves two effects per queued event. A build with no terminal panes has no pty to write to at all. -== `host.VTable`: who serves the core +== `Host.VTable`: who serves the core -The seam itself is one struct of twenty-one optional function pointers -(`host.VTable`): the `std.mem.Allocator` shape, and the generalization of -two vtables this codebase already grew on its own: the tty shell's -terminal-query hook, which this replaced, and `macos.Runtime`, which survives as -the C ABI that shell's host still enters through. +`host_io.Host` holds a context pointer and optional callbacks. macOS enters +its native shell through the separate `Runtime` C ABI. ```zig pub const VTable = struct { /// The ONLY place the process may sleep. - pull_wait_input: ?*const fn ( + wait_input: ?*const fn ( ctx: ?*anyopaque, timeout_ms: u32, ) void = null, - push_present: ?*const fn ( + present: ?*const fn ( ctx: ?*anyopaque, surface: *const pardes.Surface, ) void = null, // ... - pull_tty_taken: ?*const fn ( + tty_taken: ?*const fn ( ctx: ?*anyopaque, pane: u8, ) bool = null, @@ -437,17 +417,16 @@ pub const VTable = struct { ``` A null method is not an error: the core substitutes a default backed by ordinary -data structures in the same process (`host.Fallback`). A `Save` lands in a real +data structures in the same process (`host_io.Fallback`). A `Save` lands in a real file under the tty host and in `Fallback.files` under a host that never wrote a filesystem method, and every path above that behaves identically. Two consequences were taken on purpose. The zero-method host IS the test harness: a `Host{}` is a complete, deterministic, in-process pardes with a virtual filesystem, a virtual clipboard and silent ptys. And `Fallback` lives on the -`Pardes` instance rather than on the host, so N cores driven by one fan-out host -each keep their own state and can run in parallel. +`Pardes` instance rather than on the host, so cores keep independent state. The fallback filesystem is not empty. It is pardes's own source, embedded -(`src/source_manifest.zig`), with `files` holding only what this session WROTE, +(`src/fs.zig`), with `files` holding only what this session WROTE, so a Save shadows the built-in copy and reading it back returns the edit. That is what makes a host with no file methods a usable pardes rather than one staring at an empty buffer. @@ -472,46 +451,11 @@ platforms, which is exactly the pair `main.zig` accepts `--attach` for: the same question asked at the command line instead of in a tag. A capability that a build cannot serve should not be a word that build offers. -=== The naming rule is compiler-enforced +=== Callback ownership -Every method name says how a fan-out must route it, and `Fanout.isPull` makes a -wrong name a compile error rather than a silent push. - -#wide(caption: [`host.Fanout.isPull`. The failure of a forgotten `pull_` is -invisible in every unit test and obvious only to the user: one Ctrl-V pasting -twice. `Fanout.all` synthesizes one wrapper per field, so adding a method to -`VTable` needs no code in the fan-out at all.])[ -```zig -/// How to route a method, read off its own name. A method that is neither -/// is a COMPILE ERROR rather than a silent push, because the failure of a -/// forgotten pull is invisible in every unit test and obvious only to the -/// user: one Ctrl-V pasting twice. -fn isPull(comptime name: []const u8) bool { - if (std.mem.startsWith(u8, name, "pull_")) return true; - if (std.mem.startsWith(u8, name, "push_")) { - if (Method(name).return_type.? != void) @compileError("Host.VTable." ++ - name ++ " reaches every host, so it cannot return a value: whose answer would it be?"); - return false; - } - @compileError("Host.VTable." ++ name ++ " must be named push_… (every host gets it) " ++ - "or pull_… (exactly one host serves it, because there is one of whatever comes back)"); -} -``` -] - -`push_` reaches every wrapped host and returns nothing — a push with an answer -would have N answers and no way to pick one. `pull_` is served by exactly one -host, because there is one of whatever comes back: one value, one sleep that -ends, one `Event.paste` for one Ctrl-V, one `lsp_resp` per request id. - -The six `pull_` methods are therefore exactly the six places a single answer -exists: `pull_wait_input` (the one place the process may sleep), -`pull_tty_taken`, `pull_gpio_toggle`, `pull_read_clipboard`, `pull_lsp` and -`pull_pipe`. `pull_tty_taken` is a pull and not a pushed fact because the -`execute` that asks — has a program taken this pane's tty? — must choose a -destination inside its own update, and effects drain after; a pushed fact would -mean every host probing every pane's processes every frame to answer a question -asked when a human middle-clicks a word. +Each core has one host. The detached host explicitly broadcasts shared state +and routes clipboard reads and link opening to the originating frontend. +There is no name-based fan-out layer. == The frame, as a frontend writes it @@ -519,7 +463,7 @@ asked when a human middle-clicks a word. pub fn pump(p: *Pardes, h: Host) !void { p.host = h; const v = h.vtable; - if (v.pull_wait_input) |f| + if (v.wait_input) |f| f(h.ctx, if (p.animationActive()) animation.frame_ms else 0); while (p.nextQueued()) |ev| p.update(ev); @@ -527,12 +471,12 @@ pub fn pump(p: *Pardes, h: Host) !void { // A quitting frame has already freed // what it would draw. if (p.quit) return; - if (v.push_poll_frame) |f| f(h.ctx); + if (v.poll_frame) |f| f(h.ctx); _ = p.frame_arena.reset(.retain_capacity); const surface = try p.render( p.frame_arena.allocator()); - if (v.push_present) |f| f(h.ctx, surface); - if (v.push_post_present) |f| f(h.ctx); + if (v.present) |f| f(h.ctx, surface); + if (v.post_present) |f| f(h.ctx); // ... } ``` @@ -543,12 +487,12 @@ then every queued event, to completion; then every effect, to completion, including effects `perform` queued; then one arena reset and one render; then present, then post-present. -Animation time is not spent in here. `pull_wait_input` was told how long it may +Animation time is not spent in here. `wait_input` was told how long it may sleep, and a display clock wakes faster than that on input, so only the host knows when a real frame interval has passed. Each spends it by handing back one `.tick`. -`push_post_present` is split from `push_present` because it must observe a frame +`post_present` is split from `present` because it must observe a frame the user has actually seen: panel-presentation acknowledgement and pointer refresh both depend on that, and a hook that ran before the pixels landed would acknowledge a frame that was never shown. @@ -581,14 +525,10 @@ The core's one `pointerOperand` primitive owns click-word expansion and is share verbatim by right-click and the delayed hover preview; that policy stays beside input because it also observes live pane selections and wrapped grid coordinates. -Pane implementations are similarly flat and direct: `file_pane.zig`, -`term_pane.zig`, `image_pane.zig`, `output_pane.zig`, and `pdf_pane.zig` own -their kind-specific storage and operations. `pardes.zig` keeps the layout, input -dispatch, cross-pane invariants, and the small calls joining those modules. -There is no pane vtable or callback layer; the kind is already plain data, so a -direct switch/call is the shortest boundary. The large end-to-end PDF cases live -in `pdf_pane_integration_test.zig`, keeping pane-specific fixtures and raster -assertions out of that core file as well. +`panes.zig` keeps each pane kind's storage and operations together. +`layout.zig` owns placement and presentation state. `pardes.zig` handles input +and cross-pane state directly, without a pane vtable. Integration fixtures live +in `test/panes.zig`, `test/output.zig`, and `test/pdf.zig`. = State @@ -600,57 +540,30 @@ grow with what you open; everything else is sized at init. ```zig Pardes - ncol + col_weight[6], col_terms[6][16], col_n[6] - panes: [16]?*Pane // slot array; id = index - active: usize - drag: Drag // none | border_v | - // border_h | move | - // tag | select - settings: runtime_config.State - rects: [16]Rect // where each pane - // landed, this frame - panel_tracks: [16]?Track // live panes, - // serial-guarded - presented_panel_tracks:[16]?Track - closing_panel_tracks: [16]Track // dense visual - // tombstones, no owner - presented_cells + panel_cell_diffs + ncol + col_weight[6], col_panes[6][16], col_n[6] + panes: [16]?*Pane + active + drag + config.Runtime + rects: [16]layout.Rect + presentation: layout.Presentation effects: [limits.effect_cap]Effect + head/len in_q: [64]Event + head/len - fallback: host.Fallback // per instance - fs: acmefs.State // inert until --fs + fallback: host_io.Fallback + fs: fs.Namespace Pane - tag: TagLine // live prefix (cwd/path) - // + editable tail - vt, stream: ghostty-vt Terminal and its stream — - on EVERY pane, not just terminals, - which is what lets the same keys and - the same parity suite drive a file - and a shell - file: ?file_pane.State - image: ?image_pane.State - pdf: PdfSlot // payload presence is the - // kind; none = terminal - vweight: f32 - mode: enum { normal, insert, tty } - cursor: absolute body position // rides the - // scrollback, not the screen - msel/vsel + sels[63] + nsel // the primary - // range, and up to 63 more - ovl: ?term_pane.EditBuffer // ONE typed run, - // anchored to an absolute row - undo: two stacks, not one — term_pane.Snapshot - history for an edit buffer, and - file_pane.State history for file content - -file_pane.State = path + bytes + line index - + Syn (tree-sitter bytes) -image_pane.State = decoded RGBA + petscii cache -pdf_pane.State = MuPDF document + continuous - layout + search/selection/outline - + bounded per-page raster relay - + frame placement decisions + serial + mode + vweight + terminal: ?*panes.Terminal.State + file: ?panes.File.State + image: ?panes.Image.State + pdf: PdfSlot + cwd: none | inherited(*Pane) | owned([]u8) + tag_tail + prompt + cursor + selections + ovl: ?panes.Terminal.EditBuffer + +Terminal.State = VT + stream + replay + reply +File.State = path + bytes + line index + syntax + undo +Image.State = decoded pixels + render cache +Pdf.State = document + layout + raster cache + search ``` `MAX_PANES` is 16 and `MAX_COLS` is 6 (`pardes.MAX_PANES`, `pardes.MAX_COLS`). Sixteen panes @@ -777,11 +690,11 @@ does not touch. == Runtime settings are one table -User-settable runtime choices are one plain `runtime_config.State`: booleans, +User-settable runtime choices are one plain `config.Runtime`: booleans, theme index, owned bounded shell/font strings, requested/effective font facts, one panel-transition enum, and scene-effect booleans -(`runtime_config.State`). A compile-time `settings` array -(`runtime_config.settings`) generates each setting builtin and the rows of the +(`config.Runtime`). A compile-time `settings` array +(`config.Runtime.settings`) generates each setting builtin and the rows of the single `Config` query. It has no callbacks and no parallel query registry to drift from it. @@ -871,7 +784,7 @@ selectable, yankable and executable but READ-ONLY: every edit op measures from `tag_tail` is a fixed `[max_tag_tail]u8`, and the bound IS the storage (`limits.max_tag_tail`): every writer — `appendTag`, `tagInsert`, -`restoreDumpTail`, the acmefs `tag` file — refuses input that does not fit +`restoreDumpTail`, the 9P `tag` file — refuses input that does not fit rather than truncating it. The schema limit and the buffer therefore can never disagree, which is what lets a dump reader reject data before copying it into a pane. @@ -884,10 +797,10 @@ clicking a tag never changes a pane's mode. == Eleven transitions -`panel_animation.zig` is backend-neutral data and math: the transition +`layout.zig` is backend-neutral data and math: the transition vocabulary, easing, exact endpoint progress, stable per-cell noise, and a POD track. `Transition` has twelve members counting `off` -(`panel_animation.Transition`), with explicit numeric values because they +(`layout.Transition`), with explicit numeric values because they cross both GUI shader ABIs — GLSL receives the enum in an instance `uvec4` and the Core Image kernel receives it as a float, so spelling the numbers keeps a source reorder from changing pixels. @@ -963,7 +876,7 @@ An `.ascii` diff is a `{ from: u8, to: u8 }` pair, and the core composes it by incrementing or decrementing the printable byte. Short walks move one value per frame; a longer walk is crossed by eased character skips and finishes within `ascii_max_movement_frames`, which is 12 -(`panel_animation.ascii_max_movement_frames`). Frame +(`layout.ascii_max_movement_frames`). Frame zero is the exact old byte and the endpoint is exact, so an intermediate frame is always valid UTF-8. `Track.frame_count` carries the core-computed duration for these data-dependent effects — zero selects the effect preset, and the ASCII @@ -997,11 +910,9 @@ DOM web is a separate platform, not a shader GUI: retaining selectable HTML and CSS is more important than duplicating the renderer in canvas, so it exposes neither effect family. -`EffectCode ` writes the actual backend math, host submission, and -shader or grid source segments embedded by the build into an ordinary output -pane. This makes the implementation inspectable after installation and makes -sharing explicit: several builtins can quite honestly print the same shader with -different uniform bits. +`EffectCode ` lists the current backend's build-embedded source paths +under `/virtual`. Look opens each full file without a source checkout. +Shared implementations share paths. == The ASCII fast paths @@ -1039,7 +950,7 @@ configuration — 40×12, no tree-sitter — which was the largest single item t `\t`, `\r`, the C0 controls and DEL are excluded by the range test and keep their existing handling. -The second is `file_pane.fitEnd` (`file_pane.fitEnd`), which decides +The second is `panes.File.fitEnd` (`panes.File.fitEnd`), which decides where a soft-wrapped row breaks. It asks `modal.nextGrapheme` and `graphemeDisplayWidth` once per character, and a 640-column line asks 640 times. @@ -1133,8 +1044,8 @@ from `src/detached/server.zig:34-59`.])[ row(-0.92, text(5.6pt)[`read_clipboard`'s answer is not a reply message: it comes back as an ordinary `Event.paste`]) row(-1.40, text(5.6pt)[the gaps `0x13`..`0x17`, `0x1e` were `output` `eof` `lsp_resp` `pipe_resp` `file_changed` `tick`: deleted, not renumbered,]) row(-1.78, text(5.6pt)[when the daemon took the disk --- a decodable `output` let an attached peer forge a pane's text]) - row(-2.16, text(5.6pt)[never on the wire: `pull_wait_input` (it IS the poll loop), the two informationless frame pushes, the four `pull_`s]) - row(-2.56, text(5.6pt)[that answer their own caller, and `push_fs_reply` --- with N frontends, N#sym.minus 1 would get an answer they never asked for]) + row(-2.16, text(5.6pt)[never on the wire: `wait_input` (it IS the poll loop), the two informationless frame pushes, the four `pull_`s]) + row(-2.56, text(5.6pt)[that answer their own caller, and `fs_reply` --- with N frontends, N#sym.minus 1 would get an answer they never asked for]) }) ] ] @@ -1159,10 +1070,8 @@ detach, kill the terminal, attach from another one, and the build that was running in pane 3 is still running and has been scrolling into the core the whole time. -Of the twenty-one `VTable` methods, the detached core's `Session` implements -seventeen and leaves four null: `push_post_present`, `pull_gpio_toggle`, -`pull_lsp`, `pull_pipe`. It mounts its own `/dev/fuse` and polls it in the same -`poll(2)` as its frontends, so `--fs` works in a daemon and needs no thread. +The detached core also serves the default 9P socket. Its event loop drains +filesystem transactions alongside frontend and terminal input. Nothing blocks indefinitely, and that property is what a detached session is *for*. Every descriptor is non-blocking; the single `poll(2)` is the only place @@ -1238,17 +1147,10 @@ path, because neither of them writes anything. effects, because it is not an effect the session performs on the world: it is one frontend being told it is done. -Four `VTable` methods are named in the file with their reasons for *not* being on -the wire. `pull_wait_input` IS the server's poll loop. `push_poll_frame` and -`push_post_present` carry no information — `frame` already arrives exactly once -per pump at the same place in the order, so two more messages per frame per -client would say nothing the frame does not. `pull_tty_taken`, -`pull_gpio_toggle`, `pull_lsp` and `pull_pipe` are answers the caller waits for -or work dispatched off the loop, and a round trip inside `update` is the one -thing this transport must never do. `push_fs_reply` cannot be a broadcast at all: -the transport that asked is the one holding the request, so with N frontends, -N−1 would receive the answer to a request they never made — which is why the -acme mount stays in the detached process and `Event.fs_req` has no `ClientTag`. +Host polling, presentation, process queries and worker dispatch stay local to +the session owner. They are not frontend wire messages. The owner also serves +9P: each filesystem reply returns to its requesting connection, not to attached +frontends. `Event.fs_req` therefore has no `ClientTag`. === The codec is architecture- and build-neutral @@ -1272,11 +1174,8 @@ by a build with nowhere to put it. `version` is a `u16` checked on connect and refused loudly, because two builds of pardes are routinely on one machine — `zig build` replaces the binary under a running session — and a frontend decoding another version's frame layout would -paint garbage and blame the terminal. `ClientTag` is exhaustive on purpose, which -is the opposite of `fuse.zig`'s `Opcode`: there a newer *kernel* adds opcodes, -and a non-exhaustive enum is the only way to receive one without undefined -behaviour, whereas here both ends are pardes and an unknown tag is a corrupt or -hostile stream. +paint garbage and blame the terminal. `ClientTag` is exhaustive: both ends are +pardes, and an unknown tag is not a valid message in this protocol version. `max_payload` is 16 MiB, derived rather than chosen: a full frame of the largest grid this protocol admits (512×128) at a worst case of one run per cell is @@ -1386,14 +1285,13 @@ in order to. == An object, not a module `-Dplatform=esp32p4` emits ONE freestanding riscv32 object exporting the C ABI in -`src/esp32p4.zig`; the `zig_p4` package links it beside its own `_start`, its +`src/esp32p4.zig`; the sibling `05-zig-p4` toolchain links it beside its own `_start`, its generated linker script, and its UART driver (the `pardes-esp32p4` object `build.zig` emits). Not an executable, because the entry point is over there. Not a library, because `addLibrary` bundles a `compiler_rt` the firmware already has. -And not a module exposed through `build.zig.zon`, which is what it was first and -is the interesting part. A dependency in the OTHER direction was built and +Historically, it was a module exposed through `build.zig.zon`. A dependency in the OTHER direction was built and reverted: nesting this package's roughly 30-package graph under `zig-p4`'s broke every build in that repo, not just the firmware one. `std/Build.zig:2091` exceeded its @@ -1401,18 +1299,16 @@ exceeded its seven cached tree-sitter versions failed to compile because their `build.zig` uses APIs removed in 0.16, and the fetch materialised 2.6 GB across 42,736 files into a repo whose entire claim is that Zig is its only dependency. This direction -is free: `zig_p4` declares no dependencies at all, so it enlarges nothing here, -and its `build()` early-returns when it is not the root. - -The seam stays bytes over eight C functions, because bytes are the right seam for -a serial line and because that is the arrangement the board was measured -through. `-Desp32p4-firmware` adds the other half in this tree — the firmware -executable rooted at `src/esp32p4/app.zig`, the flashable image, and the steps -that write it to a board and talk to it — and the image it produces is -byte-identical to the one the toolchain repo produces from the same sources. It -is opt-in and not out of timidity: `esp32p4.firmware()` reads ESP-IDF's register -headers at CONFIGURE time and exits non-zero when there is no checkout, so a bare -`-Dplatform=esp32p4` must not call it. +was possible because `zig_p4` declared no dependencies of its own. The current +editor build no longer imports that package; the firmware toolchain is separate. + +The C ABI carries terminal bytes. After building the object here, `zig build +-Dpardes` in `../05-zig-p4` links `src/esp32p4/app.zig` into the firmware image. +That toolchain needs ESP-IDF register headers; the local object build does not. +The standalone GPIO 9P image instead uses `zig build +-Dapp=../02-pardes-code/src/esp32p4_9p.zig` there. It does not link the editor: +`src/esp32p4_gpio.zig` holds its fixed namespace and `src/esp32p4_9p.zig` drives +the UART protocol loop. The object is also the compile probe. Rooted at `src/esp32p4.zig` it drags the whole core through the riscv32 backend by actually calling it, so `llvm-size` on @@ -1430,12 +1326,14 @@ any other input, because firmware has no `TIOCGWINSZ`. == The memory budget is one table -`src/limits.zig` is every board-shaped capacity in one place. These numbers used +`src/memory.zig`'s `limits` contains the board-shaped capacities. These numbers used to be nine `platform == .esp32p4` tests scattered across nine files, each one a separate place to forget — and they are not nine decisions. They are ONE decision, how much memory this build is allowed to spend, taken nine times where no reader could see the total. +The original budget table below is historical; `memory.limits` is authoritative. + ```zig /// `board` is the ESP32-P4 firmware's budget: /// a 384 KiB heap and a 240 KiB chunk of L2MEM @@ -1519,7 +1417,7 @@ pub const arena = struct { ] What does NOT belong in this table is capability switches. `terminal_panes`, -`board_memory.enabled`, `hosted` and `font_picker` answer "does this build have +`builtins.Board.enabled`, `hosted` and `font_picker` answer "does this build have the thing at all", which is a question about the platform and not about a budget, so they stay next to the thing they gate. @@ -1554,84 +1452,58 @@ megabytes against a 1.5 MiB partition (`build.zig:298`). MuPDF is refused for th same reason (`build.zig:297`). The board gains four words nothing else has, all gated on -`board_memory.enabled == (platform == .esp32p4)`: `Peek`, `Poke`, `Hexdump` and +`builtins.Board.enabled == (platform == .esp32p4)`: `Peek`, `Poke`, `Hexdump` and `Gpio`. Each takes an address or a pin, so none can have a leader path — a key path names a builtin and can never carry an operand. `Gpio` is the one that goes through the host seam (@seam) rather than reaching the registers directly, and -`pull_gpio_toggle`'s comment says why: driving a pad correctly is not one +`gpio_toggle`'s comment says why: driving a pad correctly is not one register. It is the IO MUX function select, the GPIO matrix output route, the pad's drive and input-buffer bits, and the output enable, keyed by a per-pin table. The firmware already owns that code and checks it against ESP-IDF's own headers on the die; a second copy in the core would be a second copy nobody tests. -`board_memory.zig` makes the target the *witness* rather than the gate: `enabled` +`builtins.Board` makes the target the *witness* rather than the gate: `enabled` is keyed on the platform, and a `comptime` block then refuses to compile if that platform is hosted, is not freestanding, or is wasm — because whatever else `esp32p4` means, it has to still be a machine whose addresses are the bus's -(`board_memory.enabled`). +(`builtins.Board.enabled`). = The control filesystem -== acmefs as a pure transaction - -plan9's acme serves `/mnt/acme`: a directory per window holding `addr`, `body`, -`ctl`, `data`, `event`, `tag`, and a program that opens those files IS an editor -extension — no plugin API, no embedded interpreter, no rebuild. `pardes --fs` -serves the same tree over Linux FUSE (`src/fuse.zig`), and `src/acmefs.zig` is -the whole of what the files MEAN. - -A filesystem is a request/response protocol driven by other processes, which is -exactly the kind of concurrency the core does not have and must not grow. acme -answers it with a thread per in-flight request — `xfidallocthread`, a `Channel` -per `Xfid`, a `QLock` per window. pardes cannot and should not, so `acmefs.zig` -is a pure main-thread transaction, `handle(p, req) Reply`: no thread, no waiting, -no callback, no allocation on the hot path, freestanding-safe, and unit-testable -with no FUSE anywhere near it. - -Requests arrive as an ordinary `Event.fs_req` and answers leave as an ordinary -`Effect.fs_reply`, so the transport is the queue every other host/core message -already uses. A backend with no threads at all is not a special case: it either -never sends a request, or sends one from its own frame loop. And acme's blocking -`event` read — which waits for the user to do something, parking the `Xfid` in -`w->eventx` until a later `winevent` sends it a message — is `Status.again` here: -"nothing consumed, ask me again". The waiting lives in the host, which is where -the kernel's request already is. The core keeps no waiter list and no wakeups. - -One divergence from acme is deliberate: acme counts RUNES, pardes counts BYTES, -clamped to grapheme boundaries. Every offset in this filesystem — `addr`, `data`, -the event records' q0/q1, `index`'s lengths — is a byte offset, because pardes is -byte-addressed end to end (selections, look spots, LSP offsets) and a second -coordinate system would mean an O(n) conversion at every boundary and a lossy -`addr=dot`. acme pays that cost the other way round: it keeps the document as -`Rune*` and converts on every utf read, in a function that carries a -"BUG: stupid code: scan from beginning" comment for its cache miss. The two agree -for ASCII, which is what scripts compute with. - -`Pardes.fs` is `acmefs.State`, zero-initialised and inert: a core nobody scripts -pays one branch per edit and nothing else. - -== The nested socket - -`src/nested.zig` listens on `/pardes-.sock`, where `` is -`$XDG_RUNTIME_DIR` — a per-user 0700 tmpfs the login session already cleans up — -or `~/.local/state/pardes` when the session has none. It accepts exactly one -verb, `Look [:]`, and nothing else, which is the security property. -That is how a `pardes ` run inside a pardes hands the file to the outer -session instead of stacking a second full-screen UI inside one of its panes. - -It arrives as an `Event.command` rather than a direct `executeBuiltinLine` call -so that it gets the trailing sync and the ordinary effect drain: `Look` on a -directory emits a `.spawn` the shell has to perform. - -The detached sockets live in the same per-user directory under a different name, -`pardes-detached-.sock`, and both sides derive the path from one predicate -in one file so that the side which binds and the side which connects cannot -disagree (`server.socketPath` and `server.sessionPath`). A frontend that finds a socket at -a derivable path which is not a private one of ours refuses with `NotPrivate` -rather than reporting "no session": a planted socket collects every keystroke -typed into the frontend that trusts it, and the two cases need different answers -from a human. +== Namespace and transactions + +`src/fs.zig` owns Look resolution and the editor's file interface. An ordinary +Look checks the OS first, then the virtual tree. `/n/os` and `/n/self` select +those mounts explicitly; `/virtual` names the embedded and self-reflecting tree. +Named remote mounts live under `/n/` and retain their identity through Save. + +Every native session opens a 9P2000 Unix socket. The wire root exposes `os` and +`self`, without the editor's `/n` prefix. `self/pane/` contains `body`, +`tag`, `ctl`, `addr`, `data`, `event`, and selection files. Offsets are UTF-8 +bytes. `self/screen` freezes rendered cells and styles for the lifetime of an open. +See `docs/fs.md` for the public paths and commands. + +`src/9p.zig` implements the protocol without OS dependencies; `src/9p_io.zig` +owns native sockets and the client. Requests enter through `Event.fs_req`, and +`Effect.fs_reply` carries replies. A pending event read returns `Status.again`; +the native listener owns waiting and retries. Filesystem mutation runs on the +same thread as editing. + +== Nested Look + +Pane shells inherit `PARDES_9P`, `PARDES_PANE` (the pane serial), and +`PARDES_FORWARD_LOOK`. A child launch resolves its OS-relative argument in +the child's working directory, then writes `look ` to the parent's +`self/pane//ctl`. Explicit `/virtual` and `/n` paths resolve in the +parent. The ordinary filesystem update performs layout and drains host effects. + +`--nested` starts a separate editor and disables forwarding from its direct +pane shells. Its 9P socket remains available for control and plugins. There is +no executable-name discovery or separate Look listener. + +Detached frontends use `pardes-detached-.sock` in the same runtime +directory. The shared Unix socket conventions live in `src/9p_io.zig`. = Build @@ -1694,7 +1566,7 @@ is the only thing two of them are allowed to disagree about. Line numbers are out(2.12, [`pardes-gui`], [`-Dplatform=gui` --- SDL3, a FreeType atlas, SPIR-V]) out(1.72, [`pardes.wasm`], [`-Dplatform=web -Dtarget=wasm32-freestanding -Ddump=`, plus a vanilla DOM shell]) out(1.32, [`libpardes.a`], [`-Dplatform=macos`, then the `macos-app` step #sym.arrow.r `pardes.app` (Darwin host)]) - out(0.92, [`pardes-esp32p4.o`], [`-Dplatform=esp32p4` (target forced, `:171`); `-Desp32p4-firmware` adds the image and the board steps]) + out(0.92, [`pardes-esp32p4.o`], [`-Dplatform=esp32p4`; the sibling `05-zig-p4` toolchain builds the firmware image]) row(0.46, text(5.6pt)[a bare `zig build` emits the FIRST TWO together, because those two are what installing pardes means; every other spelling is one invocation each]) row(0.18, text(5.6pt)[beside them, from the same graph: `pardes-isolate` (tty only --- the filesystem is not compiled in) and `hxdiff`'s headless core]) @@ -1758,7 +1630,8 @@ Native dependencies are ghostty, vaxis, uucode (shared config), zstbi, SDL (a pinned fork, lazy), FreeType, MuPDF (`-Dmupdf`, on by default everywhere but the web and the board, lazy), ZLS, mvzr (the regex engine behind `s`/`S`), zig-tree-sitter with 29 grammars for 28 languages (markdown takes two, block and -inline), and `zig_p4` — all pinned through `zig fetch` and wired in `build.zig`. +inline) — all pinned through `zig fetch` and wired in `build.zig`. The board's +`05-zig-p4` firmware toolchain is a separate sibling checkout. Generated during the build: `highlights.scm` into an options module; the vendored helix and zed theme sources into the generated half of the theme ring; @@ -1932,9 +1805,9 @@ over CDP with real DOM pointer and touch events and diff `test/web-snapshots/`; `image-harness` and `pdf-harness` snapshot native PIXEL output through kitty graphics and SDL; `macos-e2e` is an offscreen AppKit snapshot suite over its own seven scripts (`test/macos-snapshots/`: boot, cwd, drop, font, keys, rotate, -trackpad). `esp32p4-test` runs an on-die suite on real hardware, and -`esp32p4-image-size` and `esp32p4-image-check` answer the flash-budget question -with no board attached. +trackpad). In the sibling `05-zig-p4` toolchain, `zig build selftest` flashes and +runs `src/esp32p4/selftest.zig` on real hardware. The local freestanding object +and the standalone GPIO 9P image can both be compiled without a board attached. Two suites are differential rather than golden: `hxdiff` compares the core's motion and operator results against helix case by case, and `hxparity` compares @@ -1970,10 +1843,6 @@ plugin system, because the control filesystem is the extension point (@fs) and needs no API of its own. No async runtime in the core — the shells may thread, the core is single-threaded by construction. -No 9P. The library boundary is exactly where acme put the file server, and the -FUSE mount is already that server with a Linux transport instead of a 9P one; a -sixth shell could serve `Surface` and `Event` over 9P without touching the core. - "No config files" held until the startup file (`docs/config.md`) arrived, and that is the narrowest thing the phrase could still cover: a list of builtin COMMANDS run before the first frame — no schema, no new vocabulary, and no key remapping. @@ -2080,14 +1949,11 @@ selection, the document outline into `+PdfSections`, fit-width/fit-height and a themed duotone tint — the last three named in the pane's own live tag. Tutor: embedded text as file pane. -*Language.* ZLS compiled in and called IN-PROCESS — no subprocess, no -JSON-RPC, no daemon and nothing cached between queries — behind a -one-function seam (`lsp.query`) so that swapping a backend touches nothing -else. `.zig` only, and on demand only: helix's five `g` gotos, ten builtins -under `SPC l`, `]d`/`[d`, `=`, and insert-mode Tab after a `.`. Answers become -rows in `+Search`, `+Hover` or `+Lsp` — output buffers of the same kind `/`, -Find and Help fill — or, for a rename, byte ranges the core applies in one undo. The -web shell has no threads and therefore no backend at all. +*Language.* The native host snapshots each request and runs it outside the UI +loop. Zig uses the in-process ZLS backend; configured external language servers +use the JSON-RPC client. Results become output rows or edits, applied only while +the request's pane and revision still match. Web and board builds have no +language backend. See `docs/lsp.md` for configuration and commands. *Chrome.* One theme per `.zig` file, folded into the ring at comptime: ours first — helix (default; near-black page, chrome one grey-ramp step off it, @@ -2119,14 +1985,14 @@ starting point without adding inheritance or a second theme vocabulary. Colors toggles all recolor passes; Debug stats overlay; Dump writes state ZON. Eleven panel transitions and three scene bits are settings, one builtin each, generated from -`runtime_config.settings`. +`config.Runtime.settings`. *Session.* `--detach[=name]` runs a core with no terminal; `--attach[=name]` makes a thin frontend over a unix socket; `Attach [name]` (`SPC s a`) hands a running frontend's screen to a detached core, connecting before it swaps; `Detach` (`SPC s D`) leaves a session that carries on. Up to 32 frontends on one session, all showing the same screen; the pane shells belong to the daemon and -outlive every frontend. `--fs` serves the acme control tree over FUSE; a nested +outlive every frontend. The default 9P socket serves the control tree; a nested `pardes ` hands its argument to the outer session over the per-pid socket. `pardes --version` prints the manifest version and the commit it was configured from. diff --git a/docs/detached.md b/docs/detached.md index 0902f78a..9b9ea4ef 100644 --- a/docs/detached.md +++ b/docs/detached.md @@ -1,332 +1,57 @@ # Detached sessions -One pardes core with no terminal of its own, and any number of thin frontends -attached to it over a unix socket. The core holds every piece of state — the -text, the undo history, the layout, the pane shells — and outlives every -frontend that comes and goes. +A detached session owns the editor core, pane shells, files, undo history and +layout. TTY and SDL frontends can join and leave without ending the session. -The model is the ESP32-P4 serial console, which is why it is worth naming. On -the board, pardes runs as firmware and the host side is a dumb wire: keystrokes -in, bytes out, and the console performs no effects at all. A detached session is -that arrangement with a typed wire instead of raw ANSI: input in, cells out, and -a frontend that does almost nothing. +```sh +pardes --detach=work & +pardes --attach=work +pardes-gui --attach=work +``` -## Using it +Bare `--detach` names the session after its process ID. Bare `--attach` +requires exactly one listening session. The detached process runs in the +foreground unless the shell backgrounds it. - pardes --detach # a core named by this process's pid - pardes --detach=work # ...named `work` - pardes --attach # become a frontend of the one session there is - pardes --attach=work # ...of `work` - pardes-gui --attach=work # the SDL window is a frontend too +Inside an editor, `Attach work` (`SPC s a`) switches the current window to +that session. `Detach` (`SPC s D`) closes only that frontend. It does not +turn a local editor into a detached session. -And from inside a running editor, as ordinary acme words — type one in a tag and -execute it, or press its leader chord: +## Files and transport - Attach # hand this window to the session there is - Attach work # ...to `work` (chord: SPC s a) - Detach # leave the session, and leave it running - # (chord: SPC s D) +The frontend socket is `pardes-detached-.sock` under +`$XDG_RUNTIME_DIR`, or `~/.local/state/pardes` when no runtime directory is +set. A second session cannot replace a live listener with the same name. +Shared Unix socket handling lives in `src/9p_io.zig`. -`Attach` switches **in place**: the window, the terminal and the process stay, -and what changes is where the state lives. The ordering is the feature — see -[The in-place switch](#the-in-place-switch). +The session also opens its default 9P socket. Optional TCP and QUIC listeners, +runtime mounts and the control filesystem belong to the session, not its +frontends. See [fs.md](fs.md). -`Detach` is its counterpart and the smaller of the two: only the frontend that -ran the word leaves. The session, its pane shells and every other attached -frontend are untouched, so leaving is a success — the terminal prints where to -come back to and exits 0. It routes ORIGIN-ELSE-PRIMARY, the same rule -`read_clipboard` takes, because it answers something one particular human just -did. +## Ownership -In a session with nothing to detach from, `Detach` says so on the pane's message -row and does nothing. That falls out of the design rather than being special- -cased: a local shell leaves `push_detach` null on its vtable, and `perform` -reports `NotAttached` for a null method. `Detach` does NOT turn a local session -into a daemon — that is the true inverse of the in-place switch, it needs real -daemonisation, and it is deliberately not this feature. +One poll loop owns all core mutation, frontend connections, PTY I/O and file +watch notifications. LSP and selection-pipe workers own request snapshots and +post completions through a bounded mailbox. Each subprocess is reaped by its +owner; PTY reaping cannot consume a language server or filter's exit status. -Bare `--attach` and bare `Attach` mean "the session that is there", because -bare `--detach` names itself by its own pid and nobody can be expected to read -a pid out of `$XDG_RUNTIME_DIR`. With exactly one session listening that is the -one meant; with none or several, `detached_client.resolve` says which case it is -rather than picking one. +Restore constructs a replacement core before changing the current one. It then +joins old work, clears obsolete completions, replaces panes and watches, and +sends a fresh frame to the existing frontends. -## Where the socket lives +Frontends provide input and presentation. Clipboard writes are broadcast; +clipboard reads, browser opens and Detach go to the originating frontend, +falling back to the primary attachment. Frontends never spawn pane shells, +write session files or install file watches. -`nested.socketDir`: `$XDG_RUNTIME_DIR`, else `~/.local/state/pardes`, created -`0700` by `nested.ensureSocketDir` — never `/tmp`, because this socket carries -keystrokes into a live editor, and a world-writable directory means both that -somebody else can plant a listener at a path you will derive and that a file -they planted cannot be unlinked. `server.socketPath` spells the name -`pardes-detached-.sock` and refuses a name that is empty or holds a `/` -or a NUL, because either would move the address somewhere else. The prefix -differs from `nested.socketPath`'s `pardes-.sock` so that nested.zig's -sweeper, which recognises only an all-digit pid, can never unlink a live -session called `work`. +## Wire and tests -`bind(2)` decides who owns a name, because on a unix socket it is an atomic -exclusive create. `listen` does not unlink first: a name whose socket ANSWERS -is a live session and the bind is allowed to fail, and the only file this -process removes is one `alive` proved dead — which it says only of a connect -that was REFUSED. An unconditional unlink-before-bind is how a second -`--detach=work` used to take the socket away from every frontend attached to -the first. The window `alive` cannot see is stated in its own comment: a -session between its `bind` and its `listen(2)` also answers ECONNREFUSED, it -is two syscalls wide, and the loser of that race loses a NAME rather than a -session. +`src/detached/wire.zig` owns the versioned frontend protocol. Frames are full +grids or changes relative to each frontend's last queued frame. A new +attachment receives a full grid. Output queues and per-poll work are bounded; +a lagging frontend cannot hold the session's event loop. -Both ends vet, through one predicate — `server.zig` `vetted`, which asks `ours` -three questions of the DIRECTORY and then the same three of the SOCKET: the -right file type, our uid, and nothing granted to group or other. A frontend -that checks only one of the two has checked neither. `Client.open` asks -`access(F_OK)` before it vets, so a mistyped session name reports `NoSession` -rather than `NotPrivate` — the latter promises that the socket IS there and is -reachable by somebody else, which is a different sentence to say to a human. - -## What the daemon owns - -**Everything with an operating system under it.** The eight effects that were -once routed to one frontend to perform — `push_spawn`, `push_pty_write`, -`push_pty_resize`, `push_write_file`, `push_write_dump`, `push_watch_file`, -`push_watch_theme`, `push_dump_themes` — are performed by the daemon itself, -through `src/host_io.zig` and `src/file_watch.zig`. - -This is the whole design and it is worth stating why. A unix socket means the -core and its frontends are on the same machine, so there is no question of whose -disk or whose process table is meant. Given that, the pane shells belong to the -long-lived process: a shell forked by a frontend dies with that frontend, and -then the session has a pane with no shell in it — which contradicts the one -promise a detached session makes. It also meant `tty.zig` had to reimplement the -shell's own Host, so `spawn`, `writeFile`, `watchFile` and `dumpThemes` each -existed twice in that file, and a second frontend would have been a third copy. - -A consequence worth knowing: the daemon forks its pane shells whether or not -anybody is attached. Start a session, attach nothing, and `ps --ppid ` -already shows a shell. - -The daemon is **single-threaded**. Its pane pty masters and its one watch -descriptor live in the same `poll(2)` that accepts frontends — -`poll_slots = 1 + max_clients + MAX_PANES + 1` = 50 descriptors — so there is no -thread per pane and no thread per client. Pty output enters the core as -`core.update(.{ .output = ... })` out of a stack buffer, so a chunk is never -duplicated. Dead shells are reaped with `waitpid(-1, WNOHANG)`. - -Two capabilities exist only because the ptys are here: `pull_tty_taken` can -answer whether a pane's shell has a full-screen program in it (so an `Exec` is -typed into vim instead of at the shell), and `push_poll_frame` reports each -pane's live cwd to its tag. A frontend could do neither — it had the pid but no -core to report to. - -## What a frontend does - -Input and screen, and exactly three effects: - -| message | routing | what the frontend does | -|---|---|---| -| `set_clipboard` | broadcast | put it on **this** display's clipboard | -| `read_clipboard` | origin, else primary | read this display's clipboard, send it back as an ordinary paste | -| `open_link` | origin, else primary | open it in **this** display's browser | - -Those three survive on the wire because each needs the human's own display and -cannot be done by a process nobody is looking at. Everything else the frontend -receives is a `frame`. It never forks a shell, writes a file, or watches a path. - -`read_clipboard` and `open_link` go to the frontend whose event was applied most -recently, because both answer something a human just did: the paste must come -from the keyboard that asked for it, and a link must open in front of the person -who clicked it. The fallback to primary — the lowest attached slot, i.e. the -oldest surviving attachment — covers an effect no input caused. - -## The wire - -`src/detached/wire.zig`, protocol `version` 1, checked on connect and refused -loudly, because `zig build` replaces the binary under a running session and a -frontend decoding another version's frame layout would paint garbage and blame -the terminal. Every message is `tag:u8, len:u32le, payload[len]` — `header_len` -is 5 — and `max_payload` is 16 MiB, a bound derived from the two messages that -set it: a full frame of the largest grid the protocol admits (`max_cols` 512 by -`max_rows` 128, worst case one run per cell, about 1.6 MiB) and one paste, -which the tty frontend already caps at 4 MiB. - -**Core to frontend: eight messages** (`ServerTag`), and the split between the -two ranges is the design rather than housekeeping: - - # 0x01..0x0f — SESSION CONTROL. Not effects; the session talking about - # itself and about this connection's membership of it. - welcome = 0x01 refuse = 0x02 frame = 0x03 quit = 0x04 detach = 0x05 - - # 0x10.. — one `push_` method each, in Host.VTable's own order. Only the - # three that need THIS human's display are here; see "What the daemon owns". - set_clipboard = 0x10 read_clipboard = 0x11 open_link = 0x12 - -**Frontend to core: eleven** (`ClientTag`), and the whole set is a handshake, a -goodbye, and what a keyboard, a mouse, a trackpad or a window manager produces: - - hello = 0x01 bye = 0x02 - - key = 0x10 mouse = 0x11 resize = 0x12 paste = 0x18 - command = 0x19 pdf_scroll = 0x1a pinch = 0x1b - touch_scroll = 0x1c pointer_leave = 0x1d - -Six numbers are missing from that input run — `0x13..0x17` and `0x1e` — and the -gaps are left rather than tidied away, because renumbering is a change every -deployed frontend feels. They were `output`, `eof`, `lsp_resp`, `pipe_resp`, -`file_changed` and `tick`: the machine-local host's own reports, which stopped -being a frontend's business when the daemon took the disk and the process -table. Deleting them was not housekeeping either. `server.zig` `apply` routes -any decoded non-resize event straight into `core.update`, so while those tags -decoded an attached peer could forge a pane's output, forge an `eof` for a -shell that was still running — and unlike the daemon's own `paneEof` that path -never called `closePty`, so the master stayed open and the shell was orphaned -for the life of the session — or replace a pane's text with bytes the next -`Save` would write to disk. - -The format is deliberately **architecture-neutral**: explicit little-endian -widths, no `usize` anywhere, no struct blits, a length prefix on every slice, -tags chosen in this file rather than taken from `@intFromEnum` of a core type, -floats as their binary32 bit pattern inside an explicit `u32`, and a bool that -is 0 or 1 and a decode error otherwise. A pointer-sized field would be 4 bytes -on a 32-bit frontend and 8 here, so none is sent. It is **build-neutral** for -the same reason: `Event.resize.cell_pixels` exists only in a build with native -PDF placement compiled in, so it is always on the wire and dropped on arrival -by a build with nowhere to put it. Today both ends are x86-64 Linux; the -neutrality is what makes a riscv32 end possible later without a format change. - -Frames are diffs against what a client actually has. A client with bytes still -owed to the kernel is **skipped** for this frame and its mirror is left alone, -so a slow frontend sees fewer, larger frames rather than a growing queue. - -## The in-place switch - -`Attach` is a core-side word that emits `Effect.attach{pane, name}`; the frontend -drains it with `Pardes.takeAttach()` beside the existing `takeRestore()`. No host -method performs it, because attaching replaces the core the call is running -inside — so a shell that never polls simply cannot attach, and `host.zig` needed -no change. - -**Connect first, swap second.** The frontend opens the client and, only on a -handshake that actually succeeded, tears the local session down — reaping pane -shells, closing watches, unmounting the control filesystem, deinitialising the -core — and enters the thin attached loop. On any failure it changes *nothing*: -the message row on the pane that ran the word says why, and the editor carries -on locally with every pane and its undo history intact. A failed `Attach` is a -no-op, never a half-dead editor. - -## Fairness, and why no client can stall another - -* Every descriptor is non-blocking; one `poll(2)` per pump covers all of them. -* Frames are not queued (above), so coalescing costs no byte surgery. -* A client's out-queue holds control messages and is capped at 1 MiB - (`out_backlog`). The cap is checked *before* an append, so an oversized - message still goes out whole and what gets refused is a client that has - stopped draining: it is closed, its peers untouched, and it may reattach and - be sent a full frame. -* The TABLE is accounted too, not just each slot: `session_backlog` bounds every - in-queue and out-queue together, and a drained client hands back anything - above one `read_chunk` (`idle_retain`, 16 KiB). Its value is *derived* — - `2 * wire.max_payload`, 32 MiB — and the derivation is the fix. It used to be - a literal `4 << 20`, which was by coincidence exactly tty.zig's - `max_paste_bytes`; since a client's `in` grows to hold one WHOLE message, a - frontend assembling the very paste `max_payload` is sized for crossed the - table's ceiling *while still receiving it*, and the session closed its only - frontend mid-paste with the diagnostic for a peer that had stopped reading. -* A PANE is not a client, so it cannot be closed to reclaim anything. Its - shell's input is queued behind a POLLOUT on the descriptor already in the set, - bounded by `pty_backlog` (1 MiB), and past that the write is REFUSED and said - out loud on the pane's own message row — dropping input silently loses half a - command line, and killing a shell to reclaim a megabyte destroys work. Before - this, one `write(2)` to a master could park the whole daemon: `sleep 3600` - plus a paste larger than the pty's 4 KiB input buffer meant no frame to any - frontend, fifteen other masters unread, no `accept`, no watch drain. -* `max_clients` (32) is a **refusal**, not a queue. The listener is always - accepted from even when the table is full, so the refusal can be spoken — a - level-triggered `poll` on a backlog nobody accepts returns ready forever and - spins a core. A connection that never says `hello` also loses its slot, after - `greet_deadline_default_ms` (5 s), the one number both ends of this transport - time the handshake against; a slot held by silence is the same denial as a - queue arrived at from the other end. -* A peer speaking another protocol version gets `refuse(version)` and the - session survives. So does a peer that sends a byte the decoder does not know. - -## Limits - -Stated rather than papered over: - -* **The screen is shared, at the smallest common grid.** Two frontends of - different sizes converge on the smaller; the larger window letterboxes. Same - semantics as tmux. -* **A frontend asks for at most `max_cols` x `max_rows`** (512x128, above). - A window bigger than that — a 4K display at a small font is already past 128 - rows — attaches at 512x128 and letterboxes the rest, exactly as it does - beside a smaller frontend. `client.zig` clamps the hello and every resize, - because the geometry itself does not fit the wire: an unclamped one was - refused by the session's decoder as `BadValue`, and that refusal reaches the - frontend as a bare hangup with no reason attached. -* **No LSP and no selection pipe** in a detached session. The daemon implements - **seventeen** of `Host.VTable`'s **twenty-one** methods — fewer than the tty - and SDL shells, which install nineteen each, everything but - `pull_gpio_toggle` and `push_detach` — and it is the only host that - implements `push_detach` at all. The four it leaves null divide cleanly. - Two are real losses: `pull_lsp` and `pull_pipe` want a worker pool this - deliberately single-threaded loop has not got. They fall back - to the core's in-process defaults rather than failing, so the features are - quiet rather than broken. The other two are not losses at all: there is no - moment "after the frame is on screen" for a process with no screen - (`push_post_present`), and no pads to toggle on a PC (`pull_gpio_toggle`). -* **`--fs` is inert rather than refused.** `main.zig` hands `opts.fs` to - `server.run`, which imports no `fs_service` and mounts nothing, and `spawn` - passes a null mount directory to `host_io.forkShell`. So - `pardes --detach --fs` gives a session with no control filesystem, no - `PARDES_FS` in its pane shells, and no diagnostic saying so. -* **Frontends are the terminal and the SDL window only.** `main.zig` dispatches - `--attach` to `tty.run` and `gui.run`, and refuses any other platform with - `pardes: --attach needs the tty or gui shell`. The other three shells — - `-Dplatform=web`, `-Dplatform=macos`, `-Dplatform=esp32p4` — are entered by - their own hosts and never link that file at all. -* **Linux.** The code carries darwin branches (`sun_path` is 104 there rather - than 108, and SIGPIPE is per-socket rather than per-write), but only Linux is - built and tested. -* A pane's shell is the daemon's child, so `Kill` in a frontend ends a shell for - everybody attached. That is what one shared session means. -* **An attached window does not get the session's font.** `Font ` is a - core setting raised through `takeFontRequest`, and an attached SDL frontend - has no core to raise it — so a window that would load `MartianMono-NrRg` - locally keeps its embedded Adwaita Mono when attached, and its cell metrics - differ from the same window run locally. The choice belongs to the session but - the fonts belong to the display, so closing this means putting the request on - the wire; nothing does today. -* **An attached pane tagline is drawn in the body face**, not the condensed - tagline face. The compacted band is positioned from the pane rectangle that - produced it, and the wire carries cells rather than rectangles: in a - multi-column layout two panes' tag runs touch, so the origin cannot be - recovered from the frame alone without bleeding one column's band into its - neighbour. Painting and hit-testing therefore agree on the body grid, which is - what keeps a click landing on the glyph it was aimed at; the visible cost is - one row per pane of looser tracking. - -## Tests - -`zig build unit-test` runs twelve tests in `src/detached/client.zig`. Eleven -drive a real core over a real socket: a frontend is greeted -and sent a screen; input comes back as a diff; two frontends share one screen -at the smallest common grid; a frontend that dies takes nothing with it; a -wrong-version peer is refused, loudly; a peer that sends an undefined tag byte -loses its slot and not the session; the session outlives every frontend, keeps -the grid where the last one left it, and lets the next one take it over; the -three surviving effects route as documented above — `set_clipboard` to both -frontends, `read_clipboard` and `open_link` to the origin, and to the primary -once the origin is gone; the client table refuses rather than queues; and a -frontend that stops reading is dropped; and a window bigger than the protocol's -grid attaches at `max_cols` x `max_rows` rather than being refused, which is -also the one test that drives a full frame of the largest grid the wire carries. - -The twelfth builds no harness, opens no socket and touches no core, and that is -the point: it pins the DESIGN rather than the behaviour, and `wire.zig` has its -mirror, one test per direction. "A frontend is never asked to fork, write, or -watch" walks -`wire.ServerMsg`'s fields BY NAME — so re-adding `spawn` fails with the name in -the failure — and then every one of the 256 tag bytes, so a session built -before this change cannot talk a frontend into forking either. "A session is -never told to do a frontend's remembering" is the same argument pointed at the -other end, and it is what keeps the six deleted `ClientTag` numbers -undecodable. +`zig build unit-test` covers encoding, session ownership, real frontend +connections, worker completion and Restore. `zig build fs-test` drives +detached sessions through an independent 9P client. The snapshot suites also +exercise attach, detach and shared screen behavior. diff --git a/docs/fs.md b/docs/fs.md new file mode 100644 index 00000000..8805ffde --- /dev/null +++ b/docs/fs.md @@ -0,0 +1,89 @@ +# Filesystem + +Every native session serves 9P2000 on a Unix socket. Pane shells receive +`PARDES_9P` (socket path) and `PARDES_PANE` (pane serial). The socket is +`$XDG_RUNTIME_DIR/pardes-9p-.sock`, or lives under +`~/.local/state/pardes` when XDG_RUNTIME_DIR is unset. Detached sessions use +their session name; `--9p=` overrides it. + +A `pardes ` launched from a pane forwards Look to that pane over 9P. +`--nested` opens a separate editor and disables forwarding from its pane shells; +its 9P service stays available. + +Look resolves the OS filesystem first, then the editor's virtual filesystem. +Explicit paths bypass that search: + +| Editor path | Meaning | 9P server path | +|---|---|---| +| `/n/os/proc/self` | OS filesystem | `/os/proc/self` | +| `/n/self/pane/2/body` | pane 2's text | `/self/pane/2/body` | +| `/virtual/src/pardes.zig` | source embedded in this build | `/self/src/pardes.zig` | +| `/n/peer/self/pane/2/body` | another session's text | peer's `/self/pane/2/body` | + +`--mount=peer=work` mounts the named session `work`; the dial can also be an +absolute socket path, `unix!/path`, `tcp!IP!port`, or `quic!IP!port`. +At runtime, use `Mount peer dial` and +`Unmount peer`. There are eight named mounts; `os` and `self` are reserved. +Unmount refuses mounts still used by a pane, its working directory, or a +pending Save. Mounts are saved in dumps. Save uses the file's original mount. + +`pardes --9p-tcp='tcp!127.0.0.1!5640'` adds a TCP listener alongside the Unix +socket. Build with `-Dquic=true` and system OpenSSL 3.6+ to enable QUIC; +`--9p-quic='quic!127.0.0.1!5641'` adds its listener. Both accept numeric +IPv4/IPv6 addresses, not DNS names. Listener port zero chooses a free port; +`/self/listeners` reports all active dial addresses. + +All connections have session access, including `os`. TCP is unencrypted. +QUIC uses an ephemeral TLS identity without peer verification or login. +It carries 9P2000 on one bidirectional stream with ALPN `pardes-9p`. +The four application connection slots are shared across transports; +OpenSSL's internal buffers are separate, dynamically allocated memory. + +[Plan9port's client](https://9fans.github.io/plan9port/man/man1/9p.html) can +drive Unix or TCP without a kernel mount: + +```sh +9p -n -a "unix!$PARDES_9P" read self/index +9p -n -a 'tcp!127.0.0.1!5640' read self/index +``` + +For [Linux v9fs](https://www.kernel.org/doc/html/latest/filesystems/9p.html), +use `version=9p2000,cache=none,access=any` and `trans=unix`, or `trans=tcp` +with `port=5640`. Set `uname`, `dfltuid`, and `dfltgid` for the local user. +Leave `aname` empty: the mount root contains `os` and `self`. Kernel mounts +are not part of the test suite. Neither 9P2000.u nor 9P2000.L is implemented. +Existing Plan9port/v9fs clients need a userspace bridge for QUIC. + +The server root contains `os` and `self`. Under `self`, `index` lists panes, +`new/ctl` creates a pane and returns its serial, and `pane/` contains +`body`, `tag`, `ctl`, `addr`, `data`, `event`, and selection files. Terminal +panes additionally have `pty/{ctl,status,data}`. + +`EffectCode ` lists the current backend's embedded implementation +files under `/virtual`; Look opens their full source. + +`self/screen` returns JSON with `cols`, `rows`, `cursor`, a `styles` table, +and row-major `cells` of `[grapheme, style_index]`. Each open freezes one +frame until close, including across multiple reads. The independent client +can inspect it directly with `Client(socket_path).screen()`. + +A terminal `body` freezes its history on the first read of each open handle; +later reads use the same bytes while output continues. Close and reopen for +newer history, or read `pty/data` for the live stream. Screen and terminal-body +snapshots share 32 handle slots, released on close or disconnect. + +Writing `body` appends; opening it with truncation replaces its contents. +`addr` selects a range and `data` replaces that range. `ctl` accepts `name`, +`look `, `get`, `put`, `del`, and `delete`. Look resolves from that pane +without editing its body or tag. Holding `event` open redirects the pane's Look +and Exec clicks to that client; writing a record back performs the action. + +This is a control filesystem, not a complete POSIX export. Native filenames +may contain up to 255 bytes. Existing regular OS files support read, write, +and truncation to zero; protocol create, remove, rename, and other metadata +changes are refused. Ownership, permissions, and timestamps are synthetic. + +`zig build fs-test` drives real sessions using the independent Python client +in `test/ninep.py`. `zig build 9p-test` checks the freestanding wire protocol; +`zig build fs-bench` measures filesystem transactions in the core. +`zig build fs-test quic-test -Dquic=true` also exercises QUIC mounts and I/O. diff --git a/docs/helix-keys.md b/docs/helix-keys.md index 048d8397..dcd5b7cb 100644 --- a/docs/helix-keys.md +++ b/docs/helix-keys.md @@ -13,21 +13,20 @@ single key — it is the anchor-only divergence every insert-mode edit shares and is described in the Files list at the bottom instead. Code map, by SYMBOL — line numbers rot, names do not. Body-normal key -RECOGNITION is `src/normal_input.zig`: a pure state machine over `Role` (one +RECOGNITION is `modal.Normal` in `src/modal.zig`: a state machine over `Role` (one per bound command, matched against `src/config.zig`'s chord lists by `Pardes.normalInput`), a `Prefix`, a `MatchSub` and a count, emitting a semantic `Action` that both the text and PDF adapters consume. It knows nothing about panes or text. `src/pardes.zig` then EXECUTES: `Key`, `handleKey` (intercept order is load-bearing, see the comment there), -`handleNormal` (marshals `Pane`'s compact fields in and out of -`normal_input.State` through `paneNormalState` / `putPaneNormalState`), +`handleNormal` (parses directly into `Pane.normal`), `executeNormalAction`, `setPaneRange` (helix range → pane state), `enterInsert`, `handleInsert` / `insertKey`, `insertTab`, `normalDelete`, -`normalYank`, `normalPaste`, `normalChange`, `replaySels`, `multiOnce`. Undo -and redo are per pane kind, in `file_pane` and `term_pane`. Pure text math: +`normalYank`, `pasteText`, `normalChange`, `replaySels`, `multiOnce`. Undo +and redo live in `panes.File` and `panes.Terminal`. Pure text math: `src/modal.zig` (the `hx*` family is the helix-semantics layer: gap offsets + ranges over the flat text). Differential harness: `test/hxdiff.zig` + -`test/hxcases/*.jsonl` + `test/hxcases/regen.sh` — see "Differential testing" +`test/hxcases/*.jsonl` — see "Differential testing" at the bottom. ## Global semantic divergences (read first) @@ -147,13 +146,10 @@ Status: `todo` → set to `done (phase N)` as rows land. All rows verified against keymap.md. Edit ops on terminal panes obey the immutable-output rule (typed runs only) — same as `d`/`c` today. -Phase 2's recognition state is now `normal_input.State`: a count (accumulator, +`Pane.normal` holds `modal.Normal.State`: a count (accumulator, capped 0xffff), a `Prefix` (`g` `z` `m` `f` `F` `t` `T` `r` `]` `[`), a `MatchSub` (m-mode's `i`/`a`/`s`/`r`/`d`) and one `held_char` for `mr`'s -``. `Pane` keeps the same four fields it always did — `count`, -`pending`, `pending2`, `pending_ch` — but purely as compact per-pane storage, -marshalled in and out by `paneNormalState` / `putPaneNormalState`; the -comments on them say so. `find_op`/`find_ch` (the `Alt-.` repeat target) stay +``. `find_op`/`find_ch` (the `Alt-.` repeat target) stay on `Pane`, because the parser emits `repeat_find` without remembering what was found. Pure text math in `modal.zig`: `findChar`, `matchBracket`, `paragraphFwd`/`paragraphBwd`/`paragraphRange`, textobject/surround ranges, @@ -379,7 +375,7 @@ Files (all in `test/hxcases/`): immutable) or doc-marked terminal no-ops. tty case texts keep motions inside the content (the tty motion surface trims trailing blank rows, so ge/G-to-last-line style assertions stay off tty). -- `goldens.jsonl` — checked-in helix results, regenerated by `regen.sh` +- `goldens.jsonl` — checked-in helix results, regenerated by `zig build hxdiff-update` (runs the `hx-harness` binary from the helix checkout; override with `$HX_HARNESS`). Only needed when cases change — the diff itself runs offline. @@ -417,7 +413,11 @@ Run it: # "editing a shell pane behaves like editing a # file" cannot drift. The case's own "pane" field # is ignored; exemptions in parity-waivers.jsonl - sh test/hxcases/regen.sh # regenerate goldens + zig build hxdiff-live # compare a fresh reference without changing goldens + zig build hxdiff-update # update goldens only after the live comparison passes + +Set `-Dhelix-harness=/path/to/hx-harness` or `HX_HARNESS`; otherwise these +steps look for `hx-harness` on PATH. The helix half (`hx-harness`) lives on the `pardes-harness` branch: a full headless `Application` with LSP/tree-sitter/auto-pairs/word- diff --git a/docs/lsp.md b/docs/lsp.md index 4b2e5ba2..82586344 100644 --- a/docs/lsp.md +++ b/docs/lsp.md @@ -1,551 +1,72 @@ -# Language intelligence in pardes - -Three things landed together, and only the first two are permanent: - -1. **An async execution model.** The core stays a state machine; slow work goes - to a worker and comes back as an event. -2. **A helix-exact keymap** for every LSP command. -3. **A seam** (`src/lsp/lsp.zig`) with exactly one function behind it, so competing - backends can be swapped, measured, and thrown away. - -## The async model - -There was none before this: every effect the core emitted was fire-and-forget -(`spawn`, `write`, `save_file`) or instantaneous. A language query is the first -thing pardes asks for that *answers later*, so it needed a request/response -shape — and got the smallest one that works. - -``` -core shell worker - | Effect .lsp{id,kind, | | - | pane,offset,arg} | | - |-------------------------->| | - | | snapshot path + content | - | |--------------------------->| - | | | lsp.query(...) - | | Event .lsp_resp{id,rows} | - |<--------------------------|<---------------------------| - | lspResponse -> atomic edit, jump, or results buffer | -``` - -The shell already ran this exact pattern for pty readers, so the async part is -about thirty lines per shell: `src/tty/tty.zig` uses `io.concurrent` + the -vaxis loop queue, `src/gui/gui.zig` uses a detached thread (`lspThread`) + the -mutex queue it already had. A FREESTANDING core compiles in no backend at all -(`zls_backend = !freestanding_core` in `build.zig`), which is both the web -shell and the ESP32-P4 object: there `lsp.supports` is empty, which makes -`lspRequest` return before it emits, so the effect is never even raised. -`web.zig` leaves the host's `pull_lsp` null and exports nothing for a -response; a host that links a backend would add both. - -**In a DETACHED session every query answers EMPTY.** -`src/detached/server.zig`'s vtable implements seventeen of `host.zig`'s -twenty-one methods, and `pull_lsp` is one of the four it leaves null — a -worker pool is precisely what its deliberately single-threaded loop does not -have. A null method is NOT automatically a dropped effect: `perform` decides -that per arm, and the `.lsp` arm's answer is to synthesise one on the spot — -an `lsp_resp` Event with `rows = ""`, fed straight back into `update`. `.pipe` -one arm below does the same, yielding -`pipe_resp{ .success = false, .outputs = &.{} }`, so a null `pull_pipe` is a -pipe REPORTED as failed rather than one that hangs. So `gd` jumps nowhere, -`gr` finds no references, `SPC l r` renames nothing (an empty edit list parses -as none) and Tab after a dot offers nothing — all of it indistinguishable from -a backend that found nothing, which is exactly what the seam's "no rows is a -legal answer" rule promises. - -**Tab still indents — still, not always.** The empty response reaches -`lspResponse`, whose `rows.len == 0` prong performs the indent the Tab prong -skipped, but only while the cursor has not moved. That -guard holds on the ordinary path because of `pump`'s order: `pull_wait_input`, -then the queued events, then the effects, then render. The effect Tab emitted -is performed after every keystroke that was ALREADY readable in the same -round, since `pull_wait_input` applies a whole batch and not one event (the -tty shell says so at the head of `waitInput` — "block for one event, then -apply the whole pending batch" — and the daemon's poll loop drains every -readable client `.event` straight into `core.update`). One keystroke per wake -is the normal case and the indent lands in the same frame, before render. In a -BURST where the key after Tab was readable in that same poll round, `cur_col` -has moved by the time the prong runs, the guard fails, and the Tab really is -eaten. The local shells have the same race over a wider window, so this is a -property of the late-indent repair rather than of detaching. - -None of the detached core's four null methods silently drops a reachable -effect. `pull_lsp` and -`pull_pipe` have the fallbacks above; `pull_gpio_toggle` is -`orelse return Error.NoPads` (`board_memory.zig`), which lands on the message -row; `push_post_present` is a `pump` hook fired after presenting, and there is -nothing to notify in a process with no screen. `push_fs_reply` was listed here -too until the daemon began mounting its own `/dev/fuse`; it implements that one -now, and `--fs` works in a detached session. - -Four rules make it safe: - -- **The worker never touches the core.** Path, source, arg and root are copied - into an `LspJob` before it starts — one per shell, in `src/tty/tty.zig` and - `src/gui/gui.zig`. The user keeps typing while a - query is in flight; a borrowed slice would be a use-after-free the length of - one keystroke. -- **One query in flight, identified by a monotonic id.** A second press bumps - the id, which makes the older answer stale. The pending request also records - the pane serial, so a closed-and-reused slot cannot accept its response. -- **Mutating answers are revision-checked.** Rename records the file revision - sent to the worker and applies nothing if the user edited before it answered. -- **No rows is a legal answer.** A backend that cannot answer appends nothing, - which is indistinguishable from a language server still starting up, and the - core does nothing. There is no error path to render. - -## Results are `+Search` rows - -Every backend renders into one format: - -``` -sub/file.zig:LINE:COL text under the asking window's dir -sub/file.zig:LINE:COL-ENDCOL text -/abs/path/elsewhere.zig:LINE:COL text anywhere else -``` - -1-based line and column. The second form carries the answer's -RANGE where the protocol gave one on a single line (a token, a symbol's name), -and a look on it SELECTS that span rather than parking at its first cell — so -`gd` lands on the whole name, and a `gr` reference stepped to with `n`/`N` and -opened with Enter arrives with the reference itself selected. This is the -format `look.zig` already resolves, `runSearch` already produces and `n`/`N` -already walk — so: - -- **one row from a goto** → jump straight there (`lookAt`) -- **several rows** → an output buffer, which `n`/`N` walk: a step SELECTS a row - and Enter opens it - -which means helix's multi-result picker required **no picker code at all**. The -`+Search` buffer *is* the picker. Non-location answers (`hover`, `code_action`, -`format`) open `+Hover`/`+Lsp` instead, which are prose and not places — `n`/`N` -find nothing look-able in a documentation blurb and walk straight past it to the -next pane on the ring. - -Rename is deliberately the one exception to rows as presentation. The backend -emits `@edit START END` records through `lsp.edit()`, using half-open byte -offsets into the exact `Req.source` snapshot it resolved. The core validates -that every range is ordered, non-overlapping and in bounds, checks that the -pane serial and file revision still match, then substitutes the requested name -across all ranges with one allocation and one undo transaction. A malformed, -stale, or empty response changes nothing. The current ZLS backend resolves and -renames references in the **current file only**; it does not claim a workspace -rename. - -**A path UNDER `Req.root` is written relative to it; everything else keeps its -full absolute path** (`lsp.rel`). `Req.root` is the directory of the file the -query was asked about — the window that generated the buffer — and a `gr` over -one file was otherwise the same forty-character prefix repeated down the whole -pane, with the part you came to read pushed off the right edge. The short form -resolves because `Req.root` is also the directory the results buffer is *named* -in (`output_pane.open`), and a look resolves a relative word against the -directory of the pane it was clicked in — which is that buffer. - -Under, never "shorter": a hit outside the tree is **not** walked up to with -`../`. An absolute path resolves from anywhere and says where it is; a `../..` -chain says neither, and stops being true the moment the row is read anywhere but -beside its own buffer. `test/snapshots/lsprelpath.snap` pins both directions end -to end — the enum one level up (absolute row) and one level down (`inner/tint.zig`, -a stripped path that still has a separator in it), each with the `n` step that -selects it and the Enter that opens it, plus a right click. - -This is the rule `look.grep` follows for its own rows, and since the client -landed it is spelled ONCE: look.zig's `grep` calls `lsp.rel` for its `shown` -paths rather than keeping the inline twin this paragraph used to complain -about. - -`completion` is the kind this shape changes the most. Every other editor answers -a dot with a popup of NAMES to insert; a seam that returns locations cannot -insert anything, so this one answers with the candidates' **declarations** — -one row each, in the same `+Search` buffer, steppable with `n`: - -``` -path:LINE:COL-ENDCOL name the candidate's declaration line -a.zig:2:5-13 verdigris verdigris, -``` - -The name comes first because that is the thing you would type — the row answers -"what goes here" before "where does it come from", which is the order the -question was asked in; every other kind here answers a WHERE, and for this one -the location is the evidence rather than the answer. It is not padded into a -column: the location in front of it is already ragged, so there is nothing to -align to. That is a different and arguably better answer to "what goes here": -you get the word AND you can read the definition rather than a list of words. It -is the one location kind that does NOT jump on a single row, because with one -candidate you still want to see the list rather than be teleported into it. - -A results buffer is REFILLED rather than reopened when the same kind is asked -again — the rule `runSearch` always had, and which the language path was -missing. It survived being missing while every query was a deliberate press -(`gr` twice left two identical lists and you closed one); Tab after a dot is an -ordinary typing keystroke, and measured, twenty of them stacked **fifteen** -byte-identical `+Search` panes, crushed the file to one visible line, and then -ran `freeSlot` out so the key was silently eaten for the rest of the session. -Unlike a search the ARGUMENT is not part of the identity: a language query is -asked about a different symbol every time with the same (usually empty) arg, so -the kind is the unit. - -Making it work needed one trick. A completion is asked for exactly when the -line is half-typed, and a half-typed line does not parse: `switch (e) { . }` -loses the whole switch to the parser's error recovery, taking with it every -ancestor an expected-type resolution needs. ZLS answers this with a private -token scanner welded to its `*Server`. `lsp_zls.completionSource` instead makes -the tree PARSE — it splices a placeholder in after the dot, in the six -spellings a half-typed line can need — three shapes, each with and without a -closer still hanging: a switch prong (`_p => {},` / `_p => {}, }`), an -unterminated statement (`_p;` / `_p)`), and a bare identifier (`_p` / `_p }`) -— and keeps the one that both makes -the dot reachable in the tree and leaves the fewest parse errors. Everything -after that is ZLS's ordinary public resolution over an ordinary tree. - -**A Tab the backend cannot answer still indents.** The keystroke has already -diverted by the time "no rows" comes back, so `lspResponse` performs the indent -the Tab prong skipped — on the condition that the cursor has not moved since, -so nobody who kept typing gets four spaces landing behind their hands. Without -that, a dot in a comment, in a string, or on a line nothing can be made of ate -the keystroke outright. With several cursors Tab never diverts at all: a -language query is a per-keystroke action inside a per-selection replay, so -asking would stop the replay dead and collapse the multicursor. - -### What it costs, and what it cannot do - -Per press. **Both timings date from 2026-08-09**, change `lmlltvrx`, and have -not been re-measured; the line count beside the first was refreshed once -afterwards, on 2026-08-12 in change `vwtlskzr`, and `src/pardes.zig` is 16 466 -lines today. The ReleaseFast column is `zig build lspbench`, which is pinned to -ReleaseFast in `build.zig` and always has been — so the Debug column came from -running the installed editor by hand and the repo records no harness for it. A -completion parses the buffer once per placeholder spelling it tries, so the -first row scales with the file: re-run rather than trusting either number. - -| | ReleaseFast | Debug (what `zig build` installs) | -|---|---|---| -| a switch arm in `src/pardes.zig` (14.6k lines) | 8.8 ms | 87 ms | -| `std.` — 91 candidates, each alias-resolved into the stdlib | 26 ms | 204 ms | - -It is a worker thread, so the editor does not block. On the TTY shell the -second press of Tab then joins the first query on the UI thread — -`old.cancel(s.io)` on the one in-flight future, and a backend that ignores -cancellation means waiting out a query the user already abandoned -(`src/tty/tty.zig`, the `lsp` vtable entry, whose own comment says so). The -GUI shell does not join: it spawns another thread per request and lets the -core's monotonic id make the older answer stale, so it pays memory instead of -latency. Pre-existing and shared by every LSP kind — not this feature's to -fix, but it is what a fast double-Tab feels like on a terminal. - -Known limitations, in the order you will meet them: - -- **`@This()` anywhere in a container makes the whole container unresolvable**, - so `var list: std.ArrayList(u8) = .` — the most common decl literal in this - codebase — answers nothing. This is not the completion filter: `hover` and a - plain field access on the same struct return nothing either. It is the case a - user hits first, and it is upstream of everything here. -- **Only the break AT THE CURSOR is repaired.** Zig's error recovery runs - forward, so an unrepaired break earlier in the file swallows the declaration - the cursor is in and the answer is empty. While typing you normally have one - broken spot, which is the case this works for. -- **A dependency module** (`@import("vaxis")`) cannot be typed at all, for the - same reason `gd` on `vaxis.init` finds nothing. -- **`error.`** is not handled — the position context is `.error_access`, which - no branch claims. - - -## The protocol client: every other language - -`src/lsp/lsp_client.zig` is the second backend behind the same seam: a real -LSP client — JSON-RPC 2.0, `Content-Length` frames — speaking to child -processes. Nothing in it knows any single language; `specs` is a table of -(binary, languageId, extensions, root markers), and rust-analyzer, clangd, -gopls, typescript-language-server and pyright are rows in it. The seam asks -each backend `speaks(path) and supports(kind)` in order, so `.zig` stays with -the in-process analyser (cold is warm, no process) and everything else routes -here. `SPC l i` prints both sections; `backend_name` is `zls-inproc+lsp-client`. - -**One server per spec, one reader thread per server, and the reader is not -optional.** A real server TALKS: rust-analyzer streams `$/progress` for the -whole minutes-long index of a big workspace, publishes diagnostics nobody -asked for, and asks its own `workspace/configuration` questions mid-flight. -The reader owns the read side of the socketpair, routes responses to the one -waiting query (a mailbox under the connection's mutex), answers -server-to-client requests so the server never blocks on us, feeds the -diagnostics store, and narrates state changes through the STATUS SINK — a -callback both native shells register at startup and post to their event -queue, so "rust-analyzer: cargo check 88% 955/1083" lands on the same -transient message row a save narrates into (`message.stamp`, verb `lsp`, on -the ACTIVE pane — server state is session news, not a fact about the pane -that asked). Chatty progress is throttled to one post per 150ms per server -and deduplicated; state CHANGES (starting, ready, exited, errors) always -land, and repeating the row already shown never does — which is also what -makes the settled state deterministic for the snapshot goldens. - -Nothing may wedge the editor, and nothing healthy may be killed for being -busy: - -- every write and every mailbox wait is deadline-bounded (8s handshake, 4s - request); a query the server does not answer in time returns no rows and - sends `$/cancelRequest`; -- three CONSECUTIVE timeouts mean wedged and force a restart — but only - while the server is idle. One with active `$/progress` (rust-analyzer - mid-`cargo check` over a thousand crates) is demonstrably alive, already - narrating its own excuse on the message row, and killing it would throw - the index away right before it pays off. This rule exists because the - first run against a thousand-crate workspace did exactly that; -- a failed spawn or handshake is NOT a session disable: it backs off - exponentially (10s doubling to 2min, reset by the next success), because - the failure that taught this was a rustup shim deciding to download the - project's whole pinned toolchain before launching the real server. Only a - missing binary disables a spec, once, with a message saying which env var - overrides it; -- a server that dies is reaped by whoever saw it die (the reader on EOF, - `shutdownIf` on a transport error), the fd is closed by the READER ALONE — - `shutdown(2)` first, so a polled fd number is never recycled under a - thread still watching it — and the next query respawns, generation-checked - so a stale worker can neither adopt nor kill its successor's server. - -`PARDES_LSP_RS` / `_C` / `_GO` / `_TS` / `_PY` override each spec's binary -(a path or a PATH name); the empty string disables the spec. The snapshot -harness pins `_RS` to `test/lspmock.zig`'s deterministic mock and empties -the rest, so `test/snapshots/lsp-client.snap` (gd across files, gr spans, -n/Enter) and `lsp-client-edit.snap` (format apply, rename apply, one-step -undo for each) drive the REAL client — spawn, handshake, reader, narration — -against answers a golden can quote. `zig build lspprobe -- gd :` -is the same seam from the command line, for pointing at any real workspace; -comma-separated kinds share one server so a big index is paid for once. - -The root is helix's `find_root` rule: walking up from the file, the TOP-MOST -directory holding one of the spec's markers wins (a cargo workspace's root -`Cargo.toml` beats the member crate's), the closest `.git` is the fallback, -the asking directory the last resort. A second project in the same session -becomes a workspace FOLDER when the server advertises support. Position -encoding is negotiated to utf-8 and the server's ANSWER is believed; the -utf-16 conversion is implemented in both directions for servers that refuse. -Diagnostics PULL (`textDocument/diagnostic`, LSP 3.17) is preferred when the -server advertises it — rust-analyzer does — and the push store fed by the -reader answers otherwise, `]d` stepping either for free. - -### Mutating answers really mutate now - -The seam grew a second record form beside rename's `@edit`: `@put START END -TEXT` carries a per-range replacement, percent-encoded onto the one line a -record is allowed to be (`lsp.put`). The core decodes, validates (ordered, -non-overlapping, in bounds, revision unchanged) and applies ALL records as -one undo transaction, then says so on the message row ("formatted 1 -range(s)", "renamed 2 range(s)"). So: - -- `=` FORMATS, like helix — through the client it applies the server's - TextEdits; through ZLS it applies one span covering everything `zig fmt` - would change. The two non-edit answers stay prose in `+Lsp`: a file that - does not parse, and (client-side) a server with no formatter. -- `SPC l r` through the client applies a WorkspaceEdit that stays inside the - asked-about file. One that spans OTHER files (a real workspace rename) - arrives as location rows instead and opens as a PREVIEW list in the same - buffer `gr` fills — applying a fraction of a workspace rename silently - would be worse than either. The ZLS backend still resolves and renames - current-file references via `@edit`, exactly as before. - -### Four kinds helix does not have - -The hierarchy kinds are two-step in the protocol (prepare at the cursor, -then follow the item), are gated on the server capability so an old server -costs zero round trips, and their answers are LOCATIONS — the one thing this -seam renders for free. helix has no binding for any of the four (checked -against helix-term/src/keymap/default.rs). - -| keys | kind | what the rows are | -|---|---|---| -| `SPC l c` | incoming_calls | one row per CALL SITE, under the caller's name | -| `SPC l C` | outgoing_calls | the callees' declarations | -| `SPC l t` | supertypes | the types this one extends/implements | -| `SPC l T` | subtypes | the types that extend/implement this one | - -All four behave like `gr`: a list to walk with `n`/`N`, and a lone answer is -a jump. - -## Which ZLS, and which stdlib - -Both are decided at build time, and `SPC l i` prints both. - -`build.zig.zon` pins ZLS to a COMMIT rather than a tag — -`git+https://github.com/zigtools/zls#3e0d082084be43e36865136a138c1fe2023b33ca`, -on the 0.16.x branch — because master requires Zig 0.17-dev and no tagged -release both builds on 0.16 and exports the internals this backend calls. -`build.zig` spells the same commit a second time, as the top-level -`const zls_version = "0.16.1-dev+3e0d0820"`, and hands it to the ZLS package's -own `-Dversion-string` and to `pardes_config.zls_version`. The duplication is -unavoidable rather than sloppy — the semver half (`0.16.1-dev`) exists nowhere -in the manifest — and it is load-bearing, because ZLS's build otherwise -derives that string from `git describe`, which has nothing to read in a -fetched package with no `.git`. The two are made to AGREE BY CONSTRUCTION: a -`comptime` block right below the constant takes the short hash after the `+`, -takes the pinned commit after the `#` in `zon.dependencies.zls.url`, and -`@compileError`s unless the first is a prefix of the second. A `.zon` bump -that forgets `build.zig` is therefore a build error, not a `SPC l i` naming a -build nobody linked. - -`std` is the harder half. ZLS resolves `@import("std")` through `zig_lib_dir` -and through nothing else, and this backend sets `zig_exe_path = null` on -purpose — asking the `zig` binary is the subprocess the whole design exists to -avoid. So `build.zig` bakes `b.graph.zig_lib_directory.path` into -`pardes_config.zig_lib_dir`: the exact stdlib pardes itself was compiled -against, which is what makes `gd` on `std.mem.count` land in the real -`mem.zig`. `zigLibPath()` (`src/lsp/lsp_zls.zig`) reads `ZIG_LIB_DIR` from the -environment FIRST and falls back to the baked path, so a user who moved the -toolchain can point the backend at it without rebuilding. With no lib dir at -all every `std` symbol is a silent miss — which is why `SPC l i` reports -whether the directory OPENS rather than only which one was compiled in. - -That same `zig_exe_path = null` is the dependency-module limitation above. -ZLS resolves a relative `.zig` path from the filesystem and `std` from the lib -dir, but any other import name — every dependency in `build.zig.zon` — it can -only answer by running `zig build --build-runner` to discover the module -graph. With no zig binary that branch returns nothing, so `@import("vaxis")` -is a silent miss by construction rather than by omission. - -## The keymap is helix's, exactly - -Verified against `helix-term/src/keymap/default.rs`, not from memory. - -| keys | command | notes | -|---|---|---| -| `gd` | definition | jumps on a single result, lists on several | -| `gD` | declaration | | -| `gy` | type definition | | -| `gi` | implementation | | -| `gr` | references | | -| `SPC l k` | hover | opens `+Hover` | -| `SPC l r` | rename | tag input; applies same-file edits in one undo step, PREVIEWS a multi-file WorkspaceEdit as rows | -| `SPC l a` | code action | | -| `SPC l h` | select references | | -| `SPC l s` / `SPC l S` | document / workspace symbols | `S` takes a query | -| `SPC l d` / `SPC l D` | document / workspace diagnostics | | -| `]d` / `[d` | next / prev diagnostic | steps the list, asks for one if absent | -| `]D` / `[D` | last / first diagnostic | | -| `=` | format | applies the formatter's edits in one undo step | -| `Ctrl`+left-click | definition | the mouse spelling of `gd` | -| `Tab` in INSERT mode, right after a `.` | completion | what could go here, and where each of those is defined | -| `SPC l c` / `SPC l C` | incoming / outgoing calls | beyond helix — see the client section | -| `SPC l t` / `SPC l T` | supertypes / subtypes | beyond helix — see the client section | - -Tab is the one key here that is not helix's and not a goto. helix's `Tab` -completes; pardes's shows you the CANDIDATES' DECLARATIONS in a `+Search` -buffer and inserts nothing, because that is what a seam returning locations can -honestly do — see below. It only diverts where an answer is possible: on a -terminal, in an output buffer, or in a file the backend does not speak -(`lsp.speaks`, which the core asks and the backend answers), Tab indents -exactly as it always did. A Tab that silently does nothing would be worse than -not having the feature. Nothing about the mode changes either — the pane is -still in insert, so typing goes on and walking the answer with `n` means -pressing `Esc` first, the same as for every other results buffer. - -Ctrl-click rides the ordinary left-click drag rather than firing on the press: -a click does not place the modal cursor until RELEASE, so a query asked at -press time would answer about wherever the cursor previously sat. A ctrl-DRAG -still selects, and still asks about where it started. - -**The gotos are helix's exactly; the leader commands are helix's letters under -an `l` prefix.** `g` and `[`/`]` had no conflicts — `gd/gD/gy/gi/gr` and -`]d/[d` were all free, so they stay where helix puts them. The bare `` -letters were NOT free, and an earlier pass took them anyway, displacing `SPC d` -(Del), `SPC k` (Kill), `SPC s?` (Dump/Restore) and `SPC h t` (Tutor). That is -the wrong trade: those are pardes's most-pressed keys and predate the language -work, whereas an LSP command is something you reach for deliberately and can -afford one keystroke more. So every one of them keeps helix's own letter and -gains the prefix — `k` becomes `SPC l k`, `d` becomes `SPC l d` -— and nothing pardes had moved at all. - -Two more live in the same group because they belong to it, not to helix (the -four hierarchy kinds above are also pardes's own — helix has no spelling for -them): - -| keys | builtin | what it shows | -|---|---|---| -| `SPC l i` | `Lspinfo` | BOTH backends: which ZLS and which stdlib (and whether it opens); every protocol server's state, root, encoding and capabilities; and each side's last 24 queries with timings, row counts and **the errors `query` swallowed** | -| `SPC l w` | `Lspwhy` | why the definition query at the cursor answers what it does — narrated by whichever backend the file routes to | - -These exist because of the seam's own contract: a backend never fails loudly, -which is right for an editor — a thrown analyser must not take the process with -it — but it makes a broken backend and a correct one that found nothing look -identical. `Lspwhy` narrates the REAL resolution path (the position context is -the analyser's own answer, threaded out through a trace) rather than -re-deriving it beside the code, because a debug view that reimplements the -logic is one that can disagree with it. `Lspinfo` answers from ANY pane, -including one with no file, since it is about the backend rather than a -document — which matters precisely when the pane you are in is the problem. - -`K` is **not** hover — in helix it is `keep_selections`. It was checked; do not -"fix" it. - -## Writing a backend - -`src/lsp/lsp.zig` is the seam, and since the protocol client landed it holds a -LIST of backends, asked in order: the first one that `speaks` the file's -language and claims the kind in `supports` answers. An implementation supplies -three things and touches nothing else (`backend_name` is the SEAM's, not -yours — one literal in `lsp.zig` naming the compiled-in combination; a backend -with unsolicited news to deliver may additionally accept the status sink, as -`setStatusSink` shows): - -```zig -pub fn query(gpa, arena, req: Req, out: *std.Io.Writer) void -pub fn speaks(path: []const u8) bool -pub const supports: std.EnumSet(Kind) -pub const backend_name = "..." -``` - -`supports` says what a backend can do; `speaks` says what it can do it TO, and -exists for the one key that must not be eaten when the answer is no — see the -Tab note above. - -`query` runs on a worker thread with no access to the core — everything it may -read is in `req` (`path`, `source` (NUL-terminated), `offset`, `arg`, `root`). -`out` is a plain `std.Io.Writer`: the shell owns the buffer behind it (an -`Io.Writer.Allocating`), so a backend never allocates the result, never frees -it, and cannot get the allocator wrong. `arena` is freed wholesale on return; -`gpa` is for a backend's own scratch. Use `lsp.row()` to emit a location, -`lsp.spanRow()` for one that carries the RANGE it matched (the -`path:LINE:COL-ENDCOL` form above), `lsp.rel()` to spell a path against -`req.root`, `lsp.lineCol()` to convert an offset, and `lsp.edit()` for each -half-open range of a rename response. Location -rows are byte-identical across backends; rename ranges are consumed by the core -and never rendered. `rel` allocates nothing — it returns a slice of what you -hand it. - -## How the implementations are judged - -`zig build lspbench` — same harness, same corpus, same 22 probes, every backend. -The corpus is pardes's own `src/`, plus `test/lspfixture/`: five of the probes -point at fixtures rather than at real source. Three of them do because on -clean, already formatted code the correct answer to `diagnostics`, -`workspace_diagnostics` and `format` is nothing, and that is indistinguishable -from a backend that has neither — `broken.zig` carries an unused local and a -misformatted fn, so all three have real work. The other two are the half-typed -dot, whose two shapes are `dotcomplete.zig` (a switch prong that parses -everywhere but at the dot) and `dothalf.zig` (a line also missing its -terminator). The 22 probes cover 15 of the 17 `lsp.Kind`s -— `definition` three times, `document_symbols` twice, `completion` five times, -and the two introspection kinds (`status`, `explain`) not at all. - -- **Feature completeness.** Which kinds return rows, and whether the rows - contain what they should. The harness trusts *results*, not the `supports` - flag: a kind claimed but returning nothing is reported as `CLAIMED-EMPTY`, - and a kind that answers without claiming is `unclaimed-works`. Correctness - is a substring the rows must contain, so returning a confident wrong location - scores worse than returning nothing. - -- **Latency.** `cold` (first query, index construction included) and `warm` - (median of 20). They differ by orders of magnitude for an indexing backend - and both matter: cold is what the first keypress costs, warm is what every - one after it costs. -- **Memory.** Peak RSS delta (`VmHWM`), so a backend that frees its index - before returning still pays for having built it. -- **Lines of code.** Not measured by the harness — it is `jj diff --stat` - against the base commit. Less is better, and vendoring a library is not free - but is charged in build time and dependency surface rather than in lines we - maintain. - -The user-visible rename contract is also pinned through the actual TTY, -leader prompt, worker and ZLS backend by `test/snapshots/lsp-rename.snap`: both -resolved occurrences change, a shadowed local does not, and undo/redo treats -the response as one transaction. - -Run `zig build lspbench -- --json` for machine-readable output. +# Language intelligence + +Native hosts, including detached sessions, run language queries on workers. +Zig uses the linked ZLS analyser; other languages use the protocol client in +`src/lsp/lsp_client.zig`. Web and board builds have no language backend. + +`Lspinfo` (`SPC l i`) reports backend versions, server state, capabilities, +recent timings and errors. `Lspwhy` (`SPC l w`) traces resolution at the cursor. + +## Commands + +| Keys | Action | +|---|---| +| `gd`, `gD`, `gy`, `gi`, `gr` | Definition, declaration, type, implementation, references | +| `SPC l k` | Hover | +| `SPC l r` | Rename | +| `SPC l a` | Code action | +| `SPC l h` | Select references | +| `SPC l s`, `SPC l S` | Document and workspace symbols | +| `SPC l d`, `SPC l D` | Document and workspace diagnostics | +| `]d`, `[d`, `]D`, `[D` | Next, previous, last and first diagnostic | +| `=` | Format | +| Ctrl-click | Definition | +| Insert-mode Tab after `.` | Candidate declarations | +| `SPC l c`, `SPC l C` | Incoming and outgoing calls | +| `SPC l t`, `SPC l T` | Supertypes and subtypes | + +A single goto result jumps directly; several results open a reusable search +buffer. `n`/`N` select result rows and Enter opens them. Hover opens prose. +Locations use one-based `path:line:column` or `path:line:column-endcolumn`. +Paths below the requesting pane's directory are relative to that directory; +other paths stay absolute. + +Completion lists declarations and inserts nothing. An unanswered Tab indents +only if the cursor has not moved since the request. Multicursor Tab indents +without starting a query. + +Formatting and same-file rename apply as one undo transaction. A workspace +rename spanning other files opens a preview; it does not partially apply the +rename. The linked Zig analyser resolves current-file references and relative +imports, but does not run the build runner to discover dependency modules. + +## Configuration and testing + +`PARDES_LSP_RS`, `_C`, `_GO`, `_TS` and `_PY` override the server executable +for each language. An empty value disables that server. `ZIG_LIB_DIR` overrides +the stdlib path recorded at build time. `Lspinfo` shows the effective settings. + +The protocol client keeps one child server per configured language, negotiates +position encoding, and handles both push and pull diagnostics. Startup and +request waits have deadlines; failed starts back off before retrying. Worker +status messages reach the host event queue. + +`zig build lspprobe -- gd :` queries the same backend from +the command line. `zig build lspbench` measures it. Snapshot tests use a Zig +mock server with deterministic answers while exercising the real client, +including process startup, framing, edits and undo. `zig build fs-test` also +checks language-worker delivery in TTY and detached sessions over 9P. + +## Ownership + +`host_io.Lsp.Job` owns copies of the source, path, argument and root. Workers +never read the live core. A request ID and pane serial reject obsolete replies; +mutating replies also require the original file revision. Restore cancels +owned work before replacing the core. + +Backends implement `query`, `speaks` and `supports` in `src/lsp/`. The first +backend supporting the file and query answers; status queries visit all of +them. Results are written to the caller's writer. Use `lsp.row` or `spanRow` +for locations, `edit` for rename ranges, and `put` for replacement text. +Edit records use half-open byte offsets into the request's source snapshot. +The core validates the complete response before applying it. diff --git a/docs/macos.md b/docs/macos.md index 0ce29bf9..2d9fb1cc 100644 --- a/docs/macos.md +++ b/docs/macos.md @@ -430,59 +430,17 @@ without a Force Touch trackpad and when the user has feedback switched off, so a check here would only be a second place to be wrong — and it would be wrong the moment an external trackpad is plugged in mid-session. -## libproc, twice - -Two features on this backend want to know something about a process that is not -us, and on Linux both answers live in `/proc`. Darwin's equivalent is libproc, -and it answers both. - -**A pane's cwd** (`look.shellCwd`) is `readlink("/proc//cwd")` there and -`proc_pidinfo(PROC_PIDVNODEPATHINFO)` here. The tag shows it and a relative -`Look` resolves against it, so it has to follow the shell rather than stay -where the pane was spawned. - -*When* it is read differs from the other three shells, and deliberately. The -tty, SDL and detached-daemon hosts poll every pane every frame — inline in the -tty loop's frame path (`src/tty/tty.zig:1228-1231`), `pollCwds` in the SDL -shell (`src/gui/gui.zig:4048`) and `pollFrame` in the daemon -(`src/detached/server.zig:889`); here the drain has just finished -saying exactly which shells produced bytes, and nothing else can have moved -one — a `cd` is a command, and a shell that ran a command writes at least its -next prompt. So `refreshCwds` reads only for panes flagged by that tick's -output and an idle session costs no syscalls at all. `test/macos-snapshots/ -cwd.snap` holds the gating to it: the tag must be right after a `cd` and must -survive a tick with nothing in it. - -**A pardes inside a pardes** (`src/nested.zig`) walks the ancestor chain -looking for our own executable, and hands the file over rather than stacking a -second full-screen UI inside a pane. `readlink("/proc//exe")` becomes -`proc_pidpath`, and the `PPid:` line of `/proc//status` becomes -`proc_bsdinfo.pbi_ppid`. That struct is hand-written, which is a thing to get -silently wrong: a field ordering that puts something else where `ppid` should -be still returns a plausible number, so a unit test compares `parentOf(getpid())` -against `getppid()`. - -The socket half needed real portability work rather than a second spelling. -Darwin has no `SOCK_CLOEXEC` and no `accept4`, so the flag is set with an -`fcntl` after the fact — a race only against a fork on another thread, and -every caller is past that (`src/nested.zig:81-93` names all three). -`sun_path` is 104 bytes here against 108 there, so no -buffer in the file spells a number any more; they are all sized from the field -itself, and an address that does not fit is refused rather than truncated into -a path pointing somewhere else. - -Identity gained a third sibling. `bin/pardes`, `bin/pardes-gui` and -`pardes.app/Contents/MacOS/pardes` are three installed frontends of one -program, so `samePardesExecutable` (`src/nested.zig:142`) compares BASENAMES -and ignores the paths entirely — the GUI may be system-wide while the tty -frontend sits in the user's own bin directory. `familyTail` strips a leading -`pardes-gui` or `pardes` and requires what is left to be either empty or a -valid `-os-arch` pair, which is what keeps `pardes-snap` and `pardes-perf` out -of the family; `sameFamily` then accepts two names whose tails agree, or either -of which has no tail at all. The bundle's copy always has none, because -`CFBundleExecutable` is a fixed string: the bundled `pardes-macos-aarch64` is -called `pardes` and nothing in the name records what it was. So `pardes -src/foo.zig` inside the app's own shell opens a pane in the app. +## Shell directories and nested Look + +`host_io.shellCwd` reads a shell's working directory using +`proc_pidinfo(PROC_PIDVNODEPATHINFO)`; Linux uses `/proc//cwd`. +The macOS host refreshes directories for panes that produced output, so idle +frames do not poll every shell. `test/macos-snapshots/cwd.snap` covers the tag +after `cd` and after an idle tick. + +Nested launches use the same inherited 9P address and pane serial as other +native hosts. They do not inspect ancestor processes or executable names. +See [the control filesystem](fs.md) for paths and transports. ## Fonts and zoom @@ -672,8 +630,7 @@ stacked filters. Their canonical macOS source is `shaders/crt.ci.metal`, which the build installs as `pardes.app/Contents/Resources/crt.ci.metal`. The Swift shell loads that exact asset with `CIKernel.kernels(withMetalString:)` and runs it through a `CIContext` created from the system Metal device. The source is -also a Zig build import, so `EffectCode` can print what the app executes when -the source checkout is absent. +also archived by the Zig build; `EffectCode` links to it without a checkout. The ordinary CoreText/attachment/cursor pass is one function. With any scene bit or panel track it targets a retained, backing-scale bitmap; kernels then @@ -726,8 +683,8 @@ panes move and dissolve together. Scene CRT/ripple/glitch runs once after the panel composition. With no scene bit and no panel track the retained bitmap, Core Image context, -and Metal passes are bypassed. `EffectCode Panel*` embeds this actual Metal -file plus its direct Swift owner, the same files the app executes. +and Metal passes are bypassed. `EffectCode Panel*` links to this Metal +file and its Swift owner in the virtual filesystem. ### Transparent themes @@ -870,7 +827,7 @@ already do (`install_app_bin`, `install_plist`, `install_scene_kernel`, `CIKernel.kernels(withMetalString:)`. The same file is also an anonymous module import named `effect-source-crt.ci.metal`, added by `build.zig`'s per-shell module wiring under `if (shell == .macos)`, which is what lets - `EffectCode` print the exact source the app executes. + `EffectCode` link to the exact source the app executes. Its three entry points are `extern "C" [[stitchable]]`: the runtime compiler looks for stitchable functions and rejects the WHOLE source with "cannot find a valid stitchable Metal function in the source" when there are none, which @@ -981,12 +938,11 @@ prohibited) `NSWindow`, over a real core with real ptys: ```sh zig build macos-e2e -Dplatform=macos # run it zig build macos-e2e -Dplatform=macos -- --update # regenerate the goldens +zig build macos-e2e -Dplatform=macos -- test/macos-snapshots/rotate.snap ``` -The step always hands the harness the whole `test/macos-snapshots` directory, -so naming one script after `--` runs it IN ADDITION to the suite rather than -instead of it. To run a single script, invoke -`zig-out/bin/pardes-macos-e2e test/macos-snapshots/rotate.snap` directly. +With no paths, the harness runs `test/macos-snapshots`. Paths after `--` +select scripts or directories instead. The executable stays in the build cache. `test/macos_e2e.swift` links the same Swift sources the app does, minus `main.swift`, into a second binary — test scaffolding does not ship inside the @@ -1098,7 +1054,7 @@ used for the table and is not folded into the direct-path claim. - **Detached sessions.** `Attach` (`SPC s a`) and `Detach` (`SPC s D`) exist on every hosted platform, this one included, because `Builtin.enabled` is `pardes.hosted`. Neither works here. `Detach` emits `Effect.detach`, this - host fills in no `push_detach`, and `Pardes.perform` therefore reports + host fills in no `detach`, and `Pardes.perform` therefore reports `error.NotAttached` on the pane's message row (`src/pardes.zig:6837`). `Attach` emits `Effect.attach`, which `perform` turns into an `attach_req` the shell is supposed to consume from OUTSIDE `pump` with `takeAttach` — and diff --git a/docs/registry.typ b/docs/registry.typ index 40af056a..4d386f1c 100644 --- a/docs/registry.typ +++ b/docs/registry.typ @@ -153,6 +153,10 @@ #v(0.8em) +*Historical decision record.* Entries retain their original source references +and states. For the current filesystem and transport interface, use +`docs/fs.md`; FUSE and the proof-of-concept examples are no longer supported. + #context { let es = query().map(e => e.value) let order = ("open", "investigating", "blocked", "decided", "deferred", "landed", "rejected") @@ -488,21 +492,21 @@ rejected whole. Buildable, and much cheaper than anyone assumed. But it is a SECOND FIRMWARE IMAGE, not a second role for this one: the editor owns UART0 bidirectionally (`esp32p4.zig:611`, `uart.zig:43-44,122-130`) and JP1 - exposes no second P4 UART (`board_memory.zig:382-383`). The board is either + exposes no second P4 UART (`board9p.Header`). The board is either an editor or a filesystem at any one time. Say that plainly rather than implying both. ] #note("review", "2026-08-27")[ The reframing the draft misses: the board already exposes `Peek`, `Poke`, - `Hexdump` and `Gpio` as acme words (`src/board_memory.zig:306-349,410-472`, - registered `src/builtins.zig:1009,1025,1038`). All four cap at 4,096 bytes + `Hexdump` and `Gpio` as acme words (`src/builtins.zig`, `Board` namespace). + All four cap at 4,096 bytes per command and the cap's stated reason is the 115200 console. So the whole 2³² address space is already reachable — by *typing a word into a tag*, with the answer landing in an output pane. Nothing is machine-readable and nothing is remote. A 9P tree is that same capability with names instead of verbs, and `mem/`, `gpio/pinout` and `prof` are backed by functions that - exist today (`board_memory.readWord:136`, `:368-390`, + exist today (`builtins.Board.readWord`, `builtins.Board.gpio`, `pardes_esp32p4_frame_prof` at `esp32p4.zig:985`, already exported). Four more files need one new C-ABI extern each; four have no implementation at all. That inventory belongs in the note, not a wishlist. @@ -1022,7 +1026,7 @@ Three of those four are achievable. One is not, and `9P-20` says which. `.lsp_resp`, `.pipe_resp` (`src/pardes.zig:6896-6907`). `pull_wait_input` cannot become an event because it is how events arrive. That leaves two: `pull_gpio_toggle`, whose answer is only used to set a message row - (`board_memory.zig:422-428`), so deferring it a frame is invisible; and + (`builtins.Board.gpio`), so deferring it a frame is invisible; and `pull_tty_taken`, which is reached from `takesCommandLine` (`pardes.zig:6465-6469`) at four call sites, two of which loop over all 16 panes — 16 probes per call, collapsing to ONE read if `proc` is a single file diff --git a/docs/web.md b/docs/web.md index 8d3a3d66..3a68f958 100644 --- a/docs/web.md +++ b/docs/web.md @@ -43,8 +43,8 @@ shells, and two comptime capability flags say so once each. It exposes no `PanelCurtain`, `PanelScramble`, `PanelType`) because `capabilities.panel_transitions` is `pardes.hosted`, and no `Crt`/`Ripple`/`Glitch` because `capabilities.scene_shaders` is -`platform == .gui or platform == .macos` (`src/builtins.zig:41-42`; the words -themselves carry those availabilities in `src/runtime_config.zig:155-168`). +`platform == .gui or platform == .macos` (`builtins.capabilities`; the words +themselves carry those availabilities in `config.Runtime.settings`). Applying either faithfully would require a second canvas renderer and give up the DOM renderer's selectable/accessibility contract. Theme fades and the delayed, side-effect-free Look hover remain grid animations and continue to use @@ -57,14 +57,14 @@ zig build web \ -Dplatform=web \ -Dtarget=wasm32-freestanding \ -Dtree-sitter=zig \ - -Ddump=test/web-snapshots/source-list.dump.zon + -Ddump=test/web-snapshots/touch.dump.zon ``` The output is in `zig-out/web`. Serve that directory over HTTP; browsers do not allow a useful WASM module load from `file:` URLs. The browser shell has no argv: every command-line flag `src/main.zig` parses — -`--tty`, `--tty-toggle`, `-n`, `-l`, `--fs[=]`, `--nested`, +`--tty`, `--tty-toggle`, `-n`, `-l`, `--9p=`, `--mount==`, `--nested`, `--detach[=]`, `--attach[=]`, `-h`/`--help`, `--version`, and the positional file-or-directory — is native-only, because the wasm module roots at `src/web.zig` and never links `main.zig`. State comes from the embedded dump @@ -92,7 +92,7 @@ the browser can never call. same gate: both are `enabled = pardes.hosted` (`src/builtins.zig`, and `pardes.hosted` is `tty or gui or macos`), because a detached session is a unix socket and a page has none. Nothing in this shell speaks -`src/detached/wire.zig`, and `push_detach` is one of the null vtable methods +`src/detached/wire.zig`, and `detach` is one of the null vtable methods below. `src/web.zig` fills in six methods of `Host.VTable` and leaves the rest null, @@ -120,8 +120,8 @@ failure; `wait_input` is null because wasm must never block, and `watch_theme` and `dump_themes` are not reachable without a config directory. Of the remaining nulls, `post_present` and `poll_frame` are per-frame host bookkeeping a page has none of, `detach` has no session to leave, `gpio_toggle` belongs to -the ESP32-P4 board, and `fs_reply` answers a FUSE control filesystem that -`--fs` is the only way to ask for. `quit` sets the +the ESP32-P4 board, and native filesystem replies are delivered by the 9P +listener. `quit` sets the core's flag, which `pardes_should_quit` reports so the page can stop its frame loop. @@ -133,7 +133,7 @@ zig build web-harness \ -Dplatform=web \ -Dtarget=wasm32-freestanding \ -Dtree-sitter=zig \ - -Ddump=test/web-snapshots/source-list.dump.zon + -Ddump=test/web-snapshots/touch.dump.zon # Real headless Chrome, DOM rendering, and browser touch input. zig build web-e2e -- cgit v1.3