path: root/docs/typ/building.typ

// Building pardes, for contributors: platforms, options, tests and the
// release gates, then how the pieces fit: the core and its shells, the
// threads, detached sessions, the 9P engine, the listeners and mounts.
#import "style.typ": key, keys, btn, chord, word, tag, addr, file, cmd, doc, pairs

= Platforms

Zig *0.16.0* (`build.zig.zon` pins `minimum_zig_version`). Dependencies are
fetched and pinned by the manifest; the terminal build needs no system
package. The SDL shell builds SDL3 and FreeType from source. PDF support
builds MuPDF and is on by default (`-Dmupdf=false` drops it). 9P over QUIC
(`-Dquic=true`) uses system OpenSSL 3.6+ and pkg-config.

#pairs(
  [`zig build`], [the terminal shell and the SDL window together, *installed into `~/.local/bin`* (`pardes`, `pardes-gui`, and `pardes-v9fs`, the #word("Tty9p") helper)],
  [`zig build -Dplatform=tty`], [the terminal shell alone],
  [`zig build -Dplatform=gui`], [the SDL3 window alone],
  [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=<dump.zon>`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)],
  [`zig build -Dplatform=macos`], [an AppKit and CoreText app over a static `libpardes.a` (`docs/macos.md`)],
  [`zig build -Dplatform=esp32p4`], [a freestanding riscv32 editor object for the ESP32-P4, with a 384 KiB heap],
)

A bare `zig build` installs into `~/.local`; `--prefix <dir>` installs
elsewhere, except `--prefix zig-out`, which counts as no prefix. With
`-Dplatform` the default prefix is `zig-out`, so always pass `-Dplatform`
while developing. Test, benchmark and run steps build what they need
without installing. `pardes --version` prints `pardes <version>` (from
`build.zig.zon`), plus the commit for release builds and the `~/.local`
install; `pardes --help` lists every flag.

The ESP32-P4 firmware is linked in the sibling `../05-zig-p4` toolchain,
which needs ESP-IDF register headers: after building the editor object
here, `zig build -Dpardes` there links `src/esp32p4/app.zig`. The separate
GPIO 9P image is `zig build -Dapp=../02-pardes-code/src/esp32p4_9p.zig`
there; its namespace is in `src/esp32p4_gpio.zig`.

== Build options

`zig build --help` lists the options for the selected platform.

#pairs(
  [`-Dplatform`], [`tty`, `gui`, `web`, `macos`, `esp32p4`; absent: the tty and SDL shells together, installed into `~/.local`],
  [`-Dstatic`], [bool, `false`],
  [`-Dquic`], [bool, `false`: 9P over QUIC with system OpenSSL 3.6+],
  [`-Dmupdf`], [bool; on natively, off for web and esp32p4],
  [`-Djpx`], [bool, `true`: JPEG 2000, and with it scanned PDFs],
  [`-Dtree-sitter`], [`disabled`, `zig`, `minimal`, `full`; `full` natively, `zig` for web, `disabled` for esp32p4],
  [`-Dembed-sources`], [bool, `false`: serve the sources under #file("/src")],
  [`-Dstamp-commit`], [bool: the git commit in `--version` and crash records; on for release builds and the `~/.local` install],
  [`-Dtheme-animation`], [bool; on except for esp32p4],
  [`-Dworkspace-tag`], [bool: draw the workspace tag row; on except for macOS, whose menu bar carries it],
  [`-Dprebuilt-shaders`], [bool: embed the committed SPIR-V; on for a bare `zig build`, off with `-Dplatform`; `zig build shaders` refreshes it with glslc],
  [`-Dtracy`], [path to a Tracy checkout; off],
  [`-Dmacos-identity`], [codesigning identity for `pardes.app`; `-` (ad hoc)],
  [`-Ddump`], [a `dump.zon` to embed in the web shell],
  [`-Dtest-filter`], [run only tests whose name contains it; a filter that matches nothing fails],
  [`-Dtest-rebuild`], [bool: fresh Zig test compilation],
  [`-Dhelix-harness`], [reference executable for live differential tests; `HX_HARNESS`, else `hx-harness` on `PATH`],
  [`-Desp32p4-cols`, `-Desp32p4-rows`], [the board's grid, 56 by 14],
)

= Tests

```
zig build unit-test        module and shell unit tests (the perf gate runs with it)
zig build test-build       compile the unit-test programs without running them
zig build core-test        core tests without native shell tests
zig build pane-test        pane, output, PDF and namespace integration tests
zig build syntax-test      tree-sitter tests without building the editor
zig build syntax           deterministic per-byte highlighting snapshots
zig build fs-test          real sessions and mounts over 9P (with a short 9P monkey)
zig build monkey-9p        random 9P operations checking the documented rules
zig build agent-session-test  interactive session driver checks over 9P
zig build 9p-test          freestanding protocol tests
zig build 9p-io-test       native 9P transports and client
zig build quic-test -Dquic=true  optional QUIC transport tests
zig build v9fs-test        a real kernel mount (needs sudo -v; fails, not skips, without it)
zig build snap             scripted input traces against frozen golden grids
zig build monkey           random snapshot scripts hunting panics (not a gate)
zig build hxdiff           differential suite against helix's own behaviour
zig build hxparity         file-pane vs pty-pane editing parity
zig build hxgolf           every helix-golf example, step by step, against helix
zig build perf-gate        a gesture on the 50k-line file over 3x its baseline fails
zig build mupdf-check      compile, link, render and search docs/design.pdf
zig build web-snap         browser highlighting and touch interactions
zig build web-e2e          Chrome-driven DOM end-to-end suite
zig build cheatsheet       render the cheatsheet PDF (needs typst)
```

- `-Dtest-filter=<text>` applies to every unit-test binary. A failed test's
  printed trace can be stale: run it alone with the filter.
- `snap -- --record=DIR` writes snapshots without touching goldens,
  `snap -- --update` replaces goldens (re-record one script by name, after
  reading its diff, never all), `snap -- --no-retry` makes the first
  failure decisive. Snapshot scripts can use `snap9p` to capture core cells
  through 9P.
- `zig build monkey -Dplatform=tty -- <seeds> [steps] [--from=N] [--out=DIR] [--keep]` writes one random script per seed and keeps any
  that panics; a seed always makes the same script. Each crash found gets a
  fix and a regression script in `test/snapshots/`.
- `zig build hxdiff -- --strict cases.jsonl reference.jsonl [waivers.jsonl]` runs custom differential cases. `docs/helix-keys.md`
  tracks helix parity key by key and names the reference helix build;
  `docs/selections.md` is the selection model under normal mode.
- Benchmarks (`perf`, `pdf-bench`, `pdf-scroll-bench`,
  `pdf-sections-bench`, `lspbench`, `fs-bench`) take `-- --json`; measure
  with `-Doptimize=ReleaseFast`. `perf -- --base old.json` refuses reports
  whose build metadata differs.
- `zig build history -- run 'ancestors(@, 2)' DIR -- zig build unit-test`
  records a command's output and runtimes across revisions; `history -- compare a.json b.json 1.20` fails past that runtime ratio.
- `python3 -B test/agent_session.py <pardes> --ready 'text' --min-rows N -- command args` drives an interactive command in a private shell and checks
  it through 9P.

== Release gates

A release passes all of these first: `unit-test` with `-Dplatform=tty` and
with `-Dplatform=gui`, `core-test`, `fs-test`, `snap`, the GUI goldens
(`python3 -B test/gui_golden.py <a Debug pardes-gui>`, a hidden window),
`web`, and the ReleaseSafe tty and ReleaseFast gui builds. The committed
`docs/typ/cheatsheet-a4.pdf` is rendered again when the docs change.

= The core and its shells

One core, five shells (tty, gui, web, macos, esp32p4). The core owns
editing, layout, rendering and the control filesystem; a shell turns native
input into `pardes.Event`, presents `pardes.Surface`, and performs host
effects such as spawning processes.

- The core is a state machine: input arrives as an `Event` through
  `update`; output leaves as a `Surface` from `render(arena)` and as a ring
  of `Effect` values. Effects are fixed-size values with no lifetime ties
  into the core; unbounded content is read off the core when an effect is
  drained (`save_text` carries a path and the pane's serial, so a reused
  slot writes nothing). `emit` refuses when the ring is full and never
  evicts.
- Pty bytes leave as 64-byte `.write` effects; overflow parks in a per-pane
  buffer that `nextEffect` drains in order. `postEvent` is a 64-entry value
  queue for events that borrow no slices.
- `pump` runs: wait for input (the host owns the sleep), flush paused 9P
  write batches, drain queued events, drain effects, settle the turn, then
  render and present only when a frame is needed and someone is looking.
- `host_io.Host` is a context pointer and a vtable of optional callbacks; a
  null method is not an error, and `Host{}` is a complete in-process pardes
  that the tests use. Comptime decides what a build has (`pardes.platform`,
  `hosted`, `can_attach`, `terminal_panes`, `pdf_enabled`); the vtable
  decides who serves it.
- The root module is chosen by platform: `main.zig` (tty, gui),
  `web.zig`, `macos.zig`, `esp32p4.zig`. The web and macOS shells are
  libraries whose host owns `main()`.
- `memory.limits` is the one home for capacities that differ on the board
  (panes 16, columns 6, selections 64, the tag's 512 bytes). Each core
  allocator is a thread-safe stack-fallback allocator, under a
  DebugAllocator in Debug builds.

Code map: `src/panes.zig` and the pane kinds it names (`src/File.zig`,
`src/Terminal.zig`, ...), `src/layout.zig`, `src/fs.zig` (host access,
mounts and resolution) and `src/pardes.zig` (input), with one file per
thing beside them (`edit.zig`, `normal.zig`, `look.zig`, `exec.zig`,
`mouse.zig`, ...). `src/ninep/` is the control tree, `src/detached/` the
wire, the detached core and the frontend client, `src/lsp/` the
language-server client; `test/` holds harnesses, goldens and helix cases,
`build/` the snapshot suite's build step.

== Threads

The core is single-threaded under a turn mutex, `pardes.turn`. The editor
thread holds the turn and lets go of it while it waits for input and while
it is out in a host syscall mid-step; a 9P connection task takes the turn
in those gaps to answer a request. While a step is out, a request that
would change a pane parks (`Status.again`) and is retried when the editor
rests. 9P requests are not events: they enter through `Pardes.serveFs`, and
only one that changes a pane costs a frame. Language-server and
selection-pipe workers post completions through a bounded mailbox; a
`host_io.Lsp.Job` owns copies of the source, path and arguments, never
reading the live core, and a request id, the pane's serial and (for an
edit) the file's revision reject a stale reply. A process that never calls
`turn.start` (the tests, the board, the browser) has no second thread.

== Detached sessions

One poll loop in `src/detached/server.zig` owns core mutation, frontend
connections, pty I/O and file watches. The frontend socket is
`pardes-detached-<name>.sock` beside the session's 9P socket; a second
session cannot take a live name. Up to 32 frontends attach; frames go to
every one, while clipboard reads, browser opens and #word("Detach") go to
the frontend that asked (else the first attached). Frames are full grids
or changes against what each frontend last received, never queued: a
frontend with unsent bytes skips that frame, and one that stops draining
past 1 MiB of control backlog is closed, its peers untouched. Frontends
never spawn shells, write session files or watch files. #word("Attach")
connects first and swaps second: only after the handshake does a shell
give up its own core. Restore builds the replacement core before touching
the current one.

`src/detached/wire.zig` is the versioned protocol (version 10): fixed-width
little-endian fields, a 5-byte header (tag, then a u32 length), payloads up
to 16 MiB and grids up to 512 by 128. A comptime hash of `pardes.Chrome`
forces a version bump when that struct changes.

= 9P

The wire format, the client and server connections, the file-server engine
and the Unix, TCP and QUIC transports are the `cloud9` package
(`git.sr.ht/~gbrls/cloud9`), pinned in `build.zig.zon` and fetched into
`zig-pkg/`. Re-pin with
`zig fetch --save=cloud9 git+https://git.sr.ht/~gbrls/cloud9#<commit>`, or
use `.cloud9 = .{ .path = "../cloud9" }` while editing both. cloud9's own
`zig build test`, `transport-test`, `quic-test -Dquic=true`, `fuzz` and
`differential` cover the shared code.

- The engine (`fs.Server`) owns fids, permissions, directory reads,
  flushes and what a hangup releases; it allocates nothing and makes no OS
  calls. The control tree in `src/ninep/` is its backend: `tree.zig`
  (nodes, dispatch), `pane.zig`, `ctl.zig`, `cols.zig`, `addr.zig`,
  `pty.zig`, `events.zig` (event and log), `screen.zig`, `sources.zig`.
  `src/9p.zig` names the editor's and the board's engine settings: msize
  65536, 256 fids, 128 held reads a connection, names up to 255 bytes; the
  board has 32 fids.
- A read, write, open, clunk, remove or truncating wstat can park; a walk,
  attach, stat, create or rename cannot. Every Rread is clamped to the
  count and the msize; Tflush answers the original request first.
- Unix and TCP listeners run on cloud9's `serve.Runner`: an accept task per
  listener, a reader and a writer task per connection, 16 connections. QUIC
  (`src/9p_quic.zig`) still runs on the editor's poll loop in
  `src/9p_io.zig`.
- A body write takes only whole UTF-8 sequences and answers a short count
  for the rest. Consecutive writes from one open at the end of the text are
  held and applied as one edit, flushed by any other request, the open's
  release, 64 MiB, or a 20 ms pause in the editor's step.

== Writes through a mount <writes-through-a-mount>

A mount cuts a big write into pieces of at most one message (msize 65536,
less the header), and each command line runs once it is whole. A write
that does not fill its message is whole, so its last line runs even
without a newline (`printf Save > exec`), unless it is a multiple of 4096
bytes: that is where a writer's buffer (stdio, a mount's page cache)
filled and cut a line, so its tail waits for the next write or the close.
A line held to the close (such a tail, or an `Edit` block never ended)
runs there, and its failure is only in the log, as its `err` record: the
close reports no error, and the write that sent it had already succeeded.
So a script that needs a line's result ends the write with a newline.

== Listeners <listeners>

`--9p-tcp='tcp!127.0.0.1!5640'` adds TCP; `--9p-quic='quic!127.0.0.1!5641'`
adds QUIC (built with `-Dquic=true`; ALPN `pardes-9p`, an ephemeral TLS
identity, no peer verification). Addresses are numeric IPv4 or IPv6; port
0 picks one; #file("/listeners") reads them back. Every connection has full
session access, #file("/os") included, and TCP is unencrypted: use
loopback. Unix and TCP share 16 connection slots; a 17th client's Tversion
gets `too many connections` (and the log
`err - 9p: too many connections (N turned away)`). QUIC has 16 of its own.
plan9port and v9fs need a userspace bridge for QUIC.

`$XDG_RUNTIME_DIR/9p` is the machine's `/srv`: servers post themselves
there by name, and `9ns --mntgen` mounts the whole registry. A pardes whose
socket is in the runtime directory posts
`$XDG_RUNTIME_DIR/9p/pardes/<name>`, a symlink to its socket, and unposts
it on a clean stop if it is still its own; one whose socket fell back to
`~/.local/state/pardes` posts nothing. Posting first sweeps the group: a
symlink whose socket refuses a connect is removed with its socket. pardes
binds its own socket rather than going through `cloud9.post`, which takes
only flat names. A reader of the registry must `stat` through the symlink.

== Kernel mounts and Tty9p

For Linux v9fs use `version=9p2000,cache=none,access=any`, `trans=unix`
(or `trans=tcp` with `port=`), `uname`, `dfltuid` and `dfltgid` for the
local user, and an empty `aname`. Linux follows `O_TRUNC` with a `Twstat`
of zero length and an `mtime` hint; pardes takes the truncation and drops
the hint.

#word("Tty9p") starts the normal shell and queues a quoted helper command,
which bash and fish run at their first prompt as a foreground job, so
sudo has the terminal. The unprivileged launcher makes a private temporary
mountpoint and runs `sudo -E`; the elevated `pardes-v9fs` helper makes a
private mount namespace, mounts the session's socket with
`trans=unix,version=9p2000,cache=none,access=any,nosuid,nodev,noexec`,
drops every root id and capability, and runs the shell as the user (with
the caller's `PATH` again). The namespace and the mount go with its last
process. Nothing setuid and no passwordless sudo rule is installed; the
helper takes explicit paths and a command and is no restricted broker, so
never grant it passwordless sudo. The host finds the helper beside its own
executable, or at `PARDES_V9FS_HELPER`. Code: `src/linux/v9fs.zig`;
`zig build v9fs-terminal-test` and `v9fs-driver-test` need no privileges.

= Other notes

`docs/` keeps the platform and design notes beside this book:
`web.md` and `macos.md` (the browser and macOS shells), `effects.md` and
`render-pipeline.md` (visual effects), `helix-keys.md` and `selections.md` (helix parity),
`lsp-evaluation.md`, `ui-review.md`, `divergences.md` (bookmarks off
`main`) and `open-questions.md`. `next-steps.txt` is a wishlist and
`transactions.txt` records one open structural gap against helix.