1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
|
// Building pardes, for contributors: platforms, options, tests, the
// release gates and sending patches, then how the pieces fit: the core
// and its shells, the threads, the daemon and its sessions, the 9P engine, the
// listeners and mounts.
#import "style.typ": key, keys, btn, chord, word, tag, addr, file, cmd, doc, pairs
For contributors: how pardes is built, tested and released, and how its
pieces fit. To install it, see #doc("setup", section: "install").
= Platforms
Zig *0.17.0* (`build.zig.zon` pins `minimum_zig_version`). Dependencies are
fetched and pinned by the manifest; the terminal build needs no system
package. The SDL shell builds SDL3 and FreeType from source. Pictures and
PDFs are addons, each its own step: `zig build image`, `zig build pdf`
(`pdf.so`, over MuPDF, its own FreeType and zlib in it), or `zig build
addons` for both; the binaries hold neither. MuPDF, and the FreeType and
zlib the terminal build takes only for it, are lazy: a build fetches them
only with `-Dpdf`, which is on by itself once MuPDF is fetched, so a fresh
checkout's `zig build -Dplatform=tui` downloads none of them, and its
first `zig build pdf -Dpdf` (or `addons`, `static`, `mupdf-check`, a PDF
bench) fetches MuPDF once; without it those steps say to add `-Dpdf`, and
the tests leave their PDF cases out. (Zig 0.17's configure is handed the
options and never the steps asked for, so a step cannot fetch for itself.)
MuPDF is compiled only for the steps and tests that use `pdf.so`.
#pairs(
[`zig build`], [the terminal shell and the SDL window together, *installed into `~/.local/bin`* (`pardes` and `pardes-gui`)],
[`zig build -Dplatform=tui`], [the terminal shell alone],
[`zig build -Dplatform=gui`], [the SDL3 window alone],
[`zig build -Dplatform=macos`], [an AppKit and CoreText app over a static `libpardes.a` (`docs/macos.md`)],
[`zig build static -Dplatform=tui -Dtarget=x86_64-linux-musl`], [one statically linked `pardes` with the image and PDF addons compiled in (`/addons` lists them `builtin`); musl's, as glibc links only dynamically, and never the default],
[`zig build static -Dplatform=gui`], [`pardes-gui` with the two addons compiled in, linked dynamically (SDL and the GPU take the display's libraries)],
)
Off Linux the addons are always compiled in, as in a static build:
`Addon.zig` loads a library through a Linux memfd, and `Addon load`
elsewhere says `not on this platform`. A gui for macOS built on another
system takes the SDK SDL needs with `-Dmacos-sdk=<path>`.
A bare `zig build` installs into the prefix (`zig-out`, or `--prefix
<dir>`) and copies the binaries into `~/.local/bin`; `-Dhome-install=false`
leaves that copy out. Zig 0.17 keeps the prefix from the build script, so
`--prefix` alone no longer stops it. With `-Dplatform` nothing is copied,
so always pass `-Dplatform` while developing. Test, benchmark and run steps build what they need
without installing. `pardes --version` prints `pardes <version>` (from
`build.zig.zon`), plus the commit for release builds and the `~/.local`
install, and on a second line the build's semantics hash, which a replica
must share with its session; `pardes --help` lists every flag.
== Build options
`zig build --help` lists the options for the selected platform.
#pairs(
[`-Dplatform`], [`tui`, `gui`, `macos`; absent: the tui and SDL shells together, installed into `~/.local`],
[`-Dpdf`], [bool: build the pdf addon, fetching MuPDF the first time; default on once MuPDF is fetched, and off Linux],
[`-Djpx`], [bool, `true`: JPEG 2000, and with it scanned PDFs],
[`-Dtree-sitter`], [`disabled`, `zig`, `minimal`, `full`; `full`],
[`-Dembed-sources`], [bool, `false`: serve the sources under #file("/src")],
[`-Dhome-install`], [bool: a bare `zig build` also copies the binaries into `~/.local/bin`; default true],
[`-Dstamp-commit`], [bool: the git commit in `--version` and crash records; on for release builds and the `~/.local` install],
[`-Dtheme-animation`], [bool, `true`],
[`-Dworkspace-tag`], [bool: draw the workspace tag row; on except for macOS, whose menu bar carries it],
[`-Dmacos-sdk`], [path: the macOS SDK SDL's headers and frameworks come from, for a gui built for macOS elsewhere],
[`-Dprebuilt-shaders`], [bool: embed the committed SPIR-V; on for a bare `zig build`, off with `-Dplatform`; `zig build shaders` refreshes it with glslc],
[`-Dtracy`], [path to a Tracy checkout; off],
[`-Dmacos-identity`], [codesigning identity for `pardes.app`; `-` (ad hoc)],
[`-Dtest-filter`], [run only tests whose name contains it; a filter that matches nothing fails],
[`-Dtest-rebuild`], [bool: fresh Zig test compilation],
[`-Dhelix-harness`], [reference executable for live differential tests; `HX_HARNESS`, else `hx-harness` on `PATH`],
)
= Tests
```
zig build unit-test module and shell unit tests (the perf gate runs with it)
zig build test-build compile the unit-test programs without running them
zig build core-test core tests without native shell tests
zig build shell-test the native shell's own unit tests, without the perf gate
zig build pane-test pane, output and namespace integration tests
zig build kitty-test kitty graphics conformance of terminal panes, at the core
zig build kitty-harness kitty graphics through a pane end to end (a fake kitty
terminal outside on the tui, the GPU capture on the gui;
kitten icat and chafa too when installed)
zig build syntax-test tree-sitter tests without building the editor
zig build syntax deterministic per-byte highlighting snapshots
zig build behaviour-test behaviour through the 9P tree, in process (test/behaviour/),
and the T1 runner's short run of random 9P requests
zig build process-test real sessions over their sockets: mounts, flags, shells (test/process/),
and the tty's own cases, each a pardes in a pty against its golden
(test/process/tty_*.zig, test/snapshots/); -Dplatform=gui runs the
socket cases on the hidden window instead
zig build monkey-gui random input to the hidden window, every effect setting (-Dplatform=gui)
zig build 9p-test freestanding protocol tests
zig build 9p-io-test native 9P transports and client
zig build hxdiff differential suite against helix's own behaviour
(-- --parity: each case in a file pane and a pty pane, which must agree)
zig build hxgolf every helix-golf example, step by step, against helix
zig build perf-gate a gesture on the 50k-line file over 3x its baseline fails
zig build mupdf-check the PDF addon's MuPDF: compile, link, render and search docs/design.pdf
zig build cheatsheet render the cheatsheet PDF (needs typst)
```
- `-Dtest-filter=<text>` applies to every unit-test binary. A failed test's
printed trace can be stale: run it alone with the filter.
- `process-test -- <filter>` runs the cases whose name holds it (`tty
editing:` a family of the tty's, `tty editing: grep` one).
`process-test -- --update <filter>` writes a tty case's captures to its
golden: one case at a time, then a plain run, and read the golden's diff
for moved shell output (a flake) before keeping it. A tty case that fails
runs again alone with a wider settle probe before it counts; what it drew
is kept in `/tmp/pardes-snap/<name>/drew.golden`.
- `behaviour-test -- --monkey <seeds> [--steps M]` types random keys, leader
bursts, mouse presses, drags, wheel turns and tiny resizes at an in-process
session, a seed a child, hunting panics and hangs; a seed always makes the
same input, and a failure is shrunk to the steps that make it. `-- --seed N`
runs the random 9P requests with the documented rules checked after each.
Each crash found gets a fix and a regression case.
- `behaviour-test -- --replay` records each monkey seed's sessions as tapes
and replays them (three replicas and the dump); `process-test -- --replay
[--replay-with=<pardes-gui>]` records and replays the tty cases' real
sessions, their shells' and files' answers on the tape.
- The tutor's practice blocks (`# keys:`, `# before`, `# after` in
`src/tutor.txt`) are typed by hand; nothing runs them. What pins that
behaviour is `hxdiff`, which replays `test/hxcases` against goldens
recorded from a real helix, and its `--parity`, which runs each case in a
file pane and in a shell and demands they agree.
- `zig build hxdiff -- --strict cases.jsonl reference.jsonl [waivers.jsonl]` runs custom differential cases. `docs/helix-keys.md`
tracks helix parity key by key and names the reference helix build;
`docs/selections.md` is the selection model under normal mode.
- Benchmarks (`perf`, `pdf-scroll-bench`, `pdf-pointer-bench`,
`lspbench`, `fs-bench`) take `-- --json`; measure
with `-Doptimize=ReleaseFast`. `perf -- --base old.json` refuses reports
whose build metadata differs.
- `zig build history -- run 'ancestors(@, 2)' DIR -- zig build unit-test`
records a command's output and runtimes across revisions; `history -- compare a.json b.json 1.20` fails past that runtime ratio.
- `python3 -B test/agent_session.py <pardes> --ready 'text' --min-rows N -- command args` drives an interactive command in a private shell and checks
it through 9P.
== Release gates
A release passes all of these first: `unit-test` with `-Dplatform=tui` and
with `-Dplatform=gui`, `core-test`, `behaviour-test`, `process-test` (on both:
its tty cases on the tui, the hidden window on the gui), `hxdiff` and its
`--parity`, the replicas harness and the soak (`test/soak.py`), and the
ReleaseSafe tui and ReleaseFast gui builds. The committed
`docs/typ/cheatsheet-a4.pdf` is rendered again when the docs change.
= Sending patches <patches>
Mail patches to the list `~gbrls/[email protected]`; it is the preferred way to contribute:
#cmd("git config format.subjectPrefix \"PATCH pardes\"\ngit send-email --to=~gbrls/[email protected] HEAD^")
#link("https://git-send-email.io")[git-send-email.io] sets `git send-email` up. The mirrors (#doc("setup", section: "install")) are for cloning, as a GitHub one will be; pull requests are not the preferred route.
= The core and its shells
One core, three shells (tui, gui, macos). The core owns
editing, layout, rendering and the virtual filesystem it serves over 9P; a shell turns native
input into `pardes.Event`, presents `pardes.Surface`, and performs host
effects such as spawning processes.
- The core is a state machine: input arrives as an `Event` through
`update`; output leaves as a `Surface` from `render(arena)` and as a ring
of `Effect` values. Effects are fixed-size values with no lifetime ties
into the core; unbounded content is read off the core when an effect is
drained (`save_text` carries a path and the pane's serial, so a reused
slot writes nothing). `emit` refuses when the ring is full and never
evicts.
- Pty bytes leave as 64-byte `.write` effects; overflow parks in a per-pane
buffer that `nextEffect` drains in order. `postEvent` is a 64-entry value
queue for events that borrow no slices.
- `pump` runs: wait for input (the host owns the sleep), flush paused 9P
write batches, drain queued events, drain effects, settle the turn, then
render and present only when a frame is needed and someone is looking.
- `host_io.Host` is a context pointer and a vtable of optional callbacks; a
null method is not an error, and `Host{}` is a complete in-process pardes
that the tests use. Comptime decides what a build has (`pardes.platform`,
`can_attach`); the vtable
decides who serves it.
- The root module is chosen by platform: `main.zig` (tui, gui),
`macos.zig`. The macOS shell is a library whose host owns `main()`.
- `memory.limits` is the one home for the core's capacities (selections,
the tag's bytes, undo, the arenas' regions). Each core
allocator is a DebugAllocator over its arena's reserved region, thread
safe, its checks in Debug builds only.
Code map: `src/panes.zig` and the pane kinds it names (`src/File.zig`,
`src/Terminal.zig`, ...), `src/layout.zig`, `src/fs.zig` (host access,
mounts and resolution) and `src/pardes.zig` (input), with one file per
thing beside them (`edit.zig`, `normal.zig`, `look.zig`, `exec.zig`,
`mouse.zig`, ...). `src/ninep/` is the control tree, `src/daemon.zig` the
per-user daemon and a terminal's stub, `src/wire.zig` the input
wire a remote client will speak, `src/lsp/` the
language-server client; `test/` holds harnesses, goldens and helix cases,
`build/` the process tests' build steps.
== Threads
The core is single-threaded under a turn mutex, `pardes.turn`. The editor
thread holds the turn and lets go of it while it waits for input and while
it is out in a host syscall mid-step; a 9P connection task takes the turn
in those gaps to answer a request. While a step is out, a request that
would change a pane parks (`Status.again`) and is retried when the editor
rests. 9P requests are not events: they enter through `Pardes.serveFs`, and
only one that changes a pane costs a frame. Language-server and
selection-pipe workers post completions through a bounded mailbox; a
`host_io.Lsp.Job` owns copies of the source, path and arguments, never
reading the live core, and a request id, the pane's serial and (for an
edit) the file's revision reject a stale reply. A process that never calls
`turn.start` (the tests) has no second thread.
== The daemon and detached sessions
One `pardes --daemon` per user (`src/daemon.zig`) listens on
`pardes-daemon.sock` in the runtime directory, its `.lock` flocked for its
life; the first session that finds none starts it (a double fork and an
exec) and it exits a second after it holds nothing. It is nobody's parent:
a session registers on one connection, sends it a dup of every pty master
it forks (so a session killed outright never hangs its shells up, and
#word("Adopt") takes them into another), says whether a terminal or a
window shows it, and is handed a terminal (`pardes --attach`) or a window
request (`pardes-gui --attach`) through it. Messages are `SOCK_SEQPACKET`
datagrams: a line of words and the fds it carries.
`pardes` in a terminal forks its session off before anything starts: the
session, in a session of its own (`setsid`), draws on the terminal it
opened, and the process the shell waits on stays as its stub, passing it
`SIGWINCH`, `SIGTERM`, `SIGINT` and `SIGQUIT` and ending with its status.
#word("Detach"), or the terminal's end (EIO, a hangup), puts
the terminal back and lets it go; the stub ends (exit 0) and the session
runs on with nothing drawn, until `pardes --attach` hands it another
terminal. A window's #word("Detach") closes the window and keeps the
process; `pardes-gui --attach` opens one again. `PARDES_DAEMON=off` (the
process tests) runs a session as before the daemon: in the terminal's
own process, ended by its hangup.
A replica (a client running its own copy of the core, phase 5 of
`docs/next-daemon.md`) connects to `pardes-replica.sock`, a stream socket
beside it that `ssh -L` can forward. `src/wire.zig` is its
protocol (version 14): fixed-width little-endian fields, a 5-byte header
(tag, then a u32 length), the hello, a ping, the input events. The hello
names a session and carries the client's semantics hash (the protocol,
release, commit, platform, build options and every dependency's hash); the
daemon refuses another build's, or a session nobody has, in words, and
hands an accepted connection to its session (`SCM_RIGHTS`), which welcomes
it and from then on is the one it talks to. Today the session answers
pings and nothing more.
#word("Upgrade") (`src/upgrade.zig`) execs the path the session was started
from, at a step boundary, after asking it `--version`: the session's dump
is committed to the state file as `upgrade-<pid>` (the build before wrote
`pardes-upgrade-<pid>.dump`, which this one still reads, for one release), and its pty masters,
terminals, stubs and daemon link stay open across the exec, named in a
record on the new command line (`--upgraded=`) ahead of the arguments it
was started with. The pid stays, so the shells stay its children; the new
process loads the dump with every view and takes the fds back. One that
cannot load it execs the old binary again (kept open from
`/proc/self/exe`), or with none starts afresh with the shells adopted and
the dump left for a Restore. The daemon re-execs the same way, its
registry on its command line. `python3 -B test/daemon.py --upgrade <old>
<new>` (and `--upgrade --gui`) is the end-to-end test.
= 9P
The wire format, the client and server connections, the file-server engine
and the Unix and TCP transports are the `cloud9` package
(`git.sr.ht/~gbrls/cloud9`), pinned in `build.zig.zon` and fetched into
`zig-pkg/`. Re-pin with
`zig fetch --save=cloud9 git+https://git.sr.ht/~gbrls/cloud9#<commit>`, or
use `.cloud9 = .{ .path = "../cloud9" }` while editing both. cloud9's own
`zig build test`, `transport-test`, `fuzz` and `differential` cover the
shared code.
- The engine (`fs.Server`) owns fids, permissions, directory reads,
flushes and what a hangup releases; it allocates nothing and makes no OS
calls. The control tree in `src/ninep/` is its backend: `tree.zig`
(nodes, dispatch), `pane.zig`, `ctl.zig`, `cols.zig`, `addr.zig`,
`pty.zig`, `events.zig` (event and log), `screen.zig`, `sources.zig`.
`src/9p.zig` names the editor's engine settings: msize 65536, 256 fids,
128 held reads a connection, names up to 255 bytes.
- A read, write, open, clunk, remove or truncating wstat can park; a walk,
attach, stat, create or rename cannot. Every Rread is clamped to the
count and the msize; Tflush answers the original request first.
- Unix and TCP listeners run on cloud9's `serve.Runner`: an accept task per
listener, a reader and a writer task per connection, 16 connections.
- A body write takes only whole UTF-8 sequences and answers a short count
for the rest. Consecutive writes from one open at the end of the text are
held and applied as one edit, flushed by any other request, the open's
release, 64 MiB, or a 20 ms pause in the editor's step.
== Writes through a mount <writes-through-a-mount>
A mount cuts a big write into pieces of at most one message (msize 65536,
less the header), anywhere, and each command line runs once its newline
comes; nothing is read into a write's size. A last line with no newline
(`printf Save > exec`) runs when the file is closed, as does an #word("Edit")
block never ended, and its failure is then only in the log, as its `err`
record: the close reports no error, and the write that sent it had
already succeeded. So a script that needs a line's result ends it with a
newline. A line over 1 MiB is refused once, and the rest of it, through
its newline, is dropped.
== Listeners <listeners>
`--9p-tcp='tcp!127.0.0.1!5640'` adds TCP. Addresses are numeric IPv4 or IPv6; port
0 picks one; #file("/listeners") reads them back. Every connection has full
session access, #file("/os") included, and TCP is unencrypted: use
loopback. Unix and TCP share 16 connection slots; a 17th client's Tversion
gets `too many connections` (and the log
`err - 9p: too many connections (N turned away)`).
`$XDG_RUNTIME_DIR/9p` is the machine's `/srv`: servers post themselves
there by name, and `9ns --mntgen` mounts the whole registry. A pardes whose
socket is in the runtime directory posts
`$XDG_RUNTIME_DIR/9p/pardes/<name>`, a symlink to its socket, and unposts
it on a clean stop if it is still its own; one whose socket fell back to
`~/.local/state/pardes` posts nothing. Posting first sweeps the group: a
symlink whose socket refuses a connect is removed with its socket. pardes
binds its own socket rather than going through `cloud9.post`, which takes
only flat names. A reader of the registry must `stat` through the symlink.
== Kernel mounts
For Linux v9fs use `version=9p2000,cache=none,access=any`, `trans=unix`
(or `trans=tcp` with `port=`), `uname`, `dfltuid` and `dfltgid` for the
local user, and an empty `aname`. Linux follows `O_TRUNC` with a `Twstat`
of zero length and an `mtime` hint; pardes takes the truncation and drops
the hint.
= Other notes
`docs/` keeps the platform and design notes beside this book:
`macos.md` (the macOS shell; `web.md` says what the browser shell was),
`render-pipeline.md` (a frame, and what the frontends draw on their own), `helix-keys.md` and `selections.md` (helix parity),
`lsp-evaluation.md`, `ui-review.md`, `divergences.md` (bookmarks off
`main`) and `open-questions.md`. `next-steps.txt` is a wishlist and
`transactions.txt` records one open structural gap against helix.
|