diff options
| -rw-r--r-- | docs/next-tests.md | 39 | ||||
| -rw-r--r-- | docs/typ/building.typ | 54 |
2 files changed, 61 insertions, 32 deletions
diff --git a/docs/next-tests.md b/docs/next-tests.md index 054958b5..838202eb 100644 --- a/docs/next-tests.md +++ b/docs/next-tests.md @@ -22,8 +22,8 @@ agent's survey. The `e2e` line (kbdin, mousein) is its first step. `focused`), plus `signal`, the exit code, pty bytes and captured frames by mode. It is mostly test/snapshot.zig's launching and 9P client moved out: - - **the `.snap` scripts** become one user of it (the interpreter and the - goldens stay in snapshot.zig); + - **the `.snap` scripts** became its tty cases (2026-11-09, below: The + suites), their goldens kept in test/snapshots/; - **process tests (T2)** are plain Zig tests in test/process/<label>.zig, under a `process-test` step; - **the window's tests (T4)** are `Process` in its window mode. @@ -40,8 +40,8 @@ agent's survey. The `e2e` line (kbdin, mousein) is its first step. Ported (2026-11-04, a fan-out of three): - **fs.py's 38 cases:** 36 process tests in test/process/, a few at T1 in test/behaviour/. - - **monkey9p.py:** test/monkey9p.zig, at T1 (`monkey-session`, its smoke - run part of behaviour-test). + - **monkey9p.py:** test/monkey9p.zig, at T1 (the T1 runner: its smoke + run part of behaviour-test, `behaviour-test -- --seed N` the rest). - **The build steps fs-test, fs-discovery-test, monkey-9p, monkey-9p-smoke and agent-session-test are gone.** - **fs.py is kept** as the helpers the developer tools import (session, @@ -112,7 +112,7 @@ agent's survey. The `e2e` line (kbdin, mousein) is its first step. |---|---|---| | T1 | the core through `serveFs`, in process | most behaviour; each case also through a replica that must agree | | T2 | a real `pardes` over its socket, a Zig runner | daemon, attach, mounts, shells, flags | -| T3 | terminal bytes, the `.snap` scripts | tty rendering and keys | +| T3 | terminal bytes: process-test's tty cases, a pardes in a pty of its own (test/process/tty_*.zig) | tty rendering and keys | | T4 | the window: `pardes-gui` hidden (`PARDES_TEST`, offscreen), its input through the same 9P files, frames captured once its status file says idle | what only pixels show | | perf | T1 or T4 gestures, timed | latency gates against baselines | @@ -122,11 +122,9 @@ images, fonts, edges). Built (2026-11-04): test/process/window.zig's monkey (a smoke run in process-test, all 40 Motion x Lift x Bloom with `zig build monkey-gui -Dplatform=gui`) drives the window's own input (Detached.typeKeys and mouse: its pty and OSC 777) and checks it settles, -alive, with no crash record; its goldens (`zig build gui-golden --Dplatform=gui`, PARDES_GUI_GOLDEN=update to rewrite) hash 17 scenes' -frames from Detached.capture. The old notes: test/gui_golden.py's 17 frame hashes -(test/gui-goldens.txt) are an exact gate for one machine, never a CI one, -and move to the Zig runner with T2. gui.zig's own tests: 19 M, 17 B, 47 U. +alive, with no crash record. Its goldens (15 scenes' frame hashes, exact +for one machine) went on 2026-11-09: what they caught, gui.zig's unit +tests and the tty cases catch (The suites, below). gui.zig's own tests: 19 M, 17 B, 47 U. Performance: test/perf.zig times gestures through `update` and `render` with a few internal reads (12), against test/perf-baseline-*.json (the @@ -138,6 +136,27 @@ component stays a U-level bench of that component. U tests (a stable library at its own pub functions with no core: regexp, diff, sam addresses, walk, wire, Text, modal's parse) stay as they are. +## The suites (2026-11-09, step 11 of the clean-room plan) + +Sixteen harnesses folded into the kept set: T1, hxdiff, one process suite, +one soak. Each fold was checked by a break of the code its cases guard, +failing the kept case as it failed the dropped one. + +| was | is | +|---|---| +| `snap`: 103 `.snap` scripts through pardes-snap (test/snapshot.zig) | the tty's own process cases, test/process/tty_{boot,editing,layout,themes,images,terminals}.zig, each a script's steps as `Tty` calls (test/tty.zig), its golden in test/snapshots/ unchanged; `process-test -- --update <filter>` rewrites one | +| `snap-driver-test` (the runner's retry and report) | process_test.zig's pool: a child a case, a core's worth at once, a failure run again alone with the wider settle probe | +| `monkey` (random snapshot scripts) | `behaviour-test -- --monkey <seeds> [--steps M]`: the same input mix at an in-process session (/kbdin, /mousein, Session.resize), a panic or a hang a failure, shrunk | +| `monkey-test` (did not compile) | gone: the generator is test/monkey9p.zig's genInput | +| `replay` (every script's and 20 seeds' tapes) | `behaviour-test -- --replay` (each seed's sessions taped in memory and replayed: three replicas and the dump) and `process-test -- --replay [--replay-with=<pardes>]` (the tty cases' real sessions, their hosts' answers on the tape) | +| `monkey-session`, `monkey-session-smoke` | `behaviour-test` bare runs the smoke; `-- --seed N ...` the full 9P monkey | +| `hxparity` | `hxdiff -- --parity` | +| `gui-golden` (15 frame hashes, this machine's) | the window's unit tests (`unit-test -Dplatform=gui`), the image and kitty harnesses' window arms, the tty cases for what the core draws, `monkey-gui` | + +What stays as it is: the image, kitty and pdf harnesses (end to end +through real programs and the GPU capture), pane-test (pane, output and +namespace integration), replicas, the soak and test/daemon.py, perf. + ## Inventory (2026-11-03, a regex classifier: about ±15%) | class | tests | lines | diff --git a/docs/typ/building.typ b/docs/typ/building.typ index 41fadbb2..f78f83a9 100644 --- a/docs/typ/building.typ +++ b/docs/typ/building.typ @@ -77,6 +77,7 @@ must share with its session; `pardes --help` lists every flag. zig build unit-test module and shell unit tests (the perf gate runs with it) zig build test-build compile the unit-test programs without running them zig build core-test core tests without native shell tests +zig build shell-test the native shell's own unit tests, without the perf gate zig build pane-test pane, output and namespace integration tests zig build kitty-test kitty graphics conformance of terminal panes, at the core zig build kitty-harness kitty graphics through a pane end to end (a fake kitty @@ -84,18 +85,17 @@ zig build kitty-harness kitty graphics through a pane end to end (a fake kitt kitten icat and chafa too when installed) zig build syntax-test tree-sitter tests without building the editor zig build syntax deterministic per-byte highlighting snapshots -zig build behaviour-test behaviour through the 9P tree, in process (test/behaviour/) -zig build process-test real sessions over their sockets: mounts, flags, shells (test/process/); - -Dplatform=gui runs each case on the hidden window instead +zig build behaviour-test behaviour through the 9P tree, in process (test/behaviour/), + and the T1 runner's short run of random 9P requests +zig build process-test real sessions over their sockets: mounts, flags, shells (test/process/), + and the tty's own cases, each a pardes in a pty against its golden + (test/process/tty_*.zig, test/snapshots/); -Dplatform=gui runs the + socket cases on the hidden window instead zig build monkey-gui random input to the hidden window, every effect setting (-Dplatform=gui) -zig build gui-golden the hidden window's scenes against this machine's goldens (-Dplatform=gui) -zig build monkey-session random in-process 9P requests checking the documented rules zig build 9p-test freestanding protocol tests zig build 9p-io-test native 9P transports and client -zig build snap scripted input traces against frozen golden grids -zig build monkey random snapshot scripts hunting panics (not a gate) zig build hxdiff differential suite against helix's own behaviour -zig build hxparity file-pane vs pty-pane editing parity + (-- --parity: each case in a file pane and a pty pane, which must agree) zig build hxgolf every helix-golf example, step by step, against helix zig build perf-gate a gesture on the 50k-line file over 3x its baseline fails zig build mupdf-check the PDF addon's MuPDF: compile, link, render and search docs/design.pdf @@ -104,18 +104,27 @@ zig build cheatsheet render the cheatsheet PDF (needs typst) - `-Dtest-filter=<text>` applies to every unit-test binary. A failed test's printed trace can be stale: run it alone with the filter. -- `snap -- --record=DIR` writes snapshots without touching goldens, - `snap -- --update` replaces goldens (re-record one script by name, after - reading its diff, never all), `snap -- --no-retry` makes the first - failure decisive. Snapshot scripts can use `snap9p` to capture core cells - through 9P. -- `zig build monkey -Dplatform=tui -- <seeds> [steps] [--from=N] [--out=DIR] [--keep]` writes one random script per seed and keeps any - that panics; a seed always makes the same script. Each crash found gets a - fix and a regression script in `test/snapshots/`. +- `process-test -- <filter>` runs the cases whose name holds it (`tty + editing:` a family of the tty's, `tty editing: grep` one). + `process-test -- --update <filter>` writes a tty case's captures to its + golden: one case at a time, then a plain run, and read the golden's diff + for moved shell output (a flake) before keeping it. A tty case that fails + runs again alone with a wider settle probe before it counts; what it drew + is kept in `/tmp/pardes-snap/<name>/drew.golden`. +- `behaviour-test -- --monkey <seeds> [--steps M]` types random keys, leader + bursts, mouse presses, drags, wheel turns and tiny resizes at an in-process + session, a seed a child, hunting panics and hangs; a seed always makes the + same input, and a failure is shrunk to the steps that make it. `-- --seed N` + runs the random 9P requests with the documented rules checked after each. + Each crash found gets a fix and a regression case. +- `behaviour-test -- --replay` records each monkey seed's sessions as tapes + and replays them (three replicas and the dump); `process-test -- --replay + [--replay-with=<pardes-gui>]` records and replays the tty cases' real + sessions, their shells' and files' answers on the tape. - The tutor's practice blocks (`# keys:`, `# before`, `# after` in `src/tutor.txt`) are typed by hand; nothing runs them. What pins that behaviour is `hxdiff`, which replays `test/hxcases` against goldens - recorded from a real helix, and `hxparity`, which runs each case in a + recorded from a real helix, and its `--parity`, which runs each case in a file pane and in a shell and demands they agree. - `zig build hxdiff -- --strict cases.jsonl reference.jsonl [waivers.jsonl]` runs custom differential cases. `docs/helix-keys.md` tracks helix parity key by key and names the reference helix build; @@ -132,9 +141,10 @@ zig build cheatsheet render the cheatsheet PDF (needs typst) == Release gates A release passes all of these first: `unit-test` with `-Dplatform=tui` and -with `-Dplatform=gui`, `core-test`, `behaviour-test`, `process-test`, `snap`, the GUI goldens -(`zig build gui-golden -Dplatform=gui`, a hidden window; `PARDES_GUI_GOLDEN=update` rewrites them), -and the ReleaseSafe tui and ReleaseFast gui builds. The committed +with `-Dplatform=gui`, `core-test`, `behaviour-test`, `process-test` (on both: +its tty cases on the tui, the hidden window on the gui), `hxdiff` and its +`--parity`, the replicas harness and the soak (`test/soak.py`), and the +ReleaseSafe tui and ReleaseFast gui builds. The committed `docs/typ/cheatsheet-a4.pdf` is rendered again when the docs change. = Sending patches <patches> @@ -185,7 +195,7 @@ thing beside them (`edit.zig`, `normal.zig`, `look.zig`, `exec.zig`, per-user daemon and a terminal's stub, `src/wire.zig` the input wire a remote client will speak, `src/lsp/` the language-server client; `test/` holds harnesses, goldens and helix cases, -`build/` the snapshot suite's build step. +`build/` the process tests' build steps. == Threads @@ -224,7 +234,7 @@ the terminal back and lets it go; the stub ends (exit 0) and the session runs on with nothing drawn, until `pardes --attach` hands it another terminal. A window's #word("Detach") closes the window and keeps the process; `pardes-gui --attach` opens one again. `PARDES_DAEMON=off` (the -snapshot harness) runs a session as before the daemon: in the terminal's +process tests) runs a session as before the daemon: in the terminal's own process, ended by its hangup. A replica (a client running its own copy of the core, phase 5 of |
