summaryrefslogtreecommitdiff
path: root/docs/next-tests.md
blob: 054958b552f3142786ebfd2f79d0998f88344474 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
# Behavioural tests: free to rewrite the code

> I think we need a larger project to turn most of the tests into
> behavioural ones so that we are free to rewrite the code.

Planned 2026-11-03 by the user and the coordinator, from an inventory
agent's survey. The `e2e` line (kbdin, mousein) is its first step.

## Decided by the user

- **A test goes in and comes out through public surfaces only:** the 9P
  tree (pane files, `ctl`, `look`/`exec`, `log`, `/screen`, `kbdin`,
  `mousein`), the process (flags, daemon, attach, signals, terminal bytes),
  and files on disk and effects. A rewrite that keeps those keeps its tests.
- **T1, in-process 9P, is the main tier.** The core is driven only through
  `serveFs` (src/ninep/testing.zig's `call`, `rd`, `wr`): as behavioural
  as a client on a socket, and milliseconds a case.
- **One process runner, not Python (the user, 2026-11-04):** test/process.zig's
  `Process` starts `pardes` detached (over its socket), in a pty, or as the
  hidden window, in a throwaway root, and talks 9P to it with Session's
  method names (`read`, `write`, `refused`, `keys`, `click`, `screen`,
  `focused`), plus `signal`, the exit code, pty bytes and captured frames
  by mode. It is mostly test/snapshot.zig's launching and 9P client moved
  out:
  - **the `.snap` scripts** become one user of it (the interpreter and the
    goldens stay in snapshot.zig);
  - **process tests (T2)** are plain Zig tests in test/process/<label>.zig,
    under a `process-test` step;
  - **the window's tests (T4)** are `Process` in its window mode.

  Built (2026-11-04):
  - **test/process.zig:** `Files` (a running pardes's tree over its socket,
    dialled by pid or path) and `Detached` (a session in a throwaway root,
    PARDES_DAEMON=off, so nothing outlives it).
  - **The pty mode** is e2e_harness's Harness with `process.of`.
  - **The `process-test` step** runs test/process_test.zig over
    test/process/*.zig. Its first case is fs.py's keys_and_mouse.
  - **snap9p** reads /screen through `Files`.

  Ported (2026-11-04, a fan-out of three):
  - **fs.py's 38 cases:** 36 process tests in test/process/, a few at T1
    in test/behaviour/.
  - **monkey9p.py:** test/monkey9p.zig, at T1 (`monkey-session`, its smoke
    run part of behaviour-test).
  - **The build steps fs-test, fs-discovery-test, monkey-9p, monkey-9p-smoke
    and agent-session-test are gone.**
  - **fs.py is kept** as the helpers the developer tools import (session,
    new_pane, execute).
  - **agent_session_test.py is kept,** run by hand: it tests the
    agent_session.py tool.
  - **No Python left in the build:** gui_monkey.py and gui_golden.py are
    test/process/window.zig's two cases (T4, below).

  Each backend's own (2026-11-04, the user: an effect's test is the
  backend's):
  - **`zig build process-test -Dplatform=tui` drives `pardes`, `-Dplatform=gui`
    drives `pardes-gui`,** hidden (PARDES_TEST, SDL_VIDEO_DRIVER=offscreen,
    no display of the user's) on a pty of its own and a 160x50 grid.
    `Detached.frame()` waits for the window to say it is idle and reads its
    capture.
  - **A case about one frontend's own behaviour says `process.only(.tui)`.**
    On the window, 33 cases pass and 5 skip.
  - **Every process the runner starts is kept off the user's display**
    (Child.spawn: WAYLAND_DISPLAY and DISPLAY unset, SDL_VIDEO_DRIVER=offscreen),
    including a second pardes-gui a case starts. A window's grid is its
    case's (`Detached.Options.grid`), which sizes its terminal too: a test
    window takes its terminal's size.
  - **Fixed, found by running each case on the window:**
    - **A new terminal's first pty/run waited for ever:** after a resize,
      which a window always has, its greeting `ls` was typed at the same
      first prompt (src/ninep/pty.zig). The run is now the shell's first
      command.
    - **The canary's Newcol was refused:** the runner's 80x24 terminal had
      shrunk the window.
  - **Flaky, seen once in about six full tui runs:** flags' "a tty editor
    killed by SIGTERM or SIGHUP puts its terminal back and its socket goes".
    It passes alone 3 of 3.

  Before the port, the plan was: the Python suites in the build (fs.py, monkey9p.py, gui_monkey.py,
  agent_session_test.py) move onto it or onto Session, then
  leave the build. The developer tools (ui_review, record_demos,
  appearance_gallery…) stay Python, as examples. A real process performs
  its effects, so tests that need one can land at T2.
- **Layout (from a ponytail review, 2026-11-03):**
  - **test/session.zig is the T1 `Session`:** a 9P client's walk, open,
    read or write, clunk, by path, with the core opaque.
  - **test/behaviour/<label>.zig holds the tests,** a file a section of
    the book named by its label (`other-files` is other_files.zig), each
    test's name its label first.
  - **The `behaviour-test` step:** its module imports `session` alone, so
    a behavioural test that reaches for `pardes` does not compile. That
    boundary is the lint.
  - **Expectations are inline;** goldens stay where they are.
- **A surface is added when a test being migrated needs one,** as a word in
  a file that exists where it can be (a pane ctl's read already ends
  `collapsed fit:… tint:…`; rio's wctl read is its window's rectangle).
- **A migration is one commit:** the behavioural test in, the M test it
  replaces out, and the break it was checked against named. Phases 2 and 4
  are one.
- **A replica's agreement is checked opt-in** (a Session option, to come),
  for kbdin and mousein input, which a replica takes as events.
- **Later, not now:**
  - The reference as the spec: one test per documented behaviour, named by
    its section (docs/typ/reference.typ for the 9P surface, guide.typ and
    the cheatsheet for keys and mouse). A sample of 20 reference rows found
    18 in some test; `restoredcol` and the EROFS mapping in none.
  - Phase 2 as a multi-agent workflow, an agent per area and a reviewer.

## Tiers

| tier | drives | for |
|---|---|---|
| T1 | the core through `serveFs`, in process | most behaviour; each case also through a replica that must agree |
| T2 | a real `pardes` over its socket, a Zig runner | daemon, attach, mounts, shells, flags |
| T3 | terminal bytes, the `.snap` scripts | tty rendering and keys |
| T4 | the window: `pardes-gui` hidden (`PARDES_TEST`, offscreen), its input through the same 9P files, frames captured once its status file says idle | what only pixels show |
| perf | T1 or T4 gestures, timed | latency gates against baselines |

The window (T4): its core is the one T1 drives, so a behaviour the cells
show is tested at T1; T4 is for what only pixels show (scene effects,
images, fonts, edges). Built (2026-11-04): test/process/window.zig's
monkey (a smoke run in process-test, all 40 Motion x Lift x Bloom with
`zig build monkey-gui -Dplatform=gui`) drives the window's own input
(Detached.typeKeys and mouse: its pty and OSC 777) and checks it settles,
alive, with no crash record; its goldens (`zig build gui-golden
-Dplatform=gui`, PARDES_GUI_GOLDEN=update to rewrite) hash 17 scenes'
frames from Detached.capture. The old notes: test/gui_golden.py's 17 frame hashes
(test/gui-goldens.txt) are an exact gate for one machine, never a CI one,
and move to the Zig runner with T2. gui.zig's own tests: 19 M, 17 B, 47 U.

Performance: test/perf.zig times gestures through `update` and `render`
with a few internal reads (12), against test/perf-baseline-*.json (the
perf-gate step); the PDF, rollback, locations and fs benches time
components. A gesture is to be written as T1 or T4 input (kbdin, mousein),
so a rewrite is timed by the same case it is tested by; a bench of one
component stays a U-level bench of that component.

U tests (a stable library at its own pub functions with no core: regexp,
diff, sam addresses, walk, wire, Text, modal's parse) stay as they are.

## Inventory (2026-11-03, a regex classifier: about ±15%)

| class | tests | lines |
|---|---|---|
| B, behavioural already | 252 | 6.3k |
| U, a library's unit | 682 | 15.4k |
| M, to migrate | 722 | 26.7k |
| X, pinned to the implementation | 30 | 1.6k |

Most M: pardes.zig 116, test/panes.zig 104 (of 124), ninep/ctl.zig 82
(in through 9P, checked through `p.panes`), test/output.zig 34,
ninep/pane.zig 28, test/pdf.zig 25.

What the M tests reach into, and the public surface for it:

| internal | tests | surface |
|---|---|---|
| `p.active` | 242 | `/focus` |
| a file's text | 173 | `pane/n/body` |
| `body.cur_*`, `sel` | 126 | `dot`, `sel`, `addr` |
| path, dirty | 121 | `name`, `dirty`, `/index` |
| click cells from `p.rects` | 158 | **missing**: pane geometry |
| `body.mode` | 90 | **missing**: a mode file |
| Alt, Shift keys | 18 | **missing**: modifiers in `kbdin` |
| notices, places | 28 | **missing** |
| `.pdf.?` | 25 | **missing**: page, viewport |
| terminal cursor, overlay | 60 | partial |

## Progress

Two fan-outs (2026-11-03 and 2026-11-04; a migrator per area, then a
reviewer running breaks of its own): 211 tests moved, 216 in
test/behaviour/, plus the first process test.

| area | moved | M left (most excluded: effects, T2, images, LSP) | X kept |
|---|---|---|---|
| src/ninep/ctl.zig | 9 + 29 + 62 | 37 | 14 |
| src/ninep/pane.zig, test/panes.zig | 30 + 27 | 16 | 19 |
| src/pardes.zig, test/output.zig, test/pdf.zig | 28 + 14 + 21 | 13 | 51 |

What the M left need, most first:
- **Effects performed:** Session counts a save and drops it; a save
  record, /recent after a save, and a terminal's pty bytes come only once
  a host performs the effect. A Session option that performs saves (and
  keeps pty writes) unlocks a group in each area.
- **A real host (T2):** terminals' output and shells, command panes
  finishing, Term and pty/ctl exec, a missing interpreter.
- **Image panes:** they need the decoder worker Session does not start.
- **LSP:** a fake language server.
- **Not reached yet:** jumps and Back/Forward, Zerox, Get, Restore,
  DumpConfig, ThemeFile, and name edge cases.

Traps the migrators found in Session, each now in its tests' code:
- **It never ticks the core,** so a record a step makes (a pane's `new`)
  shows in /log only at the next request.
- **A last line with no newline is held until the clunk;** its failure
  goes to /log only, and the write reports success. End ctl lines with
  `\n`.
- **`write` appends (`>>`); `replace` truncates first (`>`).**
- **`read` sets no `said`.**

## Phases
## Phases

0. Inventory. Done, above.
1. The T1 `Session` API (open, type, click, read pane files and screen
   text, expect; through a replica too), the missing surfaces in the
   table's order, and a lint that fails a behavioural test touching
   `p.panes[`, `.file.?`, `p.rects` and the like.
2. Migrate, in the order the code is to be rewritten: ninep/ctl.zig (its
   checks only), test/pdf.zig (step 3 of the pane-kinds plan),
   pardes.zig and test/panes.zig, replica, the gui and tui shells,
   host_io. A migrated test must fail on a build broken on purpose.
3. Gaps: a test for each documented behaviour with none.
4. Delete the M and X tests the new ones cover.