summaryrefslogtreecommitdiff
path: root/docs/pardes-rendering.md
blob: 0c89724d915eef8fd87167d1cb23470c8b7426f0 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
# Pardes and the shared GPU executor

Pardes already draws its interface with SDL_GPU. Its addon ABI originally
accepted cells and CPU RGBA rasters only. The optional GPU path lets an addon
submit a scene, render it into a texture on the frontend's device, and composite
that texture into the pane without reading pixels back to the CPU.

## What is shared now

`src/gpu/Executor.zig` is the executor previously in `src/host/gpu.zig`. It
owns shaders, pipelines, buffers and offscreen targets, but no window, SDL
renderer, sketch state or image manager. Both sketchbook and pardes use it.

- `init(allocator, device)` borrows an existing SDL GPU device.
- `record(command_buffer, pass)` records a pass into the caller's command
  buffer. The caller submits it. Shader and pipeline creation happens before
  drawing; errors propagate to the caller.
- `run(pass)` records and submits a pass itself; sketchbook uses this convenience.
- `output(image)` borrows the target texture and dimensions. It is valid until
  resize, `forget(image)`, or `deinit()`. Release external wrappers before those
  operations. The executor owns the texture.
- `postProcess(command_buffer, source, target, post)` records a fullscreen
  second pass sampling `source` into a distinct `target`, on the same command
  buffer. The executor owns and reuses its nearest-neighbour sampler.
- Upload, compute, asynchronous buffer download and pipeline caching remain
  available. Their submission policy is unchanged.

Sketchbook's host wraps the output in an SDL_Renderer image. Pardes samples it
directly in its existing image compositor, using premultiplied blending.

Embedders request the `gpu-only` build option, take the `abi`, `gpu` and
`gpu_scene` modules, and supply the `gpu` module's `c` import from their existing
SDL translation. This avoids building a second host or creating another device.
Pardes currently uses a sibling checkout dependency (`../sketchbook`); pin a
published revision when these two changes are released together.

## Keep GPU support optional

Pardes ABI 3.1 appends one optional submission callback and its context to
`pa_paint`. The old fields and required callbacks are unchanged. A GPU addon
checks the host ABI before reading the extension; a null callback means it is
unavailable. The Zig helper `Paint.gpu(bytes)` performs both checks.

The callback copies the scene during the call, so stack buffers are safe. An
addon still paints cells or a raster. Accepting the scene only means the host
can carry it: a terminal, the native macOS frontend, or a frontend that refuses
the shader displays the fallback. Neither ordinary addons nor the core need
SDL or a GPU. The callback transports opaque display data; graphics types do
not enter the main C ABI.

`src/gpu/scene.zig` is the separate, versioned scene format and Zig codec. SGPU
version 1 carries one self-contained render pass: SPIR-V vertex and fragment
shaders, vertex layouts, draws, uniforms, blending, clipping, MSAA and stencil.
It contains no pointers or device handles. `Pass.image` is `.none` on the wire;
the frontend supplies its own target. A clear colour is required, so a frame
can be skipped or a new replica attached without replaying old GPU work.

The encoding is `SGPU`, a little-endian u32 version, then the current
`abi.gpu.Pass` fields in declaration order, recursively. Slice counts and enums
are u32; scalars use little endian; optionals and booleans use a 0/1 byte. No
native padding is serialized. Changing these fields requires a format version
change. The decoder bounds allocation, dimensions, draws, shader metadata,
vertex ranges and uniform sizes before the executor sees the pass. This is
structural validation, not a shader sandbox: use shaders from a trusted session.

SGPU version 2 keeps the v1 pass encoding and appends one `PostProcess`:
vertex shader, fragment shader, and fragment uniform bytes. The vertex stage
has no uniforms or inputs; the fragment stage has exactly one implicit sampler
at set 2, binding 0, reading the completed first pass. `decodeFrame` accepts both
versions; `encode` still writes v1 and `encodePostProcess` writes v2. The same
frame allocation, shader and uniform limits apply. This deliberately supports
one postprocessing pass, without adding a render graph or C ABI fields.

The current limits are 4 MiB per scene, 4096 pixels per side, 64 pipelines,
4096 draws, and 16 KiB per stage's uniform block. Version 1 deliberately has no
storage-buffer handles or compute commands. Local sketchbook applications keep
the existing richer API; persistent remote GPU resources need their own
lifetime and recovery protocol before exposing them to addons.

Pardes carries scene bytes in its authoritative pane-frame stream, including
changes where the cells are identical. Replicas need no addon code. Unchanged
scenes are not resent or rerendered. Shader bytes are currently included in each
changed scene; this is suitable for the snake example, not a large streaming
mesh. Closing a pane releases its frontend target. Unsupported frontends ignore
the scene and display the fallback.

## First consumer: SnakeShader

In the pardes checkout:

```
zig build snake-shader -Dplatform=tui -Dpdf=false -Doptimize=ReleaseSafe
zig build -Dplatform=gui -Dpdf=false -Doptimize=ReleaseSafe
```

In the rebuilt GUI session, execute:

```
Addon load /absolute/path/to/pardes/zig-out/lib/libsnake_shader.so
SnakeShader
```

The build requires `glslc`. The example sends board occupancy and a small set
of parameters to a fragment shader. `h j k l` and arrows steer, `r` restarts;
losing focus pauses movement. A two-turn buffer supports quick corners and
corrections, with 60 ms of grace for late input and collision recovery. The
board fills its pane with whole square cells and resizes dynamically. Shrinking
translates the snake inward, shortening only a tail that cannot fit; the score
is preserved. The last active view owns the board dimensions, so an idle replica
does not resize another player's game. On an older host it remains a text snake.
Building a new binary does not upgrade a running session.

`snake.post.frag` samples the board in a separate GPU pass and adds a three-column
RGB phosphor mask and four-row scanlines, a nine-tap bloom and filmic colour
mapping. Hovering adds a local glow and rim to the cell. A 3.5% food pulse and
0.6% phosphor shimmer keep a paused or finished game subtly alive. Visible panes
animate at 20 Hz; hidden panes stop requesting animation ticks. Its intermediate and final textures
are reused and resized together; closing the pane releases both. No CPU readback
or extra command-buffer submission is needed. A host without SGPU v2 displays
the text fallback. Use `test/gpu_snake.py BINARY ADDON OUTPUT --crt` to check the
actual RGB triads and scanline modulation, as well as the game colours.
The target covers the pane with an opaque background and centres the whole-cell
grid in any remaining margin, so fallback text cannot leak around a successful
render. These effects require no new scene commands or C ABI fields.

`test/gpu_snake_interaction.py GUI V1_ADDON CURRENT_ADDON OUTPUT` checks live
record migration, a second reload, column resize, hover/leave, and pixel changes
while movement is paused, preserving unsaved text and shell identities. The addon
also tests buffered turns, grace deadlines, geometry, state loading and optional
GPU fallback. Its version 2 record accepts version 1 saved games.

## Next steps: move pardes's low-level renderer

1. **Texture bindings and instancing.** Extend the shared pipeline/draw model
   with sampled textures, samplers and instance layouts/counts. Pardes's glyph
   atlas and cell renderer need these before they can use the executor. Add
   pixel tests for glyph masks, clipping and straight/premultiplied alpha.
2. **Extract common resource operations.** Move texture creation, staged full
   and dirty-row uploads, transfer buffers and sampler caching out of
   `pardes/src/gui/gui.zig`. Keep ownership explicit and use the caller's
   command buffer. Test resize, partial-upload failure, window close and device
   teardown against both hosts.
3. **Move the compositor a pass at a time.** Start with native image quads,
   then decorations and cell instances. Preserve ordering of pane content,
   tags, notices and selection, clipping and window opacity. Pardes continues
   to produce layout, shaped text and draw data; those are editor semantics.
4. **Persistent addon resources, when a consumer needs them.** Add
   content-addressed shader/mesh resources and explicit updates to the separate
   scene protocol. Define per-session/per-view ownership, quotas, invalidation,
   reconnect and missed-frame recovery before adding compute/storage buffers.
   Keep all of this behind the optional callback, not new mandatory addon ops.
5. **Other frontends.** Implement the scene executor on the native macOS path
   (shader translation/backend choice needs deciding), preserving fallbacks.
   Sketchbook's WebGPU implementation is another possible consumer; SGPU v1
   currently carries SPIR-V, not WGSL.
6. **Measure before widening the API.** Compare CPU record time, GPU time,
   uploads, submissions and memory for many panes, large geometry and repeated
   reload/resize. Batch compute/uploads into caller command buffers when
   measurements justify it. Add allocation limits across targets, rather than
   relying only on per-scene bounds, before treating this as a large-scene API.

Do not migrate text shaping, editor layout, input routing or session state into
the renderer. Do not add raw device pointers to pardes's addon ABI. The shared
layer should execute graphics work; the applications decide what to draw.

## Validation (2026-11-12)

The two builds used Zig 0.17.0, ReleaseSafe, the SDL frontend, PDF disabled,
and the same pre-existing pardes edits. The baseline was the checkout before
the GPU changes. Three interleaved runs per build scrolled the same 10,000-line
file at a 100×30 grid. Three further runs restarted the shader snake every
16 ms to change its uniforms. Startup's first 500 ms was excluded.

| Workload | Frames | Median frame work | p95 frame work | Frames over 16.7 ms |
| --- | ---: | ---: | ---: | ---: |
| Text scrolling, before | 458 | 7.627 ms | 16.068 ms | 16 |
| Text scrolling, after | 462 | 7.519 ms | 14.838 ms | 8 |
| Shader snake | 350 | 7.414 ms | 14.208 ms | 5 |

These are FrameStats update + core render + present times, **not GPU timer
queries or a frame-rate guarantee**. The offscreen driver cannot present to a
headless Vulkan surface here, so the test includes GPU readback and software
presentation/capture. Both builds use the same path. Run-to-run variation was
substantial; the result supports no measured scrolling regression, not a claim
of a speedup. Median core-render time was 77 µs before, 78.5 µs after, and
124 µs for the shader snake. Shader images were checked for the board's solid
green segments and orange food, and visually inspected.

Reproduce in pardes with `test/gpu_snake.py BINARY ADDON OUTPUT --text` for
scrolling, omitting `--text` for the shader. Each output directory keeps the
raw FrameStats TSV, screenshot, stderr and JSON summary. Use matching build
options, an otherwise idle machine, and interleave the baseline and changed
build; do not measure while compiling.

`test/gpu_upgrade.py OLD_GUI NEW_GUI OLD_SNAKE SHADER_SNAKE OUTPUT` checks
in-place upgrade, unsaved text, shell identity, saved game state, addon reload,
and a second GPU replica. Explicitly loaded addons must be loaded again after
an upgrade; their placeholder panes retain the saved state in the meantime.
The test exposed an existing missing addon generation field in pardes's dump
codec; saving it is necessary for a replica joining after an addon reload.

Pardes's full core suite passed 967 of 969 tests. The same two failures also
occurred in the baseline (965 of 967): the full-screen tty handoff test and
the terminal dump test's cleanup crash. They remain unresolved; the GPU
changes introduced no additional failures in this comparison. Addon fixtures,
old ABI 3.0 addon compatibility, shader pixels, and the isolated upgrade/reload/
replica checks passed. The live session then upgraded with its text contents
and shell process IDs preserved.

The sketchbook ReleaseSafe suite covers vector GPU/CPU pixel comparisons,
render targets, replay/reload, exports and scene playback. Its existing clock
test compared struct padding; it now compares fields, avoiding an unrelated
ReleaseSafe test failure. Its vector benchmark recorded the two small sample
scenes in 15–16 µs, with CPU parse+draw at 278–315 µs (the gate is 1 ms).

The CRT follow-up passed the scene codec tests, addon fixtures, Sketchbook's
ReleaseSafe suite, v1 rendering compatibility, and the isolated upgrade/reload/
column-resize/pane-close/replica test. Native captures verified RGB triads and
scanline contrast, including a narrow pane without fallback text leaking through.
Three interleaved runs of the final version measured median frame work of
10.409 ms before and 10.585 ms with CRT, with p95 at 16.671 and 17.512 ms
(328 frames each). This is a small measured extra cost, not a zero-overhead
claim; the same offscreen capture limitations above apply. The live session
was upgraded and pane 42 reloaded with its game state, text and shells preserved.

The game-feel follow-up passed seven addon tests, its updated scripted fixture,
GPU colour/CRT checks, and the interaction test for migration, reload, resizing,
hover/leave and animation while paused. The upgrade/reload/replica test also
passed. Migration exposed two existing host bugs: reload passed the replacement
library's version for the old record, and restored records retained the old
version after successful migration. Pardes now preserves the input record version
until loading succeeds and saves subsequent records with the current version;
a focused core regression test covers both paths. No ABI fields were added.

Three interleaved before/after runs compared the previous CRT addon with the
resizable board, bloom, colour mapping, hover glow and idle animation:

| Workload | Frames | Median frame work | p95 frame work |
| --- | ---: | ---: | ---: |
| Previous CRT snake | 378 | 4.759 ms | 14.616 ms |
| New snake and effects | 369 | 5.271 ms | 15.313 ms |
| Text scrolling, before (one run) | 136 | 12.848 ms | 19.067 ms |
| Text scrolling, after (one run) | 133 | 13.275 ms | 19.027 ms |

The effects added about 0.5 ms to the pooled median in this capture. Individual
snake-run medians ranged from 4.6 to 12.7 ms, so system variation was substantial;
the scrolling pair is a smoke check, not evidence of zero regression. These are
the same offscreen/readback measurements described above, not GPU timer queries.
Visible paused games now request animation frames instead of remaining idle.
The full core and Sketchbook suites were not repeated for this follow-up.
The live session was upgraded and pane 42 migrated to version 2, retaining its
score, length, step count, focus, file contents and shell identities; the open
source-directory listing refreshed from disk.