diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/pardes-rendering.md | 253 |
1 files changed, 253 insertions, 0 deletions
diff --git a/docs/pardes-rendering.md b/docs/pardes-rendering.md new file mode 100644 index 0000000..0c89724 --- /dev/null +++ b/docs/pardes-rendering.md @@ -0,0 +1,253 @@ +# Pardes and the shared GPU executor + +Pardes already draws its interface with SDL_GPU. Its addon ABI originally +accepted cells and CPU RGBA rasters only. The optional GPU path lets an addon +submit a scene, render it into a texture on the frontend's device, and composite +that texture into the pane without reading pixels back to the CPU. + +## What is shared now + +`src/gpu/Executor.zig` is the executor previously in `src/host/gpu.zig`. It +owns shaders, pipelines, buffers and offscreen targets, but no window, SDL +renderer, sketch state or image manager. Both sketchbook and pardes use it. + +- `init(allocator, device)` borrows an existing SDL GPU device. +- `record(command_buffer, pass)` records a pass into the caller's command + buffer. The caller submits it. Shader and pipeline creation happens before + drawing; errors propagate to the caller. +- `run(pass)` records and submits a pass itself; sketchbook uses this convenience. +- `output(image)` borrows the target texture and dimensions. It is valid until + resize, `forget(image)`, or `deinit()`. Release external wrappers before those + operations. The executor owns the texture. +- `postProcess(command_buffer, source, target, post)` records a fullscreen + second pass sampling `source` into a distinct `target`, on the same command + buffer. The executor owns and reuses its nearest-neighbour sampler. +- Upload, compute, asynchronous buffer download and pipeline caching remain + available. Their submission policy is unchanged. + +Sketchbook's host wraps the output in an SDL_Renderer image. Pardes samples it +directly in its existing image compositor, using premultiplied blending. + +Embedders request the `gpu-only` build option, take the `abi`, `gpu` and +`gpu_scene` modules, and supply the `gpu` module's `c` import from their existing +SDL translation. This avoids building a second host or creating another device. +Pardes currently uses a sibling checkout dependency (`../sketchbook`); pin a +published revision when these two changes are released together. + +## Keep GPU support optional + +Pardes ABI 3.1 appends one optional submission callback and its context to +`pa_paint`. The old fields and required callbacks are unchanged. A GPU addon +checks the host ABI before reading the extension; a null callback means it is +unavailable. The Zig helper `Paint.gpu(bytes)` performs both checks. + +The callback copies the scene during the call, so stack buffers are safe. An +addon still paints cells or a raster. Accepting the scene only means the host +can carry it: a terminal, the native macOS frontend, or a frontend that refuses +the shader displays the fallback. Neither ordinary addons nor the core need +SDL or a GPU. The callback transports opaque display data; graphics types do +not enter the main C ABI. + +`src/gpu/scene.zig` is the separate, versioned scene format and Zig codec. SGPU +version 1 carries one self-contained render pass: SPIR-V vertex and fragment +shaders, vertex layouts, draws, uniforms, blending, clipping, MSAA and stencil. +It contains no pointers or device handles. `Pass.image` is `.none` on the wire; +the frontend supplies its own target. A clear colour is required, so a frame +can be skipped or a new replica attached without replaying old GPU work. + +The encoding is `SGPU`, a little-endian u32 version, then the current +`abi.gpu.Pass` fields in declaration order, recursively. Slice counts and enums +are u32; scalars use little endian; optionals and booleans use a 0/1 byte. No +native padding is serialized. Changing these fields requires a format version +change. The decoder bounds allocation, dimensions, draws, shader metadata, +vertex ranges and uniform sizes before the executor sees the pass. This is +structural validation, not a shader sandbox: use shaders from a trusted session. + +SGPU version 2 keeps the v1 pass encoding and appends one `PostProcess`: +vertex shader, fragment shader, and fragment uniform bytes. The vertex stage +has no uniforms or inputs; the fragment stage has exactly one implicit sampler +at set 2, binding 0, reading the completed first pass. `decodeFrame` accepts both +versions; `encode` still writes v1 and `encodePostProcess` writes v2. The same +frame allocation, shader and uniform limits apply. This deliberately supports +one postprocessing pass, without adding a render graph or C ABI fields. + +The current limits are 4 MiB per scene, 4096 pixels per side, 64 pipelines, +4096 draws, and 16 KiB per stage's uniform block. Version 1 deliberately has no +storage-buffer handles or compute commands. Local sketchbook applications keep +the existing richer API; persistent remote GPU resources need their own +lifetime and recovery protocol before exposing them to addons. + +Pardes carries scene bytes in its authoritative pane-frame stream, including +changes where the cells are identical. Replicas need no addon code. Unchanged +scenes are not resent or rerendered. Shader bytes are currently included in each +changed scene; this is suitable for the snake example, not a large streaming +mesh. Closing a pane releases its frontend target. Unsupported frontends ignore +the scene and display the fallback. + +## First consumer: SnakeShader + +In the pardes checkout: + +``` +zig build snake-shader -Dplatform=tui -Dpdf=false -Doptimize=ReleaseSafe +zig build -Dplatform=gui -Dpdf=false -Doptimize=ReleaseSafe +``` + +In the rebuilt GUI session, execute: + +``` +Addon load /absolute/path/to/pardes/zig-out/lib/libsnake_shader.so +SnakeShader +``` + +The build requires `glslc`. The example sends board occupancy and a small set +of parameters to a fragment shader. `h j k l` and arrows steer, `r` restarts; +losing focus pauses movement. A two-turn buffer supports quick corners and +corrections, with 60 ms of grace for late input and collision recovery. The +board fills its pane with whole square cells and resizes dynamically. Shrinking +translates the snake inward, shortening only a tail that cannot fit; the score +is preserved. The last active view owns the board dimensions, so an idle replica +does not resize another player's game. On an older host it remains a text snake. +Building a new binary does not upgrade a running session. + +`snake.post.frag` samples the board in a separate GPU pass and adds a three-column +RGB phosphor mask and four-row scanlines, a nine-tap bloom and filmic colour +mapping. Hovering adds a local glow and rim to the cell. A 3.5% food pulse and +0.6% phosphor shimmer keep a paused or finished game subtly alive. Visible panes +animate at 20 Hz; hidden panes stop requesting animation ticks. Its intermediate and final textures +are reused and resized together; closing the pane releases both. No CPU readback +or extra command-buffer submission is needed. A host without SGPU v2 displays +the text fallback. Use `test/gpu_snake.py BINARY ADDON OUTPUT --crt` to check the +actual RGB triads and scanline modulation, as well as the game colours. +The target covers the pane with an opaque background and centres the whole-cell +grid in any remaining margin, so fallback text cannot leak around a successful +render. These effects require no new scene commands or C ABI fields. + +`test/gpu_snake_interaction.py GUI V1_ADDON CURRENT_ADDON OUTPUT` checks live +record migration, a second reload, column resize, hover/leave, and pixel changes +while movement is paused, preserving unsaved text and shell identities. The addon +also tests buffered turns, grace deadlines, geometry, state loading and optional +GPU fallback. Its version 2 record accepts version 1 saved games. + +## Next steps: move pardes's low-level renderer + +1. **Texture bindings and instancing.** Extend the shared pipeline/draw model + with sampled textures, samplers and instance layouts/counts. Pardes's glyph + atlas and cell renderer need these before they can use the executor. Add + pixel tests for glyph masks, clipping and straight/premultiplied alpha. +2. **Extract common resource operations.** Move texture creation, staged full + and dirty-row uploads, transfer buffers and sampler caching out of + `pardes/src/gui/gui.zig`. Keep ownership explicit and use the caller's + command buffer. Test resize, partial-upload failure, window close and device + teardown against both hosts. +3. **Move the compositor a pass at a time.** Start with native image quads, + then decorations and cell instances. Preserve ordering of pane content, + tags, notices and selection, clipping and window opacity. Pardes continues + to produce layout, shaped text and draw data; those are editor semantics. +4. **Persistent addon resources, when a consumer needs them.** Add + content-addressed shader/mesh resources and explicit updates to the separate + scene protocol. Define per-session/per-view ownership, quotas, invalidation, + reconnect and missed-frame recovery before adding compute/storage buffers. + Keep all of this behind the optional callback, not new mandatory addon ops. +5. **Other frontends.** Implement the scene executor on the native macOS path + (shader translation/backend choice needs deciding), preserving fallbacks. + Sketchbook's WebGPU implementation is another possible consumer; SGPU v1 + currently carries SPIR-V, not WGSL. +6. **Measure before widening the API.** Compare CPU record time, GPU time, + uploads, submissions and memory for many panes, large geometry and repeated + reload/resize. Batch compute/uploads into caller command buffers when + measurements justify it. Add allocation limits across targets, rather than + relying only on per-scene bounds, before treating this as a large-scene API. + +Do not migrate text shaping, editor layout, input routing or session state into +the renderer. Do not add raw device pointers to pardes's addon ABI. The shared +layer should execute graphics work; the applications decide what to draw. + +## Validation (2026-11-12) + +The two builds used Zig 0.17.0, ReleaseSafe, the SDL frontend, PDF disabled, +and the same pre-existing pardes edits. The baseline was the checkout before +the GPU changes. Three interleaved runs per build scrolled the same 10,000-line +file at a 100×30 grid. Three further runs restarted the shader snake every +16 ms to change its uniforms. Startup's first 500 ms was excluded. + +| Workload | Frames | Median frame work | p95 frame work | Frames over 16.7 ms | +| --- | ---: | ---: | ---: | ---: | +| Text scrolling, before | 458 | 7.627 ms | 16.068 ms | 16 | +| Text scrolling, after | 462 | 7.519 ms | 14.838 ms | 8 | +| Shader snake | 350 | 7.414 ms | 14.208 ms | 5 | + +These are FrameStats update + core render + present times, **not GPU timer +queries or a frame-rate guarantee**. The offscreen driver cannot present to a +headless Vulkan surface here, so the test includes GPU readback and software +presentation/capture. Both builds use the same path. Run-to-run variation was +substantial; the result supports no measured scrolling regression, not a claim +of a speedup. Median core-render time was 77 µs before, 78.5 µs after, and +124 µs for the shader snake. Shader images were checked for the board's solid +green segments and orange food, and visually inspected. + +Reproduce in pardes with `test/gpu_snake.py BINARY ADDON OUTPUT --text` for +scrolling, omitting `--text` for the shader. Each output directory keeps the +raw FrameStats TSV, screenshot, stderr and JSON summary. Use matching build +options, an otherwise idle machine, and interleave the baseline and changed +build; do not measure while compiling. + +`test/gpu_upgrade.py OLD_GUI NEW_GUI OLD_SNAKE SHADER_SNAKE OUTPUT` checks +in-place upgrade, unsaved text, shell identity, saved game state, addon reload, +and a second GPU replica. Explicitly loaded addons must be loaded again after +an upgrade; their placeholder panes retain the saved state in the meantime. +The test exposed an existing missing addon generation field in pardes's dump +codec; saving it is necessary for a replica joining after an addon reload. + +Pardes's full core suite passed 967 of 969 tests. The same two failures also +occurred in the baseline (965 of 967): the full-screen tty handoff test and +the terminal dump test's cleanup crash. They remain unresolved; the GPU +changes introduced no additional failures in this comparison. Addon fixtures, +old ABI 3.0 addon compatibility, shader pixels, and the isolated upgrade/reload/ +replica checks passed. The live session then upgraded with its text contents +and shell process IDs preserved. + +The sketchbook ReleaseSafe suite covers vector GPU/CPU pixel comparisons, +render targets, replay/reload, exports and scene playback. Its existing clock +test compared struct padding; it now compares fields, avoiding an unrelated +ReleaseSafe test failure. Its vector benchmark recorded the two small sample +scenes in 15–16 µs, with CPU parse+draw at 278–315 µs (the gate is 1 ms). + +The CRT follow-up passed the scene codec tests, addon fixtures, Sketchbook's +ReleaseSafe suite, v1 rendering compatibility, and the isolated upgrade/reload/ +column-resize/pane-close/replica test. Native captures verified RGB triads and +scanline contrast, including a narrow pane without fallback text leaking through. +Three interleaved runs of the final version measured median frame work of +10.409 ms before and 10.585 ms with CRT, with p95 at 16.671 and 17.512 ms +(328 frames each). This is a small measured extra cost, not a zero-overhead +claim; the same offscreen capture limitations above apply. The live session +was upgraded and pane 42 reloaded with its game state, text and shells preserved. + +The game-feel follow-up passed seven addon tests, its updated scripted fixture, +GPU colour/CRT checks, and the interaction test for migration, reload, resizing, +hover/leave and animation while paused. The upgrade/reload/replica test also +passed. Migration exposed two existing host bugs: reload passed the replacement +library's version for the old record, and restored records retained the old +version after successful migration. Pardes now preserves the input record version +until loading succeeds and saves subsequent records with the current version; +a focused core regression test covers both paths. No ABI fields were added. + +Three interleaved before/after runs compared the previous CRT addon with the +resizable board, bloom, colour mapping, hover glow and idle animation: + +| Workload | Frames | Median frame work | p95 frame work | +| --- | ---: | ---: | ---: | +| Previous CRT snake | 378 | 4.759 ms | 14.616 ms | +| New snake and effects | 369 | 5.271 ms | 15.313 ms | +| Text scrolling, before (one run) | 136 | 12.848 ms | 19.067 ms | +| Text scrolling, after (one run) | 133 | 13.275 ms | 19.027 ms | + +The effects added about 0.5 ms to the pooled median in this capture. Individual +snake-run medians ranged from 4.6 to 12.7 ms, so system variation was substantial; +the scrolling pair is a smoke check, not evidence of zero regression. These are +the same offscreen/readback measurements described above, not GPU timer queries. +Visible paused games now request animation frames instead of remaining idle. +The full core and Sketchbook suites were not repeated for this follow-up. +The live session was upgraded and pane 42 migrated to version 2, retaining its +score, length, step count, focus, file contents and shell identities; the open +source-directory listing refreshed from disk. |
