From 1cb36ee39a8bbc2cf61c28003670d760841d6d3e Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Mon, 16 Nov 2026 22:02:43 -0300 Subject: gpu: add a post-process pass with optional depth sampling, and describe the executor pardes shares `f.postProcess(source, target, .{...})` records a fullscreen pass that samples a completed render into a distinct target image: colour at fragment binding 0 and, with `sample_depth`, depth at 1, which needs the source's latest pass to have depth and one sample (missing depth, multisampled depth and feedback loops are rejected). The scene format's version 4 carries it, the executor records it on the caller's command buffer and owns the sampler, the web host does the same with its texture and sampler pairs, and native profiling reports the work as `postprocess`. The README says how to use it. docs/pardes-rendering.md describes the executor as pardes uses it for addon scenes: a borrowed SDL GPU device, `record` into the caller's command buffer, `run`, `output` and `postProcess`. The working copy as it stood on 2026-11-16; message written by Claude from the diff, at the author's request. --- docs/pardes-rendering.md | 253 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 docs/pardes-rendering.md (limited to 'docs/pardes-rendering.md') diff --git a/docs/pardes-rendering.md b/docs/pardes-rendering.md new file mode 100644 index 0000000..0c89724 --- /dev/null +++ b/docs/pardes-rendering.md @@ -0,0 +1,253 @@ +# Pardes and the shared GPU executor + +Pardes already draws its interface with SDL_GPU. Its addon ABI originally +accepted cells and CPU RGBA rasters only. The optional GPU path lets an addon +submit a scene, render it into a texture on the frontend's device, and composite +that texture into the pane without reading pixels back to the CPU. + +## What is shared now + +`src/gpu/Executor.zig` is the executor previously in `src/host/gpu.zig`. It +owns shaders, pipelines, buffers and offscreen targets, but no window, SDL +renderer, sketch state or image manager. Both sketchbook and pardes use it. + +- `init(allocator, device)` borrows an existing SDL GPU device. +- `record(command_buffer, pass)` records a pass into the caller's command + buffer. The caller submits it. Shader and pipeline creation happens before + drawing; errors propagate to the caller. +- `run(pass)` records and submits a pass itself; sketchbook uses this convenience. +- `output(image)` borrows the target texture and dimensions. It is valid until + resize, `forget(image)`, or `deinit()`. Release external wrappers before those + operations. The executor owns the texture. +- `postProcess(command_buffer, source, target, post)` records a fullscreen + second pass sampling `source` into a distinct `target`, on the same command + buffer. The executor owns and reuses its nearest-neighbour sampler. +- Upload, compute, asynchronous buffer download and pipeline caching remain + available. Their submission policy is unchanged. + +Sketchbook's host wraps the output in an SDL_Renderer image. Pardes samples it +directly in its existing image compositor, using premultiplied blending. + +Embedders request the `gpu-only` build option, take the `abi`, `gpu` and +`gpu_scene` modules, and supply the `gpu` module's `c` import from their existing +SDL translation. This avoids building a second host or creating another device. +Pardes currently uses a sibling checkout dependency (`../sketchbook`); pin a +published revision when these two changes are released together. + +## Keep GPU support optional + +Pardes ABI 3.1 appends one optional submission callback and its context to +`pa_paint`. The old fields and required callbacks are unchanged. A GPU addon +checks the host ABI before reading the extension; a null callback means it is +unavailable. The Zig helper `Paint.gpu(bytes)` performs both checks. + +The callback copies the scene during the call, so stack buffers are safe. An +addon still paints cells or a raster. Accepting the scene only means the host +can carry it: a terminal, the native macOS frontend, or a frontend that refuses +the shader displays the fallback. Neither ordinary addons nor the core need +SDL or a GPU. The callback transports opaque display data; graphics types do +not enter the main C ABI. + +`src/gpu/scene.zig` is the separate, versioned scene format and Zig codec. SGPU +version 1 carries one self-contained render pass: SPIR-V vertex and fragment +shaders, vertex layouts, draws, uniforms, blending, clipping, MSAA and stencil. +It contains no pointers or device handles. `Pass.image` is `.none` on the wire; +the frontend supplies its own target. A clear colour is required, so a frame +can be skipped or a new replica attached without replaying old GPU work. + +The encoding is `SGPU`, a little-endian u32 version, then the current +`abi.gpu.Pass` fields in declaration order, recursively. Slice counts and enums +are u32; scalars use little endian; optionals and booleans use a 0/1 byte. No +native padding is serialized. Changing these fields requires a format version +change. The decoder bounds allocation, dimensions, draws, shader metadata, +vertex ranges and uniform sizes before the executor sees the pass. This is +structural validation, not a shader sandbox: use shaders from a trusted session. + +SGPU version 2 keeps the v1 pass encoding and appends one `PostProcess`: +vertex shader, fragment shader, and fragment uniform bytes. The vertex stage +has no uniforms or inputs; the fragment stage has exactly one implicit sampler +at set 2, binding 0, reading the completed first pass. `decodeFrame` accepts both +versions; `encode` still writes v1 and `encodePostProcess` writes v2. The same +frame allocation, shader and uniform limits apply. This deliberately supports +one postprocessing pass, without adding a render graph or C ABI fields. + +The current limits are 4 MiB per scene, 4096 pixels per side, 64 pipelines, +4096 draws, and 16 KiB per stage's uniform block. Version 1 deliberately has no +storage-buffer handles or compute commands. Local sketchbook applications keep +the existing richer API; persistent remote GPU resources need their own +lifetime and recovery protocol before exposing them to addons. + +Pardes carries scene bytes in its authoritative pane-frame stream, including +changes where the cells are identical. Replicas need no addon code. Unchanged +scenes are not resent or rerendered. Shader bytes are currently included in each +changed scene; this is suitable for the snake example, not a large streaming +mesh. Closing a pane releases its frontend target. Unsupported frontends ignore +the scene and display the fallback. + +## First consumer: SnakeShader + +In the pardes checkout: + +``` +zig build snake-shader -Dplatform=tui -Dpdf=false -Doptimize=ReleaseSafe +zig build -Dplatform=gui -Dpdf=false -Doptimize=ReleaseSafe +``` + +In the rebuilt GUI session, execute: + +``` +Addon load /absolute/path/to/pardes/zig-out/lib/libsnake_shader.so +SnakeShader +``` + +The build requires `glslc`. The example sends board occupancy and a small set +of parameters to a fragment shader. `h j k l` and arrows steer, `r` restarts; +losing focus pauses movement. A two-turn buffer supports quick corners and +corrections, with 60 ms of grace for late input and collision recovery. The +board fills its pane with whole square cells and resizes dynamically. Shrinking +translates the snake inward, shortening only a tail that cannot fit; the score +is preserved. The last active view owns the board dimensions, so an idle replica +does not resize another player's game. On an older host it remains a text snake. +Building a new binary does not upgrade a running session. + +`snake.post.frag` samples the board in a separate GPU pass and adds a three-column +RGB phosphor mask and four-row scanlines, a nine-tap bloom and filmic colour +mapping. Hovering adds a local glow and rim to the cell. A 3.5% food pulse and +0.6% phosphor shimmer keep a paused or finished game subtly alive. Visible panes +animate at 20 Hz; hidden panes stop requesting animation ticks. Its intermediate and final textures +are reused and resized together; closing the pane releases both. No CPU readback +or extra command-buffer submission is needed. A host without SGPU v2 displays +the text fallback. Use `test/gpu_snake.py BINARY ADDON OUTPUT --crt` to check the +actual RGB triads and scanline modulation, as well as the game colours. +The target covers the pane with an opaque background and centres the whole-cell +grid in any remaining margin, so fallback text cannot leak around a successful +render. These effects require no new scene commands or C ABI fields. + +`test/gpu_snake_interaction.py GUI V1_ADDON CURRENT_ADDON OUTPUT` checks live +record migration, a second reload, column resize, hover/leave, and pixel changes +while movement is paused, preserving unsaved text and shell identities. The addon +also tests buffered turns, grace deadlines, geometry, state loading and optional +GPU fallback. Its version 2 record accepts version 1 saved games. + +## Next steps: move pardes's low-level renderer + +1. **Texture bindings and instancing.** Extend the shared pipeline/draw model + with sampled textures, samplers and instance layouts/counts. Pardes's glyph + atlas and cell renderer need these before they can use the executor. Add + pixel tests for glyph masks, clipping and straight/premultiplied alpha. +2. **Extract common resource operations.** Move texture creation, staged full + and dirty-row uploads, transfer buffers and sampler caching out of + `pardes/src/gui/gui.zig`. Keep ownership explicit and use the caller's + command buffer. Test resize, partial-upload failure, window close and device + teardown against both hosts. +3. **Move the compositor a pass at a time.** Start with native image quads, + then decorations and cell instances. Preserve ordering of pane content, + tags, notices and selection, clipping and window opacity. Pardes continues + to produce layout, shaped text and draw data; those are editor semantics. +4. **Persistent addon resources, when a consumer needs them.** Add + content-addressed shader/mesh resources and explicit updates to the separate + scene protocol. Define per-session/per-view ownership, quotas, invalidation, + reconnect and missed-frame recovery before adding compute/storage buffers. + Keep all of this behind the optional callback, not new mandatory addon ops. +5. **Other frontends.** Implement the scene executor on the native macOS path + (shader translation/backend choice needs deciding), preserving fallbacks. + Sketchbook's WebGPU implementation is another possible consumer; SGPU v1 + currently carries SPIR-V, not WGSL. +6. **Measure before widening the API.** Compare CPU record time, GPU time, + uploads, submissions and memory for many panes, large geometry and repeated + reload/resize. Batch compute/uploads into caller command buffers when + measurements justify it. Add allocation limits across targets, rather than + relying only on per-scene bounds, before treating this as a large-scene API. + +Do not migrate text shaping, editor layout, input routing or session state into +the renderer. Do not add raw device pointers to pardes's addon ABI. The shared +layer should execute graphics work; the applications decide what to draw. + +## Validation (2026-11-12) + +The two builds used Zig 0.17.0, ReleaseSafe, the SDL frontend, PDF disabled, +and the same pre-existing pardes edits. The baseline was the checkout before +the GPU changes. Three interleaved runs per build scrolled the same 10,000-line +file at a 100×30 grid. Three further runs restarted the shader snake every +16 ms to change its uniforms. Startup's first 500 ms was excluded. + +| Workload | Frames | Median frame work | p95 frame work | Frames over 16.7 ms | +| --- | ---: | ---: | ---: | ---: | +| Text scrolling, before | 458 | 7.627 ms | 16.068 ms | 16 | +| Text scrolling, after | 462 | 7.519 ms | 14.838 ms | 8 | +| Shader snake | 350 | 7.414 ms | 14.208 ms | 5 | + +These are FrameStats update + core render + present times, **not GPU timer +queries or a frame-rate guarantee**. The offscreen driver cannot present to a +headless Vulkan surface here, so the test includes GPU readback and software +presentation/capture. Both builds use the same path. Run-to-run variation was +substantial; the result supports no measured scrolling regression, not a claim +of a speedup. Median core-render time was 77 µs before, 78.5 µs after, and +124 µs for the shader snake. Shader images were checked for the board's solid +green segments and orange food, and visually inspected. + +Reproduce in pardes with `test/gpu_snake.py BINARY ADDON OUTPUT --text` for +scrolling, omitting `--text` for the shader. Each output directory keeps the +raw FrameStats TSV, screenshot, stderr and JSON summary. Use matching build +options, an otherwise idle machine, and interleave the baseline and changed +build; do not measure while compiling. + +`test/gpu_upgrade.py OLD_GUI NEW_GUI OLD_SNAKE SHADER_SNAKE OUTPUT` checks +in-place upgrade, unsaved text, shell identity, saved game state, addon reload, +and a second GPU replica. Explicitly loaded addons must be loaded again after +an upgrade; their placeholder panes retain the saved state in the meantime. +The test exposed an existing missing addon generation field in pardes's dump +codec; saving it is necessary for a replica joining after an addon reload. + +Pardes's full core suite passed 967 of 969 tests. The same two failures also +occurred in the baseline (965 of 967): the full-screen tty handoff test and +the terminal dump test's cleanup crash. They remain unresolved; the GPU +changes introduced no additional failures in this comparison. Addon fixtures, +old ABI 3.0 addon compatibility, shader pixels, and the isolated upgrade/reload/ +replica checks passed. The live session then upgraded with its text contents +and shell process IDs preserved. + +The sketchbook ReleaseSafe suite covers vector GPU/CPU pixel comparisons, +render targets, replay/reload, exports and scene playback. Its existing clock +test compared struct padding; it now compares fields, avoiding an unrelated +ReleaseSafe test failure. Its vector benchmark recorded the two small sample +scenes in 15–16 µs, with CPU parse+draw at 278–315 µs (the gate is 1 ms). + +The CRT follow-up passed the scene codec tests, addon fixtures, Sketchbook's +ReleaseSafe suite, v1 rendering compatibility, and the isolated upgrade/reload/ +column-resize/pane-close/replica test. Native captures verified RGB triads and +scanline contrast, including a narrow pane without fallback text leaking through. +Three interleaved runs of the final version measured median frame work of +10.409 ms before and 10.585 ms with CRT, with p95 at 16.671 and 17.512 ms +(328 frames each). This is a small measured extra cost, not a zero-overhead +claim; the same offscreen capture limitations above apply. The live session +was upgraded and pane 42 reloaded with its game state, text and shells preserved. + +The game-feel follow-up passed seven addon tests, its updated scripted fixture, +GPU colour/CRT checks, and the interaction test for migration, reload, resizing, +hover/leave and animation while paused. The upgrade/reload/replica test also +passed. Migration exposed two existing host bugs: reload passed the replacement +library's version for the old record, and restored records retained the old +version after successful migration. Pardes now preserves the input record version +until loading succeeds and saves subsequent records with the current version; +a focused core regression test covers both paths. No ABI fields were added. + +Three interleaved before/after runs compared the previous CRT addon with the +resizable board, bloom, colour mapping, hover glow and idle animation: + +| Workload | Frames | Median frame work | p95 frame work | +| --- | ---: | ---: | ---: | +| Previous CRT snake | 378 | 4.759 ms | 14.616 ms | +| New snake and effects | 369 | 5.271 ms | 15.313 ms | +| Text scrolling, before (one run) | 136 | 12.848 ms | 19.067 ms | +| Text scrolling, after (one run) | 133 | 13.275 ms | 19.027 ms | + +The effects added about 0.5 ms to the pooled median in this capture. Individual +snake-run medians ranged from 4.6 to 12.7 ms, so system variation was substantial; +the scrolling pair is a smoke check, not evidence of zero regression. These are +the same offscreen/readback measurements described above, not GPU timer queries. +Visible paused games now request animation frames instead of remaining idle. +The full core and Sketchbook suites were not repeated for this follow-up. +The live session was upgraded and pane 42 migrated to version 2, retaining its +score, length, step count, focus, file contents and shell identities; the open +source-directory listing refreshed from disk. -- cgit v1.3