summaryrefslogtreecommitdiff
path: root/docs/pardes-rendering.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/pardes-rendering.md')
-rw-r--r--docs/pardes-rendering.md253
1 files changed, 253 insertions, 0 deletions
diff --git a/docs/pardes-rendering.md b/docs/pardes-rendering.md
new file mode 100644
index 0000000..0c89724
--- /dev/null
+++ b/docs/pardes-rendering.md
@@ -0,0 +1,253 @@
+# Pardes and the shared GPU executor
+
+Pardes already draws its interface with SDL_GPU. Its addon ABI originally
+accepted cells and CPU RGBA rasters only. The optional GPU path lets an addon
+submit a scene, render it into a texture on the frontend's device, and composite
+that texture into the pane without reading pixels back to the CPU.
+
+## What is shared now
+
+`src/gpu/Executor.zig` is the executor previously in `src/host/gpu.zig`. It
+owns shaders, pipelines, buffers and offscreen targets, but no window, SDL
+renderer, sketch state or image manager. Both sketchbook and pardes use it.
+
+- `init(allocator, device)` borrows an existing SDL GPU device.
+- `record(command_buffer, pass)` records a pass into the caller's command
+ buffer. The caller submits it. Shader and pipeline creation happens before
+ drawing; errors propagate to the caller.
+- `run(pass)` records and submits a pass itself; sketchbook uses this convenience.
+- `output(image)` borrows the target texture and dimensions. It is valid until
+ resize, `forget(image)`, or `deinit()`. Release external wrappers before those
+ operations. The executor owns the texture.
+- `postProcess(command_buffer, source, target, post)` records a fullscreen
+ second pass sampling `source` into a distinct `target`, on the same command
+ buffer. The executor owns and reuses its nearest-neighbour sampler.
+- Upload, compute, asynchronous buffer download and pipeline caching remain
+ available. Their submission policy is unchanged.
+
+Sketchbook's host wraps the output in an SDL_Renderer image. Pardes samples it
+directly in its existing image compositor, using premultiplied blending.
+
+Embedders request the `gpu-only` build option, take the `abi`, `gpu` and
+`gpu_scene` modules, and supply the `gpu` module's `c` import from their existing
+SDL translation. This avoids building a second host or creating another device.
+Pardes currently uses a sibling checkout dependency (`../sketchbook`); pin a
+published revision when these two changes are released together.
+
+## Keep GPU support optional
+
+Pardes ABI 3.1 appends one optional submission callback and its context to
+`pa_paint`. The old fields and required callbacks are unchanged. A GPU addon
+checks the host ABI before reading the extension; a null callback means it is
+unavailable. The Zig helper `Paint.gpu(bytes)` performs both checks.
+
+The callback copies the scene during the call, so stack buffers are safe. An
+addon still paints cells or a raster. Accepting the scene only means the host
+can carry it: a terminal, the native macOS frontend, or a frontend that refuses
+the shader displays the fallback. Neither ordinary addons nor the core need
+SDL or a GPU. The callback transports opaque display data; graphics types do
+not enter the main C ABI.
+
+`src/gpu/scene.zig` is the separate, versioned scene format and Zig codec. SGPU
+version 1 carries one self-contained render pass: SPIR-V vertex and fragment
+shaders, vertex layouts, draws, uniforms, blending, clipping, MSAA and stencil.
+It contains no pointers or device handles. `Pass.image` is `.none` on the wire;
+the frontend supplies its own target. A clear colour is required, so a frame
+can be skipped or a new replica attached without replaying old GPU work.
+
+The encoding is `SGPU`, a little-endian u32 version, then the current
+`abi.gpu.Pass` fields in declaration order, recursively. Slice counts and enums
+are u32; scalars use little endian; optionals and booleans use a 0/1 byte. No
+native padding is serialized. Changing these fields requires a format version
+change. The decoder bounds allocation, dimensions, draws, shader metadata,
+vertex ranges and uniform sizes before the executor sees the pass. This is
+structural validation, not a shader sandbox: use shaders from a trusted session.
+
+SGPU version 2 keeps the v1 pass encoding and appends one `PostProcess`:
+vertex shader, fragment shader, and fragment uniform bytes. The vertex stage
+has no uniforms or inputs; the fragment stage has exactly one implicit sampler
+at set 2, binding 0, reading the completed first pass. `decodeFrame` accepts both
+versions; `encode` still writes v1 and `encodePostProcess` writes v2. The same
+frame allocation, shader and uniform limits apply. This deliberately supports
+one postprocessing pass, without adding a render graph or C ABI fields.
+
+The current limits are 4 MiB per scene, 4096 pixels per side, 64 pipelines,
+4096 draws, and 16 KiB per stage's uniform block. Version 1 deliberately has no
+storage-buffer handles or compute commands. Local sketchbook applications keep
+the existing richer API; persistent remote GPU resources need their own
+lifetime and recovery protocol before exposing them to addons.
+
+Pardes carries scene bytes in its authoritative pane-frame stream, including
+changes where the cells are identical. Replicas need no addon code. Unchanged
+scenes are not resent or rerendered. Shader bytes are currently included in each
+changed scene; this is suitable for the snake example, not a large streaming
+mesh. Closing a pane releases its frontend target. Unsupported frontends ignore
+the scene and display the fallback.
+
+## First consumer: SnakeShader
+
+In the pardes checkout:
+
+```
+zig build snake-shader -Dplatform=tui -Dpdf=false -Doptimize=ReleaseSafe
+zig build -Dplatform=gui -Dpdf=false -Doptimize=ReleaseSafe
+```
+
+In the rebuilt GUI session, execute:
+
+```
+Addon load /absolute/path/to/pardes/zig-out/lib/libsnake_shader.so
+SnakeShader
+```
+
+The build requires `glslc`. The example sends board occupancy and a small set
+of parameters to a fragment shader. `h j k l` and arrows steer, `r` restarts;
+losing focus pauses movement. A two-turn buffer supports quick corners and
+corrections, with 60 ms of grace for late input and collision recovery. The
+board fills its pane with whole square cells and resizes dynamically. Shrinking
+translates the snake inward, shortening only a tail that cannot fit; the score
+is preserved. The last active view owns the board dimensions, so an idle replica
+does not resize another player's game. On an older host it remains a text snake.
+Building a new binary does not upgrade a running session.
+
+`snake.post.frag` samples the board in a separate GPU pass and adds a three-column
+RGB phosphor mask and four-row scanlines, a nine-tap bloom and filmic colour
+mapping. Hovering adds a local glow and rim to the cell. A 3.5% food pulse and
+0.6% phosphor shimmer keep a paused or finished game subtly alive. Visible panes
+animate at 20 Hz; hidden panes stop requesting animation ticks. Its intermediate and final textures
+are reused and resized together; closing the pane releases both. No CPU readback
+or extra command-buffer submission is needed. A host without SGPU v2 displays
+the text fallback. Use `test/gpu_snake.py BINARY ADDON OUTPUT --crt` to check the
+actual RGB triads and scanline modulation, as well as the game colours.
+The target covers the pane with an opaque background and centres the whole-cell
+grid in any remaining margin, so fallback text cannot leak around a successful
+render. These effects require no new scene commands or C ABI fields.
+
+`test/gpu_snake_interaction.py GUI V1_ADDON CURRENT_ADDON OUTPUT` checks live
+record migration, a second reload, column resize, hover/leave, and pixel changes
+while movement is paused, preserving unsaved text and shell identities. The addon
+also tests buffered turns, grace deadlines, geometry, state loading and optional
+GPU fallback. Its version 2 record accepts version 1 saved games.
+
+## Next steps: move pardes's low-level renderer
+
+1. **Texture bindings and instancing.** Extend the shared pipeline/draw model
+ with sampled textures, samplers and instance layouts/counts. Pardes's glyph
+ atlas and cell renderer need these before they can use the executor. Add
+ pixel tests for glyph masks, clipping and straight/premultiplied alpha.
+2. **Extract common resource operations.** Move texture creation, staged full
+ and dirty-row uploads, transfer buffers and sampler caching out of
+ `pardes/src/gui/gui.zig`. Keep ownership explicit and use the caller's
+ command buffer. Test resize, partial-upload failure, window close and device
+ teardown against both hosts.
+3. **Move the compositor a pass at a time.** Start with native image quads,
+ then decorations and cell instances. Preserve ordering of pane content,
+ tags, notices and selection, clipping and window opacity. Pardes continues
+ to produce layout, shaped text and draw data; those are editor semantics.
+4. **Persistent addon resources, when a consumer needs them.** Add
+ content-addressed shader/mesh resources and explicit updates to the separate
+ scene protocol. Define per-session/per-view ownership, quotas, invalidation,
+ reconnect and missed-frame recovery before adding compute/storage buffers.
+ Keep all of this behind the optional callback, not new mandatory addon ops.
+5. **Other frontends.** Implement the scene executor on the native macOS path
+ (shader translation/backend choice needs deciding), preserving fallbacks.
+ Sketchbook's WebGPU implementation is another possible consumer; SGPU v1
+ currently carries SPIR-V, not WGSL.
+6. **Measure before widening the API.** Compare CPU record time, GPU time,
+ uploads, submissions and memory for many panes, large geometry and repeated
+ reload/resize. Batch compute/uploads into caller command buffers when
+ measurements justify it. Add allocation limits across targets, rather than
+ relying only on per-scene bounds, before treating this as a large-scene API.
+
+Do not migrate text shaping, editor layout, input routing or session state into
+the renderer. Do not add raw device pointers to pardes's addon ABI. The shared
+layer should execute graphics work; the applications decide what to draw.
+
+## Validation (2026-11-12)
+
+The two builds used Zig 0.17.0, ReleaseSafe, the SDL frontend, PDF disabled,
+and the same pre-existing pardes edits. The baseline was the checkout before
+the GPU changes. Three interleaved runs per build scrolled the same 10,000-line
+file at a 100×30 grid. Three further runs restarted the shader snake every
+16 ms to change its uniforms. Startup's first 500 ms was excluded.
+
+| Workload | Frames | Median frame work | p95 frame work | Frames over 16.7 ms |
+| --- | ---: | ---: | ---: | ---: |
+| Text scrolling, before | 458 | 7.627 ms | 16.068 ms | 16 |
+| Text scrolling, after | 462 | 7.519 ms | 14.838 ms | 8 |
+| Shader snake | 350 | 7.414 ms | 14.208 ms | 5 |
+
+These are FrameStats update + core render + present times, **not GPU timer
+queries or a frame-rate guarantee**. The offscreen driver cannot present to a
+headless Vulkan surface here, so the test includes GPU readback and software
+presentation/capture. Both builds use the same path. Run-to-run variation was
+substantial; the result supports no measured scrolling regression, not a claim
+of a speedup. Median core-render time was 77 µs before, 78.5 µs after, and
+124 µs for the shader snake. Shader images were checked for the board's solid
+green segments and orange food, and visually inspected.
+
+Reproduce in pardes with `test/gpu_snake.py BINARY ADDON OUTPUT --text` for
+scrolling, omitting `--text` for the shader. Each output directory keeps the
+raw FrameStats TSV, screenshot, stderr and JSON summary. Use matching build
+options, an otherwise idle machine, and interleave the baseline and changed
+build; do not measure while compiling.
+
+`test/gpu_upgrade.py OLD_GUI NEW_GUI OLD_SNAKE SHADER_SNAKE OUTPUT` checks
+in-place upgrade, unsaved text, shell identity, saved game state, addon reload,
+and a second GPU replica. Explicitly loaded addons must be loaded again after
+an upgrade; their placeholder panes retain the saved state in the meantime.
+The test exposed an existing missing addon generation field in pardes's dump
+codec; saving it is necessary for a replica joining after an addon reload.
+
+Pardes's full core suite passed 967 of 969 tests. The same two failures also
+occurred in the baseline (965 of 967): the full-screen tty handoff test and
+the terminal dump test's cleanup crash. They remain unresolved; the GPU
+changes introduced no additional failures in this comparison. Addon fixtures,
+old ABI 3.0 addon compatibility, shader pixels, and the isolated upgrade/reload/
+replica checks passed. The live session then upgraded with its text contents
+and shell process IDs preserved.
+
+The sketchbook ReleaseSafe suite covers vector GPU/CPU pixel comparisons,
+render targets, replay/reload, exports and scene playback. Its existing clock
+test compared struct padding; it now compares fields, avoiding an unrelated
+ReleaseSafe test failure. Its vector benchmark recorded the two small sample
+scenes in 15–16 µs, with CPU parse+draw at 278–315 µs (the gate is 1 ms).
+
+The CRT follow-up passed the scene codec tests, addon fixtures, Sketchbook's
+ReleaseSafe suite, v1 rendering compatibility, and the isolated upgrade/reload/
+column-resize/pane-close/replica test. Native captures verified RGB triads and
+scanline contrast, including a narrow pane without fallback text leaking through.
+Three interleaved runs of the final version measured median frame work of
+10.409 ms before and 10.585 ms with CRT, with p95 at 16.671 and 17.512 ms
+(328 frames each). This is a small measured extra cost, not a zero-overhead
+claim; the same offscreen capture limitations above apply. The live session
+was upgraded and pane 42 reloaded with its game state, text and shells preserved.
+
+The game-feel follow-up passed seven addon tests, its updated scripted fixture,
+GPU colour/CRT checks, and the interaction test for migration, reload, resizing,
+hover/leave and animation while paused. The upgrade/reload/replica test also
+passed. Migration exposed two existing host bugs: reload passed the replacement
+library's version for the old record, and restored records retained the old
+version after successful migration. Pardes now preserves the input record version
+until loading succeeds and saves subsequent records with the current version;
+a focused core regression test covers both paths. No ABI fields were added.
+
+Three interleaved before/after runs compared the previous CRT addon with the
+resizable board, bloom, colour mapping, hover glow and idle animation:
+
+| Workload | Frames | Median frame work | p95 frame work |
+| --- | ---: | ---: | ---: |
+| Previous CRT snake | 378 | 4.759 ms | 14.616 ms |
+| New snake and effects | 369 | 5.271 ms | 15.313 ms |
+| Text scrolling, before (one run) | 136 | 12.848 ms | 19.067 ms |
+| Text scrolling, after (one run) | 133 | 13.275 ms | 19.027 ms |
+
+The effects added about 0.5 ms to the pooled median in this capture. Individual
+snake-run medians ranged from 4.6 to 12.7 ms, so system variation was substantial;
+the scrolling pair is a smoke check, not evidence of zero regression. These are
+the same offscreen/readback measurements described above, not GPU timer queries.
+Visible paused games now request animation frames instead of remaining idle.
+The full core and Sketchbook suites were not repeated for this follow-up.
+The live session was upgraded and pane 42 migrated to version 2, retaining its
+score, length, step count, focus, file contents and shell identities; the open
+source-directory listing refreshed from disk.