diff options
| author | Gabriel Schneider <[email protected]> | 2026-11-16 22:02:43 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-11-17 12:15:01 -0300 |
| commit | 1cb36ee39a8bbc2cf61c28003670d760841d6d3e (patch) | |
| tree | d7d01eb5b3a40920a8f71d091a106561ca56b282 | |
| parent | 6933f99ddb2d7103515d26838496309a807bba26 (diff) | |
| download | sketchbook-1cb36ee39a8bbc2cf61c28003670d760841d6d3e.tar.gz sketchbook-1cb36ee39a8bbc2cf61c28003670d760841d6d3e.zip | |
gpu: add a post-process pass with optional depth sampling, and describe the executor pardes shares
`f.postProcess(source, target, .{...})` records a fullscreen pass that samples a completed render into a distinct target image: colour at fragment binding 0 and, with `sample_depth`, depth at 1, which needs the source's latest pass to have depth and one sample (missing depth, multisampled depth and feedback loops are rejected). The scene format's version 4 carries it, the executor records it on the caller's command buffer and owns the sampler, the web host does the same with its texture and sampler pairs, and native profiling reports the work as `postprocess`. The README says how to use it.
docs/pardes-rendering.md describes the executor as pardes uses it for addon scenes: a borrowed SDL GPU device, `record` into the caller's command buffer, `run`, `output` and `postProcess`.
The working copy as it stood on 2026-11-16; message written by Claude from the diff, at the author's request.
| -rw-r--r-- | README.md | 11 | ||||
| -rw-r--r-- | docs/pardes-rendering.md | 253 | ||||
| -rw-r--r-- | src/abi.zig | 8 | ||||
| -rw-r--r-- | src/gpu/Executor.zig | 41 | ||||
| -rw-r--r-- | src/gpu/scene.zig | 55 | ||||
| -rw-r--r-- | src/host/Host.zig | 10 | ||||
| -rw-r--r-- | src/sketch/clock.zig | 2 | ||||
| -rw-r--r-- | src/sketch/root.zig | 27 | ||||
| -rw-r--r-- | src/sketch/web.zig | 5 | ||||
| -rw-r--r-- | web/host.js | 53 |
10 files changed, 443 insertions, 22 deletions
@@ -1170,6 +1170,17 @@ released by the host on session reset; call `model.release(host)` to release the earlier. Vertex storage layout and shader uniform layout are checked at comptime; GLSL uniform declarations are generated from the Zig types. +A separate fullscreen pass can sample a completed render with +`f.postProcess(source, target, .{ .vertex = vertex_shader, .fragment = fragment_shader, +.sample_depth = true, .fragment_uniforms = uniforms })`. Allocate distinct source +and target images in setup, then draw the target image after this command. +Native fragment bindings are color at 0 and depth at 1 (set 2). Web GLSL uses +separate texture/sampler pairs at 0/1 and 2/3 (set 2). Fragment uniforms use set 3, +binding 0. Depth sampling requires the source's latest pass to have `.depth = true` +and `.samples = 1`; missing depth, multisampled depth and feedback loops are +rejected. Omit `sample_depth` for a color-only pass. Native profiling reports this +work separately as `postprocess`. + `profile` also includes `p95_ms` and `p99_ms`: nearest-rank percentiles of the latest 2048 measured commands per label. Mean and worst cover every command since reset. `profile-trace` exposes those retained samples as CSV (`label,sample,ns`), diff --git a/docs/pardes-rendering.md b/docs/pardes-rendering.md new file mode 100644 index 0000000..0c89724 --- /dev/null +++ b/docs/pardes-rendering.md @@ -0,0 +1,253 @@ +# Pardes and the shared GPU executor + +Pardes already draws its interface with SDL_GPU. Its addon ABI originally +accepted cells and CPU RGBA rasters only. The optional GPU path lets an addon +submit a scene, render it into a texture on the frontend's device, and composite +that texture into the pane without reading pixels back to the CPU. + +## What is shared now + +`src/gpu/Executor.zig` is the executor previously in `src/host/gpu.zig`. It +owns shaders, pipelines, buffers and offscreen targets, but no window, SDL +renderer, sketch state or image manager. Both sketchbook and pardes use it. + +- `init(allocator, device)` borrows an existing SDL GPU device. +- `record(command_buffer, pass)` records a pass into the caller's command + buffer. The caller submits it. Shader and pipeline creation happens before + drawing; errors propagate to the caller. +- `run(pass)` records and submits a pass itself; sketchbook uses this convenience. +- `output(image)` borrows the target texture and dimensions. It is valid until + resize, `forget(image)`, or `deinit()`. Release external wrappers before those + operations. The executor owns the texture. +- `postProcess(command_buffer, source, target, post)` records a fullscreen + second pass sampling `source` into a distinct `target`, on the same command + buffer. The executor owns and reuses its nearest-neighbour sampler. +- Upload, compute, asynchronous buffer download and pipeline caching remain + available. Their submission policy is unchanged. + +Sketchbook's host wraps the output in an SDL_Renderer image. Pardes samples it +directly in its existing image compositor, using premultiplied blending. + +Embedders request the `gpu-only` build option, take the `abi`, `gpu` and +`gpu_scene` modules, and supply the `gpu` module's `c` import from their existing +SDL translation. This avoids building a second host or creating another device. +Pardes currently uses a sibling checkout dependency (`../sketchbook`); pin a +published revision when these two changes are released together. + +## Keep GPU support optional + +Pardes ABI 3.1 appends one optional submission callback and its context to +`pa_paint`. The old fields and required callbacks are unchanged. A GPU addon +checks the host ABI before reading the extension; a null callback means it is +unavailable. The Zig helper `Paint.gpu(bytes)` performs both checks. + +The callback copies the scene during the call, so stack buffers are safe. An +addon still paints cells or a raster. Accepting the scene only means the host +can carry it: a terminal, the native macOS frontend, or a frontend that refuses +the shader displays the fallback. Neither ordinary addons nor the core need +SDL or a GPU. The callback transports opaque display data; graphics types do +not enter the main C ABI. + +`src/gpu/scene.zig` is the separate, versioned scene format and Zig codec. SGPU +version 1 carries one self-contained render pass: SPIR-V vertex and fragment +shaders, vertex layouts, draws, uniforms, blending, clipping, MSAA and stencil. +It contains no pointers or device handles. `Pass.image` is `.none` on the wire; +the frontend supplies its own target. A clear colour is required, so a frame +can be skipped or a new replica attached without replaying old GPU work. + +The encoding is `SGPU`, a little-endian u32 version, then the current +`abi.gpu.Pass` fields in declaration order, recursively. Slice counts and enums +are u32; scalars use little endian; optionals and booleans use a 0/1 byte. No +native padding is serialized. Changing these fields requires a format version +change. The decoder bounds allocation, dimensions, draws, shader metadata, +vertex ranges and uniform sizes before the executor sees the pass. This is +structural validation, not a shader sandbox: use shaders from a trusted session. + +SGPU version 2 keeps the v1 pass encoding and appends one `PostProcess`: +vertex shader, fragment shader, and fragment uniform bytes. The vertex stage +has no uniforms or inputs; the fragment stage has exactly one implicit sampler +at set 2, binding 0, reading the completed first pass. `decodeFrame` accepts both +versions; `encode` still writes v1 and `encodePostProcess` writes v2. The same +frame allocation, shader and uniform limits apply. This deliberately supports +one postprocessing pass, without adding a render graph or C ABI fields. + +The current limits are 4 MiB per scene, 4096 pixels per side, 64 pipelines, +4096 draws, and 16 KiB per stage's uniform block. Version 1 deliberately has no +storage-buffer handles or compute commands. Local sketchbook applications keep +the existing richer API; persistent remote GPU resources need their own +lifetime and recovery protocol before exposing them to addons. + +Pardes carries scene bytes in its authoritative pane-frame stream, including +changes where the cells are identical. Replicas need no addon code. Unchanged +scenes are not resent or rerendered. Shader bytes are currently included in each +changed scene; this is suitable for the snake example, not a large streaming +mesh. Closing a pane releases its frontend target. Unsupported frontends ignore +the scene and display the fallback. + +## First consumer: SnakeShader + +In the pardes checkout: + +``` +zig build snake-shader -Dplatform=tui -Dpdf=false -Doptimize=ReleaseSafe +zig build -Dplatform=gui -Dpdf=false -Doptimize=ReleaseSafe +``` + +In the rebuilt GUI session, execute: + +``` +Addon load /absolute/path/to/pardes/zig-out/lib/libsnake_shader.so +SnakeShader +``` + +The build requires `glslc`. The example sends board occupancy and a small set +of parameters to a fragment shader. `h j k l` and arrows steer, `r` restarts; +losing focus pauses movement. A two-turn buffer supports quick corners and +corrections, with 60 ms of grace for late input and collision recovery. The +board fills its pane with whole square cells and resizes dynamically. Shrinking +translates the snake inward, shortening only a tail that cannot fit; the score +is preserved. The last active view owns the board dimensions, so an idle replica +does not resize another player's game. On an older host it remains a text snake. +Building a new binary does not upgrade a running session. + +`snake.post.frag` samples the board in a separate GPU pass and adds a three-column +RGB phosphor mask and four-row scanlines, a nine-tap bloom and filmic colour +mapping. Hovering adds a local glow and rim to the cell. A 3.5% food pulse and +0.6% phosphor shimmer keep a paused or finished game subtly alive. Visible panes +animate at 20 Hz; hidden panes stop requesting animation ticks. Its intermediate and final textures +are reused and resized together; closing the pane releases both. No CPU readback +or extra command-buffer submission is needed. A host without SGPU v2 displays +the text fallback. Use `test/gpu_snake.py BINARY ADDON OUTPUT --crt` to check the +actual RGB triads and scanline modulation, as well as the game colours. +The target covers the pane with an opaque background and centres the whole-cell +grid in any remaining margin, so fallback text cannot leak around a successful +render. These effects require no new scene commands or C ABI fields. + +`test/gpu_snake_interaction.py GUI V1_ADDON CURRENT_ADDON OUTPUT` checks live +record migration, a second reload, column resize, hover/leave, and pixel changes +while movement is paused, preserving unsaved text and shell identities. The addon +also tests buffered turns, grace deadlines, geometry, state loading and optional +GPU fallback. Its version 2 record accepts version 1 saved games. + +## Next steps: move pardes's low-level renderer + +1. **Texture bindings and instancing.** Extend the shared pipeline/draw model + with sampled textures, samplers and instance layouts/counts. Pardes's glyph + atlas and cell renderer need these before they can use the executor. Add + pixel tests for glyph masks, clipping and straight/premultiplied alpha. +2. **Extract common resource operations.** Move texture creation, staged full + and dirty-row uploads, transfer buffers and sampler caching out of + `pardes/src/gui/gui.zig`. Keep ownership explicit and use the caller's + command buffer. Test resize, partial-upload failure, window close and device + teardown against both hosts. +3. **Move the compositor a pass at a time.** Start with native image quads, + then decorations and cell instances. Preserve ordering of pane content, + tags, notices and selection, clipping and window opacity. Pardes continues + to produce layout, shaped text and draw data; those are editor semantics. +4. **Persistent addon resources, when a consumer needs them.** Add + content-addressed shader/mesh resources and explicit updates to the separate + scene protocol. Define per-session/per-view ownership, quotas, invalidation, + reconnect and missed-frame recovery before adding compute/storage buffers. + Keep all of this behind the optional callback, not new mandatory addon ops. +5. **Other frontends.** Implement the scene executor on the native macOS path + (shader translation/backend choice needs deciding), preserving fallbacks. + Sketchbook's WebGPU implementation is another possible consumer; SGPU v1 + currently carries SPIR-V, not WGSL. +6. **Measure before widening the API.** Compare CPU record time, GPU time, + uploads, submissions and memory for many panes, large geometry and repeated + reload/resize. Batch compute/uploads into caller command buffers when + measurements justify it. Add allocation limits across targets, rather than + relying only on per-scene bounds, before treating this as a large-scene API. + +Do not migrate text shaping, editor layout, input routing or session state into +the renderer. Do not add raw device pointers to pardes's addon ABI. The shared +layer should execute graphics work; the applications decide what to draw. + +## Validation (2026-11-12) + +The two builds used Zig 0.17.0, ReleaseSafe, the SDL frontend, PDF disabled, +and the same pre-existing pardes edits. The baseline was the checkout before +the GPU changes. Three interleaved runs per build scrolled the same 10,000-line +file at a 100×30 grid. Three further runs restarted the shader snake every +16 ms to change its uniforms. Startup's first 500 ms was excluded. + +| Workload | Frames | Median frame work | p95 frame work | Frames over 16.7 ms | +| --- | ---: | ---: | ---: | ---: | +| Text scrolling, before | 458 | 7.627 ms | 16.068 ms | 16 | +| Text scrolling, after | 462 | 7.519 ms | 14.838 ms | 8 | +| Shader snake | 350 | 7.414 ms | 14.208 ms | 5 | + +These are FrameStats update + core render + present times, **not GPU timer +queries or a frame-rate guarantee**. The offscreen driver cannot present to a +headless Vulkan surface here, so the test includes GPU readback and software +presentation/capture. Both builds use the same path. Run-to-run variation was +substantial; the result supports no measured scrolling regression, not a claim +of a speedup. Median core-render time was 77 µs before, 78.5 µs after, and +124 µs for the shader snake. Shader images were checked for the board's solid +green segments and orange food, and visually inspected. + +Reproduce in pardes with `test/gpu_snake.py BINARY ADDON OUTPUT --text` for +scrolling, omitting `--text` for the shader. Each output directory keeps the +raw FrameStats TSV, screenshot, stderr and JSON summary. Use matching build +options, an otherwise idle machine, and interleave the baseline and changed +build; do not measure while compiling. + +`test/gpu_upgrade.py OLD_GUI NEW_GUI OLD_SNAKE SHADER_SNAKE OUTPUT` checks +in-place upgrade, unsaved text, shell identity, saved game state, addon reload, +and a second GPU replica. Explicitly loaded addons must be loaded again after +an upgrade; their placeholder panes retain the saved state in the meantime. +The test exposed an existing missing addon generation field in pardes's dump +codec; saving it is necessary for a replica joining after an addon reload. + +Pardes's full core suite passed 967 of 969 tests. The same two failures also +occurred in the baseline (965 of 967): the full-screen tty handoff test and +the terminal dump test's cleanup crash. They remain unresolved; the GPU +changes introduced no additional failures in this comparison. Addon fixtures, +old ABI 3.0 addon compatibility, shader pixels, and the isolated upgrade/reload/ +replica checks passed. The live session then upgraded with its text contents +and shell process IDs preserved. + +The sketchbook ReleaseSafe suite covers vector GPU/CPU pixel comparisons, +render targets, replay/reload, exports and scene playback. Its existing clock +test compared struct padding; it now compares fields, avoiding an unrelated +ReleaseSafe test failure. Its vector benchmark recorded the two small sample +scenes in 15–16 µs, with CPU parse+draw at 278–315 µs (the gate is 1 ms). + +The CRT follow-up passed the scene codec tests, addon fixtures, Sketchbook's +ReleaseSafe suite, v1 rendering compatibility, and the isolated upgrade/reload/ +column-resize/pane-close/replica test. Native captures verified RGB triads and +scanline contrast, including a narrow pane without fallback text leaking through. +Three interleaved runs of the final version measured median frame work of +10.409 ms before and 10.585 ms with CRT, with p95 at 16.671 and 17.512 ms +(328 frames each). This is a small measured extra cost, not a zero-overhead +claim; the same offscreen capture limitations above apply. The live session +was upgraded and pane 42 reloaded with its game state, text and shells preserved. + +The game-feel follow-up passed seven addon tests, its updated scripted fixture, +GPU colour/CRT checks, and the interaction test for migration, reload, resizing, +hover/leave and animation while paused. The upgrade/reload/replica test also +passed. Migration exposed two existing host bugs: reload passed the replacement +library's version for the old record, and restored records retained the old +version after successful migration. Pardes now preserves the input record version +until loading succeeds and saves subsequent records with the current version; +a focused core regression test covers both paths. No ABI fields were added. + +Three interleaved before/after runs compared the previous CRT addon with the +resizable board, bloom, colour mapping, hover glow and idle animation: + +| Workload | Frames | Median frame work | p95 frame work | +| --- | ---: | ---: | ---: | +| Previous CRT snake | 378 | 4.759 ms | 14.616 ms | +| New snake and effects | 369 | 5.271 ms | 15.313 ms | +| Text scrolling, before (one run) | 136 | 12.848 ms | 19.067 ms | +| Text scrolling, after (one run) | 133 | 13.275 ms | 19.027 ms | + +The effects added about 0.5 ms to the pooled median in this capture. Individual +snake-run medians ranged from 4.6 to 12.7 ms, so system variation was substantial; +the scrolling pair is a smoke check, not evidence of zero regression. These are +the same offscreen/readback measurements described above, not GPU timer queries. +Visible paused games now request animation frames instead of remaining idle. +The full core and Sketchbook suites were not repeated for this follow-up. +The live session was upgraded and pane 42 migrated to version 2, retaining its +score, length, step count, focus, file contents and shell identities; the open +source-directory listing refreshed from disk. diff --git a/src/abi.zig b/src/abi.zig index d91fa23..7177f14 100644 --- a/src/abi.zig +++ b/src/abi.zig @@ -370,6 +370,8 @@ pub const Command = union(enum) { /// Renders a GPU pass into an image made by `Host.newImage`, when /// `Host.gpu` says the host can. gpu: gpu.Pass, + /// Fullscreen pass sampling a completed GPU image, into a different image. + post_process: struct { source: Image, target: Image, effect: gpu.PostProcess }, /// Bytes into a buffer made by `Host.newBuffer`, at `offset`. upload: struct { buffer: Buffer, offset: u32 = 0, bytes: []const u8 }, /// Runs a compute shader over buffers (see `abi.gpu`). @@ -579,8 +581,10 @@ pub const gpu = struct { draws: []const Draw, }; - /// A fullscreen triangle sampling one completed pass at fragment binding 0. + /// Fullscreen triangle: color at fragment binding 0, optional depth at 1. pub const PostProcess = struct { + /// Requires a preceding single-sample pass with depth enabled. + sample_depth: bool = false, vertex: Shader, fragment: Shader, fragment_uniforms: []const u8 = &.{}, @@ -813,7 +817,7 @@ pub const Sketch = struct { pub const hash: u64 = blk: { @setEvalBranchQuota(1_000_000); var h: u64 = 0; - for (.{ Rect, Color, BlendMode, Corners, Event, Mode, Stream, Input, Image, Audio, Cursor, Voice, Command, gpu.Pass, gpu.Pipeline, gpu.Draw, gpu.Dispatch, Host, Host.VTable, Context, Field, Layout, Info, Memory, Sketch }) |T| { + for (.{ Rect, Color, BlendMode, Corners, Event, Mode, Stream, Input, Image, Audio, Cursor, Voice, Command, gpu.Pass, gpu.Pipeline, gpu.Draw, gpu.Dispatch, gpu.PostProcess, Host, Host.VTable, Context, Field, Layout, Info, Memory, Sketch }) |T| { h = std.hash.Wyhash.hash(h, @typeName(T) ++ layout.signature(T)); } break :blk h; diff --git a/src/gpu/Executor.zig b/src/gpu/Executor.zig index 412d0ed..460b2d5 100644 --- a/src/gpu/Executor.zig +++ b/src/gpu/Executor.zig @@ -155,6 +155,8 @@ const Target = struct { msaa_fresh: bool = true, stencil_samples: u8 = 0, stencil: ?*c.SDL_GPUTexture = null, + depth_valid: bool = false, + depth_sampleable: bool = false, }; pub fn init(gpa: std.mem.Allocator, device: *c.SDL_GPUDevice) Gpu { @@ -229,25 +231,41 @@ pub fn run(g: *Gpu, pass: abi.gpu.Pass) !void { /// Records into the caller's command buffer; the caller submits it. /// Uploads cycle their buffers, allowing several passes in one frame. pub fn record(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, pass: abi.gpu.Pass) !void { - return g.recordWithInput(cmd, pass, null); + return g.recordWithInput(cmd, pass, &.{}); +} + +pub fn runPostProcess(g: *Gpu, source: abi.Image, target: abi.Image, post: abi.gpu.PostProcess) !void { + const start = g.startTiming(); + const cmd = c.SDL_AcquireGPUCommandBuffer(g.device) orelse return error.GpuCommandBuffer; + errdefer _ = c.SDL_CancelGPUCommandBuffer(cmd); + try g.postProcess(cmd, source, target, post); + try g.submit(cmd, "postprocess", start); } /// A second pass on the same command buffer; input and output must differ. pub fn postProcess(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, source: abi.Image, target: abi.Image, post: abi.gpu.PostProcess) !void { if (source == target) return error.FeedbackLoop; - const input = g.output(source) orelse return error.MissingImage; + const input = g.targets.get(source) orelse return error.MissingImage; + var textures: [2]*c.SDL_GPUTexture = undefined; + textures[0] = input.texture; + if (post.sample_depth) { + if (!input.depth_valid or input.stencil == null) return error.MissingDepthAttachment; + if (input.stencil_samples != 1) return error.MultisampleDepthUnsupported; + if (!input.depth_sampleable) return error.DepthSamplingUnsupported; + textures[1] = input.stencil.?; + } try g.recordWithInput(cmd, .{ .image = target, .width = input.width, .height = input.height, .pipelines = &.{.{ .vertex = post.vertex, .fragment = post.fragment, .stride = 0, .attributes = &.{}, .blend = .none }}, .draws = &.{.{ .pipeline = 0, .vertex_count = 3, .fragment_uniforms = post.fragment_uniforms }}, - }, input.texture); + }, textures[0 .. 1 + @as(usize, @intFromBool(post.sample_depth))]); } -fn recordWithInput(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, pass: abi.gpu.Pass, input: ?*c.SDL_GPUTexture) !void { +fn recordWithInput(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, pass: abi.gpu.Pass, input: []const *c.SDL_GPUTexture) !void { if (pass.width == 0 or pass.height == 0 or pass.draws.len == 0 and pass.clear == null) return; - if (input != null and g.post_sampler == null) { + if (input.len != 0 and g.post_sampler == null) { g.post_sampler = c.SDL_CreateGPUSampler(g.device, &std.mem.zeroInit(c.SDL_GPUSamplerCreateInfo, .{ .min_filter = c.SDL_GPU_FILTER_NEAREST, .mag_filter = c.SDL_GPU_FILTER_NEAREST, @@ -269,7 +287,7 @@ fn recordWithInput(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, pass: abi.gpu.Pass, in if (d.pipeline >= pass.pipelines.len) return error.BadPipeline; const p = pass.pipelines[d.pipeline]; if ((p.depth_test != null or p.depth_write) and !pass.depth) return error.MissingDepthAttachment; - if (built[d.pipeline] == null) built[d.pipeline] = try g.pipeline(pass.pipelines[d.pipeline], samples, pass.stencil or pass.depth, @intFromBool(input != null)); + if (built[d.pipeline] == null) built[d.pipeline] = try g.pipeline(pass.pipelines[d.pipeline], samples, pass.stencil or pass.depth, @intCast(input.len)); } // Every draw's vertices go up in one copy. @@ -314,13 +332,14 @@ fn recordWithInput(g: *Gpu, cmd: *c.SDL_GPUCommandBuffer, pass: abi.gpu.Pass, in depth.texture = t.stencil; depth.load_op = if (pass.depth) c.SDL_GPU_LOADOP_CLEAR else c.SDL_GPU_LOADOP_DONT_CARE; depth.clear_depth = 1; - depth.store_op = c.SDL_GPU_STOREOP_DONT_CARE; + depth.store_op = if (pass.depth) c.SDL_GPU_STOREOP_STORE else c.SDL_GPU_STOREOP_DONT_CARE; depth.stencil_load_op = c.SDL_GPU_LOADOP_CLEAR; depth.stencil_store_op = c.SDL_GPU_STOREOP_DONT_CARE; depth.clear_stencil = 0; } const rp = c.SDL_BeginGPURenderPass(cmd, &color, 1, if (pass.stencil or pass.depth) &depth else null) orelse return error.GpuRenderPass; - if (input) |texture_| c.SDL_BindGPUFragmentSamplers(rp, 0, &.{ .texture = texture_, .sampler = g.post_sampler.? }, 1); + t.depth_valid = pass.depth; + for (input, 0..) |texture_, slot| c.SDL_BindGPUFragmentSamplers(rp, @intCast(slot), &.{ .texture = texture_, .sampler = g.post_sampler.? }, 1); var bound: ?*c.SDL_GPUGraphicsPipeline = null; // State already set in this pass, so draws that repeat it skip the call. var vertex_uniforms: ?[]const u8 = null; @@ -575,9 +594,13 @@ fn renderTarget(g: *Gpu, image: abi.Image, w: u32, h: u32, samples: u8, stencil: t.msaa_fresh = true; } if (stencil and (t.stencil == null or t.stencil_samples != samples)) { + const sampled_usage = c.SDL_GPU_TEXTUREUSAGE_DEPTH_STENCIL_TARGET | c.SDL_GPU_TEXTUREUSAGE_SAMPLER; + const sampleable = samples == 1 and c.SDL_GPUTextureSupportsFormat(g.device, g.stencil_format, c.SDL_GPU_TEXTURETYPE_2D, sampled_usage); + const replacement = try g.texture(w, h, g.stencil_format, if (sampleable) sampled_usage else c.SDL_GPU_TEXTUREUSAGE_DEPTH_STENCIL_TARGET, samples); if (t.stencil) |s| c.SDL_ReleaseGPUTexture(g.device, s); - t.stencil = try g.texture(w, h, g.stencil_format, c.SDL_GPU_TEXTUREUSAGE_DEPTH_STENCIL_TARGET, samples); + t.stencil = replacement; t.stencil_samples = samples; + t.depth_sampleable = sampleable; } return t; } diff --git a/src/gpu/scene.zig b/src/gpu/scene.zig index e5ff3d0..d5e1db5 100644 --- a/src/gpu/scene.zig +++ b/src/gpu/scene.zig @@ -2,6 +2,7 @@ //! Version 1: "SGPU", u32 version, then Pass fields in declaration order. //! Version 2 appends one fullscreen PostProcess sampling that pass's output. //! Version 3 adds depth fields and an optional PostProcess after the pass. +//! Version 4 adds optional depth sampling to PostProcess. //! Integers/enums/f32 are little endian; slices have a u32 count, optionals //! and bools a byte (0 or 1). No pointers, native padding, or device handles. //! Storage buffers are not part of this version: each frame stands alone. @@ -22,11 +23,12 @@ pub fn encode(w: *std.Io.Writer, pass: abi.gpu.Pass) !void { pub fn encodePostProcess(w: *std.Io.Writer, pass: abi.gpu.Pass, post: abi.gpu.PostProcess) !void { try validate(pass); try validatePostProcess(post); + if (post.sample_depth and (!pass.depth or pass.samples != 1)) return error.BadScene; try w.writeAll("SGPU"); - const version: u32 = if (pass.depth) 3 else 2; + const version: u32 = if (post.sample_depth) 4 else if (pass.depth) 3 else 2; try w.writeInt(u32, version, .little); try put(w, pass, version); - if (version == 3) try put(w, @as(?abi.gpu.PostProcess, post), version) else try put(w, post, version); + if (version >= 3) try put(w, @as(?abi.gpu.PostProcess, post), version) else try put(w, post, version); } /// Allocations belong to the caller's frame arena. Never run unvalidated @@ -42,13 +44,16 @@ pub fn decodeFrame(arena: std.mem.Allocator, bytes: []const u8) !Frame { var r: std.Io.Reader = .fixed(bytes); if (!std.mem.eql(u8, try r.take(4), "SGPU")) return error.BadScene; const version = try r.takeInt(u32, .little); - if (version < 1 or version > 3) return error.BadScene; + if (version < 1 or version > 4) return error.BadScene; var budget: usize = max_bytes * 4; const pass = try get(abi.gpu.Pass, arena, &r, &budget, version); - const post: ?abi.gpu.PostProcess = if (version == 3) try get(?abi.gpu.PostProcess, arena, &r, &budget, version) else if (version == 2) try get(abi.gpu.PostProcess, arena, &r, &budget, version) else null; + const post: ?abi.gpu.PostProcess = if (version >= 3) try get(?abi.gpu.PostProcess, arena, &r, &budget, version) else if (version == 2) try get(abi.gpu.PostProcess, arena, &r, &budget, version) else null; if (r.end != r.seek) return error.BadScene; try validate(pass); - if (post) |p| try validatePostProcess(p); + if (post) |p| { + try validatePostProcess(p); + if (p.sample_depth and (!pass.depth or pass.samples != 1)) return error.BadScene; + } return .{ .pass = pass, .post = post }; } @@ -57,6 +62,11 @@ fn depthField(comptime T: type, comptime name: []const u8) bool { (T == abi.gpu.Pipeline and (std.mem.eql(u8, name, "depth_test") or std.mem.eql(u8, name, "depth_write"))); } +fn absentField(comptime T: type, comptime name: []const u8, version: u32) bool { + return (version < 3 and depthField(T, name)) or + (version < 4 and T == abi.gpu.PostProcess and std.mem.eql(u8, name, "sample_depth")); +} + fn put(w: *std.Io.Writer, value: anytype, version: u32) anyerror!void { const T = @TypeOf(value); switch (@typeInfo(T)) { @@ -74,7 +84,7 @@ fn put(w: *std.Io.Writer, value: anytype, version: u32) anyerror!void { for (value) |v| try put(w, v, version); }, .@"struct" => |s| inline for (s.field_names) |name| { - if (version >= 3 or !depthField(T, name)) try put(w, @field(value, name), version); + if (!absentField(T, name, version)) try put(w, @field(value, name), version); }, else => @compileError("unsupported scene field"), } @@ -117,7 +127,7 @@ fn get(comptime T: type, a: std.mem.Allocator, r: *std.Io.Reader, budget: *usize .@"struct" => |s| { var out: T = undefined; inline for (s.field_names, s.field_types) |name, F| { - @field(out, name) = if (version < 3 and comptime depthField(T, name)) std.mem.zeroes(F) else try get(F, a, r, budget, version); + @field(out, name) = if (absentField(T, name, version)) std.mem.zeroes(F) else try get(F, a, r, budget, version); } return out; }, @@ -227,7 +237,7 @@ test "GPU postprocess keeps v1 readable and validates the complete second pass" bad = post; bad.fragment.storage_buffers = 1; try std.testing.expectError(error.BadShader, validatePostProcess(bad)); - out.written()[4] = 4; + out.written()[4] = 5; try std.testing.expectError(error.BadScene, decodeFrame(arena.allocator(), out.written())); } @@ -261,3 +271,32 @@ test "depth uses v3 and requires an attachment; legacy frames keep depth disable const legacy = try decode(arena.allocator(), out.written()); try std.testing.expect(!legacy.depth and legacy.pipelines[0].depth_test == null and !legacy.pipelines[0].depth_write); } + +test "depth postprocess uses v4 and rejects missing or multisampled source depth" { + const code = std.mem.asBytes(&[_]u32{ 0x07230203, 0, 0, 0, 0 }); + const pipelines = [_]abi.gpu.Pipeline{.{ .vertex = .{ .code = code }, .fragment = .{ .code = code }, .stride = 0, .attributes = &.{} }}; + var pass: abi.gpu.Pass = .{ .image = .none, .width = 32, .height = 24, .depth = true, .pipelines = &pipelines, .draws = &.{.{ .pipeline = 0, .vertex_count = 3 }} }; + var post: abi.gpu.PostProcess = .{ .sample_depth = true, .vertex = .{ .code = code }, .fragment = .{ .code = code } }; + var out: std.Io.Writer.Allocating = .init(std.testing.allocator); + defer out.deinit(); + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + try encodePostProcess(&out.writer, pass, post); + try std.testing.expectEqual(4, out.written()[4]); + try std.testing.expect((try decodeFrame(arena.allocator(), out.written())).post.?.sample_depth); + for (0..out.written().len) |i| { + _ = decodeFrame(arena.allocator(), out.written()[0..i]) catch continue; + return error.AcceptedTruncatedScene; + } + pass.depth = false; + try std.testing.expectError(error.BadScene, encodePostProcess(&out.writer, pass, post)); + pass.depth = true; + pass.samples = 4; + try std.testing.expectError(error.BadScene, encodePostProcess(&out.writer, pass, post)); + pass.samples = 1; + post.sample_depth = false; + out.clearRetainingCapacity(); + try encodePostProcess(&out.writer, pass, post); + try std.testing.expectEqual(3, out.written()[4]); + try std.testing.expect(!(try decodeFrame(arena.allocator(), out.written())).post.?.sample_depth); +} diff --git a/src/host/Host.zig b/src/host/Host.zig index 745d9a1..035a004 100644 --- a/src/host/Host.zig +++ b/src/host/Host.zig @@ -447,6 +447,7 @@ fn draw(self: *Host, now: u64, started: u64) !u64 { .audio => |a| self.audio.push(a.key, a.samples, a.clear), .quit => self.quit = true, .gpu => |pass| if (self.passes) |*g| self.runGpu(g, pass) catch |err| self.print("gpu pass: {t} ({s})", .{ err, c.SDL_GetError() }), + .post_process => |p| if (self.passes) |*g| self.runPostProcess(g, p.source, p.target, p.effect) catch |err| self.print("gpu postprocess: {t} ({s})", .{ err, c.SDL_GetError() }), .upload => |u| if (self.passes) |*g| g.write(u.buffer, u.offset, u.bytes) catch |err| self.print("gpu upload: {t} ({s})", .{ err, c.SDL_GetError() }), .compute => |d| if (self.passes) |*g| g.compute(d) catch |err| self.print("gpu compute: {t} ({s})", .{ err, c.SDL_GetError() }), .microphone => self.audio.listen(), @@ -981,6 +982,15 @@ fn apiEnv(ctx: *anyopaque, name: []const u8) ?[]const u8 { return hostOf(ctx).environ.get(name); } +fn runPostProcess(self: *Host, g: *Gpu, source: abi.Image, target: abi.Image, post: abi.gpu.PostProcess) !void { + const input = g.output(source) orelse return error.MissingImage; + const before = g.output(target); + const changed = if (before) |t| t.width != input.width or t.height != input.height else true; + if (changed) self.images.unwrap(target); + try g.runPostProcess(source, target, post); + if (g.output(target)) |t| try self.images.wrap(target, t.texture, t.width, t.height); +} + fn runGpu(self: *Host, g: *Gpu, pass: abi.gpu.Pass) !void { const before = g.output(pass.image); const changed = if (before) |t| t.width != pass.width or t.height != pass.height else true; diff --git a/src/sketch/clock.zig b/src/sketch/clock.zig index e64e2a8..2d63e12 100644 --- a/src/sketch/clock.zig +++ b/src/sketch/clock.zig @@ -85,7 +85,7 @@ test "N updates of one tick each step exactly as one update of N ticks" { var sum: u32 = 0; for (per) |n| sum += n; try std.testing.expectEqual(sum, total); - try std.testing.expectEqual(std.mem.toBytes(one), std.mem.toBytes(all)); + try std.testing.expectEqualDeep(one, all); try std.testing.expectEqual(@as(u32, Clock.max_steps), per[300]); try std.testing.expectEqual(@as(u64, total), all.count); } diff --git a/src/sketch/root.zig b/src/sketch/root.zig index 434478c..cb20ab9 100644 --- a/src/sketch/root.zig +++ b/src/sketch/root.zig @@ -232,6 +232,12 @@ pub const Frame = struct { kept.draws = draws; f.emit(.{ .gpu = kept }); } + /// Samples the source color and optionally depth into a distinct image. + pub fn postProcess(f: *Frame, source: Image, destination: Image, post: abi.gpu.PostProcess) void { + var kept = post; + kept.fragment_uniforms = f.keep(u8, post.fragment_uniforms); + f.emit(.{ .post_process = .{ .source = source, .target = destination, .effect = kept } }); + } /// `bytes` into a buffer made by `Host.newBuffer`. pub fn upload(f: *Frame, buffer: abi.Buffer, offset: u32, bytes: []const u8) void { f.emit(.{ .upload = .{ .buffer = buffer, .offset = offset, .bytes = bytes } }); @@ -469,3 +475,24 @@ test "setup receives session resources without a Frame and reports errors" { try std.testing.expectEqual(@as(u32, 1), session.calls); try std.testing.expect(seen); } + +test "postprocess preserves image handles and copies transient uniforms" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + var commands: std.ArrayList(abi.Command) = .empty; + var ctx: abi.Context = undefined; + ctx.commands = &commands; + var frame: Frame = undefined; + frame.ctx = &ctx; + frame.scratch = arena.allocator(); + var uniforms = [_]u8{ 1, 2, 3, 4 }; + const source: Image = @fromBackingInt(@intCast(1)); + const target: Image = @fromBackingInt(@intCast(2)); + frame.postProcess(source, target, .{ .vertex = .{ .code = "vertex" }, .fragment = .{ .code = "fragment" }, .sample_depth = true, .fragment_uniforms = &uniforms }); + uniforms[0] = 9; + const post = commands.items[0].post_process; + try std.testing.expectEqual(source, post.source); + try std.testing.expectEqual(target, post.target); + try std.testing.expect(post.effect.sample_depth); + try std.testing.expectEqualSlices(u8, &.{ 1, 2, 3, 4 }, post.effect.fragment_uniforms); +} diff --git a/src/sketch/web.zig b/src/sketch/web.zig index 682bff6..937b491 100644 --- a/src/sketch/web.zig +++ b/src/sketch/web.zig @@ -43,6 +43,7 @@ const js = struct { /// A `Dispatch` or a `Pass`, described in JSON (see `describe`). extern "gpu" fn compute(ptr: [*]const u8, len: usize) void; extern "gpu" fn pass(ptr: [*]const u8, len: usize) void; + extern "gpu" fn postProcess(ptr: [*]const u8, len: usize) void; /// Answers through `sk_download`. extern "gpu" fn download(id: u32, offset: u32, size: u32, key: u32) void; extern "audio" fn rate() u32; @@ -234,6 +235,10 @@ fn Driver(comptime App: type) type { const json = describe(v); js.pass(json.ptr, json.len); }, + .post_process => |v| { + const json = describe(v); + js.postProcess(json.ptr, json.len); + }, .download => |v| js.download(@backingInt(v.buffer), v.offset, v.size, v.key), .open_file => |v| js.openFile(v.command.ptr, v.command.len, v.pattern.ptr, v.pattern.len), // Synthesised voices are not on the web yet; quitting a page diff --git a/web/host.js b/web/host.js index 7559dfc..efc335d 100644 --- a/web/host.js +++ b/web/host.js @@ -176,7 +176,7 @@ const gpu = { // Multisample and stencil attachments, kept per image. attachment(entry, kind, format, samples) { const key = `${kind} ${samples}`; - if (!entry.attachments[key]) entry.attachments[key] = gpu.device.createTexture({size: [entry.w, entry.h], format, sampleCount: samples, usage: GPUTextureUsage.RENDER_ATTACHMENT}); + if (!entry.attachments[key]) entry.attachments[key] = gpu.device.createTexture({size: [entry.w, entry.h], format, sampleCount: samples, usage: GPUTextureUsage.RENDER_ATTACHMENT | (kind === 'stencil' && samples === 1 ? GPUTextureUsage.TEXTURE_BINDING : 0)}); return entry.attachments[key]; }, // Copies the latest render into the image's canvas, once a frame, for @@ -234,9 +234,11 @@ const gpu = { }; const depthStencil = p.stencil || p.depth ? { view: gpu.attachment(entry, 'stencil', 'depth24plus-stencil8', samples).createView(), - depthLoadOp: 'clear', depthClearValue: 1, depthStoreOp: 'discard', + depthLoadOp: 'clear', depthClearValue: 1, depthStoreOp: p.depth ? 'store' : 'discard', stencilLoadOp: 'clear', stencilClearValue: 0, stencilStoreOp: 'discard', } : undefined; + entry.depthValid = p.depth; + entry.depthSamples = samples; // Every draw's bytes are written before the pass is submitted. const draws = p.draws.map(d => ({ d, @@ -263,6 +265,52 @@ const gpu = { gpu.device.queue.submit([commands.finish()]); }, + // Fragment group 2: color texture/sampler at 0/1, optional depth at 2/3. + postProcess({source, target, effect}) { + if (source === target) throw new Error('FeedbackLoop'); + const input = images[source]; + if (!input?.texture) throw new Error('MissingImage'); + if (effect.sample_depth && !input.depthValid) throw new Error('MissingDepthAttachment'); + if (effect.sample_depth && input.depthSamples !== 1) throw new Error('MultisampleDepthUnsupported'); + const output = gpu.target(target, input.w, input.h); + if (!output) return; + const key = `post ${effect.vertex.hash} ${effect.fragment.hash} ${effect.sample_depth}`; + let pipeline = gpu.pipelines.get(key); + if (!pipeline) { + const entries = [ + {binding: 0, visibility: GPUShaderStage.FRAGMENT, texture: {sampleType: 'float'}}, + {binding: 1, visibility: GPUShaderStage.FRAGMENT, sampler: {type: 'non-filtering'}}, + ]; + if (effect.sample_depth) entries.push( + {binding: 2, visibility: GPUShaderStage.FRAGMENT, texture: {sampleType: 'unfilterable-float'}}, + {binding: 3, visibility: GPUShaderStage.FRAGMENT, sampler: {type: 'non-filtering'}}, + ); + const layouts = [[], [], entries, effect.fragment.uniform_buffers ? [{binding: 0, visibility: GPUShaderStage.FRAGMENT, buffer: {type: 'uniform'}}] : []]; + pipeline = gpu.device.createRenderPipeline({ + layout: gpu.device.createPipelineLayout({bindGroupLayouts: layouts.map(entries => gpu.device.createBindGroupLayout({entries}))}), + vertex: {module: gpu.module(effect.vertex)}, + fragment: {module: gpu.module(effect.fragment), targets: [{format: 'rgba8unorm'}]}, + primitive: {topology: 'triangle-list'}, + }); + gpu.pipelines.set(key, pipeline); + } + gpu.postSampler ??= gpu.device.createSampler({minFilter: 'nearest', magFilter: 'nearest'}); + const entries = [{binding: 0, resource: input.texture.createView()}, {binding: 1, resource: gpu.postSampler}]; + if (effect.sample_depth) entries.push( + {binding: 2, resource: input.attachments['stencil 1'].createView({aspect: 'depth-only'})}, + {binding: 3, resource: gpu.postSampler}, + ); + const uniforms = gpu.uniforms(effect.fragment_uniforms); + const commands = gpu.device.createCommandEncoder(); + const pass = commands.beginRenderPass({colorAttachments: [{view: output.texture.createView(), loadOp: 'clear', storeOp: 'store'}]}); + pass.setPipeline(pipeline); + gpu.bind(pass, pipeline, [[], [], entries, gpu.uniform(uniforms)]); + pass.draw(3); + pass.end(); + gpu.device.queue.submit([commands.finish()]); + output.depthValid = false; + }, + // Reads a buffer back; it reaches the sketch before a later frame. arrived: [], download(id, offset, size, key) { @@ -604,6 +652,7 @@ const imports = { }, compute(ptr, len) { if (gpu.device) gpu.compute(JSON.parse(string(ptr, len))); }, pass(ptr, len) { if (gpu.device) gpu.pass(JSON.parse(string(ptr, len))); }, + postProcess(ptr, len) { if (gpu.device) gpu.postProcess(JSON.parse(string(ptr, len))); }, download(id, offset, size, key) { if (gpu.buffers[id]) gpu.download(id, offset, size, key); }, }, audio: { |
