diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 39 |
1 files changed, 33 insertions, 6 deletions
@@ -125,7 +125,7 @@ Session (a trail in an offscreen image) is stale then; repaint it from State. `draw` gets a `Frame`: `clear`, `fill`, `shape`, `shapeColors`, `image`, -`imageAt`, `target`, `blend`, `text`, `print`, `clip`, `canvas`, `cursor`, `voice`, `gain`, `audio`, `microphone`, `pixels`, `gpu`, +`imageAt`, `target`, `blend`, `text`, `print`, `clip`, `canvas`, `cursor`, `voice`, `gain`, `audio`, `audioBuffer`, `microphone`, `pixels`, `gpu`, `compute`, `upload`, `download`, `openFile`, `run`, `quit`, plus the three allocators and `f.host` (`loadImage`, `decodeImage`, `newImage`, `newBuffer`, `loadSound`, `decodeSound`, `newSound`, `samples`, `readFile`, `env`, `log`, `textSize`). Commands may be built @@ -174,7 +174,8 @@ Store holds, say). `f.download(buffer, .{ .size = n, .key = k })` reads a GPU buffer back without stalling: the bytes arrive in a later frame's `in.downloads`, a frame or two behind. For measurements a shader made (a pitch, a level), -not for pixels. +not for pixels; sound a shader made goes to the speakers with +`f.audioBuffer` (see sound, below), which does not wait a frame. Geometry and colors are duck-typed. Anything with `x`/`y` is a point, anything that also has `w`/`h` (or `width`/`height`) is a rect, in any number type, and @@ -449,8 +450,31 @@ topping its queue up to 2048 samples (about 43 ms) ahead of the device, so what `draw` declares is heard up to that much later. `f.audio(key, samples, .{})` streams PCM, and `input.stream(key)` reports -what the host has queued and played: the sketch chooses how far ahead it -keeps a stream. `f.microphone()` keeps the default recording device open +what the host has queued and played, and its `underruns` (pushes that found +it empty though the frame before had pushed to it: each a gap). The sketch +chooses how far ahead it keeps a stream; `input.lead` is how far the host +would: the slowest of the last 64 frames' worth of samples (a stream is fed +once a frame), plus the device's pull (what it takes from a stream at once, +1024 samples by SDL's default), plus half a pull for each recent underrun. +A sketch that makes its sound as it goes pushes `in.lead -| +in.stream(key).queued` a frame, and a change it makes is heard after what +is queued ahead of it, about a pull: 19 to 27 ms after `draw` on this +machine at 60 Hz. `SDL_AUDIO_DEVICE_SAMPLE_FRAMES=256` in the environment +asks SDL for 5 ms pulls instead, and a change leaves in about 6.5 ms (on +PipeWire every client then runs at that size). `f.audioBuffer(key, buffer, .{ .count += n })` streams `n` samples a compute shader wrote (f32 from `.offset` +bytes): the host reads them back within the frame, waiting for the GPU's +work so far (about 0.2 ms), and queues them. A GPU synth is then: + +```zig +const n: u32 = @min(f.in.lead -| f.in.stream(1).queued, max_block); +if (n > 0) { + f.compute(.{ .kernel = synth, .readwrite = &.{ss.block}, .uniforms = ..., .groups = .{ (n + 63) / 64, 1, 1 } }); + f.audioBuffer(1, ss.block, .{ .count = n }); +} +``` + +(`examples/spheres` is one.) `f.microphone()` keeps the default recording device open while declared, and the next frame's `input.microphone` holds what it heard (to `draw` only: it is a device, not an event, so it does not replay). `host.samples(audio)` gives a loaded sound's samples, mono at @@ -1109,8 +1133,11 @@ Add `.sketchbook = .{ .path = "..." }` to your `build.zig.zon`. silent there, whatever its sound (loaded, decoded or made with `newSound`), and so are groups' gains; `decodeSound` gives `.none`, and `loadSound` knows only sounds the page decoded (opened or dropped - files). What does sound: streams (`f.audio`) and the microphone, mixed - on the main thread (a ScriptProcessor), so a busy page can crackle. + files). What does sound: streams (`f.audio`, `f.audioBuffer`) and the + microphone, mixed on the main thread (a ScriptProcessor), so a busy page + can crackle. A GPU buffer streamed there arrives a frame or two later + (WebGPU reads back only asynchronously), counted as queued meanwhile, and + `in.lead` is a fixed 100 ms. The reusable GPU executor and the pardes integration are described in [docs/pardes-rendering.md](docs/pardes-rendering.md), including the remaining |
