summaryrefslogtreecommitdiff
path: root/examples/micscope
diff options
context:
space:
mode:
Diffstat (limited to 'examples/micscope')
-rw-r--r--examples/micscope/README.md158
-rw-r--r--examples/micscope/build.zig54
-rw-r--r--examples/micscope/build.zig.zon10
-rw-r--r--examples/micscope/shaders/tuner.comp134
-rw-r--r--examples/micscope/shaders/tuner_difference.comp49
-rw-r--r--examples/micscope/src/layouts.zig37
-rw-r--r--examples/micscope/src/main.zig1299
-rw-r--r--examples/micscope/src/tuner.zig132
-rw-r--r--examples/micscope/test/README.md22
-rw-r--r--examples/micscope/test/profile.script167
-rw-r--r--examples/micscope/test/record.script23
-rw-r--r--examples/micscope/test/reference.flacbin0 -> 3841704 bytes
-rw-r--r--examples/micscope/test/render.script18
-rw-r--r--examples/micscope/test/sweep.wavbin0 -> 192044 bytes
-rw-r--r--examples/micscope/test/tones.wavbin0 -> 480044 bytes
-rw-r--r--examples/micscope/test/transport.script98
-rw-r--r--examples/micscope/test/tuner.script31
-rw-r--r--examples/micscope/web/icon-192.pngbin0 -> 4204 bytes
-rw-r--r--examples/micscope/web/icon-512.pngbin0 -> 9911 bytes
-rw-r--r--examples/micscope/web/icon.svg26
20 files changed, 2258 insertions, 0 deletions
diff --git a/examples/micscope/README.md b/examples/micscope/README.md
new file mode 100644
index 0000000..875ebdf
--- /dev/null
+++ b/examples/micscope/README.md
@@ -0,0 +1,158 @@
+# micscope
+
+Records the microphone or opens sound files, plays them forwards or
+backwards, and shows each as a waveform, its spectrogram on the same time
+axis, and the spectrum at the playhead. A [sketchbook](../..) sketch, Linux only for now.
+
+The window shows a vertical strip of these panels, laid out in a world
+and seen through a camera, as in a game. Takes stack downwards in the
+order they were made, so each keeps its place; at the bottom is now: what
+the microphone hears, its last 6 s scrolling by, with the spectrum of the
+newest samples. Recording turns that panel red and fills it with the new
+take; stopping leaves the take there and puts the live view under it. The
+camera follows the bottom until you pan away, and follows again once you
+pan back down. Zoomed out, it overviews the whole strip. Every take keeps
+its own time zoom and playhead.
+
+A tuner reads the live sound (or the recording, or the selected take at
+its playhead) and shows it on that panel's spectrum: a line at the pitch,
+ticks at its harmonics, the two semitones either side shaded, and that
+shaded span magnified across the top: the nearest notes of the 12-tone
+chromatic scale, the pitch as a needle among them (green within 5 cents,
+yellow within 15), and the note, cents and Hz. It is not a guitar tuner
+in particular: it finds 30 Hz to 4.2 kHz, a five-string bass's low B to a
+piccolo's top, against any A4 (`[` and `]`, 440 Hz by default).
+
+Panels are rendered at world resolution and the camera only scales them,
+so moving or zooming it recomputes nothing.
+
+The waveform, spectrogram and spectrum shaders are sketchbook's `scope`
+module's (`src/scope/shaders/`): the live view is a `scope.Scope`, and the
+take panels run `scope.kernels` and `scope.pipelines` over the tape. The
+tuner's are micscope's own.
+
+| Shader | Does |
+| --- | --- |
+| `spectrum.comp` | Blackman-Harris window around the playhead, zero-padded 2×, radix-2 Stockham FFT (8192 points, one workgroup), dBFS per bin, then one value per log-spaced pixel column: a Catmull-Rom spline through the bins where columns are narrower than a bin, the loudest bin where they span several. Smoothed in time (fast attack, slow release) with falling peaks. |
+| `waveform.comp` | Per pixel column: min, max and RMS of its samples, plus the interpolated signal at the column's edges, so zoomed past one sample per pixel the columns still join into a line. Recomputed only when the view or tape changes. |
+| `spectrogram.comp` | One workgroup per waveform column, so it lines up with the waveform in time. A 2048-sample Blackman-Harris window zero-padded to 4096, transformed as 2048 complex points in shared memory (the real-input trick) and split into 2049 bins. Below 500 Hz a second analysis runs on the signal low-passed and decimated 4×: the same 2048 points span 170 ms and resolve ~6 Hz instead of 23, three windows a quarter apart averaged (Welch) so noise does not speckle; blended with the first between 250 and 500 Hz. Zoomed out, each column averages up to four windows spread across its samples, so it neither skips nor flickers. Rows are log frequency, interpolated like the spectrum. |
+| `tuner_difference.comp`, `tuner.comp` | Pitch by YIN rather than by the strongest FFT peak, which is often a harmonic (low strings, voices and brass put more into the 2nd or 3rd) or missing altogether (a missing fundamental). The difference function d(τ) = Σ(x[j] − x[j−τ])² over 2048 samples for lags up to 1700, spread over 107 workgroups; then in one workgroup, normalised by its running mean (a parallel scan), the first dip under 0.12 (the shortest period that explains the signal, so no octave drops) followed to its bottom, and a parabola through the raw dip for the period between samples. Read back to the CPU (`f.download`), which follows the median of the last five clear estimates, restarts on a jump of more than a semitone, and only changes the note shown 60 cents away from it. |
+| `waveform.frag`, `spectrogram.frag`, `spectrum.frag` | Full-screen triangles that draw those buffers: the waveform and spectrum anti-aliased with grids, the spectrogram through an inferno heat gradient. |
+
+The takes share one tape buffer on the GPU (each analysis reads from an
+offset), uploaded once. A take's waveform and spectrogram are recomputed
+only when its view changes, and its panel redrawn only when something on
+it moved; only the selected take's spectrum runs every frame.
+
+The live view (the scope module's) uploads the microphone's last 1.4 s each frame and analyses
+only the columns that became complete, into ring buffers the fragment
+shaders read from the oldest column. A column waits until everything its
+spectrogram windows reach has been heard (6160 samples past its centre),
+so it is analysed once and final: the live spectrogram runs about 130 ms
+behind, the live spectrum does not.
+
+## Keys
+
+| Key | |
+| --- | --- |
+| R | record a new take / stop |
+| Space | play / pause the selected take, resuming in the last direction |
+| Right / Left | play forwards / backwards |
+| Home / End | seek to the start / end |
+| Up / Down | select the older / newer take |
+| Delete | delete the selected take |
+| O | open a file (WAV, FLAC, Ogg Vorbis) as a take, through the system dialog |
+| click, drag | select a take; on its timeline, seek |
+| wheel, middle or right drag, Page Up / Down | pan the camera |
+| Ctrl + wheel, or a pinch | zoom what is under the pointer: a take's time on its timeline (Z unzooms), the camera elsewhere |
+| 0 | the whole strip, or back to full size |
+| - / = | waveform gain |
+| [ / ] | the tuner's A4, 1 Hz down / up |
+| \` | the sketchbook console: `open <path>` opens a file too |
+
+The buttons (REC, PLAY, REV, OPEN, DEL, ALL, A4−, A4+) do what the keys
+do, for touch screens: there, one finger clicks and drags, two fingers
+pan, and a pinch zooms like ctrl+wheel.
+
+The canvas is the window, and the layout follows it. Wide, each panel has
+the spectrum beside the waveform and spectrogram, and the buttons sit in
+the header; narrower than 760 px (a phone held upright), the spectrum goes
+under them at full width, and the buttons along the bottom, where thumbs
+are.
+
+## Rendering reference
+
+`test/reference.flac` puts a mastered song, a city street and a sweep on one
+screen (sources and licences in `test/README.md`); `zig build test` plays it
+through `test/render.script` and leaves screenshots in `.zig-cache/micscope/`.
+
+What it settled for the spectrogram's low end, checked against a plain
+numpy STFT of the same columns:
+
+- One long window resolves the street's two engine hums but shows noise as
+ speckle full of holes: a single window's spectrum of noise scatters as
+ much as its level.
+- Multitaper averaging (sine tapers, DPSS with NW 2 to 4) calms the noise but
+ widens the hums into one slab and leaks into silence, which a 96 dB scale
+ shows. Wrong trade for lines.
+- Averaging three windows a quarter apart in time keeps the hums sharp and
+ the kicks distinct, and settles the noise. That is what it does.
+
+`test/tones.wav` has five notes where the strongest peak is not the note
+(a bass E1 and a guitar E2 louder in their 2nd harmonic, a D4 without its
+fundamental), an A4 12 cents sharp and a piccolo C7; `test/tuner.script`
+screenshots the tuner on each. All read within 0.1 cent.
+
+## GPU time
+
+`zig build profile` zooms the reference one step per frame, recomputing the
+waveform and spectrogram every frame, and prints GPU time per command
+(`--profile-gpu`: each command timed alone on an idle GPU). On an RTX 3050:
+
+| Case | GPU per frame |
+| --- | --- |
+| playing, views cached | ~0.45 ms |
+| zooming the 32 s reference | ~1.1 ms on average, spectrogram 1.5 ms on the frames it recomputes |
+| zooming a 10 minute tape | ~1.6 ms on average, spectrogram 2.1 ms (worst 4.6 ms) |
+| zooming the reference, in the strip with the live view and both tuners | ~1.15 ms on average; the live view adds a spectrum (~0.15 ms) and a few columns, each tuner ~0.1 ms |
+
+## Building
+
+Zig 0.17 and `glslc` (shaderc) on PATH; for the web build also `naga`
+(`cargo install naga-cli`), which translates the shaders to WGSL, and to
+serve it `miniserve` and `openssl`.
+
+ zig build run # with hot reload: `zig build lib --watch` alongside
+ zig build test # unit tests and the headless scripts in test/
+ zig build profile # GPU time per command while zooming
+ zig-out/bin/micscope # the standalone build
+ zig build serve # the web build at https://<this machine>:8042/
+ # (`zig build web --watch` alongside rebuilds it)
+
+## On the web
+
+The same sketch, compiled to wasm32-freestanding, on WebGPU: every shader
+runs as on native, translated to WGSL, and the tuner's readback is a
+WebGPU buffer mapped a frame or two later. Sound and the microphone go
+through Web Audio (the page asks for the microphone, and sound starts at
+the first click or key, as browsers require); OPEN decodes a file with the
+browser's own decoders. Takes last as long as the page.
+
+It is an installable app (a PWA): browsers offer to add it to the home
+screen, and once opened it works offline, from a copy its service worker
+keeps of the page, the code and the wasm. The icon is `web/icon.svg`,
+rendered to the PNGs beside it with `resvg -w N -h N icon.svg icon-N.png`. It needs a browser
+with WebGPU (Chrome, Edge, Safari 26; Firefox on Linux with
+`dom.webgpu.enabled` set in about:config), and a secure context, which
+`zig build serve`'s https gives it on any device on the network once the
+browser has accepted its self-signed certificate (miniserve prints the
+addresses as QR codes, for phones).
+
+Each take is a file of its samples (mono 32-bit floats at the host's
+rate), `micscope.take-<id>.f32` beside the save (in `.sketchbook/`
+under `zig build run`), written as it records (`Host.appendFile`). State
+lists the takes, so they are still there after a restart. A take starts
+its file empty (`Host.writeFile`). A deleted take's file stays: a seek
+back, or any mark from before the delete, finds it again. Takes stop at ten minutes; the
+strip holds 64. A `micscope.state.store` from before is not read: its takes
+stay in it.
diff --git a/examples/micscope/build.zig b/examples/micscope/build.zig
new file mode 100644
index 0000000..2de639d
--- /dev/null
+++ b/examples/micscope/build.zig
@@ -0,0 +1,54 @@
+const std = @import("std");
+const sketchbook = @import("sketchbook");
+
+pub fn build(b: *std.Build) void {
+ const target = b.standardTargetOptions(.{});
+ const optimize = b.standardOptimizeOption(.{});
+ const sb = sketchbook.init(b, .{ .target = target, .optimize = optimize });
+
+ // The tuner's shaders, as SPIR-V for the native host and WGSL for the
+ // web; their uniform blocks come from the Zig structs in src/layouts.zig.
+ // The waveform, spectrogram and spectrum are the scope module's.
+ const layouts = b.createModule(.{ .root_source_file = b.path("src/layouts.zig") });
+ const shaders = sb.addShaders(.{
+ .layouts = .{ .module = layouts, .types = &.{"TunerParams"} },
+ .sources = &.{
+ b.path("shaders/tuner_difference.comp"),
+ b.path("shaders/tuner.comp"),
+ },
+ });
+
+ const app = sb.app(.{
+ .name = "micscope",
+ .root_source_file = b.path("src/main.zig"),
+ .imports = &.{
+ .{ .name = "layouts", .module = layouts },
+ .{ .name = "shaders", .module = shaders.module },
+ .{ .name = "scope", .module = sb.modules.scope },
+ },
+ });
+ const steps = app.addSteps(.{
+ // Installable, and it opens offline (icons from web/icon.svg).
+ .web = .{ .pwa = .{
+ .description = "Records the microphone, shows its waveform, spectrogram and spectrum, and tunes any instrument.",
+ .background = "#0b0c0f",
+ .theme = "#0b0c0f",
+ .icons = &.{
+ .{ .source = b.path("web/icon-192.png"), .size = 192 },
+ .{ .source = b.path("web/icon-512.png"), .size = 512 },
+ },
+ } },
+ .scripts = &.{ b.path("test/transport.script"), b.path("test/record.script"), b.path("test/render.script"), b.path("test/tuner.script") },
+ });
+
+ // GPU time per command while zooming, which recomputes every frame.
+ const profile = b.addRunArtifact(steps.exe);
+ profile.addArgs(&.{ "--headless", "--fresh", "--profile-gpu", "--script" });
+ profile.addFileArg(b.path("test/profile.script"));
+ profile.addArg("--state");
+ _ = profile.addOutputFileArg("profile.state");
+ profile.setCwd(b.path("."));
+ profile.has_side_effects = true;
+ profile.stdio = .inherit;
+ b.step("profile", "Print GPU time per command while zooming the reference").dependOn(&profile.step);
+}
diff --git a/examples/micscope/build.zig.zon b/examples/micscope/build.zig.zon
new file mode 100644
index 0000000..9c595cf
--- /dev/null
+++ b/examples/micscope/build.zig.zon
@@ -0,0 +1,10 @@
+.{
+ .name = .micscope,
+ .version = "0.0.1",
+ .dependencies = .{
+ .sketchbook = .{ .path = "../.." },
+ },
+ .minimum_zig_version = "0.17.0",
+ .paths = .{""},
+ .fingerprint = 0x780fbc0f374f0ce0,
+}
diff --git a/examples/micscope/shaders/tuner.comp b/examples/micscope/shaders/tuner.comp
new file mode 100644
index 0000000..1907f7f
--- /dev/null
+++ b/examples/micscope/shaders/tuner.comp
@@ -0,0 +1,134 @@
+#version 450
+#extension GL_GOOGLE_include_directive : require
+// The pitch of the newest (or the playhead's) sound, by YIN (de Cheveigné
+// and Kawahara, 2002), in one workgroup. Picking the strongest FFT peak
+// finds the loudest harmonic, which is often not the note: low strings,
+// voices and brass put more energy in the 2nd or 3rd harmonic, and a
+// missing fundamental has no peak at all. YIN looks for the period instead:
+//
+// 1. the difference function d(tau) = sum over the window of
+// (x[j] - x[j + tau])^2, which dips to 0 at every multiple of the
+// period, whatever the harmonics' balance;
+// 2. normalised by its running mean, d'(tau) = d(tau) * tau / sum d(1..tau),
+// which starts at 1 and removes the dip-at-zero bias, so the dips are
+// comparable;
+// 3. the first dip below an absolute threshold, then the bottom of it: the
+// shortest period that explains the signal, which keeps it from
+// dropping an octave onto a multiple of the period;
+// 4. a parabola through that dip of d and its neighbours, for a period
+// between samples: the semitone is 6 %, the cent 0.06 %, and whole samples
+// alone would be 30 cents apart at 1 kHz.
+layout(local_size_x = 256) in;
+
+#include "TunerParams.glsl"
+
+layout(set = 0, binding = 0) readonly buffer Tape { float tape[]; };
+// d(tau), from tuner_difference.comp, which does the bulk of the work.
+layout(set = 0, binding = 1) readonly buffer Difference { float difference[]; };
+// freq, clarity, level, period: layouts.Pitch.
+layout(set = 1, binding = 0) buffer Result { vec4 result; };
+
+// Match layouts.zig.
+const uint W = 2048;
+const uint LAGS = 1700;
+const uint THREADS = 256;
+
+// Each thread owns CHUNK consecutive lags.
+const uint CHUNK = (LAGS + THREADS - 1u) / THREADS;
+shared float d[THREADS * CHUNK];
+// d before normalising: the parabola goes through these, since the
+// normalisation's factor tau tilts a dip by more than a cent at high pitch.
+shared float raw[THREADS * CHUNK];
+shared float squares[THREADS];
+shared float sums[THREADS];
+shared uint first_dip;
+shared float deepest[THREADS];
+shared uint deepest_at[THREADS];
+
+void main() {
+ uint t = gl_LocalInvocationID.x;
+ uint tau_max = min(LAGS - 2u, uint(ceil(u.rate / u.min_freq)));
+ uint tau_min = max(2u, uint(floor(u.rate / u.max_freq)));
+
+ // 1. The difference function (tuner_difference.comp), and the window's
+ // energy.
+ int first = int(u.end) - int(W);
+ float sq = 0.0;
+ for (uint j = t; j < W; j += THREADS) {
+ int at = first + int(j);
+ float v = at >= 0 && at < int(u.samples) ? tape[u.offset + uint(at)] : 0.0;
+ sq += v * v;
+ }
+ squares[t] = sq;
+ float own = 0.0;
+ for (uint i = 0u; i < CHUNK; i++) {
+ uint tau = t * CHUNK + i;
+ float v = tau >= 1u && tau <= tau_max + 1u ? difference[tau] : 0.0;
+ raw[tau] = v;
+ own += v;
+ }
+ sums[t] = own;
+ if (t == 0u) first_dip = 0xffffffffu;
+ barrier();
+
+ // 2. Normalised by the running sum: a scan of the threads' chunk sums
+ // (Hillis-Steele), then each chunk in order.
+ for (uint stride = 1u; stride < THREADS; stride <<= 1u) {
+ float add = t >= stride ? sums[t - stride] : 0.0;
+ barrier();
+ sums[t] += add;
+ barrier();
+ }
+ float running = t > 0u ? sums[t - 1u] : 0.0;
+ float low = 2.0;
+ uint low_at = 0u;
+ for (uint i = 0u; i < CHUNK; i++) {
+ uint tau = t * CHUNK + i;
+ running += raw[tau];
+ float v = tau == 0u ? 1.0 : (running > 0.0 ? raw[tau] * float(tau) / running : 1.0);
+ d[tau] = v;
+ if (tau >= tau_min && tau <= tau_max) {
+ // 3. The first dip under the threshold, and the deepest.
+ if (v < u.threshold) atomicMin(first_dip, tau);
+ if (v < low) {
+ low = v;
+ low_at = tau;
+ }
+ }
+ }
+ deepest[t] = low;
+ deepest_at[t] = low_at;
+ barrier();
+ for (uint stride = THREADS / 2u; stride > 0u; stride >>= 1u) {
+ if (t < stride && deepest[t + stride] < deepest[t]) {
+ deepest[t] = deepest[t + stride];
+ deepest_at[t] = deepest_at[t + stride];
+ }
+ barrier();
+ }
+ if (t != 0u) return;
+
+ float energy = 0.0;
+ for (uint i = 0u; i < THREADS; i++) energy += squares[i];
+ float level = energy > 0.0 ? 10.0 * log(energy / float(W)) / log(10.0) : -160.0;
+
+ // The first dip followed to its bottom; failing one, the deepest, which
+ // the CPU may still reject as unclear.
+ uint best = first_dip;
+ if (best == 0xffffffffu) {
+ best = max(deepest_at[0], tau_min);
+ } else {
+ while (best + 1u <= tau_max && d[best + 1u] < d[best]) best++;
+ }
+
+ // 4. The parabola through the dip.
+ float a = raw[best - 1u];
+ float b = raw[best];
+ float c = raw[best + 1u];
+ float bend = a - 2.0 * b + c;
+ float shift = bend > 0.0 ? clamp(0.5 * (a - c) / bend, -0.5, 0.5) : 0.0;
+ float period = float(best) + shift;
+ float depth = d[best];
+ bool silent = energy <= 0.0;
+ result = vec4(silent ? 0.0 : u.rate / period, silent ? 0.0 : clamp(1.0 - depth, 0.0, 1.0), level, period);
+}
diff --git a/examples/micscope/shaders/tuner_difference.comp b/examples/micscope/shaders/tuner_difference.comp
new file mode 100644
index 0000000..c7d1b6e
--- /dev/null
+++ b/examples/micscope/shaders/tuner_difference.comp
@@ -0,0 +1,49 @@
+#version 450
+#extension GL_GOOGLE_include_directive : require
+// YIN's difference function for tuner.comp, spread over many workgroups:
+// d(tau) = sum over the newest W samples of (x[j] - x[j - tau])^2. Each
+// workgroup takes LAGS_PER_GROUP lags, with THREADS / LAGS_PER_GROUP
+// threads summing every so-many term of each, then adding up.
+layout(local_size_x = 256) in;
+
+#include "TunerParams.glsl"
+
+layout(set = 0, binding = 0) readonly buffer Tape { float tape[]; };
+layout(set = 1, binding = 0) buffer Difference { float d[]; };
+
+// Match layouts.zig and tuner.comp.
+const uint W = 2048;
+const uint LAGS = 1700;
+const uint THREADS = 256;
+const uint LAGS_PER_GROUP = 16;
+const uint PER_LAG = THREADS / LAGS_PER_GROUP;
+
+shared float x[W + LAGS];
+shared float partial[THREADS];
+
+void main() {
+ uint t = gl_LocalInvocationID.x;
+ int first = int(u.end) - int(W + LAGS);
+ for (uint i = t; i < W + LAGS; i += THREADS) {
+ int at = first + int(i);
+ x[i] = at >= 0 && at < int(u.samples) ? tape[u.offset + uint(at)] : 0.0;
+ }
+ barrier();
+
+ uint tau = gl_WorkGroupID.x * LAGS_PER_GROUP + t / PER_LAG;
+ uint lane = t % PER_LAG;
+ float sum = 0.0;
+ if (tau < LAGS) {
+ for (uint j = lane; j < W; j += PER_LAG) {
+ float diff = x[LAGS + j] - x[LAGS + j - tau];
+ sum += diff * diff;
+ }
+ }
+ partial[t] = sum;
+ barrier();
+ for (uint stride = PER_LAG / 2u; stride > 0u; stride >>= 1u) {
+ if (lane < stride) partial[t] += partial[t + stride];
+ barrier();
+ }
+ if (lane == 0u && tau < LAGS) d[tau] = partial[t];
+}
diff --git a/examples/micscope/src/layouts.zig b/examples/micscope/src/layouts.zig
new file mode 100644
index 0000000..be95799
--- /dev/null
+++ b/examples/micscope/src/layouts.zig
@@ -0,0 +1,37 @@
+//! What the tuner's shaders read, as Zig structs: the only definition. The
+//! shaders include GLSL generated from these (see sketchbook's `glsl.zig`).
+//! No imports, so the shader build can compile this alone. The views' own
+//! blocks are the scope module's (`scope.layouts`).
+
+/// tuner.comp: the YIN window is the `tuner_window + tuner_lags` samples
+/// ending at `end`, read like the other analyses (`offset`, `samples`).
+pub const tuner_window = 2048;
+pub const tuner_lags = 1700;
+
+pub const TunerParams = extern struct {
+ offset: u32,
+ end: u32,
+ samples: u32,
+ rate: f32,
+ /// The pitch range searched.
+ min_freq: f32,
+ max_freq: f32,
+ /// YIN's absolute threshold on the normalised difference.
+ threshold: f32,
+ pad: u32 = 0,
+
+ pub const glsl = .{ .uniform = .{ .set = 2, .instance = "u" } };
+};
+
+/// What tuner.comp writes (std430, read back by the CPU).
+pub const Pitch = extern struct {
+ /// Hz; 0 when nothing periodic was found.
+ freq: f32,
+ /// 1 minus YIN's normalised difference at the period: 1 is perfectly
+ /// periodic.
+ clarity: f32,
+ /// RMS of the window, dBFS.
+ level: f32,
+ /// The period in samples, interpolated.
+ period: f32,
+};
diff --git a/examples/micscope/src/main.zig b/examples/micscope/src/main.zig
new file mode 100644
index 0000000..99aac76
--- /dev/null
+++ b/examples/micscope/src/main.zig
@@ -0,0 +1,1299 @@
+//! micscope: records the microphone or opens sound files, plays them
+//! forwards or backwards, and shows each as a waveform, its spectrogram on
+//! the same time axis, and the spectrum at its playhead. The window is a
+//! vertical strip: on top, what the microphone hears right now (or the take
+//! being recorded); under it every take, newest first. The signal
+//! processing (windowed FFTs, log-frequency interpolation, smoothing,
+//! waveform reduction) and all the drawing run on the GPU; the CPU only
+//! moves samples: microphone to tape, tape to the GPU once, tape to the
+//! speakers.
+//!
+//! Memory by lifetime: State is the transport, the selection and the views,
+//! the takes (their ids and names), changed by `update` from keys and the
+//! mouse, and replayable. A take's samples are a file of its own, written
+//! by `draw` as it records and by `open` (see `Host.appendFile`): they are
+//! still there after a restart. Session is this process's copy of the
+//! takes' samples, the playback cursor, the microphone's recent past and
+//! GPU objects. Transport changes and seeks are `sk.Request`s, which `draw`
+//! carries out once each.
+const std = @import("std");
+const sk = @import("sketch");
+const gpu = sk.abi.gpu;
+const scope = @import("scope");
+const layouts = scope.layouts;
+/// The tuner's blocks; its shaders are `shaders`.
+const tuner_layouts = @import("layouts");
+const shaders = @import("shaders");
+const tuner = @import("tuner.zig");
+
+pub const name = "micscope";
+pub const width = 1280;
+pub const height = 900;
+pub const commands = .{ "open", "next-id" };
+
+/// The longest take: ten minutes at the host's rate.
+const max_seconds = 600;
+const max_takes = 64;
+/// The playback stream's key.
+const stream = 1;
+/// How far ahead playback keeps the stream filled.
+const lead_seconds = 0.06;
+/// The live view's span.
+const live_seconds = 6;
+
+pub const Transport = enum { stopped, recording, forward, backward };
+
+/// The part of a take its timeline shows, as fractions of it.
+pub const View = struct {
+ lo: f32 = 0,
+ hi: f32 = 1,
+};
+
+pub const State = struct {
+ transport: sk.Request(Transport) = .{},
+ /// The direction space resumes in.
+ backward: bool = false,
+ /// To a fraction of the selected take.
+ seek: sk.Request(f32) = .{},
+ scrubbing: bool = false,
+ dialog: sk.Request(void) = .{},
+ /// Oldest first.
+ takes: sk.List(Take, max_takes) = .{},
+ /// The next take's id. `draw` moves it past take files already there
+ /// with the `next-id` command: other runs' (`--fresh` keeps the files),
+ /// or the future a seek went back from.
+ next_id: u32 = 1,
+ /// The take the transport and the keys work on, by index (oldest 0).
+ selected: u32 = 0,
+ views: [max_takes]View = @splat(.{}),
+ /// Maps the world (the panels, stacked) to the screen.
+ camera: sk.Camera = .{},
+ /// The camera keeps the bottom, where now is, in view.
+ follow: bool = true,
+ panning: bool = false,
+ /// The pointer's height when panning last moved.
+ grab: f32 = 0,
+ gain: f32 = 1,
+ /// The tuner's reference pitch: A4, in Hz.
+ a4: f32 = 440,
+};
+
+pub const Take = struct {
+ /// Names its file, `take-<id>.f32`: its mono samples at
+ /// `Input.sample_rate`. Stays the same when earlier takes are deleted:
+ /// Session and the GPU caches go by it.
+ id: u32 = 0,
+ /// A file's name, or empty for the microphone.
+ source: [64]u8 = @splat(0),
+ source_len: u32 = 0,
+};
+
+/// A take's samples in `Session.tape`.
+const Clip = struct {
+ id: u32 = 0,
+ start: usize = 0,
+ len: usize = 0,
+ /// Where the listener is, in samples into the take.
+ heard: f64 = 0,
+};
+
+pub const Session = struct {
+ transport: sk.Handled = .{},
+ seek: sk.Handled = .{},
+ dialog: sk.Handled = .{},
+ /// State's takes' samples, read from their files once a process, one
+ /// after another in the order read. A recording grows the last one.
+ tape: std.ArrayList(f32) = .empty,
+ clips: sk.List(Clip, max_takes) = .{},
+ /// The take playback follows, by id, and the next of its samples to
+ /// queue.
+ playing: u32 = 0,
+ cursor: usize = 0,
+ /// The microphone's recent past and the live view.
+ live: scope.Scope = .{},
+ /// Pitch, by tuner.comp: of the microphone (live, or recording), and
+ /// of the selected take at its playhead.
+ tuners: [2]tuner.Tracker = @splat(.{}),
+ /// The take `tuners[1]` follows, by id.
+ tuned: u32 = 0,
+ gpu: Gpu = .{},
+};
+
+/// Which tracker a readback belongs to: its index, as the download key.
+const Tuned = enum(u32) { input, take };
+
+/// GPU objects, made on first use.
+const Gpu = struct {
+ ready: bool = false,
+ /// `Session.tape`, and how many of its samples are in it.
+ tape: sk.Buffer = .none,
+ capacity: usize = 0,
+ uploaded: usize = 0,
+ /// spectrum.comp's FFT, shared by every panel.
+ scratch: sk.Buffer = .none,
+ /// tuner.comp's results, a `tuner_layouts.Pitch` each, by `Tuned`, and
+ /// the difference functions it searches.
+ pitches: [2]sk.Buffer = @splat(.none),
+ differences: [2]sk.Buffer = @splat(.none),
+ /// For the takes on screen, by take id; the least recently used is
+ /// reused.
+ panels: [16]Panel = @splat(.{}),
+};
+
+/// One panel's buffers and images, and the inputs they were last made from:
+/// unchanged, they are not made again.
+const Panel = struct {
+ id: u32 = 0,
+ used: u64 = 0,
+ wave_columns: sk.Buffer = .none,
+ spectrum_columns: sk.Buffer = .none,
+ levels: sk.Buffer = .none,
+ wave: sk.Image = .none,
+ wave_size: [2]u32 = .{ 0, 0 },
+ spectrogram: sk.Image = .none,
+ spectrogram_size: [2]u32 = .{ 0, 0 },
+ spectrum: sk.Image = .none,
+ spectrum_size: [2]u32 = .{ 0, 0 },
+ wave_key: u64 = 0,
+ spectrogram_key: u64 = 0,
+ spectrum_key: u64 = 0,
+ wave_drawn: u64 = 0,
+ spectrogram_drawn: u64 = 0,
+
+ fn forget(p: *Panel) void {
+ p.wave_key = 0;
+ p.spectrogram_key = 0;
+ p.spectrum_key = 0;
+ p.wave_drawn = 0;
+ p.spectrogram_drawn = 0;
+ }
+};
+
+// ---------------------------------------------------------------------------
+// The world: panels stacked down in the order they were made, take 0 on top
+// and the live microphone at the bottom, where now is. A take keeps its
+// place as others are added. The camera maps the world to the screen: it
+// follows the bottom unless moved, and zooms out to overview the strip.
+// Panels are rendered at world resolution and scaled by the camera, so
+// moving it never recomputes anything.
+
+const spectrogram_height = 132;
+const min_zoom = 0.3;
+/// The canvas follows the window, but no smaller than this.
+const min_width = 320;
+const min_height = 300;
+
+/// Sizes, from the canvas. Wide, a panel is a waveform over its spectrogram
+/// with the spectrum beside them, and the buttons sit in the header. Narrow
+/// (a phone), the spectrum goes under them, full width, and the buttons
+/// along the bottom, where thumbs are, big enough to hit.
+const Layout = struct {
+ w: f32,
+ h: f32,
+ narrow: bool,
+ /// World units are canvas pixels at zoom 1.
+ world_width: f32,
+ panel_height: f32,
+ /// Where the world shows, between the header and the footer or buttons.
+ view: sk.Rect,
+
+ const header = 36;
+ const footer = 22;
+ const toolbar = 56;
+ const spectrum_width = 320;
+
+ fn of(w: f32, h: f32) Layout {
+ const narrow = w < 760;
+ const margin: f32 = if (narrow) 8 else 16;
+ const bottom: f32 = if (narrow) toolbar else footer;
+ return .{
+ .w = w,
+ .h = h,
+ .narrow = narrow,
+ .world_width = w - 2 * margin,
+ .panel_height = if (narrow) 412 else 236,
+ .view = .{ .x = 0, .y = header, .w = w, .h = h - header - bottom },
+ };
+ }
+
+ /// The panel in world slot `k`.
+ fn box(l: Layout, k: u32) Box {
+ const y = @as(f32, @floatFromInt(k)) * l.panel_height;
+ const timeline_w = if (l.narrow) l.world_width else l.world_width - spectrum_width - 16;
+ const wave: sk.Rect = .{ .x = 0, .y = y + 18, .w = timeline_w, .h = 64 };
+ const spectrogram: sk.Rect = .{ .x = 0, .y = wave.y + wave.h + 2, .w = timeline_w, .h = spectrogram_height };
+ const timeline: sk.Rect = .{ .x = 0, .y = wave.y, .w = timeline_w, .h = spectrogram.y + spectrogram.h - wave.y };
+ // Under the timeline and its time labels, or beside it; room for
+ // the frequency labels under it either way.
+ const levels: sk.Rect = if (l.narrow)
+ .{ .x = 0, .y = spectrogram.y + spectrogram.h + 22, .w = l.world_width, .h = 150 }
+ else
+ .{ .x = l.world_width - spectrum_width, .y = wave.y, .w = spectrum_width, .h = timeline.h - 14 };
+ return .{
+ .whole = .{ .x = 0, .y = y, .w = l.world_width, .h = l.panel_height },
+ .wave = wave,
+ .spectrogram = spectrogram,
+ .timeline = timeline,
+ .spectrum = levels,
+ };
+ }
+
+ /// Button `i` of `buttons`: right to left in the header, or left to
+ /// right across the bottom.
+ fn button(l: Layout, i: usize) sk.Rect {
+ const n: f32 = @floatFromInt(buttons.len);
+ const at: f32 = @floatFromInt(i);
+ if (l.narrow) {
+ const w = (l.w - 8 - (n - 1) * 4) / n;
+ return .{ .x = 4 + (n - 1 - at) * (w + 4), .y = l.h - toolbar + 6, .w = w, .h = toolbar - 10 };
+ }
+ return .{ .x = l.w - 16 - (at + 1) * (button_width + 6) + 6, .y = 5, .w = button_width, .h = 26 };
+ }
+
+ fn buttonAt(l: Layout, x: f32, y: f32) ?Action {
+ for (buttons, 0..) |a, i| if (l.button(i).contains(x, y)) return a;
+ return null;
+ }
+};
+
+const Box = struct {
+ whole: sk.Rect,
+ wave: sk.Rect,
+ spectrogram: sk.Rect,
+ /// Waveform and spectrogram: one time axis, where clicks seek and
+ /// ctrl+wheel (or a pinch) zooms.
+ timeline: sk.Rect,
+ spectrum: sk.Rect,
+
+ fn toScreen(b: Box, cam: sk.Camera) Box {
+ return .{
+ .whole = cam.toScreen(b.whole),
+ .wave = cam.toScreen(b.wave),
+ .spectrogram = cam.toScreen(b.spectrogram),
+ .timeline = cam.toScreen(b.timeline),
+ .spectrum = cam.toScreen(b.spectrum),
+ };
+ }
+};
+
+const World = struct {
+ /// Slots `0..takes` are takes; slot `takes` is the live view, unless a
+ /// recording (the last take) stands in for it.
+ takes: u32,
+ live: bool,
+
+ fn of(s: *const State) World {
+ return .{ .takes = s.takes.count(), .live = s.transport.value != .recording };
+ }
+ fn slots(w: World) u32 {
+ return w.takes + @intFromBool(w.live);
+ }
+ fn height(w: World, l: Layout) f32 {
+ return @as(f32, @floatFromInt(w.slots())) * l.panel_height;
+ }
+ /// The take at world point `p`.
+ fn takeAt(w: World, l: Layout, p: sk.shape.Point) ?u32 {
+ if (p.y < 0 or p.x < 0 or p.x > l.world_width) return null;
+ const k: u32 = @intFromFloat(@floor(p.y / l.panel_height));
+ return if (k < w.takes) k else null;
+ }
+};
+
+/// The camera's vertical range: from the world's top at the viewport's top
+/// to its bottom at the viewport's bottom.
+fn cameraRange(world: World, scale: f32, l: Layout) [2]f32 {
+ const highest = l.view.y + 4;
+ const lowest = l.view.y + l.view.h - 4 - world.height(l) * scale;
+ return .{ @min(lowest, highest), highest };
+}
+
+/// The camera as drawn: centred across, inside the world, and on the
+/// bottom while following.
+fn cameraOf(s: *const State, l: Layout) sk.Camera {
+ var cam = s.camera;
+ cam.zoom = std.math.clamp(cam.zoom, min_zoom, 1);
+ cam.x = @round((l.w - l.world_width * cam.zoom) / 2);
+ const range = cameraRange(World.of(s), cam.zoom, l);
+ cam.y = @round(if (s.follow) range[0] else std.math.clamp(cam.y, range[0], range[1]));
+ return cam;
+}
+
+/// Moves the camera, which follows again once it reaches the bottom.
+fn moveCamera(s: *State, cam: sk.Camera, in: *const sk.Input) void {
+ const l = Layout.of(in.width, in.height);
+ s.camera = cam;
+ s.follow = false;
+ s.camera = cameraOf(s, l);
+ const range = cameraRange(World.of(s), s.camera.zoom, l);
+ s.follow = s.camera.y <= range[0] + 0.5;
+}
+
+// ---------------------------------------------------------------------------
+// Update: transport, selection, views and camera, from input alone.
+
+pub fn update(s: *State, in: *const sk.Input) void {
+ if (in.started) {
+ // Nothing resumes on its own after a restart; the takes are kept.
+ s.transport.value = .stopped;
+ s.scrubbing = false;
+ s.panning = false;
+ }
+ const recording = s.transport.value == .recording;
+ for (keys) |k| if (in.pressed(k[0])) act(s, k[1], in);
+ if (in.pressed(sk.Key.equals)) s.gain = @min(s.gain * 1.25, 64);
+ if (in.pressed(sk.Key.minus)) s.gain = @max(s.gain / 1.25, 0.25);
+ if (in.pressed(sk.Key.z) and s.selected < max_takes) s.views[s.selected] = .{};
+ if (!recording and s.takes.count() > 0) {
+ // Up the strip is older.
+ if (in.pressed(sk.Key.up)) select(s, s.selected -| 1, in);
+ if (in.pressed(sk.Key.down)) select(s, @min(s.selected + 1, s.takes.count() - 1), in);
+ }
+ const l = Layout.of(in.width, in.height);
+ const page = l.view.h - l.panel_height / 2;
+ if (in.pressed(sk.Key.page_down)) panBy(s, -page, in);
+ if (in.pressed(sk.Key.page_up)) panBy(s, page, in);
+
+ const view = l.view;
+ for (in.events) |e| {
+ const cam = cameraOf(s, l);
+ const world = World.of(s);
+ const p = cam.toWorld(sk.shape.Point{ .x = e.x, .y = e.y });
+ switch (e.kind) {
+ .mouse_down => if (e.code == 1 and l.buttonAt(e.x, e.y) != null) {
+ act(s, l.buttonAt(e.x, e.y).?, in);
+ } else if (e.code == 2 or e.code == 3) {
+ // Middle or right drag pans.
+ s.panning = true;
+ s.grab = e.y;
+ } else if (e.code == 1 and !recording and view.contains(e.x, e.y)) {
+ const index = world.takeAt(l, p) orelse continue;
+ select(s, index, in);
+ const b = l.box(index);
+ if (b.timeline.contains(p.x, p.y)) {
+ s.scrubbing = true;
+ seekTo(s, fractionAt(s.views[index], b.timeline, p.x));
+ }
+ },
+ .mouse_move => if (s.panning) {
+ panBy(s, e.y - s.grab, in);
+ s.grab = e.y;
+ } else if (s.scrubbing and s.selected < world.takes) {
+ seekTo(s, fractionAt(s.views[s.selected], l.box(s.selected).timeline, p.x));
+ },
+ .mouse_up, .cancel => {
+ s.scrubbing = false;
+ s.panning = false;
+ },
+ // Ctrl+wheel, or a pinch (which browsers and the web host send
+ // as ctrl+wheel), zooms what is under it: a take's time on its
+ // timeline, the view anywhere else.
+ .wheel => if (view.contains(e.x, e.y)) {
+ const ctrl = e.modifiers() & sk.Key.Mod.ctrl != 0;
+ const over = world.takeAt(l, p);
+ if (ctrl and over != null and l.box(over.?).timeline.contains(p.x, p.y)) {
+ const index = over.?;
+ zoom(&s.views[index], fractionAt(s.views[index], l.box(index).timeline, p.x), std.math.pow(f32, 0.8, e.wheel()));
+ } else if (ctrl) {
+ var moved = cam;
+ moved.zoomAt(sk.shape.Point{ .x = e.x, .y = e.y }, e.wheel(), .{ .min = min_zoom, .max = 1 });
+ moveCamera(s, moved, in);
+ } else panBy(s, e.wheel() * 60, in);
+ },
+ else => {},
+ }
+ }
+}
+
+/// What keys and the buttons on top do.
+const Action = enum { record, play, forward, backward, start, end, open, delete, overview, a4_down, a4_up };
+
+const keys = [_]struct { sk.Key, Action }{
+ .{ .r, .record }, .{ .space, .play }, .{ .right, .forward }, .{ .left, .backward },
+ .{ .home, .start }, .{ .end, .end }, .{ .o, .open }, .{ .delete, .delete },
+ .{ .backspace, .delete }, .{ .@"0", .overview }, .{ .left_bracket, .a4_down }, .{ .right_bracket, .a4_up },
+};
+
+/// Everything a touch screen cannot reach by key, as buttons (see
+/// `Layout.button`). Labels change with what they would do.
+const buttons = [_]Action{ .a4_up, .a4_down, .overview, .delete, .open, .backward, .play, .record };
+const button_width = 52;
+
+fn act(s: *State, a: Action, in: *const sk.Input) void {
+ const now = s.transport.value;
+ switch (a) {
+ .record => if (now == .recording) change(s, .stopped) else record(s),
+ .play => change(s, switch (now) {
+ .forward, .backward => .stopped,
+ .stopped, .recording => if (s.backward) .backward else .forward,
+ }),
+ .forward => change(s, .forward),
+ .backward => change(s, .backward),
+ .start => seekTo(s, 0),
+ .end => seekTo(s, 1),
+ .open => s.dialog.send({}),
+ .delete => if (now != .recording and s.takes.count() > 0) remove(s, in),
+ .overview => {
+ // The whole strip, or back to full size.
+ const l = Layout.of(in.width, in.height);
+ var cam = cameraOf(s, l);
+ cam.zoom = if (cam.zoom < 1) 1 else l.view.h / @max(World.of(s).height(l), 1);
+ moveCamera(s, cam, in);
+ s.follow = true;
+ },
+ .a4_down => s.a4 = @max(s.a4 - 1, 400),
+ .a4_up => s.a4 = @min(s.a4 + 1, 480),
+ }
+}
+
+/// Starts a new take, unless the strip is full. It takes the live view's
+/// place, which the camera follows.
+fn record(s: *State) void {
+ s.takes.appendBounded(.{ .id = s.next_id }) catch return;
+ s.next_id += 1;
+ s.selected = s.takes.count() - 1;
+ s.views[s.selected] = .{};
+ s.follow = true;
+ change(s, .recording);
+}
+
+fn change(s: *State, to: Transport) void {
+ if (to == .forward or to == .backward) s.backward = to == .backward;
+ s.transport.send(to);
+}
+
+fn seekTo(s: *State, fraction: f32) void {
+ if (s.transport.value == .recording) return;
+ s.seek.send(std.math.clamp(fraction, 0, 1));
+}
+
+/// Selects a take and brings it into view.
+fn select(s: *State, index: u32, in: *const sk.Input) void {
+ s.selected = index;
+ const l = Layout.of(in.width, in.height);
+ var cam = cameraOf(s, l);
+ const b = cam.toScreen(l.box(index).whole);
+ const view = l.view;
+ if (b.y < view.y) cam.y += view.y - b.y + 4;
+ if (b.y + b.h > view.y + view.h) cam.y -= b.y + b.h - (view.y + view.h) + 4;
+ moveCamera(s, cam, in);
+}
+
+/// Moves the world down by `pixels` on screen.
+fn panBy(s: *State, pixels: f32, in: *const sk.Input) void {
+ var cam = cameraOf(s, Layout.of(in.width, in.height));
+ cam.y += pixels;
+ moveCamera(s, cam, in);
+}
+
+/// Deletes the selected take and selects the one above it. Its file stays:
+/// any keyframe or mark from before the delete brings the take back, and
+/// neither the sketch nor the host can tell when none is left.
+fn remove(s: *State, in: *const sk.Input) void {
+ const n = s.takes.count();
+ const index = @min(s.selected, n - 1);
+ _ = s.takes.orderedRemove(index);
+ std.mem.copyForwards(View, s.views[index .. n - 1], s.views[index + 1 .. n]);
+ s.selected = index -| 1;
+ moveCamera(s, cameraOf(s, Layout.of(in.width, in.height)), in);
+}
+
+fn fractionAt(v: View, timeline: sk.Rect, x: f32) f32 {
+ return v.lo + (x - timeline.x) / timeline.w * (v.hi - v.lo);
+}
+
+/// Scales the view by `factor` around `at`, keeping it inside the take.
+fn zoom(v: *View, at: f32, factor: f32) void {
+ const span = std.math.clamp((v.hi - v.lo) * factor, 1e-6, 1);
+ var lo = at - (at - v.lo) / (v.hi - v.lo) * span;
+ lo = std.math.clamp(lo, 0, 1 - span);
+ v.lo = lo;
+ v.hi = lo + span;
+}
+
+/// `open PATH` adds a file as a take, writing its samples to the take's
+/// file; `open` alone shows the dialog. `next-id N` moves the counter
+/// (`draw` sends it).
+pub fn command(s: *State, ss: *Session, c: *sk.Console, line: []const u8) !bool {
+ var words = std.mem.tokenizeAny(u8, line, " \t");
+ const verb = words.next() orelse return false;
+ if (std.mem.eql(u8, verb, "next-id")) {
+ s.next_id = try std.fmt.parseInt(u32, words.next() orelse return error.ExpectedId, 10);
+ return true;
+ }
+ if (!std.mem.eql(u8, verb, "open")) return false;
+ const path = std.mem.trim(u8, words.rest(), " \t");
+ if (path.len == 0) {
+ s.dialog.send({});
+ return true;
+ }
+ if (s.takes.count() >= max_takes) return error.TooManyTakes;
+ const samples = c.host.samples(c.host.loadSound(path));
+ if (samples.len == 0) return error.CannotOpen;
+ // Recording grows the last take: stop before this one goes after it.
+ if (s.transport.value == .recording) change(s, .stopped);
+ const kept = samples[0..@min(samples.len, max_seconds * c.in.sample_rate)];
+ var take: Take = .{ .id = s.next_id };
+ const base = std.fs.path.basename(path);
+ take.source_len = @intCast(@min(base.len, take.source.len));
+ @memcpy(take.source[0..take.source_len], base[0..take.source_len]);
+ // Into Session now, so the web, which keeps no files, has it too.
+ try ss.tape.appendSlice(c.session, kept);
+ try ss.clips.appendBounded(.{ .id = take.id, .start = ss.tape.items.len - kept.len, .len = kept.len });
+ var file: [32]u8 = undefined;
+ _ = c.host.writeFile(try std.fmt.bufPrint(&file, "take-{d}.f32", .{take.id}), std.mem.sliceAsBytes(kept));
+ try s.takes.appendBounded(take);
+ s.next_id += 1;
+ s.selected = s.takes.count() - 1;
+ s.views[s.selected] = .{};
+ s.follow = true;
+ change(s, .stopped);
+ c.print("opened {s}: {d:.1} s", .{ path, @as(f64, @floatFromInt(kept.len)) / @as(f64, @floatFromInt(c.in.sample_rate)) });
+ return true;
+}
+
+// ---------------------------------------------------------------------------
+// Draw: the takes, the microphone, the speakers and the GPU.
+
+pub fn draw(s: *const State, ss: *Session, f: *sk.Frame) void {
+ const in = f.in;
+ const rate: f64 = @floatFromInt(in.sample_rate);
+ const dt: f64 = @min(@as(f64, @floatFromInt(in.dt)) / std.time.ns_per_s, 0.1);
+ // As big as the window: the layout follows (see `Layout`).
+ f.canvas(@intFromFloat(@max(in.window_width, min_width)), @intFromFloat(@max(in.window_height, min_height)));
+ f.clear(0x0b0c0f);
+
+ if (ss.dialog.take(s.dialog) != null) f.openFile("open", .{ .pattern = "wav;flac;ogg" });
+ // Files are written live, never while a recording of the session plays
+ // back: like sound, a write is not replayed.
+ const writing = in.mode == .live or in.mode == .recording;
+ if (in.jumped and writing) {
+ // State came from elsewhere: the next id's file may hold samples.
+ var id = s.next_id;
+ while (f.host.readFile(f.host.appendFile(f.format("take-{d}.f32", .{id}), &.{})).len > 0) id += 1;
+ if (id != s.next_id) f.run(f.format("next-id {d}", .{id}));
+ }
+ // Requests from State. Any of them, or another take selected, flushes
+ // what is queued, so the cursor restarts from what was heard.
+ const transport = ss.transport.take(s.transport);
+ const seek = ss.seek.take(s.seek);
+ const takes = s.takes.constSlice();
+ // A take that begins here (not one a jump brought) starts its file
+ // empty, whatever a file of its id held.
+ if (transport == .recording and writing and !in.jumped and takes.len > 0) _ = f.host.writeFile(f.format("take-{d}.f32", .{takes[takes.len - 1].id}), &.{});
+ readTakes(s, ss, f);
+ const selected: ?*Clip = if (takes.len > 0) clipOf(ss, takes[@min(s.selected, takes.len - 1)].id) else null;
+ var queued: usize = in.stream(stream).queued;
+ if (selected) |t| {
+ if (transport != null or seek != null or t.id != ss.playing) {
+ f.audio(stream, &.{}, .{ .clear = true });
+ queued = 0;
+ ss.playing = t.id;
+ ss.cursor = @intFromFloat(std.math.clamp(@round(t.heard), 0, @as(f64, @floatFromInt(t.len))));
+ }
+ if (transport) |to| {
+ switch (to) {
+ .forward => if (ss.cursor >= t.len) {
+ ss.cursor = 0;
+ },
+ .backward => if (ss.cursor == 0) {
+ ss.cursor = t.len;
+ },
+ .stopped, .recording => {},
+ }
+ t.heard = @floatFromInt(ss.cursor);
+ }
+ if (seek) |fraction| if (s.transport.value != .recording) {
+ ss.cursor = @intFromFloat(@round(fraction * @as(f32, @floatFromInt(t.len))));
+ t.heard = @floatFromInt(ss.cursor);
+ };
+ }
+
+ // The microphone is always open: the live view shows it, and a
+ // recording keeps it.
+ f.microphone();
+ ss.live.push(in.microphone);
+ switch (s.transport.value) {
+ // Into the last take, at the tape's end.
+ .recording => if (takes.len > 0 and writing) if (clipOf(ss, takes[takes.len - 1].id)) |t| if (t.start + t.len == ss.tape.items.len) {
+ const room = max_seconds * in.sample_rate -| t.len;
+ const heard = in.microphone[0..@min(in.microphone.len, room)];
+ ss.tape.appendSlice(f.session, heard) catch f.host.log("out of memory: recording stopped");
+ _ = f.host.appendFile(f.format("take-{d}.f32", .{t.id}), std.mem.sliceAsBytes(heard));
+ t.len = ss.tape.items.len - t.start;
+ t.heard = @floatFromInt(t.len);
+ },
+ .forward, .backward => if (selected) |t| play(ss, t, ss.tape.items[t.start..][0..t.len], f, s.transport.value == .forward, queued, rate, dt),
+ .stopped => {},
+ }
+
+ // The tuner's readbacks, a frame or two old.
+ if (selected) |t| if (t.id != ss.tuned) {
+ ss.tuned = t.id;
+ ss.tuners[@intFromEnum(Tuned.take)] = .{};
+ };
+ for (in.downloads) |d| {
+ if (d.key >= ss.tuners.len or d.bytes.len < @sizeOf(tuner_layouts.Pitch)) continue;
+ ss.tuners[d.key].feed(std.mem.bytesToValue(tuner_layouts.Pitch, d.bytes[0..@sizeOf(tuner_layouts.Pitch)]), s.a4, @floatCast(dt));
+ }
+
+ if (!f.host.gpu()) {
+ f.text(16, Layout.header, 2, "micscope needs a GPU: the host has none.", 0xff8080);
+ return;
+ }
+ render(s, ss, f, rate, dt);
+}
+
+/// Session's tape follows State's takes: a take State no longer has leaves
+/// it, and one it lacks is read from its file (a new one's is empty).
+fn readTakes(s: *const State, ss: *Session, f: *sk.Frame) void {
+ var i: u32 = 0;
+ while (i < ss.clips.count()) {
+ const t = ss.clips.buffer[i];
+ if (for (s.takes.constSlice()) |take| {
+ if (take.id == t.id) break true;
+ } else false) {
+ i += 1;
+ continue;
+ }
+ _ = ss.clips.orderedRemove(i);
+ const tape = ss.tape.items;
+ std.mem.copyForwards(f32, tape[t.start .. tape.len - t.len], tape[t.start + t.len ..]);
+ ss.tape.shrinkRetainingCapacity(tape.len - t.len);
+ for (ss.clips.slice()[i..]) |*later| later.start -= t.len;
+ ss.gpu.uploaded = @min(ss.gpu.uploaded, t.start);
+ for (&ss.gpu.panels) |*p| if (p.id == t.id) {
+ p.id = 0;
+ p.forget();
+ };
+ }
+ for (s.takes.constSlice()) |take| if (clipOf(ss, take.id) == null) {
+ const bytes = f.host.readFile(f.host.appendFile(f.format("take-{d}.f32", .{take.id}), &.{}));
+ const samples = std.mem.bytesAsSlice(f32, @as([]align(4) const u8, @alignCast(bytes[0 .. bytes.len / 4 * 4])));
+ ss.tape.appendSlice(f.session, samples) catch return f.host.log("out of memory: takes not read");
+ ss.clips.appendBounded(.{ .id = take.id, .start = ss.tape.items.len - samples.len, .len = samples.len }) catch return;
+ };
+}
+
+fn clipOf(ss: *Session, id: u32) ?*Clip {
+ for (ss.clips.slice()) |*t| if (t.id == id) return t;
+ return null;
+}
+
+/// Keeps the stream `lead_seconds` ahead, reading the take either way, and
+/// follows the listener.
+fn play(ss: *Session, t: *Clip, tape: []const f32, f: *sk.Frame, forward: bool, queued: usize, rate: f64, dt: f64) void {
+ const lead: usize = @intFromFloat(rate * lead_seconds);
+ const want = lead -| queued;
+ // What is heard now: the cursor, less what is queued in front of it.
+ const measured: f64 = @floatFromInt(if (forward) ss.cursor -| queued else @min(ss.cursor + queued, tape.len));
+ if (forward) {
+ const n = @min(want, tape.len -| ss.cursor);
+ f.audio(stream, tape[@min(ss.cursor, tape.len)..][0..n], .{});
+ ss.cursor += n;
+ } else {
+ const n = @min(want, ss.cursor);
+ const reversed = f.scratch.alloc(f32, n) catch return;
+ for (reversed, 0..) |*sample, i| sample.* = tape[ss.cursor - 1 - i];
+ f.audio(stream, reversed, .{});
+ ss.cursor -= n;
+ }
+ // The queue drains in device-sized blocks: advance by the clock and
+ // pull gently towards the measurement, snapping when far off.
+ const direction: f64 = if (forward) 1 else -1;
+ t.heard += direction * dt * rate;
+ const off = measured - t.heard;
+ t.heard = if (@abs(off) > rate * 0.25) measured else t.heard + off * 0.15;
+ t.heard = std.math.clamp(t.heard, 0, @as(f64, @floatFromInt(tape.len)));
+}
+
+// ---------------------------------------------------------------------------
+// GPU: buffers, compute, passes.
+
+const tuner_difference_kernel: gpu.Kernel = .{
+ .code = shaders.tuner_difference_comp,
+ .readonly_buffers = 1,
+ .readwrite_buffers = 1,
+ .uniform_buffers = 1,
+ .threads = .{ 256, 1, 1 },
+};
+const tuner_kernel: gpu.Kernel = .{
+ .code = shaders.tuner_comp,
+ .readonly_buffers = 2,
+ .readwrite_buffers = 1,
+ .uniform_buffers = 1,
+ .threads = .{ 256, 1, 1 },
+};
+
+fn setup(g: *Gpu, f: *sk.Frame) void {
+ if (g.ready) return;
+ g.ready = true;
+ g.scratch = f.host.newBuffer(layouts.fft_size * 2 * @sizeOf([2]f32));
+ for (&g.pitches) |*b| b.* = f.host.newBuffer(@sizeOf(tuner_layouts.Pitch));
+ for (&g.differences) |*b| b.* = f.host.newBuffer(tuner_layouts.tuner_lags * @sizeOf(f32));
+}
+
+fn makeBuffers(p: *Panel, f: *sk.Frame) void {
+ if (p.wave_columns != .none) return;
+ p.wave_columns = f.host.newBuffer(layouts.max_columns * @sizeOf([4]f32));
+ p.spectrum_columns = f.host.newBuffer(layouts.max_columns * @sizeOf([2]f32));
+ p.levels = f.host.newBuffer(layouts.max_columns * spectrogram_height * @sizeOf(f32));
+}
+
+/// The panel for take `id`: the one it had, or the least recently used.
+fn panelFor(g: *Gpu, f: *sk.Frame, id: u32) *Panel {
+ const p = for (&g.panels) |*p| {
+ if (p.id == id) break p;
+ } else reuse: {
+ var oldest = &g.panels[0];
+ for (&g.panels) |*p| if (p.used < oldest.used) {
+ oldest = p;
+ };
+ oldest.id = id;
+ oldest.forget();
+ break :reuse oldest;
+ };
+ p.used = f.in.frame;
+ makeBuffers(p, f);
+ return p;
+}
+
+/// An image of `size`, (re)made when the size changes.
+fn target(f: *sk.Frame, image: *sk.Image, size: *[2]u32, r: sk.Rect) [2]u32 {
+ const want: [2]u32 = .{ @intFromFloat(@min(r.w, layouts.max_columns)), @intFromFloat(@min(r.h, spectrogram_height)) };
+ if (image.* == .none or size[0] != want[0] or size[1] != want[1]) {
+ if (image.* != .none) f.host.freeImage(image.*);
+ image.* = f.host.newImage(want[0], want[1]);
+ size.* = want;
+ }
+ return want;
+}
+
+/// Puts the tape on the GPU: new samples only, unless the buffer had to grow.
+fn sync(g: *Gpu, f: *sk.Frame, tape: []const f32) void {
+ if (tape.len > g.capacity or g.tape == .none) {
+ if (g.tape != .none) f.host.freeBuffer(g.tape);
+ g.capacity = @max(tape.len, g.capacity * 2, 60 * f.in.sample_rate);
+ g.tape = f.host.newBuffer(@intCast(g.capacity * @sizeOf(f32)));
+ g.uploaded = 0;
+ }
+ g.uploaded = @min(g.uploaded, tape.len);
+ if (g.uploaded < tape.len) {
+ f.upload(g.tape, @intCast(g.uploaded * @sizeOf(f32)), std.mem.sliceAsBytes(tape[g.uploaded..]));
+ g.uploaded = tape.len;
+ }
+}
+
+fn render(s: *const State, ss: *Session, f: *sk.Frame, rate: f64, dt: f64) void {
+ const g = &ss.gpu;
+ setup(g, f);
+ sync(g, f, ss.tape.items);
+ // New code may mean new shaders: make everything again (the live view
+ // sees to its own).
+ if (f.in.reloaded) for (&g.panels) |*p| p.forget();
+ const takes = s.takes.constSlice();
+ const selected = if (takes.len > 0) @min(s.selected, takes.len - 1) else 0;
+ const recording = s.transport.value == .recording;
+ const world: World = .of(s);
+ const l = Layout.of(f.in.width, f.in.height);
+ const cam = cameraOf(s, l);
+ const view = l.view;
+ // Images are made at world size; the camera only scales them.
+ const size = l.box(0);
+
+ f.clip(view);
+ // The live view, in the world's last slot. It keeps up even while it is
+ // hidden: drawn clipped away while a recording stands in for it.
+ const live = l.box(world.takes);
+ if (!world.live) f.clip(.{ .x = 0, .y = 0, .w = 0, .h = 0 });
+ ss.live.draw(f, .{ .seconds = live_seconds, .wave = live.wave, .spectrogram = live.spectrogram, .spectrum = live.spectrum, .gain = s.gain, .camera = cam });
+ f.clip(view);
+ // The tuner: on the microphone's newest sound, and on the selected take
+ // around its playhead.
+ const reach = tuner_layouts.tuner_window + tuner_layouts.tuner_lags;
+ const last = if (recording and takes.len > 0) clipOf(ss, takes[takes.len - 1].id) else null;
+ if (last) |t| {
+ tune(g, f, .input, g.tape, t.start, t.len, t.len, rate);
+ } else tune(g, f, .input, ss.live.gpu.tape, 0, scope.history, scope.history, rate);
+ if (!recording and takes.len > 0) if (clipOf(ss, takes[selected].id)) |t| {
+ const end: usize = @intFromFloat(@max(t.heard, 0));
+ tune(g, f, .take, g.tape, t.start, t.len, end + reach / 2, rate);
+ };
+ for (0..world.takes) |slot| {
+ const k: u32 = @intCast(slot);
+ const b = l.box(k).toScreen(cam);
+ if (b.whole.y + b.whole.h <= view.y or b.whole.y >= view.y + view.h) continue;
+ const tint: layouts.Tint = if (recording and k + 1 == world.takes) .recording else .take;
+ drawTake(s, g, f, size, b, clipOf(ss, takes[k].id) orelse continue, s.views[k], k == selected, tint, rate, dt);
+ }
+ f.clip(null);
+ labels(s, ss, f, world, cam, rate);
+}
+
+/// Finds the pitch of the `tuner_window` samples before `end` (of the
+/// `samples` from `offset`), and reads it back.
+fn tune(g: *Gpu, f: *sk.Frame, which: Tuned, tape: sk.Buffer, offset: usize, samples: usize, end: usize, rate: f64) void {
+ const result = g.pitches[@intFromEnum(which)];
+ const difference = g.differences[@intFromEnum(which)];
+ const params = gpu.uniform(f.scratch, tuner_layouts.TunerParams{
+ .offset = @intCast(offset),
+ .end = @intCast(end),
+ .samples = @intCast(samples),
+ .rate = @floatCast(rate),
+ .min_freq = tuner_low,
+ .max_freq = tuner_high,
+ .threshold = 0.12,
+ });
+ f.compute(.{
+ .label = "tuner_difference.comp",
+ .kernel = tuner_difference_kernel,
+ .readonly = &.{tape},
+ .readwrite = &.{difference},
+ .uniforms = params,
+ // Match tuner_difference.comp's LAGS_PER_GROUP.
+ .groups = .{ (tuner_layouts.tuner_lags + 15) / 16, 1, 1 },
+ });
+ f.compute(.{
+ .label = "tuner.comp",
+ .kernel = tuner_kernel,
+ .readonly = &.{ tape, difference },
+ .readwrite = &.{result},
+ .uniforms = params,
+ });
+ f.download(result, .{ .size = @sizeOf(tuner_layouts.Pitch), .key = @intFromEnum(which) });
+}
+
+/// The pitches the tuner looks for: a five-string bass's low B to a
+/// piccolo's top.
+const tuner_low = 30.0;
+const tuner_high = 4200.0;
+
+/// The spectrum's analysis of `tape`, then its pass.
+fn spectrum(g: *Gpu, f: *sk.Frame, tape: sk.Buffer, p: *Panel, size: [2]u32, params: layouts.SpectrumParams, rate: f64) void {
+ f.compute(.{
+ .label = "spectrum.comp",
+ .kernel = scope.kernels.spectrum,
+ .readonly = &.{tape},
+ .readwrite = &.{ g.scratch, p.spectrum_columns },
+ .uniforms = gpu.uniform(f.scratch, params),
+ });
+ f.gpu(.{
+ .label = "spectrum.frag",
+ .image = p.spectrum,
+ .width = size[0],
+ .height = size[1],
+ .clear = null,
+ .pipelines = &scope.pipelines.spectrum,
+ .draws = &.{.{
+ .pipeline = 0,
+ .vertex_count = 3,
+ .fragment_uniforms = gpu.uniform(f.scratch, layouts.SpectrumView{
+ .size = .{ @floatFromInt(size[0]), @floatFromInt(size[1]) },
+ .floor_db = scope.floor_db,
+ .ceil_db = 0,
+ .min_freq = scope.min_freq,
+ .max_freq = scope.maxFreq(rate),
+ }),
+ .fragment_storage = &.{p.spectrum_columns},
+ }},
+ });
+}
+
+fn drawTake(s: *const State, g: *Gpu, f: *sk.Frame, size: Box, b: Box, t: *const Clip, view: View, selected: bool, tint: layouts.Tint, rate: f64, dt: f64) void {
+ const p = panelFor(g, f, t.id);
+
+ // Waveform columns, when what they show changed.
+ const wave = target(f, &p.wave, &p.wave_size, size.wave);
+ const span = visible(view, t.len, wave[0]);
+ const first: u32 = @intFromFloat(@floor(span[0]));
+ const spp: f32 = @floatCast((span[1] - span[0]) / @as(f64, @floatFromInt(wave[0])));
+ const params: layouts.WaveParams = .{
+ .offset = @intCast(t.start),
+ .first = first,
+ .samples = @intCast(t.len),
+ .columns = wave[0],
+ .spp = spp,
+ .slot = 0,
+ .ring = layouts.max_columns,
+ };
+ const key = std.hash.Wyhash.hash(0, std.mem.asBytes(&params));
+ if (key != p.wave_key) {
+ p.wave_key = key;
+ f.compute(.{
+ .label = "waveform.comp",
+ .kernel = scope.kernels.waveform,
+ .readonly = &.{g.tape},
+ .readwrite = &.{p.wave_columns},
+ .uniforms = gpu.uniform(f.scratch, params),
+ .groups = .{ wave[0], 1, 1 },
+ });
+ }
+
+ // The spectrogram over the same columns.
+ const spectrogram = target(f, &p.spectrogram, &p.spectrogram_size, size.spectrogram);
+ const analysis: layouts.SpectrogramParams = .{
+ .offset = @intCast(t.start),
+ .first = first,
+ .samples = @intCast(t.len),
+ .columns = spectrogram[0],
+ .rows = spectrogram[1],
+ .spp = spp,
+ .rate = @floatCast(rate),
+ .min_freq = scope.min_freq,
+ .max_freq = scope.maxFreq(rate),
+ .slot = 0,
+ .ring = layouts.max_columns,
+ };
+ const analysis_key = std.hash.Wyhash.hash(0, std.mem.asBytes(&analysis));
+ if (analysis_key != p.spectrogram_key) {
+ p.spectrogram_key = analysis_key;
+ f.compute(.{
+ .label = "spectrogram.comp",
+ .kernel = scope.kernels.spectrogram,
+ .readonly = &.{g.tape},
+ .readwrite = &.{p.levels},
+ .uniforms = gpu.uniform(f.scratch, analysis),
+ .groups = .{ spectrogram[0], 1, 1 },
+ });
+ }
+
+ // The spectrum around the playhead, or of the latest while recording:
+ // every frame for the selected take, which plays and glides; the others
+ // only when it would change, and without smoothing.
+ const half: f64 = layouts.window / 2;
+ const len: f64 = @floatFromInt(t.len);
+ const end: f64 = if (tint == .recording) len else @min(t.heard + half, len + half);
+ const shown = target(f, &p.spectrum, &p.spectrum_size, size.spectrum);
+ var sparams: layouts.SpectrumParams = .{
+ .offset = @intCast(t.start),
+ .end = @intFromFloat(@max(end, 0)),
+ .samples = @intCast(t.len),
+ .columns = shown[0],
+ .reset = 0,
+ .rate = @floatCast(rate),
+ .min_freq = scope.min_freq,
+ .max_freq = scope.maxFreq(rate),
+ .dt = 0,
+ };
+ const spectrum_key = std.hash.Wyhash.hash(1, std.mem.asBytes(&sparams));
+ if (selected or spectrum_key != p.spectrum_key) {
+ sparams.reset = @intFromBool(!selected or p.spectrum_key == 0);
+ sparams.dt = @floatCast(dt);
+ p.spectrum_key = spectrum_key;
+ spectrum(g, f, g.tape, p, shown, sparams, rate);
+ }
+
+ const at: f32 = @floatCast((t.heard - span[0]) / (span[1] - span[0]) * @as(f64, @floatFromInt(wave[0])));
+ const playhead: f32 = if (!selected or t.len == 0 or at < 0 or at > @as(f32, @floatFromInt(wave[0]))) -1 else at;
+ const wave_view: layouts.WaveView = .{
+ .size = .{ @floatFromInt(wave[0]), @floatFromInt(wave[1]) },
+ .playhead = playhead,
+ .gain = s.gain,
+ .tint = @intFromEnum(tint),
+ .scroll = 0,
+ .filled = wave[0],
+ };
+ const wave_drawn = std.hash.Wyhash.hash(p.wave_key, std.mem.asBytes(&wave_view));
+ if (wave_drawn != p.wave_drawn) {
+ p.wave_drawn = wave_drawn;
+ f.gpu(.{
+ .label = "waveform.frag",
+ .image = p.wave,
+ .width = wave[0],
+ .height = wave[1],
+ .clear = null,
+ .pipelines = &scope.pipelines.wave,
+ .draws = &.{.{ .pipeline = 0, .vertex_count = 3, .fragment_uniforms = gpu.uniform(f.scratch, wave_view), .fragment_storage = &.{p.wave_columns} }},
+ });
+ }
+ const spectrogram_view: layouts.SpectrogramView = .{
+ .size = .{ @floatFromInt(spectrogram[0]), @floatFromInt(spectrogram[1]) },
+ .floor_db = scope.floor_db,
+ .ceil_db = -6,
+ .playhead = playhead,
+ .scroll = 0,
+ .filled = spectrogram[0],
+ };
+ const spectrogram_drawn = std.hash.Wyhash.hash(p.spectrogram_key, std.mem.asBytes(&spectrogram_view));
+ if (spectrogram_drawn != p.spectrogram_drawn) {
+ p.spectrogram_drawn = spectrogram_drawn;
+ f.gpu(.{
+ .label = "spectrogram.frag",
+ .image = p.spectrogram,
+ .width = spectrogram[0],
+ .height = spectrogram[1],
+ .clear = null,
+ .pipelines = &scope.pipelines.spectrogram,
+ .draws = &.{.{ .pipeline = 0, .vertex_count = 3, .fragment_uniforms = gpu.uniform(f.scratch, spectrogram_view), .fragment_storage = &.{p.levels} }},
+ });
+ }
+ f.image(p.wave, b.wave, .{});
+ f.image(p.spectrogram, b.spectrogram, .{});
+ f.image(p.spectrum, b.spectrum, .{});
+}
+
+/// The samples a take's timeline spans, at least 1/32 of a sample per
+/// column.
+fn visible(v: View, len: usize, columns: u32) [2]f64 {
+ const total: f64 = @floatFromInt(len);
+ var lo = v.lo * total;
+ var hi = v.hi * total;
+ const least = @as(f64, @floatFromInt(columns)) / 32;
+ if (hi - lo < least) {
+ const mid = (lo + hi) / 2;
+ lo = @max(mid - least / 2, 0);
+ hi = lo + least;
+ }
+ return .{ lo, hi };
+}
+
+// ---------------------------------------------------------------------------
+// Text.
+
+const dim = 0x7a8090;
+const bright = 0xe6e9ef;
+const red = 0xff5a5a;
+const green = 0x6fdc8c;
+const yellow = 0xffd84d;
+
+fn labels(s: *const State, ss: *Session, f: *sk.Frame, world: World, cam: sk.Camera, rate: f64) void {
+ const takes = s.takes.constSlice();
+ const recording = s.transport.value == .recording;
+ const selected = if (takes.len > 0) @min(s.selected, takes.len - 1) else 0;
+
+ // What the transport is doing, to which take.
+ const status: struct { []const u8, u24 } = switch (s.transport.value) {
+ .recording => .{ "REC", red },
+ .forward => .{ "PLAY >", 0x7fd7ff },
+ .backward => .{ "PLAY <", 0x7fd7ff },
+ .stopped => .{ "PAUSED", dim },
+ };
+ const l = Layout.of(f.in.width, f.in.height);
+ f.text(16, 10, 2, status[0], status[1]);
+ if (if (takes.len > 0) clipOf(ss, takes[selected].id) else null) |t| {
+ const times = .{ Time{ .seconds = t.heard / rate }, Time{ .seconds = @as(f64, @floatFromInt(t.len)) / rate } };
+ // As much as fits beside the buttons (or, narrow, the corner).
+ const room = if (l.narrow) l.w - 16 * 7 else l.button(buttons.len - 1).x - 16;
+ if (room >= 480) {
+ f.print(140, 10, 2, bright, "{f} / {f}", times);
+ if (room >= 740) f.print(500, 14, 1, dim, "take {d} of {d}: {s}", .{ selected + 1, takes.len, source(&takes[selected]) });
+ } else f.print(16 + 16 * 7, 14, 1, bright, "{f} / {f}", times);
+ }
+ for (buttons, 0..) |a, i| button(s, f, a, l.button(i));
+ const corner = f.format("{s}{s}", .{
+ if (s.gain != 1) f.format("gain x{d:.2} ", .{s.gain}) else "",
+ if (cam.zoom < 1) f.format("view x{d:.2}", .{cam.zoom}) else "",
+ });
+ f.text(f.in.width - 16 - @as(f32, @floatFromInt(corner.len * 8)), if (l.narrow) 14 else f.in.height - 16, 1, corner, dim);
+
+ const view = l.view;
+ f.clip(view);
+ for (0..world.slots()) |slot| {
+ const k: u32 = @intCast(slot);
+ const b = l.box(k).toScreen(cam);
+ if (b.whole.y + b.whole.h <= view.y or b.whole.y >= view.y + view.h) continue;
+ // Centred in the band above the waveform, whatever the zoom.
+ const title_y = @round(b.whole.y + (b.wave.y - b.whole.y - 8) / 2);
+ if (k == world.takes) {
+ f.print(b.whole.x, title_y, 1, green, "LIVE the microphone, the last {d} s", .{live_seconds});
+ if (ss.live.heard == 0) f.text(b.wave.x + 8, b.wave.y + 8, 1, "nothing from the microphone yet", dim);
+ if (cam.zoom >= 0.8) axes(f, b, rate, null);
+ tunerOverlay(f, b.spectrum, ss.tuners[@intFromEnum(Tuned.input)].reading, s.a4, rate, cam.zoom >= 0.8);
+ continue;
+ }
+ const t = clipOf(ss, takes[k].id) orelse continue;
+ const seconds = Time{ .seconds = @as(f64, @floatFromInt(t.len)) / rate };
+ if (recording and k + 1 == world.takes) {
+ f.print(b.whole.x, title_y, 1, red, "RECORDING take {d} {f}", .{ k + 1, seconds });
+ } else {
+ const chosen = k == selected;
+ if (chosen) f.fill(.{ .x = b.whole.x - if (l.narrow) @as(f32, 6) else 12, .y = b.wave.y, .w = 3, .h = b.timeline.h }, yellow);
+ f.print(b.whole.x, title_y, 1, @as(u24, if (chosen) bright else dim), "#{d} {s} {f}", .{ k + 1, source(&takes[k]), seconds });
+ }
+ if (cam.zoom >= 0.8) {
+ const span = visible(s.views[k], t.len, @intFromFloat(@min(l.box(0).wave.w, layouts.max_columns)));
+ axes(f, b, rate, .{ span[0] / rate, span[1] / rate });
+ }
+ const which: ?Tuned = if (recording) (if (k + 1 == world.takes) .input else null) else if (k == selected) .take else null;
+ if (which) |w| tunerOverlay(f, b.spectrum, ss.tuners[@intFromEnum(w)].reading, s.a4, rate, cam.zoom >= 0.8);
+ }
+ f.clip(null);
+ if (world.takes == 0) f.text(16, view.y + view.h - 12, 1, "No takes yet: REC records one, OPEN opens a file.", dim);
+ // Keys, where there is a keyboard to press them.
+ if (!l.narrow) f.text(16, f.in.height - 16, 1, "R SPACE LEFT RIGHT HOME END O DEL 0 [ ] UP/DOWN take wheel or 2 fingers pan CTRL+wheel or pinch zoom", dim);
+}
+
+/// The tuner on a spectrum: the pitch and its harmonics where they fall on
+/// it, the span of two semitones either side shaded, and that span
+/// magnified across the top: the nearest notes of the chromatic scale,
+/// the pitch as a needle between them, and the note in cents and Hz.
+fn tunerOverlay(f: *sk.Frame, r: sk.Rect, reading: ?tuner.Reading, a4: f32, rate: f64, detailed: bool) void {
+ const at = reading orelse return;
+ const max_freq = scope.maxFreq(rate);
+ const fade: u8 = @intFromFloat(255 * std.math.clamp(1 - at.quiet / tuner.hold_seconds, 0.2, 1));
+ const cents = at.cents();
+ const color = sk.Color.hex(if (@abs(cents) <= 5) green else if (@abs(cents) <= 15) yellow else 0xff8a4d).alpha(fade);
+ const x = struct {
+ fn of(rect: sk.Rect, hz: f32, ceiling: f32) f32 {
+ return rect.x + @log(hz / scope.min_freq) / @log(ceiling / scope.min_freq) * rect.w;
+ }
+ }.of;
+
+ const freq = scope.hz(at.pitch, a4);
+ const lo = scope.hz(@as(f32, @floatFromInt(at.note)) - 2.5, a4);
+ const hi = scope.hz(@as(f32, @floatFromInt(at.note)) + 2.5, a4);
+ f.fill(.{ .x = x(r, lo, max_freq), .y = r.y, .w = @max(x(r, hi, max_freq) - x(r, lo, max_freq), 1), .h = r.h }, sk.Color.white.alpha(fade / 10));
+ f.fill(.{ .x = @round(x(r, freq, max_freq)), .y = r.y, .w = 1, .h = r.h }, color);
+ // The harmonics the pitch accounts for.
+ for (2..9) |h| {
+ const hz = freq * @as(f32, @floatFromInt(h));
+ if (hz > max_freq) break;
+ f.fill(.{ .x = @round(x(r, hz, max_freq)), .y = r.y + r.h - 10, .w = 1, .h = 10 }, color.alpha(fade / 2));
+ }
+ if (!detailed) return;
+
+ // The magnified scale: semitones `note - 2.5` to `note + 2.5`.
+ const strip: sk.Rect = .{ .x = r.x + 6, .y = r.y + 4, .w = r.w - 12, .h = 30 };
+ const semitone = strip.w / 5;
+ const centre = strip.x + strip.w / 2;
+ f.fill(strip, sk.Color.hex(0x0b0c0f).alpha(220));
+ var c: f32 = -250;
+ while (c <= 250) : (c += 10) {
+ const whole = @mod(c, 100) == 0;
+ const h: f32 = if (whole) 10 else if (@mod(c, 50) == 0) 6 else 3;
+ f.fill(.{ .x = @round(centre + c / 100 * semitone), .y = strip.y + strip.h - h, .w = 1, .h = h }, sk.Color.hex(if (c == 0) bright else dim).alpha(if (whole) 220 else 120));
+ }
+ for (0..5) |i| {
+ const note = at.note - 2 + @as(i32, @intCast(i));
+ const label = f.format("{s}{d}", .{ tuner.name(note), tuner.octave(note) });
+ const lx = centre + (@as(f32, @floatFromInt(i)) - 2) * semitone - @as(f32, @floatFromInt(label.len * 4));
+ f.text(lx, strip.y + 4, 1, label, if (note == at.note) color else sk.Color.hex(dim).alpha(fade));
+ }
+ const needle = std.math.clamp(centre + cents / 100 * semitone, strip.x, strip.x + strip.w - 2);
+ f.fill(.{ .x = @round(needle) - 1, .y = strip.y + 14, .w = 3, .h = strip.h - 14 }, color);
+
+ // The reading.
+ const big = f.format("{s}{d}", .{ tuner.name(at.note), tuner.octave(at.note) });
+ const panel: sk.Rect = .{ .x = r.x + r.w - 6 - 124, .y = strip.y + strip.h + 4, .w = 124, .h = 50 };
+ f.fill(panel, sk.Color.hex(0x0b0c0f).alpha(200));
+ f.text(panel.x + 6, panel.y + 4, 3, big, color);
+ const whole = @round(cents);
+ f.print(panel.x + 6, panel.y + 30, 1, sk.Color.hex(bright).alpha(fade), "{s}{d:.0} cents", .{ if (whole > 0) "+" else "", if (whole == 0) 0 else whole });
+ f.print(panel.x + 6, panel.y + 40, 1, sk.Color.hex(dim).alpha(fade), "{d:.2} Hz", .{freq});
+ f.print(panel.x + 70, panel.y + 4, 1, sk.Color.hex(dim).alpha(fade), "A4 {d:.0}", .{a4});
+}
+
+fn button(s: *const State, f: *sk.Frame, a: Action, r: sk.Rect) void {
+ const now = s.transport.value;
+ const label: []const u8, const lit: ?u24 = switch (a) {
+ .record => if (now == .recording) .{ "STOP", red } else .{ "REC", null },
+ .play => if (now == .forward or now == .backward) .{ "PAUSE", 0x7fd7ff } else .{ "PLAY", null },
+ .backward => .{ "REV", if (now == .backward) 0x7fd7ff else null },
+ .open => .{ "OPEN", null },
+ .delete => .{ "DEL", null },
+ .overview => .{ "ALL", null },
+ .a4_down => .{ "A4-", null },
+ .a4_up => .{ "A4+", null },
+ .forward, .start, .end => .{ "", null },
+ };
+ f.fill(r, sk.Color.hex(if (lit) |c| c else 0x1c2029).alpha(if (lit != null) 70 else 255));
+ f.fill(.{ .x = r.x, .y = r.y + r.h - 1, .w = r.w, .h = 1 }, sk.Color.hex(lit orelse 0x2e3440));
+ f.text(r.x + (r.w - @as(f32, @floatFromInt(label.len * 8))) / 2, r.y + 9, 1, label, sk.Color.hex(lit orelse bright));
+}
+
+fn source(t: *const Take) []const u8 {
+ return if (t.source_len > 0) t.source[0..t.source_len] else "microphone";
+}
+
+/// A panel's scales: its time span (seconds; null for the live view, which
+/// ends now) under the spectrogram, frequencies up the spectrogram, and
+/// frequency and level on the spectrum.
+fn axes(f: *sk.Frame, b: Box, rate: f64, span: ?[2]f64) void {
+ const below = b.spectrogram.y + b.spectrogram.h + 4;
+ const left = if (span) |sp| f.format("{f}", .{Time{ .seconds = sp[0] }}) else f.format("-{d} s", .{live_seconds});
+ const right = if (span) |sp| f.format("{f}", .{Time{ .seconds = sp[1] }}) else "now";
+ f.text(b.wave.x, below, 1, left, dim);
+ f.text(b.wave.x + b.wave.w - @as(f32, @floatFromInt(right.len * 8)), below, 1, right, dim);
+
+ const max_freq = scope.maxFreq(rate);
+ for ([_]f64{ 100, 1000, 10000 }) |hz| {
+ const y = b.spectrogram.y + b.spectrogram.h * (1 - @as(f32, @floatCast(@log(hz / scope.min_freq) / @log(max_freq / scope.min_freq))));
+ f.fill(.{ .x = b.spectrogram.x, .y = y, .w = 6, .h = 1 }, sk.Color.white.alpha(140));
+ f.text(b.spectrogram.x + 9, y - 4, 1, hzLabel(f, hz), sk.Color.white.alpha(170));
+ }
+
+ for ([_]f64{ 50, 200, 1000, 5000, 20000 }) |hz| {
+ if (hz > max_freq) continue;
+ const x = b.spectrum.x + @as(f32, @floatCast(@log(hz / scope.min_freq) / @log(max_freq / scope.min_freq))) * b.spectrum.w;
+ f.text(@min(x + 3, b.spectrum.x + b.spectrum.w - 24), b.spectrum.y + b.spectrum.h + 4, 1, hzLabel(f, hz), dim);
+ }
+ var db: f32 = -24;
+ while (db > scope.floor_db) : (db -= 24) {
+ const y = b.spectrum.y + b.spectrum.h * (db / scope.floor_db);
+ f.print(b.spectrum.x + 4, y + 3, 1, dim, "{d}", .{db});
+ }
+}
+
+fn hzLabel(f: *sk.Frame, hz: f64) []const u8 {
+ return if (hz >= 1000) f.format("{d}k", .{hz / 1000}) else f.format("{d}", .{hz});
+}
+
+const Time = struct {
+ seconds: f64,
+
+ pub fn format(t: Time, w: *std.Io.Writer) std.Io.Writer.Error!void {
+ const ms: u64 = @intFromFloat(@max(t.seconds, 0) * 1000);
+ try w.print("{d}:{d:0>2}.{d:0>3}", .{ ms / 60_000, ms / 1000 % 60, ms % 1000 });
+ }
+};
+
+test "zoom keeps the view inside the take and around the pointer" {
+ var v: View = .{};
+ zoom(&v, 0.5, 0.5);
+ try std.testing.expectApproxEqAbs(@as(f32, 0.25), v.lo, 1e-6);
+ try std.testing.expectApproxEqAbs(@as(f32, 0.75), v.hi, 1e-6);
+ zoom(&v, 0.0, 4);
+ try std.testing.expectEqual(@as(f32, 0), v.lo);
+ try std.testing.expectEqual(@as(f32, 1), v.hi);
+}
+
+test "the world stacks takes in order with the live view last" {
+ const world: World = .{ .takes = 3, .live = true };
+ try std.testing.expectEqual(@as(u32, 4), world.slots());
+ const l = Layout.of(width, height);
+ try std.testing.expectEqual(@as(?u32, 0), world.takeAt(l, .{ .x = 10, .y = 1 }));
+ try std.testing.expectEqual(@as(?u32, 2), world.takeAt(l, .{ .x = 10, .y = 2 * l.panel_height + 1 }));
+ try std.testing.expectEqual(@as(?u32, null), world.takeAt(l, .{ .x = 10, .y = 3 * l.panel_height + 1 }));
+}
+
+test "the camera follows the bottom and stays inside the world, wide or narrow" {
+ for ([_]Layout{ .of(width, height), .of(390, 844) }) |l| {
+ var s: State = .{};
+ s.takes.len = 5;
+ const cam = cameraOf(&s, l);
+ // Following: the live view's bottom sits at the viewport's bottom.
+ try std.testing.expectApproxEqAbs(l.view.y + l.view.h - 4, cam.toScreen(l.box(5).whole).y + l.panel_height, 1);
+ s.follow = false;
+ s.camera.y = 1e6;
+ try std.testing.expectApproxEqAbs(l.view.y + 4, cameraOf(&s, l).toScreen(l.box(0).whole).y, 1);
+ }
+}
+
+test "a narrow layout stacks the spectrum under the timeline, inside the panel" {
+ const l: Layout = .of(390, 844);
+ try std.testing.expect(l.narrow);
+ const b = l.box(0);
+ try std.testing.expect(b.spectrum.y > b.timeline.y + b.timeline.h);
+ try std.testing.expect(b.spectrum.y + b.spectrum.h + 16 <= b.whole.y + b.whole.h);
+ try std.testing.expectEqual(l.world_width, b.timeline.w);
+ // Buttons fill the bottom, inside the canvas, without overlapping.
+ for (0..buttons.len) |i| {
+ const r = l.button(i);
+ try std.testing.expect(r.x >= 0 and r.x + r.w <= l.w and r.y + r.h <= l.h);
+ try std.testing.expect(r.w >= 40);
+ }
+}
+
+test {
+ _ = tuner;
+}
+
diff --git a/examples/micscope/src/tuner.zig b/examples/micscope/src/tuner.zig
new file mode 100644
index 0000000..159798b
--- /dev/null
+++ b/examples/micscope/src/tuner.zig
@@ -0,0 +1,132 @@
+//! From pitch estimates (tuner.comp's YIN, one per frame) to a steady note on
+//! the 12-tone equal-tempered chromatic scale, for any instrument: notes are
+//! MIDI numbers (69 is A4, at `a4` Hz), so the range is whatever the
+//! detector finds, from a bass's low B to a piccolo's top.
+//!
+//! Raw estimates jitter and, at a note's attack or release, misfire. So:
+//! unclear or quiet estimates are ignored; the median of the last few
+//! clear ones is followed, smoothed; a jump of over a semitone restarts the
+//! median, so a new note shows at once; and the note shown only changes
+//! once the pitch is 60 cents from it, so a note held a quarter tone sharp
+//! does not flicker between two names.
+const std = @import("std");
+const scope = @import("scope");
+const layouts = @import("layouts");
+
+/// Estimates clearer than this are a note (YIN's threshold is 0.12, so
+/// clarity 0.88 at least when its first dip was found).
+pub const min_clarity = 0.85;
+/// Quieter than this is silence, dBFS.
+pub const min_level = -60.0;
+/// How long the last note stays (shown as fading) after the sound stops.
+pub const hold_seconds = 1.0;
+
+pub fn name(note: i32) []const u8 {
+ return scope.note_names[@intCast(@mod(note, 12))];
+}
+
+pub fn octave(note: i32) i32 {
+ return @divFloor(note, 12) - 1;
+}
+
+pub const Reading = struct {
+ /// The note it is nearest, as a MIDI number.
+ note: i32,
+ /// The pitch, as a fractional MIDI number.
+ pitch: f32,
+ /// Seconds since a clear estimate: 0 while the note sounds.
+ quiet: f32,
+
+ pub fn cents(r: Reading) f32 {
+ return (r.pitch - @as(f32, @floatFromInt(r.note))) * 100;
+ }
+};
+
+pub const Tracker = struct {
+ recent: [5]f32 = @splat(0),
+ count: u32 = 0,
+ reading: ?Reading = null,
+
+ pub fn feed(t: *Tracker, p: layouts.Pitch, a4: f32, dt: f32) void {
+ const clear = p.freq > 0 and p.clarity >= min_clarity and p.level >= min_level;
+ if (!clear) {
+ t.count = 0;
+ if (t.reading) |*r| {
+ r.quiet += dt;
+ if (r.quiet > hold_seconds) t.reading = null;
+ }
+ return;
+ }
+ const m = scope.midi(p.freq, a4);
+ if (t.count > 0 and @abs(m - t.median()) > 1) t.count = 0;
+ if (t.count == t.recent.len) {
+ std.mem.copyForwards(f32, t.recent[0 .. t.recent.len - 1], t.recent[1..]);
+ t.count -= 1;
+ }
+ t.recent[t.count] = m;
+ t.count += 1;
+ const target = t.median();
+
+ const fresh = t.count == 1 or t.reading == null or t.reading.?.quiet > 0;
+ var r: Reading = if (fresh) .{ .note = @intFromFloat(@round(target)), .pitch = target, .quiet = 0 } else t.reading.?;
+ r.pitch += (target - r.pitch) * 0.4;
+ r.quiet = 0;
+ if (@abs(r.pitch - @as(f32, @floatFromInt(r.note))) > 0.6) r.note = @intFromFloat(@round(r.pitch));
+ t.reading = r;
+ }
+
+ fn median(t: *const Tracker) f32 {
+ var sorted = t.recent;
+ std.mem.sort(f32, sorted[0..t.count], {}, std.sort.asc(f32));
+ return sorted[t.count / 2];
+ }
+};
+
+fn pitchOf(freq: f32) layouts.Pitch {
+ return .{ .freq = freq, .clarity = 0.97, .level = -20, .period = 48_000 / freq };
+}
+
+test "notes and octaves on the chromatic scale" {
+ try std.testing.expectEqualStrings("E", name(40));
+ try std.testing.expectEqual(@as(i32, 2), octave(40));
+ try std.testing.expectEqualStrings("B", name(23));
+ try std.testing.expectEqual(@as(i32, 0), octave(23));
+}
+
+test "a held note settles, in cents from it" {
+ var t: Tracker = .{};
+ // A4, 12 cents sharp.
+ const sharp = scope.hz(69.12, 440);
+ for (0..20) |_| t.feed(pitchOf(sharp), 440, 1.0 / 60.0);
+ const r = t.reading.?;
+ try std.testing.expectEqual(@as(i32, 69), r.note);
+ try std.testing.expectApproxEqAbs(@as(f32, 12), r.cents(), 0.5);
+}
+
+test "one wild estimate does not move the note" {
+ var t: Tracker = .{};
+ for (0..10) |_| t.feed(pitchOf(110), 440, 1.0 / 60.0);
+ t.feed(pitchOf(220), 440, 1.0 / 60.0);
+ t.feed(pitchOf(110), 440, 1.0 / 60.0);
+ try std.testing.expectEqual(@as(i32, 45), t.reading.?.note);
+}
+
+test "a new note shows at once, a quarter tone does not flicker" {
+ var t: Tracker = .{};
+ for (0..10) |_| t.feed(pitchOf(110), 440, 1.0 / 60.0);
+ for (0..3) |_| t.feed(pitchOf(146.83), 440, 1.0 / 60.0);
+ try std.testing.expectEqual(@as(i32, 50), t.reading.?.note);
+ // 55 cents sharp of D3 stays D3.
+ for (0..30) |_| t.feed(pitchOf(scope.hz(50.55, 440)), 440, 1.0 / 60.0);
+ try std.testing.expectEqual(@as(i32, 50), t.reading.?.note);
+}
+
+test "silence holds the note a while, then lets go" {
+ var t: Tracker = .{};
+ for (0..10) |_| t.feed(pitchOf(330), 440, 1.0 / 60.0);
+ const quiet: layouts.Pitch = .{ .freq = 0, .clarity = 0, .level = -120, .period = 0 };
+ for (0..30) |_| t.feed(quiet, 440, 1.0 / 60.0);
+ try std.testing.expect(t.reading != null);
+ for (0..60) |_| t.feed(quiet, 440, 1.0 / 60.0);
+ try std.testing.expect(t.reading == null);
+}
diff --git a/examples/micscope/test/README.md b/examples/micscope/test/README.md
new file mode 100644
index 0000000..6f4b833
--- /dev/null
+++ b/examples/micscope/test/README.md
@@ -0,0 +1,22 @@
+# Test material
+
+- `sweep.wav`: a 2 s log sweep (40 Hz to 16 kHz) over a quiet 440 Hz tone,
+ 48 kHz mono, generated for these tests.
+- `tones.wav`: five 1 s notes for the tuner, generated for these tests
+ (harmonics with a little noise, 48 kHz mono): E1 with its 2nd harmonic
+ loudest, E2 with a weak fundamental, A4 12 cents sharp, D4 with no
+ fundamental at all, and C7.
+- `reference.flac`: the rendering reference, all on one screen. 15 s of a
+ loudly mastered electronic track (sub-bass kicks, hats, a full mix), then
+ 15 s of a Berlin street (traffic, a close vehicle pass, broadband noise),
+ then `sweep.wav`; 48 kHz stereo, joined with short crossfades.
+
+ - Square Punch, "Thundercat" (Plasma Pumper EP, 2011), from 3:36:
+ [Wikimedia Commons](https://commons.wikimedia.org/wiki/File:Square_Punch_-_Thundercat.flac),
+ CC BY-SA 3.0.
+ - LG, "traffic karl-liebknechtstrasse 111223", from 1:18:
+ [Wikimedia Commons](https://commons.wikimedia.org/wiki/File:141977_lg_traffic-karl-liebknechtstrasse-111223.wav),
+ from [Freesound](http://freesound.org/people/LG/sounds/141977/),
+ CC BY-SA 3.0.
+
+ The excerpt is licensed CC BY-SA 3.0 like its sources.
diff --git a/examples/micscope/test/profile.script b/examples/micscope/test/profile.script
new file mode 100644
index 0000000..aa56b3e
--- /dev/null
+++ b/examples/micscope/test/profile.script
@@ -0,0 +1,167 @@
+# Zooms in and out one ctrl+wheel step per frame on the reference take, so
+# waveform and spectrogram are recomputed every frame: run by
+# `zig build profile` with --profile-gpu, which prints GPU time per command.
+console open test/reference.flac
+frames 3
+key right
+hold left_ctrl
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
+wheel 470 150 -0.15
+frames 1
diff --git a/examples/micscope/test/record.script b/examples/micscope/test/record.script
new file mode 100644
index 0000000..843d802
--- /dev/null
+++ b/examples/micscope/test/record.script
@@ -0,0 +1,23 @@
+# Records (from whatever microphone the host gives a headless run) into a
+# new take, stops, and plays it back.
+expect takes.len 0
+key r
+expect transport.value recording
+expect takes.len 1
+frames 60
+screenshot .zig-cache/micscope/recording.png
+key r
+expect transport.value stopped
+key space
+expect transport.value forward
+frames 10
+# Each recording is another take; opening a file while recording ends it.
+key r
+expect takes.len 2
+expect selected 1
+console open test/sweep.wav
+expect transport.value stopped
+expect takes.len 3
+frames 2
+expect takes.len 3
+screenshot .zig-cache/micscope/strip.png
diff --git a/examples/micscope/test/reference.flac b/examples/micscope/test/reference.flac
new file mode 100644
index 0000000..7ad1c63
--- /dev/null
+++ b/examples/micscope/test/reference.flac
Binary files differ
diff --git a/examples/micscope/test/render.script b/examples/micscope/test/render.script
new file mode 100644
index 0000000..71b50c7
--- /dev/null
+++ b/examples/micscope/test/render.script
@@ -0,0 +1,18 @@
+# The rendering reference (see README.md): song, street and sweep in one
+# take. Screenshots land in .zig-cache/micscope/ for review.
+console open test/reference.flac
+expect transport.value stopped
+frames 3
+screenshot .zig-cache/micscope/reference.png
+# Playing: the playhead across waveform and spectrogram, the spectrum.
+key right
+frames 90
+expect-audio 1
+screenshot .zig-cache/micscope/reference-playing.png
+# Zoomed into the street: the low end's hum lines and the noise floor.
+key space
+hold left_ctrl
+wheel 660 150 4
+release left_ctrl
+frames 3
+screenshot .zig-cache/micscope/reference-street.png
diff --git a/examples/micscope/test/sweep.wav b/examples/micscope/test/sweep.wav
new file mode 100644
index 0000000..1f131e3
--- /dev/null
+++ b/examples/micscope/test/sweep.wav
Binary files differ
diff --git a/examples/micscope/test/tones.wav b/examples/micscope/test/tones.wav
new file mode 100644
index 0000000..b2cf3ea
--- /dev/null
+++ b/examples/micscope/test/tones.wav
Binary files differ
diff --git a/examples/micscope/test/transport.script b/examples/micscope/test/transport.script
new file mode 100644
index 0000000..2015682
--- /dev/null
+++ b/examples/micscope/test/transport.script
@@ -0,0 +1,98 @@
+# Opens a file as a take and plays it both ways: a 2 s log sweep (40 Hz to
+# 16 kHz) over a quiet 440 Hz tone.
+console open test/sweep.wav
+frames 2
+expect takes.len 1
+expect selected 0
+expect transport.value stopped
+key right
+expect transport.value forward
+frames 30
+expect-audio 1
+screenshot .zig-cache/micscope/forward.png
+key left
+expect transport.value backward
+expect backward true
+frames 20
+key space
+expect transport.value stopped
+# Space resumes the way it last went.
+key space
+expect transport.value backward
+key space
+# Seeking: keys, and a click on the take's timeline.
+key end
+expect seek.value 1
+click 400 150
+expect seek.serial 2
+# Ctrl+wheel (or a pinch) on a take's timeline zooms its time around the
+# pointer; Z zooms back out.
+hold left_ctrl
+wheel 400 150 3
+release left_ctrl
+key z
+key right
+frames 30
+screenshot .zig-cache/micscope/played.png
+# A second take goes under the first and is selected; up is older.
+console open test/sweep.wav
+expect takes.len 2
+expect selected 1
+key up
+expect selected 0
+key down
+expect selected 1
+# A click on the first take selects it.
+click 400 150
+expect selected 0
+frames 2
+screenshot .zig-cache/micscope/two-takes.png
+# Deleting it leaves the second. Its file stays: going back to before the
+# deletion reads it again.
+console save two
+key delete
+expect takes.len 1
+expect selected 0
+console load two
+frames 2
+expect takes.len 2
+key delete
+expect takes.len 1
+# A counter that names a take file already written (--fresh keeps the
+# files; a seek back finds the future's) is moved past it by draw.
+console set next_id 1
+frames 2
+expect next_id 3
+# More takes than fit: the camera follows the bottom until the wheel pans
+# it, then 0 shows the whole strip.
+console open test/sweep.wav
+console open test/sweep.wav
+console open test/sweep.wav
+expect follow true
+wheel 400 400 2
+expect follow false
+frames 2
+screenshot .zig-cache/micscope/panned.png
+key 0
+expect follow true
+frames 2
+screenshot .zig-cache/micscope/overview.png
+# Ctrl+wheel off the timelines zooms the view.
+hold left_ctrl
+wheel 1150 400 1
+release left_ctrl
+frames 2
+screenshot .zig-cache/micscope/zoomed-view.png
+# The buttons on top do what the keys do, for touch screens.
+click 832 18
+expect transport.value recording
+click 832 18
+expect transport.value stopped
+click 890 18
+expect transport.value forward
+click 890 18
+expect transport.value stopped
+click 1238 18
+expect a4 441
+frames 2
+screenshot .zig-cache/micscope/buttons.png
diff --git a/examples/micscope/test/tuner.script b/examples/micscope/test/tuner.script
new file mode 100644
index 0000000..4e0e480
--- /dev/null
+++ b/examples/micscope/test/tuner.script
@@ -0,0 +1,31 @@
+# The tuner on a take, at its playhead: five notes where the strongest FFT
+# peak is not (or not only) the note. Expected, by screenshot:
+# E1 0 cents (41.2 Hz) bass, 2nd harmonic loudest
+# E2 0 cents (82.4 Hz) guitar, weak fundamental
+# A4 +12 cents (443.1 Hz) sharp
+# D4 0 cents (293.7 Hz) fundamental missing
+# C7 0 cents (2093 Hz) piccolo
+console open test/tones.wav
+click 107 150
+frames 20
+screenshot .zig-cache/micscope/tuner-e1.png
+click 290 150
+frames 20
+screenshot .zig-cache/micscope/tuner-e2.png
+click 472 150
+frames 20
+screenshot .zig-cache/micscope/tuner-a4.png
+click 654 150
+frames 20
+screenshot .zig-cache/micscope/tuner-d4.png
+click 837 150
+frames 20
+screenshot .zig-cache/micscope/tuner-c7.png
+# Another reference pitch: A4 = 443 makes the sharp A4 in tune.
+click 472 150
+key right_bracket
+key right_bracket
+key right_bracket
+expect a4 443
+frames 20
+screenshot .zig-cache/micscope/tuner-a443.png
diff --git a/examples/micscope/web/icon-192.png b/examples/micscope/web/icon-192.png
new file mode 100644
index 0000000..0b26078
--- /dev/null
+++ b/examples/micscope/web/icon-192.png
Binary files differ
diff --git a/examples/micscope/web/icon-512.png b/examples/micscope/web/icon-512.png
new file mode 100644
index 0000000..0b5538c
--- /dev/null
+++ b/examples/micscope/web/icon-512.png
Binary files differ
diff --git a/examples/micscope/web/icon.svg b/examples/micscope/web/icon.svg
new file mode 100644
index 0000000..1220011
--- /dev/null
+++ b/examples/micscope/web/icon.svg
@@ -0,0 +1,26 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
+ <!-- micscope's icon: a waveform over its spectrogram's harmonics, in the
+ app's colours. Full-bleed, so it also works masked. Rendered to the
+ PNGs beside it with: resvg -w N -h N icon.svg icon-N.png -->
+ <defs>
+ <linearGradient id="heat" gradientUnits="userSpaceOnUse" x1="96" y1="0" x2="416" y2="0">
+ <stop offset="0" stop-color="#2b0a4a"/>
+ <stop offset="0.25" stop-color="#b8325a"/>
+ <stop offset="0.6" stop-color="#f98c0a"/>
+ <stop offset="1" stop-color="#fcf3a6"/>
+ </linearGradient>
+ </defs>
+ <rect width="512" height="512" fill="#0b0c0f"/>
+ <!-- harmonics: a fundamental and its overtones, fading -->
+ <g stroke="url(#heat)" stroke-linecap="round">
+ <line x1="96" y1="392" x2="416" y2="392" stroke-width="22"/>
+ <line x1="96" y1="340" x2="416" y2="340" stroke-width="16" opacity="0.85"/>
+ <line x1="96" y1="300" x2="416" y2="300" stroke-width="11" opacity="0.65"/>
+ <line x1="96" y1="270" x2="416" y2="270" stroke-width="7" opacity="0.45"/>
+ </g>
+ <!-- the waveform -->
+ <path d="M96 176 C120 96 152 96 176 176 S232 256 256 176 S312 96 336 176 S392 256 416 176"
+ fill="none" stroke="#6fdc8c" stroke-width="18" stroke-linecap="round"/>
+ <!-- the tuner's needle -->
+ <rect x="246" y="76" width="20" height="356" rx="10" fill="#ffd84d" opacity="0.9"/>
+</svg>