# micscope Records the microphone or opens sound files, plays them forwards or backwards, and shows each as a waveform, its spectrogram on the same time axis, and the spectrum at the playhead. A [sketchbook](../..) sketch, Linux only for now. The window shows a vertical strip of these panels, laid out in a world and seen through a camera, as in a game. Takes stack downwards in the order they were made, so each keeps its place; at the bottom is now: what the microphone hears, its last 6 s scrolling by, with the spectrum of the newest samples. Recording turns that panel red and fills it with the new take; stopping leaves the take there and puts the live view under it. The camera follows the bottom until you pan away, and follows again once you pan back down. Zoomed out, it overviews the whole strip. Every take keeps its own time zoom and playhead. A tuner reads the live sound (or the recording, or the selected take at its playhead) and shows it on that panel's spectrum: a line at the pitch, ticks at its harmonics, the two semitones either side shaded, and that shaded span magnified across the top: the nearest notes of the 12-tone chromatic scale, the pitch as a needle among them (green within 5 cents, yellow within 15), and the note, cents and Hz. It is not a guitar tuner in particular: it finds 30 Hz to 4.2 kHz, a five-string bass's low B to a piccolo's top, against any A4 (`[` and `]`, 440 Hz by default). Panels are rendered at world resolution and the camera only scales them, so moving or zooming it recomputes nothing. The waveform, spectrogram and spectrum shaders are sketchbook's `scope` module's (`src/scope/shaders/`): the live view is a `scope.Scope`, and the take panels run `scope.kernels` and `scope.pipelines` over the tape. The tuner's are micscope's own. | Shader | Does | | --- | --- | | `spectrum.comp` | Blackman-Harris window around the playhead, zero-padded 2×, radix-2 Stockham FFT (8192 points, one workgroup), dBFS per bin, then one value per log-spaced pixel column: a Catmull-Rom spline through the bins where columns are narrower than a bin, the loudest bin where they span several. Smoothed in time (fast attack, slow release) with falling peaks. | | `waveform.comp` | Per pixel column: min, max and RMS of its samples, plus the interpolated signal at the column's edges, so zoomed past one sample per pixel the columns still join into a line. Recomputed only when the view or tape changes. | | `spectrogram.comp` | One workgroup per waveform column, so it lines up with the waveform in time. A 2048-sample Blackman-Harris window zero-padded to 4096, transformed as 2048 complex points in shared memory (the real-input trick) and split into 2049 bins. Below 500 Hz a second analysis runs on the signal low-passed and decimated 4×: the same 2048 points span 170 ms and resolve ~6 Hz instead of 23, three windows a quarter apart averaged (Welch) so noise does not speckle; blended with the first between 250 and 500 Hz. Zoomed out, each column averages up to four windows spread across its samples, so it neither skips nor flickers. Rows are log frequency, interpolated like the spectrum. | | `tuner_difference.comp`, `tuner.comp` | Pitch by YIN rather than by the strongest FFT peak, which is often a harmonic (low strings, voices and brass put more into the 2nd or 3rd) or missing altogether (a missing fundamental). The difference function d(τ) = Σ(x[j] − x[j−τ])² over 2048 samples for lags up to 1700, spread over 107 workgroups; then in one workgroup, normalised by its running mean (a parallel scan), the first dip under 0.12 (the shortest period that explains the signal, so no octave drops) followed to its bottom, and a parabola through the raw dip for the period between samples. Read back to the CPU (`f.download`), which follows the median of the last five clear estimates, restarts on a jump of more than a semitone, and only changes the note shown 60 cents away from it. | | `waveform.frag`, `spectrogram.frag`, `spectrum.frag` | Full-screen triangles that draw those buffers: the waveform and spectrum anti-aliased with grids, the spectrogram through an inferno heat gradient. | The takes share one tape buffer on the GPU (each analysis reads from an offset), uploaded once. A take's waveform and spectrogram are recomputed only when its view changes, and its panel redrawn only when something on it moved; only the selected take's spectrum runs every frame. The live view (the scope module's) uploads the microphone's last 1.4 s each frame and analyses only the columns that became complete, into ring buffers the fragment shaders read from the oldest column. A column waits until everything its spectrogram windows reach has been heard (6160 samples past its centre), so it is analysed once and final: the live spectrogram runs about 130 ms behind, the live spectrum does not. ## Keys | Key | | | --- | --- | | R | record a new take / stop | | Space | play / pause the selected take, resuming in the last direction | | Right / Left | play forwards / backwards | | Home / End | seek to the start / end | | Up / Down | select the older / newer take | | Delete | delete the selected take | | O | open a file (WAV, FLAC, Ogg Vorbis) as a take, through the system dialog | | click, drag | select a take; on its timeline, seek | | wheel, middle or right drag, Page Up / Down | pan the camera | | Ctrl + wheel, or a pinch | zoom what is under the pointer: a take's time on its timeline (Z unzooms), the camera elsewhere | | 0 | the whole strip, or back to full size | | - / = | waveform gain | | [ / ] | the tuner's A4, 1 Hz down / up | | \` | the sketchbook console: `open ` opens a file too | The buttons (REC, PLAY, REV, OPEN, DEL, ALL, A4−, A4+) do what the keys do, for touch screens: there, one finger clicks and drags, two fingers pan, and a pinch zooms like ctrl+wheel. The canvas is the window, and the layout follows it. Wide, each panel has the spectrum beside the waveform and spectrogram, and the buttons sit in the header; narrower than 760 px (a phone held upright), the spectrum goes under them at full width, and the buttons along the bottom, where thumbs are. ## Rendering reference `test/reference.flac` puts a mastered song, a city street and a sweep on one screen (sources and licences in `test/README.md`); `zig build test` plays it through `test/render.script` and leaves screenshots in `.zig-cache/micscope/`. What it settled for the spectrogram's low end, checked against a plain numpy STFT of the same columns: - One long window resolves the street's two engine hums but shows noise as speckle full of holes: a single window's spectrum of noise scatters as much as its level. - Multitaper averaging (sine tapers, DPSS with NW 2 to 4) calms the noise but widens the hums into one slab and leaks into silence, which a 96 dB scale shows. Wrong trade for lines. - Averaging three windows a quarter apart in time keeps the hums sharp and the kicks distinct, and settles the noise. That is what it does. `test/tones.wav` has five notes where the strongest peak is not the note (a bass E1 and a guitar E2 louder in their 2nd harmonic, a D4 without its fundamental), an A4 12 cents sharp and a piccolo C7; `test/tuner.script` screenshots the tuner on each. All read within 0.1 cent. ## GPU time `zig build profile` zooms the reference one step per frame, recomputing the waveform and spectrogram every frame, and prints GPU time per command (`--profile-gpu`: each command timed alone on an idle GPU). On an RTX 3050: | Case | GPU per frame | | --- | --- | | playing, views cached | ~0.45 ms | | zooming the 32 s reference | ~1.1 ms on average, spectrogram 1.5 ms on the frames it recomputes | | zooming a 10 minute tape | ~1.6 ms on average, spectrogram 2.1 ms (worst 4.6 ms) | | zooming the reference, in the strip with the live view and both tuners | ~1.15 ms on average; the live view adds a spectrum (~0.15 ms) and a few columns, each tuner ~0.1 ms | ## Building Zig 0.17 and `glslc` (shaderc) on PATH; for the web build also `naga` (`cargo install naga-cli`), which translates the shaders to WGSL, and to serve it `miniserve` and `openssl`. zig build run # with hot reload: `zig build lib --watch` alongside zig build test # unit tests and the headless scripts in test/ zig build profile # GPU time per command while zooming zig-out/bin/micscope # the standalone build zig build serve # the web build at https://:8042/ # (`zig build web --watch` alongside rebuilds it) ## On the web The same sketch, compiled to wasm32-freestanding, on WebGPU: every shader runs as on native, translated to WGSL, and the tuner's readback is a WebGPU buffer mapped a frame or two later. Sound and the microphone go through Web Audio (the page asks for the microphone, and sound starts at the first click or key, as browsers require); OPEN decodes a file with the browser's own decoders. Takes last as long as the page. It is an installable app (a PWA): browsers offer to add it to the home screen, and once opened it works offline, from a copy its service worker keeps of the page, the code and the wasm. The icon is `web/icon.svg`, rendered to the PNGs beside it with `resvg -w N -h N icon.svg icon-N.png`. It needs a browser with WebGPU (Chrome, Edge, Safari 26; Firefox on Linux with `dom.webgpu.enabled` set in about:config), and a secure context, which `zig build serve`'s https gives it on any device on the network once the browser has accepted its self-signed certificate (miniserve prints the addresses as QR codes, for phones). Each take is a file of its samples (mono 32-bit floats at the host's rate), `micscope.take-.f32` beside the save (in `.sketchbook/` under `zig build run`), written as it records (`Host.appendFile`). State lists the takes, so they are still there after a restart. A take starts its file empty (`Host.writeFile`). A deleted take's file stays: a seek back, or any mark from before the delete, finds it again. Takes stop at ten minutes; the strip holds 64. A `micscope.state.store` from before is not read: its takes stay in it.