summaryrefslogtreecommitdiff
path: root/examples/live/README.md
blob: 781fc7ada68c66434a20f39f18ed17fdc083d4d0 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
# live

A program typed over its own picture, and heard, in pardes's editor
(helix's keys, the language's grammar colouring it and drawing the name
each line defines a size larger). Each line names an
expression; the `out` line is what you hear (-1..1), the `color` line what
you see. At every keystroke that leaves a program that parses, the
program is compiled into two tapes of at most 128 instructions, and one
interpreter on the GPU runs them per sample and per pixel. While a line
is broken, the last good program keeps playing and the problem is marked
under its byte.

```
bpm   = 120
beat  = t * bpm / 60
kick  = exp(-7 * fract(beat))
thump = sin(tau * 48 * t * (1 + 2 * kick)) * kick * 0.5
out   = thump
r     = sqrt(x * x + y * y)
color = hsv(fract(beat / 16 + r), 0.6, smoothstep(0.06, 0, abs(r - 0.4 - 0.2 * kick)))
```

The picture pulses with the kick because both lines say `kick`.

## Running it

`zig build run` (with `zig build lib --watch -fincremental` beside it for
hot reload: an edit of main.zig is back in 0.07-0.09 s). It needs pardes
beside sketchbook (`../02-pardes-code` of sketchbook's directory), whose
editor it takes as a library.

**Keys** are helix's: `i` (or `a`, `o`, `O`) to type, Escape back to
normal mode, where `h j k l w b` move, `x` takes a line, `d` deletes, `c`
changes, `r` replaces a character, `u` and `U` undo and redo, `%` takes
it all. The mode is in the corner (NOR, INS, SEL). The pointer drags.

**Hear this.** Alt-o grows the selection to the expression around it,
Alt-i shrinks it back, Alt-n and Alt-p go to the next and previous: after
any of them, that expression alone is heard (as `out`) and seen (grey,
unless it is a colour) until the next key. Escape, or any edit, goes back
to the program. `f*` then `;` then `mi(` then Alt-o, in
`thump = sin(tau * 48 * t ...)`, hears the sine alone.

**Drag a number.** Press on a number and drag sideways: it moves a step
every 4 pixels, its step from how it is written (`220` by ones, `0.25`
by hundredths), and the program follows as you drag. The whole drag is
one step to undo.

**Scrub** through the console (backtick): `sessions` lists the sessions
recorded, `play session N` plays one back (`play session N at 12` paused
at 12 s in),
`pause`, `seek 5000` (to 5 s in), `step -60` (60 frames back), `stop`
(live again from that moment, the session kept as it was). A seek brings back the text, the tapes and
the picture of that moment, and the sound with them. `capture` appends
what is heard to `capture.f32` beside the save (mono f32 at the
device's rate) until `capture` again; `set crossfade false` lets an edit
click.

`zig build test` runs the language's tests, the grammar against the
language (test/grammar.zig), and test/live.script and test/story.script
(the storyboard: a tone, a drag, a gradient, a kick, a `)` broken and
mended, a probe, a bass and a ring, the session played back and sought)
headless; `zig build probe` times the interpreter on the GPU
(`--profile-gpu`).

## The language

A line is `name = expression`, a name defined once and before it is used,
so a program has no loops or recursion and always ends. Expressions:
numbers (`0.5`, `120`, `1e-3`), names, `+ - * /`, unary `-`, parentheses,
calls; `#` starts a comment. The builtins (src/Lang.zig's one table):

| | |
| --- | --- |
| `t` | seconds |
| `x`, `y` | across and up from the centre, in half the window's height (`y` -1 to 1, `x` -1.78 to 1.78 on 16:9: a circle is round); 0 in sound |
| `tau`, `pi` | a turn and half a turn, in radians |
| `sin(a)`, `cos(a)` | of radians |
| `exp(a)`, `log(a)`, `pow(a, b)`, `sqrt(a)` | e to the a, its inverse, a to the b (a not negative), square root |
| `abs(a)`, `floor(a)`, `fract(a)` | without its sign, the whole number at or below, a - floor(a) |
| `min(a, b)`, `max(a, b)`, `clamp(v, lo, hi)` | the smaller, the larger, v kept between |
| `mix(a, b, w)` | a + (b - a) * w |
| `step(edge, v)` | 0 below edge, 1 from it on |
| `smoothstep(lo, hi, v)` | 0 to 1 smoothly as v goes from lo to hi, either way round |
| `atan2(y, x)` | the angle of the point (x, y) |
| `noise(x, y, z)` | smooth noise, -1..1, a new value about every 1; y and z 0 when left out |
| `saw(p)`, `sqr(p)`, `tri(p)` | waves of period 1 (cycles, not radians), -1..1; saw and tri rise through 0 at 0, sqr is 1 then -1 |
| `seq(i, v0, v1, ...)` | the value at floor(i), round the list: a step pattern, 16 values at most |
| `hsv(h, s, v)`, `rgb(r, g, b)` | a colour (hue 0..1 round the wheel); only as the whole `color` line |

Capacities: 128 instructions a tape (a line reaches only what it uses),
512 terms and 128 names in the text, nesting 32 deep, 4 KiB of text. Past
any of them is a problem like a syntax error, in words, at its byte.

## What it cannot say

No state between samples or between frames: every sample is a function of
its time alone, every pixel of x, y and t. So no filters, feedback, delay
lines or reverb, and no picture that remembers the last frame. That is the
price of computing every sample alone on the GPU, and what makes a replayed
moment sound and look as it did. An echo can be said by evaluating again
at an earlier time (`t - 0.25` in a copy of the line). No user functions,
no vectors but the final colour, and `t` is a 32-bit float: past ten
minutes or so a high pitch begins to roughen.

## How it works

src/Lang.zig compiles the text in `update` (no allocation: fixed arrays on
the stack) into tapes of `{op, a, b, c}`, instruction i writing register
i; State keeps the editor's value (pardes-editor: 39 KiB of plain data,
4 KiB of text), both tapes, the probe's, the problem, the drag and the
time. grammar/grammar.js, generated into the committed grammar/src/parser.c,
is the same language for tree-sitter: its parse colours the text
(queries/highlights.scm, in pardes's orchard theme) and sizes a
definition's name (that query's `(definition name: (name))`, 1.2 times
the rest), gives Alt-o its nodes and a new line its indent
(queries/indents.scm). The code is one sketchbook paragraph (`f.paragraph`,
a span a run of one class), and every range, caret and mark sits where
`sk.text.places` lays its byte out; `update` finds the number under a
press with the same places, so a mixed size moves nothing out of step.
The canvas is the window's pixels, so the text is never scaled. The tapes
go to the GPU as data: the color tape as picture.frag's uniforms, the out
tape (with the one it replaces, for the crossfade) in a storage buffer for
synth.comp, both including shaders/vm.glsl, a loop over the tape with a
switch per op. No shader is compiled after the first frame, so typing
never stalls one. The picture is drawn at most 720 rows high and scaled
to the window, and a color tape over 64 instructions at half that: on an
RTX 3050, 128 instructions took 2.7 ms at 1280x720 and 0.85 at half; 60
took 1.4 at full. The sound is `pub const audio_pull =
256` and a block a frame, as much as keeps the host's lead queued: a
change is heard a pull or so after the frame that made it.