1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
|
# canim
Animations of how a program ran, from a `ztrace` recording, on sketchbook's
`anim` (live scenes). The frame is a debugger's whole context, as pwndbg
shows it, with one lens over it; the names, sizes and colours of everything
in it are the design language's catalogue (`../design/README.md`), and how
it moves is `../design/motion.md`. Code and design share one set of names:
a block of the catalogue is a function or a component of the same name here.
```sh
zig build record # the recordings, into zig-out/recordings
zig build run -Dscene=fib # a scene in a window, hot reload (zig build lib --watch)
zig build export -Dscene=fib -- DIR --fps 60 --size 1920x1080 # frames as PNGs (or - for raw RGBA)
zig build videos -Doptimize=ReleaseFast # every scene: out/NAME-1080.mp4 and out/NAME-2160.mp4 (x264, 60 fps)
zig build test # unit tests, each scene run headless, and the gallery against its pictures
zig build gallery # the gallery's pictures drawn anew into test/gallery
zig build bench # what a recording costs a frame
zig build bench -- NAME BY [FUNCTION] # a recording's steps (BY: moment, line, over), with what each changed
python3 tools/chart.py chart out/fib-1080.mp4 --out out/motion/fib # the designer's timing chart and five checks
```
Needs what sketchbook and ztrace need, and `setarch`, `clang` (the pass
plugin), `ffmpeg` (videos), python with numpy and Pillow (charts). The
`sqlite` scene also needs SQLite's amalgamation (`-Dsqlite=DIR`, the
directory of `sqlite3.c` and `sqlite3.h`; by default
`~/05-genizah/sqlite/sqlite-amalgamation-3500400`) and `objdump`; where
the directory is not there the scene is not made, and the rest builds
and tests as before.
Dependencies are by path (`..` is ztrace, `../../sketchbook`,
`../../02-pardes-code`); `zig-pkg` is a symlink to pardes's.
## A scene
```zig
const anim = @import("anim");
const canim = @import("canim");
const recordings = @import("recordings");
pub const background = canim.background;
pub const scene = anim.Live(frame, struct {});
fn frame(s: *anim.Frame) void {
const rec = canim.open(recordings.steps); // a recording the build made and embedded
var at = canim.cursor(s, rec, rec.lineMoment(10, rec.call("main", 0).enter));
at.levels = .{ .idea = .calls, .source_rows = 10 }; // what a step is timed by
at.journey(.line, rec.call("fib", 0).exit.?); // the clock's whole way
// The stage: the band (the lens) over the program column and the state column.
const ctx = canim.context(s, at, .{ .split = canim.layout.split.subject });
const tree = canim.callTree(s, .{ .cursor = at, .stage = ctx.stage, .function = "fib" });
const joins = ctx.ties(s); // thread, link, projection; add it last
var says: canim.Narrator = .{};
canim.establish(s, .{ ctx.clock, ctx.registers, ctx.stack, ctx.source, ctx.disasm, tree, joins });
says.line(s, "`fib` calls itself."); // old caption out, a gap, new one in, then stillness
at.walk(s, .line, rec.lineMoment(6, rec.call("fib", 0).enter)); // each step in full, timed by its content
at.hurry(s, .line, rec.call("fib", 4).enter, .{}); // a run: ramps in and out, counted on the clock
ctx.stage.relayout(s, canim.layout.split.none); // a change of split, 48 f
s.wait(canim.theme.time.before_cut);
}
```
`frame` runs every frame and builds the whole timeline each time; a
component is an `anim` object (`fadeIn`, `fadeOut`) that draws the program's
state at the cursor, at the box its `Stage` gives it now.
**The cursor** (`src/cursor.zig`) moves in steps from one moment straight to
a later one (`Recording.Stop`: `.moment`, `.line`, `.over`, `.call`,
`.syscall`). For each step it writes a timetable, a `Plan`: the cascade.
The line fires (12 f); each level the step touched strikes 18 f after the one
before, in the order CPU, memory, idea (a level the scene does not show, or
the step did not touch, or touched nothing drawn, gives up its slot; a
drawing made of bytes strikes when the projection from them arrives, 24 f
after them, and its write 12 f later still; a call's node when the
projection from its return address does, 36 f after); the writes of one step strike 6 f apart; a hold to read; the
mark advances last; a rest. `walk` plays steps in full, `hurry` ramps the
pace down and up again (each step 0.6 of the one before) to a run (0.30 s a
step) or a sprint, `step` takes an explicit `Plan` (`Array.swap`,
`syscalls.step`). Components read it through `Cursor.now(at)`.
## The scenes
| Scene | Shows | Split, lens | Recording |
| --- | --- | --- | --- |
| `title` | the series and the legend of marks | a card | |
| `fib` | recursion: each call a frame and a node | `subject`, `calls` | `steps` |
| `list` | a list reversed: each `next` turned | `subject`, `nodes` | `list` |
| `sort` | insertion sort: compares and swaps | `subject`, `array` | `sort` |
| `syscalls` | open, read, close, write: the kernel's side | `subject`, `syscalls` | `hello` |
| `zoom` | one line of C as its instructions, the call's bytes | `none`, the camera at x2 | `steps` |
| `sqlite` | a row's way to disk in SQLite: from `sqlite3_step` to `sqlite3BtreeInsert` (three continues, the row's struct on the stack), the commit's `unixWrite` nine calls under `main`, `pwrite64` of page 2 into the kernel, and the page read as a B-tree leaf | `subject` with the backtrace in the drawing's place, then `syscalls`; its view a `hexdump.Dump` | `sqlite` |
| `gallery` | every block in its states | all three | `gallery` |
## The blocks
The catalogue's layers, and where each block is. Every block is drawn in
every state the code has in the `gallery` scene.
| Layer | Block | Where | Notes |
| --- | --- | --- | --- |
| 0 tokens | grid, colour, type, the splits | `src/theme.zig` | per "Token changes"; JetBrains Mono NL (`.code*` faces) for all mono, Medium under 30 |
| 1 atom | `address` (`lead`, `name`, `rwx`), `value` (and `Ui.typed`: `fd`, `bool`, `char`), `arrow` (`points`, `holds`, `more`), `mark`s, `plaque` (`chip`), `pin` (`row`, `under`), `bracket` (`quiet`, `tie`, `thread`), `tag` | `src/ui.zig` (`Ui.address`, `.value`, `.arrow`, `.written`, `.read`, `.cellWritten`, `.cellRead`, `.current`, `.trail`, `.unknown`, `.chip`, `.pinRow`, `.pinUnder`, `.bracket`, `.panel`) | marks in the context: `armed`, `struck`, `held` (a tick), `read`; in a lens's cell the same with a bar |
| | `byte` | `src/rows.zig` `bytes` | `number`, `pointer`, `text` (a string's ink), `allocator` (also bytes that are no field of what a dump is about), `unread`; `pad`; `bytesAt` moves the bytes right of a long (stack) address |
| 2 row | `chain`, `reg` (and `rip` with the flags), `slot`, `line`, `bytes`, `call` (`span`, `callRow`) | `src/rows.zig` | `chain` follows `zt.telescope.chase`; cut at its far end, `more` |
| | `insn` | `src/disasm.zig` | columns at 2, 31, 40, 134, 180, 264, 564; operands whole, the annotation from 564 or a character after them; then a size word goes, then the operands give way, never inside `[]`; `current`, `ran` (every instruction of the last three steps), `ahead`, `outside`; the jump lane from the jump just before the pc's moment |
| 3 panel | `registers`, `stack`, `backtrace` | `src/context.zig` | the stack's window on `[rsp, rbp+16)` keeping the touched word in view, rsp and rbp's pins travelling (keeping their place among the names they leave and join); a window that moves more than 16 words fades out and in round memory's strike; a frame's name longer than the spans' room shortens the chains beside it; `call` spans and the backtrace (`#n`, name, arguments at 384, file and line) from `zt.telescope.frames`, the frames two stops share sliding to their new depth |
| | `source` | `src/source.zig` | the whole file in a window, its middle third; numbers as wide as the window needs (SQLite's have five digits), the text moved right by as much; a window more than two windows away fades out and in |
| | `disasm` | `src/disasm.zig` | 17 rows (9 as subject), fewer above near a function's start |
| | `hexdump` | `src/hexdump.zig` | a lens's view (`hexdump.panel`): `bytes` rows and their reading, veiled at a sprint; `Dump`, a view of a region by the fields a scene gives it (`Field`: where, how long, read as a number, big-endian or not, a pointer, text, or left to the eye), the rows that hold a field and a `more` row for those left out, marks the scene plays, the bytes a syscall was given as the kernel took them, and only what the debugger read at the stop (`seen`) |
| | lenses: `nodes`, `array`, `calls`, `syscalls` | `src/nodes.zig`, `src/array.zig`, `src/calls.zig` (`Tree`), `src/syscalls.zig` (`Crossing`) | the drawing in `band` (over the program column), its view in `view` (over the state column): `nodes` and `array` a dump (`hexdump.panel`), `calls` the backtrace with arguments, `syscalls` the bytes the call is given or fills |
| 4 tie | `thread`, `link` (and its `pin`), `projection`, `echo` | `src/ties.zig` | panels publish places (`anchor`); `Ties`, added last, draws in the channels: a link climbs to a node by the gutter's left lane and goes down to a slot by the right, its name riding its head, a target past the stack's window a stub and a row count; a projection from the view's bytes, or from a frame's return address to its call's node |
| 5 frame | `stage` and `relayout` | `src/stage.zig` | splits `none`, `beside`, `subject` (`theme.layout.split`): the band is `band` (the drawing) and `view`; `relayout` fades the drawing round its move; `holdStack` holds the stack's window for the camera, with `free` slots |
| | `clock`, `caption`, `veil`, `card` | `src/words.zig`, `src/root.zig` (`Narrator`, `Camera`) | the clock still while a step plays, its piece fading in once the mark lands; a caption stays its reading time; the title card is `scenes/title.zig` |
| 6 verb | `fire`, `arm`, `strike`, `cool`, `read`, `extend`, `link`, `project`, `travel` (pins), `advance`, `cascade`, `run`, `sprint`, `establish` (`Context.establish`: a drawing's elements fade in rising, no lines), `scroll`, `relayout`, `visit`, `say` | `src/cursor.zig` (the timetable), the blocks (each draws its state at the instant) | tokens in `theme.time`; a level gives up its slot when nothing it touched is drawn (`Recording.drawn`) |
`Recording` (`src/Recording.zig`) is the index every block reads: lines,
calls, syscalls, allocations, which registers and bytes each moment changed
and read (a debugger's `touch` reads count), each moment's `touch` and
`branch`, and each instruction's line.
## The recordings
`build.zig` records four programs of `tools/` under `ztrace record` (the
debugger: registers, the stack, what they point at, the instructions round
the pc with their lines and callees' parameter counts, at every stop), each
by a bare name in a directory of its own with an empty environment, without
address randomisation, with the tools built `ReleaseFast` (a step through
the agent's or the runtime's code is a moment, and a slow tracer stretches
the clock's retry loops it steps through). The instrumented ones are built
`-no-pie -fno-stack-protector -Wl,-z,now`: short addresses, no resolver
and no canary stepped through that the source does not say.
| Name | Program | Stops | Moments, bytes | Also |
| --- | --- | --- | --- | --- |
| `steps` | `fib.c`, static musl, `-O0` | every instruction | 1999, 336 KB | |
| `hello` | `hello.c`, static musl | every line | 233, 101 KB | reads `tools/note` |
| `sort` | `sort.c`, `ztrace cc --instr vars --only sort,a` | every instruction, `--from main` | 5076, 617 KB | `--ring`: the variables' names and types; each `__zt_var` call is stepped (4 of them) |
| `list` | `list.c`, `ztrace cc --instr vars --only push,reverse,head` | every instruction, `--from reverse` | 3421, 528 KB | `--alloc --preload` the agent: allocator headers are read; the pushes (the agent's and the runtime's first calls) run through |
| `gallery` | the old `list` by line, kept in `test/gallery/list.ztrace` | | 134, 88 KB | the gallery's pictures depend on the drawing only |
| `sqlite` | `sqlite.c` (three rows into a table, committed) with SQLite 3.50.4's amalgamation, system clang `-O2 -g -fno-omit-frame-pointer -no-pie -fno-stack-protector -Wl,-z,now`, glibc | `--by break`: `sqlite3_step`, `sqlite3BtreeInsert`, `unixWrite`, and the instruction after its `pwrite64` (found by objdump); every syscall recorded, `-s 4096`, `--stack 512` | 181 (42 stops, 138 syscalls), 95 KB | made only with `-Dsqlite=DIR` (default `~/05-genizah/sqlite/sqlite-amalgamation-3500400`), in a new temporary directory each time; `sqlite3.c` is embedded as the source |
A recording that begins inside a call (`--from`) has that call open from
its first stop (`Recording.index`, from the frame chain). Writes are the
bytes whose value changed, to the element the instruction stored (its
`touch`); a store of the same value is kept apart (`store_spans`) and
counts as assigning a local. A register's change over a step is net: a
call's rsp put back is not struck. The vDSO's `clock_gettime` is stepped
through under the runtime and the agent: its retry loop makes the counts
vary a little from one recording to the next. An address no symbol
holds is named from its module's first mapping (`list+2c89`).
## Motion, as measured
`tools/chart.py chart` on the 1080p exports in `out/` (the designer's five
checks; its regions follow the two-panel band now). Encoded with `-g 600`:
x264's keyframe at 10, 20, 30 s is a one-frame fade the instrument sees.
| | fib | list | sort | syscalls | zoom | sqlite |
| --- | --- | --- | --- | --- | --- | --- |
| POP, round 2 / 3 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | – / 0 |
| CROWD, round 2 / 3 | 7 / 2 | 2 / 7 | 10 / 6 | 5 / 1 | 4 / 3 | – / 1 |
| RUSH, round 2 / 3 | 0 / 0 | 0 / 2 | 2 / 0 | 1 / 2 | 0 / 0 | – / 1 |
| AIR, round 3 | 0 | 0 | 0 | 0 | 1 | 4 |
| seconds, round 3 | 41.5 | 60.5 | 42.0 | 38.5 | 32.0 | 64.6 |
(sort as the coordinator re-exported it without the camera; syscalls and
sqlite after the crossing's changes: a register carries a call only while
it holds the argument at the stop shown, `rax` is struck after the
kernel's `returns` row; fib, list and zoom are the round-3 exports, made
before the stack's and backtrace's motions below.)
What fires, and why it is left:
- CROWD: fib 12.05 and 27.20, list 30.62 and 32.18, sort 17.58 are the
veil coming over the registers, stack and disassembly at the first
shortened step (one veil over three panels); zoom 9.65 and 21.55 the
camera (the regions are fixed on the screen); zoom 11.75 the stack's
window scrolling to its hold under the camera (the stack fills every
region there). The rest (list 15.92, 19.32 and the others like them,
sort 11.68, 27.12, 29.42 and the others, syscalls 35.35, sqlite 43.40)
are an `advance`: the thread, the plaques and rip move as one mark, the
source and the disassembly scroll with it, and a link or projection
goes back into its pin, all on the catalogue's one frame.
- RUSH: list 29.20 and 29.55, syscalls 29.37 and 29.72, sqlite 9.10: a
caption arrives, and the next change the instrument sees is the
keyframe at 30.00 (10.00) (a fade of every region over one frame,
0.00% at its peak).
- AIR: a caption's reading time on a still picture: the last one's and
the 2 s on the finished picture (zoom 26.88, 3.1 s; sqlite 60.02,
4.6 s), and in sqlite three more, each a dense caption over a stop
(11.38, 7.2 s: the arguments; 32.72, 3.5 s: the commit's backtrace;
50.03, 4.9 s: the page's header), its fields framed as it names them.
## What is not done
- Back pointers between nodes still run under the nodes: a page keyline
round each pin's stub and head shows them passing behind it (there is
no lane for them in `beside`).
- Names keep their side when one of two leaves a pin and when a pin's
name travels; a name that arrives where another already is takes the
right, and once it has landed the pin's own order (variables, then
registers) may put it on the other side. (In the stack's lane, rsp or
rbp travelling alone keeps its place among the names it leaves and
joins; both travelling to rows with other names is not handled.)
- The hexdump's reading of a `char` or `bool` field (the lens's values
and cells read them by type; the dump's reading column does not).
- A link to a slot past the stack's window ends in its stub and row count:
drawn in the sqlite scene only (rsi to the struct on the caller's
stack, `23 rows`, 4.2 s).
- The calls tree's leaf spacing holds for fib(5); past about ten leaves
nodes and badges collide (the catalogue's known case).
- `heap` and `pages` lenses are not in the band (`Timeline` is the old
far view).
- A recording holds the bytes it has seen, each as last seen: it cannot
tell a byte read again and unchanged from one not read since. Memory
nothing watches (SQLite's page cache) is read at a stop only where a
pointer leads (32 bytes), so a view of it says what the stop read
(`hexdump.Fields.seen`); a syscall's buffer is shown as the kernel took
it, from its capture. The recorder should say what it read at each stop.
- A frame's file and line are known only where the recording stopped in
it: return addresses are recorded with names, not lines, so a deep
backtrace's older rows have none.
- A program's frame arguments are in the backtrace only for calls the
recording tells (an instrumented program's); at a breakpoint the
callee's are in its registers, which the scene's captions name.
- The sqlite scene changes what is over the band (the backtrace, then the
kernel's side) and its view (the SQL at rbx, the row, the page) by a
fade, not by a verb of the catalogue; the backtrace in the band's
drawing is a split the catalogue does not have (`subject` with the
backtrace where the drawing goes: eleven rows).
- `zig build test` prints "failed command" for each scripted scene: that
is the run echoing its stderr (the Vulkan headless warning), not a
failure; the summary line is the verdict.
|