1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
|
# computeranim
Two things that meet at a file:
- **ztrace**, a small tracer/debugger: many ways to watch a program run, one
event format, one recording.
- **canim**, animations of what was recorded (registers, memory, syscalls,
data structures, source), written as code, exported as frames for ffmpeg.
```sh
zig build # zig-out/bin/ztrace
zig build test # everything; test-<area> for one
zig build bench-store # what a recording costs a moment
ztrace record -o fib.ztrace -- ./fib 10 # under ptrace, an instruction a moment
ztrace record --syscalls -- ls / # ... or a syscall a moment
ztrace run --preload libztrace-agent.so -- ./prog # in-process, no ptrace
ztrace dump fib.ztrace
```
## The pieces
| Where | What |
| --- | --- |
| `src/trace/event.zig` | `Event` (64 bytes: time, thread, kind, six words) plus a payload; `Kind`; `Regs`; `Sink` |
| `src/trace/Ring.zig` | events from the target to the tracer through shared memory; a producer can be held at a kind (`stop_mask`) until `resumeAll` |
| `src/trace/client.zig`, `host.zig` | the ring's two ends: inside the target, inside the tracer |
| `src/trace/store.zig` | `Recorder` (a `Sink`) and `Trace`: the recording, in xitdb |
| `src/trace/ptrace.zig` | the out-of-process backend |
| `src/agent/` | the in-process backend: a library preloaded or linked into the target |
| `llvm/` | compiler instrumentation and its runtime |
| `src/trace/kernel*` | perf events and BPF |
| `anim/` | canim, a package of its own (it needs sketchbook and pardes; ztrace does not): the animation library and its scenes |
| `design/` | the visual system the animations follow, with mock frames |
| `src/cli/` | the `ztrace` command, a file a subcommand |
| `targets/` | programs to trace |
| `build/` | each area's part of the build |
Every backend produces `Event`s into a `Sink`. The recorder is one; the ring
is another, with the recorder at its far end.
## A recording
An xitdb file, kept small. A **moment** stores only its events, encoded
compactly (registers as the ones that changed since the thread's last). The
program's **state** is stored at a keyframe every 256 moments, each sharing
what did not change with the one before; the state at any moment is the
keyframe before it with the moments since replayed over it. `Trace.moment(i)`
is any of them, in any order: time travel is an index.
| `Moment` | from the events |
| --- | --- |
| `regs(tid)` | any event carrying `Regs` as its payload |
| `read(addr, buf, known)` | `mem_write`: every byte seen so far, and which are known |
| `heap()`, `allocation(addr)` | `alloc`, `free`: the live allocations |
| `stack(tid)` | `call`, `ret`: the calls a thread is inside |
| `variables()` | `variable`: the live named variables; a local goes with its call |
| `line(tid)` | `line` |
| `mappings()`, `mapping(addr)` | `map`, `unmap` |
| `string(id)` | `string` |
| `events()` | everything, as it came |
The recorder ends a moment at the kinds in `Recorder.cut` (by default a
step, a breakpoint, a syscall's exit, a signal, a source line, a mark). Of a
memory write it keeps only the bytes that differ from what it already
holds, so a backend that reads a region again pays for what changed in it.
It stops with `error.RecordingFull` at `max_bytes` (1 GiB): a recording is
meant to be megabytes.
What a moment costs (`zig build bench-store`; release, this machine):
| each moment | bytes | ns |
| --- | --- | --- |
| a mark | 29 | 560 |
| a step with registers | 57 | 1100 |
| ... and a word pushed on the stack | 75 | 960 |
| a word written somewhere in 1 MiB of heap | 717 | 14700 |
| 4 KiB written again, unchanged | 376 | 24000 |
| 4 KiB of new bytes | 12078 | 584000 |
An instruction-level recording of `fib(12)` under ptrace, 142 000 moments
with registers and the stack, is 8 MB at 197 000 moments a second; reading
it back is 178 000 moments a second in order, a millisecond to any one.
Scattered writes and new bulk data are what cost: each chunk of 64 bytes
that changed is written at the next keyframe with its path in the tree.
## Depends on
- [sketchbook](../sketchbook): window, drawing, shaders, hot reload, the
manim-style `anim` module, frame export.
- [pardes](../02-pardes-code): tree-sitter parsing, highlighting, themes.
- [xitdb](https://github.com/xit-vcs/xitdb): the recording.
x86-64 Linux, Zig 0.17.
|