diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 63 |
1 files changed, 62 insertions, 1 deletions
@@ -18,6 +18,8 @@ ztrace record --syscalls --strace -- ls / # ... or a syscall, printed as strac ztrace dbg -- ./fib 10 # step, break, look, and go back ztrace run --preload libztrace-agent.so -- ./prog # in-process, no ptrace ztrace cc -- clang sort.c -o sort && ztrace run -- ./sort # compiled in: lines, calls, variables, stores +ztrace perf --faults --syscalls --watch counter -- ./prog # the kernel's view, at full speed +ztrace probe --dry-run --usdt provider:name -- ./prog # ... with BPF, for root ztrace dump fib.ztrace ``` @@ -34,7 +36,7 @@ ztrace dump fib.ztrace | `src/trace/Symbols.zig`, `syscalls.zig` | what an address means (modules, ELF symbols, DWARF lines); syscalls by name and shape | | `src/agent/`, `include/ztrace.h` | the in-process backend: a library preloaded or linked into the target, and what the target can say through it | | `llvm/pass.cpp`, `src/instr/` | compiler instrumentation: the LLVM pass, the runtime it calls, the type descriptions | -| `src/trace/kernel*` | perf events and BPF | +| `src/trace/perf.zig`, `bpf.zig`, `usdt.zig` | the kernel's view: perf events for any user, BPF programs for root, the static probes of `<sys/sdt.h>` | | `anim/` | canim, a package of its own (it needs sketchbook and pardes; ztrace does not): the animation library and its scenes | | `design/` | the visual system the animations follow, with mock frames | | `src/cli/` | the `ztrace` command, a file a subcommand | @@ -313,6 +315,65 @@ EFAULT faults in the handler; syscalls made from the vDSO never enter the kernel and are not seen, as under ptrace; a target that needs SIGSYS or its own alternate signal stack is told it has them. +## The kernel's view + +Two tiers, by what a machine lets its user do. Both hold the program at its +first instruction until they are attached, then let it run at full speed: +no stops, records out of a kernel ring buffer, stamped with the clock every +other backend uses. + +**`ztrace perf`** needs no privilege: `perf_event_open` as +`kernel.perf_event_paranoid=2` allows it, which is user space only, on one's +own processes. `perf.Session.open(gpa, pid, options, sink)` is the same for +a tracer that has its own target (stopped, or it misses the start). + +| option | events | costs the program | bytes an event | +| --- | --- | --- | --- | +| `-e`, `--interval MS` | `sample`: a counter's total | nothing measurable | 27 | +| `--sample HZ`, `--callchain` | `sample`: pc, callers | 3% at 40 kHz with callers | 24, 48 | +| `--faults` | `fault` and the registers | 0.3 us a fault (the fault itself: 0.9) | 40 | +| `--switches` | `sched` | nothing measurable | 10 | +| `--syscalls` | `syscall_enter` with the path or bytes, `syscall_exit` | 0.4 us a syscall (ptrace: 10), and a second at the end | 23 | +| `--watch SYMBOL` | `mem_write` or `mem_read`, and the pc | 0.9 us a hit | 17 | +| `--break SYMBOL` | `probe`, five arguments, the registers | 1.3 us a hit | 73 | +| always | `map`, `thread_start`, `thread_exit`, `mark` (comm), `exit` | | | + +(Release build, an i7-11700; a ptrace stop is 5 us. It decodes and records +one to four million events a second; what the kernel could not put in its +ring is counted as lost: `--ring`.) `--threads` follows threads and +children, a ring per CPU merged by time. Moments end where the recorder +ends them anyway, a syscall's exit or a megabyte of events, unless `--cut` +asks for one at every fault and hit. What to know: + +- Only user space is seen. A page the kernel touches first (`read()` into + fresh memory) faults unseen; context switches and migrations cannot be + counted, only reported as events. +- A huge page is one fault for 2 MiB. An anonymous mapping the kernel + merges with its neighbour is reported as the merged whole. +- Syscalls come from the 736 per-syscall tracepoints, the only ones a user + may read: found by trying every id, closed at a millisecond each. +- There are four debug registers; a watchpoint covers 1, 2, 4 or 8 aligned + bytes; the record comes after the access, so the value is read when it + is decoded, later (or not at all, once the program is gone). + +**`ztrace probe`** is for root (CAP_BPF and CAP_PERFMON) and says so +otherwise; `--dry-run` builds everything and lists it. BPF programs, built +here as instruction arrays for one process, on the raw tracepoints +`sys_enter`, `sys_exit` and `sched_switch` (both threads of a switch, +through the kernel's BTF), on kprobes, on uprobes, and on USDT probes, +whose arguments are read in the kernel at the probe. They write `Event`s +into one BPF ring buffer and count what did not fit. A syscall costs 0.15 +us, a probe 0.5 to 0.8 (in a VM), 22 to 73 bytes an event. + +**USDT**, the static probes of DTrace: `usdt.probes(gpa, elf, bias)` reads +the notes `<sys/sdt.h>` leaves (provider, name, pc, semaphore, arguments) +and `usdt.Arg.parse(spec).fetch(regs, memory)` gets an argument at a +stopped thread. No BPF in it: a ptrace breakpoint on the nop does as well. + +`zig build test-kernel` runs the first tier and USDT on `targets/kernel_*`, +and the BPF programs in a small BPF machine. `zig build test-kernel-vm` +runs them for real: this machine's kernel in QEMU, where we are root. + ## Depends on - [sketchbook](../sketchbook): window, drawing, shaders, hot reload, the |
