| Commit message (Collapse) | Author | Age |
| |
|
|
|
|
|
|
| |
The loader's syscalls on the way to main used up the count: --steps 120
on a dynamic program left ninety instructions.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Y9MNCKgYCGiANnEwcEQxa
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
record --heap keeps the arena, the tcache and, of every chunk, its size
word and the links of those on a list: free chunks stay readable, which
--alloc dropped at the free. zt.heap.glibc gives the chunks in address
order, the tcache's and the bins' chains (safe-linking as the stored
value and the pointer it protects), the top; each with the address of
the word it was read from, each chain saying how it ended. ztrace show
prints it. Layouts for glibc 2.44 (no fastbins; a tcache made at the
first free, counting the slots left) and 2.40; the arena found by symbol
or by the struct whose next points at itself. Against pwndbg on 2.40:
the same at every stop. Three bytes a moment and a sixth slower on the
list by instruction.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Y9MNCKgYCGiANnEwcEQxa
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
backtrace and frames follow .debug_frame or .eh_frame where there is
one (PLT entries expanded into exact rows) and the frame pointer where
there is none: the caller is no longer lost in a prologue, in a function
that keeps no frame, in a PLT entry or inside libc's syscall wrappers.
Against gdb: the same chains on fib and on SQLite, where the old walk
lost __sbrk and unixSync.
A recording holds, once a function, its unwind rows and its locals on
the stack (DW_OP_fbreg, with the block each lives in), and the process's
first stack pointer. telescope.slots says of each word what it is: a
saved register and which, a return address, a local by name, argc, argv,
envp, auxv. Four bytes a moment on the list by instruction, and no
slower.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Y9MNCKgYCGiANnEwcEQxa
|
| |
|
|
|
|
|
|
|
| |
The 128 bytes under the stack pointer are recorded with the stack; what
an instruction reads or writes is recorded with --around bytes each side
of it, as what a pointer leads to is: a viewer's memory panel shows the
neighbourhood, not the word.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
| |
--around BYTES of memory each side of what a pointer leads to, for a
viewer that shows the region and not the word; --disasm ROWS above and
below the pc; and every word of the --stack bytes is followed as a
pointer, not the first sixteen.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
| |
The debugger records the call each frame returns from, as an instruction
with its line; canim reads it. main is at list.c:41 while reverse()
runs, not at 42, and the rows of a stop deep in SQLite have their lines.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
| |
A write's buffer was recorded as the syscall's payload only: a reader of
the moment's memory found there what an earlier stop had seen of it
(zeros, in SQLite's page). And record --from with --by syscall took the
loader's first syscall for the place and gave up its breakpoint. Both
found by the first animation of a real program.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
| |
It was the address in the file, which in a program that is not relocated
is the address again: list!402c89 for 402c89. Now list+2c89.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
| |
By instruction, a dynamic program's loader and C library start are a
million moments nobody watches: the list of anim/tools from main to its
exit is 3 743.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An instruction's string carries its source line and, for a direct call,
how many parameters the callee takes (DWARF); telescope.frames gives each
frame's extent on the stack; with --alloc the debugger watches each live
allocation and the allocator's header before it, changes only, 256 KiB a
stop at most. C++ names are shown and found as the source has them (the
command links the C++ runtime for its demangler).
SQLite 3.50.4 through every backend, measured (README, bench/sqlite.sh).
It found two faults: a file compiled by its full path was recorded under
a path joined onto the compile directory, and ztrace run noticed a
target's exit up to 10 ms late.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
| |
debug info
Events from the target's ring pass a tap on their way to the sink: a
variable declared there (the pass's --instr vars, the agent's ztrace_watch)
joins the regions recorded at every stop. With nothing else instrumented
that is names and types from the compiler, values and everything else from
the debugger. The instrumentation runtime is built stripped, so stepping
by line runs through its calls.
|
| |
|
|
|
| |
names(nr) gives fd, buf, count for read and its like: what a viewer of a
syscall labels an argument with, where it had only the argument's kind.
|
| |
|
|
|
|
|
|
| |
A function is decoded once from its start, so the instructions before the
pc are known: each instruction's string now says the length of the one
before it, and ztrace show prints eight above and eight below, as pwndbg
does. A call, a jump or a return at the pc is a branch event with its
target (named) and, for a conditional jump, whether the flags take it.
|
| |
|
|
|
|
|
|
|
|
|
| |
At a stop the instruction at the pc is decoded against the registers it
will run with (capstone's operands): a touch event for each memory operand
(address, size, read or write), what is there recorded with it, and what a
write left there read back at the next stop. An instruction-level
recording so has every memory write of the stepped thread, nothing watched.
The frames of the call chain are recorded and their return addresses
named; telescope.backtrace walks them from the recording and ztrace show
prints them. Process.read no longer panics on address 0.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For a viewer to show what pwndbg's context shows, the debugger records it
while the memory is there: each register and the top of the stack followed
as pointers three deep (--telescope), every code address met named, eight
instructions from the pc disassembled by capstone (built from source for
x86, into the command only; pinned as in 03-codenomicom). zt.telescope
reads a chain back with the kind of memory each link points into, and
ztrace show prints the context, coloured on a terminal as pwndbg colours.
The thirty thousand strings of a small run showed that a string map in
each keyframe costs 700 bytes of tree a string: strings are now one table
for the recording, appended a batch and read whole at open. fib(12) by
instruction with everything on: 9 MB, 71 000 moments/s.
|
| |
|
|
|
|
|
|
|
|
|
| |
emit compared a head loaded earlier with a tail read later: when other
producers reserved and the consumer consumed in between, the tail was
past it and the subtraction underflowed. A panic in safe builds (the
ring's own thread test aborted twice in 200 runs; a debug agent in a
threaded target would die the same way), a spurious wait otherwise.
Saturating: not full, and the exchange that follows fails and reloads.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
not a handler's
The kernel starts a signal handler with default PKRU (key 0 only) and
default MXCSR and x87 control word, the thread's own saved in the frame.
The SIGSYS handler makes the thread's syscalls and clones its threads
from there, so under --syscalls:
- a pkey_alloc's new rights were undone when the handler returned;
- a thread was born with no right to keyed memory (V8's compiler
threads fault reading the code space: d8 died of SEGV_PKUERR, and the
35 seconds were its core dump, not a spin);
- a buffer in keyed memory was EFAULT to the kernel, and a fault to the
agent reading it for the event;
- a thread started without its maker's float modes (flush-to-zero).
Both handlers now take the thread's PKRU and control words from the frame
first (signal.adopt), and the SIGSYS one writes PKRU back to the frame
after the syscall. Watched memory is read, and hooked code patched, with
every key open, so a key the thread itself cannot use hides nothing.
test-agent: agent_pkey (a keyed page read, given to syscalls, watched,
read by a later thread; skips itself without keys), and agent_sys's
thread checks its MXCSR. README: --alloc and programs with their own
malloc; the core dump.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
| |
musl's printf and Zig programs write through writev: pardes drawing an
image showed nothing of the kitty graphics commands it sends until this.
The README says what Lua and pardes were run with and what they showed.
|
| |
|
|
|
|
|
|
| |
Lua 5.4 through ztrace cc with everything instrumented, on a ten-line
script: 3.2 million moments, and at a keyframe every 256 the keyframes
were half of its 304 MB. At 4096 it is 156 MB and records in 1.7 s
instead of 3.9; a seek to any moment is a millisecond on average, three at
worst, and reading in order 6 µs a moment. The README has the table.
|
| |
|
|
|
|
|
|
|
| |
Debugger.usdt arms the USDT probes of every mapped module as breakpoints
whose hook records a probe event with its arguments and lets the program
run on, raising a probe's semaphore first; record and dbg take
usdt:PROVIDER:NAME (* for any), running the loader through when the probe
is a library's. glibc's own 78 probes are there for the asking:
memory_sbrk_more shows malloc growing the heap.
|
| |
|
|
|
|
| |
--preload and --alloc on ztrace perf: what the agent says and what the
kernel sees are merged by time into one recording (perf.Sorted, as probe
merges BPF and perf). test-together runs a list under both.
|
| |
|
|
|
|
|
|
|
|
|
| |
Symbols come from Symbols (perf.lookup is gone), string ids from
event.stringId, and ptrace.Process is used as it is now. A USDT
semaphore of the program itself is raised by the tracer: the kernel's
counter landed on the wrong word of a small lld-linked target, whose
page of the file is mapped writable twice. test-kernel also runs the
library tests of perf, usdt and bpf.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
| |
Syscall entries and kprobes send the event alone (a kernel function's
registers are not the thread's; a syscall's are its arguments), a
probe's event is not inherited (it failed the target's clone). The
per-syscall tracepoints are told entry from exit on the events
themselves, not on a second set. The VM has two CPUs and a time limit.
README: the kernel's view, with what each mode costs.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Programs built as instruction arrays for one process (no compiler, no
libbpf), writing Events into a BPF ring buffer; a drop counter in a map;
task_struct offsets from the kernel's BTF; a USDT probe's arguments read
in the kernel at the probe. Without CAP_BPF and CAP_PERFMON it says so;
--dry-run builds and lists everything. Tested offline by running the
programs in a small BPF machine, and for real by test-kernel-vm, which
boots this kernel in QEMU where we are root.
ztrace perf: moments are no longer cut per event unless --cut;
--max-size, and a full recording stops the target and says so.
perf.Sorted merges two backends' events by time.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
perf_event_open on a held child: counters, a sampling profile, every
page fault with its registers, maps, threads, context switches, every
syscall (the per-syscall tracepoints, the only ones open to a user),
and the debug registers as watchpoints and breakpoints that do not stop
the target. One time-ordered stream of Events into a Sink; --threads
follows with a ring per CPU. usdt.zig: the notes of <sys/sdt.h> and
their arguments from a Regs and a memory reader, independent of BPF.
test-kernel runs all of it on targets/kernel_*.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
ztrace record and dbg take --ring, --preload and --alloc: the target gets
the ring ztrace run gives it, and the debugger drains it while the target
runs (polling for the stop, so a full ring cannot hold both) and before it
reports a stop. A program compiled by ztrace cc, with the agent preloaded,
stopped at breakpoints, is one recording in order; test-together checks it.
Recording such a program showed the debugger's file names and the pass's
strings both counted from 1: event.stringId (a hash, top bit set) is now
what every producer without a table of its own uses.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
zig build bench-agent runs each capability against its native loop and
prints what it costs and writes; README has the table. ztrace run looks
at the ring a little longer before sleeping (a target that finds the
tracer asleep pays a futex wake an event). A hooked function's name is
defined as the string of its address, so the recorder's call stack reads
fib. A static link without libc's malloc says which -u to add. The
archive carries no stack probes, so cc links it in debug builds too.
Tests for --wrap (musl, zig cc) and for a recording that fills.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
| |
A full ring made consumer and producer trade a futex wake for every
record: 2.8 M events/s through ztrace run where the recorder alone does
12 M. One comparison in advance.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
what a recording may hold
Syscall User Dispatch now runs its handler on an alternate stack a thread
(a thread's own stack may not hold a signal frame; node makes such
threads), and makes clone, clone3 and vfork itself: the child leaves
through the gate with the target's registers instead of being stepped
into with the trap flag. Paths and written buffers are cut at 256 bytes
(--capture), a watch or snapshot above 1 MiB is refused, and ztrace run
ends a recording that is full (--max-size) with the target killed and the
file closed.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
libztrace-agent.so (preloaded) and libztrace-agent.a (linked in), one
source: map and thread_start at startup; alloc/free from the interposed
allocator; syscall_enter/exit through Syscall User Dispatch; call/ret
from int3 hooks; mem_write from watched regions diffed in-process; the
target-side API (include/ztrace.h, and the ztrace-agent module for Zig).
ztrace run turns each on (--alloc --syscalls --hook --watch), chooses the
moments (--cut), and stops the target at event kinds without ptrace
(--stop-at, --read, -i). test-agent records real targets and checks the
recordings.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
| |
The gettid per event is gone (a thread-local cache; a forked child resets
it), and environ/getenv are public: the agent reads its ZTRACE_* options
the way the ring's fd was read.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
| |
The consumer cached nothing and re-read head for every record, the
producers tail: the two cache lines bounced on each event. From the LLVM
runtime's profile: an event through the ring 65 ns to 29 ns.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
llvm/pass.cpp is an LLVM pass plugin: lines, stores, loads, calls and
variables, each call into the runtime carrying names and type descriptions,
selected by globs (-ztrace-only, -ztrace-mem), before the optimizer by
default. src/instr/rt.zig is the runtime (libztrace-rt.a), which also
answers clang's own coverage and -finstrument-functions hooks;
src/instr/types.zig the grammar of the type descriptions and its parser.
ztrace cc adds all of it to a compiler command.
test-llvm compiles fib, two sorts, a linked list and a BFS through ztrace
cc and asserts on their recordings; bench-llvm measures what each selection
costs.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
| |
Comments only: call's callee is a name's string id when the backend has
names, a3 its type; ret's aux is the size of the value; a variable's aux
its parameter number; a pc may be 0; a mem_write may hold the first bytes
of a large write.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
| |
Session.spawn stores Options.stop_mask in the ring before it forks, so a
tracer that steps a target by its events is not racing the target's first
one.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
| |
A linked list, a graph search in Zig, the same C under glibc and musl, two
threads. The tests drive the debugger over them: breakpoint counts, the
lines a step walks (as gdb), a list snapshotted by a hook, a watch's
words, the syscalls that tell glibc from musl, threads, attach, and
scripted dbg and record sessions. The bench measures steps, stops and
memory access from bare ptrace up to a recording.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
threads; ztrace dbg
ptrace.zig follows the threads a target makes, attaches, tells a step from
a breakpoint, and writes memory in one call (through /proc/pid/mem where
the page is read-only). Debugger.zig is the layer above: cont, stepi, step
(a source line, through std.debug's DWARF), breakpoints with hooks, syscall
stops and hooks with the path or buffer captured, watched regions sent as
the words that changed, and every stop an event. Symbols.zig is modules,
ELF symbols and line tables; syscalls.zig their names and shapes.
ztrace dbg is a small REPL on it that records as it goes and looks back
into the recording; ztrace record stops by instruction, line, syscall or
breakpoint, prints syscalls as strace does, and ends cleanly when the
recording is full.
Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Folding every event into persistent state at every moment cost 750 bytes a
step and 2 KB a scattered 8-byte write: the tree's path copied each time.
Now a moment stores its events alone, in a compact code (a flags byte,
varints, time as a difference, registers as the ones that changed), and the
state (threads, memory, heap, variables, mappings, strings) is written at a
keyframe a batch, sharing with the one before. A reader replays at most a
batch over a keyframe, and keeps four positions replayed, so playing
forward reads one moment a step.
Of a memory write the recorder keeps the spans that differ from what it
holds, so a region read again costs what changed in it. It stops at
max_bytes (1 GiB).
A step with registers: 749 bytes to 57. fib(12) by instruction, 142 000
moments: 111 MB to 8 MB, 197 000 moments/s. bench-store measures the rest.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mappings in each
A transaction a moment cost 10 µs and 1 KB before the moment's own data:
the list's path copied each time. Moments now gather 256 to a transaction,
each frozen by its own cursor so the list grows in place, and a thread's
fields sit in the moment's map instead of two maps down: a small moment is
284 bytes and 1 µs, an instruction step with registers 780 bytes.
A moment also folds alloc/free into the live heap, call/ret into each
thread's stack, variable into the live variables (a local leaves with its
call), line, and map/unmap, so a viewer reads any of them at any moment
without replaying.
|
|
|
One 64-byte Event for every backend; a shared-memory ring from target to
tracer with stop/resume; a Recorder that folds events into xitdb moments
(registers, memory in 64-byte chunks with known-byte masks, strings) and a
Trace that reads any moment back; a minimal ptrace Process; ztrace record,
run and dump. Per-area build files so the areas grow apart.
|