summaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'README.md')
-rw-r--r--README.md76
1 files changed, 63 insertions, 13 deletions
diff --git a/README.md b/README.md
index f166675..71142c3 100644
--- a/README.md
+++ b/README.md
@@ -67,10 +67,11 @@ is any of them, in any order: time travel is an index.
| `string(id)` | `string` |
| `events()` | everything, as it came |
-From a debugger's recording, `zt.telescope`: `instruction(m, addr, buf)` is an instruction's text, its source `line`, and for a direct call its callee's `params` (DWARF's); `frames(m, regs, out)` is the call chain with each frame's extent on the stack, `low` to `high` (a `stack()` frame's is up to `sp + 8`), walked by the call frame information the recording holds (below), else by frame pointer; `slots(m, regs, low, out)` is what each stack word from `low` is (`out[i]` the word at `low + 8 i`), a `Slot`: its `frame`, an index into `frames` (null in none), and its `role`: `.saved` (the register a function saved for its caller there: `"rbx"`, `"rbp"`, `"r12"` to `"r15"`), `.return_address`, `.local` (a variable or parameter: `name`, `size`, and `offset`, where the word is in it), or the process's first stack, `.argc`, `.argv` i, `.envp` i (i = argc, envc: the zero that ends them) and `.auxv` i (the auxiliary vector's i-th word, two an entry); `.none` where the recording does not say; `function(m, pc)` is the function that holds a pc as the recorded walk went through it, its `start` and `len` by its call frame information (a PLT stub's 16 bytes), for every frame of every stop; `got(m, slot)` is what a GOT slot holds and whether that is still `lazy` (its name has `@plt`: the way back into the PLT, so the next call runs the loader's resolver) or bound.
+From a debugger's recording, `zt.telescope`: `instruction(m, addr, buf)` is an instruction's text, its source `line`, and for a direct call its callee's `params` (DWARF's); `frames(m, regs, out)` is the call chain with each frame's extent on the stack, `low` to `high` (a `stack()` frame's is up to `sp + 8`), walked by the call frame information the recording holds (below), else by frame pointer; `slots(m, regs, low, out)` is what each stack word from `low` is (`out[i]` the word at `low + 8 i`), a `Slot`: its `frame`, an index into `frames` (null in none), and its `role`: `.saved` (the register a function saved for its caller there: `"rbx"`, `"rbp"`, `"r12"` to `"r15"`; or `"canary"`, its copy of the stack protector's value, below), `.return_address`, `.local` (a variable or parameter, a `Local`: `name`, `size`, and `offset`, where the word is in it: 0 at its first byte, 8 a word on, -4 its high half), or the process's first stack, `.argc`, `.argv` i, `.envp` i (i = argc, envc: the zero that ends them) and `.auxv` i (the auxiliary vector's i-th word, two an entry); `.none` where the recording does not say. Where small locals share a word, `role` is the one that begins first in it and `also` the others in the order they begin (two ints: `i` at offset 0 and `n` at -4, so `i, n`; a viewer writes `n, high half` from `n`'s -4 and size 4). At an instruction that saves a register (`push rbp`, the frame being made) the word it is about to write, under rsp, has the register's role already, in frame 0: the call frame information says where it goes from that instruction on (the row that begins after it); `function(m, pc)` is the function that holds a pc as the recorded walk went through it, its `start` and `len` by its call frame information (a PLT stub's 16 bytes), for every frame of every stop; `got(m, slot)` is what a GOT slot holds and whether that is still `lazy` (its name has `@plt`: the way back into the PLT, so the next call runs the loader's resolver) or bound.
For a call followed down, each frame has its module (`mapping`), whether it has source (its instructions' `line`), its function's bounds (`function`), and, from a caller, three instructions round its call: the return address, the call `back` from it, and the row before the call `back` from that.
A function as a graph (IDA's, Binary Ninja's): `blocks(m, pc, out)` gives the basic blocks of the function that holds `pc`, a `Graph`: its bounds, its blocks in address order, the one that holds the pc (`here`), and whether every instruction's text is recorded (`text`; then `instruction` reads each block's rows). A `Block` has its bounds, how it ends (`Debugger.End`: `fall` into the next, an unconditional `jump`, a conditional `branch`, `ret`, an `indirect` jump whose targets are not known, a `tail` call out of the function, a call that does not return, `noreturn`: the function's last instruction, or `abort`, `exit`, `__stack_chk_fail` and their kin) and its edges: `to`, where a jump, a taken branch, a tail call or a call that does not return goes, and `falls`, the block after it for `fall` and a branch not taken. A call that returns ends no block. The debugger records them once a function, at the first stop in it, decoded from its own instructions (its bounds from its symbol, else its unwind information: a stripped library's function has them too), with every instruction's text and the names of what its calls and its jumps out of it reach, up to `Debugger.function_text` instructions (2048); past that the blocks without the text (`sqlite3VdbeExec`, 11 077 instructions: 2 510 blocks). Code with no bounds (the C runtime's start, a JIT's) has none. Which edges a run took: by instruction, a stop whose instruction is a jump or a return has a `branch` event (a2 its pc, a0 where it goes, a1 1 when it will go there); its pc is the last instruction of the block that holds it, and the next stop is at `to` when a1 is 1, else at `falls` (a `ret`'s, at its caller). By line or by breakpoint there are no stops between, so a reader knows the blocks the stops are in and, at a stop on a jump, the edge it is about to take; the path between two stops it does not know. What it costs: the list of `anim/tools` by instruction, 243 bytes a moment against 224 (every function stopped in, recorded whole once) and 25 600 moments a second against 26 300; the first stop in `sqlite3BtreeInsert` (1 045 instructions) 118 KB and 2 to 4 ms, in `sqlite3VdbeExec` 80 KB (its blocks alone) and 6 ms; SQLite by breakpoint on `sqlite3BtreeInsert` 317 bytes a moment against 310.
Code is named at every pc, a function's first instruction too, and a PLT stub and a GOT slot for the function they lead to: `open@plt+0`, `open@got+0` (from the relocations and the dynamic symbols; an IFUNC's in libc by the function its resolver is, `libc.so.6!strlen@plt+0`); of the names libc gives one address (`open`, `open64`, `__open64`) the one the program's relocations use. A slot is named when an instruction jumps or calls through it.
+Data is named as code is, by the global or static object of the symbol tables that holds it, `+` how far into it in decimal: `head+0`, `big+4096`, `libc.so.6!main_arena+96`, `libc.so.6!_IO_2_1_stdout_+0` (libc's statics from its separate debug file). A reader gets it as it gets a code name, the string whose id is the address (`Moment.string`, `m.trace.strings.get`), for every address an instruction touches and every pointer the telescope follows into a module's data, its .bss past the file's last page included. An address no object holds (a string literal, the stack, the heap) has none; the debugger looks each new address up once.
With `--alloc`, every live allocation and the 16 bytes of allocator header before it are in `read` at each stop (its first 4 KiB; what changed, a stop at a time; 256 KiB of them at most, lowest addresses first: `Debugger.allocation_watch`).
Recorded `--by insn`, the instructions that ran between two moments are the `rip` of each moment between them; `--by line` does not keep them.
A byte in `read` is as it was when the debugger last sent it: the stack window, the watched regions and the live allocations at every stop, a `read`'s or `write`'s buffer at that syscall (`-s` bytes of it). Memory it looked at once and not since is known and may be stale.
@@ -303,11 +304,35 @@ moment against 249, 12 000 moments a second against 16 000.
A stop also says what its instruction is about to do to memory, worked
out from capstone's operands and the registers: a `touch` event (the
-address, the size, read or write) with what is there, and after the step
-what a write left there. So an instruction-level recording has every
-memory write the stepped thread makes, with nothing watched and nothing
-compiled in. A call, a jump or a return at the pc is a `branch` event:
-where it goes, and for a conditional jump whether it will.
+address, the size, read or write) with what is there (and `--around`
+bytes each side), and after the step what a write left there. So an
+instruction-level recording has every memory write the stepped thread
+makes, with nothing watched and nothing compiled in. The stack's own
+instructions touch memory no operand names, and have their touch after
+their operands' (a2 the pc, as every touch): `push`, `pushfq` and `call`
+write the 8 bytes at rsp - 8 (the pushed register, the flags, the return
+address), `enter` the saved rbp there; `pop`, `popfq` and `ret` read the 8
+at rsp, `leave` the saved rbp at rbp (2 bytes for `pushf`, `push ax`: the
+operand-size prefix). The string instructions (`movs`, `stos`, `lods`,
+`cmps`, `scas`) name theirs, at rdi and rsi; repeated (`rep`), the touch is
+the range the count covers, rcx elements up from there, or down to there
+with the direction flag set, none when rcx is 0. Stepped by instruction a
+`rep` stops after each element (rcx one less at each stop, the range one
+element shorter, the pc the same); of a range only its first 4 KiB, from
+where it begins, is recorded at a stop. A call, a jump or a return at the
+pc is a `branch` event: where it goes, and for a conditional jump whether
+it will. What the stack's touches and the names of data cost, on the list
+of `anim/tools` by instruction (`--from main --stack 1024 --around 80
+--disasm 48`, -O0, 2 464 moments): 205 more touches and 135 names, 362
+bytes a moment against 356, and 11 800 moments a second against 12 100
+(the median of eight runs each).
+
+How far an instruction moves rsp or rbp (`push` -8, `sub rsp, 0x18`
+-0x18, `leave` up to the caller's frame, `ret` +8) is no event: it is the
+next stop's register less this one's, both `Moment.regs` of the same
+thread. By instruction the next stop is the next moment, or the one after
+the syscall's when the instruction is a `syscall`; by line or breakpoint
+it is the sum of everything between.
**The call chain** is walked at every stop by the unwind tables
(`.eh_frame`, `.debug_frame`, through `std.debug.Dwarf.Unwind`), as gdb
@@ -323,16 +348,41 @@ places by it (`DW_OP_fbreg`), each with its size and its block's range of
code. `zt.telescope.frames` walks the same chain again from the recording,
which needs no ELF file, and `slots` names the words: the saved registers
and return addresses of every frame; the locals of -O0 and -Og code where
-the frame's pc is in their block (a leaf's in the red zone too); and argc,
-argv, envp and the auxiliary vector, from where the stack began (the
-string `event.frameId(0)`, `Debugger.Start`, read from `/proc/pid/stat`).
+the frame's pc is in their block (a leaf's in the red zone too), every one
+that shares a word; the stack protector's copy; and argc, argv, envp and
+the auxiliary vector, from where the stack began (the string
+`event.frameId(0)`, `Debugger.Start`, read from `/proc/pid/stat`).
+`ztrace show FILE MOMENT` prints each stack word's role after its chain:
+`saved rbp`, `saved canary`, `return address`, `buf`, `buf+8` (a word into
+it), `i[0..4], n[4..8]` (the bytes of the word each fills), `argv[1]`.
+
+**The canary.** A function built with the stack protector copies the
+value at `fs:[0x28]` into its frame and checks it before it returns: that
+word's role is `.saved = "canary"`, from the store on. It is found by
+seeing the store, not by position: when the debugger records a function's
+frame it decodes its first 64 instructions, once, for a `mov reg,
+fs:[0x28]` and the next store of that register to the stack (`mov [rbp -
+8], rax` at -O0 by gcc and clang, `mov [rsp + 0x18], rax` in libc), and
+records the word as a local with no name, placed from the frame base (from
+the CFA in a function without debug info). By position it would be a guess:
+the canary is under the saved registers, but with padding between, and a
+function with no array has none at all. It cannot be known where the store
+is past the first 64 instructions (a prologue shrink-wrapped behind an
+early return), where its register is neither the frame base nor the one
+the CFA is kept in at the store (`mov [rsp + 8], rax` in a function that
+keeps rbp, at -O2), and in a recording made without a disassembler.
+
What it does not say: a local in a register or a list of places (-O1 and
up), or in a block whose code is in pieces (`DW_AT_ranges`); a local's
name in a prologue or an epilogue where the frame base is rbp and rbp is
-not yet or no longer the frame's; one of two small locals sharing a word
-(the first keeps it); a frame whose CFA is an expression (a signal frame)
-or that has no unwind tables, which falls back on the frame pointer;
-inlined calls, which have no frame (gdb lists them).
+not yet or no longer the frame's; a frame whose CFA is an expression (a
+signal frame) or that has no unwind tables, which falls back on the frame
+pointer; inlined calls, which have no frame (gdb lists them). In
+`targets/stack.c`'s loop (gcc and clang, -O0) and in libc's `sbrk`
+between its `push rbp` and its `mov rbp, rsp` (the list of `anim/tools`),
+the roles are where gdb's `info frame` and `info locals` put the saved
+registers and the locals (`&i`, `&n`, `&buf`); gdb names no canary, whose
+word holds what `fs:[0x28]` does.
What it costs, on `fib(12)` by instruction (142 000 moments): 154 000
moments a second and 64 bytes a moment with the disassembly and the