crash: a panic writes itself down beside the init file, where stderr cannot lose it

Every crash this program has ever had went to stderr and nowhere else, and
stderr is the one place it cannot keep anything. In the tty shell stderr IS the
screen, so the trace lands on the grid the terminal is being reset out of; the
SDL and AppKit shells have no terminal at all; a --detach session's goes
wherever its launcher left it. src/crash.zig appends a record to
<config dir>/crashes first: one line naming the build (version, commit, UTC,
os-arch, pid) and under it the panic message and the frames behind it.

RETURN ADDRESSES and not the symbolised trace, which is measured rather than
chosen. `std.debug.writeCurrentStackTrace` called from a panic handler BEFORE
defaultPanic wedges the process at 0% CPU: symbolising reads DWARF, that read
can itself panic, and the staging which turns a nested panic into "aborting due
to recursive panic" is defaultPanic's own and private. Reproduced in a
standalone build with this program's std_options_debug_io and inside a test
binary. `captureCurrentStackTrace` only walks frames, so the addresses go in
the file and `addr2line -e` finishes the job; stderr still gets the symbolised
trace from defaultPanic, unchanged.

The AppKit shell gets a panic handler of its own here too: the macOS build
roots at macos.zig, so main.zig's had never run there — in the shell with the
least useful stderr of the four. The config directory is COPIED rather than
borrowed, because that host's lives in an arena its own errdefer frees. One
record at a time, so two panicking threads cannot interleave into one buffer.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_016Q4RATpafkwahrovHQLKRf

5 files changed, 320 insertions, 0 deletions

diff --git a/docs/config.md b/docs/config.md
index 7982ccee..b95f207e 100644
--- a/docs/config.md
+++ b/docs/config.md
@@ -93,6 +93,44 @@ and `ayu_mirage` is helix's `ayu_mirage.toml`. The suffix goes on all of them
 rather than only the eight that clash today, so a name cannot move when either
 project gains or loses a file. A name that is not in the ring is ignored.
 
+## Crash records
+
+A panic appends to `crashes` in that same directory, beside `init`, and only
+then prints to stderr (`src/crash.zig`, wired into the panic handlers in
+`main.zig` and — because the macOS build roots there — `macos.zig`). stderr is
+the one place this program cannot keep a trace: in the TTY shell stderr IS the
+screen, so the trace lands on a grid the terminal is being reset out of; the SDL
+and AppKit shells have no terminal at all; and a `--detach` session's stderr
+goes wherever its launcher left it. The file is appended, never rewritten, and
+each record is one line of build metadata, the panic message, and the return
+addresses behind it:
+
+```text
+pardes 0.0.2 (a1b2c3d) 2026-09-03T11:20:44Z linux-x86_64 pid 48812
+panic: index out of bounds: index 4, len 4
+  0x11ccb5a
+  0x11cc84c
+  0x11cc67a
+```
+
+`addr2line -e <the pardes binary>` turns those into source lines, against the
+build the metadata line names. They are addresses rather than the symbolised
+trace stderr gets for a measured reason: symbolising from inside a panic
+handler, before `std.debug.defaultPanic` has run, HANGS the process — reading
+DWARF can itself panic, and the staging that turns a nested panic into
+"aborting due to recursive panic" is `defaultPanic`'s own and private. Walking
+frames is safe; symbolising them is not.
+
+Everything about it is best effort and silent: no config directory (a launch
+with no `HOME`) means no file, and a directory that cannot be created or opened
+leaves the panic exactly as it was before — stderr alone. The directory itself
+is created if it does not exist, because the user who never wrote an `init` is
+as likely as any other to hit a bug. One record at a time: two threads panicking
+at once would otherwise interleave into one buffer, so the second falls straight
+through to stderr. Only panics come here; a SIGSEGV is caught one level lower
+(`main.zig`'s `debug.handleSegfault`) and unwinding one needs the signal's saved
+CPU context.
+
 ## Runtime theme files
 
 `ThemeFile <path>` loads one complete theme from a `.zon` file. An absolute
diff --git a/src/CHANGELOG.md b/src/CHANGELOG.md
index 714497b6..4fef47fc 100644
--- a/src/CHANGELOG.md
+++ b/src/CHANGELOG.md
@@ -2,6 +2,30 @@
 
 ## 0.0.2
 
+- A panic is written down somewhere it survives. Every crash this program has
+  ever had went to stderr and nowhere else, and stderr is the one place it
+  cannot keep anything: in the TTY shell stderr IS the screen, so the trace
+  lands on the grid the terminal is being reset out of and the next `clear`
+  takes it; the SDL and AppKit shells have no terminal at all; a `--detach`
+  session's goes wherever its launcher left it, which is usually nowhere. So
+  the message and the frames behind it are now appended to `crashes` beside the
+  `init` file, under one line naming the build that produced them — version,
+  commit, UTC timestamp, os-arch and pid — which is exactly the set a bug
+  report needs and the set a reporter cannot be asked to reconstruct after the
+  fact. `src/crash.zig` runs inside the panic handler, so it takes no lock of
+  this program's, allocates nothing of its own, and every failure is swallowed:
+  a crash file that could not be written must not become the crash, and stderr
+  still gets its own copy either way. The file gets RETURN ADDRESSES rather
+  than the symbolised trace, and that is measured rather than chosen —
+  `writeCurrentStackTrace` called from a panic handler *before* `defaultPanic`
+  wedges the process at 0% CPU: symbolising reads DWARF, that read can panic,
+  and the staging which turns a nested panic into "aborting due to recursive
+  panic" is `defaultPanic`'s own and private. Walking frames is safe, so the
+  addresses go in the file and `addr2line -e` against the build named on the
+  line above them finishes the job. The AppKit shell got a panic handler of its
+  own in the process: the macOS build roots at `macos.zig`, so the one in
+  `main.zig` had never run there — in the shell with the least useful stderr of
+  the four.
 - A filtered terminal costs what an unfiltered one does. `Filter`'s second
   stage asked `RGB.contrast` for every cell it painted, and that call ends in
   `std.math.pow` six times over — a libm round trip per cell, per frame, to
diff --git a/src/crash.zig b/src/crash.zig
new file mode 100644
index 00000000..8de2abed
--- /dev/null
+++ b/src/crash.zig
@@ -0,0 +1,226 @@
+//! The crash file: what a panic leaves behind for the human who has to report it.
+//!
+//! stderr is where a panic has always gone, and stderr is the worst place this
+//! program has. In the tty shell stderr IS the screen — the trace lands on the
+//! grid the terminal is being reset out of, and the next redraw or the next
+//! `clear` takes it with it. The GUI and the AppKit shells have no terminal at
+//! all, so it goes to a console nobody has open. A detached session's goes
+//! wherever the launcher left it, which is usually /dev/null. So a record is
+//! appended to `<config dir>/crashes` as well, under one line naming the build
+//! that produced it: a bug report needs the version and the commit, and a user
+//! who runs pardes already knows where its config lives.
+//!
+//! The record is the metadata line, the panic message, and the RETURN
+//! ADDRESSES — not the symbolised trace stderr gets. `record` says why in as
+//! many words: symbolising from out here hangs the process, measured.
+//!
+//! Panics only. A SIGSEGV never reaches a panic handler (main.zig `debug`
+//! hooks that path separately) and unwinding one needs the signal's saved cpu
+//! context, which is a different job than this one.
+
+const std = @import("std");
+const builtin = @import("builtin");
+const pardes = @import("pardes.zig");
+
+/// Beside `init`, so `Config` opens the directory that holds both.
+pub const name = "crashes";
+
+/// The config directory `user_config.load` resolved, COPIED rather than
+/// borrowed: main.zig hands over an arena slice that outlives the process, but
+/// the AppKit host's lives in a `config_arena` its own `errdefer` frees on a
+/// failed init and its teardown frees at quit — and a panic after either would
+/// have built a path out of freed memory and then created a directory at it.
+/// A panic handler is the one caller that cannot check whether its input is
+/// still alive, so it does not borrow.
+var dir_buf: [1024]u8 = undefined;
+var dir_len: usize = 0;
+
+/// Called where the launcher sets `Options.config_dir`. Ignores a path too long
+/// to hold, which is a crash file that never appears rather than a truncated
+/// path pointing somewhere real.
+pub fn setDir(path: []const u8) void {
+    if (path.len == 0 or path.len > dir_buf.len) return;
+    @memcpy(dir_buf[0..path.len], path);
+    dir_len = path.len;
+}
+
+/// Built at comptime out of the two halves main.zig's `--version` prints, and
+/// for the same reason: there is nothing here to format at runtime. A build
+/// from a tarball says `pardes 0.0.1` rather than inventing a revision.
+const build_id = if (pardes.commit) |c|
+    "pardes " ++ pardes.version ++ " (" ++ c ++ ")"
+else
+    "pardes " ++ pardes.version;
+
+/// The whole record is formatted here before a byte of it is written, because
+/// the fd is opened `O_APPEND` and one `write(2)` of the lot is what keeps two
+/// crashing processes from interleaving their records line by line. Static
+/// rather than a local: a panic handler may be running on a thread stack of a
+/// few pages.
+var scratch: [8 << 10]u8 = undefined;
+
+/// Deep enough to reach `main` through any of this program's loops, and the
+/// only sizing decision here: the addresses are 8 bytes each and cost a line.
+var addr_buf: [64]usize = undefined;
+
+/// One at a time, and never inside itself. `defaultPanic` gets both properties
+/// from a private `panic_stage` and from the stderr lock, and neither is
+/// reachable from out here: two threads panicking at once would interleave
+/// `@memcpy`s into one `scratch` and each write out the mixture, and a panic
+/// raised INSIDE this function — the walk below is the plausible place — would
+/// re-enter it with the fd still open. Both end here instead, and the loser
+/// falls through to `defaultPanic`, which still says everything on stderr.
+var recording: std.atomic.Value(bool) = .init(false);
+
+/// Append one crash record, or silently do nothing. Called FROM the panic
+/// handler, so it takes no lock of this program's, allocates nothing of its
+/// own, and does its own file I/O: a static buffer, one `open`, one `write`.
+/// Every failure is swallowed — a crash file that could not be written must not
+/// become the crash, and stderr is about to get the message either way.
+pub fn record(msg: []const u8, first_trace_addr: ?usize) void {
+    if (dir_len == 0) return;
+    if (recording.swap(true, .seq_cst)) return;
+    // Released on the way out, which matters to nobody at panic time — the
+    // process is about to abort — and lets a test call this twice.
+    defer recording.store(false, .seq_cst);
+    const config_dir = dir_buf[0..dir_len];
+    var path_buf: [1024:0]u8 = undefined;
+    if (config_dir.len + 1 + name.len >= path_buf.len) return;
+
+    // The directory is where the config WOULD be, not where it is: a user who
+    // never wrote an init file has no config directory, and the first thing
+    // they have to report is this. Parents are assumed and EEXIST is the
+    // normal answer, exactly as dump.zig bets for its own directory.
+    @memcpy(path_buf[0..config_dir.len], config_dir);
+    path_buf[config_dir.len] = 0;
+    _ = std.c.mkdir(path_buf[0..config_dir.len :0], 0o755);
+
+    path_buf[config_dir.len] = '/';
+    @memcpy(path_buf[config_dir.len + 1 ..][0..name.len], name);
+    path_buf[config_dir.len + 1 + name.len] = 0;
+    const path = path_buf[0 .. config_dir.len + 1 + name.len :0];
+
+    const fd = std.c.open(path, .{
+        .ACCMODE = .WRONLY,
+        .CREAT = true,
+        .APPEND = true,
+    }, @as(std.c.mode_t, 0o600));
+    if (fd < 0) return;
+    defer _ = std.c.close(fd);
+
+    var w: std.Io.Writer = .fixed(&scratch);
+
+    // The metadata line. Same clock and the same civil-time arithmetic as
+    // dump.zig's filename, printed as UTC because a crash file is read by
+    // whoever the reporter sends it to and their zone is not the reporter's.
+    var ts: std.c.timespec = undefined;
+    _ = std.c.clock_gettime(.REALTIME, &ts);
+    const es: std.time.epoch.EpochSeconds = .{ .secs = @intCast(@max(0, ts.sec)) };
+    const yd = es.getEpochDay().calculateYearDay();
+    const md = yd.calculateMonthDay();
+    const ds = es.getDaySeconds();
+    w.print("\n" ++ build_id ++ " {d:0>4}-{d:0>2}-{d:0>2}T{d:0>2}:{d:0>2}:{d:0>2}Z {s}-{s} pid {d}\n", .{
+        yd.year,
+        md.month.numeric(),
+        @as(u8, md.day_index) + 1,
+        ds.getHoursIntoDay(),
+        ds.getMinutesIntoHour(),
+        ds.getSecondsIntoMinute(),
+        @tagName(builtin.os.tag),
+        @tagName(builtin.cpu.arch),
+        // Unsigned: `{d}` prints a leading '+' for a positive SIGNED int, which
+        // is the same note main.zig makes where it names a `--detach` session
+        // after this pid.
+        @as(u32, @intCast(std.c.getpid())),
+    }) catch {};
+
+    // ...and under it the message stderr is about to print, then the return
+    // addresses behind it.
+    //
+    // ADDRESSES AND NOT THE SYMBOLISED TRACE, which is the one thing here that
+    // is not simply "the same bytes stderr gets". `std.debug.writeCurrentStack
+    // Trace` — the call `defaultPanic` makes a moment later, into a writer of
+    // its own — HANGS THE PROCESS when it is made from a panic handler before
+    // `defaultPanic` has run. Symbolising reads DWARF, and on a machine with
+    // debug info to fetch that read can itself panic; `defaultPanic` survives
+    // that because its private `panic_stage` turns the nested panic into
+    // "aborting due to recursive panic" and an abort. Outside it there is no
+    // such staging, and the recursion ends in a futex nobody will post: a
+    // panicking pardes stopped dead instead of dying. Measured, not guessed —
+    // with the stack-trace call the process wedged at 0% CPU with libc's debug
+    // info half-open, and it did it identically inside a test binary and in a
+    // standalone build with this program's `std_options_debug_io`.
+    //
+    // `captureCurrentStackTrace` only walks frames, so it is safe here, and the
+    // addresses resolve with `addr2line -e <the pardes binary>` against the
+    // build the line above names. stderr still gets the symbolised trace from
+    // `defaultPanic`, unchanged.
+    w.print("panic: {s}\n", .{msg}) catch {};
+    const trace = std.debug.captureCurrentStackTrace(.{
+        .first_address = first_trace_addr orelse @returnAddress(),
+        .allow_unsafe_unwind = true,
+    }, &addr_buf);
+    for (trace.return_addresses) |a| w.print("  0x{x}\n", .{a}) catch {};
+
+    const written = w.buffered();
+    var off: usize = 0;
+    while (off < written.len) {
+        const n = std.c.write(fd, written.ptr + off, written.len - off);
+        if (n <= 0) break; // including EINTR: the record is best effort
+        off += @intCast(n);
+    }
+}
+
+test "a crash record names the build, appends, and carries the panic message" {
+    const io = std.testing.io;
+    const gpa = std.testing.allocator;
+    var tmp = std.testing.tmpDir(.{});
+    defer tmp.cleanup();
+    var base_buf: [std.fs.max_path_bytes]u8 = undefined;
+    const base_len = try tmp.dir.realPath(io, &base_buf);
+
+    // A directory that does NOT exist yet, which is the case that matters: the
+    // user who has never written an init file is the likeliest reporter.
+    const config_dir = try std.fs.path.join(gpa, &.{ base_buf[0..base_len], "pardes" });
+    defer gpa.free(config_dir);
+    const saved_len = dir_len;
+    defer dir_len = saved_len;
+    setDir(config_dir);
+
+    record("first boom", null);
+    record("second boom", null);
+
+    const path = try std.fs.path.join(gpa, &.{ config_dir, name });
+    defer gpa.free(path);
+    const bytes = try std.Io.Dir.cwd().readFileAlloc(io, path, gpa, .limited(1 << 20));
+    defer gpa.free(bytes);
+
+    // The metadata line, then the message, for each of the two panics: the
+    // file is appended to, never rewritten.
+    try std.testing.expect(std.mem.count(u8, bytes, build_id ++ " ") == 2);
+    try std.testing.expect(std.mem.indexOf(u8, bytes, "panic: first boom\n") != null);
+    const second = std.mem.indexOf(u8, bytes, "panic: second boom\n").?;
+    try std.testing.expect(std.mem.indexOf(u8, bytes, "panic: first boom\n").? < second);
+    // ...and the platform and a UTC stamp on that line, which is the half a
+    // version string cannot give: `1970-` would mean the clock read failed.
+    const stamp = std.mem.indexOf(u8, bytes, @tagName(builtin.os.tag) ++ "-" ++ @tagName(builtin.cpu.arch)).?;
+    try std.testing.expect(stamp < second);
+    try std.testing.expect(std.mem.indexOf(u8, bytes, "1970-") == null);
+
+    // ...and at least one return address under the FIRST message, which is the
+    // only part of this that can quietly produce nothing: an unwinder that
+    // refuses the strategy hands back an empty trace and the loop writes no
+    // lines at all. Asserted between the two records so it cannot be satisfied
+    // by the second one's.
+    const first_msg = std.mem.indexOf(u8, bytes, "panic: first boom\n").?;
+    const addr = std.mem.indexOfPos(u8, bytes, first_msg, "\n  0x").?;
+    try std.testing.expect(addr < second);
+
+    // No directory, no file: a core that never resolved a config directory
+    // panics exactly as it did before this existed.
+    dir_len = 0;
+    record("unrecorded", null);
+    const again = try std.Io.Dir.cwd().readFileAlloc(io, path, gpa, .limited(1 << 20));
+    defer gpa.free(again);
+    try std.testing.expectEqual(bytes.len, again.len);
+}
diff --git a/src/macos.zig b/src/macos.zig
index 4c7eb97f..a9c9d0f3 100644
--- a/src/macos.zig
+++ b/src/macos.zig
@@ -41,6 +41,19 @@ const lsp_host = @import("lsp_host.zig"); // the shared snapshot + worker body
 const host_api = @import("host.zig"); // LspRequest and the vtable's own types
 const tracy = @import("tracy.zig"); // no-op unless -Dtracy names a checkout
 const selection_pipe = @import("selection_pipe.zig"); // Job, runJob and Tasks
+const crash = @import("crash.zig");
+
+/// This file is the ROOT of the macOS build (build.zig: the AppKit shell is a
+/// library whose host owns main()), so `std.builtin.panic` resolves here and
+/// not in src/main.zig — a handler written only there would never run in the
+/// app, which is the shell with the least useful stderr of the four. No
+/// terminal to restore either, which is the rest of what main.zig's does.
+pub const panic = std.debug.FullPanic(struct {
+    fn call(msg: []const u8, ret_addr: ?usize) noreturn {
+        crash.record(msg, ret_addr);
+        std.debug.defaultPanic(msg, ret_addr);
+    }
+}.call);
 
 extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: c_int) c_int;
 
@@ -679,6 +692,10 @@ fn initCore(runtime: ?*const Runtime, cols_arg: u16, rows_arg: u16) !void {
         opts.startup_config = found.bytes;
         opts.startup_config_path = found.path;
         opts.config_dir = found.dir;
+        // This host has no terminal at all, so the panic trace stderr gets goes
+        // to a Console.app nobody has open. `panic` above writes it beside the
+        // init file too, and this is where it learns the directory.
+        if (found.dir) |d| crash.setDir(d);
     }
 
     pardes.image.start(io, allocs.image);
diff --git a/src/main.zig b/src/main.zig
index 1fcc2770..41495a48 100644
--- a/src/main.zig
+++ b/src/main.zig
@@ -46,9 +46,15 @@ fn logFn(
 // A panic must restore the terminal (cooked mode, main screen, mouse off)
 // before the trace prints, or it lands garbled in a raw alt screen. recover()
 // no-ops unless the vaxis tty is live, so gui/tty share the handler.
+//
+// Then the same message and trace are appended to `<config dir>/crashes`
+// BEFORE stderr gets them, because stderr is the one place this program cannot
+// keep them: see crash.zig. Silent on every failure, so the fallback is
+// exactly the behaviour that was here before.
 pub const panic = if (is_emscripten) std.debug.FullPanic(std.debug.defaultPanic) else std.debug.FullPanic(struct {
     fn call(msg: []const u8, ret_addr: ?usize) noreturn {
         @import("vaxis").recover();
+        @import("crash.zig").record(msg, ret_addr);
         std.debug.defaultPanic(msg, ret_addr);
     }
 }.call);
@@ -338,6 +344,11 @@ fn nativeMain(init: std.process.Init) !void {
     opts.startup_config = found.bytes;
     opts.startup_config_path = found.path;
     opts.config_dir = found.dir;
+    // The panic handler above writes beside that init file, and this is the
+    // only place it can learn where that is — it runs with no `Options` in
+    // reach. Set for every native entry through this file, including the
+    // `--detach` daemon below, whose stderr nobody is reading.
+    if (found.dir) |d| @import("crash.zig").setDir(d);
     // `--detach` is the core with no terminal and `--attach` is a terminal
     // with no core, so the two together are a contradiction with no useful
     // reading. Refused rather than resolved by declaration order, which would
@@ -425,6 +436,10 @@ fn parseCtrlKey(raw: []const u8) ?u21 {
 // needs its own line here.
 test {
     _ = @import("user_config.zig");
+    // Reached only from the panic handler and from `nativeMain`, neither of
+    // which a test build analyses — so without this line the crash file has no
+    // test at all.
+    _ = @import("crash.zig");
     _ = @import("allocators.zig");
     _ = @import("fs_service.zig");
     // acme's control filesystem, both halves, and NOT their own b.addTest