summaryrefslogtreecommitdiff
path: root/src/memory.zig
Commit message (Collapse)AuthorAge
* a replica client's join costs the session's step its mutable state alone: ↵Gabriel Schneider2026-10-30
| | | | | | | | | | the dump leaves the blobs out (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted where they go) and its terminals' emulators (a size: the first delta brings each), the client's writer thread writes the welcome and the dump with the blobs put back in place, and each blob is pinned until written (memory.zig `pin`: its free put off); the texts the session never digested are hashed first on a thread of their own (walk.zig `unhashed`, `makeWords`), the join waiting a boundary for them The client is unchanged: what it reads is the dump `dumpBytes` makes, byte for byte (a test splices one and compares). ReleaseFast, the session's worst boundary while three clients join one after another (`t2/join2.py`), before and after: typical (4.1 MB of files, 1 MB of scrollback, an 8.1 MB dump) 12.4 to 1.6 ms; big (38.6 MB, 8 MB, 76 MB) 51.3 to 1.5 ms; a 100 MB file open beside typical's (254 MB) 172.5 to 1.6 ms. Of the dump the step makes 113 to 122 KB. PARDES_REPLICA_STATS says each join's time and the state's size. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* the walker's digest: a blob's hash made once and folded in as a word (a 100 ↵Gabriel Schneider2026-10-30
| | | | | | | | | | MB file was 5 ms a digest, two a unit, on the authority); blob fields annotated `.blob`, which the rollback copy's pins read too walk.zig: a field named in a type's `replica.blob` (a file's text, line index and location rows, an undo snapshot's text, a register's values and joined text, the rows an edit dropped, a query's saved rows) holds immutable blobs. A digest hashes one of 4 KiB or more once, keeps the word by its address and length, and folds the length and word in; `forget` drops the word, from memory.freeRaw and freeUnfilled, through which every such blob is freed raw. Only a Debug build (the tests, the harness, replay) hashes the bytes at every digest all the same and panics where the kept word is not their hash: a blob written in place, or freed past its word. A release build, ReleaseSafe the user's own as well as ReleaseFast, takes the word and never panics on a stale one: at worst a divergence, which a replica records and then stops at (nothing resets or rejoins it yet; the notes say so). File.zig: an edit's free of the old text went to `rawFree` itself and so kept its word; the next text of the same length at the same address folded in the old one (the ReleaseFast harness diverged in 162 of 180 replica runs, in `file.content`), and it goes through freeUnfilled now. Registers.zig: `put` makes a new values array rather than writing the old one. rollback.zig: the copy pins what `.blob` names (`pinned` and `pinnedDeep` gone). Measured, ReleaseFast on the perf cores: with a 100 MB file open (PARDES_DIGEST_BENCH=100, replica.zig's bench), a unit of a key into it went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the word), a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients): rollback p99 9.5 -> 2.6 ms at 30 ms (1.5 and 1.4 ms at 80 and 207), copy p99 0.27 ms; none diverged, none unequal, no delta wrong. Tests: a blob hashed once, its kept word no longer its hash once written in place, its word gone when freed. docs/next-daemon.md: the follow-up. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* phase 5 part A: the speculation and rollback engine, in one process: a ↵Gabriel Schneider2026-10-29
| | | | | | | | | | | | | | | | walker copy put back where it was, replicas that apply their own input at once and take the authority's tape as the truth, and a harness of typists over delayed links src/rollback.zig: the replica core's allocator (`Epoch`) knows every allocation, and while a copy is held puts off frees of what was there before and refuses to grow it in place, so what the core held stays where it was. The copy (`Copy`) is the digest's walk saving the bytes of every allocation it reaches, field by field (the derived caches and this process's own too: a safe build's free writes over what it frees, and the frame arena points at what the epoch then frees), but the queues a step drains and the frame statistics; a file's text, its line index and its undo snapshots are pinned instead (made anew at each edit, never written; now freed raw everywhere), checked unchanged in a safe build. Put back, each allocation's bytes go back over it, the epoch's allocations are freed and its put-off frees forgotten; let go, they are made. Tree-sitter's trees, the C heap's, are held the same way (syntax.held_trees). src/replica.zig: an `Authority` records its core's every input to a tape in memory and hands each client's input back as a unit (origin, sequence number, its tape); a `Replica` applies its own input at once, as the authority will (update, then pump), recorded by its own tape (tape.zig's speculation mode), and for each unit in order drops its pending head when the authority recorded the same (inputs, host calls and answers: the fast path), else puts the copy back, replays the units since and this one (every digest checked) and applies its pending inputs again. A host call or a syscall stops a speculation: it is answered harmlessly, undone when it returns, and the input waits for the authority's answer; a clock or environment read is a guess the comparison checks; an emulator is the authority's (terminal.zig's writers stop a speculation; a safe build asserts none changed). The view's events (a resize, the pointer) wait. Messages are bytes: no transport yet. Each client has a view of its own (src/views.zig): the authority goes into it before the client's input (`toView`, taped), the replica's speculation the same, and the copy walks the views not live too (`parked`, each pane's `pv`). test/replicas.zig (`zig build replicas`): one authority and N replicas, a view each (`--shared` puts them in one), each client typing monkey.zig's mix at 50-200 ms over its own link (half the RTT plus 10% jitter, in order), the authority's pty echoing 2 ms later; gates: no replica diverges, every replica's digest is the authority's at quiescence; reported per RTT: inputs applied at once, stopped, waiting, fast paths, rollbacks (mispredicted, foreign), and what a speculative apply, a copy and a rollback cost. 3 clients x 100 inputs at 30, 80 and 207 ms: no divergence over 12 Debug seeds and 20 ReleaseFast ones with a filled heap; ReleaseFast p99 a speculative apply under 0.2 ms, a rollback under 1.6 ms. `zig build rollback-bench` times the real copy now (take, release, an edit, restore). What the replicas found, fixed: a release build reads what the heap had where Debug's fill made two cores agree: `col_panes` and `restore_buf` digested undefined (zeroed now), `terminal.State.reply_dropped` never set at create, `Text.sels` and `PendingPipe.ranges` digested past their length (a walker `len` cuts a slice too). PARDES_REPLICA_FILL fills a replica's allocations to show what is left. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* FreeBSD: the tty pardes builds for it and runs there, terminals, 9P and the ↵Gabriel Schneider2026-10-29
| | | | | | | | | | | | | | | | | | | | | | | | daemon included Cross-compiled with -Dtarget=x86_64-freebsd.15.0 (15 ships libutil.so.10; Zig's default 14.0 links .so.9) and run on the FreeBSD 15.1 box: - 9p_io serves there: fstatat, not Linux's statx. madvise is Linux's. - WINCH reaches the sigwait thread through a handler that does nothing: FreeBSD drops a signal ignored by default as it is sent. - A dead terminal answers a write with ENXIO, which std does not expect: a Debug build traced it to stderr, that same terminal, and recursed until the stack ran out (a session whose ssh dropped hung). No tracing there. - waitid's FreeBSD numbers and 64-bit id: a command pane's exit is seen. - No /proc: a shell's directory and program by sysctl, ttyTaken by the foreground group, the foreground job's name by its program. - File watching by FreeBSD 15's inotify. - ghostty's kitty loader: off Linux a stat comes before a t=f open (a FIFO hung it); FreeBSD's shm_open refuses O_NONBLOCK. - vaxis's test terminal has no read off Linux: no input thread for it. - test-install: the test-build programs installed into <prefix>/test, to run where the build cannot. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Arenas are regions mapped once with a page bitmap under a DebugAllocator; ↵Gabriel Schneider2026-10-23
| | | | | | | | | | | | | | the shells allocate from one too Each arena (pardes, frame, lsp, tree_sitter, image, pdf and the new host) maps its region once and tracks 4 KiB pages in a bitmap; a full region spills to malloc and counts it. Freed runs of 64 KiB to 1 MiB go back at once, the rest at memory.trim. In a release build init.gpa is memory.host over libc's malloc. Measured against the old arenas: typing in a 7.4 MB file 1.95 ms a key (was 2.0), a V8 Callstack page 142 MiB (was 134) and 42 MiB after close (was 69). Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Callstack lists every caller and pages a tree too big for one query; memory ↵Gabriel Schneider2026-10-23
| | | | | | | | | | | | | | | | bounds and resident-size fixes Callstack walks while the server answers, counts on the message row past a second, puts test callers last at each level, and ends a tree past one query's memory (memory.limits.arena.lsp_query, a 256 MiB region taken and given back per query) in a row whose Callstack shows the next page. Grep reads whole files. New hard limits: undo text per file, image decode pixels, the LSP snapshot of other open files, one paste. Safe builds no longer fill worst-case buffers with 0xAA (memory.unfilled), big regions are kept off transparent huge pages, and malloc_trim runs after big results. Also: Look's git show pages into a +Diff, and dropped marks in File. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Build with Zig 0.17Gabriel Schneider2026-10-22
| | | | | | | | | | | ghostty, libvaxis and uucode move to ghostty's 0.17 pins, cloud9 to its 0.17 main, SDL to castholm's release-3.4.x (3.4.18), zstbi and tree-sitter to forks with 0.17 build files (see build.zig.zon). A bare zig build copies the binaries into ~/.local/bin (-Dhome-install=false leaves it out): 0.17 keeps the prefix from build scripts. A terminal's edited copy goes when the program clears the screen. tinywin.snap is gone; perf baselines are re-recorded. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* A selection holds up to 1024 ranges on the desktop, 64 on the boardGabriel Schneider2026-10-01
| | | | | | | | | | | The limit was 64 everywhere, which helix-golf's enumerate_and_align outgrows (%s |\d+ makes 121 ranges). It is now memory.limits.selections, per platform, and the counts that hold it are u16. The ranges stay inline in each Text, so a pane grows by about 80 KiB on the desktop (three texts and the regex prompt's snapshot); the Pane size budget in Terminal.zig moves from 128 to 160 KiB and says why. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Record /log whether or not anyone reads it, with the editor's messages in itGabriel Schneider2026-10-01
| | | | | | | | | | | | | | | /log is now one ring (64 KiB, 4 KiB on the board) that records new, del, rename and save, and a msg line for everything the editor says. An open freezes it, so cat log shows what happened lately and ends; writing follow to that same open makes reads past it wait for newer records, and a follower the ring outran reads lost N first. The message log keeps the pane's serial, not its reusable slot. A test now fails when the tree serves a file /README never mentions; it caught pty/status, now documented along with typing through pty/data. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Keep the arena buffers out of Debug binariesGabriel Schneider2026-10-01
| | | | | | | | The five arena buffers are zeroed globals instead of undefined fields, so a Debug build keeps them in .bss rather than writing 160 MB of 0xAA filler into every binary (Debug pardes 839 MB -> 246 MB). Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Check each subsystem's memory under one DebugAllocator, not twoGabriel Schneider2026-10-01
| | | | | | | | | | | | | | | | | | In a Debug build memory.zig put each subsystem allocator (core, frame, lsp, tree-sitter, image, PDF) under a DebugAllocator of its own, which is what reports that subsystem's leaks and makes memory.deinit panic on one. Beneath it, past the fixed buffer, was the allocator the shell passed in: init.gpa, which std.process.Init already makes a DebugAllocator in Debug. Every block that spilled out of a fixed buffer was tracked, checked and given stack traces twice, and in a test both layers sat on std.testing.allocator. A Debug build now puts the page allocator beneath the subsystem DebugAllocators, so each allocation passes through exactly one. The shells' own gpa is still std.process.Init's DebugAllocator for everything else, and a release build is unchanged: the subsystems sit on the gpa, no DebugAllocator anywhere. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* Answer 9P on the connection's task, so a session can open its own treeGabriel Schneider2026-10-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The editor's loop was the only thing that could answer a 9P request, which made the editor's own syscalls through a mount of its own tree -- a Look at /mnt/9p/pardes/<me>/anything under a `9ns --mntgen` view, a Save into it -- requests only the blocked loop could serve. The name-based refusal that followed (ownMountSuffix) and the in-process routing of a mount of oneself (Client.sameSession) were patches over that, and both are gone, with the mailbox that shipped every request to the editor's thread. One rule replaces them, `pardes.turn`: the core is single-threaded, the editor's thread has the turn by default and gives it up in two kinds of gap -- while it waits for input and while a step of it is out in a host syscall -- and a cloud9 connection task takes it in those gaps to answer. `out` counts the steps that are out, from any thread: while one is, the core reads consistently but that step still holds pointers into it, so a request that would change a pane (a write, a truncation, an rmdir) is parked in the engine and retried when the turn is next given up with nothing out, and the editor's own wake waits for the count to reach zero. It is never a write of its own that a step waits on out there -- writes come from a shell performing a save between steps -- so a parked request is never the syscall's own, and making a pane or rendering a screen need not park: every yield sits before its step's mutation, so the layout and the surface are whole under it. A changing request that queued effects is answered once the editor has performed them (`echo Save > exec` returns with the file written, as acme's `put` does), and it settles the way a step does, because without that a /log reader waited for the user's next keystroke. Every host syscall on a user path has to give the turn up, not fs.zig's alone: the first end-to-end run hung in `inotify_add_watch` performing the new pane's watch effect. PDFs and images are read whole at open, so no draw goes out into the host. The core's allocator takes its fixed buffer through the lock-free interface, since a connection task allocates while the editor's thread is out in a syscall that allocates too. A Restore puts the replacement in first and releases every task waiting on the old core. cloud9 (pinned at eb1a104) parks an open, a truncating wstat, a clunk and a remove on `again`, not only reads and writes, and answers a parked job whose fid was clunked without asking the backend. Verified: test/selfmount.py runs the editor under `9ns --mntgen` and Looks at, reads and Saves its own tree through the mount; a unit test pins that a change parks while the editor is out mid-step and lands when it rests, while a read is answered in the window. 9P over the Unix socket against a tty session, same machine, Debug builds: a read of /index 278us -> 61us, a truncating body write 1184us -> 609us, exec Save 718us -> 583us; the gesture benchmark is unchanged (geometric mean 0.997 over 53 cells). Also from the reviews: a notice chip over an image or PDF pane was painted out by the picture drawn after the cells, so pictures give up the rows; in the GUI a tree-sitter context band painted over the chip, so body layers are emitted first; a message is one row of printable text, its 256-byte cut never leaves half a glyph, and one wider than its pane keeps its tail (the file name, the reason) rather than its head. Co-Authored-By: Claude Fable 5.1 <[email protected]>
* Flatten the 9P control tree and move it out of fs.zigGabriel Schneider2026-10-01
| | | | | | | | | | | | | | | The served tree loses the self/ level: /index /ctl /new /log /screen /listeners /pane/<n>/... /os, with /src only in -Dembed-sources=true builds (default off, on for esp32p4). ctl speaks the editor's own language with two lowercase verbs, look TEXT and exec TEXT, plus acme's addr verbs; the new/ factory directory becomes one clone file; cons is gone (exec Msg); name and sel are files; stats report real lengths, modes and mtimes; /log streams pane new/del/rename/save events. The tree code lives in src/ninep/ (tree, pane, ctl, addr, pty, events, screen, sources); fs.zig keeps host access, mounts, resolution and find/grep. Same engine and transports. README (fs-help.txt) and docs rewritten; tests updated and extended. Co-Authored-By: Claude Fable 5.1 <[email protected]>
* Refactor panes and filesystem; replace FUSE with 9PGabriel Schneider2026-09-07
Consolidate pane, layout, memory and host code. Serve 9P by default over Unix sockets, with runtime mounts and optional TCP/QUIC transports. Remove FUSE and obsolete proof-of-concept examples. Fix highlighting and terminal-history performance, expand differential and stress-test infrastructure, sort navigation results while preserving the next occurrence, add syntax-colored Braille minimaps, remove SPC-k, and document 9P interaction as a repository skill.