summaryrefslogtreecommitdiff
path: root/src/walk.zig
Commit message (Collapse)AuthorAge
* tests: at the end of their files: every top-level test block of the 46 files ↵Gabriel Schneider8 days
| | | | | | | that had one mid-file (the locality analysis's 45 and test/hxdiff.zig; gui.zig, macos.zig, look, host_io, 9p_io, terminal, tui, mouse, fs, replica, layout among them, 11,429 test lines), with the functions only tests call, moved after the file's code in their order, a mechanical move (.scratch/suites/tests_to_end.py), zig fmt'd; the unit suites pass as before (tui 1486/1487, gui 1536/1537, core 956/956 on both), macos.zig only parse-checked here Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* kinds #14, the core's PDF code gone: pdf.zig and pdf_view.zig leave and ↵Gabriel Schneider10 days
| | | | | | | every pdf_enabled branch with them; look, Jump, Recent, dumps (one from before the addon opens as its pane), the watch, mouse, keys, the rail and the tag go through Kind for any kind's pane; test/pdf.zig's behaviours are T1 cases over 9P (addons_pdf.zig) and pane cases (a Shadow over a replica, a file changed under it), the harnesses load pdf.so by path Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* digest: a blob of 4 KiB and up (a file's text, a register's text, a file's ↵Gabriel Schneider11 days
| | | | | | | rows, an undo entry's bytes) is taken as its length, never its bytes, a text's revision saying what it holds, where it was hashed once and kept as a word by its address (walk.zig's blob words, from when no text was written in place): since the journal a session's edit writes its text in place, and since the change before a replica's does too, so the word went at every edit and the next unit's digest hashed the whole text again, 100 MB of it in a big file, on the session's step and the replica's; the words go with what kept them (walk.forget, wordOf, forgetAll, unhashed, makeWords, the bench's blob_words_on, memory.freeRaw's and unpin's forgets, File.edit's and grow's), and so does the join's hashing thread (replica.Peers' held, texts, hasher and rounds, joins, hashTexts, unhashedDone, the `hashed` note), which made the words of the texts a session never digested before a join's digest took them, its pins with it: a client is joined at the boundary its connection comes at; the semantics change, so the wire's version is 20 (a client of the build before is refused at its hello, as any other build's, and test/daemon.py's own hello says 20); a blob under 4 KiB is hashed whole as before, so a small text's bytes stay in the digest, and the replicas harness (test/replicas.zig) compares every file's text byte for byte at quiescence beside the digest; tests: walk.zig's 'a blob is hashed once and folded in as a word; one written in place is not its word, which a Debug build checks; a freed one's word goes' becomes 'a blob of 4 KiB and up is digested by its length, what it holds told by its keeper's revision; a smaller one by its bytes' Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* dumps: a Dump is a labelled entry of the state file (src/Db.zig, ↵Gabriel Schneider11 days
| | | | | | | architecture §14.3), one fsynced transaction: `dumps/<label>` holds the session as a database of its own, which the walk writes by field name into xitdb's maps (walk.Put: a struct a map keyed by its fields' names, a union a map of its one field, records a list, plain numbers one bytes value tagged with their kind and width, an enum its name, a number a uint or an int, an optional an absent key, a type with dumpWrite its bytes, a .blob text of 4 KiB and up its BLAKE3 hash) and another build reads by name (walk.Get: defaults, notes, a pane it cannot read left out, as the byte loader did), with the hashes of the texts it keeps once under `blobs/`, so an unchanged text is written once across dumps; labels are `pardes-<date>-<time>` (`-2` and on in one second), `last` names the newest, the newest 32 are kept and the texts no kept one names are dropped in the same transaction, and at a session's end the file is compacted into `pardes.xitdb.new` and renamed over once it is past twice what it holds plus 64 MiB (another session notices the new file by its inode); a dump travels as an image, a database in memory holding it alone, deterministic (std.Io.failing's zero ids, so the replay's fixpoint holds): what Dump hands its host, what `Dump export <path>` and $PARDES_DUMP write whole (the snapshot goldens' `Restore <path>` unchanged), what a Restore of a label is read into, what a handover carries (`handover-<seq>`), and what `-l` and `Restore <path>` read; Restore takes a label or a path (bare: this session's last dump, else the newest), a label the file keeps none of refused in words (`Restore: no dump labelled X`, asked of the authority as the closed panes are); `/dumps` lists them, `<label> <date> <size> <panes>`, the newest first; Upgrade commits `upgrade-<pid>` before the exec, its record naming the state file and the label (a record without one, an older build's, names its `.dump` file, read as before), and the label stays; the byte dump stays for the join, the closed panes and the old `.dump` files (dump.byteDump), and dump.outPath, uniqueIn and upgradePath go for dump.label; measured (Debug, a session with a 3.2 MB file and three scratch panes, 20 dumps): the image on the loop 64-66 ms (59 ms of it BLAKE3 of the file, 0.65 ms in ReleaseFast) where the byte dump was 0.7 ms, the commit 9-16 ms with its fsyncs where the file was written in 2, a Restore 1.8 ms to read and 11.4 to load, and the file 3.55 MB after the first dump and 379 KB a dump after it with the big text unchanged (3.58 MB a dump with it edited each time, as twenty `.dump` files would be); tests: dump.zig's 'two dumps in one second get names of their own' and 'a dump makes every directory missing on the way to its DumpDir' become Db.zig's 'two dumps with one label get labels of their own' and 'a Dump makes every directory missing on the way to its DumpDir, and the file there', 'a session read by name, field by field as another build reads it, is the same session' becomes 'a session's byte dump read by name...' beside the new 'a session's image, read by name as every load reads it, is the same session, and dumps to the same bytes again', two replica tests compare the join's stream with dump.byteDump, the Handover tests read the carried image, upgrade's record test carries a label, the root listing tests and the monkey list `dumps`, the behaviour suite gains five root-ctl tests (a Dump and Restore round trip through 9P with `/dumps` and the tag's word, a label kept none of refused in words, the 33rd dump dropping the first, an unchanged text written once across two dumps by the file's size, a killed Dump leaving the one before it whole), the test host performs a Dump's write and a Restore as a host does, the monkey lets go of its log follower at a Restore, and daemon.py's upgrade case finds the kept dump in `/dumps` Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* targets: the DOM web shell goes (src/web.zig and src/web/, its harnesses ↵Gabriel Schneider11 days
| | | | | | | web_harness, web_snapshot, web_driver_test and web-snapshots/, tools/web_fixture.py, -Dplatform=web, -Ddump, and the web, web-harness, web-snap, web-e2e and web-driver-test steps), and with it what only a freestanding build needed: `hosted` and every guard on it (every build has a host now), memory's stack-fallback arenas and reduced_target sizes, the emscripten main, the tree-sitter libc shim and the stb stub, lsp's has_client, the web arms of the platform switches (-Dplatform is tui, gui or macos); the macOS files pass zig ast-check; docs/web.md is a paragraph on what it was Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* memory: freeUnfilled, freeRaw for a slice of u8 written out again, goes; its ↵Gabriel Schneider12 days
| | | | | | | callers (File.zig's three, walk.zig's, memory.zig's tests) call freeRaw, and walk.zig's comments name it alone Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* walk: hash names isAtomic and isBitset where it spelled them out again, ↵Gabriel Schneider12 days
| | | | | | | plainBytes is flatHash and isSigned is written at its one use; a dump's float of another width is incompatible, as any other changed kind, not converted; unlockBlobs is the store it wrapped, Dump.shared was memo.count(), elements takes the slice it is always given, and five names nothing outside reads lose pub; two doc lines go back to lengthOf Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* walk: dead branches out: hashMap and its dispatch (no HashMap is reachable ↵Gabriel Schneider12 days
| | | | | | | from a digest's roots: a map in Pardes would not compile past opaqueTo), and the dump's arms for errors with Load.errorNamed (no dumped type holds one, and Load refuses anyerror); the Kind numbers stay, an old dump's table still names them, and the tape keeps its errors Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* walk: an enum is digested by its name (its word, a fixed-width hash), not ↵Gabriel Schneider2026-10-30
| | | | | | its number: registry.Builtin, made at compile time, has Tty9p in a Linux build and not a FreeBSD one, so a +LocationsConfig (its output's origin a Builtin) digested otherwise in each, and a Linux client of a FreeBSD session holding one could not join (the hunt's H19: `the session's dump loaded as another session`, main's window client of main's session on the box; after, a terminal and a window client here join one with a +Search, a directory, a renamed file and a +LocationsConfig, and agree); walk's test, and test/cross_replay.py's session holds a +LocationsConfig and a renamed file (a FreeBSD tape replayed here diverged at pane 2.file.output before) Co-Authored-By: Claude Opus 5.5 <[email protected]>
* web: the build links no libc, so what is the C heap's is a hosted build's ↵Gabriel Schneider2026-10-30
| | | | | | alone: terminal.create takes emulator_gpa (std.heap.c_allocator) only in a hosted build, as only one has a screen-state replica, and walk's blob lists (a replica client's, a join's) only with libc; `zig build web` failed on main at terminal.zig:601, and at walk.zig:115 behind it; it builds, and test/web-snapshots/touch.mjs passes (the driver's own CDP-deadline test fails as on main, untouched) Co-Authored-By: Claude Opus 5.5 <[email protected]>
* tape: an error crosses by its name, a stat as fs.OsStat, and the tape's ↵Gabriel Schneider2026-10-30
| | | | | | shape is in wire.semantics: a replica of another build read its session's units as its own build's, so a window client of a terminal's session, or of the box's, read every error by a number its build gives another (451 of 505 names numbered otherwise in the Linux terminal's and window's builds, 244 between Linux and FreeBSD: pardes-gui replaying a terminal's tape panicked at `unexpected error code, found error.NonLocalPath`, and so did main's window client of a session on the box at its Look of a fifo) and a FreeBSD stat out of step (nlink 8 bytes there, 4 here); a name this build's set lacks is a divergence (error.NoSuchError), never another error; an anyerror (a host's report) reads back as one of tape.Failure's, a Debug build panicking at one outside it as it is taped; tape.shape (each taped type's fields, tags, enum names and numbers) in semantics refuses, in words, builds whose tapes differ (the stat taped as before: the Linux and FreeBSD builds hash to two), and a site whose result tape.results does not list is a compile error; walk's test of an error by its name Co-Authored-By: Claude Opus 5.5 <[email protected]>
* replica: a far client's content cache and zstd: the session lists its join's ↵Gabriel Schneider2026-10-30
| | | | | | blobs by BLAKE3 hash first (wire 16 `blobs`), the client finds each in the core it leaves or on disk ($XDG_CACHE_HOME/pardes/blobs, hashed again on load, a bad file a miss, the least recently used evicted past 1 GiB) and asks for the rest (`want`), and everything after the welcome goes in zstd envelopes (level 6 over 8 MiB for the join, an undo snapshot being a near copy of its text, then level 3), so `ssh -C` leaves the default command; a far client (through ssh, or PARDES_REPLICA_FAR) keeps what it was sent and, at its end, what it made; a hangup ends a remote client the normal way; through the box a client holding the blobs is drawn in 2.4 s where main took 7.0 to 11.9, and a first join of a session with undo in 7.1 s where main took 11.9; libzstd 1.5.7 vendored, its allocations on c_heap, a second copy refused by shared-libs-check; tests for the cache's verify-on-load, its eviction and a far join's blobs sent once; test/daemon.py's far clients Co-Authored-By: Claude Opus 5.5 <[email protected]>
* walk: an optional slice or pointer loads through a local, not into an ↵Gabriel Schneider2026-10-30
| | | | | | optional set to an undefined payload: a slice's null is its address, so that optional was undefined and ReleaseSafe's optimizer took it as null; a command pane's `?[]u8` command read one byte long, every field after it out of step, and Restore of a session with a command pane in it said "end of dump" (Debug's 0xAA filler and ReleaseFast read it right); a test of an optional slice and the field after it aborts in ReleaseSafe without it Co-Authored-By: Claude Opus 5.5 <[email protected]>
* a replica client's join costs the session's step its mutable state alone: ↵Gabriel Schneider2026-10-30
| | | | | | | | | | the dump leaves the blobs out (walk.zig `Dump.deferred`: a `.blob`'s texts of 4 KiB and up, noted where they go) and its terminals' emulators (a size: the first delta brings each), the client's writer thread writes the welcome and the dump with the blobs put back in place, and each blob is pinned until written (memory.zig `pin`: its free put off); the texts the session never digested are hashed first on a thread of their own (walk.zig `unhashed`, `makeWords`), the join waiting a boundary for them The client is unchanged: what it reads is the dump `dumpBytes` makes, byte for byte (a test splices one and compares). ReleaseFast, the session's worst boundary while three clients join one after another (`t2/join2.py`), before and after: typical (4.1 MB of files, 1 MB of scrollback, an 8.1 MB dump) 12.4 to 1.6 ms; big (38.6 MB, 8 MB, 76 MB) 51.3 to 1.5 ms; a 100 MB file open beside typical's (254 MB) 172.5 to 1.6 ms. Of the dump the step makes 113 to 122 KB. PARDES_REPLICA_STATS says each join's time and the state's size. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* replica clients' terminals as screen state: the session's Peers keep a ↵Gabriel Schneider2026-10-30
| | | | | | | | | | Screens (rterm's), a peer each: the rows a pump's steps read before its unit, the deltas after it, and before each client input's step, the rows it read before its unit; a joiner's terminals afresh after its dump; acks, rows asked and resets from the client (0x04) taken at the boundary; units carry no pty bytes and a word of digest The client applies a delta or rows (0x06) in stream order and asks for the rows it shows and lacks after each frame; its dump's emulators are loaded on the C heap (walk.zig `.alloc` of a pane's `terminal`, dump.zig `terminal_gpa`), as `terminal.create` makes a screen replica's. Its load checked under its own tape (no emulator in the digest). tty.zig: with replica clients, a wait takes one step of a person's input and none after a pty's output (`split`): the next is held for the next wait, so a step reads the terminals as the deltas after each unit bring them. Screens: `dropPeer` (clients come and go), its numbers reused. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* the walker's digest: a blob's hash made once and folded in as a word (a 100 ↵Gabriel Schneider2026-10-30
| | | | | | | | | | MB file was 5 ms a digest, two a unit, on the authority); blob fields annotated `.blob`, which the rollback copy's pins read too walk.zig: a field named in a type's `replica.blob` (a file's text, line index and location rows, an undo snapshot's text, a register's values and joined text, the rows an edit dropped, a query's saved rows) holds immutable blobs. A digest hashes one of 4 KiB or more once, keeps the word by its address and length, and folds the length and word in; `forget` drops the word, from memory.freeRaw and freeUnfilled, through which every such blob is freed raw. Only a Debug build (the tests, the harness, replay) hashes the bytes at every digest all the same and panics where the kept word is not their hash: a blob written in place, or freed past its word. A release build, ReleaseSafe the user's own as well as ReleaseFast, takes the word and never panics on a stale one: at worst a divergence, which a replica records and then stops at (nothing resets or rejoins it yet; the notes say so). File.zig: an edit's free of the old text went to `rawFree` itself and so kept its word; the next text of the same length at the same address folded in the old one (the ReleaseFast harness diverged in 162 of 180 replica runs, in `file.content`), and it goes through freeUnfilled now. Registers.zig: `put` makes a new values array rather than writing the old one. rollback.zig: the copy pins what `.blob` names (`pinned` and `pinnedDeep` gone). Measured, ReleaseFast on the perf cores: with a 100 MB file open (PARDES_DIGEST_BENCH=100, replica.zig's bench), a unit of a key into it went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the word), a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients): rollback p99 9.5 -> 2.6 ms at 30 ms (1.5 and 1.4 ms at 80 and 207), copy p99 0.27 ms; none diverged, none unequal, no delta wrong. Tests: a blob hashed once, its kept word no longer its hash once written in place, its word gone when freed. docs/next-daemon.md: the follow-up. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* walk.zig: a slice a `len` cuts (a text's room for its ranges) is dumped ↵Gabriel Schneider2026-10-29
| | | | | | whole and loaded with the room it had, its elements plain numbers or no build Co-Authored-By: Claude Opus 5.5 <[email protected]>
* web: the browser shell builds again and loads a dump of a native build: a ↵Gabriel Schneider2026-10-29
| | | | | | | | | | theme by its name, a field compiled out passed over, no disk asked; touch.dump replaces the hand-written ZON fixture Fixed for wasm32-freestanding: counts as usize, no threads, stb_image, tape or host stat there. touch.mjs follows the renderer's 13 tag-layer fields, helix's retuned colours and the tag column. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* walk.zig: a dump's type table is made once a process, not for every store ↵Gabriel Schneider2026-10-29
| | | | | | record and dump Co-Authored-By: Claude Opus 5.5 <[email protected]>
* docs/next-daemon.md: phase 2 notes (the walker dump, crossing builds, load ↵Gabriel Schneider2026-10-29
| | | | | | and revive, what the design missed); the GUI's Restore test keeps the dump's serial Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dump: a pane another build cannot give a terminal for is a terminal still; a ↵Gabriel Schneider2026-10-29
| | | | | | session read by name is tested; nothing of a load goes to stderr Co-Authored-By: Claude Opus 5.5 <[email protected]>
* dump: one walker dump of the whole session, by name across builds, for ↵Gabriel Schneider2026-10-29
| | | | | | | | | | | Restore, -l, Handover and the store; dump.zig's hand-written schema gone A dump is walk.zig's walk of the core with a table of every type it reaches (fields, enum and union names, number widths), read back by name; terminals are ghostty's snapshot (screens, history, kitty images); a failed load frees what it made. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* phase 5 part A: the speculation and rollback engine, in one process: a ↵Gabriel Schneider2026-10-29
| | | | | | | | | | | | | | | | walker copy put back where it was, replicas that apply their own input at once and take the authority's tape as the truth, and a harness of typists over delayed links src/rollback.zig: the replica core's allocator (`Epoch`) knows every allocation, and while a copy is held puts off frees of what was there before and refuses to grow it in place, so what the core held stays where it was. The copy (`Copy`) is the digest's walk saving the bytes of every allocation it reaches, field by field (the derived caches and this process's own too: a safe build's free writes over what it frees, and the frame arena points at what the epoch then frees), but the queues a step drains and the frame statistics; a file's text, its line index and its undo snapshots are pinned instead (made anew at each edit, never written; now freed raw everywhere), checked unchanged in a safe build. Put back, each allocation's bytes go back over it, the epoch's allocations are freed and its put-off frees forgotten; let go, they are made. Tree-sitter's trees, the C heap's, are held the same way (syntax.held_trees). src/replica.zig: an `Authority` records its core's every input to a tape in memory and hands each client's input back as a unit (origin, sequence number, its tape); a `Replica` applies its own input at once, as the authority will (update, then pump), recorded by its own tape (tape.zig's speculation mode), and for each unit in order drops its pending head when the authority recorded the same (inputs, host calls and answers: the fast path), else puts the copy back, replays the units since and this one (every digest checked) and applies its pending inputs again. A host call or a syscall stops a speculation: it is answered harmlessly, undone when it returns, and the input waits for the authority's answer; a clock or environment read is a guess the comparison checks; an emulator is the authority's (terminal.zig's writers stop a speculation; a safe build asserts none changed). The view's events (a resize, the pointer) wait. Messages are bytes: no transport yet. Each client has a view of its own (src/views.zig): the authority goes into it before the client's input (`toView`, taped), the replica's speculation the same, and the copy walks the views not live too (`parked`, each pane's `pv`). test/replicas.zig (`zig build replicas`): one authority and N replicas, a view each (`--shared` puts them in one), each client typing monkey.zig's mix at 50-200 ms over its own link (half the RTT plus 10% jitter, in order), the authority's pty echoing 2 ms later; gates: no replica diverges, every replica's digest is the authority's at quiescence; reported per RTT: inputs applied at once, stopped, waiting, fast paths, rollbacks (mispredicted, foreign), and what a speculative apply, a copy and a rollback cost. 3 clients x 100 inputs at 30, 80 and 207 ms: no divergence over 12 Debug seeds and 20 ReleaseFast ones with a filled heap; ReleaseFast p99 a speculative apply under 0.2 ms, a rollback under 1.6 ms. `zig build rollback-bench` times the real copy now (take, release, an edit, restore). What the replicas found, fixed: a release build reads what the heap had where Debug's fill made two cores agree: `col_panes` and `restore_buf` digested undefined (zeroed now), `terminal.State.reply_dropped` never set at create, `Text.sels` and `PendingPipe.ranges` digested past their length (a walker `len` cuts a slice too). PARDES_REPLICA_FILL fills a replica's allocations to show what is left. Co-Authored-By: Claude Opus 5.5 <[email protected]>
* walk.zig: the reflective walker, one comptime @typeInfo walk that hashes a ↵Gabriel Schneider2026-10-29
value (the replay digest) and encodes and decodes it (the tape's values) Values, never addresses: a pointer is followed, a slice is its elements, an ArrayList its items, a hash map its entries in no order, a DynamicBitSet its bits; an allocator, an Io, a lock or a worker's future is machinery and walks to nothing. A type it cannot see into (a pointer to an opaque thing or a function, a many-pointer, an untagged union) is a compile error naming the struct and the field, until the struct says what the field is: pub const replica = .{ .local = .{...}, .derived = .{...}, .authority = .{...}, .len = .{ .{ "jumps", "njumps" } } }; local (this process's own: frames, animations, caches of the host's), derived (made again from the rest), authority (held by the session that performs effects alone) are in no digest; `len` hashes a fixed array only as far as its length field says, since the rest is undefined (0xAA in a safe build, and a slice in it faults). A name that is no field is a compile error, so a rename cannot put a field back in quietly. `fieldDigests`/`fieldNames` give a digest a field, which is how a divergence is narrowed down. encode/decode are native byte order and an error is its number in this build: for the process that wrote it, or one of the same build. Phase 2 is to make the complete dump on the same walk (docs/next-daemon.md); nothing uses it yet. Co-Authored-By: Claude Opus 5.5 <[email protected]>