summaryrefslogtreecommitdiff
path: root/src/Db.zig
Commit message (Collapse)AuthorAge
* fix: Recent's one-time import from the files before the state file read ↵Gabriel Schneider9 days
| | | | | | | nothing, and said nothing, for a directory whose path passed its room (a bufPrint caught to ""): a directory past max_path_bytes is said in the message, Recent: the files before the state file were not read: their directory's path is N bytes, too long; the import test gains such a window Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* idioms #9, every catch unreachable this slice added is a fact of the code: ↵Gabriel Schneider11 days
| | | | | | | each formats integers, literals, an error's name or a slice of a fixed array (the daemon's [128]u8 names, the cache's path_buf, main's cwd_buf, those two buffers now sized from their arrays), so all 39 stay, each with its reason on its own line Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* idioms #5, more formats that cannot overflow: /proc paths of a pid, the ↵Gabriel Schneider11 days
| | | | | | | daemon's pty, close and clients words, a pid or a serial for the environment, a capture's header and status line and a cache's .new name are catch unreachable with their reason, where a catch return, null or continue stood for an overflow no buffer here can have Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* idioms #1, std.fs's deprecated aliases in the hosts: std.fs.path and ↵Gabriel Schneider11 days
| | | | | | | std.fs.max_path_bytes are std.Io.Dir.path and std.Io.Dir.max_path_bytes in 0.17, so the hosts name them there Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* state file, a torn one mended: a power loss inside the writeback window ↵Gabriel Schneider12 days
| | | | | | | after Recent's commit (not fsynced) can leave its root header on disk and its bytes not, and xitdb then refuses the file, which pardes set aside as `pardes.xitdb.broken` with every dump in it, whole but unreached; now the open scans back from the file's end for the last commit whose trailer verifies (its CRC-64 over the file id and its own place, its root a map written before it) and whose maps read whole, puts the root header back at it under the file's lock and opens the file there (src/Db.zig `mend`, xitdb's layout read by hand, xitdb itself checking the result), so the Recent's change or two after it are lost and the dumps before it are not; the file is set aside only when no commit is whole, its bytes as they were; on a 105 MB state file (Debug) a whole one opens in 78-93 us, one torn in Recent's last commit after a Dump in 5.0-5.5 ms, and one with no commit whole, scanned to its start, in 429-438 ms before it is set aside; tests: Db.zig's 'one a power loss tore is opened at its last whole commit, the dumps before it kept; it is set aside only when none is whole' is new (a file cut short in its last commit, one with garbage after its last commit and its root header pointing into it, one torn in Recent's commit after a Dump: the Dump read back and loaded each time, its text with it, Recent's change kept or gone as the tear says; one with no commit whole set aside byte for byte), and fails with the mend switched off Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* blob cache: the trim keeps the most recently used texts that fit under the ↵Gabriel Schneider12 days
| | | | | | | cap, the newest first, and lets the rest go, where it let the least recently used go until the rest was under it: a text past the cap alone (a big file's, edited since the client joined) took every older one with it, the small ones that fit too, so a client after it with that cap was sent them all again (daemon.py's far clients E and F at a 1 MiB cap: F found nothing on disk, the 6.5 KB notes gone with the 1.1 MB file's texts; now the notes stay); tests: replica.zig's 'the blob cache stays under its cap, the least recently used going first, the file compacted' becomes '... going first and one past the cap alone without them, the file compacted', a 20 KB text past its 15 KB cap put newest, gone, and the three older ones kept Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* migration: a state file that holds no Recent yet reads it once from the ↵Gabriel Schneider12 days
| | | | | | | files before it, 0.24's $XDG_STATE_HOME/pardes/recent (merged as another session's commit is), else the `store` before that (its `@r` records, the oldest first, as W4's one-time read did), and leaves both where they are (Options.recent_import names their directory, set by main.zig alone, so no test reads the user's); an old `.dump` file, the byte dump, stays readable for one release: `-l <path>` and `Restore <path>` read it, told from an image by its first bytes rather than by its name (the snapshot and process harnesses' dumps are `.dump.zon` files, and a renamed file is the same file), and an Upgrade from the build before, whose record names its `.dump` file and no label, loads it and deletes it as before, so the next Upgrade commits to the state file; tests: Db.zig's 'Recent is read once from the file it replaced, else from the store before that, both left where they are' (in place of W4's 'the store Recent's file replaced is read once, its newest records first, over the file's list'), dump.zig's 'an old dump's file, a byte dump, is read by -l and by a Restore of its path, for one release', and upgrade.zig's 'an upgrade from the build before reads the dump file its record names, and deletes it once loaded' Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* blob cache: a far client's blobs are kept in `cache.xitdb` (src/Db.zig's ↵Gabriel Schneider12 days
| | | | | | | Cache, in $XDG_CACHE_HOME/pardes, else ~/.cache/pardes), a file of its own and never the state file (expendable, and up to a gigabyte): `blobs/<hash>` the texts, `seqs/<hash>` and `seen` (a sorted map, a number to its hash, the least recently used first) their order, `size` and `next`; a join's blobs are put in one transaction by the client's storing thread, with those it found there made the most recently used (a hit takes a new number), and at the client's end what it made since is put and the cache trimmed: past the cap (PARDES_BLOB_CACHE_MB, 1 GiB) the oldest go in one transaction and the file is compacted into `cache.xitdb.new` and renamed over it, where a directory of files was scanned, sorted and deleted from at each join; each use opens the file for itself (a database a thread, as xitdb asks), a text read back is hashed again (one kept wrong is a miss, and goes), and a file that is no database is started afresh; replica.zig's Cache (its directory, the hex names, put by rename, trim by scan) goes for Db.Cache; tests: replica.zig's 'a cached blob is hashed again as it is read: one cut short, corrupted, planted or longer is a miss, and goes' and 'the blob cache stays under its cap, the least recently used going first' are rewritten for the file (texts kept wrong under a hash through put, the order made by hits, and the file seen compacted), 'a blob cache that is no database is started afresh' is new, the far join test keeps its biggest blob wrong through the cache rather than a file, and daemon.py's far clients read the cache file's size, cut it short by hand where they corrupted files, and check the cap after a client's end where they planted junk files Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* dumps: a Dump is a labelled entry of the state file (src/Db.zig, ↵Gabriel Schneider12 days
| | | | | | | architecture §14.3), one fsynced transaction: `dumps/<label>` holds the session as a database of its own, which the walk writes by field name into xitdb's maps (walk.Put: a struct a map keyed by its fields' names, a union a map of its one field, records a list, plain numbers one bytes value tagged with their kind and width, an enum its name, a number a uint or an int, an optional an absent key, a type with dumpWrite its bytes, a .blob text of 4 KiB and up its BLAKE3 hash) and another build reads by name (walk.Get: defaults, notes, a pane it cannot read left out, as the byte loader did), with the hashes of the texts it keeps once under `blobs/`, so an unchanged text is written once across dumps; labels are `pardes-<date>-<time>` (`-2` and on in one second), `last` names the newest, the newest 32 are kept and the texts no kept one names are dropped in the same transaction, and at a session's end the file is compacted into `pardes.xitdb.new` and renamed over once it is past twice what it holds plus 64 MiB (another session notices the new file by its inode); a dump travels as an image, a database in memory holding it alone, deterministic (std.Io.failing's zero ids, so the replay's fixpoint holds): what Dump hands its host, what `Dump export <path>` and $PARDES_DUMP write whole (the snapshot goldens' `Restore <path>` unchanged), what a Restore of a label is read into, what a handover carries (`handover-<seq>`), and what `-l` and `Restore <path>` read; Restore takes a label or a path (bare: this session's last dump, else the newest), a label the file keeps none of refused in words (`Restore: no dump labelled X`, asked of the authority as the closed panes are); `/dumps` lists them, `<label> <date> <size> <panes>`, the newest first; Upgrade commits `upgrade-<pid>` before the exec, its record naming the state file and the label (a record without one, an older build's, names its `.dump` file, read as before), and the label stays; the byte dump stays for the join, the closed panes and the old `.dump` files (dump.byteDump), and dump.outPath, uniqueIn and upgradePath go for dump.label; measured (Debug, a session with a 3.2 MB file and three scratch panes, 20 dumps): the image on the loop 64-66 ms (59 ms of it BLAKE3 of the file, 0.65 ms in ReleaseFast) where the byte dump was 0.7 ms, the commit 9-16 ms with its fsyncs where the file was written in 2, a Restore 1.8 ms to read and 11.4 to load, and the file 3.55 MB after the first dump and 379 KB a dump after it with the big text unchanged (3.58 MB a dump with it edited each time, as twenty `.dump` files would be); tests: dump.zig's 'two dumps in one second get names of their own' and 'a dump makes every directory missing on the way to its DumpDir' become Db.zig's 'two dumps with one label get labels of their own' and 'a Dump makes every directory missing on the way to its DumpDir, and the file there', 'a session read by name, field by field as another build reads it, is the same session' becomes 'a session's byte dump read by name...' beside the new 'a session's image, read by name as every load reads it, is the same session, and dumps to the same bytes again', two replica tests compare the join's stream with dump.byteDump, the Handover tests read the carried image, upgrade's record test carries a label, the root listing tests and the monkey list `dumps`, the behaviour suite gains five root-ctl tests (a Dump and Restore round trip through 9P with `/dumps` and the tag's word, a label kept none of refused in words, the 33rd dump dropping the first, an unchanged text written once across two dumps by the file's size, a killed Dump leaving the one before it whole), the test host performs a Dump's write and a Restore as a host does, the monkey lets go of its log follower at a Restore, and daemon.py's upgrade case finds the kept dump in `/dumps` Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
* store: Recent is a value of the user's state file, `pardes.xitdb` in the ↵Gabriel Schneider12 days
DumpDir (src/Db.zig: the xitdb database the architecture's §14 puts everything pardes keeps on disk in), which every session shares: read at a session's start (now after the startup config, whose DumpDir says where the file is) and again on use when the file's committed size has moved, written at the end of a step that changed it in one transaction under the file's lock, taken without waiting (busy, the next step tries again), another session's commit read and merged under this one's changes first; not fsynced, as the `recent` file was not (a xitdb commit costs 20 us without fsync and 5.2 ms with its two on this btrfs); a file there that is no database of pardes's is kept aside as `pardes.xitdb.broken` and a new one started; Store.zig keeps the closed panes alone (its file half goes: the path and its inode/size/mtime stamp, load, refresh, read, flush and write, and W4's one-time read of the old `store`, which comes back with the migration), Options.recent_path becomes Options.state (off, disk, memory), main.recentPath goes, a Restore's new core takes the old one's database, and the test host's sessions keep theirs in a `.memory` database, the same code; measured (Debug, 200 files listed): flush after an open 133-135 us (W4's file: 124), refresh after another session's write 239-247 us (198), refresh with nothing new 3 us (1), a session started on it 1144-1194 us (1214), and a Recent change adds 12,325 bytes to the file (its 12,089 bytes of lines and 236 of xitdb's); tests: Store.zig's four Recent tests become Db.zig's three (two windows sharing Recent through a busy lock, the newest 200, a file that is no database kept aside; the old store's migration test goes with that read), the closed-pane test keeps its second window on a state file in its DumpDir, the behaviour suite gains 'two sessions on one state file share Recent', and the process tests and the canary look for the state file where they looked for `recent` Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge