diff options
| author | Gabriel Schneider <[email protected]> | 2026-11-05 15:40:52 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-11-05 19:16:37 -0300 |
| commit | b48f79a1c19c4c96c30d29d5f8ac5ad19c8e3475 (patch) | |
| tree | 65a7620c2826bb03167c32e1df4681ed5c0b4f55 /docs/next-store.md | |
| parent | cd712084c68b9e2bad350066a92a96ce78a9ff35 (diff) | |
| download | pardes-b48f79a1c19c4c96c30d29d5f8ac5ad19c8e3475.tar.gz pardes-b48f79a1c19c4c96c30d29d5f8ac5ad19c8e3475.zip | |
docs: the state file and the dumps as they are now: the reference's Dump and Restore paragraph (a label, `/dumps`, `Dump export <path>` to carry one, Restore of a label or a path, the words for a label the file keeps none of), its DumpDir row (the state file's directory, one that cannot be read kept aside), `/recent` kept in the state file, `/dumps` among the root files, the snapshots and the stats, and the log's `dump` and `restore` records; building.typ's Upgrade (`upgrade-<pid>` in the state file, the build before's `.dump` still read); setup.typ's new list of the files pardes keeps (the startup file and the crash record, the state file, the blob cache, the runtime directory's, and the build before's to delete); next-store.md's decision (everything on disk through xitdb, with the measured costs) over W4's, the closed panes under it and its tests as they are; and the changelog's entry, what a user must know: labels where dump files were, `Dump export`, `.dump` files read for one release, Recent read once from its old file, DumpDir now the state file's directory, the old cache directory no longer read, an unreadable state file kept aside
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Y9MNAtZenFv4weoMD3cge
Diffstat (limited to 'docs/next-store.md')
| -rw-r--r-- | docs/next-store.md | 122 |
1 files changed, 91 insertions, 31 deletions
diff --git a/docs/next-store.md b/docs/next-store.md index 89997ee2..3c8a2f7d 100644 --- a/docs/next-store.md +++ b/docs/next-store.md @@ -1,4 +1,4 @@ -# The store: Recent's file, and this session's closed panes +# The store: the state file, the blob cache, and this session's closed panes > We should have some persistent state that is bounded (a houndred or so > MiB at most) stored on the fs and shared among all pardes windows. It will @@ -6,22 +6,66 @@ > picker and this should include panes from the current session that were > deleted/closed. -## The decision (2026-11-05) +## The decision (2026-11-05, the persistence step): everything on disk through xitdb -**Recent is a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent` -(else `~/.local/state/pardes/recent`), a line a file, the newest first: -`<line> <col> <aline> <acol> <path>`. Every session reads it at start, and -again on use (Recent, `/recent`) when a stat of it says another session -wrote it since. A step that changed the list writes it at its end: `flock` -on the file (held by another session: the next step tries again, so a -frame never waits for one; at exit the lock is waited for), the file read -again with this session's changes put over it (`recent.List.merge`) when -another session wrote it since, and the whole list written over it in -place. A line that is no entry is passed over: a missing, unreadable or -foreign file is an empty list, a reader meeting a write half done reads it -again once its stat moves on, a line a crash tore is one entry fewer, and -pardes always starts. A rename of a new file over it (atomic) cost twice -the write in Debug for nothing the lenient read does not already give. +What pardes keeps on disk goes through xitdb (`src/Db.zig`; architecture +ยง14): two files, both BLAKE3-keyed, both with xitdb's header-last commit, +so a kill or a full disk leaves the last commit whole. + +**The state file**, `pardes.xitdb` in the DumpDir (`$XDG_DATA_HOME/pardes`, +else `~/.local/share/pardes`), one a user: Recent is documented as shared +by every window, a Restore is any session's. Its root is a map: + +- `recent`: Recent's 200 lines as one value, the format the `recent` file + had. A session reads it at start (after the startup config, whose DumpDir + says where the file is) and on use when the file's committed size has + moved; a step that changed it reads, merges (`recent.List.merge`) and + writes it in one transaction under the file's lock, taken without + waiting: busy, the next step tries again. Not fsynced, as the file never + was: a commit is 20 us without fsync and 5.2 ms with xitdb's two on this + machine's btrfs, against 124-148 us for the whole flush after an open. + Measured: 133-135 us for that flush (Debug, 200 files), and 12,325 bytes + of file a Recent change (its 12,089 bytes of lines and 236 of xitdb's). +- `dumps/<label>`: a dump each, `pardes-<date>-<time>` (UTC, `-2` and on in + one second), `upgrade-<pid>`: its number, date, pane count and size, the + session as a database of its own (walk.Put's maps, by field name: another + build reads it by name), and the hashes of the texts it keeps beside it. + One Dump is one fsynced transaction; the newest 32 are kept, the texts no + kept one names dropped in the same transaction. +- `blobs/<hash>`: each such text (a `.blob` field of 4 KiB and up) once: + an unchanged text is written once across dumps. +- `last`, `seq`: the newest dump's label, the next number. + +At a session's end the file is compacted into `pardes.xitdb.new` and +renamed over it once it is past twice what it holds plus 64 MiB, under the +lock; another session notices the new file by its inode and opens it. A +file there that is no database is kept aside as `pardes.xitdb.broken`. + +**A dump travels as an image**: a database in memory holding one dump in +the same shape (deterministic: no random ids, so one session is one +image). The core makes one at a Dump (the host puts it in the state file +under its label, or writes it whole for `Dump export <path>` and +`$PARDES_DUMP`), a Restore of a label is read into one, a handover's +carried panes are one (`handover-<seq>`), and `-l <file>` and +`Restore <path>` read one, an old byte dump (`.dump`) too for one release. + +**The far client's blob cache** is `cache.xitdb` in `$XDG_CACHE_HOME/pardes`: +expendable and up to a gigabyte, so never the state file. `blobs/<hash>`, +`seen` (a sorted map of numbers to hashes, the least recently used first), +`size`; past the cap (`PARDES_BLOB_CACHE_MB`, 1 GiB) the oldest go and the +file is compacted, at a client's end. + +**Not in a database**: the closed panes (this session's, in memory, as +below), the crash record (one `write(2)` from the panic handler, which may +not lock or allocate), the startup file and themes (edited by hand), and +the kernel's and the shells' files (sockets, locks, the registry, rc +snippets). + +**Migration**: Recent is read once from the `recent` file this decision +replaces, else the `store` before it, when the state file holds none yet; +both are left where they are. + +## Closed panes **Closed panes are this session's, in memory**: the newest 64, within 64 MiB together, each a one-pane dump (`dump.savePane`, at most 8 MiB; a @@ -30,6 +74,12 @@ and where it was. Panes lists them after the open panes, `(closed)`; Enter, or a Look at `@pN`, brings one back beside the pane that took its rows. Restore keeps them; the session's end drops them. +## Before it, the same day + +**Recent was a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent`, +under `flock`, written whole in place; the state file keeps those lines as +one value and reads the file once. + ## What went, and why The store this replaced (2026-10-24, Store.zig 1,135 lines) was one @@ -46,21 +96,31 @@ or closes a file. What a user loses: the closed panes of other sessions, which the log kept on disk until a compaction and nothing ever listed. -The first session of a build with this reads the old `store` once more, -when it is newer than `recent`: its `@r` records' bodies are this file's -lines. The old file is left in place for an older pardes still running. +W4's `recent` file read the old `store` once more while it was the newer +of the two; the state file reads `recent`, else `store`, once (the +migration above), and leaves both for an older pardes still running. ## Tests -Store.zig: two windows share Recent (each reads the other's writes on use; -a lock held by a third party delays a write, never loses it); the file -holds the newest 200, newest first; a foreign file's Recent lines are read -and the next write mends it; the old store read once, its newest records -first; a session keeps its newest 64 closed panes; a closed pane comes back -where it was, its rows from the pane that took them, listed `(closed)` in -Panes and unlisted once back, and another session lists none. -recent.zig: a merge puts this session's changes over the file's list. -test/process/other_files.zig: a file opened and closed is in `/recent` -closed and still is in the next session; two sessions share Recent, and a -Del'd pane comes back from `@p` only where it was. test/behaviour/ -other_files.zig: `/recent` and Recent's rows. +Db.zig: two windows share Recent (each reads the other's commits on use; a +lock held by a third party delays a write, never loses it); the file holds +the newest 200, newest first; a file that is no database is kept aside and +a new one started; Recent is read once from the `recent` file, else the +`store`, both left in place; two dumps of one label get labels of their own; +a Dump makes the directories of its DumpDir. Store.zig: a session keeps its +newest 64 closed panes; a closed pane comes back where it was, its rows from +the pane that took them, listed `(closed)` in Panes and unlisted once back, +and another session lists none. recent.zig: a merge puts this session's +changes over the list read. dump.zig: an image read by name is the session, +and dumps to the same bytes again; an old `.dump` is read by `-l` and +Restore. upgrade.zig: an older build's record loads its `.dump` file. +replica.zig: the blob cache rehashes what it reads, keeps its cap by least +recent use and compaction, and starts afresh over a file that is no +database. test/behaviour/root_ctl.zig: a Dump and Restore round trip through +9P, a missing label refused in words, the 33rd dump dropping the first, an +unchanged text written once across two dumps, a killed Dump leaving the one +before it whole. test/behaviour/other_files.zig: two sessions on one state +file share Recent. test/process/other_files.zig: a file opened and closed is +in `/recent` closed and still is in the next session; two sessions share +Recent. test/daemon.py: an Upgrade keeps its dump in `/dumps`, and far +clients' cache. |
