diff options
Diffstat (limited to 'docs/next-store.md')
| -rw-r--r-- | docs/next-store.md | 122 |
1 files changed, 91 insertions, 31 deletions
diff --git a/docs/next-store.md b/docs/next-store.md index 89997ee2..3c8a2f7d 100644 --- a/docs/next-store.md +++ b/docs/next-store.md @@ -1,4 +1,4 @@ -# The store: Recent's file, and this session's closed panes +# The store: the state file, the blob cache, and this session's closed panes > We should have some persistent state that is bounded (a houndred or so > MiB at most) stored on the fs and shared among all pardes windows. It will @@ -6,22 +6,66 @@ > picker and this should include panes from the current session that were > deleted/closed. -## The decision (2026-11-05) +## The decision (2026-11-05, the persistence step): everything on disk through xitdb -**Recent is a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent` -(else `~/.local/state/pardes/recent`), a line a file, the newest first: -`<line> <col> <aline> <acol> <path>`. Every session reads it at start, and -again on use (Recent, `/recent`) when a stat of it says another session -wrote it since. A step that changed the list writes it at its end: `flock` -on the file (held by another session: the next step tries again, so a -frame never waits for one; at exit the lock is waited for), the file read -again with this session's changes put over it (`recent.List.merge`) when -another session wrote it since, and the whole list written over it in -place. A line that is no entry is passed over: a missing, unreadable or -foreign file is an empty list, a reader meeting a write half done reads it -again once its stat moves on, a line a crash tore is one entry fewer, and -pardes always starts. A rename of a new file over it (atomic) cost twice -the write in Debug for nothing the lenient read does not already give. +What pardes keeps on disk goes through xitdb (`src/Db.zig`; architecture +ยง14): two files, both BLAKE3-keyed, both with xitdb's header-last commit, +so a kill or a full disk leaves the last commit whole. + +**The state file**, `pardes.xitdb` in the DumpDir (`$XDG_DATA_HOME/pardes`, +else `~/.local/share/pardes`), one a user: Recent is documented as shared +by every window, a Restore is any session's. Its root is a map: + +- `recent`: Recent's 200 lines as one value, the format the `recent` file + had. A session reads it at start (after the startup config, whose DumpDir + says where the file is) and on use when the file's committed size has + moved; a step that changed it reads, merges (`recent.List.merge`) and + writes it in one transaction under the file's lock, taken without + waiting: busy, the next step tries again. Not fsynced, as the file never + was: a commit is 20 us without fsync and 5.2 ms with xitdb's two on this + machine's btrfs, against 124-148 us for the whole flush after an open. + Measured: 133-135 us for that flush (Debug, 200 files), and 12,325 bytes + of file a Recent change (its 12,089 bytes of lines and 236 of xitdb's). +- `dumps/<label>`: a dump each, `pardes-<date>-<time>` (UTC, `-2` and on in + one second), `upgrade-<pid>`: its number, date, pane count and size, the + session as a database of its own (walk.Put's maps, by field name: another + build reads it by name), and the hashes of the texts it keeps beside it. + One Dump is one fsynced transaction; the newest 32 are kept, the texts no + kept one names dropped in the same transaction. +- `blobs/<hash>`: each such text (a `.blob` field of 4 KiB and up) once: + an unchanged text is written once across dumps. +- `last`, `seq`: the newest dump's label, the next number. + +At a session's end the file is compacted into `pardes.xitdb.new` and +renamed over it once it is past twice what it holds plus 64 MiB, under the +lock; another session notices the new file by its inode and opens it. A +file there that is no database is kept aside as `pardes.xitdb.broken`. + +**A dump travels as an image**: a database in memory holding one dump in +the same shape (deterministic: no random ids, so one session is one +image). The core makes one at a Dump (the host puts it in the state file +under its label, or writes it whole for `Dump export <path>` and +`$PARDES_DUMP`), a Restore of a label is read into one, a handover's +carried panes are one (`handover-<seq>`), and `-l <file>` and +`Restore <path>` read one, an old byte dump (`.dump`) too for one release. + +**The far client's blob cache** is `cache.xitdb` in `$XDG_CACHE_HOME/pardes`: +expendable and up to a gigabyte, so never the state file. `blobs/<hash>`, +`seen` (a sorted map of numbers to hashes, the least recently used first), +`size`; past the cap (`PARDES_BLOB_CACHE_MB`, 1 GiB) the oldest go and the +file is compacted, at a client's end. + +**Not in a database**: the closed panes (this session's, in memory, as +below), the crash record (one `write(2)` from the panic handler, which may +not lock or allocate), the startup file and themes (edited by hand), and +the kernel's and the shells' files (sockets, locks, the registry, rc +snippets). + +**Migration**: Recent is read once from the `recent` file this decision +replaces, else the `store` before it, when the state file holds none yet; +both are left where they are. + +## Closed panes **Closed panes are this session's, in memory**: the newest 64, within 64 MiB together, each a one-pane dump (`dump.savePane`, at most 8 MiB; a @@ -30,6 +74,12 @@ and where it was. Panes lists them after the open panes, `(closed)`; Enter, or a Look at `@pN`, brings one back beside the pane that took its rows. Restore keeps them; the session's end drops them. +## Before it, the same day + +**Recent was a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent`, +under `flock`, written whole in place; the state file keeps those lines as +one value and reads the file once. + ## What went, and why The store this replaced (2026-10-24, Store.zig 1,135 lines) was one @@ -46,21 +96,31 @@ or closes a file. What a user loses: the closed panes of other sessions, which the log kept on disk until a compaction and nothing ever listed. -The first session of a build with this reads the old `store` once more, -when it is newer than `recent`: its `@r` records' bodies are this file's -lines. The old file is left in place for an older pardes still running. +W4's `recent` file read the old `store` once more while it was the newer +of the two; the state file reads `recent`, else `store`, once (the +migration above), and leaves both for an older pardes still running. ## Tests -Store.zig: two windows share Recent (each reads the other's writes on use; -a lock held by a third party delays a write, never loses it); the file -holds the newest 200, newest first; a foreign file's Recent lines are read -and the next write mends it; the old store read once, its newest records -first; a session keeps its newest 64 closed panes; a closed pane comes back -where it was, its rows from the pane that took them, listed `(closed)` in -Panes and unlisted once back, and another session lists none. -recent.zig: a merge puts this session's changes over the file's list. -test/process/other_files.zig: a file opened and closed is in `/recent` -closed and still is in the next session; two sessions share Recent, and a -Del'd pane comes back from `@p` only where it was. test/behaviour/ -other_files.zig: `/recent` and Recent's rows. +Db.zig: two windows share Recent (each reads the other's commits on use; a +lock held by a third party delays a write, never loses it); the file holds +the newest 200, newest first; a file that is no database is kept aside and +a new one started; Recent is read once from the `recent` file, else the +`store`, both left in place; two dumps of one label get labels of their own; +a Dump makes the directories of its DumpDir. Store.zig: a session keeps its +newest 64 closed panes; a closed pane comes back where it was, its rows from +the pane that took them, listed `(closed)` in Panes and unlisted once back, +and another session lists none. recent.zig: a merge puts this session's +changes over the list read. dump.zig: an image read by name is the session, +and dumps to the same bytes again; an old `.dump` is read by `-l` and +Restore. upgrade.zig: an older build's record loads its `.dump` file. +replica.zig: the blob cache rehashes what it reads, keeps its cap by least +recent use and compaction, and starts afresh over a file that is no +database. test/behaviour/root_ctl.zig: a Dump and Restore round trip through +9P, a missing label refused in words, the 33rd dump dropping the first, an +unchanged text written once across two dumps, a killed Dump leaving the one +before it whole. test/behaviour/other_files.zig: two sessions on one state +file share Recent. test/process/other_files.zig: a file opened and closed is +in `/recent` closed and still is in the next session; two sessions share +Recent. test/daemon.py: an Upgrade keeps its dump in `/dumps`, and far +clients' cache. |
