summaryrefslogtreecommitdiff
path: root/docs/next-store.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/next-store.md')
-rw-r--r--docs/next-store.md122
1 files changed, 91 insertions, 31 deletions
diff --git a/docs/next-store.md b/docs/next-store.md
index 89997ee2..3c8a2f7d 100644
--- a/docs/next-store.md
+++ b/docs/next-store.md
@@ -1,4 +1,4 @@
-# The store: Recent's file, and this session's closed panes
+# The store: the state file, the blob cache, and this session's closed panes
> We should have some persistent state that is bounded (a houndred or so
> MiB at most) stored on the fs and shared among all pardes windows. It will
@@ -6,22 +6,66 @@
> picker and this should include panes from the current session that were
> deleted/closed.
-## The decision (2026-11-05)
+## The decision (2026-11-05, the persistence step): everything on disk through xitdb
-**Recent is a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent`
-(else `~/.local/state/pardes/recent`), a line a file, the newest first:
-`<line> <col> <aline> <acol> <path>`. Every session reads it at start, and
-again on use (Recent, `/recent`) when a stat of it says another session
-wrote it since. A step that changed the list writes it at its end: `flock`
-on the file (held by another session: the next step tries again, so a
-frame never waits for one; at exit the lock is waited for), the file read
-again with this session's changes put over it (`recent.List.merge`) when
-another session wrote it since, and the whole list written over it in
-place. A line that is no entry is passed over: a missing, unreadable or
-foreign file is an empty list, a reader meeting a write half done reads it
-again once its stat moves on, a line a crash tore is one entry fewer, and
-pardes always starts. A rename of a new file over it (atomic) cost twice
-the write in Debug for nothing the lenient read does not already give.
+What pardes keeps on disk goes through xitdb (`src/Db.zig`; architecture
+ยง14): two files, both BLAKE3-keyed, both with xitdb's header-last commit,
+so a kill or a full disk leaves the last commit whole.
+
+**The state file**, `pardes.xitdb` in the DumpDir (`$XDG_DATA_HOME/pardes`,
+else `~/.local/share/pardes`), one a user: Recent is documented as shared
+by every window, a Restore is any session's. Its root is a map:
+
+- `recent`: Recent's 200 lines as one value, the format the `recent` file
+ had. A session reads it at start (after the startup config, whose DumpDir
+ says where the file is) and on use when the file's committed size has
+ moved; a step that changed it reads, merges (`recent.List.merge`) and
+ writes it in one transaction under the file's lock, taken without
+ waiting: busy, the next step tries again. Not fsynced, as the file never
+ was: a commit is 20 us without fsync and 5.2 ms with xitdb's two on this
+ machine's btrfs, against 124-148 us for the whole flush after an open.
+ Measured: 133-135 us for that flush (Debug, 200 files), and 12,325 bytes
+ of file a Recent change (its 12,089 bytes of lines and 236 of xitdb's).
+- `dumps/<label>`: a dump each, `pardes-<date>-<time>` (UTC, `-2` and on in
+ one second), `upgrade-<pid>`: its number, date, pane count and size, the
+ session as a database of its own (walk.Put's maps, by field name: another
+ build reads it by name), and the hashes of the texts it keeps beside it.
+ One Dump is one fsynced transaction; the newest 32 are kept, the texts no
+ kept one names dropped in the same transaction.
+- `blobs/<hash>`: each such text (a `.blob` field of 4 KiB and up) once:
+ an unchanged text is written once across dumps.
+- `last`, `seq`: the newest dump's label, the next number.
+
+At a session's end the file is compacted into `pardes.xitdb.new` and
+renamed over it once it is past twice what it holds plus 64 MiB, under the
+lock; another session notices the new file by its inode and opens it. A
+file there that is no database is kept aside as `pardes.xitdb.broken`.
+
+**A dump travels as an image**: a database in memory holding one dump in
+the same shape (deterministic: no random ids, so one session is one
+image). The core makes one at a Dump (the host puts it in the state file
+under its label, or writes it whole for `Dump export <path>` and
+`$PARDES_DUMP`), a Restore of a label is read into one, a handover's
+carried panes are one (`handover-<seq>`), and `-l <file>` and
+`Restore <path>` read one, an old byte dump (`.dump`) too for one release.
+
+**The far client's blob cache** is `cache.xitdb` in `$XDG_CACHE_HOME/pardes`:
+expendable and up to a gigabyte, so never the state file. `blobs/<hash>`,
+`seen` (a sorted map of numbers to hashes, the least recently used first),
+`size`; past the cap (`PARDES_BLOB_CACHE_MB`, 1 GiB) the oldest go and the
+file is compacted, at a client's end.
+
+**Not in a database**: the closed panes (this session's, in memory, as
+below), the crash record (one `write(2)` from the panic handler, which may
+not lock or allocate), the startup file and themes (edited by hand), and
+the kernel's and the shells' files (sockets, locks, the registry, rc
+snippets).
+
+**Migration**: Recent is read once from the `recent` file this decision
+replaces, else the `store` before it, when the state file holds none yet;
+both are left where they are.
+
+## Closed panes
**Closed panes are this session's, in memory**: the newest 64, within
64 MiB together, each a one-pane dump (`dump.savePane`, at most 8 MiB; a
@@ -30,6 +74,12 @@ and where it was. Panes lists them after the open panes, `(closed)`;
Enter, or a Look at `@pN`, brings one back beside the pane that took its
rows. Restore keeps them; the session's end drops them.
+## Before it, the same day
+
+**Recent was a file of at most 200 lines**, `$XDG_STATE_HOME/pardes/recent`,
+under `flock`, written whole in place; the state file keeps those lines as
+one value and reads the file once.
+
## What went, and why
The store this replaced (2026-10-24, Store.zig 1,135 lines) was one
@@ -46,21 +96,31 @@ or closes a file.
What a user loses: the closed panes of other sessions, which the log kept
on disk until a compaction and nothing ever listed.
-The first session of a build with this reads the old `store` once more,
-when it is newer than `recent`: its `@r` records' bodies are this file's
-lines. The old file is left in place for an older pardes still running.
+W4's `recent` file read the old `store` once more while it was the newer
+of the two; the state file reads `recent`, else `store`, once (the
+migration above), and leaves both for an older pardes still running.
## Tests
-Store.zig: two windows share Recent (each reads the other's writes on use;
-a lock held by a third party delays a write, never loses it); the file
-holds the newest 200, newest first; a foreign file's Recent lines are read
-and the next write mends it; the old store read once, its newest records
-first; a session keeps its newest 64 closed panes; a closed pane comes back
-where it was, its rows from the pane that took them, listed `(closed)` in
-Panes and unlisted once back, and another session lists none.
-recent.zig: a merge puts this session's changes over the file's list.
-test/process/other_files.zig: a file opened and closed is in `/recent`
-closed and still is in the next session; two sessions share Recent, and a
-Del'd pane comes back from `@p` only where it was. test/behaviour/
-other_files.zig: `/recent` and Recent's rows.
+Db.zig: two windows share Recent (each reads the other's commits on use; a
+lock held by a third party delays a write, never loses it); the file holds
+the newest 200, newest first; a file that is no database is kept aside and
+a new one started; Recent is read once from the `recent` file, else the
+`store`, both left in place; two dumps of one label get labels of their own;
+a Dump makes the directories of its DumpDir. Store.zig: a session keeps its
+newest 64 closed panes; a closed pane comes back where it was, its rows from
+the pane that took them, listed `(closed)` in Panes and unlisted once back,
+and another session lists none. recent.zig: a merge puts this session's
+changes over the list read. dump.zig: an image read by name is the session,
+and dumps to the same bytes again; an old `.dump` is read by `-l` and
+Restore. upgrade.zig: an older build's record loads its `.dump` file.
+replica.zig: the blob cache rehashes what it reads, keeps its cap by least
+recent use and compaction, and starts afresh over a file that is no
+database. test/behaviour/root_ctl.zig: a Dump and Restore round trip through
+9P, a missing label refused in words, the 33rd dump dropping the first, an
+unchanged text written once across two dumps, a killed Dump leaving the one
+before it whole. test/behaviour/other_files.zig: two sessions on one state
+file share Recent. test/process/other_files.zig: a file opened and closed is
+in `/recent` closed and still is in the next session; two sessions share
+Recent. test/daemon.py: an Upgrade keeps its dump in `/dumps`, and far
+clients' cache.