# The store: bounded state shared by every pardes window > We should have some persistent state that is bounded (a houndred or so > MiB at most) stored on the fs and shared among all pardes windows. It will > store stuff for like the recent files picker. We should also have a panes > picker and this should include panes from the current session that were > deleted/closed. ## What was there `recent.zig` kept 200 files in `$XDG_STATE_HOME/pardes/recent`, one line a file. Each process read it once at start and, at the end of any step that changed it, wrote the whole of its own list back (`saveRecent`: truncate, write). With two windows that is last-writer-wins: window A opens `a.zig`, window B opens `b.zig`, whichever saves last erases the other's entry, and neither ever sees the other's files until it restarts. A crash between the truncate and the write leaves an empty or half file (the reader passed bad lines over, so it never stopped pardes starting, but the list was gone). Closed panes were nowhere: a closed file's dot was its Recent entry, a closed scratch, terminal or +Errors was gone. ## The format (argued first; the code follows it) One file, `$XDG_STATE_HOME/pardes/store`, an append-only log of framed records, plain enough to `cat`: ``` @ \n\n ``` The first record is `@h` with the body `pardes-store 1`; a file that does not start with it is not a store (garbage, an older format) and reads as empty. The kinds: | kind | body | meaning | |---|---|---| | `h` | `pardes-store 1` | the header, a version | | `r` | ` ` | a recent file and its place: the old `recent` line, unchanged | | `c` | ` \t\n` | a pane closed in session ``, with where it was, and the pane itself as a one-pane Dump | | `x` | ` ` | that closed pane was reopened: no longer listed | Folding the log in order gives the state: a path's latest `r` wins and its order is the order of last writes (so every window folds the same list); a `c` not followed by its `x` is a closed pane. **Why a closed pane is a Dump.** `dump.zig` already turns any pane into data and back: a file's dot and scroll (its text only when unsaved), a scratch's or an output pane's text with its origin and rows' metadata, a terminal's scrollback (a shell comes back running under its old history, a command pane finished, as Restore does), a PDF's page, an image. Reusing it means Panes can bring back every kind of pane with the same fidelity as Restore, through the same validation (`dump.readZon` checks a record before anything is built from it), and nothing new to keep in step when a pane grows a field. **Why on disk and not in memory**, since only this session reads its own closed panes: their text is what makes them big (a terminal's history, a long +Errors, a discarded unsaved buffer), and the store is where the bound lives; in memory each window keeps only an index (serial, name, place, offset). Restore keeps them (same process, same session). A session that crashed leaves its records for the next compaction to drop. Position, not a verdict: an in-memory list would be simpler, and I would accept that pushback; the cap argument is what tipped it. **Bounds.** The file never passes `max_bytes` (100 MiB): an append that would compacts first. A compaction keeps the newest live records within half of that, so it is not redone at once. A record is at most `max_record` (8 MiB): a terminal whose dump is bigger is kept without its history (it comes back as a fresh shell in its directory), any other pane that big is not kept. Recent keeps `recent.cap` (200) files; a session keeps its 64 newest closed panes. ## The sharing - **Writers** take `flock(LOCK_EX | LOCK_NB)` on `store.lock`, a file of its own (the store's inode changes at each compaction, so locking it would let two writers hold locks on two different files). Busy, the records wait in memory for the next step: a window never blocks a frame on another's compaction. At exit the lock is waited for. - Under the lock a writer first reads what others appended since it last read (so its list's order is the log's), then writes its records with one positional write at the end of the file, then notes the new end. - **Readers** take no lock. On use (Recent, Panes, `/recent`, before an append) one `stat` of the path: the same inode and size as last read, nothing to do; the same inode and longer, read the tail; another inode (a compaction renamed a new file over it) or shorter, read it all again. - **Torn and garbage records.** A record whose length runs past the end of the file is being written (or its writer died): reading stops before it and tries again next time. A complete record with a bad CRC or a bad header is garbage: the reader resynchronises at the next `\n@` that starts a valid record, and marks the file for compaction, which the next writer does under the lock (rewriting only valid records). A missing, unreadable or foreign file is an empty store: pardes always starts. - **Compaction**, under the lock: fold the whole file, write the live records (Recent, oldest first so the fold gives the same order; closed panes of sessions whose pid is alive, copied byte for byte) to `store.new`, `fsync`, rename it over `store`. Readers see the new inode. Triggered when the file is past twice its live size plus 1 MiB, when it holds garbage, or when an append would pass the cap. - **Startup reads headers, not bodies**: a new session has no closed panes of its own, so every `c` body is skipped by its length; only `r` bodies (a line each) are read. ### Rejected 1. **Rewrite the whole file under a lock** (today's scheme, made safe with read-merge-write and a rename): correct, but every open or close costs the whole file, and with closed panes' text in it that is megabytes per Del. 2. **SQLite**: one more C library to vendor and hook into Zig's allocators (the shared-deps policy), WAL needs shared memory, for three record kinds and a fold. 3. **A file per entry** (maildir): atomic by rename and lock-free, but the bound means listing and statting a directory of thousands of files on every use, two windows race deleting the same oldest, and "what changed" is not one `stat`. 4. **A file per window, merged by readers**: no lock, but a merge needs wall clocks across processes for order, every read opens N files, and dead windows' files need a reaper that races the living. 5. **A fixed-size mmap'd ring**: always 100 MiB on disk, cross-process writes into shared pages need atomics, a writer that dies mid-record corrupts in place, and a format change is a migration. 6. **A watcher thread / inotify** (spec says no) or **a daemon** (none exists; pardes must start without one): a `stat` on use is cheaper than either and as fresh as anything that is looked at. The old `recent` file was read once into a store that did not exist yet, so nobody's list was lost; with the store in use, that import went (2026-11-04). ## As built | piece | where | |---|---| | the log: framing, reader, resync, fold, flush, compaction | `src/Store.zig` (`Pardes.store`) | | Recent as records: `Entry.changed`, `List.note` (a record read), `changedFirst` | `src/recent.zig` | | a pane to and from a Dump, one at a time | `dump.dumpPane`, `dump.restorePane` (lifted out of `dumpState` / `initDump`) | | a closed pane kept: before its teardown, its rows' gainer after | `Pardes.removePane`, `Pardes.removeColumn` (`Store.keep`, `Store.close`) | | back where it was | `Store.reopen` → `layout.giveBack` (rows from the gainer alone), `layout.attachFrom` for an attachment whose owner is open, else `exec.placeNew` | | Panes: open panes, then this session's closed ones, `@p7 notes/a.md (closed)`, dimmer (the gutter's colour between the match marks) | `Query.startSource` / `makeNameRows` / `land` | | Recent and `/recent` read the store on use | `Query.startSource`, `ninep/tree.zig` | | the path | `main.storePath`: `$XDG_STATE_HOME/pardes/store` (`Options.store_path`, was `recent_path`) | Which closes are kept: every `removePane` / `removeColumn` of a pane the user had (announced: a pane made and refused in one step never was), not a query pane nor a +Preview. Exit and Restore keep nothing (Restore's new core takes the old session's list, `Store.clone`). Brought back, a pane is a new pane (a fresh serial, as Restore gives); its row goes (`x` record). A Look at `@p7` naming a pane this session closed brings it back instead of the "no pane @p7" miss: the row's text is its address, as everywhere. That is the only change to Look, in its miss branch. **The turn.** Every host call is made out of the turn (`Store.host(f, args)`, one call each), so a store under a mount of this session's own tree cannot deadlock, and nothing in the store or Recent changes while a step is out. `/recent` (a 9P read) reads the store only when no step is out; otherwise it answers the list as it was. ## Measured (Debug, the `zz store timings` test, STORE_TIMES=) A store as a running session leaves it: 200 files and 90 closed panes of 1 MiB (94 MB). | what | cost | |---|---| | a session starting on it (`load`: headers only, every `c` body skipped) | 3.4 ms | | a refresh with nothing new (one `stat`) | ~1 µs | | a flush of one record (lock, read the tail, one write) | 36 µs | | a compaction of the 94 MB file (CRC of every record, copy the live, rename) | 278 ms, to 51 MB (the newest within half the cap) | ## Tests `store: two windows share Recent...` (two cores on one file, a lock held by a third party delays a write and loses nothing, a fresh window folds the same list), `store: the cap holds...` (3000 files and 30 closed 20 KB scratches against a 48 KiB / 256 KiB cap: never past it, compactions happen, the newest 200 files and the newest closed pane survive), `store: a garbage or torn file...` (a foreign file, a torn header, junk, then a whole record; a half-written record waited for; mended by the next writer), `store: a closed pane comes back...` (from the file, not memory; its rect, the gainer's and another column's as before the close; listed `(closed)` in Panes, Enter brings it back, unlisted after a re-read; another session lists none). `test/fs.py store_shared_between_windows`: two live sessions over 9P. The related suites (dump, Restore, query, Find, Grep, Del, Delcol, attach, collapse, jump, Edit, recent) pass. The tty build compiles. Found, not mine: `fs.py recent_survives_restart` fails on the converge baseline too, at its Recent step (the query pane's rows are relative names now, the test looks for absolute ones). ## Open questions, and my position on each 1. **Closed panes on disk or in memory?** On disk (argued above). The honest counter: only this session reads them, and a Del now costs a dump and a write. Measured, a flush is 36 µs; the dump of a big terminal is the cost (its history base64'd), capped at 8 MiB. 2. **Recent's cap stays 200**, now for every window together. Shared, it fills faster; 1000 is cheap for the fuzzy scorer. I kept 200 to change one thing at a time; I would raise it. 3. **"open" means open here.** A file open in another window reads `(closed)` in this one's Recent (its place is the other window's last close). Saying `open elsewhere` needs the store to know which windows are alive and what they hold (a record per open, its pid): doable, not done. 4. **Compaction is on the loop** (278 ms for 94 MB, Debug). It happens when the file doubles past what was live, or reaches the cap, which a window that closes many big terminals does. Off the loop it would need the disk engine's worker and the lock held across frames; I would do it only if it shows up. A store of Recent alone is ~20 KB to rewrite (not timed separately). 5. **A crashed session's closed panes** are dropped at the next compaction (its pid is gone). Keeping them a day for a "what was I doing" word would be cheap: the records are there. 6. **A dirty file pane closed with the second Del** comes back with its unsaved text (it is in its dump, by Restore's rules; not tested here). That is a feature (undo a discard) and a surprise (the discard was asked for twice). I kept it; say if a discard should be final. ## What I would push back on - "shared among all pardes windows" for the closed panes: they are this session's by the spec, so the sharing is only of the file and its bound. If the bound is the only reason, an in-memory list with a byte budget is less code (no `c`/`x` records, no offsets); I chose the disk for the memory and Restore, but this is the part I hold least firmly. - The 100 MiB: nothing in Recent comes near it; only closed terminals' history does. A smaller cap (16 MiB) would bound compaction to ~50 ms and lose little. 100 MiB is what was asked, so it is the default. - Reopening a closed pane makes a new serial: `@p7` in old text names a pane that is now `@p12`. Restore does the same and logs `restored 7 12`; the store could log the same, not done. ## Rough - The dim tint of a closed Panes row is the gutter colour (`lineno`), not checked by eye in every theme. - A Delcol'd pane comes back where `placeNew` puts it (its column is gone). - An attachment whose owner was closed with it comes back as a plain pane next to wherever placement puts it (its owner's serial is new).