summaryrefslogtreecommitdiff
path: root/src/host/Sessions.zig
Commit message (Collapse)AuthorAge
* Dial by name, and fs in every sketch's binary, answering with the sketch closedGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A run a person watches (not headless, a script or an export) posts its tree as $XDG_RUNTIME_DIR/9p/sketchbook/<name>, the sketch's Info.name; --9p NAME posts under another name, --9p /path listens at a socket path as before. A name a live run holds is refused, a stale one replaced; a default post that fails says so and the run goes on without one, an explicit --9p that fails ends it as before. (cloud9's post() takes names without a slash, so control.zig makes the sketchbook/ directory and binds there itself, with post.probe deciding live or stale.) `fs VERB [PATH [WORDS]]`, after the options, in the generic host (`sketchbook LIBRARY fs ...`) and in every executable built with sketchbook (`drat-meout fs read state/scene`), is cloud9.verbs.run on an Fsys: the running sketch's tree when one is posted (or at --9p), else the same tree served in this process over a socketpair, on a thread (Control.serveOffline, the same respond with `offline` set), from the save's newest State (--state's save, else the default one; read only: Sessions.openRead, no session started, restored into today's layout), else from the State's declared defaults (init is not run). README, commands, ctl's read and state/ answer; a write and status, log, screen.png and the traces answer in words, "drat-meout is not running: this is its save's newest State, frame 61 of <save>". No window, no SDL. sketchbook.init's modules gain cloud9, so an app's own tests can dial its executable with cloud9.Fsys. Tests: a new step, test-fs (in test): the replay fixture posted by name under a private runtime directory, driven with its own binary's fs: ls, a state write and read, a refused value in words, a live click, a screenshot read, help of state/dots and ctl, quit and the name gone; then closed: a state file read from the defaults, status refused in words. test-fs, test-control, test-paths, test-setup, test-sessions: 56/56 steps, 27/27 tests. drat-meout's copy builds unchanged and answers fs with the game closed: 94 leaves, 16,522 files, 17,608 values, 10,408 directories. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* The cap: a save is at most 64 MiB, State half of it, history dropped oldest ↵Gabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | first A save grew without bound. Now Sessions.max_save is 64 MiB and layout.max_state 32 MiB, so history always has room (the compile error says so: "State is N bytes, and may be 32 MiB, half of a save, so history has room: ..."). A commit that leaves the file past the cap trims it (Sessions.trim): one transaction drops history oldest first, a keyframe by removing its pages (its frame, batch and time stay, so keyframe numbers, keyBefore and marks by index still resolve) and a batch of frames by emptying its slot, until the keyframes' written pages and the batches left count three quarters of the cap; then the file is compacted into `<path>.new` and renamed over it, and the run goes on in the new file (compact now reopens). A dropped keyframe's pages may still be shared by a later one, so a round can free less than it counted: the next round lowers the count by the overshoot, four rounds at most. Never dropped: the present (the newest keyframe and the batches from its own on), the keyframe of the newest mark of each name, the keyframe of the newest clip of each name and every batch holding one of its frames. A session left with nothing loses its keyframes, batches and stretches; its summary stays. Asked for, what was dropped is error.HistoryDropped, which the console says in words ("That part of the history is gone: a save keeps 64 MiB, the newest."). Marks are pinned under the cap: a mark of a new name needs room for a whole State, history trimmed to make it; without, it is refused, error.SaveFull, the console saying how to make room: save over a mark's name (the older one then goes with history) or start another save. A mark over an existing name is always taken. (The spec's "room" rule, refusing only after a trim failed, let one mark take the file past the cap; room for a whole State keeps the cap hard, at the price of refusing early for a big State.) Builds that do not record (`history` false, from Info.record) use the same mechanism with the history limit zero: every commit holding a keyframe drops all it may (no compaction unless past the cap), and the exit compacts, so the disk holds at most the last minute of input while it runs and the present and the marks after. The exit compaction a recording build made past twice its contents plus 64 MiB is gone: the cap's compaction does that work. Measured (Debug, unit test): a trim of a 64 MiB file to 48 MiB takes 165-187 ms (two rounds of drop and compaction, the first ending 0.5% over); a no-history trim at a keyframe, 0-5 ms. ReleaseFast not measured. The console's `marks` reads each mark's keyframe fields, not its State: an older mark of a name may have gone with history (listed "dropped" when its session went whole). Docs: record means "keep history" in App.Options, sketch.Build and abi.Info. Tests: zig build test, 168/168 steps, 137/137 tests. New, in Sessions.zig: 100 keyframes of 1 MiB each with marks at the 5th and 20th: the file never past the cap, the oldest keyframes and frames HistoryDropped, both marks and the newest keyframe whole, the present's frames readable; without history, 20 keyframes and a mark: only the mark and the present stay, the file compacted at exit under three times the State and the present reopening whole; 2 MiB marks until one is refused (SaveFull, the file under the cap), then one over an old name taken and the file under the cap, every mark whole. micscope 67/67, 10/10; spheres 75/75, 3/3; noc e2e 292/292; the drat-meout copy 87/87, 30/30. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* The present lives in the save: one file a sketch, the state file goneGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A sketch kept its live State in a mapped state file and its history in a sessions database beside it. Now the database is the save, the one file: the present is its newest keyframe and the frames logged after it, and a run starts there (Sessions.present: the newest session with a keyframe, its last keyframe loaded and the frames after it read). Host restores it (migrating across a layout change, as a seek does) and replays the frames through `update`, unlogged, `started` as it was fed (true only from a session's first frame); played by another build, they are not replayed, and the run says how many frames that was. The run's first keyframe then starts from the present's pages, so a session costs what changed since the last one. --fresh still starts from defaults, the save untouched. State is process memory (memory.State: one page-aligned allocation, its layout, and its layout header for the stretches); a reload's adopt allocates the new layout's defaults and migrates the old bytes into it, and returns null when unchanged. Gone: StateFile, its slots, used bits, resize, dropSnapshots and the adopt that rewrote a file's header; the sketch-facing `Host.state` keeps `layout`; Timeline restores into `state.live()`. `adopt` no longer runs `init`: openPresent does when there is no present. Every run saves: the database is opened in a file for every run but a scripted one from fresh state (memory, nothing on disk) and an export (none). So release builds keep their State as before, and marks and clips work there; `.record` keeps or trims history (next change). The save is `<stem>.xitdb`, `<stem>` being --state without `.xitdb` or `.state`: `--state x.state` and `--state x.xitdb` name the same save. Defaults: `.sketchbook/<name>.xitdb` under zig build run, and `$XDG_DATA_HOME/<name>/<name>.xitdb`. A sketch's appended files are `<stem>.<name>` (micscope's takes: `.sketchbook/micscope.take-3.f32`, where Phase 1 wrote `micscope.state.take-3.f32`; that naming never left this workspace, so nothing of the user's is renamed). A scripted run from fresh state or an export appends to `<stem>.<pid>.<name>` and removes those at exit; the private state file and the exit's moving of files by prefix are gone from root.zig. Old state files: in a new save, `<stem>.state` (one slot or eight, any of the three formats) is imported once: slot 0 becomes the present (in today's layout), slots 1-7 marks "1".."7" (migrated to it, as before); the file is read through a read-only mapping and left exactly as it was. It is never read again: the save then has a session. A file that is no state file is said and left. .sketchbook/'s move of an older build's `<stem>.state` from beside it stays, now by stem. sketchbook-probe is deleted (86 lines, its build step, Sketchbook's probe_exe, App.addProbe): it mapped the state file. Reading the save instead would show State as of the newest keyframe, up to a minute old, never the frames after it, which only `update` can turn into State; and it would need a read-only open the host has no other use for. The planned 9P state/ tree reads live State from the process. A kill -9 now loses what the session log always lost: the frames not yet committed (under a second: in test-kill 15 of 255 frames, the next run starting from frame 240's State), keyframes and marks queued since the last commit (a mark commits at once, synced), the session's end time and an open clip's end. The mapped file lost nothing to a kill. A power loss reopens at the last commit the disk holds whole (`mend`); commits are fsynced only for marks and at exit, so what the OS had not written back is lost (not measured); the mapped file, never synced, could tear page by page. Measured: drat-meout's State, 20 extra jump keyframes in a 2,400-frame script, 100,708 B more than the same frames without them: about 5 KB a keyframe, where the whole State (17.7-20 KiB) was written each time. Tests: zig build test, 168/168 steps. test-kill runs for a recording and a non-recording build, and the second run checks `updates` is 240 (replayed), not only the session's frame count. test-paths: the XDG directory holds `replay.xitdb` alone; .sketchbook/ holds `.gitignore` and `replay.xitdb`; the next run's State is the one the last left; an older `old.state` beside .sketchbook/ is moved in once, never over a file, and left as it was when it is no state file. test-same: a run from fresh state leaves no file at all. Rewritten: memory.zig's state file tests as State's (migrate, unchanged) and OldState's (refused untouched; drat-meout's v1 fixture imports every value, the file unchanged), Host's snapshot import as importOld (State live, two marks, the file byte for byte as it was), Host's setup test without a file, the timeline test on a State. micscope 67/67, 10/10; spheres 75/75, 3/3; noc e2e 292/292; the drat-meout copy 87/87, 30/30. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Keyframes share pages: each writes only the pages of State that changedGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A keyframe was a whole copy of State, so a 4 MiB State edited once a minute wrote 4 MiB a minute. Now a keyframe stores State as `pages`, an xitdb list of 4 KiB blobs (the last one shorter), with its `size`: it starts from the list of the keyframe before it (its slot written, and the cursor frozen, so a keyframe queued in the same commit is copied, not changed), and puts only the pages whose Wyhash differs from that keyframe's, which Sessions keeps in memory (`hashes`, one u64 a page). An all-zero page is an empty slot: no bytes. `written` records the bytes of pages a keyframe wrote, each page being written by exactly one keyframe, so the live size the exit compaction weighs is the batches plus every keyframe's `written`. A State unchanged since the last keyframe is still skipped (but before a reload), now by its page hashes. Keyframes from before pages keep their whole `state` bytes, and `load` reads both (`Loaded.paged` says which). Export and import shared pages once: `copy` remembers each blob's place in the source and the slot it was copied to, and writes that slot again for a page another keyframe already brought (xitdb takes slot writes). Measured (Debug, unit tests): a 4 MiB State, one u64 edited a minute, grows the database 144-153 KB a minute, 86 KB of it the minute's 3,600 ticks; a keyframe alone in a commit is 10,160 B with one changed page and 1,376 B with none (it was 4 MiB either way). A 1 MiB session of ten keyframes a byte apart exports to 1,133,486 B (it was over 10 MiB). The hash pass over State, best of five on a loaded machine: 2.2, 36 and 71 ms at 1, 16 and 32 MiB in Debug; 0.04, 1.3 and 2.9 ms in ReleaseFast. Not done (the spec's later fast paths, nothing needs them yet): restore copying pages without defaults and migration for an unchanged layout, the divergence check comparing pages first, formatPath skipping leaves. Tests: zig build test, 166/166 steps, 134/134 tests. New: a 4 MiB State edited each minute for ten minutes, under 230 KB a minute and every keyframe reading back with exactly the edits made before it; a lone keyframe under 12 KiB with one changed page and under 8 KiB with none; an exported ten-keyframe session under 1 MiB and an eighth, reading back. Changed: the compaction test's live size counts five 8-byte keyframes, not six (the first State is all zeros, an empty page). test-kill, test-past, test-import, test-same and test-paths unchanged. micscope 67/67 steps; spheres 75/75, 3/3 tests; noc e2e 292/292; the drat-meout copy 87/87 steps, 30/30 tests. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Bound a scripted run's memory sessions; check a script records the same twiceGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | A scripted run from fresh state records in memory, and a very long one grew without end. It stops at 256 MiB (ten hours or more of play at 60 Hz; a write the memory database cannot make, out of memory included, is the same): once, a warning, and no more frames, while what it holds still reads (marks, clips, past sessions). test-same plays test/past.script twice into two databases and compares them value by value with test/same_sessions.zig, but for a session's started and ended (the wall clock): every frame, keyframe, stretch and mark follows from the script alone. A third run from fresh state leaves its state file and no database. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Mend a sessions database a crash tore, compact one at exitGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | | A database whose last commit a power loss tore (its root header written, its bytes not all) was refused and renamed aside, a new one begun. Now it is opened at the last commit before the tear whose trailer verifies and whose sessions read whole, scanned back from the end, as pardes's state file is; renaming it to <path>.broken.<seconds> stays the last resort, when no commit is whole, and nothing is deleted. Every commit leaves the nodes it copied behind, about a kilobyte a second of play: a clean exit compacts the file (into <path>.new, synced, renamed over it) once it is 64 MiB and past twice what its batches and keyframes hold plus 64 MiB. Tests: a database of two sessions, cut at six places inside its last commit, opens at the commit before with every frame of both; one with no whole commit is kept aside byte for byte; a compacted one is smaller and holds every frame and keyframe. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Export a session to a file of its own, and import oneGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | export-session [S] FILE writes session S (this run's by default, committed first), with the marks and clips that point into it, as session 0 of a new database: written beside FILE, synced, renamed over it. import FILE appends each session of such a file to this database, from "import <file name>", its marks and clips renamed NAME@<file name> (cut to 32 bytes). The copy is value by value, the same few lines for both, over whatever a session holds; sessions lists where one came from. test-import records test/past.script's session, exports it, imports it into another database and replays it there: the mark it carries is the State half a second in exactly, and two seconds in is frame 120's. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Play past sessions back, re-synced at their keyframesGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | play session S [at SECONDS] restores the session's keyframe before that time (a binary search over its keyframes' times), replays its frames up to it and leaves it paused there; without a time it plays from the start. At each later keyframe the replay reaches, the State is compared with it over the leaves migration kept: a keyframe taken of what update made (periodic, mark, before a reload, exit) that differs is reported once, "diverged before frame F: N values, first <field>", and every keyframe is restored, so a replay across a jump the log does not hold (set, load, a sketch command) goes on from the State that followed it. A session built from another source or build says so. The console's status line names the session, frame and time, the mark, the clip and where it records. test-past records a session, replays it by the same build (the mark's State exactly, frame 120's two seconds in, no divergence), then by a build with an edited fixture (one divergence, before the mark). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Make snapshots named marks and recordings named clips of sessionsGabriel Schneider3 hours
| | | | | | | | | | | | | | | | | | | | | The state file keeps live state only (SKETCH03, one slot). save NAME is a keyframe and a mark in the sessions database, synced to disk; load, diff, snapshot and expect-snapshot take mark names, through the keyframe's own layout migrated to today's, compared over the leaves migration kept. A file from before (SKETCH01/02) has its used snapshot slots imported as marks "1".."7" at open, then drops them; a run that records nothing leaves them alone. record NAME, stop and play NAME make and play clips: a frame range of a session and its start keyframe, nothing overwritten, nothing invalidated by a later save, no capacity. The 64 MiB .inputs file, its --fresh memset and Timeline's Bank are gone; numbers still work as names, so every existing script runs unchanged. Event times in a clip are the session's, not restarted at record. reload.script also checks that a field a reload drops is still in the keyframe taken before it (expect-key). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
* Record every run's frames to a sessions database in xitdbGabriel Schneider3 hours
Live play was recorded only after someone typed record N, into 16 fixed slots. Now every run is a session in one xitdb database per sketch, beside its state file (<name>.xitdb): an append-only log of the frames live update was fed, one commit a second of session time, and keyframes of the State at start, every minute, on jumps, before and after reloads and at exit. A kill -9 loses at most the uncommitted second (test-kill). Whether a build records travels in the sketch: App.Options.record, null meaning Debug, reaching the host as Info.record. Scripted runs from fresh state record in memory, exports not at all. Each stretch of a session keeps its build, layout and Info.source: the repository revision (one jj or git call per build, build_info), zig version, backend and mode. Sessions.zig takes plain data (bytes, events, strings, a path, an Io), so a second host can drive it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T