From c7ebfb6d18c44fec676da3d683e0b4e54001bd21 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Mon, 16 Nov 2026 23:52:52 -0300 Subject: README: sessions, marks and clips, where a sketch's files live The host section said what the 7 snapshot slots, the 16 recordings and the .inputs file did. It now says what the code does: the state file holds live State only; where files go (.sketchbook/ under zig build run, the XDG data directory for a shipped game, sketch.state for a script or an export); a Sessions bullet (what is logged and keyframed, who records by App.Options.record, memory for scripts, the commit cadence and what a crash or a power loss loses, mend and the .broken last resort, measured sizes and compaction, export-session and import, and what a playtest file holds of the player); time travel over marks, clips and past sessions; the console, and scripts' expect-snapshot NAME, expect-key and kill. The intro's table, input.jumped, Data that grows, the test list, Build API's addRun and .record, and the web's known limits follow. The hot-reload bullet and the Animation section are other work's, untouched. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T --- README.md | 168 +++++++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 127 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index 626a4e4..6c123a2 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ An app is an ordinary module. Every declaration is optional: | Declaration | Meaning | | --- | --- | | `name`, `width`, `height` | Window title and initial canvas | -| `State` | Saved to a file, snapshotted, replayed: what `update` changes. No pointers and no host handles (compile errors). | +| `State` | Saved to a file, kept in every session's keyframes and marks, replayed: what `update` changes. No pointers and no host handles (compile errors). | | `Session` | This process: survives reloads, never saved. Caches, image and buffer handles, playback cursors. | | `Store` | On disk, at the same address in every run: survives reloads and restarts. Big data that is not replayed (a recording, a document). May point into itself; no host handles. | | `init(state, host)` | Seed brand-new state, after the declared defaults | @@ -116,7 +116,7 @@ can replay differently. `input.started` is true on the first update after the program starts, never after a reload: state is saved, so this is where to stop what should not resume on its own. In `draw`, `input.jumped` says State got where it is other than by `update` from what the last `draw` -saw: the first frame, a seek, step, snapshot load, `set`, `reset`, play or +saw: the first frame, a seek, step, a mark's `load`, `set`, `reset`, play or record start, a sketch command, a reload. Anything `draw` piles up in Session (a trail in an offscreen image) is stale then; repaint it from State. @@ -363,9 +363,9 @@ display's rate it steps once a frame, jitter and all: a 60 Hz frame is after a reload that shrinks it. The default buffer is the element's own defaults, never `std.mem.zeroes` (an enum may have no 0). -**Data that grows.** State is snapshotted, replayed and migrated, so it -holds what `update` changes, at a size fixed in its type. Store is not -snapshotted or replayed, so what is in it does not follow a seek. Three +**Data that grows.** State is kept in keyframes, replayed and migrated, so +it holds what `update` changes, at a size fixed in its type. Store is not +kept in keyframes or replayed, so what is in it does not follow a seek. Three cases, by who needs the data: - `update` needs it: a `List` in State, sized for the worst case (the @@ -598,38 +598,116 @@ object. - **Hot reload.** It watches the library file and loads each new build through a symlink with a fresh name (a build must replace the file, as `zig build` does, not rewrite it in place). -- **State file.** `.state` holds the live state plus 7 snapshot slots, - shared via `MAP_SHARED`. Its header carries the reflected layout, so a - layout change migrates every slot field by field, with enums matched by tag - name, and `sketchbook-probe` can print any sketch's state. Arrays and - vectors migrate element by element: the elements both layouts have, each - field by field, so an element type can gain a field and a capacity can - grow (new elements take their defaults) or shrink (the rest is dropped; a - count field such as `sk.List`'s `len` is kept, and `List` clamps it where - it is read). An array whose type is unchanged is copied whole. Files from - before element leaves (`SKETCH01`) are read and rewritten; a file that - is not a state file is refused, never truncated. +- **State file.** `.state` holds the live State, shared via + `MAP_SHARED`, so the file is up to date as it changes. Its header carries + the reflected layout, so a layout change migrates it field by field, with + enums matched by tag name, and `sketchbook-probe` can print any sketch's + state. Arrays and vectors migrate element by element: the elements both + layouts have, each field by field, so an element type can gain a field + and a capacity can grow (new elements take their defaults) or shrink (the + rest is dropped; a count field such as `sk.List`'s `len` is kept, and + `List` clamps it where it is read). An array whose type is unchanged is + copied whole. Files from before element leaves (`SKETCH01`) are read and + rewritten, and the snapshot slots of files from before sessions become + marks `1` to `7` in a run that records; a file that is not a state file + is refused, never truncated. - **Store file.** `.state.store` is a heap mapped at a fixed address (`0x2000_0000_0000`, 64 GiB reserved), grown in place, locked to one - process. Power-of-two size classes with free lists in the file. Snapshots - and replays do not cover it (only `draw` and `command` change it). -- **Time travel.** Recordings, replay, seek and step work on any sketch, - because they only need state bytes and events. A recording is whole - frames; playback, `seek` and `step N` (frames) stop only between them. - Restoring a snapshot (play, seek, step, `load`) clears the pointer, - button and key state, and a recording begins with events that bring back - what was held when it started (a pointer move, a button or key down), so - a replay starts from the same input whatever the live pointer did since. - A recording carries an identity of the build that made it (its file's - inode, size and modification time); `play` warns when the running build - differs (edited `update`, a new compiler), since the replay may then - diverge. + process. Power-of-two size classes with free lists in the file. + Keyframes, marks and replays do not cover it (only `draw` and `command` + change it). +- **Where files go.** `zig build run`, `probe` and `export` keep a sketch's + files in `.sketchbook/`: `.state`, `.state.store` and + `.xitdb`. The host makes the directory with a `.gitignore` of `*`, + so neither git nor jj takes in a database that grows (the jj snapshot a + build makes would commit one while it is under a megabyte), and moves in, + once, the state and store an older build kept in the project's root; + when both exist, nothing moves and the host says which is used. Without + `--state`, a run keeps them in `$XDG_DATA_HOME//` + (`~/.local/share//` without it), which is where a shipped game's + go; a script or an export uses `sketch.state` in the working directory. +- **Sessions.** Every run of a build that records is a session in + `.xitdb`, one xitdb database per sketch beside its state file: an + append-only log of the events live `update` was fed, in whole frames, and + keyframes, the State at the start, every minute while it changes, at + every jump (`load`, `set`, `reset`, a seek, a sketch command), before and + after a reload, at every mark and at exit. Each stretch of a session + between reloads keeps its build's identity, its layout and the source it + was built from: the repository's revision (`jj: `, else + `git:`, `+dirty` with uncommitted changes, read once per `zig + build`), Zig's version, the backend and the mode. Nothing in it is ever + overwritten; marks and clips are names pointing into it. + + `App.Options.record` decides who records, at build time: null, the + default, means Debug builds. A playtest build of a release opts in with + one line, `.record = b.option(bool, "record", "collect sessions in a + release build")`. A script from `--fresh` state records in memory (time + travel works, nothing touches the disk; it stops at 256 MiB, ten hours or + more of play, keeping what it has), a script without `--fresh` to the + file, an export nothing. A build that does not record has no marks or + clips, and the console says so. The console's status line ends with + `rec file`, `rec memory` or `rec off`, and `sessions` says where. + + A commit is a second of play (of event time, so a script commits on the + same frames every run), and is made at once for a reload, a mark, a + clip's start or end, an export and exit; only a mark and exit are synced + to disk. A crash (a panic, `kill -9`) loses the frames not yet committed, + under a second: the session lists as `(crashed)`, and a clip it left + open ends where the session does. A power loss loses what the kernel had + not written back (30 s past the last sync, by default); the torn end is + mended at the next start, the file opened at its last whole commit, and + a file still unreadable is renamed to `.broken.`, never + deleted, and a new one begun. Live State is in the state file either way. + + A minute of drat-meout's play is 268 KB with the pointer moving every + frame (16 MB an hour), 164 KB idle (86 KB of ticks, the rest keyframes). + Every commit leaves the nodes it copied behind, about a kilobyte; a clean + exit compacts the file once it is past twice what its sessions hold plus + 64 MiB. Nothing is compressed or forgotten yet. + + Playtests: `export-session [S] FILE` writes session S (this run's when + none is given) with its marks and clips to a database of its own, for a + player to send; `import FILE` adds the sessions in such a file to this + sketch's (`sessions` lists them as `import `), its marks and clips + named `NAME@`; `play session N` replays one, and its stretch names + the commit to build to replay it without divergence. What such a file holds + of the player: every input event (pointer positions, keys, the text typed + into the sketch), the console lines that changed State (an `f.openFile` + answer is a local path), and the date the session started; nothing about + the machine or the user. +- **Time travel.** Marks, clips, replay, seek and step work on any sketch, + because they only need state bytes and events. `save NAME` marks the + State now: a keyframe with a name, the newest of a name winning. `load + NAME` (or `load #K`, the K-th in `marks`) brings it back through the + layout it was taken in, migrated to today's, and `diff NAME` compares it + with live leaf by leaf over the fields both layouts have. `record NAME` + starts a clip from the mark `snapshot NAME` chose (saved first if it is + new, restored if it exists, so several clips branch from the same bytes) + and `stop` ends it; `play NAME` restores its keyframe and replays its + frames; `pause`, `seek start|end|MS` and `step N` (frames) move within + it, stopping only between frames. `play session S [at SECONDS]` replays a + past session from its start, or from the keyframe before that time, + paused there. At each later keyframe `update` produced, the replay is + compared with what was recorded: a difference is said once, `diverged + before frame F: N values, first `, and the replay goes on from the + recorded State, which also carries it over the jumps the log does not + hold. A clip or session made by another build says so, since the replay + may then differ. Going back and playing on is `play` or `seek`, then + `stop`: live goes on from there in the same session, nothing + overwritten. Names are words of up to 32 bytes, and numbers are names, so + `save 3` and `play 0` work as they always have. Restoring State (play, + seek, step, `load`) clears the pointer, button and key state, and a clip + begins with events that bring back what was held when it started (a + pointer move, a button or key down), so a replay starts from the same + input whatever the live pointer did since. Event times are the session's: + `in.now` does not restart at 0 in a clip. - **Console and overlays.** `get`/`set ` (`get` of a vector, - array or struct shows its values, in memory order), snapshots, recordings and - the memory, scope and frame timing overlays come from the host; unknown lines go to the - sketch. Paths index arrays and vector lanes: `get pos[0]`, - `set boids.buffer[3].pos[1] 20`. `diff N` compares live state with - snapshot N leaf by leaf, so padding (a Debug `@Vector`'s spare lane) + array or struct shows its values, in memory order), marks, clips, + sessions (`marks`, `clips` and `sessions` list the last few) and the + memory, scope and frame timing overlays come from the host; unknown lines + go to the sketch. Paths index arrays and vector lanes: `get pos[0]`, + `set boids.buffer[3].pos[1] 20`. `diff NAME` compares live state with + mark NAME leaf by leaf, so padding (a Debug `@Vector`'s spare lane) never counts. - **Scripts** (`--headless --fresh --script file`) drive the same routing and frames as a player: `click`, `key`, `hold`/`release` (a key down until @@ -639,18 +717,22 @@ object. (`expect pos[0] 250 ~0.5` within a tolerance; a failure prints what it found), `await field value [~tol]` (frames until it holds, a minute at most), - `expect-snapshot N` (live state equals snapshot N, leaf by leaf, printing - the first differences), `expect-log`/`expect-no-log text` (the console's + `expect-snapshot NAME` (live state equals mark NAME, leaf by leaf, + printing the first differences), `expect-key WHY field value` (the + newest keyframe of this session taken for WHY, `before_reload`, `mark`, + `exit`..., holds the value, read in its own layout), + `expect-log`/`expect-no-log text` (the console's recent lines), `expect-audio`, `screenshot`, `expect-same-screenshot a b [max [x y w h]]`, `expect-different-screenshot a b [x y w h]`, `expect-pixel file x y r g b - a [tol]`, `load`, `reload`, `await-reload`. A failed + a [tol]`, `load`, `reload`, `await-reload`, `kill` (death by SIGKILL, + for tests of what a crash keeps). A failed `expect`, or a console line that errors (a sketch command returning an error included), fails the script. Headless always means offscreen, even with `SDL_VIDEO_DRIVER` set, and GPU work still runs, into a texture. A scripted `--fresh` run works on files of its own (`.`), renamed to the given `--state` at exit, so two builds running the same - script at once do not share them. + script at once do not share them; its sessions are in memory. - **Export.** `--export DIR` plays the sketch headless, from fresh state, and saves every frame as `DIR/000000.png`, `000001.png`, ...; `--export -` writes them to stdout instead, as raw RGBA: @@ -750,7 +832,7 @@ and returns plain `std.Build` objects: | `app.addScriptExecutable()` | the same, built in `script_optimize` mode, as `e2e` plays scripts with | | `app.addWeb(.{ .assets = ... })` | the wasm module and a ready-to-serve directory, built from the app's modules with every shader module swapped for its WGSL twin; tests can read its state by path (indices too: `list[2].x`) with `sk_field` (numbers) and `sk_tag` (enum tag names) | | `app.addServe(install)` | `miniserve` serving an installed web directory over https, on every interface, with a self-signed certificate made once by `openssl`: https makes the page a secure context on any device, which WebGPU and the microphone need. Its defaults are environment variables, so miniserve options after `--` replace them | -| `app.addRun(lib, .{})` | the generic host running the installed library | +| `app.addRun(lib, .{})` | the generic host running the installed library, its files in `.sketchbook/` | | `app.addScript(exe_or_lib, file, .{})` | a headless, fresh-state scripted run, of the executable or of the library through the host (so the script can `reload` it) | | `app.addExport(exe, .{ .fps, .frames, .size })` | a headless export of the executable's frames, as PNGs in `.frames`, a directory of the cache | | `app.addProbe(state)` | the probe on a state file | @@ -763,6 +845,9 @@ for sketches whose per-pixel loops crawl in Debug; null (the default) is `optimize`. Only Zig code changes mode (the host's and the app's), so it costs one more compile, not SDL's. A `b.option` makes it a `-D` flag. +`sb.app(.{ ..., .record = true })` records sessions in every mode (see +The host); null, the default, records in Debug builds only. + **Hot reload, checked.** `addReloadCheck` builds the app's library as written, and again from a copy of its sources with textual edits applied, then plays a script on the first through the generic host with `{v2}` @@ -857,7 +942,8 @@ Add `.sketchbook = .{ .path = "..." }` to your `build.zig.zon`. antialiased natively only inside a target. Without a GPU renderer, native draws aliased. A stroke that overlaps itself blends twice natively where it overlaps, once on the web. -- On the web: nothing persists (Store lasts as long as the page), +- On the web: nothing persists (Store lasts as long as the page) and no + sessions are recorded, synthesised voices are silent, `loadSound` knows only sounds the page decoded, multisampling is 4× at most, and mixing runs on the main thread (a ScriptProcessor), so a busy page can crackle. @@ -891,7 +977,7 @@ never unloaded. Only the recorded `Event` and the reflected `Layout` are `zig build test` in this directory runs the unit tests, a headless reload through `dlopen`, where a counter sketch changes its layout mid-run and keeps -its values, and a replay test (`test/replay.script`): a recording played, +its values, and a replay test (`test/replay.script`): a clip played, sought, stepped and replayed after a live pointer move, each ending on the state it was stopped at. -- cgit v1.3