diff options
| author | Gabriel Schneider <[email protected]> | 2026-11-17 01:07:16 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-11-17 12:15:01 -0300 |
| commit | 62cce7f3d96aa054b2bc2055dce86dd8e4205256 (patch) | |
| tree | 55a8e7d748d51809b9154d0d6d39ef6e6b662bb1 /src/host/control.zig | |
| parent | 70be2e0776d2e61a2d018b264ef5602e7e2c7f86 (diff) | |
| download | sketchbook-62cce7f3d96aa054b2bc2055dce86dd8e4205256.tar.gz sketchbook-62cce7f3d96aa054b2bc2055dce86dd8e4205256.zip | |
Keyframes share pages: each writes only the pages of State that changed
A keyframe was a whole copy of State, so a 4 MiB State edited once a
minute wrote 4 MiB a minute. Now a keyframe stores State as `pages`, an
xitdb list of 4 KiB blobs (the last one shorter), with its `size`: it
starts from the list of the keyframe before it (its slot written, and
the cursor frozen, so a keyframe queued in the same commit is copied,
not changed), and puts only the pages whose Wyhash differs from that
keyframe's, which Sessions keeps in memory (`hashes`, one u64 a page).
An all-zero page is an empty slot: no bytes. `written` records the
bytes of pages a keyframe wrote, each page being written by exactly one
keyframe, so the live size the exit compaction weighs is the batches
plus every keyframe's `written`. A State unchanged since the last
keyframe is still skipped (but before a reload), now by its page hashes.
Keyframes from before pages keep their whole `state` bytes, and `load`
reads both (`Loaded.paged` says which).
Export and import shared pages once: `copy` remembers each blob's place
in the source and the slot it was copied to, and writes that slot again
for a page another keyframe already brought (xitdb takes slot writes).
Measured (Debug, unit tests): a 4 MiB State, one u64 edited a minute,
grows the database 144-153 KB a minute, 86 KB of it the minute's 3,600
ticks; a keyframe alone in a commit is 10,160 B with one changed page
and 1,376 B with none (it was 4 MiB either way). A 1 MiB session of ten
keyframes a byte apart exports to 1,133,486 B (it was over 10 MiB). The
hash pass over State, best of five on a loaded machine: 2.2, 36 and 71
ms at 1, 16 and 32 MiB in Debug; 0.04, 1.3 and 2.9 ms in ReleaseFast.
Not done (the spec's later fast paths, nothing needs them yet): restore
copying pages without defaults and migration for an unchanged layout,
the divergence check comparing pages first, formatPath skipping leaves.
Tests: zig build test, 166/166 steps, 134/134 tests. New: a 4 MiB State
edited each minute for ten minutes, under 230 KB a minute and every
keyframe reading back with exactly the edits made before it; a lone
keyframe under 12 KiB with one changed page and under 8 KiB with none;
an exported ten-keyframe session under 1 MiB and an eighth, reading
back. Changed: the compaction test's live size counts five 8-byte
keyframes, not six (the first State is all zeros, an empty page).
test-kill, test-past, test-import, test-same and test-paths unchanged.
micscope 67/67 steps; spheres 75/75, 3/3 tests; noc e2e 292/292;
the drat-meout copy 87/87 steps, 30/30 tests.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
Diffstat (limited to 'src/host/control.zig')
0 files changed, 0 insertions, 0 deletions
