summaryrefslogtreecommitdiff
path: root/src/host
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-11-17 01:13:36 -0300
committerGabriel Schneider <[email protected]>2026-11-17 12:15:01 -0300
commit1ce323f2f21b94f9c4fc4e7a6986612cf30cf183 (patch)
tree87342ff0ca8bf9e08ba8060ffddd6a64bfca7adf /src/host
parent62cce7f3d96aa054b2bc2055dce86dd8e4205256 (diff)
downloadsketchbook-1ce323f2f21b94f9c4fc4e7a6986612cf30cf183.tar.gz
sketchbook-1ce323f2f21b94f9c4fc4e7a6986612cf30cf183.zip
The present lives in the save: one file a sketch, the state file gone
A sketch kept its live State in a mapped state file and its history in a sessions database beside it. Now the database is the save, the one file: the present is its newest keyframe and the frames logged after it, and a run starts there (Sessions.present: the newest session with a keyframe, its last keyframe loaded and the frames after it read). Host restores it (migrating across a layout change, as a seek does) and replays the frames through `update`, unlogged, `started` as it was fed (true only from a session's first frame); played by another build, they are not replayed, and the run says how many frames that was. The run's first keyframe then starts from the present's pages, so a session costs what changed since the last one. --fresh still starts from defaults, the save untouched. State is process memory (memory.State: one page-aligned allocation, its layout, and its layout header for the stretches); a reload's adopt allocates the new layout's defaults and migrates the old bytes into it, and returns null when unchanged. Gone: StateFile, its slots, used bits, resize, dropSnapshots and the adopt that rewrote a file's header; the sketch-facing `Host.state` keeps `layout`; Timeline restores into `state.live()`. `adopt` no longer runs `init`: openPresent does when there is no present. Every run saves: the database is opened in a file for every run but a scripted one from fresh state (memory, nothing on disk) and an export (none). So release builds keep their State as before, and marks and clips work there; `.record` keeps or trims history (next change). The save is `<stem>.xitdb`, `<stem>` being --state without `.xitdb` or `.state`: `--state x.state` and `--state x.xitdb` name the same save. Defaults: `.sketchbook/<name>.xitdb` under zig build run, and `$XDG_DATA_HOME/<name>/<name>.xitdb`. A sketch's appended files are `<stem>.<name>` (micscope's takes: `.sketchbook/micscope.take-3.f32`, where Phase 1 wrote `micscope.state.take-3.f32`; that naming never left this workspace, so nothing of the user's is renamed). A scripted run from fresh state or an export appends to `<stem>.<pid>.<name>` and removes those at exit; the private state file and the exit's moving of files by prefix are gone from root.zig. Old state files: in a new save, `<stem>.state` (one slot or eight, any of the three formats) is imported once: slot 0 becomes the present (in today's layout), slots 1-7 marks "1".."7" (migrated to it, as before); the file is read through a read-only mapping and left exactly as it was. It is never read again: the save then has a session. A file that is no state file is said and left. .sketchbook/'s move of an older build's `<stem>.state` from beside it stays, now by stem. sketchbook-probe is deleted (86 lines, its build step, Sketchbook's probe_exe, App.addProbe): it mapped the state file. Reading the save instead would show State as of the newest keyframe, up to a minute old, never the frames after it, which only `update` can turn into State; and it would need a read-only open the host has no other use for. The planned 9P state/ tree reads live State from the process. A kill -9 now loses what the session log always lost: the frames not yet committed (under a second: in test-kill 15 of 255 frames, the next run starting from frame 240's State), keyframes and marks queued since the last commit (a mark commits at once, synced), the session's end time and an open clip's end. The mapped file lost nothing to a kill. A power loss reopens at the last commit the disk holds whole (`mend`); commits are fsynced only for marks and at exit, so what the OS had not written back is lost (not measured); the mapped file, never synced, could tear page by page. Measured: drat-meout's State, 20 extra jump keyframes in a 2,400-frame script, 100,708 B more than the same frames without them: about 5 KB a keyframe, where the whole State (17.7-20 KiB) was written each time. Tests: zig build test, 168/168 steps. test-kill runs for a recording and a non-recording build, and the second run checks `updates` is 240 (replayed), not only the session's frame count. test-paths: the XDG directory holds `replay.xitdb` alone; .sketchbook/ holds `.gitignore` and `replay.xitdb`; the next run's State is the one the last left; an older `old.state` beside .sketchbook/ is moved in once, never over a file, and left as it was when it is no state file. test-same: a run from fresh state leaves no file at all. Rewritten: memory.zig's state file tests as State's (migrate, unchanged) and OldState's (refused untouched; drat-meout's v1 fixture imports every value, the file unchanged), Host's snapshot import as importOld (State live, two marks, the file byte for byte as it was), Host's setup test without a file, the timeline test on a State. micscope 67/67, 10/10; spheres 75/75, 3/3; noc e2e 292/292; the drat-meout copy 87/87, 30/30. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016Y9MNBECMWcJD5pdS4SR6T
Diffstat (limited to 'src/host')
-rw-r--r--src/host/Console.zig2
-rw-r--r--src/host/Host.zig214
-rw-r--r--src/host/Sessions.zig42
-rw-r--r--src/host/memory.zig303
-rw-r--r--src/host/probe.zig86
-rw-r--r--src/host/root.zig56
-rw-r--r--src/host/timeline.zig22
7 files changed, 295 insertions, 430 deletions
diff --git a/src/host/Console.zig b/src/host/Console.zig
index 75da732..ffa6876 100644
--- a/src/host/Console.zig
+++ b/src/host/Console.zig
@@ -350,7 +350,7 @@ fn listSessions(self: *Console, host: *Host) !void {
switch (s.mode()) {
.file => self.print("recording to {s}", .{s.path}),
.memory => self.log("recording in memory (scripted run)"),
- .off => return self.log("not recording (release build: .record = true collects)"),
+ .off => return self.log("not recording (an export)"),
}
const n = try s.count(.sessions);
var i = n -| 4;
diff --git a/src/host/Host.zig b/src/host/Host.zig
index 3a2a25f..3485d79 100644
--- a/src/host/Host.zig
+++ b/src/host/Host.zig
@@ -27,15 +27,17 @@ pub const Options = struct {
library: ?[]const u8 = null,
/// Local 9P control socket, opt-in.
ninep: ?[]const u8 = null,
- /// State file; the sessions database goes next to it, `<name>.xitdb`
- /// for `<name>.state`. Empty: `$XDG_DATA_HOME/<name>/<name>.state`
+ /// The save, a sessions database: `<stem>.xitdb`, for a path
+ /// `<stem>.xitdb`, `<stem>.state` (where a state file from before is
+ /// imported from) or `<stem>`. Empty: `$XDG_DATA_HOME/<name>/<name>.xitdb`
/// (`~/.local/share` without it), for a shipped game.
state: []const u8 = "",
/// Playing a script: from fresh state, sessions go to memory only.
script: bool = false,
/// Offscreen video, silent audio, no vsync: for scripts and CI.
headless: bool = false,
- /// Forget the state file's contents: scripts start from a known place.
+ /// Start from State's defaults, not the save's present: scripts start
+ /// from a known place.
fresh: bool = false,
/// Time every GPU command alone and print the times at exit.
profile_gpu: bool = false,
@@ -70,7 +72,7 @@ images: gfx.Images,
/// Plays the 2D commands: targets, blend, antialiased shapes.
painter: gfx.Painter,
audio: sound.Audio,
-state: memory.StateFile,
+state: memory.State,
session: memory.Session,
/// A failed resource setup disables app callbacks while keeping host controls alive.
session_ready: bool = false,
@@ -91,8 +93,11 @@ reloads: usize = 0,
/// file's inode, size and modification time): sessions' stretches carry it,
/// and `play` warns on a mismatch.
build: u64 = 0,
-/// The state file's path when the host chose it (`Options.state`), `gpa`'s.
+/// The save's path when the host chose it (`Options.state`), `gpa`'s.
state_path: []const u8 = "",
+/// `Options.state` without `.xitdb` or `.state`: the save is
+/// `<stem>.xitdb`, the sketch's own files `<stem>.<name>`.
+stem: []const u8 = "",
/// Copies of the sketch's info strings, replaced on every load.
info_arena: std.heap.ArenaAllocator,
name: []const u8 = "",
@@ -176,15 +181,13 @@ pub fn init(self: *Host, gpa: std.mem.Allocator, io: std.Io, environ: *const std
var data_buf: [std.fs.max_path_bytes]u8 = undefined;
const state_path = if (given.state.len > 0) "" else blk: {
const data = environ.get("XDG_DATA_HOME") orelse try std.fmt.bufPrint(&data_buf, "{s}/.local/share", .{environ.get("HOME") orelse "."});
- const path = try std.fmt.allocPrint(gpa, "{s}/{s}/{s}.state", .{ data, info.name, info.name });
+ const path = try std.fmt.allocPrint(gpa, "{s}/{s}/{s}.xitdb", .{ data, info.name, info.name });
errdefer gpa.free(path);
try std.Io.Dir.cwd().createDirPath(io, std.fs.path.dirname(path).?);
break :blk path;
};
errdefer gpa.free(state_path);
if (state_path.len > 0) options.state = state_path;
- var state = try memory.StateFile.open(gpa, options.state);
- errdefer state.close();
self.* = .{
.gpa = gpa,
@@ -197,13 +200,14 @@ pub fn init(self: *Host, gpa: std.mem.Allocator, io: std.Io, environ: *const std
.images = .{ .gpa = gpa, .renderer = renderer.? },
.painter = .{ .renderer = renderer.?, .images = undefined, .smooth = gpu != null },
.audio = .{ .gpa = gpa },
- .state = state,
+ .state = .init(gpa),
.session = .init(gpa),
.timeline = undefined,
.sessions = .{ .gpa = gpa, .io = io },
.sketch = sketch,
.library = library,
.state_path = state_path,
+ .stem = stemOf(options.state),
.info_arena = .init(gpa),
.frame_arena = .init(gpa),
.input = .{ .exporting = options.export_to != null },
@@ -224,10 +228,8 @@ pub fn init(self: *Host, gpa: std.mem.Allocator, io: std.Io, environ: *const std
self.build = fileId(io, options.library orelse "/proc/self/exe");
try self.adopt(sketch);
self.openSessions(info.record);
- // Before --fresh: it never touches the history.
- if (importSnapshots(&self.state, &self.sessions)) |n| {
- if (n > 0) self.print("{d} snapshots of {s} are marks now", .{ n, options.state });
- } else |err| self.print("snapshots of {s} kept in it: {t}", .{ options.state, err });
+ // Before --fresh, which never touches the save.
+ self.openPresent();
if (options.fresh) self.resetState();
if (!options.headless and options.export_to == null) {
self.presentation = Presentation.init(gpa, self.window) catch null;
@@ -258,14 +260,19 @@ pub fn deinit(self: *Host) void {
self.files.deinit(self.gpa);
for (self.retired.items) |bytes| std.posix.munmap(bytes);
self.retired.deinit(self.gpa);
+ // A scripted run from fresh state, or an export, leaves no files.
+ const private = (self.options.script or self.options.export_to != null) and self.options.fresh;
var written = self.written.keyIterator();
- while (written.next()) |path| self.gpa.free(path.ptr[0 .. path.len + 1]);
+ while (written.next()) |path| {
+ if (private) std.Io.Dir.cwd().deleteFile(self.io, path.*) catch {};
+ self.gpa.free(path.ptr[0 .. path.len + 1]);
+ }
self.written.deinit(self.gpa);
self.timeline.deinit();
self.session.deinit(self.gpa);
self.made.images.deinit(self.gpa);
self.made.buffers.deinit(self.gpa);
- self.state.close();
+ self.state.deinit(self.gpa);
self.pending.deinit(self.gpa);
self.frame_arena.deinit();
self.info_arena.deinit();
@@ -330,7 +337,7 @@ test "timing clock drives ordinary ticks without moving input or replay time" {
}
pub fn live(self: *Host) []u8 {
- return self.state.slot(0);
+ return self.state.live();
}
/// One frame: reload if a build replaced the library, feed input through
@@ -696,7 +703,7 @@ pub fn runLine(self: *Host, line: []const u8, now: u64) !void {
const replay = self.console.execute(self, line, now) catch |err| {
// The sketch has printed its own error already.
if (err == error.NotRecording) {
- self.console.log("Not recording (a release build: .record = true collects): no marks or clips.");
+ self.console.log("Not recording (an export): no marks or clips.");
} else if (err != error.CommandFailed) self.console.print("{t}. Type help for usage.", .{err});
if (self.timeline.restored) self.jumped = true;
try self.restoredInput(now);
@@ -738,7 +745,6 @@ pub fn resetState(self: *Host) void {
self.sketch.defaults(.state, bytes.ptr);
const ctx = self.context();
self.sketch.init(&ctx, bytes.ptr);
- self.state.markUsed(0, true);
}
/// Loads the watched library again.
@@ -757,14 +763,14 @@ pub fn load(self: *Host, path: []const u8) !void {
self.library = library;
self.reloads += 1;
self.build = fileId(self.io, path);
- self.sessions.stretch(.{ .build = self.build, .source = self.source, .layout = self.state.layoutBytes() }) catch |err| self.print("session: {t}", .{err});
+ self.sessions.stretch(.{ .build = self.build, .source = self.source, .layout = self.state.header }) catch |err| self.print("session: {t}", .{err});
self.print("reload {d}: {s}", .{ self.reloads, path });
}
fn adopt(self: *Host, sketch: abi.Sketch) !void {
const info = sketch.info();
if (info.abi != abi.hash) return error.AbiMismatch;
- const adoption = try self.state.adopt(self.gpa, info.state, sketch.defaults);
+ const migrated = try self.state.adopt(self.gpa, info.state, sketch.defaults);
const session_reset = try self.session.adopt(self.gpa, info.session, sketch.defaults);
if (session_reset) self.releaseMade();
_ = self.info_arena.reset(.retain_capacity);
@@ -777,12 +783,7 @@ fn adopt(self: *Host, sketch: abi.Sketch) !void {
self.sketch = sketch;
self.timeline.defaults = sketch.defaults;
self.reloaded = true;
- switch (adoption) {
- .unchanged => {},
- .created => {},
- .migrated => |m| self.print("state layout changed: kept {d} fields, reset {d} to defaults", .{ m.kept, m.reset }),
- }
- if (!self.state.used(0)) self.resetState();
+ if (migrated) |m| self.print("state layout changed: kept {d} fields, reset {d} to defaults", .{ m.kept, m.reset });
if (session_reset) {
const ctx = self.context();
self.session_ready = self.sketch.setup(&ctx);
@@ -932,42 +933,86 @@ fn consoleKey(e: abi.Event) ?Console.Key {
};
}
-/// Starts this run's session (Sessions.zig): in a file beside the state
-/// file when the build records, in memory for a script from fresh state,
-/// nowhere for an export. A database that cannot be opened is said, and
-/// the run goes on unrecorded.
+/// Opens the save (Sessions.zig) and starts this run's session in it: a
+/// file, `<stem>.xitdb`; in memory for a script from fresh state; nowhere
+/// for an export. A save that cannot be opened is said, and the run goes
+/// on unsaved.
fn openSessions(self: *Host, record: bool) void {
const o = self.options;
- const mode: Sessions.Mode = if (o.export_to != null) .off else if (o.script and o.fresh) .memory else if (record) .file else .off;
+ const mode: Sessions.Mode = if (o.export_to != null) .off else if (o.script and o.fresh) .memory else .file;
var buf: [std.fs.max_path_bytes]u8 = undefined;
- const stem = if (std.mem.endsWith(u8, o.state, ".state")) o.state[0 .. o.state.len - ".state".len] else o.state;
- const path = std.fmt.bufPrint(&buf, "{s}.xitdb", .{stem}) catch "sketch.xitdb";
- const first: Sessions.Stretch = .{ .build = self.build, .source = self.source, .layout = self.state.layoutBytes() };
+ const path = std.fmt.bufPrint(&buf, "{s}.xitdb", .{self.stem}) catch "sketch.xitdb";
+ const first: Sessions.Stretch = .{ .build = self.build, .source = self.source, .layout = self.state.header };
self.sessions = Sessions.open(self.gpa, self.io, mode, path, first, wallClock(self.io)) catch |err| blk: {
- self.print("not recording: {s}: {t}", .{ path, err });
+ self.print("not saved: {s}: {t}", .{ path, err });
break :blk .{ .gpa = self.gpa, .io = self.io };
};
- if (o.headless) return;
- switch (self.sessions.mode()) {
- .file => self.print("recording to {s}", .{path}),
- .memory => {},
- .off => if (o.export_to == null) self.print("not recording (release build: .record = true collects)", .{}),
- }
+ if (o.headless or self.sessions.mode() != .file) return;
+ self.print("{s} to {s}", .{ if (record) "recording" else "saving", path });
+}
+
+/// The State this run starts from: the save's present, its newest keyframe
+/// and the frames logged after it, replayed; in a new save, a state file
+/// from before (`<stem>.state`), imported once and left as it was; else
+/// `init`'s.
+fn openPresent(self: *Host) void {
+ var events: std.ArrayList(abi.Event) = .empty;
+ defer events.deinit(self.gpa);
+ const found = self.sessions.present(&events) catch |err| blk: {
+ self.print("the save's present did not read ({t}): starting from defaults", .{err});
+ break :blk null;
+ };
+ const key = found orelse {
+ if (self.sessions.session != 0 or self.sessions.mode() != .file) return self.resetState();
+ var buf: [std.fs.max_path_bytes]u8 = undefined;
+ const old = std.fmt.bufPrint(&buf, "{s}.state", .{self.stem}) catch return self.resetState();
+ const n = importOld(self.gpa, old, &self.state, &self.sessions, self.sketch.defaults) catch |err| {
+ if (err != error.FileNotFound) self.print("{s} not imported ({t}): kept as it was", .{ old, err });
+ return self.resetState();
+ };
+ return self.print("{s} imported: its State, and {d} snapshots as marks; the file is kept as it was", .{ old, n });
+ };
+ defer key.free(self.gpa);
+ self.timeline.restore(key) catch |err| {
+ self.print("the save's present did not restore ({t}): starting from defaults", .{err});
+ return self.resetState();
+ };
+ self.timeline.restored = false;
+ var frames: usize = 0;
+ for (events.items) |e| frames += @intFromBool(e.kind == .tick);
+ if (frames == 0) return;
+ if (key.build != self.build) return self.print("{d} frames played after the save's last keyframe not replayed: another build", .{frames});
+ // As they were fed: `started` only for a run's first frame.
+ self.started = key.frame == 0;
+ self.update(events.items);
+ self.started = true;
}
-/// Snapshots of a state file from before sessions become marks "1".."7",
-/// then the file holds live state only. While nothing records, they stay.
-fn importSnapshots(state: *memory.StateFile, sessions: *Sessions) !usize {
- if (state.slots == 1 or sessions.mode() == .off) return 0;
+/// A state file from before (`path`) into a new save: its live State into
+/// `state`, its snapshots (slots 1..7) marks "1".."7", in `state`'s layout;
+/// the file is left as it was. Returns the number of marks.
+fn importOld(gpa: std.mem.Allocator, path: []const u8, state: *memory.State, sessions: *Sessions, defaults: *const fn (abi.Memory, [*]u8) callconv(.c) void) !usize {
+ const old = try memory.OldState.open(path);
+ defer old.close();
+ var arena: std.heap.ArenaAllocator = .init(gpa);
+ defer arena.deinit();
+ const scratch = try arena.allocator().alignedAlloc(u8, .fromByteUnits(memory.page), state.layout.size);
var n: usize = 0;
- for (1..state.slots) |i| if (state.used(i)) {
- _ = try sessions.mark(&.{'0' + @as(u8, @intCast(i))}, .import, state.slot(i));
+ for (1..old.slots) |i| if (old.slot(i)) |image| {
+ _ = try memory.restore(arena.allocator(), old.layoutBytes(), image, state.layout, scratch, defaults);
+ _ = try sessions.mark(&.{'0' + @as(u8, @intCast(i))}, .import, scratch);
n += 1;
};
- try state.dropSnapshots();
+ _ = try memory.restore(arena.allocator(), old.layoutBytes(), old.slot(0) orelse return error.CorruptStateFile, state.layout, state.live(), defaults);
return n;
}
+/// `path` without `.xitdb` or `.state`.
+pub fn stemOf(path: []const u8) []const u8 {
+ for ([_][]const u8{ ".xitdb", ".state" }) |suffix| if (std.mem.endsWith(u8, path, suffix)) return path[0 .. path.len - suffix.len];
+ return path;
+}
+
fn wallClock(io: std.Io) u64 {
return @intCast(@max(0, std.Io.Timestamp.now(io, .real).nanoseconds));
}
@@ -1093,7 +1138,11 @@ fn apiAppendFile(ctx: *anyopaque, name_ptr: [*]const u8, name_len: usize, bytes:
return "";
}
var buf: [std.fs.max_path_bytes]u8 = undefined;
- const path = std.mem.print(&buf, "{s}.{s}", .{ self.options.state, name }) catch return "";
+ const o = self.options;
+ const private = (o.script or o.export_to != null) and o.fresh;
+ // A scripted run from fresh state, or an export, writes files of its own
+ // (two runs of one build step may share a directory), gone at exit.
+ const path = (if (private) std.mem.print(&buf, "{s}.{d}.{s}", .{ self.stem, std.os.linux.getpid(), name }) else std.mem.print(&buf, "{s}.{s}", .{ self.stem, name })) catch return "";
if (replace) {
// A new file: a mapping of the old one stays valid.
std.Io.Dir.cwd().deleteFile(self.io, path) catch {};
@@ -1194,22 +1243,11 @@ test "session setup runs for restored state and layout reset, not same-layout re
Fixture.init_calls = 0;
Fixture.fail_setup = false;
const a = std.testing.allocator;
- var tmp = std.testing.tmpDir(.{});
- defer tmp.cleanup();
- var state_buf: [std.fs.max_path_bytes]u8 = undefined;
- const state_path = try std.mem.print(&state_buf, ".zig-cache/tmp/{s}/setup.state", .{&tmp.sub_path});
- {
- var state = try memory.StateFile.open(a, state_path);
- defer state.close();
- _ = try state.adopt(a, Fixture.description.state, Fixture.defaults);
- @as(*Fixture.State, @ptrCast(state.slot(0).ptr)).value = 77;
- state.markUsed(0, true);
- }
- // Exercise adoption with real persisted memory, without opening SDL devices.
+ // Exercise adoption without opening SDL devices.
var host: Host = undefined;
host.gpa = a;
- host.state = try memory.StateFile.open(a, state_path);
- defer host.state.close();
+ host.state = .init(a);
+ defer host.state.deinit(a);
host.session = .init(a);
defer host.session.deinit(a);
host.info_arena = .init(a);
@@ -1224,7 +1262,8 @@ test "session setup runs for restored state and layout reset, not same-layout re
host.input = .{};
host.commands = .empty;
try host.adopt(Fixture.sketch);
- try std.testing.expectEqual(@as(u32, 77), @as(*Fixture.State, @ptrCast(@alignCast(host.live().ptr))).value);
+ // Defaults: the present or `init` comes after (openPresent).
+ try std.testing.expectEqual(@as(u32, 1), @as(*Fixture.State, @ptrCast(@alignCast(host.live().ptr))).value);
try std.testing.expectEqual(@as(usize, 0), Fixture.init_calls);
try std.testing.expectEqual(@as(usize, 1), Fixture.setup_calls);
const session_ptr = host.session.bytes.ptr;
@@ -1265,7 +1304,7 @@ test "setup writeBuffer propagates unavailable, unknown and out-of-bounds upload
try std.testing.expectError(error.UploadOutOfBounds, client.writeBuffer(@fromBackingInt(@intCast(1)), 2, "data"));
}
-test "a state file's snapshots from before sessions become marks, and the file one slot" {
+test "a state file from before imports into a new save: its State live, its snapshots marks, the file as it was" {
const State = extern struct { v: u32 = 0 };
const D = struct {
fn f(_: abi.Memory, p: [*]u8) callconv(.c) void {
@@ -1278,32 +1317,24 @@ test "a state file's snapshots from before sessions become marks, and the file o
defer tmp.cleanup();
var buf: [std.fs.max_path_bytes]u8 = undefined;
const path = try std.mem.print(&buf, ".zig-cache/tmp/{s}/old.state", .{&tmp.sub_path});
- // An eight-slot SKETCH02 file, slots 0, 1 and 3 used: built from a new
- // one-slot file's header.
- {
- var file = try memory.StateFile.open(gpa, path);
- defer file.close();
- _ = try file.adopt(gpa, abi.layout.of(State, .state), D.f);
- }
- const one = try std.Io.Dir.cwd().readFileAlloc(io, path, gpa, .unlimited);
- defer gpa.free(one);
+ var state: memory.State = .init(gpa);
+ defer state.deinit(gpa);
+ _ = try state.adopt(gpa, abi.layout.of(State, .state), D.f);
+ // An eight-slot SKETCH02 file, slots 0, 1 and 3 used, its header State's.
const stride = memory.page;
const old = try gpa.alloc(u8, memory.header_len + memory.old_slot_count * stride);
defer gpa.free(old);
@memset(old, 0);
- @memcpy(old[0..memory.header_len], one[0..memory.header_len]);
+ @memcpy(old[0..state.header.len], state.header);
const h: *memory.Header = @ptrCast(@alignCast(old.ptr));
h.magic = memory.file_magic_v2;
h.used = 0b1011;
for ([_][2]u32{ .{ 0, 5 }, .{ 1, 11 }, .{ 3, 33 } }) |slot| std.mem.writeInt(u32, old[memory.header_len + slot[0] * stride ..][0..4], slot[1], .little);
try std.Io.Dir.cwd().writeFile(io, .{ .sub_path = path, .data = old });
- var state = try memory.StateFile.open(gpa, path);
- defer state.close();
- try std.testing.expectEqual(memory.old_slot_count, state.slots);
- var sessions = try Sessions.open(gpa, io, .memory, "", .{ .build = 1, .source = "", .layout = state.layoutBytes() }, 0);
+ var sessions = try Sessions.open(gpa, io, .memory, "", .{ .build = 1, .source = "", .layout = state.header }, 0);
defer sessions.deinit();
- try std.testing.expectEqual(@as(usize, 2), try importSnapshots(&state, &sessions));
+ try std.testing.expectEqual(@as(usize, 2), try importOld(gpa, path, &state, &sessions, D.f));
for ([_][2]u32{ .{ '1', 11 }, .{ '3', 33 } }) |want| {
const m = (try sessions.findMark(&.{@intCast(want[0])})).?;
const key = try sessions.load(m.session, m.key);
@@ -1312,17 +1343,13 @@ test "a state file's snapshots from before sessions become marks, and the file o
try std.testing.expectEqual(want[1], std.mem.readInt(u32, key.state[0..4], .little));
}
try std.testing.expectEqual(@as(?Sessions.Entry, null), try sessions.findMark("2"));
- // One slot now, live state kept, and it reopens so.
- try std.testing.expectEqual(@as(usize, 1), state.slots);
- try std.testing.expectEqual(@as(u32, 5), std.mem.readInt(u32, state.slot(0)[0..4], .little));
- var again = try memory.StateFile.open(gpa, path);
- defer again.close();
- try std.testing.expectEqualStrings(&memory.file_magic, &again.header().magic);
- try std.testing.expectEqual(@as(usize, 1), again.slots);
- try std.testing.expectEqual(@as(u32, 5), std.mem.readInt(u32, again.slot(0)[0..4], .little));
+ try std.testing.expectEqual(@as(u32, 5), std.mem.readInt(u32, state.live()[0..4], .little));
+ const after = try std.Io.Dir.cwd().readFileAlloc(io, path, gpa, .unlimited);
+ defer gpa.free(after);
+ try std.testing.expectEqualSlices(u8, old, after);
}
-test "appendFile appends to a file beside the state and names it for readFile" {
+test "appendFile appends to a file beside the save and names it for readFile" {
const io = std.testing.io;
var tmp = std.testing.tmpDir(.{});
defer tmp.cleanup();
@@ -1332,7 +1359,8 @@ test "appendFile appends to a file beside the state and names it for readFile" {
host.io = io;
host.written = .empty;
host.files = .empty;
- host.options = .{ .state = try std.mem.print(&buf, ".zig-cache/tmp/{s}/t.state", .{&tmp.sub_path}) };
+ host.options = .{};
+ host.stem = try std.mem.print(&buf, ".zig-cache/tmp/{s}/t", .{&tmp.sub_path});
defer {
var written = host.written.keyIterator();
while (written.next()) |path| host.gpa.free(path.ptr[0 .. path.len + 1]);
@@ -1340,14 +1368,14 @@ test "appendFile appends to a file beside the state and names it for readFile" {
}
const name = "take-1.f32";
const path = std.mem.span(apiAppendFile(&host, name, name.len, "ab", 2, false));
- try std.testing.expect(std.mem.endsWith(u8, path, "/t.state.take-1.f32"));
+ try std.testing.expect(std.mem.endsWith(u8, path, "/t.take-1.f32"));
try std.testing.expectEqual(path.ptr, apiAppendFile(&host, name, name.len, "cd", 2, false));
- const bytes = try tmp.dir.readFileAlloc(io, "t.state.take-1.f32", host.gpa, .unlimited);
+ const bytes = try tmp.dir.readFileAlloc(io, "t.take-1.f32", host.gpa, .unlimited);
defer host.gpa.free(bytes);
try std.testing.expectEqualStrings("abcd", bytes);
// writeFile: a new file, its old bytes gone.
_ = apiAppendFile(&host, name, name.len, "e", 1, true);
- const again = try tmp.dir.readFileAlloc(io, "t.state.take-1.f32", host.gpa, .unlimited);
+ const again = try tmp.dir.readFileAlloc(io, "t.take-1.f32", host.gpa, .unlimited);
defer host.gpa.free(again);
try std.testing.expectEqualStrings("e", again);
}
diff --git a/src/host/Sessions.zig b/src/host/Sessions.zig
index c2af557..ae649bf 100644
--- a/src/host/Sessions.zig
+++ b/src/host/Sessions.zig
@@ -1,7 +1,8 @@
-//! Every run of a sketch, recorded: one xitdb database per sketch, beside
-//! its state file, holding an append-only log of the frames live `update`
-//! was fed, and the State at the moments that matter (keyframes). Nothing
-//! in it is ever overwritten: marks and clips are names pointing into it.
+//! A sketch's save: one xitdb database holding every run of it, an
+//! append-only log of the frames live `update` was fed, and the State at
+//! the moments that matter (keyframes). The newest keyframe and the frames
+//! after it are the present a run starts from (`present`). Nothing in it
+//! is ever overwritten: marks and clips are names pointing into it.
//!
//! root = map {
//! version 1
@@ -14,7 +15,7 @@
//! started wall-clock ns at open ended ns at a clean exit, else 0
//! from "" or where it came from ("import <file>")
//! stretches list of { frame, build, source, layout }: one a load (start, reloads);
-//! `layout` is the state file's header (memory.readLayout reads it)
+//! `layout` is State's header (memory.State.header, read by memory.layoutOf)
//! batches list of bytes: a little-endian u64, the first frame's index, then
//! whole frames of raw `abi.Event`s, each frame ending with its tick
//! keys list of { frame, batch, stretch, why, time, size, pages, written, line }:
@@ -43,9 +44,8 @@ const abi = @import("abi");
pub const Disk = xitdb.Database(.buffered_file, u64);
pub const Memory = xitdb.Database(.memory, u64);
-/// `file` for interactive runs of a build that records, `memory` for a
-/// scripted run from fresh state (time travel, nothing on disk), `off`
-/// otherwise (an export, a release build).
+/// `file` for every run but two: `memory` for a scripted run from fresh
+/// state (time travel, nothing on disk), `off` for an export.
pub const Mode = enum { off, memory, file };
pub const Why = enum(u64) { start, periodic, jump, before_reload, after_reload, mark, exit, import };
@@ -691,6 +691,32 @@ pub fn load(s: *Sessions, session: u64, k: u64) !Loaded {
}
}
+/// The State a run starts from, the present: the newest keyframe of the
+/// sessions before this one, with the frames logged after it appended to
+/// `out` (replayed, they bring it to where that run stopped). Null in a
+/// new database. This run's first keyframe starts from its pages.
+pub fn present(s: *Sessions, out: *std.ArrayList(abi.Event)) !?Loaded {
+ if (s.db == .off) return null;
+ var i = s.session;
+ while (i > 0) {
+ i -= 1;
+ const sum = try s.summary(i);
+ if (sum.keys == 0) continue;
+ const newest = try s.load(i, sum.keys - 1);
+ errdefer newest.free(s.gpa);
+ try s.events(i, newest.batch, newest.frame, sum.frames, out);
+ if (newest.paged) {
+ const hashes = try s.gpa.alloc(u64, std.math.divCeil(usize, newest.state.len, page) catch unreachable);
+ for (hashes, 0..) |*h, j| h.* = std.hash.Wyhash.hash(0, newest.state[j * page .. @min(newest.state.len, (j + 1) * page)]);
+ s.gpa.free(s.hashes);
+ s.hashes = hashes;
+ s.last = .{ .session = i, .key = sum.keys - 1 };
+ }
+ return newest;
+ }
+ return null;
+}
+
/// Keyframe `k`'s frame, time and why, without its bytes.
pub fn keyAt(s: *Sessions, session: u64, k: u64) !struct { frame: u64, time: u64, why: Why } {
switch (s.db) {
diff --git a/src/host/memory.zig b/src/host/memory.zig
index 796fb39..2749ea1 100644
--- a/src/host/memory.zig
+++ b/src/host/memory.zig
@@ -1,9 +1,12 @@
//! Sketch memory by lifetime.
//!
-//! `StateFile`: one `MAP_SHARED` file holding the live state, plus a header
-//! describing the layout, so the state outlives restarts,
-//! a probe can read it from another process, and a reload with a changed
-//! layout can carry the old values over field by field.
+//! `State`: process memory in the layout the sketch declares; a reload
+//! with a changed layout carries the old values over field by field. It
+//! outlives the process in the sessions database (Sessions.zig), whose
+//! stretches keep its layout in the header format below.
+//!
+//! `OldState`: a state file from before the database held the present,
+//! read once to import it.
//!
//! `Session`: this process's memory, kept across reloads.
const std = @import("std");
@@ -14,7 +17,7 @@ const field_value = @import("fields.zig");
pub const page = std.heap.page_size_min;
pub const header_len = 64 * 1024;
-/// One slot, the live state: snapshots are marks in the sessions database.
+/// A layout header, and an old file's one slot, the live state.
pub const file_magic = "SKETCH03".*;
/// Before sessions: slot 0 live and 1..7 snapshots (`old_slot_count`), which
/// the host imports as marks and then drops (`dropSnapshots`).
@@ -287,167 +290,93 @@ fn remapEnum(from: Layout.Leaf, old: []const u8, to: Layout.Leaf, new: []u8) boo
return true;
}
-pub const StateFile = struct {
- fd: posix.fd_t,
- map: []align(page) u8,
- /// 1, or `old_slot_count` in a file from before sessions.
- slots: usize = 1,
- /// Layout of the bytes in the file, owned by `arena`.
- layout: Layout,
+/// Live State: page aligned, in the running sketch's layout; `header` is
+/// that layout as `layoutOf` reads it back, for the sessions' stretches.
+pub const State = struct {
+ bytes: []align(page) u8 = &.{},
+ layout: Layout = .{},
+ header: []const u8 = "",
+ /// Owns `layout` and `header`.
arena: std.heap.ArenaAllocator,
- /// Opens or creates `path`. A new (empty) file has no layout and no used
- /// slots: the first `adopt` fills it. A file that is not a state file
- /// this host can read is refused, never truncated: it may be someone's
- /// work, written by another build.
- pub fn open(gpa: std.mem.Allocator, path: []const u8) !StateFile {
- const fd = try posix.openat(posix.AT.FDCWD, path, .{ .ACCMODE = .RDWR, .CREAT = true }, 0o600);
- errdefer _ = posix.system.close(fd);
- var self: StateFile = .{ .fd = fd, .map = &.{}, .layout = .{}, .arena = .init(gpa) };
- errdefer self.arena.deinit();
- const len = try fileSize(fd);
- if (len > 0) {
- if (len < header_len) return error.NotAStateFile;
- self.map = try mapFile(fd, @intCast(len), .{ .READ = true, .WRITE = true });
- errdefer posix.munmap(self.map);
- try self.check();
- self.layout = try readLayout(self.arena.allocator(), self.map) orelse .{};
- return self;
- }
- try self.resize(0);
- self.header().* = .{ .size = 0, .alignment = 1, .hash = 0, .stride = 0, .used = 0, .field_count = 0, .strings_len = 0 };
- return self;
+ pub fn init(gpa: std.mem.Allocator) State {
+ return .{ .arena = .init(gpa) };
}
- pub fn close(self: *StateFile) void {
- if (self.map.len != 0) posix.munmap(self.map);
- _ = posix.system.close(self.fd);
+ pub fn deinit(self: *State, gpa: std.mem.Allocator) void {
+ gpa.free(self.bytes);
self.arena.deinit();
}
- pub fn header(self: *StateFile) *Header {
- return @ptrCast(self.map.ptr);
- }
-
- /// The header and the layout after it, as `readLayout` reads them back.
- pub fn layoutBytes(self: *StateFile) []const u8 {
- const h = self.header();
- return self.map[0 .. @sizeOf(Header) + @as(usize, h.field_count) * @sizeOf(StoredField) + h.strings_len];
+ pub fn live(self: *State) []align(page) u8 {
+ return self.bytes[0..self.layout.size];
}
- pub fn slot(self: *StateFile, index: usize) []align(page) u8 {
- std.debug.assert(index < self.slots);
- const h = self.header();
- const start = header_len + index * h.stride;
- return @alignCast(self.map[start..][0..h.size]);
+ /// Makes State hold `new`'s layout: a new allocation of its defaults
+ /// (`defaults`), then every leaf `migrate` keeps of the old State. Null
+ /// when the layout is unchanged, or there was none.
+ pub fn adopt(self: *State, gpa: std.mem.Allocator, new: abi.Layout, defaults: *const fn (abi.Memory, [*]u8) callconv(.c) void) !?Migration {
+ const first = self.bytes.len == 0;
+ if (!first and self.layout.hash == new.hash and self.layout.size == new.size) return null;
+ var arena: std.heap.ArenaAllocator = .init(gpa);
+ errdefer arena.deinit();
+ const layout = try Layout.copy(arena.allocator(), new);
+ const region = try arena.allocator().alignedAlloc(u8, .of(Header), header_len);
+ @memset(region, 0);
+ const h: *Header = @ptrCast(region.ptr);
+ h.* = .{ .size = new.size, .alignment = new.alignment, .hash = new.hash, .stride = std.mem.alignForward(usize, @max(new.size, 1), page), .used = 1, .field_count = 0, .strings_len = 0 };
+ try writeLayout(region, layout);
+ const bytes = try gpa.alignedAlloc(u8, .fromByteUnits(page), @max(new.size, 1));
+ @memset(bytes, 0);
+ defaults(.state, bytes.ptr);
+ const m = if (first) null else migrate(self.layout, self.live(), layout, bytes);
+ self.deinit(gpa);
+ self.* = .{ .bytes = bytes, .layout = layout, .header = region[0 .. @sizeOf(Header) + @as(usize, h.field_count) * @sizeOf(StoredField) + h.strings_len], .arena = arena };
+ return m;
}
+};
- pub fn used(self: *StateFile, index: usize) bool {
- return self.header().used & (@as(u32, 1) << @intCast(index)) != 0;
- }
+/// A state file from before the sessions database held the present: a
+/// layout header, then one slot (`file_magic`), or eight (`file_magic_v2`,
+/// `file_magic_v1`), the first live State and the others snapshots. Mapped
+/// read-only: imported once, never changed.
+pub const OldState = struct {
+ map: []align(page) const u8,
+ slots: usize,
- pub fn markUsed(self: *StateFile, index: usize, value: bool) void {
- const bit = @as(u32, 1) << @intCast(index);
- if (value) self.header().used |= bit else self.header().used &= ~bit;
+ /// error.FileNotFound when `path` is missing or empty; a file that is
+ /// no state file this host reads is refused.
+ pub fn open(path: []const u8) !OldState {
+ const map = mapReadOnly(path);
+ if (map.len == 0) return error.FileNotFound;
+ errdefer posix.munmap(map);
+ if (map.len < header_len) return error.NotAStateFile;
+ const h: *const Header = @ptrCast(map.ptr);
+ const slots: usize = if (std.mem.eql(u8, &h.magic, &file_magic)) 1 else if (std.mem.eql(u8, &h.magic, &file_magic_v2) or std.mem.eql(u8, &h.magic, &file_magic_v1)) old_slot_count else return error.NotAStateFile;
+ if (map.len < header_len + slots * h.stride or h.size > h.stride) return error.CorruptStateFile;
+ return .{ .map = map, .slots = slots };
}
- pub const Adoption = union(enum) {
- unchanged,
- created,
- migrated: Migration,
- };
-
- /// Makes the file hold `new`'s layout. Used slots are migrated: `defaults`
- /// writes the new default values, then unchanged fields are copied over.
- pub fn adopt(self: *StateFile, gpa: std.mem.Allocator, new: abi.Layout, defaults: *const fn (abi.Memory, [*]u8) callconv(.c) void) !Adoption {
- if (self.layout.hash == new.hash and self.layout.size == new.size and self.header().used & 1 != 0) {
- // Same type, older header: describe it again, bytes untouched.
- if (std.mem.eql(u8, &self.header().magic, &file_magic_v1)) {
- var kept = self.arena;
- defer kept.deinit();
- self.arena = .init(gpa);
- self.layout = try Layout.copy(self.arena.allocator(), new);
- try writeLayout(self.map[0..header_len], self.layout);
- self.header().magic = file_magic_v2;
- }
- return .unchanged;
- }
- const had_state = self.header().used & 1 != 0;
-
- // Keep the old bytes and layout while the file is rebuilt.
- var old_arena: std.heap.ArenaAllocator = .init(gpa);
- defer old_arena.deinit();
- const old_layout = self.layout;
- var old_slots: [old_slot_count]?[]u8 = @splat(null);
- for (old_slots[0..self.slots], 0..) |*copy, i| {
- if (self.used(i)) copy.* = try old_arena.allocator().dupe(u8, self.slot(i));
- }
- const old_used = self.header().used;
- // `old_layout` points into `self.arena`: move it aside before resetting.
- var kept = self.arena;
- defer kept.deinit();
- self.arena = .init(gpa);
-
- self.layout = try Layout.copy(self.arena.allocator(), new);
- const stride = std.mem.alignForward(usize, @max(new.size, 1), page);
- try self.resize(stride);
- const h = self.header();
- h.* = .{ .magic = if (self.slots == 1) file_magic else file_magic_v2, .size = new.size, .alignment = new.alignment, .hash = new.hash, .stride = stride, .used = old_used, .field_count = 0, .strings_len = 0 };
- try writeLayout(self.map[0..header_len], self.layout);
-
- var total: Migration = .{};
- for (old_slots, 0..) |maybe_old, i| {
- const old = maybe_old orelse continue;
- const bytes = self.slot(i);
- @memset(bytes, 0);
- defaults(.state, bytes.ptr);
- const m = migrate(old_layout, old, self.layout, bytes);
- total.kept += m.kept;
- if (i == 0) total.reset = m.reset;
- }
- if (!had_state) {
- self.markUsed(0, false);
- return .created;
- }
- return .{ .migrated = total };
+ pub fn close(self: OldState) void {
+ posix.munmap(self.map);
}
- /// Makes a file from before sessions one of a single slot, its
- /// snapshots (imported as marks by now) dropped.
- pub fn dropSnapshots(self: *StateFile) !void {
- if (self.slots == 1) return;
- const stride = self.header().stride;
- self.slots = 1;
- try self.resize(stride);
- self.header().magic = file_magic;
- self.header().used &= 1;
+ /// The header and layout, as `restore` and `layoutOf` take them.
+ pub fn layoutBytes(self: OldState) []const u8 {
+ return self.map[0..header_len];
}
- fn check(self: *StateFile) !void {
- const h = self.header();
- if (std.mem.eql(u8, &h.magic, &file_magic)) {
- self.slots = 1;
- } else if (std.mem.eql(u8, &h.magic, &file_magic_v2) or std.mem.eql(u8, &h.magic, &file_magic_v1)) {
- self.slots = old_slot_count;
- } else return error.NotAStateFile;
- if (self.map.len < header_len + self.slots * h.stride or h.size > h.stride) return error.CorruptStateFile;
- }
-
- fn resize(self: *StateFile, stride: usize) !void {
- const len = header_len + self.slots * stride;
- if (self.map.len != 0) posix.munmap(self.map);
- self.map = &.{};
- switch (posix.errno(posix.system.ftruncate(self.fd, @intCast(len)))) {
- .SUCCESS => {},
- else => |err| return posix.unexpectedErrno(err),
- }
- self.map = try mapFile(self.fd, len, .{ .READ = true, .WRITE = true });
+ /// Slot `i`'s State, null when the slot is unused.
+ pub fn slot(self: OldState, i: usize) ?[]const u8 {
+ const h: *const Header = @ptrCast(self.map.ptr);
+ if (h.used & (@as(u32, 1) << @intCast(i)) == 0) return null;
+ return self.map[header_len + i * h.stride ..][0..h.size];
}
};
/// Fills `live` (laid out as `to`) from `image`, state kept in another
-/// layout, described by `layout_bytes` (a header as `StateFile.layoutBytes`
-/// gives it): `defaults`, then every leaf `migrate` keeps. Returns the
+/// layout, described by `layout_bytes` (a header as `State.header` holds
+/// it): `defaults`, then every leaf `migrate` keeps. Returns the
/// image's layout, `arena`'s.
pub fn restore(arena: std.mem.Allocator, layout_bytes: []const u8, image: []const u8, to: Layout, live: []u8, defaults: *const fn (abi.Memory, [*]u8) callconv(.c) void) !Layout {
const from = try layoutOf(arena, layout_bytes);
@@ -458,7 +387,7 @@ pub fn restore(arena: std.mem.Allocator, layout_bytes: []const u8, image: []cons
return from;
}
-/// The layout in a header as `StateFile.layoutBytes` gives it, `arena`'s.
+/// The layout in a header as `State.header` holds it, `arena`'s.
pub fn layoutOf(arena: std.mem.Allocator, layout_bytes: []const u8) !Layout {
if (layout_bytes.len < @sizeOf(Header) or layout_bytes.len > header_len) return error.CorruptKeyframe;
const region = try arena.alignedAlloc(u8, .of(Header), header_len);
@@ -556,10 +485,6 @@ pub fn readLayout(arena: std.mem.Allocator, map: []const u8) !?Layout {
return .{ .size = h.size, .alignment = h.alignment, .hash = h.hash, .fields = leaves };
}
-pub fn mapFile(fd: posix.fd_t, len: usize, prot: posix.PROT) ![]align(page) u8 {
- return posix.mmap(null, len, prot, .{ .TYPE = .SHARED }, fd, 0);
-}
-
pub fn fileSize(fd: posix.fd_t) !u64 {
var stat: linux.Statx = undefined;
switch (linux.errno(linux.statx(fd, "", linux.AT.EMPTY_PATH, .{ .SIZE = true }, &stat))) {
@@ -715,7 +640,7 @@ test "migration keeps unchanged fields and defaults the rest" {
try std.testing.expectEqual(@as(usize, 1), m.reset);
}
-test "state file migrates live state, and a reopen reads the layout back" {
+test "State migrates to a changed layout, and keeps itself when it is unchanged" {
const V1 = extern struct { score: u32 = 0, name: [4]u8 = "none".* };
const V2 = extern struct { level: u16 = 1, score: u32 = 0 };
const D = struct {
@@ -726,24 +651,20 @@ test "state file migrates live state, and a reopen reads the layout back" {
@as(*V2, @ptrCast(@alignCast(p))).* = .{};
}
};
- var tmp = std.testing.tmpDir(.{});
- defer tmp.cleanup();
- var buf: [std.fs.max_path_bytes]u8 = undefined;
- const path = try std.mem.print(&buf, ".zig-cache/tmp/{s}/test.state", .{&tmp.sub_path});
const gpa = std.testing.allocator;
- {
- var file = try StateFile.open(gpa, path);
- defer file.close();
- try std.testing.expectEqual(StateFile.Adoption.created, try file.adopt(gpa, abi.layout.of(V1, .state), D.v1));
- file.markUsed(0, true);
- @as(*V1, @ptrCast(file.slot(0).ptr)).score = 99;
- }
- var file = try StateFile.open(gpa, path);
- defer file.close();
- try std.testing.expectEqual(abi.layout.hash(V1), file.layout.hash);
- const adoption = try file.adopt(gpa, abi.layout.of(V2, .state), D.v2);
- try std.testing.expectEqual(@as(usize, 1), adoption.migrated.reset);
- try std.testing.expectEqual(V2{ .level = 1, .score = 99 }, @as(*V2, @ptrCast(file.slot(0).ptr)).*);
+ var state: State = .init(gpa);
+ defer state.deinit(gpa);
+ try std.testing.expectEqual(@as(?Migration, null), try state.adopt(gpa, abi.layout.of(V1, .state), D.v1));
+ @as(*V1, @ptrCast(state.live().ptr)).score = 99;
+ try std.testing.expectEqual(@as(?Migration, null), try state.adopt(gpa, abi.layout.of(V1, .state), D.v1));
+ try std.testing.expectEqual(@as(u32, 99), @as(*V1, @ptrCast(state.live().ptr)).score);
+ const m = (try state.adopt(gpa, abi.layout.of(V2, .state), D.v2)).?;
+ try std.testing.expectEqual(@as(usize, 1), m.reset);
+ try std.testing.expectEqual(V2{ .level = 1, .score = 99 }, @as(*V2, @ptrCast(@alignCast(state.live().ptr))).*);
+ // The header reads back as the layout.
+ var arena: std.heap.ArenaAllocator = .init(gpa);
+ defer arena.deinit();
+ try std.testing.expectEqual(abi.layout.hash(V2), (try layoutOf(arena.allocator(), state.header)).hash);
}
test "arrays migrate element by element: fields, enums by name, shrink and growth" {
@@ -858,14 +779,15 @@ test "a file that is not a state file is refused, untouched" {
defer std.testing.allocator.free(text);
@memset(text, 'x');
try std.Io.Dir.cwd().writeFile(io, .{ .sub_path = path, .data = text });
- try std.testing.expectError(error.NotAStateFile, StateFile.open(std.testing.allocator, path));
+ try std.testing.expectError(error.NotAStateFile, OldState.open(path));
const after = try std.Io.Dir.cwd().readFileAlloc(io, path, std.testing.allocator, .unlimited);
defer std.testing.allocator.free(after);
try std.testing.expectEqualSlices(u8, text, after);
}
+ try std.testing.expectError(error.FileNotFound, OldState.open(try testPath(&buf, tmp, "none.state")));
}
-test "a state file from before element leaves (drat-meout's) keeps every value" {
+test "a state file from before element leaves (drat-meout's) imports with every value" {
const gpa = std.testing.allocator;
const io = std.testing.io;
var tmp = std.testing.tmpDir(.{});
@@ -885,30 +807,29 @@ test "a state file from before element leaves (drat-meout's) keeps every value"
const h: *const Header = @ptrCast(@alignCast(original.ptr));
const live = original[header_len..][0..h.size];
- // The layout the game had when it wrote the file, as a sketch would
- // describe it: unchanged, so nothing migrates and the header is
- // rewritten in the new format.
- const same = try asSketchLayout(a, old, &.{});
- {
- var file = try StateFile.open(gpa, path);
- defer file.close();
- try std.testing.expectEqual(StateFile.Adoption.unchanged, try file.adopt(gpa, same, NoDefaults.f));
- try std.testing.expectEqualStrings(&file_magic_v2, &file.header().magic);
- }
- var file = try StateFile.open(gpa, path);
+ const file = try OldState.open(path);
defer file.close();
- try std.testing.expectEqual(old.fields.len, file.layout.fields.len);
- try std.testing.expectEqualSlices(u8, live, file.slot(0));
-
- // A changed layout (one field gone, one new): every other leaf keeps
- // its value, the songs included (an unchanged array type, so whole).
- const changed = try asSketchLayout(a, old, "scene");
- _ = try file.adopt(gpa, changed, NoDefaults.f);
- for (old.fields) |leaf| {
- if (std.mem.eql(u8, leaf.path, "scene")) continue;
- try std.testing.expectEqualSlices(u8, leaf.bytes(live), leaf.bytes(file.slot(0)));
+ try std.testing.expectEqual(@as(usize, old_slot_count), file.slots);
+ // Into the layout the game had when it wrote the file, as a sketch would
+ // describe it: every byte; then into a changed one (one field gone, one
+ // new): every other leaf keeps its value, the songs included (an
+ // unchanged array type, so whole).
+ for ([_][]const u8{ "", "scene" }) |drop| {
+ var state: State = .init(gpa);
+ defer state.deinit(gpa);
+ _ = try state.adopt(gpa, try asSketchLayout(a, old, drop), NoDefaults.f);
+ @memset(state.live(), 0);
+ _ = try restore(a, file.layoutBytes(), file.slot(0).?, state.layout, state.live(), NoDefaults.f);
+ if (drop.len == 0) try std.testing.expectEqualSlices(u8, live, state.live());
+ for (old.fields) |leaf| {
+ if (drop.len > 0 and std.mem.eql(u8, leaf.path, drop)) continue;
+ try std.testing.expectEqualSlices(u8, leaf.bytes(live), leaf.bytes(state.live()));
+ }
}
- try std.testing.expectEqual(@as(u8, 0), file.slot(0)[h.size]);
+ // The file is as it was.
+ const after = try tmp.dir.readFileAlloc(io, "game.state", gpa, .unlimited);
+ defer gpa.free(after);
+ try std.testing.expectEqualSlices(u8, original, after);
}
const NoDefaults = struct {
diff --git a/src/host/probe.zig b/src/host/probe.zig
deleted file mode 100644
index 4b61153..0000000
--- a/src/host/probe.zig
+++ /dev/null
@@ -1,86 +0,0 @@
-//! sketchbook-probe: prints the live state of any running sketch, from its
-//! state file alone. The file header carries the layout, so the probe needs
-//! no knowledge of the sketch: it maps the same pages read-only.
-const std = @import("std");
-const memory = @import("memory.zig");
-const fields = @import("fields.zig");
-
-const usage =
- \\usage: sketchbook-probe [--watch] [--interval MS] STATE_FILE
- \\Prints every field of the live state; --watch prints again on change.
- \\
-;
-
-pub fn main(init: std.process.Init) !void {
- var path: ?[]const u8 = null;
- var watch = false;
- var interval_ms: i64 = 100;
- var args = init.minimal.args.iterate();
- _ = args.skip();
- while (args.next()) |arg| {
- if (std.mem.eql(u8, arg, "--watch")) {
- watch = true;
- } else if (std.mem.eql(u8, arg, "--interval")) {
- interval_ms = std.fmt.parseInt(i64, args.next() orelse "", 10) catch fatal("--interval needs milliseconds");
- } else if (std.mem.startsWith(u8, arg, "-")) {
- std.debug.print("{s}", .{usage});
- return;
- } else path = arg;
- }
- const file = path orelse fatal("no state file given");
- const map = memory.mapReadOnly(file);
- if (map.len < memory.header_len) fatal("not a sketchbook state file");
- const magic = map[0..8];
- if (!std.mem.eql(u8, magic, &memory.file_magic) and !std.mem.eql(u8, magic, &memory.file_magic_v2) and !std.mem.eql(u8, magic, &memory.file_magic_v1)) fatal("not a sketchbook state file");
- defer std.posix.munmap(map);
- var arena: std.heap.ArenaAllocator = .init(init.gpa);
- defer arena.deinit();
- const layout = (try memory.readLayout(arena.allocator(), map)) orelse fatal("state file has no layout yet");
- const header: *const memory.Header = @ptrCast(@alignCast(map.ptr));
- const live = map[memory.header_len..][0..header.size];
-
- const shown = try arena.allocator().dupe(u8, live);
- show(file, layout, live);
- while (watch) {
- try std.Io.sleep(init.io, .fromMilliseconds(interval_ms), .awake);
- if (std.mem.eql(u8, shown, live)) continue;
- @memcpy(shown, live);
- show(file, layout, live);
- }
-}
-
-fn show(path: []const u8, layout: memory.Layout, live: []const u8) void {
- var buf: [4096]u8 = undefined;
- var w: std.Io.Writer = .fixed(&buf);
- std.debug.print("{s}: {d} bytes, layout {x:0>16}\n", .{ path, live.len, layout.hash });
- for (layout.fields) |leaf| {
- // Whole arrays are shown by their elements' leaves.
- if (leaf.whole()) continue;
- w.end = 0;
- if (leaf.dims.len == 0) {
- fields.format(leaf, live, &w) catch {};
- } else {
- // Elements in index order, the first few.
- var it = leaf.elements();
- var n: usize = 0;
- while (it.next()) |offset| : (n += 1) {
- if (n == 8) {
- w.writeAll(" ...") catch {};
- break;
- }
- if (n > 0) w.writeAll(" ") catch {};
- fields.format(leaf.at(offset), live, &w) catch {};
- }
- }
- var name: [128]u8 = undefined;
- var nw: std.Io.Writer = .fixed(&name);
- nw.writeAll(leaf.path) catch {};
- for (leaf.dims) |d| nw.print(" [{d}]", .{d.count}) catch {};
- std.debug.print(" +0x{x:0>4} {s:<28} {s}\n", .{ leaf.offset, nw.buffered(), w.buffered() });
- }
-}
-
-fn fatal(message: []const u8) noreturn {
- std.debug.print("sketchbook-probe: {s}\n{s}", .{ message, usage });
- std.process.exit(2);
-}
diff --git a/src/host/root.zig b/src/host/root.zig
index 629c466..9c7ad77 100644
--- a/src/host/root.zig
+++ b/src/host/root.zig
@@ -2,11 +2,11 @@
//! that it hot-reloads whenever a build replaces it.
//!
//! sketchbook [options] [LIBRARY]
-//! --state FILE persistent state (default: $XDG_DATA_HOME/<name>/<name>.state;
-//! sketch.state for a script or an export)
+//! --state FILE the save (default: $XDG_DATA_HOME/<name>/<name>.xitdb;
+//! sketch.xitdb for a script or an export)
//! --headless offscreen video, silent audio
//! --script FILE play a script instead of reading devices
-//! --fresh start from default state (sessions, marks and clips are kept)
+//! --fresh start from default state (the save's history is kept)
//! --timing-marker encode frame ID and draw sample time for video analysis
//! --profile-gpu time each GPU command alone, print the times at exit
//! --console LINE run a console line once the sketch is loaded (repeatable)
@@ -93,31 +93,9 @@ pub fn main(init: std.process.Init, linked: ?abi.Sketch) !void {
} else if (try std.Io.File.stdout().isTty(init.io)) fatal("--export -: stdout is a terminal; pipe it to ffmpeg");
}
- // A scripted run from fresh state works on files of its own, moved to
- // the given path at exit: two runs of one build step (two `zig build`s
- // at once share its output path) never share a state file, nor the
- // files the sketch appends to (its sessions are in memory). An export's
- // are removed instead: nothing reads them, and a player may have the
- // given ones open.
- var private_buf: [std.fs.max_path_bytes]u8 = undefined;
- if (options.state.len == 0 and (script_path != null or options.export_to != null)) options.state = "sketch.state";
- const given = options.state;
- const private = (script_path != null or options.export_to != null) and options.fresh;
- if (!private) sketchbookDir(init.io, given);
- if (private) options.state = std.mem.print(&private_buf, "{s}.{d}", .{ given, std.os.linux.getpid() }) catch fatal("--state path too long");
- // Its files are `<private>` and `<private>.<name>`, beside the given one.
- defer if (private) moved: {
- const dir = std.Io.Dir.cwd().openDir(init.io, std.fs.path.dirname(given) orelse ".", .{ .iterate = true }) catch break :moved;
- defer dir.close(init.io);
- const from = std.fs.path.basename(options.state);
- var it = dir.iterate();
- while (it.next(init.io) catch null) |entry| {
- if (!std.mem.startsWith(u8, entry.name, from) or (entry.name.len > from.len and entry.name[from.len] != '.')) continue;
- var to: [std.fs.max_path_bytes]u8 = undefined;
- const name = std.mem.print(&to, "{s}{s}", .{ std.fs.path.basename(given), entry.name[from.len..] }) catch continue;
- if (options.export_to != null) dir.deleteFile(init.io, entry.name) catch {} else dir.rename(entry.name, dir, name, init.io) catch {};
- }
- };
+ if (options.state.len == 0 and (script_path != null or options.export_to != null)) options.state = "sketch.xitdb";
+ // A scripted run from fresh state, or an export, touches no save.
+ if (!((script_path != null or options.export_to != null) and options.fresh)) sketchbookDir(init.io, options.state);
sdl_heap.install();
{
@@ -137,11 +115,12 @@ pub fn main(init: std.process.Init, linked: ?abi.Sketch) !void {
const usage =
\\usage: sketchbook [options] [LIBRARY]
\\ --9p SOCKET local 9P controls: ctl, status, log, screen.png
- \\ --state FILE persistent state; sessions are kept beside it (default:
- \\ $XDG_DATA_HOME/<name>/<name>.state, sketch.state for --script, --export)
+ \\ --state FILE the save, one file: FILE, or FILE.xitdb for a FILE.state (where a
+ \\ state file from before is imported from, once); default:
+ \\ $XDG_DATA_HOME/<name>/<name>.xitdb, sketch.xitdb for --script, --export
\\ --headless offscreen video, silent audio, no vsync
\\ --script FILE play a script instead of reading devices
- \\ --fresh start from default state (sessions, marks and clips are kept)
+ \\ --fresh start from default state (the save's history is kept)
\\ --timing-marker encode frame ID and draw sample time for video analysis
\\ --profile-gpu time each GPU command alone, print the times at exit
\\ --console LINE run a console line once the sketch is loaded (repeatable)
@@ -154,11 +133,11 @@ const usage =
\\
;
-/// A state file in a directory named `.sketchbook` (`zig build run`'s):
-/// the directory is made, with a `.gitignore` of `*` (its sessions database
-/// grows, and a jj snapshot would commit it), and the state file an older
-/// build kept beside it is moved in, once. A file is never replaced: when
-/// both exist, nothing moves.
+/// A save in a directory named `.sketchbook` (`zig build run`'s): the
+/// directory is made, with a `.gitignore` of `*` (the save grows, and a jj
+/// snapshot would commit it), and the state file an older build kept beside
+/// it (`<stem>.state`) is moved in, once, to be imported. A file is never
+/// replaced: when both exist, nothing moves.
fn sketchbookDir(io: std.Io, state: []const u8) void {
const dir = std.fs.path.dirname(state) orelse return;
if (!std.mem.eql(u8, std.fs.path.basename(dir), ".sketchbook")) return;
@@ -170,8 +149,9 @@ fn sketchbookDir(io: std.Io, state: []const u8) void {
var from_buf: [std.fs.max_path_bytes]u8 = undefined;
var to_buf: [std.fs.max_path_bytes]u8 = undefined;
const parent = std.fs.path.dirname(dir) orelse ".";
- const from = std.fmt.bufPrintSentinel(&from_buf, "{s}/{s}", .{ parent, std.fs.path.basename(state) }, 0) catch return;
- const to = std.fmt.bufPrintSentinel(&to_buf, "{s}", .{state}, 0) catch return;
+ const stem = Host.stemOf(std.fs.path.basename(state));
+ const from = std.fmt.bufPrintSentinel(&from_buf, "{s}/{s}.state", .{ parent, stem }, 0) catch return;
+ const to = std.fmt.bufPrintSentinel(&to_buf, "{s}/{s}.state", .{ dir, stem }, 0) catch return;
switch (std.os.linux.errno(std.os.linux.renameat2(std.os.linux.AT.FDCWD, from.ptr, std.os.linux.AT.FDCWD, to.ptr, .{ .NOREPLACE = true }))) {
.SUCCESS => std.debug.print("sketchbook: moved {s} to {s}\n", .{ from, to }),
.EXIST => std.debug.print("sketchbook: {s} and {s} both exist: nothing moved, {s} is used\n", .{ from, to, to }),
diff --git a/src/host/timeline.zig b/src/host/timeline.zig
index c2eeff3..e755433 100644
--- a/src/host/timeline.zig
+++ b/src/host/timeline.zig
@@ -45,8 +45,8 @@ pub const Frames = struct {
pub const Timeline = struct {
gpa: std.mem.Allocator,
- /// Live state is its slot 0.
- state: *memory.StateFile,
+ /// Live State.
+ state: *memory.State,
/// The running sketch's: a restore starts from its defaults.
defaults: *const fn (abi.Memory, [*]u8) callconv(.c) void,
mode: abi.Mode = .live,
@@ -90,7 +90,7 @@ pub const Timeline = struct {
pub fn restore(self: *Timeline, key: Sessions.Loaded) !void {
var arena: std.heap.ArenaAllocator = .init(self.gpa);
defer arena.deinit();
- _ = try memory.restore(arena.allocator(), key.layout, key.state, self.state.layout, self.state.slot(0), self.defaults);
+ _ = try memory.restore(arena.allocator(), key.layout, key.state, self.state.layout, self.state.live(), self.defaults);
self.restored = true;
self.fed = 0;
self.next_check = 0;
@@ -202,18 +202,14 @@ pub const Timeline = struct {
test "frames are whole: input waits for its tick, in playback, seek and step" {
const gpa = std.testing.allocator;
- // Live state on plain memory: 8 bytes, no fields.
- var file: memory.StateFile = undefined;
- file.map = try gpa.alignedAlloc(u8, .fromByteUnits(memory.page), memory.header_len + memory.page);
- defer gpa.free(file.map);
- @memset(file.map, 0);
- file.slots = 1;
- file.header().* = .{ .size = 8, .alignment = 1, .hash = 1, .stride = memory.page, .used = 1, .field_count = 0, .strings_len = 0 };
- file.layout = .{ .size = 8, .hash = 1 };
+ // Live State: 8 bytes, no fields.
const D = struct {
fn f(_: abi.Memory, _: [*]u8) callconv(.c) void {}
};
- var t: Timeline = .{ .gpa = gpa, .state = &file, .defaults = D.f };
+ var state: memory.State = .init(gpa);
+ defer state.deinit(gpa);
+ _ = try state.adopt(gpa, .{ .size = 8, .alignment = 1, .hash = 1, .fields = &.{} }, D.f);
+ var t: Timeline = .{ .gpa = gpa, .state = &state, .defaults = D.f };
defer t.deinit();
// Frame k: a move at 10k ms, its tick 5 ms later; then a move with no
// tick (a frame cut off).
@@ -223,7 +219,7 @@ test "frames are whole: input waits for its tick, in playback, seek and step" {
try events.append(gpa, abi.Event.tick(1000 + k * 10_000_000 + 5_000_000, 16_666_667));
}
try events.append(gpa, .{ .kind = .mouse_move, .time = 40_000_000 });
- const key: Sessions.Loaded = .{ .frame = 0, .batch = 0, .why = .mark, .time = 0, .stretch = 0, .build = 7, .state = try gpa.dupe(u8, &@as([8]u8, @splat(0))), .line = try gpa.dupe(u8, ""), .source = try gpa.dupe(u8, ""), .layout = try gpa.dupe(u8, file.layoutBytes()) };
+ const key: Sessions.Loaded = .{ .frame = 0, .batch = 0, .why = .mark, .time = 0, .stretch = 0, .build = 7, .state = try gpa.dupe(u8, &@as([8]u8, @splat(0))), .line = try gpa.dupe(u8, ""), .source = try gpa.dupe(u8, ""), .layout = try gpa.dupe(u8, state.header) };
t.load(key, events);
try t.play(0);