summaryrefslogtreecommitdiff
path: root/examples/noc/notes/impl-core-host.md
blob: b83ae8a587ec566575f0cda2b81b7f30e82ff366 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
# Iteration 1, host half (core-host)

Design.md section A, items 9 and 10 (host side), and the owner's
"Acceptance: timeline" script support. Paths below are relative to
`lib/sketchbook/` unless they start with `sketches/`.

## What changed

**One `update` per frame** (src/host/Host.zig, timeline.zig).
- A frame is the events up to and including a `tick`. `Host.update` cuts
  whatever it is given (`timeline.frames`) and calls the sketch once per
  frame; events after the last tick are never fed.
- `Timeline.tick` (playback), `seekCount`, `seekTime` and `step` only ever
  stop between frames. `step N` now moves N *frames* (it used to move N
  input events, skipping ticks); `seek <ms>` lands on the last frame whose
  tick is due.
- `in.now`/`in.dt` in `update` come from the frame's tick: `now` is the
  tick's time, `dt` is `Event.tickDt()` (the tick's f32 seconds, rounded to
  ns). Live uses the same path, so live and replay give the same bits (E4).
  While recording, the frame is captured *before* `update`, and capture
  rewrites the times to recorded ones, so live update sees exactly the
  bytes a replay will (consequence: `in.now` in `update` restarts near 0
  when recording starts; nothing in the game or the ports reads it there).
- Input reset (E2): any snapshot restore (`play`, `seek`, `step`, console
  `load`, `record`) sets `Timeline.restored`; `Host.runLine` then clears
  `mouse_*`, `buttons`, `keys`. On `record`, the input held before the
  reset is prepended to the first frame as events: a `mouse_move` (if the
  pointer is not at 0,0), a `mouse_down` per held button, a `key_down` per
  held key. The recording format is unchanged; a sketch sees those events
  once, when recording starts.
- `abi.Event.tick(time, dt)` / `tickDt()`; the web host uses them too
  (src/sketch/web.zig `frame`), so `in.dt` agrees on both hosts.

**Inspection** (src/layout.zig, memory.zig, fields.zig, script.zig,
Console.zig, probe.zig, overlay.zig, web.zig).
- `layout.flatten` makes arrays and byte-sized-lane vectors into element
  leaves with `[]` in the path, one leaf per element *field*, not per
  element: `boids.buffer[].pos[]` with `dims = {(64, 24), (2, 4)}` (count,
  stride, outermost first; `abi.Field.rank/dims`). Each array also stays as
  a whole leaf (kind `array`) for migration.
  Decision: indices are resolved at lookup (`abi.layout.match/resolve`,
  `Layout.find`), not stored. Storing every element would put e.g. the
  game's `tracker.songs` (8 songs x 16 patterns x 16 rows x 4 notes x 2
  fields = 16k leaves) into a 64 KiB header and slow every lookup; strided
  leaves cost one row per field of the element type.
- Syntax is the Zig one: `pos[0]`, `boids.buffer[3].pos[1]`, `grid[2][1]`.
  `get`/`set`/`expect`/`await`/`sk_field`/`sk_tag` all accept it. Vectors
  of bool or odd-width ints stay one opaque leaf.
- `memory.compare` (behind `Bank.compare`, console `diff`,
  `expect-snapshot`) compares leaf by leaf, element by element, so padding
  never differs (E3). `diff` names the first differing leaf.
- `expect F V ~T` / `await F V ~T`: numeric comparison within T. The field
  is looked up again every frame, so it works across a `load`.

**Migration** (memory.zig).
- Element-wise: for every new leaf with an old leaf of the same pattern
  path, the index tuples both have (min count per dimension) are copied:
  bytes when the element type hash is equal, enums by tag name otherwise.
  Shrink drops the tail, growth keeps the new defaults, scalar arrays keep
  their common prefix, `[2]f32` <-> `@Vector(2, f32)` carry over lane by
  lane. An array whose whole type is unchanged is copied in one piece and
  its element leaves skipped.
- `List` clamp: agreed with draw-layer that `sk.List` accessors clamp `len`
  to the capacity; migration keeps `len` as it was.
- File format: `SKETCH02` (StoredField grows to 40 bytes with `rank` and a
  `dims` offset into the string table). `SKETCH01` files are read: their
  arrays are whole leaves, which migrate whenever the array type is
  unchanged (so `tracker.songs` survives). An unchanged layout on a v1
  file just rewrites the header. Any other file is refused with
  `NotAStateFile` / `CorruptStateFile`, untouched (E11). The store's
  layout header follows the same scheme (`SKSTORE2`, reads `SKSTORE1`); a
  `.store` that is not a store is refused unless `--fresh`.
- Checked by hand on a copy of `drat-meout.state` with today's game: "kept
  72 fields, reset 4" (4 fields added since the file was written), songs
  byte-identical, probe prints `tracker.songs[].bpm [8] ... 156 168`.

**Recording stamp** (timeline.zig, Host.zig, Console.zig). `Host.build` is
a hash of the library file (or `/proc/self/exe` when linked), recomputed on
each load; `record` stores it in `Recording.reserved` (old recordings read
as 0, unknown, no warning); `play` (and a seek/step that starts playback)
logs "Warning: recorded by another build of the sketch; the replay may
differ." A hash of the file, so a draw-only edit also warns: it is a
warning.

**Harness** (root.zig, Host.zig, build.zig).
- Scripted `--fresh` runs work on `<state>.<pid>` (+ `.inputs`, `.store`),
  renamed to the given `--state` at exit. Two `zig build e2e` at once share
  the Run step's output path, which caused "store is open in another
  process". Verified: two concurrent noc `e2e` builds pass.
- Segfault: `Host.init` registered the renderer's `errdefer` before the GPU
  device's, so on an error (store busy) the device was destroyed first and
  `SDL_DestroyRenderer` used it. Swapped; the store-busy path now exits
  with `StoreInUse`.
- `LeakedSdlMemory` under parallel e2e: not ours. Every leak trace is
  `VULKAN_INTERNAL_AllocateMemory` (SDL 3.4.18, SDL_gpu_vulkan.c ~1866):
  when `vkAllocateMemory` fails it frees `freeRegions` and the allocation
  but not `usedRegions` and `memoryLock`, then falls back to another memory
  type and carries on. With 12+ headless Vulkan devices, some heap runs out
  and the fallback path runs. Fix: build.zig compiles that file from a copy
  with the two frees added (`fixSdlVulkanAllocationLeak`, via draw-layer's
  `tools/reload_edit.zig`). Before: 3 of 3 parallel noc runs failed; after:
  6 of 6 passed (incl. two concurrent builds, 24 processes). Worth sending
  upstream.

**Script commands** (script.zig): `expect-snapshot N` (prints up to 5
differing leaves with both values), `expect-log`/`expect-no-log TEXT`,
`expect-same-screenshot A B [MAX [X Y W H]]`,
`expect-different-screenshot A B [X Y W H]`, `expect-pixel FILE X Y R G B
A [TOL]` (the last three also for draw-layer's reload checks).

## Timeline template

`sketches/noc/test/accel_to_mouse.timeline.script`, wired into noc's `e2e`
through the `timelines` list in `sketches/noc/build.zig`. It records
pointer input, then checks play-to-end, pause, seek end, seek middle and
play on, step from the start and back and forth, replay after a live move
(each `expect-snapshot 2`), and play-end vs seek-end screenshots. With
`Host.update` temporarily switched back to one call per batch it fails
(`position[0]: live 327.02, snapshot 2 340.14`); with the fix it passes.

## Tests

- layout.zig: element leaves and dims; vector lanes and nested array
  elements resolve by index, out-of-range and malformed paths do not.
- fields.zig: get/set/match through indexed paths (lanes, struct
  elements, enums).
- memory.zig: element-wise migration (field added to an element, enum per
  element by name, shrink, regrow, scalar prefix, `[2]f32` to `@Vector`,
  `[2][3]` to `[3][2]`); `[8]Song` gaining a field keeps every song;
  unchanged arrays copy whole; leaf-wise compare ignores every padding byte
  and names `list[1].d`; a non-state file (short and long) is refused and
  left byte-identical; `test/fixtures/drat-meout-v1.state-fixture` (a copy
  of the game's file) reopens with every byte, gets a v2 header, then
  migrates to a changed layout keeping every other leaf.
- timeline.zig: playback at 1 ms steps never yields a partial frame;
  seek-time between input and tick, seek end, step back/forward land on
  frame boundaries; a cut-off trailing frame is never fed; the stamp is
  stored; live and recorded ticks give the same dt (E4 values).
- test/replay.script (new `zig build test` step, sketch test/replay.zig):
  button held before `record` (seeded), shift held mid-recording, clicks,
  pointer chasing; `expect` with indices and `~`; then play to end, pause,
  seek end, seek start + step 1000, step -7/+7, seek 200 + play on, `diff`,
  and replay after a live move and press, each `expect-snapshot 2`. The
  sum of `in.dt` is in State, so live dt == replayed dt is checked too.
- test/reload.script: recording made by v1 plays without the warning;
  after `load {v2}` the same recording warns; `expect streak 7 ~0.5` right
  after the load.
- Passing: `zig build test` (lib/sketchbook), the game's `zig build test
  e2e`, `sketches/noc` and `sketches/shaders` `zig build e2e` (parallel,
  several times). No noc script needed new expected values.

## Deferred / notes

- Migration does not clamp a `List`'s `len` (the host does not know
  which field is a count); `get boids.len` can read more than the
  capacity after a shrink. A `pub const sketchbook_list = true` marker
  would let it; not needed while accessors clamp.
- v1 files: an array whose element type changed resets (v1 headers do not
  describe elements). After the first write everything is element-wise.
- Seeded record-start events are visible to the sketch (a held button
  arrives as a `mouse_down` on the first recorded frame).
- Fields no event carries (`width`, `window_*`, `console`, `frame`) still
  reach `update` and do not replay; documented in the README, not narrowed.
- The memory overlay lists arrays as wholes (`<N bytes>`), not elements.
- The build stamp hashes the whole file; a narrower identity (State
  layout + `update`'s code) would avoid warnings on draw-only edits.