summaryrefslogtreecommitdiff
path: root/docs/next-daemon.md
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-10-30 00:25:19 -0300
committerGabriel Schneider <[email protected]>2026-10-30 00:53:24 -0300
commitf264dbc4a646da695331fe7401fa013cbb720b6a (patch)
treeb8d960c5034d7ebfeb44fa25fa5697abbf386a9f /docs/next-daemon.md
parentbce88593014a4078dc9259855944f9ddbc19fe54 (diff)
downloadpardes-f264dbc4a646da695331fe7401fa013cbb720b6a.tar.gz
pardes-f264dbc4a646da695331fe7401fa013cbb720b6a.zip
the walker's digest: a blob's hash made once and folded in as a word (a 100 MB file was 5 ms a digest, two a unit, on the authority); blob fields annotated `.blob`, which the rollback copy's pins read too
walk.zig: a field named in a type's `replica.blob` (a file's text, line index and location rows, an undo snapshot's text, a register's values and joined text, the rows an edit dropped, a query's saved rows) holds immutable blobs. A digest hashes one of 4 KiB or more once, keeps the word by its address and length, and folds the length and word in; `forget` drops the word, from memory.freeRaw and freeUnfilled, through which every such blob is freed raw. Only a Debug build (the tests, the harness, replay) hashes the bytes at every digest all the same and panics where the kept word is not their hash: a blob written in place, or freed past its word. A release build, ReleaseSafe the user's own as well as ReleaseFast, takes the word and never panics on a stale one: at worst a divergence, which a replica records and then stops at (nothing resets or rejoins it yet; the notes say so). File.zig: an edit's free of the old text went to `rawFree` itself and so kept its word; the next text of the same length at the same address folded in the old one (the ReleaseFast harness diverged in 162 of 180 replica runs, in `file.content`), and it goes through freeUnfilled now. Registers.zig: `put` makes a new values array rather than writing the old one. rollback.zig: the copy pins what `.blob` names (`pinned` and `pinnedDeep` gone). Measured, ReleaseFast on the perf cores: with a 100 MB file open (PARDES_DIGEST_BENCH=100, replica.zig's bench), a unit of a key into it went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the word), a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients): rollback p99 9.5 -> 2.6 ms at 30 ms (1.5 and 1.4 ms at 80 and 207), copy p99 0.27 ms; none diverged, none unequal, no delta wrong. Tests: a blob hashed once, its kept word no longer its hash once written in place, its word gone when freed. docs/next-daemon.md: the follow-up. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Diffstat (limited to 'docs/next-daemon.md')
-rw-r--r--docs/next-daemon.md44
1 files changed, 44 insertions, 0 deletions
diff --git a/docs/next-daemon.md b/docs/next-daemon.md
index 55921027..681a2ed4 100644
--- a/docs/next-daemon.md
+++ b/docs/next-daemon.md
@@ -1825,6 +1825,50 @@ Follow-ups (2026-10-29):
one leaves none. The gate's baselines are recorded again from base (main
and the new perf.zig), and .scratch/daemon/base/pardes-perf-live rebuilt
from it.
+- **A blob's digest is a word, made once.** The walker's digest, taken at
+ every unit a replica is sent and every unit it replays, hashed every blob
+ whole each time: a 100 MB file was 5 ms a digest, two digests a unit, on
+ the user's own session. A blob (a field annotated `.blob`: a file's text
+ and line index, an undo snapshot, a register's values and joined text, a
+ file's location rows, the ones an edit dropped, a query's saved) is now
+ hashed once, its word kept by its address (walk.zig), and the digest
+ folds that word in. The rollback copy pins what the same annotation
+ names (`pinned` and `pinnedDeep` are gone).
+ - **Exact while every free drops the word:** every such blob is freed
+ raw, through `memory.freeRaw` or `freeUnfilled` (the pins needed raw
+ frees), and those drop its word, so an address used again is hashed
+ again. The first ReleaseFast harness run found one that did not: an
+ edit freed the old text with `rawFree` itself, the next text of the
+ same length came back at the same address, and the old word was folded
+ in (162 of 180 replica runs diverged, in `file.content`, named with
+ PARDES_REPLICA_TRACE); it goes through `freeUnfilled` now. A replica's
+ blobs born under a copy are never digested (a speculation records
+ none), so the epoch's own frees leave no word behind. A register's
+ values array was the one written in place; `put` now makes a new one.
+ ponytail: the word is kept by address, so a new free path that skips
+ those two misses it; a word carried in the struct beside its blob if
+ that bites again.
+ - **Written in place, or freed past its word:** a Debug build hashes a
+ blob's bytes at every digest and folds those in, and panics where the
+ word kept at that address and length is not their hash: a blob written
+ in place (which the pins forbid too) or freed without dropping its
+ word. The tests, the replicas harness and replay run Debug and go
+ through it (and pass); walk.zig's test writes a blob in place and sees
+ its kept word is no longer its hash. A release build (ReleaseSafe, the
+ user's own, and ReleaseFast) takes the word and never panics on a stale
+ one: at worst it shows as a digest divergence. Not recovered yet: a
+ replica that diverges records why (`Replica.diverged`) and takes no
+ more units; nothing resets it or joins it again. That is the remote
+ client's to add (a fresh join from the authority's dump), not a crash.
+ - Measured (ReleaseFast, alone on the perf cores; PARDES_DIGEST_BENCH=100
+ zig build core-test -Dtest-filter="big file open"): with a 100 MB file
+ open, a unit of a key into it (two digests) went from p50 10.45 ms,
+ p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the
+ file's word), and a digest from 5.18 ms to 10 us. The harness (20 seeds
+ x 120 inputs x 3 clients, ReleaseFast): rollback p99 at 30 ms went from
+ 9.5 to 2.6 ms (1.5 and 1.4 ms at 80 and 207 ms, about as before), copy
+ p99 0.27 ms (0.22 before, within the run's noise); none diverged, none
+ unequal, no delta wrong.
## Phase 6: upgrade notes