diff options
| author | Gabriel Schneider <[email protected]> | 2026-10-30 00:25:19 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-10-30 00:53:24 -0300 |
| commit | f264dbc4a646da695331fe7401fa013cbb720b6a (patch) | |
| tree | b8d960c5034d7ebfeb44fa25fa5697abbf386a9f /src/replica.zig | |
| parent | bce88593014a4078dc9259855944f9ddbc19fe54 (diff) | |
| download | pardes-f264dbc4a646da695331fe7401fa013cbb720b6a.tar.gz pardes-f264dbc4a646da695331fe7401fa013cbb720b6a.zip | |
the walker's digest: a blob's hash made once and folded in as a word (a 100 MB file was 5 ms a digest, two a unit, on the authority); blob fields annotated `.blob`, which the rollback copy's pins read too
walk.zig: a field named in a type's `replica.blob` (a file's text, line index and location rows, an undo snapshot's text, a register's values and joined text, the rows an edit dropped, a query's saved rows) holds immutable blobs. A digest hashes one of 4 KiB or more once, keeps the word by its address and length, and folds the length and word in; `forget` drops the word, from memory.freeRaw and freeUnfilled, through which every such blob is freed raw. Only a Debug build (the tests, the harness, replay) hashes the bytes at every digest all the same and panics where the kept word is not their hash: a blob written in place, or freed past its word. A release build, ReleaseSafe the user's own as well as ReleaseFast, takes the word and never panics on a stale one: at worst a divergence, which a replica records and then stops at (nothing resets or rejoins it yet; the notes say so). File.zig: an edit's free of the old text went to `rawFree` itself and so kept its word; the next text of the same length at the same address folded in the old one (the ReleaseFast harness diverged in 162 of 180 replica runs, in `file.content`), and it goes through freeUnfilled now. Registers.zig: `put` makes a new values array rather than writing the old one. rollback.zig: the copy pins what `.blob` names (`pinned` and `pinnedDeep` gone).
Measured, ReleaseFast on the perf cores: with a 100 MB file open (PARDES_DIGEST_BENCH=100, replica.zig's bench), a unit of a key into it went from p50 10.45 ms, p99 41.6 ms to p50 24 us, p99 6.1 ms (the first, which makes the word), a digest from 5.18 ms to 10 us. The harness (20 seeds x 120 inputs x 3 clients): rollback p99 9.5 -> 2.6 ms at 30 ms (1.5 and 1.4 ms at 80 and 207), copy p99 0.27 ms; none diverged, none unequal, no delta wrong. Tests: a blob hashed once, its kept word no longer its hash once written in place, its word gone when freed. docs/next-daemon.md: the follow-up.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Diffstat (limited to 'src/replica.zig')
| -rw-r--r-- | src/replica.zig | 51 |
1 files changed, 51 insertions, 0 deletions
diff --git a/src/replica.zig b/src/replica.zig index a96a056a..0b55a5bf 100644 --- a/src/replica.zig +++ b/src/replica.zig @@ -1011,3 +1011,54 @@ test "a delta applied under a speculation outlives its rollback: the terminal st const shown = a.core.panes[shell].?.rows; for (rows_a[rows_a.len - shown ..], rows_r[rows_r.len - shown ..]) |x, y| try std.testing.expectEqualStrings(x, y); } + +/// p50 and p99 of `samples`, sorted in place. +fn spread(samples: []u64) [2]u64 { + std.mem.sort(u64, samples, {}, std.sort.asc(u64)); + return .{ samples[samples.len / 2], samples[@min(samples.len - 1, samples.len * 99 / 100)] }; +} + +test "the authority's units with a big file open: its blobs hashed once (PARDES_DIGEST_BENCH=<MB>)" { + const arg = std.c.getenv("PARDES_DIGEST_BENCH") orelse return error.SkipZigTest; + const mb = std.fmt.parseInt(usize, std.mem.span(arg), 10) catch 100; + const gpa = std.heap.c_allocator; + const tmp = std.mem.span(std.c.getenv("TMPDIR") orelse "/tmp"); + var path_buf: [256]u8 = undefined; + const path = try std.fmt.bufPrintSentinel(&path_buf, "{s}/pardes-digest-bench.txt", .{tmp}, 0); + { + const fd = std.c.open(path, .{ .ACCMODE = .WRONLY, .CREAT = true, .TRUNC = true }, @as(std.c.mode_t, 0o644)); + if (fd < 0) return error.OpenFailed; + defer _ = std.c.close(fd); + var line: [80]u8 = undefined; + var n: usize = 0; + var i: usize = 0; + while (n < mb << 20) : (i += 1) { + const text = try std.fmt.bufPrint(&line, "line {d:0>9} of a big file, as a log or a dump is\n", .{i}); + if (std.c.write(fd, text.ptr, text.len) != @as(isize, @intCast(text.len))) return error.WriteFailed; + n += text.len; + } + } + defer _ = std.c.unlink(path); + const a = try Authority.init(gpa, .{ .cols = 120, .rows = 40, .file = path }, TestHost.host(), true); + defer a.deinit(); + _ = try a.unit(0, 0); + var unit_ns: [40]u64 = undefined; + var digest_ns: [40]u64 = undefined; + for ([_]bool{ false, true }) |on| { + walk.blob_words_on = on; + defer walk.blob_words_on = true; + walk.forgetAll(); + for (&unit_ns, &digest_ns, 0..) |*u, *d, k| { + // A key into the file: a unit as a replica is sent it. + const t0 = nowNs(); + _ = try a.local(.{ .key = .{ .cp = if (k % 2 == 0) 'j' else 'k', .text = if (k % 2 == 0) "j" else "k" } }); + u.* = nowNs() - t0; + const t1 = nowNs(); + _ = digestOf(a); + d.* = nowNs() - t1; + } + const us = spread(&unit_ns); + const ds = spread(&digest_ns); + std.debug.print("{d} MB file, blobs {s}: a unit p50 {d} us p99 {d} us; a digest p50 {d} us p99 {d} us\n", .{ mb, if (on) "hashed once" else "hashed whole", us[0] / 1000, us[1] / 1000, ds[0] / 1000, ds[1] / 1000 }); + } +} |
