diff options
Diffstat (limited to 'docs/validation.md')
| -rw-r--r-- | docs/validation.md | 73 |
1 files changed, 73 insertions, 0 deletions
diff --git a/docs/validation.md b/docs/validation.md new file mode 100644 index 0000000..acc35c3 --- /dev/null +++ b/docs/validation.md @@ -0,0 +1,73 @@ +# Validation and reproduction + +Run the commands in README.md. The Go modules in `test/differential/go.mod` and +`go.sum` pin the reference implementations and their test-only dependencies. +They are not library dependencies. The valid-traffic corpus exercises all 27 +message types, zero and nonzero payloads, 0–16 walk elements, UTF-8 names, stat +records, wstat sentinel values, and varying integer fields. Use different seeds +and `--chunk 1` for byte-at-a-time stream probes. Generated cases, their seed and +configuration, and reports are saved before a mismatch can terminate the run. + +The live session probe connects cloud9's client to go9p's StaticFile backend over +local pipes. A seeded in-memory content oracle checks writes at varying offsets, +reads, stat lengths, EOF, open/walk/clunk, and fid reuse. It does not claim coverage +of every reference-server filesystem policy. Pardes's separate Python 9P client +exercises the integrated cloud9 server over Unix, TCP, and optional QUIC. + +The standalone `zig build fuzz -- <seed> <iterations>` runner mutates frames only +for cloud9. It checks decode/re-encode identity and server pre-negotiation handling. +The native `zig build test --fuzz=10000` route is also present through std.testing, +but the installed Zig 0.16.0 build currently fails compiling its own fuzz test +runner due to incompatible StackTrace types. The standalone runner avoids that +toolchain failure; it is deterministic mutation testing, not coverage-guided. + +# Interpreting measurements + +Reports retain raw counts. Cloud9 codec timings exclude pipe I/O. Reference Go +measurements include in-memory io.Reader parsing and byte comparison. These are +not directly comparable end-to-end throughput numbers. Cloud9 syscall counts +cover one traversal of the corpus, while Go reader calls cover all repetitions. +Counts must be normalized before comparison, and an io.Reader call is not a +syscall. Byte totals distinguish corpus transport from repeated codec work. + +Go allocations are runtime.MemStats deltas across the measurement. Cloud9's zero +codec allocation count is structural evidence: no allocator or allocation calls +exist in that path. It is not a claim about process startup, std.Io, OpenSSL, +filesystem backends, or the test harness. There are no portable performance gates +or claims about kernel/disk performance in these reports. + +If this machine's /tmp quota is exhausted, set TMPDIR to an owned directory on a +filesystem with space. Unix socket tests need a short absolute path because of +sockaddr_un's path limit. No unrelated temporary files need to be removed. + +# Recorded run — 2026-09-14 + +Linux x86_64, Zig 0.16.0, OpenSSL 3.6.3, Go 1.26.5. These results describe this +checkout and machine, not a production-readiness or complete-conformance claim. + +| Check | Result | +|---|---| +| Cloud9 Debug tests | 24 protocol/session + 3 transport + 6 QUIC passed | +| Cloud9 ReleaseSafe tests | The same 33 tests passed | +| Deterministic mutation probes | 1,000,000 iterations, seed 4200; 442,898 accepted, 557,102 rejected | +| Valid wire differential, seed 4200 | 27,000 frames; 540,000 codec operations per implementation | +| Valid wire differential, seed 4201, chunk 1 | 2,700 frames; 54,000 codec operations per implementation | +| Live go9p server, seed 4200 | 7,003 requests passed | +| Live go9p server, seed 4201 | 703 requests passed | +| Pardes protocol/filesystem adapter tests | 31 passed | +| Pardes native transport/client tests (`9p-io-test`) | 8 passed both with and without QUIC | +| Pardes terminal build with QUIC | Built with Zig grammar enabled | +| Pardes dedicated Python Unix/TCP/QUIC integration | Passed IPv4/IPv6 and headless/TTY variants | +| ESP32-P4 GPIO image | Built successfully; no flash performed | + +Raw measurement reports: [batch](results/batch.json), +[byte-at-a-time](results/fragmented.json). + +The full Pardes editor integration script reached its syntax-style assertion and +failed because `fn` and `if` were not bold in the current theme. Syntax coloring +was enabled on the second run. This was left unchanged; the dedicated network +integration function was run independently and passed. A complete core-test run +passed earlier, but later full reruns encountered hard-coded /tmp shell-fixture +creation failures from the machine's quota. The final protocol and transport +checks were run separately. The `--fuzz` compiler-runner limitation is described +above. Firmware hardware behavior and non-Linux native transports were not run. |