diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/design.md | 73 | ||||
| -rw-r--r-- | docs/results/batch.json | 60 | ||||
| -rw-r--r-- | docs/results/fragmented.json | 60 | ||||
| -rw-r--r-- | docs/spec.md | 33 | ||||
| -rw-r--r-- | docs/validation.md | 73 |
5 files changed, 299 insertions, 0 deletions
diff --git a/docs/design.md b/docs/design.md new file mode 100644 index 0000000..ed9a66e --- /dev/null +++ b/docs/design.md @@ -0,0 +1,73 @@ +# API and ownership + +The public entry point is `src/root.zig`. `Msg` covers every base request and +response, excluding the reserved, nonexistent Terror. Decode returns borrowed +strings/data. Encode checks complete output size before writing; input payloads +may already occupy their exact final position in the output frame, but arbitrary +input/output overlap is not supported. Stat uses the protocol's inner length; +Rstat and Twstat add and validate the outer length separately. + +`Client` supports 16 ordinary outstanding requests and one additional flush. +Replies can arrive out of order. Tags remain reserved while flushes are pending, +even when the original request has already completed. Multiple flushes and +flushing a flush are accounted for. A completed result's slices remain valid +until the next `take()` or `hangup()`. `push()` only appends and never compacts. +Submit copies request payloads into the output buffer. An iounit returned by an +open/create is per fid; the caller must apply it when selecting atomic I/O sizes. +`maxRead()` and `maxWrite()` describe frame limits, not that per-fid guarantee. +Renegotiation requires an empty client request window. + +`Server` supports 64 ordinary outstanding requests and one additional flush. +Receive errors terminate the connection. A full request table produces NoTags; +output backpressure instead returns null without consuming the next request. +`receive()` yields at most one frame. `release()` ends its borrow and advances +input. A backend retaining any request data must copy it before release. +A successful reply copies output and frees the request tag. An output-capacity +error leaves that tag pending so the backend can drain and retry. + +The server validates transaction structure, not filesystem policy. Backends +own fid maps, authentication state, file handles, access modes, complete directory +records, atomic wstat, and cancellations. They must retire canceled work before +reusing its tag; a tag alone cannot distinguish a stale backend completion from a +new request. A version event requires backend cancellation and fid cleanup before +`negotiate()`. A partially sent response drains before a new version is accepted. +An Rflush promises no later reply to oldtag. Replying Rerror to a flush is refused. + +Caller-provided input/output buffers must be disjoint and remain stable for the +session. No protocol API takes an allocator. Server and client request tables are +fixed-size. Copies and work per frame are bounded by buffer size and fixed table +capacities; transports and application backends can have their own allocations. + +# Transports + +`transport.readFrame` and `writeFrame` adapt `std.Io.Reader` and `std.Io.Writer`. +The writer remains buffered until its owner flushes it. Reader errors leave the +stream unsuitable for further frame processing; close the connection. + +`transport.connect` and `listen` use `std.Io.net` for TCP and Unix streams. The +POSIX `connectFd`, `listenFd`, `acceptFd`, `read`, `write`, and `wait` APIs fit +caller-owned poll loops. They configure nonblocking descriptors and close-on-exec; +read/write return null for retry, and read returns zero for EOF. `wait` takes an +absolute monotonic deadline. The application closes descriptors, limits its +connections, selects addresses and deadlines, and manages Unix path permissions +and removal. No mounting or namespace policy is performed. + +`Quic(OpenSSL, alpn)` is an optional OpenSSL 3.6+ adapter with a single ordered +bidirectional stream per connection. It preserves the previous Pardes transport: +ephemeral self-signed certificates, no peer authentication. Applications needing +identity verification must provide that policy before using it across a trust +boundary. OpenSSL allocations and handshake costs are outside the allocation-free +protocol core. Accepted connections must close before their shared listener. + +# Pardes integration + +Pardes consumes the sibling package through build.zig.zon. Its `src/9p.zig` now +adapts filesystem requests and retains editor/GPIO capacities and permissions. +`src/9p_io.zig` retains mounting, discovery, the editor's event-loop scheduling, +connection limits, and error presentation. Protocol bytes, session validation, +and TCP/Unix/QUIC transport implementation come from cloud9. Pardes selects the +existing `pardes-9p` ALPN for compatibility. + +The separate `05-zig-p4` firmware build adds cloud9 to the GPIO application's +module map. UART hardware access remains in the board firmware. No hardware was +flashed by this change. diff --git a/docs/results/batch.json b/docs/results/batch.json new file mode 100644 index 0000000..b2d4d76 --- /dev/null +++ b/docs/results/batch.json @@ -0,0 +1,60 @@ +{ + "cloud9": { + "allocation_evidence": "no allocator or allocation calls in codec", + "bytes_in": 3803125, + "bytes_out": 3803125, + "codec_allocations": 0, + "codec_ns": 11364700, + "implementation": "cloud9", + "operations": 540000, + "read_calls": 54001, + "write_calls": 27000 + }, + "configuration": { + "chunk": 65536, + "frames": 27000, + "go": "go1.26.5-X:nodwarf5", + "platform": "linux/amd64", + "repeats": 20, + "rounds": 1000, + "seed": 4200 + }, + "correctness": "all generated base-9P2000 frames agree byte-for-byte", + "measurement_notes": [ + "Cloud9 codec timer excludes pipe I/O; Go timers include in-memory stream parsing and byte comparison. Do not treat these as equivalent throughput benchmarks.", + "Go allocations use runtime.MemStats deltas on a single-process run; cloud9 codec has no allocation API.", + "Cloud9 I/O counters are actual stdin/stdout syscalls; Go read counters are io.Reader calls, not syscalls.", + "These are wire-conformance probes, not differential filesystem-server semantics." + ], + "references": [ + { + "implementation": "9fans v0.0.7", + "operations": 540000, + "codec_ns": 116434888, + "codec_allocations": 2509437, + "allocated_bytes": 486133240, + "read_calls": 1080000, + "bytes": 76062500 + }, + { + "implementation": "go9p v1.18.0", + "operations": 540000, + "codec_ns": 94922755, + "codec_allocations": 3130709, + "allocated_bytes": 289069480, + "read_calls": 1080000, + "bytes": 76062500 + } + ], + "session": { + "bytes_read": 3156839, + "bytes_written": 646229, + "checks": "walk/open/write/read/stat/EOF/clunk, content oracle, fragmented requests and replies", + "read_calls": 454124, + "requests": 7003, + "rounds": 1000, + "seed": 4200, + "server": "go9p v1.18.0 StaticFile", + "write_calls": 217407 + } +} diff --git a/docs/results/fragmented.json b/docs/results/fragmented.json new file mode 100644 index 0000000..a49b23a --- /dev/null +++ b/docs/results/fragmented.json @@ -0,0 +1,60 @@ +{ + "cloud9": { + "allocation_evidence": "no allocator or allocation calls in codec", + "bytes_in": 369117, + "bytes_out": 369117, + "codec_allocations": 0, + "codec_ns": 1327036, + "implementation": "cloud9", + "operations": 54000, + "read_calls": 369118, + "write_calls": 369117 + }, + "configuration": { + "chunk": 1, + "frames": 2700, + "go": "go1.26.5-X:nodwarf5", + "platform": "linux/amd64", + "repeats": 20, + "rounds": 100, + "seed": 4201 + }, + "correctness": "all generated base-9P2000 frames agree byte-for-byte", + "measurement_notes": [ + "Cloud9 codec timer excludes pipe I/O; Go timers include in-memory stream parsing and byte comparison. Do not treat these as equivalent throughput benchmarks.", + "Go allocations use runtime.MemStats deltas on a single-process run; cloud9 codec has no allocation API.", + "Cloud9 I/O counters are actual stdin/stdout syscalls; Go read counters are io.Reader calls, not syscalls.", + "These are wire-conformance probes, not differential filesystem-server semantics." + ], + "references": [ + { + "implementation": "9fans v0.0.7", + "operations": 54000, + "codec_ns": 48042988, + "codec_allocations": 250519, + "allocated_bytes": 48088256, + "read_calls": 7382340, + "bytes": 7382340 + }, + { + "implementation": "go9p v1.18.0", + "operations": 54000, + "codec_ns": 43224680, + "codec_allocations": 312801, + "allocated_bytes": 28165920, + "read_calls": 7382340, + "bytes": 7382340 + } + ], + "session": { + "bytes_read": 282585, + "bytes_written": 63853, + "checks": "walk/open/write/read/stat/EOF/clunk, content oracle, fragmented requests and replies", + "read_calls": 40697, + "requests": 703, + "rounds": 100, + "seed": 4201, + "server": "go9p v1.18.0 StaticFile", + "write_calls": 21488 + } +} diff --git a/docs/spec.md b/docs/spec.md new file mode 100644 index 0000000..4537620 --- /dev/null +++ b/docs/spec.md @@ -0,0 +1,33 @@ +# Specification baseline + +The baseline is Plan 9's 9P2000 manual, read before defining the package boundary. +Reference implementations are comparison partners, not the specification. + +| Requirement | Primary source | Implementation | +|---|---|---| +| Little-endian framing, counted strings, tags, qids | [intro(9P)](https://9fans.github.io/plan9port/man/man9/intro.html) | `wire.zig` | +| First-message negotiation, NOTAG, msize, session reset | [version(9P)](https://9fans.github.io/plan9port/man/man9/version.html) | `Client`, `Server.negotiate`; backend releases fids | +| Authentication and attach | [attach(9P)](https://9fans.github.io/plan9port/man/man9/attach.html) | All wire/client messages; backend authentication | +| Tag reservation and cancellation completion | [flush(9P)](https://9fans.github.io/plan9port/man/man9/flush.html) | Client reservations; server Rflush retires oldtag | +| Walk bounds, cloning and partial walks | [walk(9P)](https://9fans.github.io/plan9port/man/man9/walk.html) | Codec and count checks; backend commits only full walks | +| Opening, creation, permissions and iounit | [open(9P)](https://9fans.github.io/plan9port/man/man9/open.html) | All wire/client messages; backend filesystem behavior | +| Read/write counts and directory records | [read(9P)](https://9fans.github.io/plan9port/man/man9/read.html) | Reply count checks; backend directory offsets and records | +| Fid release, including failed removal | [clunk(9P)](https://9fans.github.io/plan9port/man/man9/clunk.html), [remove(9P)](https://9fans.github.io/plan9port/man/man9/remove.html) | Client messages; backend releases handles | +| Double stat lengths and atomic metadata changes | [stat(9P)](https://9fans.github.io/plan9port/man/man9/stat.html) | `Stat`, wire length checks; backend atomic updates | +| Error replies | [error(9P)](https://9fans.github.io/plan9port/man/man9/error.html) | Counted errors, frame-size truncation, no Rerror for Tflush | + +The local 24-byte session floor and fixed request capacities are resource choices, +not additional wire requirements. TCP, Unix, and QUIC adapters carry unchanged +9P2000 frames. QUIC ALPN and authentication policy are not defined by 9P2000. + +[Tiger Style](https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TIGER_STYLE.md) +informs bounded memory, explicit ownership, assertions for internal invariants, +recoverable errors for external input, and deterministic tests. Zig standard +library conventions take precedence for names: functions use camelCase, values +use snake_case, and types use PascalCase. The code is formatted with `zig fmt`. + +The wire codec and initial client were extracted from Pardes and checked against +these rules. Cloud9 adds the reusable server connection, missing client +operations, cancellation bookkeeping, bounds checks, shared transports, and +independent conformance harness. Pardes regression tests remain with its adapter; +wire regression tests moved with the codec. diff --git a/docs/validation.md b/docs/validation.md new file mode 100644 index 0000000..acc35c3 --- /dev/null +++ b/docs/validation.md @@ -0,0 +1,73 @@ +# Validation and reproduction + +Run the commands in README.md. The Go modules in `test/differential/go.mod` and +`go.sum` pin the reference implementations and their test-only dependencies. +They are not library dependencies. The valid-traffic corpus exercises all 27 +message types, zero and nonzero payloads, 0–16 walk elements, UTF-8 names, stat +records, wstat sentinel values, and varying integer fields. Use different seeds +and `--chunk 1` for byte-at-a-time stream probes. Generated cases, their seed and +configuration, and reports are saved before a mismatch can terminate the run. + +The live session probe connects cloud9's client to go9p's StaticFile backend over +local pipes. A seeded in-memory content oracle checks writes at varying offsets, +reads, stat lengths, EOF, open/walk/clunk, and fid reuse. It does not claim coverage +of every reference-server filesystem policy. Pardes's separate Python 9P client +exercises the integrated cloud9 server over Unix, TCP, and optional QUIC. + +The standalone `zig build fuzz -- <seed> <iterations>` runner mutates frames only +for cloud9. It checks decode/re-encode identity and server pre-negotiation handling. +The native `zig build test --fuzz=10000` route is also present through std.testing, +but the installed Zig 0.16.0 build currently fails compiling its own fuzz test +runner due to incompatible StackTrace types. The standalone runner avoids that +toolchain failure; it is deterministic mutation testing, not coverage-guided. + +# Interpreting measurements + +Reports retain raw counts. Cloud9 codec timings exclude pipe I/O. Reference Go +measurements include in-memory io.Reader parsing and byte comparison. These are +not directly comparable end-to-end throughput numbers. Cloud9 syscall counts +cover one traversal of the corpus, while Go reader calls cover all repetitions. +Counts must be normalized before comparison, and an io.Reader call is not a +syscall. Byte totals distinguish corpus transport from repeated codec work. + +Go allocations are runtime.MemStats deltas across the measurement. Cloud9's zero +codec allocation count is structural evidence: no allocator or allocation calls +exist in that path. It is not a claim about process startup, std.Io, OpenSSL, +filesystem backends, or the test harness. There are no portable performance gates +or claims about kernel/disk performance in these reports. + +If this machine's /tmp quota is exhausted, set TMPDIR to an owned directory on a +filesystem with space. Unix socket tests need a short absolute path because of +sockaddr_un's path limit. No unrelated temporary files need to be removed. + +# Recorded run — 2026-09-14 + +Linux x86_64, Zig 0.16.0, OpenSSL 3.6.3, Go 1.26.5. These results describe this +checkout and machine, not a production-readiness or complete-conformance claim. + +| Check | Result | +|---|---| +| Cloud9 Debug tests | 24 protocol/session + 3 transport + 6 QUIC passed | +| Cloud9 ReleaseSafe tests | The same 33 tests passed | +| Deterministic mutation probes | 1,000,000 iterations, seed 4200; 442,898 accepted, 557,102 rejected | +| Valid wire differential, seed 4200 | 27,000 frames; 540,000 codec operations per implementation | +| Valid wire differential, seed 4201, chunk 1 | 2,700 frames; 54,000 codec operations per implementation | +| Live go9p server, seed 4200 | 7,003 requests passed | +| Live go9p server, seed 4201 | 703 requests passed | +| Pardes protocol/filesystem adapter tests | 31 passed | +| Pardes native transport/client tests (`9p-io-test`) | 8 passed both with and without QUIC | +| Pardes terminal build with QUIC | Built with Zig grammar enabled | +| Pardes dedicated Python Unix/TCP/QUIC integration | Passed IPv4/IPv6 and headless/TTY variants | +| ESP32-P4 GPIO image | Built successfully; no flash performed | + +Raw measurement reports: [batch](results/batch.json), +[byte-at-a-time](results/fragmented.json). + +The full Pardes editor integration script reached its syntax-style assertion and +failed because `fn` and `if` were not bold in the current theme. Syntax coloring +was enabled on the second run. This was left unchanged; the dedicated network +integration function was run independently and passed. A complete core-test run +passed earlier, but later full reruns encountered hard-coded /tmp shell-fixture +creation failures from the machine's quota. The final protocol and transport +checks were run separately. The `--fuzz` compiler-runner limitation is described +above. Firmware hardware behavior and non-Linux native transports were not run. |