diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/design.md | 7 | ||||
| -rw-r--r-- | docs/http.md | 216 | ||||
| -rw-r--r-- | docs/results/http-e2e.json | 45 | ||||
| -rw-r--r-- | docs/validation.md | 30 |
4 files changed, 298 insertions, 0 deletions
diff --git a/docs/design.md b/docs/design.md index ed9a66e..d8d55c4 100644 --- a/docs/design.md +++ b/docs/design.md @@ -52,6 +52,13 @@ absolute monotonic deadline. The application closes descriptors, limits its connections, selects addresses and deadlines, and manages Unix path permissions and removal. No mounting or namespace policy is performed. +`http.WebSocket` adapts established HTTP streams without owning their sockets +or buffers. `http.Client` negotiates HTTP/1.1 upgrades using `std.http.Client`, +including its certificate-verified HTTPS support. `http.Bridge` relays between +an accepted WebSocket and arbitrary standard readers/writers, including UART +adapters. The runnable application's web UI, device leases, deadlines, and +origin policy remain in `app/`. See [HTTP ownership and usage](http.md). + `Quic(OpenSSL, alpn)` is an optional OpenSSL 3.6+ adapter with a single ordered bidirectional stream per connection. It preserves the previous Pardes transport: ephemeral self-signed certificates, no peer authentication. Applications needing diff --git a/docs/http.md b/docs/http.md new file mode 100644 index 0000000..f486d52 --- /dev/null +++ b/docs/http.md @@ -0,0 +1,216 @@ +# 9P over HTTP + +cloud9 provides a reusable HTTP transport and a standalone browser gateway. +The transport carries unchanged base 9P2000 messages. Filesystem permissions, +mounting, UART configuration, and board drivers remain with their applications. + +```mermaid +flowchart LR + Browser[Browser: cloud9 WASM client] -->|WebSocket / HTTPS| Gateway[HTTP gateway] + Native[Native cloud9 client] -->|WebSocket / HTTPS| Gateway + Gateway -->|TCP or Unix stream| Server[9P server] + Gateway -->|Serial byte stream| Board[9P firmware / ESP32] +``` + +## Run + +```sh +zig build -Doptimize=ReleaseSafe +./zig-out/bin/cloud9-http --upstream tcp:127.0.0.1:564 +# Or run directly: +zig build serve -Doptimize=ReleaseSafe -- --upstream unix:/path/to/9p.sock +``` + +Open **http://127.0.0.1:8080**. Files load automatically. +The upstream must already be running: the example assumes a 9P server on port +564. For Pardes, pass its actual 9P Unix socket with `--upstream unix:PATH`. +If HTTP port 8080 is occupied, add `--listen 127.0.0.1:0`; the operating system +selects a free port and the gateway prints the URL to open. + +The binary embeds HTML, CSS, JavaScript, and WebAssembly; it can run from any +working directory. `zig build web` also installs the separate assets into +`zig-out/web` for applications that host their own frontend. Its layout is +`index.html` plus assets under `_cloud9/`. A custom host must serve `index.html` +for file routes, reserve `/_cloud9/` for the assets and endpoints, and forward +WebSocket upgrades at `/_cloud9/9p`. + +The browser uses a compact cgit-inspired table with modes, names, sizes, and +breadcrumb links. File URLs can be bookmarked, reloaded, or opened in new tabs; +browser back/forward navigation works. File URLs use ordinary paths such as +`/self/listeners` and `/notes/caf%C3%A9.txt`, with each component encoded separately. +The gateway serves the browser application at these URLs; the client resolves the +path in the upstream 9P filesystem. They are browser views, not raw-file HTTP +responses: missing files are reported by the page after the 9P lookup. + +The literal `/_cloud9/` prefix is reserved for assets, configuration, and the +WebSocket endpoint. This leaves names such as `/app.mjs`, `/config.json`, and +`/9p` available to the upstream filesystem. If the upstream itself has a root +entry named `_cloud9`, generated links escape its leading underscore as +`/%5Fcloud9/`. Percent escapes are decoded exactly once; encoded slashes, NULs, +and malformed escapes are rejected. Reverse proxies must preserve the encoded +path when forwarding requests. + +The browser establishes 9P automatically on load +and reconnects when needed, preserving unsaved edits across connection expiry. +Interrupted writes are not automatically replayed. The default attach name is +`user` and the default export name is empty; set `--user NAME` and `--tree NAME` +on the gateway when the upstream requires different values. These are 9P attach +parameters, not a browser login. The `/_cloud9/config.json` endpoint supplies these public +defaults to the page; they do not restrict other clients' attach requests. + +The browser lists directories, follows paths longer than 16 components, previews +UTF-8 text, downloads binary files, and saves edits to existing files. Saving uses +OWRITE|OTRUNC and is not atomic. Empty saves truncate the file. It respects both +negotiated msize and each open's iounit, handles short writes, and clunks temporary +fids even after partial walks or filesystem errors. Views and writes are limited +to 8 MiB. The UI does not perform a 9P authentication exchange; it attaches using +NOFID, so authenticated filesystems need their own client authentication flow. + +Options: + +| Option | Default | Meaning | +| --- | --- | --- | +| `--listen IP:PORT` | `127.0.0.1:8080` | HTTP listener; IPv4 or bracketed IPv6 | +| `--upstream tcp:IP:PORT` | `tcp:127.0.0.1:564` | A new 9P connection per WebSocket | +| `--upstream unix:PATH` | — | A new Unix connection per WebSocket | +| `--upstream file:PATH` | — | An already configured duplex device, one session at a time | +| `--origin SCHEME://HOST[:PORT]` | Listener origin | Public origin when a reverse proxy serves the gateway | +| `--user NAME` | `user` | Browser's 9P attach user (`uname`), at most 256 bytes | +| `--tree NAME` | empty | Browser's named export (`aname`), at most 256 bytes | +| `--timeout-ms N` | `300000` | Maximum connection lifetime, including HTTP headers; 0 disables it | + +The gateway bounds concurrent HTTP/WebSocket connections at 32, HTTP headers at +8 KiB, and 9P frames at 64 KiB. Excess connections close. A connection ending or +expiring cancels both relay directions before releasing buffers and descriptors. +Each network connection has a separate upstream fid/tag space. A device lease +prevents multiple clients from mixing transactions on one physical UART stream. +A new serial session starts with Tversion; the device owner is responsible for +link reset/recovery if an interrupted physical link leaves stale bytes in transit. + +For a configured serial device: + +```sh +# Set raw mode, baud rate, and flow control with your platform's serial tools. +./zig-out/bin/cloud9-http --upstream file:/dev/ttyUSB0 +``` + +The serial path transports 9P bytes; it does not interpret ESP32 boot logs or +configure pins, UART framing, baud rates, or firmware. A custom embedded runtime +can instead supply its own `std.Io.Reader` and `std.Io.Writer` to the library relay. + +## HTTPS and QUIC + +The native connector uses `std.http.Client` for HTTP/1.1 or certificate-verified +HTTPS. The browser uses its own WebSocket/TLS stack. For the runnable gateway, +terminate HTTPS in a reverse proxy that forwards WebSocket upgrades, and set +`--origin https://files.example` to the public origin. Serve assets and `/_cloud9/9p` +under that same origin. Host must match the listener or public authority; the +upgrade Origin must match the configured origin exactly. Gateway access grants +the upstream's existing 9P rights; identity/access policy belongs to the proxy +or filesystem server. The configured attach name is not proof of identity. + +HTTP/3 is **not implemented by this connector or binary**. HTTPS alone does not +select QUIC. WebSockets over HTTP/2 and HTTP/3 require Extended CONNECT support +in the HTTP stack ([RFC 8441](https://www.rfc-editor.org/rfc/rfc8441.html), +[RFC 9220](https://www.rfc-editor.org/rfc/rfc9220.html)). Such a stack can provide +an established stream to `http.WebSocket`; its handshake and stream lifecycle +remain the stack's responsibility. cloud9's existing direct `Quic` transport is +separate from HTTP/3. + +## Library API + +`cloud9.http.WebSocket` operates on caller-owned standard readers/writers, with +no allocation or socket dependency. A complete binary WebSocket message contains +one complete 9P frame. It handles masked client frames, fragmented messages, and +interleaved ping/pong/close controls. Text messages and WebSocket extensions are +outside this transport contract. Its framing follows +[RFC 6455](https://www.rfc-editor.org/rfc/rfc6455.html); the payload is the existing +[9P2000 format](https://9fans.github.io/plan9port/man/man9/intro.html). + +`receive(buffer)` returns borrowed bytes; keep that buffer stable across control +messages during a fragmented message. `send` flushes its writer; +`sendUnflushed` lets a layered connection control flushing. Client-role callers +must supply a fresh unpredictable mask. A framing/I/O error terminates the +transport. One reader and one writer may run concurrently, but writes, including +control replies, must be serialized. + +`cloud9.http.Client.connect(&http_client, url, origin)` negotiates and validates +an HTTP upgrade. The HTTP client and URL bytes must outlive the tunnel. It uses +the HTTP client's allocator, CA bundle, and I/O provider; it does not disable +certificate verification or follow redirects. `deinit()` closes the connection +instead of returning the upgraded stream to the HTTP pool. + +```zig +var http_client: std.http.Client = .{ .allocator = allocator, .io = io }; +defer http_client.deinit(); +var tunnel = try cloud9.http.Client.connect( + &http_client, + "https://files.example/_cloud9/9p", + "https://files.example", +); +defer tunnel.deinit(); + +// Drive the existing cloud9.Client state machine as with any other transport. +_ = try client.submit(.{ .version = .{} }); +try tunnel.send(client.output()); +client.wrote(client.output().len); +const reply = try tunnel.receive(frame_buffer); +if (client.push(reply) != reply.len) return error.InputFull; +const done = client.take() orelse return error.MissingReply; +``` + +`Client.receive` is a serial convenience method that answers control frames. For +concurrent drivers, use `socket.receive` and coordinate all writes, including +TLS connection flushing. The 9P client still owns tag matching, negotiated size, +reply validation, and flush semantics. Split batched client output at 9P frame +boundaries before sending individual WebSocket messages. + +For a server, call `http.accept(&request)` after checking route, Host, Origin, +and application authorization. Supply the resulting WebSocket, upstream standard +reader/writer, two disjoint frame buffers, and frame limit to `http.Bridge`, then +call `run(io, timeout)`. The relay forwards requests and replies concurrently, +including overlapping tags and delayed replies. It performs framing and direction +checks; it does not allocate fids, mount filesystems, or implement a filesystem. +Each Bridge value runs once. Timeout and cancellation stop both pumps before +returning. The caller owns and closes the underlying streams. + +`app/client.zig` is a browser ABI around the same `cloud9.Client` and `Stat` +implementations. Its WASM memory is fixed at 1 MiB and it imports no host +functions. `app/web/client.mjs` supplies asynchronous WebSocket I/O and file +operations. Each browser Client owns a separate WASM instance. The JavaScript +contains no 9P encoder or decoder. + +## End-to-end tests + +```sh +zig build test http-test transport-test +zig build e2e -Doptimize=ReleaseSafe +# Alternate Chromium executable: +CLOUD9_CHROME=/path/to/chromium zig build e2e -Doptimize=ReleaseSafe +``` + +E2E requires Node 22+ (including its global WebSocket), Go, Chromium, OpenSSL, +and Python 3 on Linux. It builds the pinned go9p v1.18.0 fixture from the existing +test-only Go module. No npm packages, public servers, physical boards, or mounted +filesystems are used. Chrome runs headless against loopback; test TLS uses a +fresh local certificate explicitly trusted by the native client. + +9P itself has connection-scoped sessions: Tversion negotiates/resets the session, +Tattach establishes a root fid, and later requests refer to fids retained by the +server. The page handles this state without presenting connection controls. + +The suite drives actual DOM controls and the shipped WASM in Chromium. It checks +automatic startup, configured attach defaults, clean file URLs and reloads, +encoded filenames, collisions with asset names, invalid path encoding, back/forward +navigation, reconnecting with unsaved edits, paged directories, UTF-8 names, literal HTML-like names, text editing and +reconnection, byte-exact multi-frame binary reads/writes and browser downloads, mobile layout, +empty truncation, +18-component walks, concurrent sessions, queued operations, and recovery after +Rerror. The native client verifies HTTP, trusted HTTPS, rejection of untrusted +certificates and hostname mismatches, overlapping requests, Unix upstreams, and +a raw PTY serial path. It also checks serial lease release and connection deadlines. +The reference server deliberately fragments reads and writes. Framing unit tests +cover length boundaries, one-byte I/O, interleaved controls, invalid frames, and +upgrade nonce validation, using only cloud9 for invalid-input checks. + +Each run leaves a JSON result and browser screenshot in `.zig-cache/e2e-*/`. diff --git a/docs/results/http-e2e.json b/docs/results/http-e2e.json new file mode 100644 index 0000000..028002e --- /dev/null +++ b/docs/results/http-e2e.json @@ -0,0 +1,45 @@ +{ + "browser": "Chromium", + "reference": "go9p v1.18.0", + "wasmHostImports": 0, + "checks": { + "binary": 180000, + "written": 150000, + "queued": 2, + "independent": "Hello from 9P.\n" + }, + "passed": [ + "clean path links and reloads", + "encoded filename round trips", + "gateway asset name collisions", + "escaped gateway-prefix file", + "invalid path encoding rejected", + "automatic connection", + "bookmarkable file URLs", + "browser back and forward", + "automatic reconnect preserves edits", + "configured attach defaults", + "directory paging", + "UTF-8 paths and contents", + "text save and reconnect", + "binary read/write", + "browser download", + "mobile layout", + "truncate to empty", + "18-component walk", + "concurrent sessions", + "queued operations", + "Rerror recovery", + "HTTP routing and origin policy", + "native HTTP", + "native verified HTTPS", + "untrusted certificate rejection", + "hostname mismatch rejection", + "Unix upstream", + "PTY serial upstream", + "serial lease released after disconnect", + "connection deadline", + "overlapping native requests", + "fragmented upstream I/O" + ] +} diff --git a/docs/validation.md b/docs/validation.md index acc35c3..98b73d6 100644 --- a/docs/validation.md +++ b/docs/validation.md @@ -71,3 +71,33 @@ passed earlier, but later full reruns encountered hard-coded /tmp shell-fixture creation failures from the machine's quota. The final protocol and transport checks were run separately. The `--fuzz` compiler-runner limitation is described above. Firmware hardware behavior and non-Linux native transports were not run. + +## HTTP transport and browser gateway (2026-09-14) + +`zig build test http-test transport-test` passed **33 tests** in both Debug and +ReleaseSafe: 24 protocol/session, 6 HTTP framing/handshake, and 3 stream transport +tests. `zig build e2e -Doptimize=ReleaseSafe` passed **22 scenarios**, using actual +Chromium, the shipped WASM, native cloud9 clients, and a pinned local go9p server. +Coverage includes binary downloads, mobile layout, multi-frame reads/writes, +verified HTTPS and certificate rejection, concurrent tags, Unix sockets, PTY +serial transport, serial lease cleanup, and connection deadlines. The serial +fixture is a pseudo-terminal; no physical UART or ESP32 hardware was exercised. +HTTP/3 is not implemented or tested. + +The [HTTP E2E report](results/http-e2e.json) records the scenarios and byte counts. +The run's screenshots and TLS fixtures remain in `.zig-cache/e2e-3HAs6G/`. +`zig build -Doptimize=ReleaseSafe` installs the standalone `cloud9-http` binary; +`zig build web -Doptimize=ReleaseSafe` also installs the separate browser assets. +See [HTTP usage and ownership](http.md) for the reusable API and runnable gateway. + +The cgit-inspired UI update passed **27 E2E scenarios**, adding automatic attach, +configured user/export defaults, bookmarkable file URLs, browser back/forward, +and automatic reconnection after expiry while preserving unsaved edits. The +latest screenshots are in `.zig-cache/e2e-IYt96E/` and the tracked HTTP report +has been updated. + +The clean-URL update passed **32 E2E scenarios**. File routes now occupy ordinary +URL paths and gateway endpoints use `/_cloud9/`. Added checks cover per-component +encoding, direct reloads, filenames matching asset names, an upstream `_cloud9` +directory, and invalid-URL rejection with navigation recovery. Latest artifacts: +`.zig-cache/e2e-YIV80o/`. Both the embedded binary and standalone web assets built. |