diff options
Diffstat (limited to 'docs/http.md')
| -rw-r--r-- | docs/http.md | 216 |
1 files changed, 216 insertions, 0 deletions
diff --git a/docs/http.md b/docs/http.md new file mode 100644 index 0000000..f486d52 --- /dev/null +++ b/docs/http.md @@ -0,0 +1,216 @@ +# 9P over HTTP + +cloud9 provides a reusable HTTP transport and a standalone browser gateway. +The transport carries unchanged base 9P2000 messages. Filesystem permissions, +mounting, UART configuration, and board drivers remain with their applications. + +```mermaid +flowchart LR + Browser[Browser: cloud9 WASM client] -->|WebSocket / HTTPS| Gateway[HTTP gateway] + Native[Native cloud9 client] -->|WebSocket / HTTPS| Gateway + Gateway -->|TCP or Unix stream| Server[9P server] + Gateway -->|Serial byte stream| Board[9P firmware / ESP32] +``` + +## Run + +```sh +zig build -Doptimize=ReleaseSafe +./zig-out/bin/cloud9-http --upstream tcp:127.0.0.1:564 +# Or run directly: +zig build serve -Doptimize=ReleaseSafe -- --upstream unix:/path/to/9p.sock +``` + +Open **http://127.0.0.1:8080**. Files load automatically. +The upstream must already be running: the example assumes a 9P server on port +564. For Pardes, pass its actual 9P Unix socket with `--upstream unix:PATH`. +If HTTP port 8080 is occupied, add `--listen 127.0.0.1:0`; the operating system +selects a free port and the gateway prints the URL to open. + +The binary embeds HTML, CSS, JavaScript, and WebAssembly; it can run from any +working directory. `zig build web` also installs the separate assets into +`zig-out/web` for applications that host their own frontend. Its layout is +`index.html` plus assets under `_cloud9/`. A custom host must serve `index.html` +for file routes, reserve `/_cloud9/` for the assets and endpoints, and forward +WebSocket upgrades at `/_cloud9/9p`. + +The browser uses a compact cgit-inspired table with modes, names, sizes, and +breadcrumb links. File URLs can be bookmarked, reloaded, or opened in new tabs; +browser back/forward navigation works. File URLs use ordinary paths such as +`/self/listeners` and `/notes/caf%C3%A9.txt`, with each component encoded separately. +The gateway serves the browser application at these URLs; the client resolves the +path in the upstream 9P filesystem. They are browser views, not raw-file HTTP +responses: missing files are reported by the page after the 9P lookup. + +The literal `/_cloud9/` prefix is reserved for assets, configuration, and the +WebSocket endpoint. This leaves names such as `/app.mjs`, `/config.json`, and +`/9p` available to the upstream filesystem. If the upstream itself has a root +entry named `_cloud9`, generated links escape its leading underscore as +`/%5Fcloud9/`. Percent escapes are decoded exactly once; encoded slashes, NULs, +and malformed escapes are rejected. Reverse proxies must preserve the encoded +path when forwarding requests. + +The browser establishes 9P automatically on load +and reconnects when needed, preserving unsaved edits across connection expiry. +Interrupted writes are not automatically replayed. The default attach name is +`user` and the default export name is empty; set `--user NAME` and `--tree NAME` +on the gateway when the upstream requires different values. These are 9P attach +parameters, not a browser login. The `/_cloud9/config.json` endpoint supplies these public +defaults to the page; they do not restrict other clients' attach requests. + +The browser lists directories, follows paths longer than 16 components, previews +UTF-8 text, downloads binary files, and saves edits to existing files. Saving uses +OWRITE|OTRUNC and is not atomic. Empty saves truncate the file. It respects both +negotiated msize and each open's iounit, handles short writes, and clunks temporary +fids even after partial walks or filesystem errors. Views and writes are limited +to 8 MiB. The UI does not perform a 9P authentication exchange; it attaches using +NOFID, so authenticated filesystems need their own client authentication flow. + +Options: + +| Option | Default | Meaning | +| --- | --- | --- | +| `--listen IP:PORT` | `127.0.0.1:8080` | HTTP listener; IPv4 or bracketed IPv6 | +| `--upstream tcp:IP:PORT` | `tcp:127.0.0.1:564` | A new 9P connection per WebSocket | +| `--upstream unix:PATH` | — | A new Unix connection per WebSocket | +| `--upstream file:PATH` | — | An already configured duplex device, one session at a time | +| `--origin SCHEME://HOST[:PORT]` | Listener origin | Public origin when a reverse proxy serves the gateway | +| `--user NAME` | `user` | Browser's 9P attach user (`uname`), at most 256 bytes | +| `--tree NAME` | empty | Browser's named export (`aname`), at most 256 bytes | +| `--timeout-ms N` | `300000` | Maximum connection lifetime, including HTTP headers; 0 disables it | + +The gateway bounds concurrent HTTP/WebSocket connections at 32, HTTP headers at +8 KiB, and 9P frames at 64 KiB. Excess connections close. A connection ending or +expiring cancels both relay directions before releasing buffers and descriptors. +Each network connection has a separate upstream fid/tag space. A device lease +prevents multiple clients from mixing transactions on one physical UART stream. +A new serial session starts with Tversion; the device owner is responsible for +link reset/recovery if an interrupted physical link leaves stale bytes in transit. + +For a configured serial device: + +```sh +# Set raw mode, baud rate, and flow control with your platform's serial tools. +./zig-out/bin/cloud9-http --upstream file:/dev/ttyUSB0 +``` + +The serial path transports 9P bytes; it does not interpret ESP32 boot logs or +configure pins, UART framing, baud rates, or firmware. A custom embedded runtime +can instead supply its own `std.Io.Reader` and `std.Io.Writer` to the library relay. + +## HTTPS and QUIC + +The native connector uses `std.http.Client` for HTTP/1.1 or certificate-verified +HTTPS. The browser uses its own WebSocket/TLS stack. For the runnable gateway, +terminate HTTPS in a reverse proxy that forwards WebSocket upgrades, and set +`--origin https://files.example` to the public origin. Serve assets and `/_cloud9/9p` +under that same origin. Host must match the listener or public authority; the +upgrade Origin must match the configured origin exactly. Gateway access grants +the upstream's existing 9P rights; identity/access policy belongs to the proxy +or filesystem server. The configured attach name is not proof of identity. + +HTTP/3 is **not implemented by this connector or binary**. HTTPS alone does not +select QUIC. WebSockets over HTTP/2 and HTTP/3 require Extended CONNECT support +in the HTTP stack ([RFC 8441](https://www.rfc-editor.org/rfc/rfc8441.html), +[RFC 9220](https://www.rfc-editor.org/rfc/rfc9220.html)). Such a stack can provide +an established stream to `http.WebSocket`; its handshake and stream lifecycle +remain the stack's responsibility. cloud9's existing direct `Quic` transport is +separate from HTTP/3. + +## Library API + +`cloud9.http.WebSocket` operates on caller-owned standard readers/writers, with +no allocation or socket dependency. A complete binary WebSocket message contains +one complete 9P frame. It handles masked client frames, fragmented messages, and +interleaved ping/pong/close controls. Text messages and WebSocket extensions are +outside this transport contract. Its framing follows +[RFC 6455](https://www.rfc-editor.org/rfc/rfc6455.html); the payload is the existing +[9P2000 format](https://9fans.github.io/plan9port/man/man9/intro.html). + +`receive(buffer)` returns borrowed bytes; keep that buffer stable across control +messages during a fragmented message. `send` flushes its writer; +`sendUnflushed` lets a layered connection control flushing. Client-role callers +must supply a fresh unpredictable mask. A framing/I/O error terminates the +transport. One reader and one writer may run concurrently, but writes, including +control replies, must be serialized. + +`cloud9.http.Client.connect(&http_client, url, origin)` negotiates and validates +an HTTP upgrade. The HTTP client and URL bytes must outlive the tunnel. It uses +the HTTP client's allocator, CA bundle, and I/O provider; it does not disable +certificate verification or follow redirects. `deinit()` closes the connection +instead of returning the upgraded stream to the HTTP pool. + +```zig +var http_client: std.http.Client = .{ .allocator = allocator, .io = io }; +defer http_client.deinit(); +var tunnel = try cloud9.http.Client.connect( + &http_client, + "https://files.example/_cloud9/9p", + "https://files.example", +); +defer tunnel.deinit(); + +// Drive the existing cloud9.Client state machine as with any other transport. +_ = try client.submit(.{ .version = .{} }); +try tunnel.send(client.output()); +client.wrote(client.output().len); +const reply = try tunnel.receive(frame_buffer); +if (client.push(reply) != reply.len) return error.InputFull; +const done = client.take() orelse return error.MissingReply; +``` + +`Client.receive` is a serial convenience method that answers control frames. For +concurrent drivers, use `socket.receive` and coordinate all writes, including +TLS connection flushing. The 9P client still owns tag matching, negotiated size, +reply validation, and flush semantics. Split batched client output at 9P frame +boundaries before sending individual WebSocket messages. + +For a server, call `http.accept(&request)` after checking route, Host, Origin, +and application authorization. Supply the resulting WebSocket, upstream standard +reader/writer, two disjoint frame buffers, and frame limit to `http.Bridge`, then +call `run(io, timeout)`. The relay forwards requests and replies concurrently, +including overlapping tags and delayed replies. It performs framing and direction +checks; it does not allocate fids, mount filesystems, or implement a filesystem. +Each Bridge value runs once. Timeout and cancellation stop both pumps before +returning. The caller owns and closes the underlying streams. + +`app/client.zig` is a browser ABI around the same `cloud9.Client` and `Stat` +implementations. Its WASM memory is fixed at 1 MiB and it imports no host +functions. `app/web/client.mjs` supplies asynchronous WebSocket I/O and file +operations. Each browser Client owns a separate WASM instance. The JavaScript +contains no 9P encoder or decoder. + +## End-to-end tests + +```sh +zig build test http-test transport-test +zig build e2e -Doptimize=ReleaseSafe +# Alternate Chromium executable: +CLOUD9_CHROME=/path/to/chromium zig build e2e -Doptimize=ReleaseSafe +``` + +E2E requires Node 22+ (including its global WebSocket), Go, Chromium, OpenSSL, +and Python 3 on Linux. It builds the pinned go9p v1.18.0 fixture from the existing +test-only Go module. No npm packages, public servers, physical boards, or mounted +filesystems are used. Chrome runs headless against loopback; test TLS uses a +fresh local certificate explicitly trusted by the native client. + +9P itself has connection-scoped sessions: Tversion negotiates/resets the session, +Tattach establishes a root fid, and later requests refer to fids retained by the +server. The page handles this state without presenting connection controls. + +The suite drives actual DOM controls and the shipped WASM in Chromium. It checks +automatic startup, configured attach defaults, clean file URLs and reloads, +encoded filenames, collisions with asset names, invalid path encoding, back/forward +navigation, reconnecting with unsaved edits, paged directories, UTF-8 names, literal HTML-like names, text editing and +reconnection, byte-exact multi-frame binary reads/writes and browser downloads, mobile layout, +empty truncation, +18-component walks, concurrent sessions, queued operations, and recovery after +Rerror. The native client verifies HTTP, trusted HTTPS, rejection of untrusted +certificates and hostname mismatches, overlapping requests, Unix upstreams, and +a raw PTY serial path. It also checks serial lease release and connection deadlines. +The reference server deliberately fragments reads and writes. Framing unit tests +cover length boundaries, one-byte I/O, interleaved controls, invalid frames, and +upgrade nonce validation, using only cloud9 for invalid-input checks. + +Each run leaves a JSON result and browser screenshot in `.zig-cache/e2e-*/`. |