path: root/docs/http.md

# 9P over HTTP

cloud9 provides a reusable HTTP transport and a standalone browser gateway.
The transport carries unchanged base 9P2000 messages. Filesystem permissions,
mounting, UART configuration, and board drivers remain with their applications.

`9web` is a **multiplexer**: it holds one upstream 9P connection (a single
`cloud9.Client`, all sixteen tags) and fans it out to any number of downstream
sessions — the browser WASM client over a WebSocket, plain 9P clients over an
optional TCP/Unix listener (a mux like plan9port's 9pserve), and an HTTP view
of the tree under `/fs/`. Each downstream keeps its own fid space; two
downstreams never share an upstream fid. On an upstream failure the gateway
reconnects and invalidates downstream fids (their next request answers EIO
until they re-attach).

```mermaid
flowchart LR
    Browser[Browser: cloud9 WASM client] -->|WebSocket| Gateway[9web mux]
    Native[Native cloud9 client] -->|WebSocket / HTTPS| Gateway
    Curl[curl / fetch / SSE] -->|HTTP /fs| Gateway
    P9[9ns / plan9port 9p] -->|TCP / Unix 9P| Gateway
    Gateway -->|one shared TCP / Unix / serial connection| Server[9P server]
```

## The multiplexer

Downstream requests are forwarded upstream on remapped tags with their fids
remapped into the shared upstream fid space; upstream replies are routed back to
the originating downstream by tag. `Tversion` is answered locally (the upstream
session is negotiated once at connect); `Tattach` is forwarded so every
downstream gets its own upstream tree root; `Tflush` is forwarded as `Tflush`.
Frames are forwarded unchanged, so directory reads, qids, iounits and stat
fields keep the upstream's exact values — the mux does not re-derive them.

This is deliberately a frame-level remux rather than an `fs.Server`/`serve.Runner`
backend: the file-server engine re-decomposes each request into filesystem
operations and re-encodes directories with synthetic stats and an entry-index
cursor, which would lose the upstream's real directory stats and complicate the
readdir byte offset. Forwarding requests unchanged is exactly the "one upstream,
many downstreams" contract. See [design.md](design.md#multiplexer-9web).

The tables are comptime-sized and nothing allocates per request: the shared
upstream has one reader task and any number of downstream forwarder tasks
serialized by one mutex, sixteen ordinary tags plus one flush tag, a fixed
upstream fid pool, and a bounded downstream-connection table. When all sixteen
upstream tags are in flight, further downstream requests wait for one to free.

## The HTTP view of the tree

`/fs/<path>` maps HTTP onto the 9P tree — file operations, not verbs:

| Request | 9P | Result |
| --- | --- | --- |
| `GET /fs/<file>` | walk, open, read | bytes; `Content-Type` sniffed (`text/plain` or `application/octet-stream`); `Range` honored (206) |
| `GET /fs/<file>?follow=1` | walk, open, blocking reads | `text/event-stream`; each read that returns data is one SSE `data:` event; also triggered by `Accept: text/event-stream` |
| `GET /fs/<dir>` | walk, open, read | JSON array of `{name, dir, length, mode, mtime, qid}`; an HTML listing with `Accept: text/html` |
| `HEAD /fs/<path>` | walk, stat | headers: `content-length`, `x-9p-qid`, `x-9p-mode`, `x-9p-mtime` |
| `PUT /fs/<file>` | walk or create, open, write | write; creates if missing, truncates unless `?append=1` |
| `PUT /fs/<dir>/` | walk, create | create a directory (trailing slash) |
| `DELETE /fs/<path>` | walk, remove | remove |

A 9P `Rerror` maps to a status with the ename in the body: *no such
file* → 404, *permission* → 403, *exists* / *not empty* / *is a directory* →
409, otherwise 500. Path components are percent-decoded once; encoded slashes,
NULs and malformed escapes are rejected. `..` is refused.

Verify against a running `9proc-demo` upstream:

```sh
9proc-demo --unix /tmp/up.sock &
9web --listen 127.0.0.1:8080 --upstream unix:/tmp/up.sock
curl -s localhost:8080/fs/                       # directory JSON
curl -s localhost:8080/fs/runtime/fn/now         # a live file
curl -s -X PUT --data-binary hi localhost:8080/fs/scratch/x   # write
curl -N localhost:8080/fs/self/log               # follow a blocking file (Pardes)
```

## Self-introspection

`--probe unix:PATH` (off by default) embeds [9proc](../9proc/README.md) and
serves the gateway's own live counters as files — `/runtime/fn/{connections,
downstreams,upstream,reconnects,negotiated}` — plus `/threads`, on a private 9P
listener. Read-only; serve it on a Unix socket or loopback.

## Run

```sh
zig build -Doptimize=ReleaseSafe
./zig-out/bin/9web --upstream tcp:127.0.0.1:564
# Or run directly:
zig build serve -Doptimize=ReleaseSafe -- --upstream unix:/path/to/9p.sock
```

Open **http://127.0.0.1:8080**. Files load automatically.
The upstream must already be running: the example assumes a 9P server on port
564. For Pardes, pass its actual 9P Unix socket with `--upstream unix:PATH`.
If HTTP port 8080 is occupied, add `--listen 127.0.0.1:0`; the operating system
selects a free port and the gateway prints the URL to open.

The binary embeds HTML, CSS, JavaScript, and WebAssembly; it can run from any
working directory. `zig build web` also installs the separate assets into
`zig-out/web` for applications that host their own frontend. Its layout is
`index.html` plus assets under `_cloud9/`. A custom host must serve `index.html`
for file routes, reserve `/_cloud9/` for the assets and endpoints, and forward
WebSocket upgrades at `/_cloud9/9p`.

The browser uses a compact cgit-inspired table with modes, names, sizes, and
breadcrumb links. File URLs can be bookmarked, reloaded, or opened in new tabs;
browser back/forward navigation works. File URLs use ordinary paths such as
`/self/listeners` and `/notes/caf%C3%A9.txt`, with each component encoded separately.
The gateway serves the browser application at these URLs; the client resolves the
path in the upstream 9P filesystem. They are browser views, not raw-file HTTP
responses: missing files are reported by the page after the 9P lookup.

The literal `/_cloud9/` prefix is reserved for assets, configuration, and the
WebSocket endpoint. This leaves names such as `/app.mjs`, `/config.json`, and
`/9p` available to the upstream filesystem. If the upstream itself has a root
entry named `_cloud9`, generated links escape its leading underscore as
`/%5Fcloud9/`. Percent escapes are decoded exactly once; encoded slashes, NULs,
and malformed escapes are rejected. Reverse proxies must preserve the encoded
path when forwarding requests.

The browser establishes 9P automatically on load
and reconnects when needed, preserving unsaved edits across connection expiry.
Interrupted writes are not automatically replayed. The default attach name is
`user` and the default export name is empty; set `--user NAME` and `--tree NAME`
on the gateway when the upstream requires different values. These are 9P attach
parameters, not a browser login. The `/_cloud9/config.json` endpoint supplies these public
defaults to the page; they do not restrict other clients' attach requests.

The browser lists directories, follows paths longer than 16 components, previews
UTF-8 text, downloads binary files, and saves edits to existing files. Saving uses
OWRITE|OTRUNC and is not atomic. Empty saves truncate the file. It respects both
negotiated msize and each open's iounit, handles short writes, and clunks temporary
fids even after partial walks or filesystem errors. Views and writes are limited
to 8 MiB. The UI does not perform a 9P authentication exchange; it attaches using
NOFID, so authenticated filesystems need their own client authentication flow.

Options:

| Option | Default | Meaning |
| --- | --- | --- |
| `--listen IP:PORT` | `127.0.0.1:8080` | HTTP listener; IPv4 or bracketed IPv6 |
| `--upstream tcp:IP:PORT` | `tcp:127.0.0.1:564` | The one shared upstream 9P connection (TCP) |
| `--upstream unix:PATH` | — | The one shared upstream 9P connection (Unix) |
| `--upstream file:PATH` | — | An already configured duplex device (serial), shared by all downstreams |
| `--serve tcp:IP:PORT\|unix:PATH` | — | Also serve plain 9P downstreams (for 9ns / plan9port clients) |
| `--probe unix:PATH\|tcp:IP:PORT` | — | Embed 9proc for self-introspection (off by default) |
| `--origin SCHEME://HOST[:PORT]` | Listener origin | Public origin when a reverse proxy serves the gateway |
| `--user NAME` | `user` | Browser's 9P attach user (`uname`), at most 256 bytes |
| `--tree NAME` | empty | Browser's named export (`aname`), at most 256 bytes |
| `--timeout-ms N` | `300000` | Maximum HTTP/WebSocket connection lifetime; 0 disables it |

The gateway bounds concurrent downstream connections at 64, HTTP headers at
8 KiB, and 9P frames at 64 KiB. Excess connections close. A connection ending or
expiring drops the session and reclaims its upstream fids. Downstream fid spaces
are isolated by the mux; the one shared upstream connection carries them all,
which also means one serial link is multiplexed rather than leased — the mux
serializes transactions over it. The upstream session starts with a single
Tversion at connect; the device owner is responsible for link reset/recovery if
an interrupted physical link leaves stale bytes in transit.

For a configured serial device:

```sh
# Set raw mode, baud rate, and flow control with your platform's serial tools.
./zig-out/bin/9web --upstream file:/dev/ttyUSB0
```

The serial path transports 9P bytes; it does not interpret ESP32 boot logs or
configure pins, UART framing, baud rates, or firmware. A custom embedded runtime
can instead supply its own `std.Io.Reader` and `std.Io.Writer` to the library relay.

## HTTPS and QUIC

The native connector uses `std.http.Client` for HTTP/1.1 or certificate-verified
HTTPS. The browser uses its own WebSocket/TLS stack. For the runnable gateway,
terminate HTTPS in a reverse proxy that forwards WebSocket upgrades, and set
`--origin https://files.example` to the public origin. Serve assets and `/_cloud9/9p`
under that same origin. Host must match the listener or public authority; the
upgrade Origin must match the configured origin exactly. Gateway access grants
the upstream's existing 9P rights; identity/access policy belongs to the proxy
or filesystem server. The configured attach name is not proof of identity.

HTTP/3 is **not implemented by this connector or binary**. HTTPS alone does not
select QUIC. WebSockets over HTTP/2 and HTTP/3 require Extended CONNECT support
in the HTTP stack ([RFC 8441](https://www.rfc-editor.org/rfc/rfc8441.html),
[RFC 9220](https://www.rfc-editor.org/rfc/rfc9220.html)). Such a stack can provide
an established stream to `http.WebSocket`; its handshake and stream lifecycle
remain the stack's responsibility. cloud9's existing direct `Quic` transport is
separate from HTTP/3.

## Library API

`cloud9.http.WebSocket` operates on caller-owned standard readers/writers, with
no allocation or socket dependency. A complete binary WebSocket message contains
one complete 9P frame. It handles masked client frames, fragmented messages, and
interleaved ping/pong/close controls. Text messages and WebSocket extensions are
outside this transport contract. Its framing follows
[RFC 6455](https://www.rfc-editor.org/rfc/rfc6455.html); the payload is the existing
[9P2000 format](https://9fans.github.io/plan9port/man/man9/intro.html).

`receive(buffer)` returns borrowed bytes; keep that buffer stable across control
messages during a fragmented message. `send` flushes its writer;
`sendUnflushed` lets a layered connection control flushing. Client-role callers
must supply a fresh unpredictable mask. A framing/I/O error terminates the
transport. One reader and one writer may run concurrently, but writes, including
control replies, must be serialized.

`cloud9.http.Client.connect(&http_client, url, origin)` negotiates and validates
an HTTP upgrade. The HTTP client and URL bytes must outlive the tunnel. It uses
the HTTP client's allocator, CA bundle, and I/O provider; it does not disable
certificate verification or follow redirects. `deinit()` closes the connection
instead of returning the upgraded stream to the HTTP pool.

```zig
var http_client: std.http.Client = .{ .allocator = allocator, .io = io };
defer http_client.deinit();
var tunnel = try cloud9.http.Client.connect(
    &http_client,
    "https://files.example/_cloud9/9p",
    "https://files.example",
);
defer tunnel.deinit();

// Drive the existing cloud9.Client state machine as with any other transport.
_ = try client.submit(.{ .version = .{} });
try tunnel.send(client.output());
client.wrote(client.output().len);
const reply = try tunnel.receive(frame_buffer);
if (client.push(reply) != reply.len) return error.InputFull;
const done = client.take() orelse return error.MissingReply;
```

`Client.receive` is a serial convenience method that answers control frames. For
concurrent drivers, use `socket.receive` and coordinate all writes, including
TLS connection flushing. The 9P client still owns tag matching, negotiated size,
reply validation, and flush semantics. Split batched client output at 9P frame
boundaries before sending individual WebSocket messages.

For a server, call `http.accept(&request)` after checking route, Host, Origin,
and application authorization. Supply the resulting WebSocket, upstream standard
reader/writer, two disjoint frame buffers, and frame limit to `http.Bridge`, then
call `run(io, timeout)`. The relay forwards requests and replies concurrently,
including overlapping tags and delayed replies. It performs framing and direction
checks; it does not allocate fids, mount filesystems, or implement a filesystem.
Each Bridge value runs once. Timeout and cancellation stop both pumps before
returning. The caller owns and closes the underlying streams.

`web/client.zig` is a browser ABI around the same `cloud9.Client` and `Stat`
implementations. Its WASM memory is fixed at 1 MiB and it imports no host
functions. `web/static/client.mjs` supplies asynchronous WebSocket I/O and file
operations. Each browser Client owns a separate WASM instance. The JavaScript
contains no 9P encoder or decoder.

## End-to-end tests

```sh
zig build test http-test transport-test
zig build e2e -Doptimize=ReleaseSafe
# Alternate Chromium executable:
CLOUD9_CHROME=/path/to/chromium zig build e2e -Doptimize=ReleaseSafe
```

E2E requires Node 22+ (including its global WebSocket), Go, Chromium, OpenSSL,
and Python 3 on Linux. It builds the pinned go9p v1.18.0 fixture from the existing
test-only Go module. No npm packages, public servers, physical boards, or mounted
filesystems are used. Chrome runs headless against loopback; test TLS uses a
fresh local certificate explicitly trusted by the native client.

9P itself has connection-scoped sessions: Tversion negotiates/resets the session,
Tattach establishes a root fid, and later requests refer to fids retained by the
server. The page handles this state without presenting connection controls.

The suite drives actual DOM controls and the shipped WASM in Chromium. It checks
automatic startup, configured attach defaults, clean file URLs and reloads,
encoded filenames, collisions with asset names, invalid path encoding, back/forward
navigation, reconnecting with unsaved edits, paged directories, UTF-8 names, literal HTML-like names, text editing and
reconnection, byte-exact multi-frame binary reads/writes and browser downloads, mobile layout,
empty truncation,
18-component walks, concurrent sessions, queued operations, and recovery after
Rerror. The native client verifies HTTP, trusted HTTPS, rejection of untrusted
certificates and hostname mismatches, overlapping requests, Unix upstreams, and
a raw PTY serial path. It also checks serial lease release and connection deadlines.
The reference server deliberately fragments reads and writes. Framing unit tests
cover length boundaries, one-byte I/O, interleaved controls, invalid frames, and
upgrade nonce validation, using only cloud9 for invalid-input checks.

Each run leaves a JSON result and browser screenshot in `.zig-cache/e2e-*/`.