summaryrefslogtreecommitdiff
path: root/docs/catalog.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/catalog.md')
-rw-r--r--docs/catalog.md177
1 files changed, 177 insertions, 0 deletions
diff --git a/docs/catalog.md b/docs/catalog.md
new file mode 100644
index 0000000..cd6eec9
--- /dev/null
+++ b/docs/catalog.md
@@ -0,0 +1,177 @@
+# 9web local catalogue (Linux)
+
+`9web --catalog FILE` runs a local website catalogue in the gateway process.
+It serves existing generated websites immediately, supervises requested builds,
+and inventories listening TCP sockets without making requests to them. No
+separate site servers are needed. The existing `--upstream` gateway mode is
+unchanged; catalogue mode and an explicit upstream are mutually exclusive.
+
+## Configuration and serving
+
+See [catalog.example.json](catalog.example.json). Every site has a unique `name`,
+`title`, absolute `cwd` and `root`, and a distinct nonzero HTTP `port`. Optional
+`build` and `watch` values are argument arrays, not shell command strings.
+Use an explicit `sh -c` only when a trusted recipe needs shell syntax.
+
+```
+9web --listen '[::]:8090' \
+ --catalog "$HOME/.config/9web/sites.json" \
+ --catalog-socket "$XDG_RUNTIME_DIR/9web/catalog.sock" \
+ --state-dir "$HOME/.local/state/9web"
+```
+
+Create the socket's parent directory first. Configuration is read at startup;
+restart the service to change it. A failed site-port bind fails startup rather
+than advertising a site that is not actually served. All site listeners use
+the dashboard's bind address. `[::]` uses a dual-stack listener for all IPv6 and
+IPv4 interfaces; `0.0.0.0` restricts this to IPv4.
+
+Each site occupies its own HTTP origin so root-relative `/assets/` and `/blog/`
+URLs work without HTML rewriting. Directory URLs redirect to a trailing slash
+and serve `index.html`; extensionless paths try `.html`. Static serving supports
+MIME types, HEAD, single byte ranges, and streaming large files. Symlink traversal,
+dotfiles, `..`, and encoded path separators are refused. Directory listings are
+available through 9P, not through site HTTP roots.
+
+The HTTP site root is static: provider-specific `_redirects`, `_headers`, and
+Worker APIs are not executed. For example the café's articles and assets work,
+but its Cloudflare poke API is not reproduced locally.
+
+## Builds and watch
+
+The dashboard has Build, Watch, Cancel, and Log controls. Builds execute in the
+configured working directory with the service's environment. One child job runs
+per site. Repeated build requests during a build coalesce into one later build.
+Build writes enqueue work immediately; serving does not wait for compilation.
+
+For managed watch, configure `inputs` as source paths relative to `cwd` (or absolute
+paths for shared libraries outside the project). The
+watcher checks source names, modification times and sizes once per second and
+invokes the build recipe on changes. It skips hidden entries, symlinks and common
+generated directories (`out`, `dist`, `build`, `node_modules`, `__pycache__`).
+List a particular build script explicitly if it lives in one of those directories.
+This is a metadata watcher, not a content hash or a Typst dependency graph.
+Changes during a build are detected by the next scan. Browser refresh is manual.
+
+Alternatively, `watch` can specify an existing persistent watcher. Its stdout and
+stderr go to the same log, and watch-off cancels its process group. A native watcher
+may itself serve a preview port, but the catalogue never requires that preview to
+serve the site's generated files. Manual builds are refused while a native watcher
+is active. Cancel disables watching and clears queued work. Server shutdown under
+systemd kills the whole service cgroup, including build descendants.
+
+Each new job replaces `<state-dir>/<name>.log`. The UI and 9P expose its last
+256 KiB; the on-disk log is not rotated during a long-running native watcher.
+Watch state and build status reset at service restart; generated output remains.
+No build or watch is started automatically.
+
+Builders currently write to their configured output directories in place.
+`9web` does **not** provide atomic publication or a last-good snapshot: a failed
+build can leave partial output, and a file truncated during a response closes that
+response. Builders needing atomic publication should generate a separate release
+directory and arrange publication themselves. The served root can be a symlink;
+symlinks below that root are refused.
+
+## 9P and HTTP controls
+
+The catalogue is itself a 9P backend, consumed by the ordinary gateway mux.
+`--serve` and WebSocket clients see the same tree as direct socket clients:
+
+```
+/index.html
+/sites/<name>/public/ complete generated output tree (read-only)
+/sites/<name>/ctl build | watch on | watch off | cancel
+/sites/<name>/status JSON state, PID, exit code, times, generation
+/sites/<name>/log last 256 KiB of current/most recent job
+/sites/<name>/events blocking stream of JSON state changes
+/services/index.json recognized HTTP/HTTPS listeners
+/listeners/index.json all observed TCP listeners, including unknown ones
+```
+
+```
+9ns --unix "$XDG_RUNTIME_DIR/9web/catalog.sock" --name web -- sh
+echo build > /mnt/9p/web/sites/entomology/ctl
+cat /mnt/9p/web/sites/entomology/status
+cat /mnt/9p/web/sites/entomology/log
+```
+
+Event opens receive the current state, then park until its revision changes.
+Each reader has its own cursor; slow readers receive the newest state, not a
+durable history of every transition. 9P flush cancels parked reads. State events
+also work through `/fs/sites/<name>/events?follow=1` as SSE. `status` is the finite
+snapshot to use for ordinary polling.
+
+The dashboard reads `GET /_cloud9/catalog.json` and uses
+`POST /_cloud9/sites/<name>/ctl` with a plain-text command. The existing
+`PUT /fs/sites/<name>/ctl` also reaches the same control implementation.
+
+## Passive discovery
+
+Every five seconds, an embedded Python 3 inventory helper reads `/proc/net/tcp*`,
+joins socket inodes to process descriptors, and recognizes executable/argument
+patterns for miniserve, Python `http.server`, Typst watch, Vite and Wrangler dev.
+9web advertises only its explicitly registered HTTP ports: its 9P listener is
+never labelled HTTP just because the owning executable is 9web.
+
+No TCP connections or HTTP probes are performed. Unknown or inaccessible process
+ownership stays unknown. Full command lines and environments are not published;
+entries contain address, actual bound port, PID, executable, cwd and the evidence
+for classification. Process races and permissions can make entries temporarily
+unknown. Discovery is in the service's network namespace, not across containers.
+The helper is a short-lived child of 9web, not another daemon.
+
+Wildcard addresses in dashboard links use the browser's current server hostname.
+Specific addresses remain specific; loopback services are labelled local-only.
+There is no automatic proxy to loopback-only services. Recognizers identify likely
+HTTP listeners from process evidence; they cannot prove application health.
+
+## Service and access
+
+[9web.service](9web.service) is a user-systemd unit template. Install the binary
+at `~/.local/libexec/9web`, the config at `~/.config/9web/sites.json`, and the unit
+at `~/.config/systemd/user/9web.service`, then:
+
+```
+systemctl --user daemon-reload
+systemctl --user enable --now 9web.service
+systemctl --user status 9web.service
+journalctl --user -u 9web.service
+```
+
+The unit supplies a private runtime directory and socket, restarts on failure,
+and kills child jobs together with the service. Lingering must be enabled if the
+user service should run without an interactive login. It needs `python3`, `setsid`,
+and the configured project build tools in PATH. Miniserve remains independent.
+
+This is a trusted-network operator interface: it has **no login or TLS**. Anyone
+who can reach the dashboard can read the catalogue and trigger configured jobs.
+Browser control requests reject foreign Origins; numeric Host validation and
+separate site origins prevent served pages from using the dashboard as a same-origin
+control API. This is not authentication. Bind to loopback or place an authenticated
+gateway in front of it when those network permissions are inappropriate. No
+arbitrary command can be supplied through HTTP or 9P; commands come from the local
+registry, which should remain writable only by its owner.
+
+## Verification
+
+```
+zig build catalog-test http-fs-test mux-test -Doptimize=ReleaseSafe
+```
+
+The catalogue integration test uses temporary real sites and processes. It checks
+immediate static serving, pretty URLs, MIME, HEAD/ranges, traversal and symlink
+rejection, Host/Origin checks, HTTP and raw 9P controls, watched rebuilds, failed
+build status, process-group cancellation, nonblocking event readers, and actual
+socket ownership discovery. Existing gateway HTTP/filesystem and mux suites cover
+the original mode.
+
+## Installed workstation configuration
+
+The initial workstation setup uses dashboard port 8090 and site ports 8091–8095:
+Entomology lab, 0x4200.cafe, Standards, Pardes website, and the older café hub archive.
+The archive is read-only (no build recipe registered). Other sites have explicit
+local build recipes and opt-in managed watches. Port 8081 remains miniserve's.
+Runtime config is `~/.config/9web/sites.json`; the service uses a separately
+installed binary so subsequent development builds do not change the running
+service's executable on disk. Source and deployment changes are described in
+the implementing git commit; no remote push is required for the local service.