From fd7775b4e9c24b6bb8f664beb7d4fd2ccecd8d85 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Mon, 12 Oct 2026 15:45:52 -0300 Subject: 9web: serve local sites with 9P build controls and passive listener discovery Add optional catalogue mode with an explicit site registry, per-site HTTP roots, MIME/index/pretty-URL handling, HEAD and byte ranges. Existing generated sites are available immediately, without preview processes. Expose build, watch, cancel, status, logs and state events through the shared 9P backend and HTTP dashboard. Supervise process groups, coalesce queued rebuilds and watch configured source paths. Discover Linux TCP listeners by socket inode ownership and executable/argument recognizers; never probe endpoints. Document configuration, the user-systemd unit, workstation ports, network access assumptions and limitations including in-place publication and static-only Worker sites. Keep the original upstream gateway mode. Validation: catalogue integration suite (including paginated directory reads, cancellation and parked events), existing HTTP filesystem and mux suites, real 9ns mount, dual-stack service restart and all five workstation sites. Real entomology build: 129 pages, zero failures. --- docs/9web.service | 18 +++++ docs/catalog.example.json | 13 ++++ docs/catalog.md | 177 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 208 insertions(+) create mode 100644 docs/9web.service create mode 100644 docs/catalog.example.json create mode 100644 docs/catalog.md (limited to 'docs') diff --git a/docs/9web.service b/docs/9web.service new file mode 100644 index 0000000..a20bd89 --- /dev/null +++ b/docs/9web.service @@ -0,0 +1,18 @@ +[Unit] +Description=9web local sites, build controls and listener catalogue +After=network.target + +[Service] +ExecStart=%h/.local/libexec/9web --listen [::]:8090 --catalog %h/.config/9web/sites.json --catalog-socket %t/9web/catalog.sock --state-dir %h/.local/state/9web +Restart=on-failure +RestartSec=2 +RuntimeDirectory=9web +RuntimeDirectoryMode=0700 +UMask=0077 +KillMode=control-group +TimeoutStopSec=10 +NoNewPrivileges=yes +Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin + +[Install] +WantedBy=default.target diff --git a/docs/catalog.example.json b/docs/catalog.example.json new file mode 100644 index 0000000..c5970f3 --- /dev/null +++ b/docs/catalog.example.json @@ -0,0 +1,13 @@ +{ + "sites": [ + { + "name": "notes", + "title": "My notes", + "cwd": "/home/me/notes", + "root": "/home/me/notes/site/out", + "port": 8091, + "build": ["just", "site"], + "inputs": ["index.typ", "lib", "notes", "site/build.py", "site/site.css"] + } + ] +} diff --git a/docs/catalog.md b/docs/catalog.md new file mode 100644 index 0000000..cd6eec9 --- /dev/null +++ b/docs/catalog.md @@ -0,0 +1,177 @@ +# 9web local catalogue (Linux) + +`9web --catalog FILE` runs a local website catalogue in the gateway process. +It serves existing generated websites immediately, supervises requested builds, +and inventories listening TCP sockets without making requests to them. No +separate site servers are needed. The existing `--upstream` gateway mode is +unchanged; catalogue mode and an explicit upstream are mutually exclusive. + +## Configuration and serving + +See [catalog.example.json](catalog.example.json). Every site has a unique `name`, +`title`, absolute `cwd` and `root`, and a distinct nonzero HTTP `port`. Optional +`build` and `watch` values are argument arrays, not shell command strings. +Use an explicit `sh -c` only when a trusted recipe needs shell syntax. + +``` +9web --listen '[::]:8090' \ + --catalog "$HOME/.config/9web/sites.json" \ + --catalog-socket "$XDG_RUNTIME_DIR/9web/catalog.sock" \ + --state-dir "$HOME/.local/state/9web" +``` + +Create the socket's parent directory first. Configuration is read at startup; +restart the service to change it. A failed site-port bind fails startup rather +than advertising a site that is not actually served. All site listeners use +the dashboard's bind address. `[::]` uses a dual-stack listener for all IPv6 and +IPv4 interfaces; `0.0.0.0` restricts this to IPv4. + +Each site occupies its own HTTP origin so root-relative `/assets/` and `/blog/` +URLs work without HTML rewriting. Directory URLs redirect to a trailing slash +and serve `index.html`; extensionless paths try `.html`. Static serving supports +MIME types, HEAD, single byte ranges, and streaming large files. Symlink traversal, +dotfiles, `..`, and encoded path separators are refused. Directory listings are +available through 9P, not through site HTTP roots. + +The HTTP site root is static: provider-specific `_redirects`, `_headers`, and +Worker APIs are not executed. For example the café's articles and assets work, +but its Cloudflare poke API is not reproduced locally. + +## Builds and watch + +The dashboard has Build, Watch, Cancel, and Log controls. Builds execute in the +configured working directory with the service's environment. One child job runs +per site. Repeated build requests during a build coalesce into one later build. +Build writes enqueue work immediately; serving does not wait for compilation. + +For managed watch, configure `inputs` as source paths relative to `cwd` (or absolute +paths for shared libraries outside the project). The +watcher checks source names, modification times and sizes once per second and +invokes the build recipe on changes. It skips hidden entries, symlinks and common +generated directories (`out`, `dist`, `build`, `node_modules`, `__pycache__`). +List a particular build script explicitly if it lives in one of those directories. +This is a metadata watcher, not a content hash or a Typst dependency graph. +Changes during a build are detected by the next scan. Browser refresh is manual. + +Alternatively, `watch` can specify an existing persistent watcher. Its stdout and +stderr go to the same log, and watch-off cancels its process group. A native watcher +may itself serve a preview port, but the catalogue never requires that preview to +serve the site's generated files. Manual builds are refused while a native watcher +is active. Cancel disables watching and clears queued work. Server shutdown under +systemd kills the whole service cgroup, including build descendants. + +Each new job replaces `/.log`. The UI and 9P expose its last +256 KiB; the on-disk log is not rotated during a long-running native watcher. +Watch state and build status reset at service restart; generated output remains. +No build or watch is started automatically. + +Builders currently write to their configured output directories in place. +`9web` does **not** provide atomic publication or a last-good snapshot: a failed +build can leave partial output, and a file truncated during a response closes that +response. Builders needing atomic publication should generate a separate release +directory and arrange publication themselves. The served root can be a symlink; +symlinks below that root are refused. + +## 9P and HTTP controls + +The catalogue is itself a 9P backend, consumed by the ordinary gateway mux. +`--serve` and WebSocket clients see the same tree as direct socket clients: + +``` +/index.html +/sites//public/ complete generated output tree (read-only) +/sites//ctl build | watch on | watch off | cancel +/sites//status JSON state, PID, exit code, times, generation +/sites//log last 256 KiB of current/most recent job +/sites//events blocking stream of JSON state changes +/services/index.json recognized HTTP/HTTPS listeners +/listeners/index.json all observed TCP listeners, including unknown ones +``` + +``` +9ns --unix "$XDG_RUNTIME_DIR/9web/catalog.sock" --name web -- sh +echo build > /mnt/9p/web/sites/entomology/ctl +cat /mnt/9p/web/sites/entomology/status +cat /mnt/9p/web/sites/entomology/log +``` + +Event opens receive the current state, then park until its revision changes. +Each reader has its own cursor; slow readers receive the newest state, not a +durable history of every transition. 9P flush cancels parked reads. State events +also work through `/fs/sites//events?follow=1` as SSE. `status` is the finite +snapshot to use for ordinary polling. + +The dashboard reads `GET /_cloud9/catalog.json` and uses +`POST /_cloud9/sites//ctl` with a plain-text command. The existing +`PUT /fs/sites//ctl` also reaches the same control implementation. + +## Passive discovery + +Every five seconds, an embedded Python 3 inventory helper reads `/proc/net/tcp*`, +joins socket inodes to process descriptors, and recognizes executable/argument +patterns for miniserve, Python `http.server`, Typst watch, Vite and Wrangler dev. +9web advertises only its explicitly registered HTTP ports: its 9P listener is +never labelled HTTP just because the owning executable is 9web. + +No TCP connections or HTTP probes are performed. Unknown or inaccessible process +ownership stays unknown. Full command lines and environments are not published; +entries contain address, actual bound port, PID, executable, cwd and the evidence +for classification. Process races and permissions can make entries temporarily +unknown. Discovery is in the service's network namespace, not across containers. +The helper is a short-lived child of 9web, not another daemon. + +Wildcard addresses in dashboard links use the browser's current server hostname. +Specific addresses remain specific; loopback services are labelled local-only. +There is no automatic proxy to loopback-only services. Recognizers identify likely +HTTP listeners from process evidence; they cannot prove application health. + +## Service and access + +[9web.service](9web.service) is a user-systemd unit template. Install the binary +at `~/.local/libexec/9web`, the config at `~/.config/9web/sites.json`, and the unit +at `~/.config/systemd/user/9web.service`, then: + +``` +systemctl --user daemon-reload +systemctl --user enable --now 9web.service +systemctl --user status 9web.service +journalctl --user -u 9web.service +``` + +The unit supplies a private runtime directory and socket, restarts on failure, +and kills child jobs together with the service. Lingering must be enabled if the +user service should run without an interactive login. It needs `python3`, `setsid`, +and the configured project build tools in PATH. Miniserve remains independent. + +This is a trusted-network operator interface: it has **no login or TLS**. Anyone +who can reach the dashboard can read the catalogue and trigger configured jobs. +Browser control requests reject foreign Origins; numeric Host validation and +separate site origins prevent served pages from using the dashboard as a same-origin +control API. This is not authentication. Bind to loopback or place an authenticated +gateway in front of it when those network permissions are inappropriate. No +arbitrary command can be supplied through HTTP or 9P; commands come from the local +registry, which should remain writable only by its owner. + +## Verification + +``` +zig build catalog-test http-fs-test mux-test -Doptimize=ReleaseSafe +``` + +The catalogue integration test uses temporary real sites and processes. It checks +immediate static serving, pretty URLs, MIME, HEAD/ranges, traversal and symlink +rejection, Host/Origin checks, HTTP and raw 9P controls, watched rebuilds, failed +build status, process-group cancellation, nonblocking event readers, and actual +socket ownership discovery. Existing gateway HTTP/filesystem and mux suites cover +the original mode. + +## Installed workstation configuration + +The initial workstation setup uses dashboard port 8090 and site ports 8091–8095: +Entomology lab, 0x4200.cafe, Standards, Pardes website, and the older café hub archive. +The archive is read-only (no build recipe registered). Other sites have explicit +local build recipes and opt-in managed watches. Port 8081 remains miniserve's. +Runtime config is `~/.config/9web/sites.json`; the service uses a separately +installed binary so subsequent development builds do not change the running +service's executable on disk. Source and deployment changes are described in +the implementing git commit; no remote push is required for the local service. -- cgit v1.3