diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/9web.service | 2 | ||||
| -rw-r--r-- | docs/catalog.example.json | 5 | ||||
| -rw-r--r-- | docs/catalog.md | 128 | ||||
| -rw-r--r-- | docs/namespace-sites-validation.md | 74 |
4 files changed, 167 insertions, 42 deletions
diff --git a/docs/9web.service b/docs/9web.service index 7109935..a861671 100644 --- a/docs/9web.service +++ b/docs/9web.service @@ -3,7 +3,7 @@ Description=9web filesystem, sites and services After=network.target [Service] -ExecStart=%h/.local/bin/9ns --mntgen -- %h/.local/libexec/9web --listen [::]:8090 --catalog %h/.config/9web/sites.json --catalog-socket %t/9web/catalog.sock --state-dir %h/.local/state/9web +ExecStart=%h/.local/bin/9ns --mntgen -- %h/.local/libexec/9web --listen [::]:8090 --catalog %h/.config/9web/sites.json --catalog-socket %t/9web/catalog.sock --state-dir %h/.local/state/9web --post 9web Restart=on-failure RestartSec=2 RuntimeDirectory=9web diff --git a/docs/catalog.example.json b/docs/catalog.example.json index c5970f3..c20c54a 100644 --- a/docs/catalog.example.json +++ b/docs/catalog.example.json @@ -4,9 +4,8 @@ "name": "notes", "title": "My notes", "cwd": "/home/me/notes", - "root": "/home/me/notes/site/out", - "port": 8091, - "build": ["just", "site"], + "root": "/home/me/.local/state/9web/public/notes", + "build": ["python3", "site/build.py", "--url-prefix", "/sites/notes/", "--prefix-output", "/home/me/.local/state/9web/public/notes"], "inputs": ["index.typ", "lib", "notes", "site/build.py", "site/site.css"] } ] diff --git a/docs/catalog.md b/docs/catalog.md index 74062bb..bc308c2 100644 --- a/docs/catalog.md +++ b/docs/catalog.md @@ -8,8 +8,10 @@ unchanged; catalogue mode and an explicit upstream are mutually exclusive. Set the top-level registry field `"filesystem": "/"` to expose the whole workstation in the Files tab. The real `/` remains the root; the catalogue is -mounted at `/_9web`. Startup refuses an existing real `/_9web` rather than hiding -it. Without this opt-in field, the previous catalogue-only tree is unchanged. +posted separately as `9web`, visible at `/mnt/9p/9web` under `9ns --mntgen`. +`/_9web` is a compatibility path alias to that mount, not another backend. +Startup refuses an existing real `/_9web` rather than hiding it. +Without this opt-in field, the previous catalogue-only tree is unchanged. Only `/` is accepted: this export is not a restricted directory sandbox. Host files support reading, editing, uploads, mkdir, rename and removal through @@ -30,7 +32,9 @@ discovered by 9ns in the usual way. ## Configuration and serving See [catalog.example.json](catalog.example.json). Every site has a unique `name`, -`title`, absolute `cwd` and `root`, and a distinct nonzero HTTP `port`. Optional +`title`, absolute `cwd` and `root`. Sites live at `/sites/<name>/` on the gateway +port. An optional `port` (`?u16`, omitted by default) retains the old separate +listener for explicit compatibility; it must be nonzero and unique. Optional `build` and `watch` values are argument arrays, not shell command strings. Use an explicit `sh -c` only when a trusted recipe needs shell syntax. @@ -47,8 +51,11 @@ than advertising a site that is not actually served. All site listeners use the dashboard's bind address. `[::]` uses a dual-stack listener for all IPv6 and IPv4 interfaces; `0.0.0.0` restricts this to IPv4. -Each site occupies its own HTTP origin so root-relative `/assets/` and `/blog/` -URLs work without HTML rewriting. Directory URLs redirect to a trailing slash +HTTP roots map into `/mnt/9p/9web/sites/<name>/public` in the service namespace. +Website responses use the same mux, 9P walks, opens and reads as `/fs`, not a +second disk-serving implementation. Root-relative sites need an **opt-in build +prefix**, described below; there is no response rewriting or Referer/cookie +routing. Directory URLs redirect to a trailing slash and serve `index.html`; extensionless paths try `.html`. Static serving supports MIME types, HEAD, single byte ranges, and streaming large files. Symlink traversal, dotfiles, `..`, and encoded path separators are refused. Directory listings are @@ -100,25 +107,64 @@ Each new job replaces `<state-dir>/<name>.log`. The UI and 9P expose its last Watch state and build status reset at service restart; generated output remains. No build or watch is started automatically. -Builders currently write to their configured output directories in place. -`9web` does **not** provide atomic publication or a last-good snapshot: a failed -build can leave partial output, and a file truncated during a response closes that -response. Builders needing atomic publication should generate a separate release -directory and arrange publication themselves. The served root can be a symlink; -symlinks below that root are refused. +Generic builders may still write in place. The optional prefix packaging step +below publishes complete preview generations instead. Public files retain their +disk descriptors for the lifetime of an open 9P fid, including across publication. +The served root can be a symlink; symlinks below that root are refused. + +## Opt-in prefix builds + +`zig build` installs `zig-out/bin/9web-prefix` (Python 3). Put it on the build +process's PATH; the installed service uses `~/.local/bin/9web-prefix`. +The site's build option is **unset by default**. Authored content, source links, +scripts and default output URLs do not acquire a local prefix. + +The workstation build entry points accept: + +``` +--url-prefix /sites/NAME/ --prefix-output /absolute/preview/output +``` + +- Entomology/Standards: `python3 site/build.py ...` +- Café: `mise exec -- sh build/build.sh ...` +- Pardes website: `mise exec -- python3 build/build.py ...` (also its build task) +- Café hub archive: `python3 build/static.py ...` + +They run the normal build, then package a **separate** output copy. For initial +publication of an existing build, add `--reuse-output` to skip compilation. +9web's configured build recipes pass the prefix explicitly; managed watch calls +the same recipe. The archive remains read-only in the deployed catalogue. + +The shared packaging tool only relocates generated HTML URL attributes and CSS +URLs/imports/URL selectors. Relative URLs, external URLs, fragments, text/code +examples, JavaScript and JSON are preserved. It does not pretend to understand +arbitrary JavaScript-generated URLs; future sites using those need their own +build-time base-path support. Worker/API routes are not implemented locally. + +Preview output is an atomically replaced symlink to a complete generation in +the same parent directory. A failed compile/package leaves the prior preview +published; deleted assets do not survive in a new generation. Old generation +directories are retained (no automatic garbage collection), so repeated builds +consume disk space. Normal output is never overwritten by prefix packaging. ## 9P and HTTP controls -The catalogue is itself a 9P backend, consumed by the ordinary gateway mux. -`--serve` and WebSocket clients see the same tree as direct socket clients: +There is one supervisor in the 9web process, with a pure 9P service for site +files, controls and discovery. In host mode this is posted through the ordinary +per-user registry as `$XDG_RUNTIME_DIR/9p/9web` (`--post NAME` changes the name). +A host-only export has a separate runner, mutex, fid tables and pending reads; +it has no build/discovery loops. It exports the process's composed `/` through +the private `--catalog-socket` to the gateway mux. The host export is **not** +posted back inside its own mount, avoiding recursive exposure. -With `"filesystem": "/"`, prefix every catalogue path below with `/_9web`. -For example, the control URL becomes `/fs/_9web/sites/entomology/ctl` and its -mounted path is `/mnt/9p/web/_9web/sites/entomology/ctl`. UI links and controls +With `"filesystem": "/"`, prefix every pure-service path below with `/mnt/9p/9web`. +For example, the control URL becomes `/fs/mnt/9p/9web/sites/entomology/ctl` and its +mounted path is `/mnt/9p/9web/sites/entomology/ctl`. UI links and controls select the prefix automatically from the catalogue snapshot's `namespace`. ``` /index.html +/index.json catalogue snapshot (sites, state, discovery) /sites/<name>/public/ complete generated output tree (read-only) /sites/<name>/ctl build | watch on | watch off | cancel /sites/<name>/status JSON state, PID, exit code, times, generation @@ -129,10 +175,10 @@ select the prefix automatically from the catalogue snapshot's `namespace`. ``` ``` -9ns --unix "$XDG_RUNTIME_DIR/9web/catalog.sock" --name web -- sh -echo build > /mnt/9p/web/sites/entomology/ctl -cat /mnt/9p/web/sites/entomology/status -cat /mnt/9p/web/sites/entomology/log +9ns --mntgen -- sh +echo build > /mnt/9p/9web/sites/entomology/ctl +cat /mnt/9p/9web/sites/entomology/status +cat /mnt/9p/9web/sites/entomology/log ``` Event opens receive the current state, then park until its revision changes. @@ -141,10 +187,11 @@ durable history of every transition. 9P flush cancels parked reads. State events also work through `/fs/sites/<name>/events?follow=1` as SSE. `status` is the finite snapshot to use for ordinary polling. -The sites and services views read `GET /_cloud9/catalog.json` while visible. +The sites and services views read `GET /_cloud9/catalog.json` while visible; +this is an HTTP-to-9P mapping to the posted service's `index.json`. Build controls use `PUT /fs/sites/<name>/ctl`, through the same HTTP-to-9P mapping as file editing. The earlier `POST /_cloud9/sites/<name>/ctl` endpoint remains -compatible for existing clients. Logs use the normal file viewer rather than a +compatible for existing clients and also writes the 9P control file. Logs use the normal file viewer rather than a separate dialog. Polling preserves site-row focus and pauses in background tabs. ## Passive discovery @@ -152,7 +199,7 @@ separate dialog. Polling preserves site-row focus and pauses in background tabs. Every five seconds, an embedded Python 3 inventory helper reads `/proc/net/tcp*`, joins socket inodes to process descriptors, and recognizes executable/argument patterns for miniserve, Python `http.server`, Typst watch, Vite and Wrangler dev. -9web advertises only its explicitly registered HTTP ports: its 9P listener is +9web identifies only its own known HTTP listening ports: its 9P listener is never labelled HTTP just because the owning executable is 9web. No TCP connections or HTTP probes are performed. Unknown or inaccessible process @@ -187,13 +234,16 @@ and the configured project build tools in PATH. Miniserve remains independent. This is a trusted-network operator interface: it has **no login or TLS**. Anyone who can reach the dashboard can read the catalogue and trigger configured jobs. +Sites under URL prefixes share the operator interface's origin: site scripts +can access filesystem writes and build controls. Prefixes are **not security +isolation**; only serve trusted code here. With `"filesystem": "/"`, they can additionally read or modify any file the service user can access, including credentials, configuration and source code, and operate control files under `/mnt/9p`. This can amount to arbitrary code execution as that user. Only enable this export for clients you fully trust. -Browser control requests reject foreign Origins; numeric Host validation and -separate site origins prevent served pages from using the dashboard as a same-origin -control API. This is not authentication. Bind to loopback or place an authenticated +Browser control requests reject foreign Origins and unexpected Hosts, but served +sites on this same origin are trusted clients too. This is not authentication. +Bind to loopback or place an authenticated gateway in front of it when those network permissions are inappropriate. No arbitrary command can be supplied to the catalogue build-control parser itself; its commands come from the local registry. A whole-host writable export removes @@ -246,8 +296,8 @@ There are three separate mechanisms, not one registration service: `$XDG_RUNTIME_DIR/9p/<name>`. 9ns lists the registry and connects on a walk. No request is made to 9web; publication lifetime belongs to the producer. * Website roots and build recipes in this catalogue are startup configuration - in `sites.json`, not self-registration. The private catalogue socket itself - currently uses `listen`, not `listenPosted`. + in `sites.json`, not self-registration. The shared supervisor posts its pure + service using `listenPosted`; the private host adapter uses `listen`. * HTTP listener discovery observes `/proc` socket ownership and process metadata. It neither registers services nor probes HTTP endpoints. @@ -260,13 +310,12 @@ Original `mntgen` creates mountpoints on demand; 9ns's registry auto-connection is additional policy. Explicit configuration is not itself contrary to Plan 9: `ROOT/docs/man/6/namespace` specifies configured mount/bind/import composition. -The coherent next design step, **not implemented here**, is to expose a pure -site/build-control 9P view and compose it into the consumer's namespace, with -HTTP serving those same paths. Shared supervision can remain in one process; -no extra daemon is required per static site. Do not post the current mixed -host/catalogue root back under its own `/mnt/9p`: that creates recursion and -possible self-dependent waits. Arbitrary-upstream composition and unifying -the separate disk-based static-site HTTP handler remain outstanding. +The pure site/build-control view is now composed into the consumer's namespace, +and HTTP serves those same paths. Shared supervision remains in one process; +no extra daemon is required per static site. Only that pure view is posted, +never the host export. This also follows `ROOT/docs/man/8/httpd`: a namespace +webroot and explicit internal URL mappings, not a separate registration API. +Composition with an arbitrary explicit upstream remains outside this mode. ## Installed workstation configuration @@ -275,14 +324,17 @@ existing `/_cloud9/` asset/protocol URLs and library module names are retained for compatibility; they are not a different application name. The workstation registry enables `"filesystem": "/"`. Files opens the host -root, including the service's live `/mnt/9p` registry mount; the virtual catalogue -is under `/_9web`. The unit wraps the separately installed 9web binary with 9ns. +root, including the service's live `/mnt/9p` registry mount; the pure catalogue +is under `/mnt/9p/9web` (old `/_9web` paths still work). The unit wraps the separately installed 9web binary with 9ns. Ordinary `--upstream` mode is unchanged. Composition with an arbitrary explicit upstream is still not implemented; this mode composes the host filesystem with the catalogue, and existing 9P servers are available through `/mnt/9p`. -The initial workstation setup uses dashboard port 8090 and site ports 8091–8095: +The workstation now uses a **single port, 8090**, for the dashboard and sites: Entomology lab, 0x4200.cafe, Standards, Pardes website, and the older café hub archive. +The prior 8091–8095 listeners are no longer configured. Site URLs are +`/sites/entomology/`, `/sites/cafe/`, `/sites/standards/`, `/sites/pardes/`, and +`/sites/cafe-hub/`. Prefix output lives in `~/.local/state/9web/public/`. The archive is read-only (no build recipe registered). Other sites have explicit local build recipes and opt-in managed watches. Port 8081 remains miniserve's. Runtime config is `~/.config/9web/sites.json`; the service uses a separately diff --git a/docs/namespace-sites-validation.md b/docs/namespace-sites-validation.md new file mode 100644 index 0000000..64858fc --- /dev/null +++ b/docs/namespace-sites-validation.md @@ -0,0 +1,74 @@ +# Namespace-backed sites: implementation and validation + +## Design + +One 9web process supervises builds and posts a pure service as `9web` in the +normal per-user 9P registry. A separate host-only export consumes its composed +namespace through 9ns. Website HTTP, Files, catalogue metadata and controls use +the same 9P mux. The host export is never posted back into itself. + +Plan 9 references reviewed locally in Principia Softwarica: `srv(3)`, +`namespace(6)`, `mntgen(4)`, `exportfs(4)`, and `httpd(8)` (namespace webroot and +internal URL mappings). A conversational critique review caught public-file +reopening across publication and two HTML prefix parsing edge cases; all have +regression tests. Explicit recipe configuration remains policy, not a +self-registration protocol. Unix listener posting is a Linux adaptation of +channel publication, not identical `/srv` semantics. + +## Deployment + +- Enabled user unit `9web.service`, wrapped with `9ns --mntgen`. +- Sole HTTP listener `[::]:8090`, reachable using LAN IPv4 and IPv6. +- Existing 8091–8095 site listeners removed from configuration; miniserve untouched. +- Whole `/`, including `/mnt/9p`, retained in Files; pure service at `/mnt/9p/9web`. +- URLs `/sites/{entomology,cafe,standards,pardes,cafe-hub}/`. +- Preview generations under `~/.local/state/9web/public/`; normal output unchanged + by prefix packaging. Initial publication reused existing builds. +- `--url-prefix` is optional in project build commands and never enabled by + default. The service explicitly supplies it for builds and managed watch. +- Optional compatibility site port is represented as `?u16 = null`, not zero. +- Configuration backup: `~/.config/9web/sites.before-namespace.json`. + +The helper `9web-prefix` is installed by `zig build` and locally at +`~/.local/bin/9web-prefix`. It relocates generated HTML/CSS only. JavaScript, +JSON, authored content and source URLs are unchanged. The archive stays +read-only in the service. Old preview generations are retained, not garbage +collected. Same-origin site scripts are trusted operator code, not isolated +from writable files or build controls. Cloudflare APIs are not implemented. + +## Companion build commits + +| Project | Commit | Scope | +|---|---|---| +| Entomology lab | `da9d227` | Optional prefix/output/reuse flags in existing builder | +| Standards | `4ad43a0` | Same optional build flags | +| 0x4200.cafe | `2d47b2b` | Optional flags in build/build.sh; default/watch preserved | +| Pardes website | `a1da335` | Build entry point with optional prefix packaging | +| Café hub archive | `3e07a67` | Optional static archive build/packaging entry point | + +Existing unrelated project edits were not committed. No remote push. + +## Checks + +`zig build -Doptimize=ReleaseSafe catalog-test host-fs-test http-fs-test +catalog-ui-test mux-test prefix-test` passes. Coverage includes: + +- Index/pretty URLs, redirects with query preservation, MIME, HEAD and byte ranges. +- Identical website bytes through namespace/9P and HTTP; traversal rejection. +- Pinned public file reads when a new generation replaces its pathname. +- Scoped HTML/CSS prefixing, CSS selectors, entities, srcset/data URLs, unchanged + scripts/code examples/external URLs and normal-output hashes. +- Atomic publication, stale-asset exclusion and preservation on failure. +- Whole-host operations and mounted 9P controls, large directories, blocking + events and repeated flush/clunk without reader exhaustion. +- Existing Files/Sites/Services shell, mobile layout, editing and history. + +Live Chromium validation (`NINE_WEB_LIVE=http://127.0.0.1:8090 node +test/web/catalog_ui.mjs zig-out/bin/9web`) also passes: `/mnt/9p/9web`, agents, +zmx, `/usr/bin`, all five site landing pages, 22 same-origin images across +Entomology and the café, and café blog/pretty article navigation. LAN IPv4 and +IPv6 site requests return 200. An actual café build requested through +`/fs/mnt/9p/9web/sites/cafe/ctl` completed with exit code 0 and published a new +prefixed generation; no preview daemon was started. + +`zig build -Doptimize=ReleaseSafe e2e` passes all 32 original gateway scenarios. |
