summaryrefslogtreecommitdiff
path: root/docs/catalog.md
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-10-12 17:15:03 -0300
committerGabriel Schneider <[email protected]>2026-11-17 12:15:16 -0300
commit7a07e24916cd5bc762996d81aa1faf550177cd6f (patch)
tree4530a8562fb3f068cc4340f1fab1f6c599721735 /docs/catalog.md
parentf9ca3fb5ba767300bec3f1fbb13fc5e6552a90e3 (diff)
downloadcloud9-7a07e24916cd5bc762996d81aa1faf550177cd6f.tar.gz
cloud9-7a07e24916cd5bc762996d81aa1faf550177cd6f.zip
9web: serve prefixed sites through the composed 9P namespace
Post only the pure site/control/discovery tree, with a separate host-only adapter and no duplicate supervision. Serve websites, metadata and legacy controls through the ordinary 9P mux; retain whole-root filesystem browsing and compatibility paths. Pin public file descriptors across publication. Add opt-in build-time HTML/CSS prefix packaging with separate atomic preview generations and unchanged JavaScript/default output. Optional compatibility ports use ?u16. Deploy all five workstation sites on [::]:8090, with /mnt/9p/9web controls. Document Plan 9 references, companion build commits, shared-origin trust and live validation. Validated catalogue, host/FUSE, HTTP FS, mux, prefix and Chromium suites; original E2E 32 scenarios; all five live sites, blog navigation, IPv4/IPv6 and real cafe build through 9P.
Diffstat (limited to 'docs/catalog.md')
-rw-r--r--docs/catalog.md128
1 files changed, 90 insertions, 38 deletions
diff --git a/docs/catalog.md b/docs/catalog.md
index 74062bb..bc308c2 100644
--- a/docs/catalog.md
+++ b/docs/catalog.md
@@ -8,8 +8,10 @@ unchanged; catalogue mode and an explicit upstream are mutually exclusive.
Set the top-level registry field `"filesystem": "/"` to expose the whole
workstation in the Files tab. The real `/` remains the root; the catalogue is
-mounted at `/_9web`. Startup refuses an existing real `/_9web` rather than hiding
-it. Without this opt-in field, the previous catalogue-only tree is unchanged.
+posted separately as `9web`, visible at `/mnt/9p/9web` under `9ns --mntgen`.
+`/_9web` is a compatibility path alias to that mount, not another backend.
+Startup refuses an existing real `/_9web` rather than hiding it.
+Without this opt-in field, the previous catalogue-only tree is unchanged.
Only `/` is accepted: this export is not a restricted directory sandbox.
Host files support reading, editing, uploads, mkdir, rename and removal through
@@ -30,7 +32,9 @@ discovered by 9ns in the usual way.
## Configuration and serving
See [catalog.example.json](catalog.example.json). Every site has a unique `name`,
-`title`, absolute `cwd` and `root`, and a distinct nonzero HTTP `port`. Optional
+`title`, absolute `cwd` and `root`. Sites live at `/sites/<name>/` on the gateway
+port. An optional `port` (`?u16`, omitted by default) retains the old separate
+listener for explicit compatibility; it must be nonzero and unique. Optional
`build` and `watch` values are argument arrays, not shell command strings.
Use an explicit `sh -c` only when a trusted recipe needs shell syntax.
@@ -47,8 +51,11 @@ than advertising a site that is not actually served. All site listeners use
the dashboard's bind address. `[::]` uses a dual-stack listener for all IPv6 and
IPv4 interfaces; `0.0.0.0` restricts this to IPv4.
-Each site occupies its own HTTP origin so root-relative `/assets/` and `/blog/`
-URLs work without HTML rewriting. Directory URLs redirect to a trailing slash
+HTTP roots map into `/mnt/9p/9web/sites/<name>/public` in the service namespace.
+Website responses use the same mux, 9P walks, opens and reads as `/fs`, not a
+second disk-serving implementation. Root-relative sites need an **opt-in build
+prefix**, described below; there is no response rewriting or Referer/cookie
+routing. Directory URLs redirect to a trailing slash
and serve `index.html`; extensionless paths try `.html`. Static serving supports
MIME types, HEAD, single byte ranges, and streaming large files. Symlink traversal,
dotfiles, `..`, and encoded path separators are refused. Directory listings are
@@ -100,25 +107,64 @@ Each new job replaces `<state-dir>/<name>.log`. The UI and 9P expose its last
Watch state and build status reset at service restart; generated output remains.
No build or watch is started automatically.
-Builders currently write to their configured output directories in place.
-`9web` does **not** provide atomic publication or a last-good snapshot: a failed
-build can leave partial output, and a file truncated during a response closes that
-response. Builders needing atomic publication should generate a separate release
-directory and arrange publication themselves. The served root can be a symlink;
-symlinks below that root are refused.
+Generic builders may still write in place. The optional prefix packaging step
+below publishes complete preview generations instead. Public files retain their
+disk descriptors for the lifetime of an open 9P fid, including across publication.
+The served root can be a symlink; symlinks below that root are refused.
+
+## Opt-in prefix builds
+
+`zig build` installs `zig-out/bin/9web-prefix` (Python 3). Put it on the build
+process's PATH; the installed service uses `~/.local/bin/9web-prefix`.
+The site's build option is **unset by default**. Authored content, source links,
+scripts and default output URLs do not acquire a local prefix.
+
+The workstation build entry points accept:
+
+```
+--url-prefix /sites/NAME/ --prefix-output /absolute/preview/output
+```
+
+- Entomology/Standards: `python3 site/build.py ...`
+- Café: `mise exec -- sh build/build.sh ...`
+- Pardes website: `mise exec -- python3 build/build.py ...` (also its build task)
+- Café hub archive: `python3 build/static.py ...`
+
+They run the normal build, then package a **separate** output copy. For initial
+publication of an existing build, add `--reuse-output` to skip compilation.
+9web's configured build recipes pass the prefix explicitly; managed watch calls
+the same recipe. The archive remains read-only in the deployed catalogue.
+
+The shared packaging tool only relocates generated HTML URL attributes and CSS
+URLs/imports/URL selectors. Relative URLs, external URLs, fragments, text/code
+examples, JavaScript and JSON are preserved. It does not pretend to understand
+arbitrary JavaScript-generated URLs; future sites using those need their own
+build-time base-path support. Worker/API routes are not implemented locally.
+
+Preview output is an atomically replaced symlink to a complete generation in
+the same parent directory. A failed compile/package leaves the prior preview
+published; deleted assets do not survive in a new generation. Old generation
+directories are retained (no automatic garbage collection), so repeated builds
+consume disk space. Normal output is never overwritten by prefix packaging.
## 9P and HTTP controls
-The catalogue is itself a 9P backend, consumed by the ordinary gateway mux.
-`--serve` and WebSocket clients see the same tree as direct socket clients:
+There is one supervisor in the 9web process, with a pure 9P service for site
+files, controls and discovery. In host mode this is posted through the ordinary
+per-user registry as `$XDG_RUNTIME_DIR/9p/9web` (`--post NAME` changes the name).
+A host-only export has a separate runner, mutex, fid tables and pending reads;
+it has no build/discovery loops. It exports the process's composed `/` through
+the private `--catalog-socket` to the gateway mux. The host export is **not**
+posted back inside its own mount, avoiding recursive exposure.
-With `"filesystem": "/"`, prefix every catalogue path below with `/_9web`.
-For example, the control URL becomes `/fs/_9web/sites/entomology/ctl` and its
-mounted path is `/mnt/9p/web/_9web/sites/entomology/ctl`. UI links and controls
+With `"filesystem": "/"`, prefix every pure-service path below with `/mnt/9p/9web`.
+For example, the control URL becomes `/fs/mnt/9p/9web/sites/entomology/ctl` and its
+mounted path is `/mnt/9p/9web/sites/entomology/ctl`. UI links and controls
select the prefix automatically from the catalogue snapshot's `namespace`.
```
/index.html
+/index.json catalogue snapshot (sites, state, discovery)
/sites/<name>/public/ complete generated output tree (read-only)
/sites/<name>/ctl build | watch on | watch off | cancel
/sites/<name>/status JSON state, PID, exit code, times, generation
@@ -129,10 +175,10 @@ select the prefix automatically from the catalogue snapshot's `namespace`.
```
```
-9ns --unix "$XDG_RUNTIME_DIR/9web/catalog.sock" --name web -- sh
-echo build > /mnt/9p/web/sites/entomology/ctl
-cat /mnt/9p/web/sites/entomology/status
-cat /mnt/9p/web/sites/entomology/log
+9ns --mntgen -- sh
+echo build > /mnt/9p/9web/sites/entomology/ctl
+cat /mnt/9p/9web/sites/entomology/status
+cat /mnt/9p/9web/sites/entomology/log
```
Event opens receive the current state, then park until its revision changes.
@@ -141,10 +187,11 @@ durable history of every transition. 9P flush cancels parked reads. State events
also work through `/fs/sites/<name>/events?follow=1` as SSE. `status` is the finite
snapshot to use for ordinary polling.
-The sites and services views read `GET /_cloud9/catalog.json` while visible.
+The sites and services views read `GET /_cloud9/catalog.json` while visible;
+this is an HTTP-to-9P mapping to the posted service's `index.json`.
Build controls use `PUT /fs/sites/<name>/ctl`, through the same HTTP-to-9P mapping
as file editing. The earlier `POST /_cloud9/sites/<name>/ctl` endpoint remains
-compatible for existing clients. Logs use the normal file viewer rather than a
+compatible for existing clients and also writes the 9P control file. Logs use the normal file viewer rather than a
separate dialog. Polling preserves site-row focus and pauses in background tabs.
## Passive discovery
@@ -152,7 +199,7 @@ separate dialog. Polling preserves site-row focus and pauses in background tabs.
Every five seconds, an embedded Python 3 inventory helper reads `/proc/net/tcp*`,
joins socket inodes to process descriptors, and recognizes executable/argument
patterns for miniserve, Python `http.server`, Typst watch, Vite and Wrangler dev.
-9web advertises only its explicitly registered HTTP ports: its 9P listener is
+9web identifies only its own known HTTP listening ports: its 9P listener is
never labelled HTTP just because the owning executable is 9web.
No TCP connections or HTTP probes are performed. Unknown or inaccessible process
@@ -187,13 +234,16 @@ and the configured project build tools in PATH. Miniserve remains independent.
This is a trusted-network operator interface: it has **no login or TLS**. Anyone
who can reach the dashboard can read the catalogue and trigger configured jobs.
+Sites under URL prefixes share the operator interface's origin: site scripts
+can access filesystem writes and build controls. Prefixes are **not security
+isolation**; only serve trusted code here.
With `"filesystem": "/"`, they can additionally read or modify any file the
service user can access, including credentials, configuration and source code,
and operate control files under `/mnt/9p`. This can amount to arbitrary code
execution as that user. Only enable this export for clients you fully trust.
-Browser control requests reject foreign Origins; numeric Host validation and
-separate site origins prevent served pages from using the dashboard as a same-origin
-control API. This is not authentication. Bind to loopback or place an authenticated
+Browser control requests reject foreign Origins and unexpected Hosts, but served
+sites on this same origin are trusted clients too. This is not authentication.
+Bind to loopback or place an authenticated
gateway in front of it when those network permissions are inappropriate. No
arbitrary command can be supplied to the catalogue build-control parser itself;
its commands come from the local registry. A whole-host writable export removes
@@ -246,8 +296,8 @@ There are three separate mechanisms, not one registration service:
`$XDG_RUNTIME_DIR/9p/<name>`. 9ns lists the registry and connects on a walk.
No request is made to 9web; publication lifetime belongs to the producer.
* Website roots and build recipes in this catalogue are startup configuration
- in `sites.json`, not self-registration. The private catalogue socket itself
- currently uses `listen`, not `listenPosted`.
+ in `sites.json`, not self-registration. The shared supervisor posts its pure
+ service using `listenPosted`; the private host adapter uses `listen`.
* HTTP listener discovery observes `/proc` socket ownership and process metadata.
It neither registers services nor probes HTTP endpoints.
@@ -260,13 +310,12 @@ Original `mntgen` creates mountpoints on demand; 9ns's registry auto-connection
is additional policy. Explicit configuration is not itself contrary to Plan 9:
`ROOT/docs/man/6/namespace` specifies configured mount/bind/import composition.
-The coherent next design step, **not implemented here**, is to expose a pure
-site/build-control 9P view and compose it into the consumer's namespace, with
-HTTP serving those same paths. Shared supervision can remain in one process;
-no extra daemon is required per static site. Do not post the current mixed
-host/catalogue root back under its own `/mnt/9p`: that creates recursion and
-possible self-dependent waits. Arbitrary-upstream composition and unifying
-the separate disk-based static-site HTTP handler remain outstanding.
+The pure site/build-control view is now composed into the consumer's namespace,
+and HTTP serves those same paths. Shared supervision remains in one process;
+no extra daemon is required per static site. Only that pure view is posted,
+never the host export. This also follows `ROOT/docs/man/8/httpd`: a namespace
+webroot and explicit internal URL mappings, not a separate registration API.
+Composition with an arbitrary explicit upstream remains outside this mode.
## Installed workstation configuration
@@ -275,14 +324,17 @@ existing `/_cloud9/` asset/protocol URLs and library module names are retained
for compatibility; they are not a different application name.
The workstation registry enables `"filesystem": "/"`. Files opens the host
-root, including the service's live `/mnt/9p` registry mount; the virtual catalogue
-is under `/_9web`. The unit wraps the separately installed 9web binary with 9ns.
+root, including the service's live `/mnt/9p` registry mount; the pure catalogue
+is under `/mnt/9p/9web` (old `/_9web` paths still work). The unit wraps the separately installed 9web binary with 9ns.
Ordinary `--upstream` mode is unchanged. Composition with an arbitrary explicit
upstream is still not implemented; this mode composes the host filesystem with
the catalogue, and existing 9P servers are available through `/mnt/9p`.
-The initial workstation setup uses dashboard port 8090 and site ports 8091–8095:
+The workstation now uses a **single port, 8090**, for the dashboard and sites:
Entomology lab, 0x4200.cafe, Standards, Pardes website, and the older café hub archive.
+The prior 8091–8095 listeners are no longer configured. Site URLs are
+`/sites/entomology/`, `/sites/cafe/`, `/sites/standards/`, `/sites/pardes/`, and
+`/sites/cafe-hub/`. Prefix output lives in `~/.local/state/9web/public/`.
The archive is read-only (no build recipe registered). Other sites have explicit
local build recipes and opt-in managed watches. Port 8081 remains miniserve's.
Runtime config is `~/.config/9web/sites.json`; the service uses a separately