lsp: a protocol client for every other language, narrated on the message row

The seam grows a second backend: src/lsp/lsp_client.zig speaks JSON-RPC to
child language servers — rust-analyzer, clangd, gopls, tsserver, pyright are
rows in a spec table — while the in-process ZLS analyser keeps .zig. One
reader thread per server owns the socket, routes responses to a mailbox
under the conn mutex (monotonic condvar), answers server-to-client requests,
feeds the diagnostics store, and narrates $/progress and state changes
through a status sink both native shells post to the transient message row:
"rust-analyzer: cargo check 88% 955/1083" lands where a save narrates, with
the same clock. Chatty progress is throttled and deduplicated; settled
states always land, which is also what makes the goldens deterministic.

Nothing wedges and nothing healthy dies: waits are deadline-bounded, a
timeout cancels and returns no rows, three consecutive timeouts restart the
server ONLY while it is idle (an indexing server is narrating its own
excuse), spawn and handshake failures back off 10s to 2min, a crash shortly
after ready counts as a failure, and only a missing binary disables a spec.
PARDES_LSP_{RS,C,GO,TS,PY} override binaries; empty disables; the snapshot
harness pins RS to test/lspmock.zig and empties the rest.

Mutating answers really mutate now: the @put record beside rename @edit
carries per-range text, so = applies the formatter (both backends) and a
same-file WorkspaceEdit rename applies atomically, one undo step, narrated
("renamed 2 range(s)"); a multi-file rename previews as rows instead of
half-applying. Malformed responses fail closed: coordinates validated not
clamped, one bad TextEdit poisons the whole edit set, poison frames kill
the connection instead of buffering forever, decoded control bytes reject a
uri, hierarchy items too deep to reserialize are skipped.

Four kinds helix does not have, on SPC l: c/C incoming/outgoing calls (rows
are call sites), t/T super/subtypes. Pull diagnostics (3.17) preferred when
advertised. Help gains a language-keys footer for the motions no builtin
row could carry; lsp.rel and look.grep now share one path-shortening rule.

zig build lspprobe drives the seam from the CLI (comma-separated kinds share
one server); measured against a 1083-crate workspace warm: gd 26ms, gr 213
rows 165ms, incoming calls 212 sites 197ms, document symbols 670 rows 347ms.
docs/lsp.md tells the whole story; lsp-evaluation.md gets an addendum.

1 files changed, 132 insertions, 13 deletions

diff --git a/docs/lsp.md b/docs/lsp.md
index b35c8eeb..4b2e5ba2 100644
--- a/docs/lsp.md
+++ b/docs/lsp.md
@@ -149,11 +149,10 @@ to end — the enum one level up (absolute row) and one level down (`inner/tint.
 a stripped path that still has a separator in it), each with the `n` step that
 selects it and the Enter that opens it, plus a right click.
 
-This is the rule `look.grep` already follows for its own rows, and it is
-spelled TWICE: `lsp.rel` here, and an inline `if` over the asking pane's
-directory in `look.zig`'s `grep` (the `shown` computation) there. Same rule,
-two implementations — they want to become one function, and `lsp.zig`'s own
-comment on `rel` says so.
+This is the rule `look.grep` follows for its own rows, and since the client
+landed it is spelled ONCE: look.zig's `grep` calls `lsp.rel` for its `shown`
+paths rather than keeping the inline twin this paragraph used to complain
+about.
 
 `completion` is the kind this shape changes the most. Every other editor answers
 a dot with a popup of NAMES to insert; a seam that returns locations cannot
@@ -250,6 +249,118 @@ Known limitations, in the order you will meet them:
 - **`error.`** is not handled — the position context is `.error_access`, which
   no branch claims.
 
+
+## The protocol client: every other language
+
+`src/lsp/lsp_client.zig` is the second backend behind the same seam: a real
+LSP client — JSON-RPC 2.0, `Content-Length` frames — speaking to child
+processes. Nothing in it knows any single language; `specs` is a table of
+(binary, languageId, extensions, root markers), and rust-analyzer, clangd,
+gopls, typescript-language-server and pyright are rows in it. The seam asks
+each backend `speaks(path) and supports(kind)` in order, so `.zig` stays with
+the in-process analyser (cold is warm, no process) and everything else routes
+here. `SPC l i` prints both sections; `backend_name` is `zls-inproc+lsp-client`.
+
+**One server per spec, one reader thread per server, and the reader is not
+optional.** A real server TALKS: rust-analyzer streams `$/progress` for the
+whole minutes-long index of a big workspace, publishes diagnostics nobody
+asked for, and asks its own `workspace/configuration` questions mid-flight.
+The reader owns the read side of the socketpair, routes responses to the one
+waiting query (a mailbox under the connection's mutex), answers
+server-to-client requests so the server never blocks on us, feeds the
+diagnostics store, and narrates state changes through the STATUS SINK — a
+callback both native shells register at startup and post to their event
+queue, so "rust-analyzer: cargo check 88% 955/1083" lands on the same
+transient message row a save narrates into (`message.stamp`, verb `lsp`, on
+the ACTIVE pane — server state is session news, not a fact about the pane
+that asked). Chatty progress is throttled to one post per 150ms per server
+and deduplicated; state CHANGES (starting, ready, exited, errors) always
+land, and repeating the row already shown never does — which is also what
+makes the settled state deterministic for the snapshot goldens.
+
+Nothing may wedge the editor, and nothing healthy may be killed for being
+busy:
+
+- every write and every mailbox wait is deadline-bounded (8s handshake, 4s
+  request); a query the server does not answer in time returns no rows and
+  sends `$/cancelRequest`;
+- three CONSECUTIVE timeouts mean wedged and force a restart — but only
+  while the server is idle. One with active `$/progress` (rust-analyzer
+  mid-`cargo check` over a thousand crates) is demonstrably alive, already
+  narrating its own excuse on the message row, and killing it would throw
+  the index away right before it pays off. This rule exists because the
+  first run against a thousand-crate workspace did exactly that;
+- a failed spawn or handshake is NOT a session disable: it backs off
+  exponentially (10s doubling to 2min, reset by the next success), because
+  the failure that taught this was a rustup shim deciding to download the
+  project's whole pinned toolchain before launching the real server. Only a
+  missing binary disables a spec, once, with a message saying which env var
+  overrides it;
+- a server that dies is reaped by whoever saw it die (the reader on EOF,
+  `shutdownIf` on a transport error), the fd is closed by the READER ALONE —
+  `shutdown(2)` first, so a polled fd number is never recycled under a
+  thread still watching it — and the next query respawns, generation-checked
+  so a stale worker can neither adopt nor kill its successor's server.
+
+`PARDES_LSP_RS` / `_C` / `_GO` / `_TS` / `_PY` override each spec's binary
+(a path or a PATH name); the empty string disables the spec. The snapshot
+harness pins `_RS` to `test/lspmock.zig`'s deterministic mock and empties
+the rest, so `test/snapshots/lsp-client.snap` (gd across files, gr spans,
+n/Enter) and `lsp-client-edit.snap` (format apply, rename apply, one-step
+undo for each) drive the REAL client — spawn, handshake, reader, narration —
+against answers a golden can quote. `zig build lspprobe -- gd <file> <l>:<c>`
+is the same seam from the command line, for pointing at any real workspace;
+comma-separated kinds share one server so a big index is paid for once.
+
+The root is helix's `find_root` rule: walking up from the file, the TOP-MOST
+directory holding one of the spec's markers wins (a cargo workspace's root
+`Cargo.toml` beats the member crate's), the closest `.git` is the fallback,
+the asking directory the last resort. A second project in the same session
+becomes a workspace FOLDER when the server advertises support. Position
+encoding is negotiated to utf-8 and the server's ANSWER is believed; the
+utf-16 conversion is implemented in both directions for servers that refuse.
+Diagnostics PULL (`textDocument/diagnostic`, LSP 3.17) is preferred when the
+server advertises it — rust-analyzer does — and the push store fed by the
+reader answers otherwise, `]d` stepping either for free.
+
+### Mutating answers really mutate now
+
+The seam grew a second record form beside rename's `@edit`: `@put START END
+TEXT` carries a per-range replacement, percent-encoded onto the one line a
+record is allowed to be (`lsp.put`). The core decodes, validates (ordered,
+non-overlapping, in bounds, revision unchanged) and applies ALL records as
+one undo transaction, then says so on the message row ("formatted 1
+range(s)", "renamed 2 range(s)"). So:
+
+- `=` FORMATS, like helix — through the client it applies the server's
+  TextEdits; through ZLS it applies one span covering everything `zig fmt`
+  would change. The two non-edit answers stay prose in `+Lsp`: a file that
+  does not parse, and (client-side) a server with no formatter.
+- `SPC l r` through the client applies a WorkspaceEdit that stays inside the
+  asked-about file. One that spans OTHER files (a real workspace rename)
+  arrives as location rows instead and opens as a PREVIEW list in the same
+  buffer `gr` fills — applying a fraction of a workspace rename silently
+  would be worse than either. The ZLS backend still resolves and renames
+  current-file references via `@edit`, exactly as before.
+
+### Four kinds helix does not have
+
+The hierarchy kinds are two-step in the protocol (prepare at the cursor,
+then follow the item), are gated on the server capability so an old server
+costs zero round trips, and their answers are LOCATIONS — the one thing this
+seam renders for free. helix has no binding for any of the four (checked
+against helix-term/src/keymap/default.rs).
+
+| keys | kind | what the rows are |
+|---|---|---|
+| `SPC l c` | incoming_calls | one row per CALL SITE, under the caller's name |
+| `SPC l C` | outgoing_calls | the callees' declarations |
+| `SPC l t` | supertypes | the types this one extends/implements |
+| `SPC l T` | subtypes | the types that extend/implement this one |
+
+All four behave like `gr`: a list to walk with `n`/`N`, and a lone answer is
+a jump.
+
 ## Which ZLS, and which stdlib
 
 Both are decided at build time, and `SPC l i` prints both.
@@ -302,16 +413,18 @@ Verified against `helix-term/src/keymap/default.rs`, not from memory.
 | `gi` | implementation | |
 | `gr` | references | |
 | `SPC l k` | hover | opens `+Hover` |
-| `SPC l r` | rename | tag input; applies current-file references in one undo step |
+| `SPC l r` | rename | tag input; applies same-file edits in one undo step, PREVIEWS a multi-file WorkspaceEdit as rows |
 | `SPC l a` | code action | |
 | `SPC l h` | select references | |
 | `SPC l s` / `SPC l S` | document / workspace symbols | `S` takes a query |
 | `SPC l d` / `SPC l D` | document / workspace diagnostics | |
 | `]d` / `[d` | next / prev diagnostic | steps the list, asks for one if absent |
 | `]D` / `[D` | last / first diagnostic | |
-| `=` | format | |
+| `=` | format | applies the formatter's edits in one undo step |
 | `Ctrl`+left-click | definition | the mouse spelling of `gd` |
 | `Tab` in INSERT mode, right after a `.` | completion | what could go here, and where each of those is defined |
+| `SPC l c` / `SPC l C` | incoming / outgoing calls | beyond helix — see the client section |
+| `SPC l t` / `SPC l T` | supertypes / subtypes | beyond helix — see the client section |
 
 Tab is the one key here that is not helix's and not a goto. helix's `Tab`
 completes; pardes's shows you the CANDIDATES' DECLARATIONS in a `+Search`
@@ -340,12 +453,14 @@ afford one keystroke more. So every one of them keeps helix's own letter and
 gains the prefix — `<space>k` becomes `SPC l k`, `<space>d` becomes `SPC l d`
 — and nothing pardes had moved at all.
 
-Two more live in the same group because they belong to it, not to helix:
+Two more live in the same group because they belong to it, not to helix (the
+four hierarchy kinds above are also pardes's own — helix has no spelling for
+them):
 
 | keys | builtin | what it shows |
 |---|---|---|
-| `SPC l i` | `Lspinfo` | which ZLS, which stdlib (and whether it opens), what the backend answers and refuses, plus the last 24 queries with timings, row counts and **the errors `query` swallowed** |
-| `SPC l w` | `Lspwhy` | why the definition query at the cursor answers what it does |
+| `SPC l i` | `Lspinfo` | BOTH backends: which ZLS and which stdlib (and whether it opens); every protocol server's state, root, encoding and capabilities; and each side's last 24 queries with timings, row counts and **the errors `query` swallowed** |
+| `SPC l w` | `Lspwhy` | why the definition query at the cursor answers what it does — narrated by whichever backend the file routes to |
 
 These exist because of the seam's own contract: a backend never fails loudly,
 which is right for an editor — a thrown analyser must not take the process with
@@ -362,9 +477,13 @@ document — which matters precisely when the pane you are in is the problem.
 
 ## Writing a backend
 
-`src/lsp/lsp.zig` is the seam. An implementation supplies three things and touches
-nothing else (`backend_name` below is the SEAM's, not yours — it is a literal in
-`lsp.zig` naming whichever backend was compiled in):
+`src/lsp/lsp.zig` is the seam, and since the protocol client landed it holds a
+LIST of backends, asked in order: the first one that `speaks` the file's
+language and claims the kind in `supports` answers. An implementation supplies
+three things and touches nothing else (`backend_name` is the SEAM's, not
+yours — one literal in `lsp.zig` naming the compiled-in combination; a backend
+with unsolicited news to deliver may additionally accept the status sink, as
+`setStatusSink` shows):
 
 ```zig
 pub fn query(gpa, arena, req: Req, out: *std.Io.Writer) void