# Language intelligence in pardes Three things landed together, and only the first two are permanent: 1. **An async execution model.** The core stays a state machine; slow work goes to a worker and comes back as an event. 2. **A helix-exact keymap** for every LSP command. 3. **A seam** (`src/lsp/lsp.zig`) with exactly one function behind it, so competing backends can be swapped, measured, and thrown away. ## The async model There was none before this: every effect the core emitted was fire-and-forget (`spawn`, `write`, `save_file`) or instantaneous. A language query is the first thing pardes asks for that *answers later*, so it needed a request/response shape — and got the smallest one that works. ``` core shell worker | Effect .lsp{id,kind, | | | pane,offset,arg} | | |-------------------------->| | | | snapshot path + content | | |--------------------------->| | | | lsp.query(...) | | Event .lsp_resp{id,rows} | |<--------------------------|<---------------------------| | lspResponse -> atomic edit, jump, or results buffer | ``` The shell already ran this exact pattern for pty readers, so the async part is about thirty lines per shell: `src/tty/tty.zig` uses `io.concurrent` + the vaxis loop queue, `src/gui/gui.zig` uses a detached thread (`lspThread`) + the mutex queue it already had. A FREESTANDING core compiles in no backend at all (`zls_backend = !freestanding_core` in `build.zig`), which is both the web shell and the ESP32-P4 object: there `lsp.supports` is empty, which makes `lspRequest` return before it emits, so the effect is never even raised. `web.zig` leaves the host's `pull_lsp` null and exports nothing for a response; a host that links a backend would add both. **In a DETACHED session every query answers EMPTY.** `src/detached/server.zig`'s vtable implements sixteen of `host.zig`'s twenty-one methods, and `pull_lsp` is one of the five it leaves null — a worker pool is precisely what its deliberately single-threaded loop does not have. A null method is NOT automatically a dropped effect: `perform` decides that per arm, and the `.lsp` arm's answer is to synthesise one on the spot — an `lsp_resp` Event with `rows = ""`, fed straight back into `update`. `.pipe` one arm below does the same, yielding `pipe_resp{ .success = false, .outputs = &.{} }`, so a null `pull_pipe` is a pipe REPORTED as failed rather than one that hangs. So `gd` jumps nowhere, `gr` finds no references, `SPC l r` renames nothing (an empty edit list parses as none) and Tab after a dot offers nothing — all of it indistinguishable from a backend that found nothing, which is exactly what the seam's "no rows is a legal answer" rule promises. **Tab still indents — still, not always.** The empty response reaches `lspResponse`, whose `rows.len == 0` prong performs the indent the Tab prong skipped, but only while the cursor has not moved. That guard holds on the ordinary path because of `pump`'s order: `pull_wait_input`, then the queued events, then the effects, then render. The effect Tab emitted is performed after every keystroke that was ALREADY readable in the same round, since `pull_wait_input` applies a whole batch and not one event (the tty shell says so at the head of `waitInput` — "block for one event, then apply the whole pending batch" — and the daemon's poll loop drains every readable client `.event` straight into `core.update`). One keystroke per wake is the normal case and the indent lands in the same frame, before render. In a BURST where the key after Tab was readable in that same poll round, `cur_col` has moved by the time the prong runs, the guard fails, and the Tab really is eaten. The local shells have the same race over a wider window, so this is a property of the late-indent repair rather than of detaching. None of the detached core's five null methods silently drops a reachable effect. `pull_lsp` and `pull_pipe` have the fallbacks above; `pull_gpio_toggle` is `orelse return Error.NoPads` (`board_memory.zig`), which lands on the message row; `push_post_present` is a `pump` hook fired after presenting, and there is nothing to notify in a process with no screen; and `push_fs_reply` is the one `perform` arm with no fallback at all, but it is only ever emitted in answer to an `Event.fs_req`, which is not on the wire (`wire.zig`: it has no `ClientTag`, because neither half of that pair may cross an attachment) and which the daemon raises none of, mounting no `/dev/fuse` by its own vtable comment. Four rules make it safe: - **The worker never touches the core.** Path, source, arg and root are copied into an `LspJob` before it starts — one per shell, in `src/tty/tty.zig` and `src/gui/gui.zig`. The user keeps typing while a query is in flight; a borrowed slice would be a use-after-free the length of one keystroke. - **One query in flight, identified by a monotonic id.** A second press bumps the id, which makes the older answer stale. The pending request also records the pane serial, so a closed-and-reused slot cannot accept its response. - **Mutating answers are revision-checked.** Rename records the file revision sent to the worker and applies nothing if the user edited before it answered. - **No rows is a legal answer.** A backend that cannot answer appends nothing, which is indistinguishable from a language server still starting up, and the core does nothing. There is no error path to render. ## Results are `+Search` rows Every backend renders into one format: ``` sub/file.zig:LINE:COL text under the asking window's dir sub/file.zig:LINE:COL-ENDCOL text /abs/path/elsewhere.zig:LINE:COL text anywhere else ``` 1-based line and column. The second form carries the answer's RANGE where the protocol gave one on a single line (a token, a symbol's name), and a look on it SELECTS that span rather than parking at its first cell — so `gd` lands on the whole name, and a `gr` reference stepped to with `n`/`N` and opened with Enter arrives with the reference itself selected. This is the format `look.zig` already resolves, `runSearch` already produces and `n`/`N` already walk — so: - **one row from a goto** → jump straight there (`lookAt`) - **several rows** → an output buffer, which `n`/`N` walk: a step SELECTS a row and Enter opens it which means helix's multi-result picker required **no picker code at all**. The `+Search` buffer *is* the picker. Non-location answers (`hover`, `code_action`, `format`) open `+Hover`/`+Lsp` instead, which are prose and not places — `n`/`N` find nothing look-able in a documentation blurb and walk straight past it to the next pane on the ring. Rename is deliberately the one exception to rows as presentation. The backend emits `@edit START END` records through `lsp.edit()`, using half-open byte offsets into the exact `Req.source` snapshot it resolved. The core validates that every range is ordered, non-overlapping and in bounds, checks that the pane serial and file revision still match, then substitutes the requested name across all ranges with one allocation and one undo transaction. A malformed, stale, or empty response changes nothing. The current ZLS backend resolves and renames references in the **current file only**; it does not claim a workspace rename. **A path UNDER `Req.root` is written relative to it; everything else keeps its full absolute path** (`lsp.rel`). `Req.root` is the directory of the file the query was asked about — the window that generated the buffer — and a `gr` over one file was otherwise the same forty-character prefix repeated down the whole pane, with the part you came to read pushed off the right edge. The short form resolves because `Req.root` is also the directory the results buffer is *named* in (`output_pane.open`), and a look resolves a relative word against the directory of the pane it was clicked in — which is that buffer. Under, never "shorter": a hit outside the tree is **not** walked up to with `../`. An absolute path resolves from anywhere and says where it is; a `../..` chain says neither, and stops being true the moment the row is read anywhere but beside its own buffer. `test/snapshots/lsprelpath.snap` pins both directions end to end — the enum one level up (absolute row) and one level down (`inner/tint.zig`, a stripped path that still has a separator in it), each with the `n` step that selects it and the Enter that opens it, plus a right click. This is the rule `look.grep` already follows for its own rows, and it is spelled TWICE: `lsp.rel` here, and an inline `if` over the asking pane's directory in `look.zig`'s `grep` (the `shown` computation) there. Same rule, two implementations — they want to become one function, and `lsp.zig`'s own comment on `rel` says so. `completion` is the kind this shape changes the most. Every other editor answers a dot with a popup of NAMES to insert; a seam that returns locations cannot insert anything, so this one answers with the candidates' **declarations** — one row each, in the same `+Search` buffer, steppable with `n`: ``` path:LINE:COL-ENDCOL name the candidate's declaration line a.zig:2:5-13 verdigris verdigris, ``` The name comes first because that is the thing you would type — the row answers "what goes here" before "where does it come from", which is the order the question was asked in; every other kind here answers a WHERE, and for this one the location is the evidence rather than the answer. It is not padded into a column: the location in front of it is already ragged, so there is nothing to align to. That is a different and arguably better answer to "what goes here": you get the word AND you can read the definition rather than a list of words. It is the one location kind that does NOT jump on a single row, because with one candidate you still want to see the list rather than be teleported into it. A results buffer is REFILLED rather than reopened when the same kind is asked again — the rule `runSearch` always had, and which the language path was missing. It survived being missing while every query was a deliberate press (`gr` twice left two identical lists and you closed one); Tab after a dot is an ordinary typing keystroke, and measured, twenty of them stacked **fifteen** byte-identical `+Search` panes, crushed the file to one visible line, and then ran `freeSlot` out so the key was silently eaten for the rest of the session. Unlike a search the ARGUMENT is not part of the identity: a language query is asked about a different symbol every time with the same (usually empty) arg, so the kind is the unit. Making it work needed one trick. A completion is asked for exactly when the line is half-typed, and a half-typed line does not parse: `switch (e) { . }` loses the whole switch to the parser's error recovery, taking with it every ancestor an expected-type resolution needs. ZLS answers this with a private token scanner welded to its `*Server`. `lsp_zls.completionSource` instead makes the tree PARSE — it splices a placeholder in after the dot, in the six spellings a half-typed line can need — three shapes, each with and without a closer still hanging: a switch prong (`_p => {},` / `_p => {}, }`), an unterminated statement (`_p;` / `_p)`), and a bare identifier (`_p` / `_p }`) — and keeps the one that both makes the dot reachable in the tree and leaves the fewest parse errors. Everything after that is ZLS's ordinary public resolution over an ordinary tree. **A Tab the backend cannot answer still indents.** The keystroke has already diverted by the time "no rows" comes back, so `lspResponse` performs the indent the Tab prong skipped — on the condition that the cursor has not moved since, so nobody who kept typing gets four spaces landing behind their hands. Without that, a dot in a comment, in a string, or on a line nothing can be made of ate the keystroke outright. With several cursors Tab never diverts at all: a language query is a per-keystroke action inside a per-selection replay, so asking would stop the replay dead and collapse the multicursor. ### What it costs, and what it cannot do Per press. **Both timings date from 2026-08-09**, change `lmlltvrx`, and have not been re-measured; the line count beside the first was refreshed once afterwards, on 2026-08-12 in change `vwtlskzr`, and `src/pardes.zig` is 16 466 lines today. The ReleaseFast column is `zig build lspbench`, which is pinned to ReleaseFast in `build.zig` and always has been — so the Debug column came from running the installed editor by hand and the repo records no harness for it. A completion parses the buffer once per placeholder spelling it tries, so the first row scales with the file: re-run rather than trusting either number. | | ReleaseFast | Debug (what `zig build` installs) | |---|---|---| | a switch arm in `src/pardes.zig` (14.6k lines) | 8.8 ms | 87 ms | | `std.` — 91 candidates, each alias-resolved into the stdlib | 26 ms | 204 ms | It is a worker thread, so the editor does not block. On the TTY shell the second press of Tab then joins the first query on the UI thread — `old.cancel(s.io)` on the one in-flight future, and a backend that ignores cancellation means waiting out a query the user already abandoned (`src/tty/tty.zig`, the `lsp` vtable entry, whose own comment says so). The GUI shell does not join: it spawns another thread per request and lets the core's monotonic id make the older answer stale, so it pays memory instead of latency. Pre-existing and shared by every LSP kind — not this feature's to fix, but it is what a fast double-Tab feels like on a terminal. Known limitations, in the order you will meet them: - **`@This()` anywhere in a container makes the whole container unresolvable**, so `var list: std.ArrayList(u8) = .` — the most common decl literal in this codebase — answers nothing. This is not the completion filter: `hover` and a plain field access on the same struct return nothing either. It is the case a user hits first, and it is upstream of everything here. - **Only the break AT THE CURSOR is repaired.** Zig's error recovery runs forward, so an unrepaired break earlier in the file swallows the declaration the cursor is in and the answer is empty. While typing you normally have one broken spot, which is the case this works for. - **A dependency module** (`@import("vaxis")`) cannot be typed at all, for the same reason `gd` on `vaxis.init` finds nothing. - **`error.`** is not handled — the position context is `.error_access`, which no branch claims. ## Which ZLS, and which stdlib Both are decided at build time, and `SPC l i` prints both. `build.zig.zon` pins ZLS to a COMMIT rather than a tag — `git+https://github.com/zigtools/zls#3e0d082084be43e36865136a138c1fe2023b33ca`, on the 0.16.x branch — because master requires Zig 0.17-dev and no tagged release both builds on 0.16 and exports the internals this backend calls. `build.zig` spells the same commit a second time, as the top-level `const zls_version = "0.16.1-dev+3e0d0820"`, and hands it to the ZLS package's own `-Dversion-string` and to `pardes_config.zls_version`. The duplication is unavoidable rather than sloppy — the semver half (`0.16.1-dev`) exists nowhere in the manifest — and it is load-bearing, because ZLS's build otherwise derives that string from `git describe`, which has nothing to read in a fetched package with no `.git`. The two are made to AGREE BY CONSTRUCTION: a `comptime` block right below the constant takes the short hash after the `+`, takes the pinned commit after the `#` in `zon.dependencies.zls.url`, and `@compileError`s unless the first is a prefix of the second. A `.zon` bump that forgets `build.zig` is therefore a build error, not a `SPC l i` naming a build nobody linked. `std` is the harder half. ZLS resolves `@import("std")` through `zig_lib_dir` and through nothing else, and this backend sets `zig_exe_path = null` on purpose — asking the `zig` binary is the subprocess the whole design exists to avoid. So `build.zig` bakes `b.graph.zig_lib_directory.path` into `pardes_config.zig_lib_dir`: the exact stdlib pardes itself was compiled against, which is what makes `gd` on `std.mem.count` land in the real `mem.zig`. `zigLibPath()` (`src/lsp/lsp_zls.zig`) reads `ZIG_LIB_DIR` from the environment FIRST and falls back to the baked path, so a user who moved the toolchain can point the backend at it without rebuilding. With no lib dir at all every `std` symbol is a silent miss — which is why `SPC l i` reports whether the directory OPENS rather than only which one was compiled in. That same `zig_exe_path = null` is the dependency-module limitation above. ZLS resolves a relative `.zig` path from the filesystem and `std` from the lib dir, but any other import name — every dependency in `build.zig.zon` — it can only answer by running `zig build --build-runner` to discover the module graph. With no zig binary that branch returns nothing, so `@import("vaxis")` is a silent miss by construction rather than by omission. ## The keymap is helix's, exactly Verified against `helix-term/src/keymap/default.rs`, not from memory. | keys | command | notes | |---|---|---| | `gd` | definition | jumps on a single result, lists on several | | `gD` | declaration | | | `gy` | type definition | | | `gi` | implementation | | | `gr` | references | | | `SPC l k` | hover | opens `+Hover` | | `SPC l r` | rename | tag input; applies current-file references in one undo step | | `SPC l a` | code action | | | `SPC l h` | select references | | | `SPC l s` / `SPC l S` | document / workspace symbols | `S` takes a query | | `SPC l d` / `SPC l D` | document / workspace diagnostics | | | `]d` / `[d` | next / prev diagnostic | steps the list, asks for one if absent | | `]D` / `[D` | last / first diagnostic | | | `=` | format | | | `Ctrl`+left-click | definition | the mouse spelling of `gd` | | `Tab` in INSERT mode, right after a `.` | completion | what could go here, and where each of those is defined | Tab is the one key here that is not helix's and not a goto. helix's `Tab` completes; pardes's shows you the CANDIDATES' DECLARATIONS in a `+Search` buffer and inserts nothing, because that is what a seam returning locations can honestly do — see below. It only diverts where an answer is possible: on a terminal, in an output buffer, or in a file the backend does not speak (`lsp.speaks`, which the core asks and the backend answers), Tab indents exactly as it always did. A Tab that silently does nothing would be worse than not having the feature. Nothing about the mode changes either — the pane is still in insert, so typing goes on and walking the answer with `n` means pressing `Esc` first, the same as for every other results buffer. Ctrl-click rides the ordinary left-click drag rather than firing on the press: a click does not place the modal cursor until RELEASE, so a query asked at press time would answer about wherever the cursor previously sat. A ctrl-DRAG still selects, and still asks about where it started. **The gotos are helix's exactly; the leader commands are helix's letters under an `l` prefix.** `g` and `[`/`]` had no conflicts — `gd/gD/gy/gi/gr` and `]d/[d` were all free, so they stay where helix puts them. The bare `` letters were NOT free, and an earlier pass took them anyway, displacing `SPC d` (Del), `SPC k` (Kill), `SPC s?` (Dump/Restore) and `SPC h t` (Tutor). That is the wrong trade: those are pardes's most-pressed keys and predate the language work, whereas an LSP command is something you reach for deliberately and can afford one keystroke more. So every one of them keeps helix's own letter and gains the prefix — `k` becomes `SPC l k`, `d` becomes `SPC l d` — and nothing pardes had moved at all. Two more live in the same group because they belong to it, not to helix: | keys | builtin | what it shows | |---|---|---| | `SPC l i` | `Lspinfo` | which ZLS, which stdlib (and whether it opens), what the backend answers and refuses, plus the last 24 queries with timings, row counts and **the errors `query` swallowed** | | `SPC l w` | `Lspwhy` | why the definition query at the cursor answers what it does | These exist because of the seam's own contract: a backend never fails loudly, which is right for an editor — a thrown analyser must not take the process with it — but it makes a broken backend and a correct one that found nothing look identical. `Lspwhy` narrates the REAL resolution path (the position context is the analyser's own answer, threaded out through a trace) rather than re-deriving it beside the code, because a debug view that reimplements the logic is one that can disagree with it. `Lspinfo` answers from ANY pane, including one with no file, since it is about the backend rather than a document — which matters precisely when the pane you are in is the problem. `K` is **not** hover — in helix it is `keep_selections`. It was checked; do not "fix" it. ## Writing a backend `src/lsp/lsp.zig` is the seam. An implementation supplies three things and touches nothing else (`backend_name` below is the SEAM's, not yours — it is a literal in `lsp.zig` naming whichever backend was compiled in): ```zig pub fn query(gpa, arena, req: Req, out: *std.Io.Writer) void pub fn speaks(path: []const u8) bool pub const supports: std.EnumSet(Kind) pub const backend_name = "..." ``` `supports` says what a backend can do; `speaks` says what it can do it TO, and exists for the one key that must not be eaten when the answer is no — see the Tab note above. `query` runs on a worker thread with no access to the core — everything it may read is in `req` (`path`, `source` (NUL-terminated), `offset`, `arg`, `root`). `out` is a plain `std.Io.Writer`: the shell owns the buffer behind it (an `Io.Writer.Allocating`), so a backend never allocates the result, never frees it, and cannot get the allocator wrong. `arena` is freed wholesale on return; `gpa` is for a backend's own scratch. Use `lsp.row()` to emit a location, `lsp.spanRow()` for one that carries the RANGE it matched (the `path:LINE:COL-ENDCOL` form above), `lsp.rel()` to spell a path against `req.root`, `lsp.lineCol()` to convert an offset, and `lsp.edit()` for each half-open range of a rename response. Location rows are byte-identical across backends; rename ranges are consumed by the core and never rendered. `rel` allocates nothing — it returns a slice of what you hand it. ## How the implementations are judged `zig build lspbench` — same harness, same corpus, same 22 probes, every backend. The corpus is pardes's own `src/`, plus `test/lspfixture/`: five of the probes point at fixtures rather than at real source. Three of them do because on clean, already formatted code the correct answer to `diagnostics`, `workspace_diagnostics` and `format` is nothing, and that is indistinguishable from a backend that has neither — `broken.zig` carries an unused local and a misformatted fn, so all three have real work. The other two are the half-typed dot, whose two shapes are `dotcomplete.zig` (a switch prong that parses everywhere but at the dot) and `dothalf.zig` (a line also missing its terminator). The 22 probes cover 15 of the 17 `lsp.Kind`s — `definition` three times, `document_symbols` twice, `completion` five times, and the two introspection kinds (`status`, `explain`) not at all. - **Feature completeness.** Which kinds return rows, and whether the rows contain what they should. The harness trusts *results*, not the `supports` flag: a kind claimed but returning nothing is reported as `CLAIMED-EMPTY`, and a kind that answers without claiming is `unclaimed-works`. Correctness is a substring the rows must contain, so returning a confident wrong location scores worse than returning nothing. - **Latency.** `cold` (first query, index construction included) and `warm` (median of 20). They differ by orders of magnitude for an indexing backend and both matter: cold is what the first keypress costs, warm is what every one after it costs. - **Memory.** Peak RSS delta (`VmHWM`), so a backend that frees its index before returning still pays for having built it. - **Lines of code.** Not measured by the harness — it is `jj diff --stat` against the base commit. Less is better, and vendoring a library is not free but is charged in build time and dependency surface rather than in lines we maintain. The user-visible rename contract is also pinned through the actual TTY, leader prompt, worker and ZLS backend by `test/snapshots/lsp-rename.snap`: both resolved occurrences change, a shadowed local does not, and undo/redo treats the response as one transaction. Run `zig build lspbench -- --json` for machine-readable output.