diff options
Diffstat (limited to 'src')
| -rw-r--r-- | src/CHANGELOG.md | 18 | ||||
| -rw-r--r-- | src/config.zig | 10 | ||||
| -rw-r--r-- | src/file_pane.zig | 61 | ||||
| -rw-r--r-- | src/grammar_manifest.zig | 5 | ||||
| -rw-r--r-- | src/pardes.zig | 154 | ||||
| -rw-r--r-- | src/pdf_pane.zig | 35 | ||||
| -rw-r--r-- | src/pdf_pane_integration_test.zig | 62 | ||||
| -rw-r--r-- | src/syntax.zig | 264 | ||||
| -rw-r--r-- | src/tutor.txt | 28 |
9 files changed, 586 insertions, 51 deletions
diff --git a/src/CHANGELOG.md b/src/CHANGELOG.md index 61e670d6..b2bc9583 100644 --- a/src/CHANGELOG.md +++ b/src/CHANGELOG.md @@ -32,3 +32,21 @@ - `zig build run-isolated` builds and runs `pardes-isolate`: the same source with one comptime option, whose host supplies only the terminal. The filesystem is not disabled in it, it is absent — the libc paths compile away. +- Typst highlighting: headings, `*bold*` spans, raw/code blocks and inline raw + spans, and `#`-prefixed function calls (including the sigil itself). +- Fenced Typst raw blocks with a language tag (e.g. ```` ```zig ````) get that + language's grammar injected, the same treatment markdown fenced blocks + already had. +- Markdown bold, italics and inline code spans are highlighted: pardes now + builds the markdown inline grammar, a second parser off the same dependency, + and re-parses the block grammar's `inline` nodes with it. Only the block + grammar was built before, so emphasis had no nodes to match. +- Escape in a PDF pane is the document's own cancel: it drops the mouse + selection and the search highlight overlay and stays where you were reading. + It used to run `Last`, which moved focus out of the pane and left every + following key going somewhere else. `Shift-Esc` is now the hop out — the same + chord that leaves a raw tty. +- `gj`/`gk` move one VISUAL line, following the automatic breaks of a + soft-wrapped body and keeping the goal column inside the row. `j`/`k` are + unchanged: whole file lines. With `Wrap` off a line is one visual row and the + two pairs are the same motion. diff --git a/src/config.zig b/src/config.zig index 685d083a..63f97db1 100644 --- a/src/config.zig +++ b/src/config.zig @@ -328,6 +328,16 @@ pub const topbar_down: []const Chord = &.{.{ .cp = 'j' }}; pub const topbar_left: []const Chord = &.{.{ .cp = 'h' }}; pub const topbar_right: []const Chord = &.{.{ .cp = 'l' }}; +/// LEAVE the pane you are in — the `Last` builtin — from a pane whose own +/// plain Escape already means something else. A terminal in raw tty mode has +/// had it since it existed (tty_toggle_alt below, same chord, same job); a PDF +/// needs it because Escape there cancels the selection and the search overlay +/// without moving focus out of the document you are reading. +/// +/// On a host that reports no modifier on Escape it arrives as a plain Escape +/// and still means what Escape always means in that pane. +pub const leave_pane: []const Chord = &.{.{ .cp = Key.escape, .shift = true }}; + // ---- raw tty mode ---- /// Ctrl-<this> toggles raw tty mode in and out (terminals only); tty is diff --git a/src/file_pane.zig b/src/file_pane.zig index 42e5dba9..80c31edb 100644 --- a/src/file_pane.zig +++ b/src/file_pane.zig @@ -559,25 +559,60 @@ pub fn refreshHighlights(p: *Pardes) void { } } +/// The width a wrapped row of THIS pane holds, in cells, or 0 when the pane is +/// not wrapping — the render decision, named once so motion cannot disagree +/// with paint. One column is left for the break marker: a row that filled its +/// last cell would have nowhere to say it continues. A pane taller than the +/// map refuses to wrap rather than record part of itself (see Pane.wrap_line). +pub fn wrapWidth(pane: *const Pane, wrap: bool) usize { + if (!wrap or pane.rows > pane.wrap_line.len) return 0; + return @max(1, @as(usize, pane.cols -| config.PREFIX_W) -| 1); +} + +pub const VisualRow = struct { start: usize, end: usize }; + +/// The visual row of `line` holding byte `col`: `[start, end)`, where `end` is +/// where the next visual row of the same line begins and equals `line.len` on +/// the last one. This is the same walk `fillBody` renders with, so `gj`/`gk` +/// step exactly the breaks a reader sees. `width == 0` (not wrapping) makes +/// the whole line one visual row, which is what collapses visual motion onto +/// textual motion instead of special-casing it upstream. +pub fn visualRow(line: []const u8, col: usize, width: usize) VisualRow { + if (width == 0) return .{ .start = 0, .end = line.len }; + var start: usize = 0; + while (true) { + const end = fitEnd(line, start, width); + if (col < end or end >= line.len) return .{ .start = start, .end = end }; + start = end; + } +} + +test "visual rows partition a line at the breaks the body renders" { + const line = "abcdefgh"; + try std.testing.expectEqual(VisualRow{ .start = 0, .end = line.len }, visualRow(line, 5, 0)); + try std.testing.expectEqual(VisualRow{ .start = 0, .end = 3 }, visualRow(line, 0, 3)); + try std.testing.expectEqual(VisualRow{ .start = 0, .end = 3 }, visualRow(line, 2, 3)); + try std.testing.expectEqual(VisualRow{ .start = 3, .end = 6 }, visualRow(line, 3, 3)); + // Past the end (a cursor on the newline) names the LAST row, and a short + // line is one row however narrow the pane is. + try std.testing.expectEqual(VisualRow{ .start = 6, .end = 8 }, visualRow(line, line.len, 3)); + try std.testing.expectEqual(VisualRow{ .start = 0, .end = 0 }, visualRow("", 0, 3)); + // A grapheme wider than the row still occupies exactly one row. + try std.testing.expectEqual(VisualRow{ .start = 0, .end = 4 }, visualRow("👩x", 0, 1)); +} + /// the body a file pane renders: `pane.rows` SCREEN rows from the scroll /// offset, each behind its right-aligned line number, then cut by hscroll. /// /// With `wrap` on a line too long for the pane takes several rows instead of /// running off the right edge, and this is where that happens — it is a render -/// property and nothing else in the editor learns about it. What every other -/// site needs is the map back: which line each row showed and at which byte -/// column it began, recorded into pane.wrap_line/wrap_col as the rows are -/// built. `wrap_n` stays 0 for an unwrapped body, and that is the value the -/// readers treat as "rows are lines", so the off path never consults an array. +/// property, and the one thing the rest of the editor reads back is the map: +/// which line each row showed and at which byte column it began, recorded into +/// pane.wrap_line/wrap_col as the rows are built. `wrap_n` stays 0 for an +/// unwrapped body, and that is the value the readers treat as "rows are +/// lines", so the off path never consults an array. pub fn bodyText(arena: std.mem.Allocator, pane: *Pane, f: *State, wrap: bool) ![]const u8 { - // the pane's own content width, less the ONE column the break marker sits - // in — a wrapped row that filled its last cell would have nowhere to say it - // wrapped. 0 means don't wrap. A pane taller than the map refuses too, - // rather than recording part of itself — see Pane.wrap_line. - const width: usize = if (!wrap or pane.rows > pane.wrap_line.len) - 0 - else - @max(1, @as(usize, pane.cols -| config.PREFIX_W) -| 1); + const width = wrapWidth(pane, wrap); pane.wrap_n = 0; // Count the exact rendered bytes first. Unwrapped source lines are not diff --git a/src/grammar_manifest.zig b/src/grammar_manifest.zig index 01df9e34..0303af17 100644 --- a/src/grammar_manifest.zig +++ b/src/grammar_manifest.zig @@ -34,6 +34,11 @@ pub const all = [_]Grammar{ .{ .name = "kotlin", .dep = "ts_kotlin", .exts = &.{ ".kt", ".kts" }, .tier = .full, .scanner = true }, .{ .name = "ocaml", .dep = "ts_ocaml", .exts = &.{ ".ml", ".mli" }, .tier = .full, .src = "grammars/ocaml/src", .scanner = true }, .{ .name = "markdown", .dep = "ts_markdown", .exts = &.{ ".md", ".markdown" }, .tier = .full, .src = "tree-sitter-markdown/src", .scanner = true, .query = "tree-sitter-markdown/queries/highlights.scm" }, + // Second grammar off the SAME dep: the split_parser markdown repo ships a + // block parser and an inline parser, and bold/italic/code-span live only in + // the inline one. `.exts` is empty because no filename ever selects it — the + // block grammar's `inline` nodes are re-parsed with it by name (forLang). + .{ .name = "markdown_inline", .dep = "ts_markdown", .exts = &.{}, .tier = .full, .src = "tree-sitter-markdown-inline/src", .scanner = true, .query = "tree-sitter-markdown-inline/queries/highlights.scm" }, .{ .name = "pascal", .dep = "ts_pascal", .exts = &.{ ".pas", ".pp", ".p" }, .tier = .full }, .{ .name = "php", .dep = "ts_php", .exts = &.{ ".php", ".phtml", ".php3", ".php4", ".php5" }, .tier = .full, .src = "php/src", .scanner = true }, .{ .name = "powershell", .dep = "ts_powershell", .exts = &.{ ".ps1", ".psm1", ".psd1" }, .tier = .full, .scanner = true }, diff --git a/src/pardes.zig b/src/pardes.zig index 405cc3f2..0a390161 100644 --- a/src/pardes.zig +++ b/src/pardes.zig @@ -254,6 +254,72 @@ test "selection pipe prompt submits exact request and Escape cancels" { try std.testing.expect(nextPipeEffect(p) == null); } +test "gj/gk step the wrapped rows a body draws while j/k keep the file's lines" { + const gpa = std.testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true }); + defer p.deinit(); + while (p.nextEffect()) |_| {} + const long = "a" ** 400; + const pane = try p.hxOpenFileContent(long ++ "\nsecond\n"); + p.settings.wrap = true; + const width = file_pane.wrapWidth(pane, true); + try std.testing.expect(width > 4 and long.len > width * 3); + + // gj holds the column INSIDE the row and lands on the next break; the line + // is one line the whole way down, which is the whole point. + pane.cur_row = 0; + pane.cur_col = 3; + pane.sticky_col = -1; + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, 0), pane.cur_row); + try std.testing.expectEqual(@as(i32, @intCast(width + 3)), pane.cur_col); + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, @intCast(width * 2 + 3)), pane.cur_col); + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'k' } }); + try std.testing.expectEqual(@as(i32, @intCast(width + 3)), pane.cur_col); + // A count is counted in rows, not lines. + p.update(.{ .key = .{ .cp = '2' } }); + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, 0), pane.cur_row); + try std.testing.expectEqual(@as(i32, @intCast(width * 3 + 3)), pane.cur_col); + + // Plain j/k are untouched: one FILE line each, whatever the body drew. + pane.cur_col = 3; + pane.sticky_col = -1; + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, 1), pane.cur_row); + p.update(.{ .key = .{ .cp = 'k' } }); + try std.testing.expectEqual(@as(i32, 0), pane.cur_row); + + // The last row of a line steps into the next line's first row, and the + // first row steps back onto the previous line's LAST row. + pane.cur_row = 0; + pane.cur_col = @intCast(long.len - 1); + pane.sticky_col = -1; + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, 1), pane.cur_row); + pane.cur_col = 0; + pane.sticky_col = -1; + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'k' } }); + try std.testing.expectEqual(@as(i32, 0), pane.cur_row); + try std.testing.expect(pane.cur_col >= @as(i32, @intCast(long.len - long.len % width))); + + // Wrap OFF: a line is one visual row, so gj/gk ARE j/k again. + p.settings.wrap = false; + pane.cur_row = 0; + pane.cur_col = 3; + pane.sticky_col = -1; + p.update(.{ .key = .{ .cp = 'g' } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(i32, 1), pane.cur_row); +} + test "selection pipe replaces all ranges atomically and undo restores them" { const gpa = std.testing.allocator; const p = try Pardes.init(gpa, .{ .tty_only = true }); @@ -8258,6 +8324,76 @@ pub const Pardes = struct { pane.sticky_col = @intCast(goal); } + /// `gj`/`gk`: one VISUAL line, following the automatic breaks a wrapped + /// body draws rather than the newlines in the file. The goal column is the + /// one INSIDE the visual row, so a run of them walks straight down a + /// paragraph; on the last visual row of a line the step crosses into the + /// next line's first row, exactly as the eye does. + /// + /// With wrapping off — Wrap unset, a terminal pane, an output pane too + /// narrow to record its map — a line is one visual row and this IS + /// verticalMove, which is why nothing upstream branches on the setting. + fn visualMove( + pane: *Pane, + pl: PaneLines, + text: []const u8, + range: modal.HxRange, + down: bool, + cnt: usize, + width: usize, + ) void { + const cur = modal.hxCursor(text, range); + const pos = panePos(pane, text, cur); + const last_row = paneLineCount(pane, text) - 1; + var row = pos.row; + var line = modal.lineSlice(text, row); + var vrow = file_pane.visualRow(line, pos.col, width); + const goal: usize = if (pane.sticky_col >= 0) + @intCast(pane.sticky_col) + else + file_pane.rawDisplayCol(line[vrow.start..vrow.end], pos.col -| vrow.start); + var steps = @max(1, cnt); + while (steps > 0) : (steps -= 1) { + if (down) { + if (vrow.end < line.len) { + vrow = file_pane.visualRow(line, vrow.end, width); + continue; + } + if (row == last_row) break; + row += 1; + line = modal.lineSlice(text, row); + vrow = file_pane.visualRow(line, 0, width); + } else { + if (vrow.start > 0) { + vrow = file_pane.visualRow(line, vrow.start - 1, width); + continue; + } + if (row == 0) break; + row -= 1; + line = modal.lineSlice(text, row); + vrow = file_pane.visualRow(line, line.len, width); + } + } + // The newline slot is a real cursor position, but the first byte of the + // NEXT visual row is not: landing there would read as two rows moved. + var target_col = vrow.start + file_pane.rawAtDisplay(line[vrow.start..vrow.end], goal); + if (vrow.end < line.len and target_col >= vrow.end) + target_col = modal.graphemeStart(line, vrow.end - 1); + const t = paneOff(pane, text, .{ .row = row, .col = target_col }); + // extend mode never walks onto the empty trailing line (helix) + if (pane.select and t == text.len and text.len > 0 and text[text.len - 1] == '\n') return; + setPaneRange(pane, pl, text, modal.hxPutCursor(text, range, t, pane.select), false); + pane.sticky_col = @intCast(goal); + } + + /// How wide a wrapped row of this pane is, or 0 when it does not wrap. + /// Only a file-backed body wraps: a terminal's rows are the emulator's + /// own, already broken where it decided to break them. + fn paneWrapWidth(p: *const Pardes, pane: *const Pane) usize { + if (pane.file == null) return 0; + return file_pane.wrapWidth(pane, p.settings.wrap); + } + /// Ctrl-d/u: scroll half a page AND move the cursor by the same rows fn halfPageMove(pane: *Pane, pl: PaneLines, text: []const u8, range: modal.HxRange, down: bool) void { const half: i32 = @max(1, @divTrunc(@as(i32, pane.rows), 2)); @@ -8414,8 +8550,8 @@ pub const Pardes = struct { if (nw == slice.len) return; return pointMove(pane, pl, text, range, ls + nw); }, - .line_down => return verticalMove(pane, pl, text, range, true, go.count), - .line_up => return verticalMove(pane, pl, text, range, false, go.count), + .line_down => return visualMove(pane, pl, text, range, true, go.count, p.paneWrapWidth(pane)), + .line_up => return visualMove(pane, pl, text, range, false, go.count, p.paneWrapWidth(pane)), .column => { const line = modal.hxLineOf(text, cur); const ls = modal.lineStartOffset(text, line); @@ -12360,6 +12496,16 @@ pub const Pardes = struct { if (comptime !pdf_enabled) return; if (pane.pdf == null) return; var state = paneNormalState(pane); + // The way OUT of a PDF, because plain Escape below is the document's + // own cancel (selection, search overlay) and never moves focus. Before + // the parser: `config.escape` matches a shifted Escape too, since shift + // is consulted only where a binding asks for it. + if (hit(key, config.leave_pane)) { + state.clear(); + putPaneNormalState(pane, state); + pane.select = false; + return p.runBuiltin(.Last, p.active, "", null); + } // A PDF has no body character to find, so its bare `f` is the direct // document-outline door. Prefix continuations still go through the // shared parser, and text/terminal panes retain `f<char>` unchanged. @@ -12385,10 +12531,6 @@ pub const Pardes = struct { if (result.page_changed) pdf_pane.resetPageChrome(pane); switch (result.host) { .none => {}, - .last => { - pane.select = false; - p.runBuiltin(.Last, p.active, "", null); - }, .leader => { p.leader_on = true; p.leader_n = 0; diff --git a/src/pdf_pane.zig b/src/pdf_pane.zig index f34a457d..8f163488 100644 --- a/src/pdf_pane.zig +++ b/src/pdf_pane.zig @@ -424,6 +424,18 @@ pub const State = if (enabled) struct { if (changed) state.invalidateRaster(state.page); } + /// Escape's cancel: everything transient a reader can SEE — the mouse + /// selection and the search overlay — and nothing that says WHERE in the + /// document they are. Page, scroll, fit and tint are what the pane is, not + /// chrome. Allocation-free, so it cannot half-cancel. + pub fn cancelChrome(state: *@This(), gpa: std.mem.Allocator) void { + state.clearDrag(); + state.clearSelection(gpa); + if (state.search_query.len == 0) return; + state.dropSearchQuery(gpa); + state.invalidateRaster(state.page); + } + fn invalidatePage(state: *@This(), gpa: std.mem.Allocator) void { state.dropSearchResults(gpa); state.dropSelection(gpa); @@ -438,13 +450,23 @@ pub const State = if (enabled) struct { state.search_hit = 0; } + /// Forget the query, its hits, and every flag derived from them. Shared by + /// the cancel above and by the replacement below, which owns new bytes the + /// caller allocated before anything here was dropped. + fn dropSearchQuery(state: *@This(), gpa: std.mem.Allocator) void { + if (state.search_query.len > 0) gpa.free(state.search_query); + state.search_query = &.{}; + state.search_hit = 0; + state.dropSearchResults(gpa); + state.highlights_dirty = false; + state.search_reveal_pending = false; + } + pub fn setSearchQuery(state: *@This(), gpa: std.mem.Allocator, query: []const u8) !void { if (std.mem.eql(u8, state.search_query, query)) return; const owned = try gpa.dupe(u8, query); - if (state.search_query.len > 0) gpa.free(state.search_query); + state.dropSearchQuery(gpa); state.search_query = owned; - state.search_hit = 0; - state.dropSearchResults(gpa); state.highlights_dirty = query.len > 0; state.search_reveal_pending = query.len > 0; state.invalidateRaster(state.page); @@ -709,7 +731,6 @@ pub const ScrollResult = struct { active_page: usize }; pub const CellPixels = struct { w: u16, h: u16 }; pub const NormalHost = enum { none, - last, leader, command_line, search, @@ -1384,7 +1405,11 @@ pub fn applyNormal( if (comptime !enabled) return .{}; var result: NormalResult = .{}; switch (semantic) { - .escape => result.host = .last, + // Escape stays in the document and cancels what is drawn over it. The + // way OUT of a PDF is Shift-Escape, which the host takes before this + // parse ever runs — a reader who pressed Escape to drop a selection + // was not asking to be moved to another pane. + .escape => state.cancelChrome(gpa), .move => |move| switch (move.motion) { .down => result.page_changed = moveRows( state, diff --git a/src/pdf_pane_integration_test.zig b/src/pdf_pane_integration_test.zig index b290616e..2e671f49 100644 --- a/src/pdf_pane_integration_test.zig +++ b/src/pdf_pane_integration_test.zig @@ -906,6 +906,68 @@ test "PDF normal adapter consumes unsupported actions and navigates page fallbac try std.testing.expect(!pane.tag_edit); } +test "Escape cancels PDF chrome in place and Shift-Escape leaves the pane" { + if (!pdf_enabled or platform == .web) return; + + const gpa = std.testing.allocator; + const p = try Pardes.init(gpa, .{ + .file = "docs/design.pdf", + .cols = 80, + .rows = 28, + }); + defer p.deinit(); + const pane = p.panes[0].?; + const pv = &pane.pdf.?; + p.native_images = true; + + // A second pane, then focus it and come back: `Last` has somewhere to go + // and the PDF is where the keys land. + pdf_pane.openSections(p, 0); + p.update(.{ .key = .{ .cp = 'w', .ctrl = true } }); + p.update(.{ .key = .{ .cp = 'j' } }); + try std.testing.expectEqual(@as(usize, 1), p.active); + p.update(.{ .key = .{ .cp = 'w', .ctrl = true } }); + p.update(.{ .key = .{ .cp = 'k' } }); + try std.testing.expectEqual(@as(usize, 0), p.active); + + // Everything a reader can see over the page: a search overlay and a live + // word selection. + try pv.setSearchQuery(p.pdf_gpa, "Pardes"); + var found = try pv.document.search(gpa, pv.page, "Pardes"); + defer found.deinit(gpa); + try std.testing.expect(found.quads.len > 0); + const quad = found.quads[0].quad; + try std.testing.expectEqual( + pdf_pane.SelectionUpdate.changed, + pv.setSelection(p.pdf_gpa, quad.ul, quad.lr, false), + ); + p.update(.{ .key = .{ .cp = '3' } }); + const page_before = pv.page; + const scroll_before = pv.document_scroll_y; + + // Escape drops the overlay and the selection, keeps the reading position, + // and does NOT hand the keyboard to another pane. + p.update(.{ .key = .{ .cp = Key.escape } }); + try std.testing.expectEqual(@as(usize, 0), p.active); + try std.testing.expect(pv.selection == null and pv.selection_text.len == 0); + try std.testing.expectEqual(@as(usize, 0), pv.search_query.len); + try std.testing.expect(pv.search_results == null); + try std.testing.expect(!pv.highlights_dirty and !pv.search_reveal_pending); + try std.testing.expectEqual(page_before, pv.page); + try std.testing.expectEqual(scroll_before, pv.document_scroll_y); + try std.testing.expectEqual(@as(u32, 0), pane.count); + try std.testing.expectEqual(@as(u21, 0), pane.pending); + // A second Escape on a bare document is inert rather than an exit. + p.update(.{ .key = .{ .cp = Key.escape } }); + try std.testing.expectEqual(@as(usize, 0), p.active); + + // Shift-Escape is the way out, and it leaves the document as it found it. + p.update(.{ .key = .{ .cp = Key.escape, .shift = true } }); + try std.testing.expectEqual(@as(usize, 1), p.active); + try std.testing.expectEqual(page_before, pv.page); + try std.testing.expectEqual(scroll_before, pv.document_scroll_y); +} + test "PDF continuous strip renders every intersecting short page" { if (!pdf_enabled or platform == .web) return; diff --git a/src/syntax.zig b/src/syntax.zig index 1494ebd3..09ff12cc 100644 --- a/src/syntax.zig +++ b/src/syntax.zig @@ -46,6 +46,34 @@ fn specCount() comptime_int { return count; } +// Upstream's typst highlights.scm names its markup honestly — +// @markup.heading.*, @markup.bold, @markup.italic, @markup.raw.block — and +// `synFor` now understands that vocabulary, so headings, bold, italics and raw +// blocks paint on their own. What is left here is exactly the two captures we +// refuse to map globally: +// +// - upstream tags call callees @function/@function.method; mapping "function" +// in `synFor` would recolour every function call in every language. +// - upstream tags the code sigil "#" @operator, and mapping "operator" would +// likewise light up every +, -, == in the codebase. +// +// Both are worth colouring *in typst specifically*: the sigil in front of every +// #let/#if/#import/#call is the visual anchor of the code/markup split, and +// upstream leaves it uncoloured even though the keyword behind it is not. +const typst_supplement = + \\ + \\(call item: (ident) @keyword) + \\(call item: (field field: (ident) @keyword)) + \\"#" @keyword + \\ +; + +fn querySrc(comptime g: grammar_manifest.Grammar) []const u8 { + const base = @field(ts_queries, g.name ++ "_highlights"); + if (comptime std.mem.eql(u8, g.name, "typst")) return base ++ typst_supplement; + return base; +} + fn initSpecs() [specCount()]Spec { var out: [specCount()]Spec = undefined; var i = 0; @@ -55,7 +83,7 @@ fn initSpecs() [specCount()]Spec { .name = g.name, .exts = g.exts, .language = @extern(LanguageFn, .{ .name = "tree_sitter_" ++ g.name }), - .query_src = @field(ts_queries, g.name ++ "_highlights"), + .query_src = querySrc(g), }; i += 1; } @@ -235,6 +263,26 @@ fn synFor(name: []const u8) Syn { .{ "title", .keyword }, .{ "uri", .string }, .{ "reference", .number }, + + // Markup grammars (markdown, typst) name prose constructs in their own + // vocabulary rather than the code vocabulary above, so none of the + // needles so far reach them. These are appended, and first-match-wins + // makes that strictly additive; the needles below were audited across + // all 27 shipped queries and occur only in the markdown and typst ones, + // so no other language is recoloured. + // + // The slot assignment is forced by the palette being four wide and by + // `synStyle` attaching the real BOLD attribute to exactly two of them, + // `keyword` and `comment`: headings take `keyword`, so bold spans have + // to land on `comment` to render actually bold. Nothing is left that + // renders italic, so emphasis can only get a colour shift (`number`). + .{ "heading", .keyword }, + .{ "strong", .comment }, + .{ "bold", .comment }, + .{ "emphasis", .number }, + .{ "italic", .number }, + .{ "literal", .string }, + .{ "raw", .string }, }) |m| { if (std.mem.indexOf(u8, name, m[0]) != null) return m[1]; } @@ -264,19 +312,61 @@ pub fn highlightFileRange(gpa: std.mem.Allocator, path: []const u8, content: []c runQuery(styles, selected, tree, 0); - if (std.mem.eql(u8, selected.name, "markdown")) injectCodeBlocks(styles, source, tree.rootNode()); + if (InjectSite.forGrammar(selected.name)) |site| injectCodeBlocks(styles, source, tree.rootNode(), site); + // Disjoint from the fenced-block pass above: `code_fence_content` is never + // an `inline` node, so the two never write the same byte. + if (std.mem.eql(u8, selected.name, "markdown")) injectMarkdownInline(styles, source, tree.rootNode()); return styles; } -fn injectCodeBlocks(styles: []u8, source: []const u8, node: ts.Node) void { - if (std.mem.eql(u8, node.kind(), "fenced_code_block")) { - highlightCodeBlock(styles, source, node); +// Markdown fenced blocks and Typst raw blocks are the same construct — a +// language tag plus a literal payload — under different node shapes, so one +// walker drives both and only the (lang, content) extraction differs. +const InjectSite = enum { + markdown_fence, + typst_raw, + + fn forGrammar(name: []const u8) ?InjectSite { + if (std.mem.eql(u8, name, "markdown")) return .markdown_fence; + if (std.mem.eql(u8, name, "typst")) return .typst_raw; + return null; + } + + fn blockKind(self: InjectSite) []const u8 { + return switch (self) { + .markdown_fence => "fenced_code_block", + .typst_raw => "raw_blck", + }; + } + + /// null when the block carries no language tag (an untagged fence, or a + /// Typst raw block written without one) — nothing to inject, leave it alone. + fn parts(self: InjectSite, block: ts.Node) ?struct { lang: ts.Node, content: ts.Node } { + switch (self) { + .markdown_fence => { + const info = childOfKind(block, "info_string") orelse return null; + return .{ + .lang = childOfKind(info, "language") orelse return null, + .content = childOfKind(block, "code_fence_content") orelse return null, + }; + }, + .typst_raw => return .{ + .lang = block.childByFieldName("lang") orelse return null, + .content = childOfKind(block, "blob") orelse return null, + }, + } + } +}; + +fn injectCodeBlocks(styles: []u8, source: []const u8, node: ts.Node, site: InjectSite) void { + if (std.mem.eql(u8, node.kind(), site.blockKind())) { + highlightCodeBlock(styles, source, node, site); return; } var i: u32 = 0; const count = node.childCount(); while (i < count) : (i += 1) { - if (node.child(i)) |c| injectCodeBlocks(styles, source, c); + if (node.child(i)) |c| injectCodeBlocks(styles, source, c, site); } } @@ -291,14 +381,12 @@ fn childOfKind(node: ts.Node, kind: []const u8) ?ts.Node { return null; } -fn highlightCodeBlock(styles: []u8, source: []const u8, block: ts.Node) void { - const info = childOfKind(block, "info_string") orelse return; - const lang_node = childOfKind(info, "language") orelse return; - const content_node = childOfKind(block, "code_fence_content") orelse return; - const langtext = source[lang_node.startByte()..lang_node.endByte()]; +fn highlightCodeBlock(styles: []u8, source: []const u8, block: ts.Node, site: InjectSite) void { + const p = site.parts(block) orelse return; + const langtext = source[p.lang.startByte()..p.lang.endByte()]; const sub_sel = (forLang(langtext) catch return) orelse return; - const cs: usize = content_node.startByte(); - const ce: usize = content_node.endByte(); + const cs: usize = p.content.startByte(); + const ce: usize = p.content.endByte(); if (ce > source.len or cs > ce) return; const parser = ts.Parser.create(); @@ -306,9 +394,60 @@ fn highlightCodeBlock(styles: []u8, source: []const u8, block: ts.Node) void { parser.setLanguage(sub_sel.lang) catch return; const tree = parser.parseString(source[cs..ce], null) orelse return; defer tree.destroy(); + // The outer grammar already painted these bytes (Typst blankets the whole + // raw block `string`), and runQuery only writes bytes it captures, so the + // outer colour would survive as a wash behind the injected code. The sub + // grammar owns the payload outright: clear it first. + @memset(styles[cs..ce], @intFromEnum(Syn.none)); runQuery(styles, sub_sel, tree, cs); } +// tree-sitter-markdown is a split grammar: the block parser bottoms out at named +// `inline` nodes whose bytes it never looks inside, and emphasis / +// strong_emphasis / code_span exist only in the companion inline parser. So +// every `inline` node is re-parsed with `markdown_inline`, which is what +// upstream's injections.scm, helix and nvim all do (per node, uncombined — the +// stray block_continuation markers inside a multi-line paragraph's range are +// just plain text to the inline parser). +// +// The grammar is resolved and the parser built once per file, not once per node: +// only the parse is inherently per node. +fn injectMarkdownInline(styles: []u8, source: []const u8, root: ts.Node) void { + // Absent in builds below the `full` tier — nothing to inject, leave the + // block grammar's colours alone. + const sub_sel = (forLang("markdown_inline") catch return) orelse return; + const parser = ts.Parser.create(); + defer parser.destroy(); + parser.setLanguage(sub_sel.lang) catch return; + inlineNodes(styles, source, root, sub_sel, parser); +} + +fn inlineNodes(styles: []u8, source: []const u8, node: ts.Node, sel: Selected, parser: *ts.Parser) void { + if (std.mem.eql(u8, node.kind(), "inline")) { + highlightInline(styles, source, node, sel, parser); + return; // `inline` nodes never nest + } + var i: u32 = 0; + const count = node.childCount(); + while (i < count) : (i += 1) { + if (node.child(i)) |c| inlineNodes(styles, source, c, sel, parser); + } +} + +fn highlightInline(styles: []u8, source: []const u8, node: ts.Node, sel: Selected, parser: *ts.Parser) void { + const s: usize = node.startByte(); + const e: usize = node.endByte(); + if (s >= e or e > source.len) return; // an empty atx heading has a zero-length inline + const tree = parser.parseString(source[s..e], null) orelse return; + defer tree.destroy(); + // Deliberately NOT clearing the range first, unlike highlightCodeBlock: the + // block query already painted a heading's inline text `keyword`, and that is + // the colour the heading must keep wherever the inline pass captures + // nothing. Painting over instead of resetting is what makes *italic* inside + // a heading recolour while the rest of the heading stays heading-coloured. + runQuery(styles, sel, tree, s); +} + /// One Syn byte per byte of content[start, end) for unified diffs/patches. /// Pure byte scan; independent of tree-sitter and the `enabled` flag. Caller frees. pub fn highlightDiff(gpa: std.mem.Allocator, content: []const u8, start_byte_raw: usize, end_byte_raw: usize) ![]u8 { @@ -384,21 +523,104 @@ test "default full grammar set highlights Typst source" { try std.testing.expectEqual(Syn.keyword, @as(Syn, @enumFromInt(short_ext[keyword_at]))); } -test "markdown highlights headings and injects fenced code blocks" { +test "Typst markup constructs paint and raw blocks inject their language" { + if (!enabled or !full_grammars) return; + start(std.testing.allocator); + defer stop(); + + const source = + \\// note + \\= Heading Title + \\Some *bold* text with `raw` inline. + \\#import "mod.typ": helper + \\#let answer = 42 + \\#emit(7, "arg") + \\```zig + \\fn widget() u8 { return 99; } + \\``` + \\ + ; + const styles = try highlightFileRange(std.testing.allocator, "paper.typ", source, 0, source.len); + defer std.testing.allocator.free(styles); + + const synAt = struct { + fn f(s: []const u8, src: []const u8, needle: []const u8, offset: usize) Syn { + return @enumFromInt(s[std.mem.indexOf(u8, src, needle).? + offset]); + } + }.f; + + // marker and text of the heading both take the bold accent + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "= Heading", 0)); + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "Heading", 0)); + // *bold* is @markup.bold, which lands on the one slot that still renders + // with the real bold attribute now that headings own `keyword`. + try std.testing.expectEqual(Syn.comment, synAt(styles, source, "bold", 0)); + try std.testing.expectEqual(Syn.string, synAt(styles, source, "raw` inline", 0)); + // the callee and the code sigil in front of it + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "emit", 0)); + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "#emit", 0)); + + // fence and lang tag keep the literal colour of the raw block... + try std.testing.expectEqual(Syn.string, synAt(styles, source, "```zig", 0)); + try std.testing.expectEqual(Syn.string, synAt(styles, source, "```zig", 3)); + // ...while the blob is reset and re-painted by the injected zig grammar. Its + // `fn` and `99` prove the injection ran; `widget` proves the reset, since the + // zig query names it @function (Syn.none here) and without clearing the blob + // first the raw block's `string` would still be washing over it. + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "fn widget", 0)); + try std.testing.expectEqual(Syn.number, synAt(styles, source, "99", 0)); + try std.testing.expectEqual(Syn.none, synAt(styles, source, "widget", 0)); + + try std.testing.expectEqual(Syn.comment, synAt(styles, source, "// note", 0)); + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "import", 0)); + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "let", 0)); + try std.testing.expectEqual(Syn.number, synAt(styles, source, "42", 0)); + try std.testing.expectEqual(Syn.string, synAt(styles, source, "\"arg\"", 0)); +} + +test "markdown highlights markup, injects inline spans and fenced code blocks" { if (!enabled or !full_grammars) return; start(std.testing.allocator); defer stop(); - const source = "# Heading\n\n```zig\nconst answer = 42;\n```\n"; + const source = + "# Heading *slant* Title\n" ++ + "\n" ++ + "Prose with **stout** and *lean* plus `snippet` inline.\n" ++ + "\n" ++ + "```zig\n" ++ + "fn gadget() u8 { return 77; }\n" ++ + "```\n"; const styles = try highlightFileRange(std.testing.allocator, "doc.md", source, 0, source.len); defer std.testing.allocator.free(styles); - const title_at = std.mem.indexOf(u8, source, "Heading").?; - const num_at = std.mem.indexOf(u8, source, "42").?; - // the heading is markdown's own highlight; the number proves the zig - // grammar was injected into the fenced block and colored its contents. - try std.testing.expectEqual(Syn.keyword, @as(Syn, @enumFromInt(styles[title_at]))); - try std.testing.expectEqual(Syn.number, @as(Syn, @enumFromInt(styles[num_at]))); + const synAt = struct { + fn f(s: []const u8, src: []const u8, needle: []const u8, offset: usize) Syn { + return @enumFromInt(s[std.mem.indexOf(u8, src, needle).? + offset]); + } + }.f; + + // The block grammar washes the whole heading `keyword`; the inline pass then + // paints the emphasis over it without resetting, so the heading keeps its + // colour everywhere the emphasis is not. + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "Title", 0)); + try std.testing.expectEqual(Syn.number, synAt(styles, source, "slant", 0)); + + // bold/italic/code-span live only in the inline grammar, so all three prove + // the inline injection ran. + try std.testing.expectEqual(Syn.comment, synAt(styles, source, "stout", 0)); + try std.testing.expectEqual(Syn.number, synAt(styles, source, "lean", 0)); + try std.testing.expectEqual(Syn.string, synAt(styles, source, "snippet", 0)); + + // the fence is inside the block query's @text.literal wash... + try std.testing.expectEqual(Syn.string, synAt(styles, source, "```zig", 0)); + // ...while the payload is cleared and re-painted by the injected zig + // grammar: `fn` and `77` prove the injection ran, and `gadget` proves the + // clear, since the zig query names it @function (Syn.none here) and without + // clearing first the fence's `string` would still be washing over it. + try std.testing.expectEqual(Syn.keyword, synAt(styles, source, "fn gadget", 0)); + try std.testing.expectEqual(Syn.number, synAt(styles, source, "77", 0)); + try std.testing.expectEqual(Syn.none, synAt(styles, source, "gadget", 0)); } test "highlightDiff colors unified diff lines by prefix" { diff --git a/src/tutor.txt b/src/tutor.txt index d6b82713..475cd04a 100644 --- a/src/tutor.txt +++ b/src/tutor.txt @@ -547,7 +547,7 @@ Zabcde gg first line ge LAST line (what vim spells G) Ngg line N Ng| column N gh line start gl line end gs first non-blank - gj line down gk line up + gj VISUAL line down gk VISUAL line up gt the view's top row gc its middle gb its bottom (top and bottom are scrolloff-clamped: three rows in, so they land where the cursor could have scrolled to anyway) @@ -557,6 +557,13 @@ Zabcde `ge` is the key that takes you to the last line — its START, the way `gg` takes you to the first line's. + `gj`/`gk` are the VISUAL pair: with soft wrap on (`SPC t w`, part 4.1, + on to start) one long line is drawn as several rows, and these two walk + those rows — the automatic breaks, not the newlines — keeping the column + they had inside the row. Plain `j`/`k` stay whole FILE lines, so on a + wrapped paragraph `j` jumps past every row of it and `gj` steps one. With + wrap off a line is one row and the two pairs are the same key. + Five more gotos live under the same prefix — gd gD gy gi gr — and they belong to the language backend, in part 4.2. @@ -841,7 +848,10 @@ abc where SPC belongs to the shell) Esc hop to the pane you were in before this one, so held down it alternates between two (body normal - mode; the same builtin as SPC j j) + mode; the same builtin as SPC j j). A PDF pane is the + one exception — there Esc cancels the selection and + the search highlights and stays in the document, and + Shift-Esc is the hop out (part 4.1) Alt-n new terminal below the active one Alt-c move the active pane into a fresh column — any pane, not just a terminal, and a no-op if it is alone in @@ -974,7 +984,10 @@ abc The modal keys work on the PAGE rather than on a cursor: j/k scroll, Ctrl-d/u and Ctrl-f/b page, `Ngg` goes to page N, `ge` is the last page. `/` searches the document's real text through MuPDF and n/N walk - the hits. + the hits. Esc is the document's own CANCEL — it drops the selection and + the search highlights and leaves you where you were reading — so the + hop to the pane before this one is Shift-Esc here, the same chord that + leaves a raw tty. The rest depends on the shell being able to place real PIXELS. Where it can, drag with the left button to select text (SPC y takes it), and @@ -1194,7 +1207,9 @@ abc k off the TOPMOST tagline = the top bar, j back down) zt zz zb zj zk Ctrl-d/u Ctrl-f/b Ctrl-o/i Ctrl-w hjkl Alt-n Alt-c - Esc = the pane you were in before this one (body normal) + Esc = the pane you were in before this one (body normal; + in a PDF it cancels that document's own selection and + search highlights instead, and Shift-Esc hops) n/N = select the next/prev look-able text, across panes (a ring); Enter opens what you landed on SPC = the leader: a key path runs a builtin (SPC ? lists @@ -1203,8 +1218,9 @@ abc SPC w hjkl focus, SPC j j the pane before this one, SPC t * the toggles, SPC l * the language backend) NOT TEXT: .pdf = the real document through MuPDF — j/k scrolls it, - `/` searches it, PdfFit/PdfTint/PdfSections sit in - its own tag + `/` searches it, Esc cancels selection+highlights and + Shift-Esc leaves the pane, PdfFit/PdfTint/PdfSections + sit in its own tag images = pixels where the shell has them, petscii glyphs where it does not (SPC t p/l/a) LANGUAGE: ZLS, compiled in, .zig only, nothing running in the |