diff options
| -rw-r--r-- | docs/next-tables.md | 176 | ||||
| -rw-r--r-- | src/File.zig | 2 | ||||
| -rw-r--r-- | src/Tables.zig | 41 |
3 files changed, 212 insertions, 7 deletions
diff --git a/docs/next-tables.md b/docs/next-tables.md new file mode 100644 index 00000000..cc9e567c --- /dev/null +++ b/docs/next-tables.md @@ -0,0 +1,176 @@ +# Tables in markdown files, as built + +> Could we do special rendering for tables in markdown files? + +A GFM pipe table in a `.md` file is drawn as a table, in its own rows: every +column's `|` on one screen column whatever the source spacing, the `|` drawn +as rules, the delimiter row (`|:--|--:|`) drawn as one rule with its colons +kept, the header bold on the tag's ground. Every byte stays in its row as the +text it is: B1 selects it, B2/B3 take its words, the chords cut and paste it, +the caret sits on it, and nothing is drawn over anything. Editing a cell +re-aligns the table on the keystroke. + +## The one idea: a pad is a tab's extra cells + +The rows are not rewritten. A cell narrower than its column is given the +cells it lacks as **virtual width before one of its bytes** (a *pad*): before +the `|` that ends it (left), before its first byte (right), split (centred). +A pad is to its byte what a tab's extra cells are to the tab: drawn blank, +each of its cells maps back to that byte, and the caret before that byte is +drawn after the pad, against its glyph. So there is no parallel map: the +existing pane-aware column functions in `File.zig` add the pads, and so does +every walk that draws a file row. + +| where | what changed | +|---|---| +| `File.rowPads` / `rowTable` | the gate: a row of a table of the text as it is now, that fits the pane | +| `File.displayOffset`, `displayEndOffset`, `byteAtRowDisplay` | `+ Tables.between(pads)` / `Tables.byteAt` | +| `File.fillBody` | writes the pads as blanks into the body string (so `paneText`, `expandedSel`, the sweep text see the drawn row) | +| `File.recolorSyntax` | its screen walk steps over the pads | +| `body_layer` | pointer-selection tint and the view-edge checks through the pane-aware functions | +| `edit.capturePointerSelection` | a sweep's columns to bytes through `byteAtRowDisplay` | +| `Text.verticalMove` / `visualMove` | j/k keep the column the table shows (a body of a file pane only) | + +Everything else (mouse.zig, exec.zig's B2 cells, look.zig's cursor word, the +caret, multi-cursor tints, `gw` labels) already went through those three +functions and needed no change. That is the evidence the mapping is in one +place: the round-trip test clicks every cell of a row with pads of all three +kinds and checks the caret lands on the byte and is drawn where it is. + +## The look (`Tables.paint`) + +Glyphs and colours only, on the cells the row has, after the syntax colours: +`|` as `│`, the delimiter row as `├──┼──┤` (its `-`, blanks and pads as `─`, +its `:` in the rule colour), the header bold on `tag_bg` (the compact context +rows' ground: tables reuse a look the editor has). The rule colour is the +theme's `border`, else `lineno`. + +## Finding the tables + +tree-sitter-markdown here does expose `pipe_table` (header, delimiter row, +cells, alignments). I did not use the tree to find them: the highlighter +parses only the lines around the view (`refreshHighlights`, a fragment from +`scroll - slack`), which starts mid-table as soon as a long table is +scrolled, and a fence is only known from the top. So `Tables.zig` finds +tables by lines with the grammar's own rules (its scanner's +`PIPE_TABLE_START`: a header's cells counted by pipes, a `:?-+:?` delimiter +row of the same count with at least one pipe; rows until a blank line or a +block that interrupts a paragraph; escaped `\|`), and the grammar is the +**oracle in a test** (`tables are the pipe_tables tree-sitter finds`: five +tables including an indented one, a no-pipe continuation row and an escaped +pipe; the fenced one and the short-delimiter one not). + +## Cost (measured) + +`.scratch/stress-tables.md`: 2.7 MB, 54 508 lines, 2000 tables of 3-6 columns +and 10-30 rows, fences every 50 tables. Debug build, 160x50, each step +`update` + a full render, the same text as `.md` and as `.txt` +(`zz tables timings`, `TABLES_TIMES=<path>`). + +| what | .md | .txt | +|---|---|---| +| a whole read of the tables (once, blocking, for comparison) | 127 ms | - | +| the first frame | 27 ms | 18 ms | +| frames until every table is read (4 ms a frame from the top) | 24 | - | +| `j` and its frame, in a table | 1.0 ms (worst 1.5) | 0.6 ms (0.9) | +| a letter typed in a cell and its frame | 3.2 ms (worst 8.5) | 2.0 ms (6.8) | + +The first version re-read the whole text on every edit: 119 ms a keystroke. +Two changes the numbers forced, both Places' patterns: + +- **An edit reflows**: the tables it did not touch shift by the lines it + added (`File.State.splice`), and only its block (blank line to blank line, + the touched tables taken in) is read again. A fence line in the edit, or + in what it replaced, reads everything (a fence changes every line after + it). A random 1500-edit test checks every result against a whole read; + more than 1000 took the short way. +- **Opening reads from the top a frame's budget at a time** (`Tables.frame_ns`, + `Pardes.nextWake` while `busy`): rows below the read point are drawn as + source until it gets there (a quarter second on this file). An edit past + the read point changes nothing read; one before it starts the read again. + +## GUI and tty + +Both draw the same grid: alignment, rules, header, the mapping are core. +What only the GUI does: the light box-drawing rules (`─│┌┐└┘├┤┬┴┼`) are drawn +by hand edge to edge (`gui.zig` `ruleArms`/`drawRule`, a `rule` bit in +`GlyphKey`, never a slice of a shaped word), so a table's `│` joins the rows +above and below at any line height and the delimiter row is one unbroken +line. In a terminal they are the terminal font's glyphs, which may leave gaps +between rows (the terminal's business). This applies to every box-drawing +cell in the GUI, terminal panes' TUIs too (as Ghostty and kitty do). + +Things only a GUI could do next, not done: pixel-thin rules between cells +rather than a cell-wide glyph (needs a cell decoration the surface does not +have); a half-cell inset so a rule does not take a whole column. + +## Rough + +- A table wider than a wrapping pane is shown as its source, wrapped (a + padded row must not wrap: wrapping is byte-fitted). `Wrap off` lays it out + again with hscroll. +- Inline markdown is not coloured inside cells: `syntax.inject` injects + `markdown_inline` into `inline` nodes, and `pipe_table_cell` is not one. +- A table in a blockquote or list item is not found (the line detector has + no container prefixes); tree-sitter finds it. +- Mixed leading pipes (`| a |` over `a |`) misalign by one cell; a leading + pipe's indentation is padded only when the pad is not at byte 0. +- Columns past 64 are not aligned. Zerox twins do not share the splice, so a + twin re-reads the whole text after an edit (sliced, so it still does not + hold a frame). +- The sweep-text fallback (`selectionText` without captured rows, a sweep + begun on the tag) reads the drawn row, so it would carry pad blanks; the + same is true of the callstack guide today. + +## Open questions, and my position on each + +1. **Hide redundant source spaces?** Position: no. Every byte stays visible; + only width is added, never taken. `| foo|` shows its two spaces. Hiding + bytes would make a click land on something you cannot see, which is the + constraint's spirit even if a space is "text". +2. **The glyph substitution: is drawing `|` as `│` drawing over text?** + Position: no; it is the byte's glyph, like a font ligature (which the GUI + already does), with the byte selectable, searchable, typed over. The + alternative, keeping `|` and only padding, is one constant away + (`rule_v`); I find the plain `|` reads as source, not as a table. +3. **Delimiter colons**: kept, in the rule colour, at the cell edges. The + alignment is visible anyway, but you edit them, so they must show. +4. **Rule colour**: `border` is faint on doom_one, especially the 1 px + horizontal rule. `lineno` or a mix toward `fg` would read better; a theme + field (`table_rule`) is the honest answer if you want it tunable. +5. **Wide tables**: source wrapped (now) versus clipped laid-out rows with + hscroll per table. Position: source; a clipped row hides text, and the + wrap is honest. A column could also be squeezed by wrapping a cell inside + it, but that is a virtual row, and virtual rows break the row = line + invariant everything here rests on. + +## Pushback + +- The NEXT brief says "check whether the grammar exposes pipe_table nodes and + use them if so". It does, and using them for detection would be worse here + (a view-local fragment parse starts mid-table; a whole-file parse is + ~seconds in Debug on the stress file, by converge's 4.8 s for 1.2 MB of + Zig). Using the grammar as the test oracle keeps its rules without its + cost. If a whole-file markdown tree ever exists (a worker parse, as + converge does for declarations), the detector could read it instead. +- Re-aligning live means a column widens under the caret as you type in its + widest cell, so the rest of the row moves right per keystroke. That is the + point of the feature, but it is motion in text you are not editing; a + `TableAlign off` setting may be wanted for heavy table editing. +- Making box-drawing rules procedural is a GUI-wide change (terminal TUIs + change look too). I think it is right (Ghostty, kitty, WezTerm all do it), + but it is a decision beyond tables. + +## Tests + +`zig build unit-test -Dplatform=gui -Dtest-filter="tables"`: +`tables: a pipe table's columns, alignments and pads, by line`; `...every +byte's cell maps back to it, and every pad cell to the byte after it`; `...is +drawn aligned, its bytes in order, the caret and a click on each byte's cell` +(also the look); `...typing in a cell re-aligns the table live, and j/k keep +the column shown`; `...an edit's tables, shifted and found again around it, +are the whole text's` (1500 random edits); `...a big text's tables are read +a frame's budget at a time`; `...wider than a wrapping pane is shown as its +source`; `tables are the pipe_tables tree-sitter finds` (syntax.zig); +`a box-drawing rule reaches its cell's edges` (gui.zig); `zz tables timings` +(skipped without `TABLES_TIMES`). No layout change, so no rects invariant. diff --git a/src/File.zig b/src/File.zig index 5db04443..cf44c69a 100644 --- a/src/File.zig +++ b/src/File.zig @@ -588,7 +588,7 @@ test "indexed source rows match uncached scans across content changes" { /// through here, so the drawn row, the caret and a click cannot disagree. pub fn rowPads(pane: *const Pane, row: i32, out: *[Tables.max_pads]Tables.Pad) []const Tables.Pad { const table = rowTable(pane, row) orelse return &.{}; - return pane.file.?.tables.padsOf(table, sourceLine(pane, row), out); + return pane.file.?.tables.padsOf(table, @intCast(row), sourceLine(pane, row), out); } /// The table `row` is laid out in, in this pane (`rowPads`). diff --git a/src/Tables.zig b/src/Tables.zig index 1dd296f7..2e65a716 100644 --- a/src/Tables.zig +++ b/src/Tables.zig @@ -594,7 +594,7 @@ fn add(t: *Tables, gpa: std.mem.Allocator, content: []const u8, header: Line, de var buf: [max_pads]Pad = undefined; while (row) |r| : (row = if (r.n == last.n) null else nextLine(content, r)) { var width = widthOf(r.text); - for (t.padsOf(&table, r.text, &buf)) |pad| width += pad.cells; + for (t.padsOf(&table, r.n, r.text, &buf)) |pad| width += pad.cells; table.width = @max(table.width, @as(u32, @intCast(@min(width, std.math.maxInt(u32))))); } try t.list.append(gpa, table); @@ -604,7 +604,7 @@ fn add(t: *Tables, gpa: std.mem.Allocator, content: []const u8, header: Line, de /// The pads of one of `table`'s rows, by byte, in order. No pad is ever at /// byte 0: a row is drawn from its first byte, so there is no cell before /// it to give. -pub fn padsOf(t: *const Tables, table: *const Table, text: []const u8, out: *[max_pads]Pad) []Pad { +pub fn padsOf(t: *const Tables, table: *const Table, line: usize, text: []const u8, out: *[max_pads]Pad) []Pad { const pipes = Pipes.of(text); var n: usize = 0; if (pipes.leading and pipes.at[0] > 0) { @@ -625,6 +625,17 @@ pub fn padsOf(t: *const Tables, table: *const Table, text: []const u8, out: *[ma // The first cell with no leading pipe starts at byte 0: it pads at // its end, whatever its alignment. const how: Align = if (from == 0) .left else t.aligns.items[table.col0 + k]; + // The delimiter row's cells keep their colons at their edges: the + // rule is drawn long between them (a pad before a closing `:`). + if (line == table.first + 1) { + var last = stop; + while (last > from and (text[last - 1] == ' ' or text[last - 1] == '\t')) last -= 1; + const at = if (last > from and text[last - 1] == ':') last - 1 else stop; + if (at == 0) continue; + out[n] = .{ .at = @intCast(at), .cells = missing }; + n += 1; + continue; + } const before: u16 = switch (how) { .left => 0, .right => missing, @@ -717,7 +728,7 @@ pub fn paint(p: *Pardes, s: *Surface, pane: *Pane, r: pardes.Rect, tx: u16, tw: const table = File.rowTable(pane, w.line) orelse continue; const text = File.sourceLine(pane, w.line); var buf: [max_pads]Pad = undefined; - const pads = f.tables.padsOf(table, text, &buf); + const pads = f.tables.padsOf(table, @intCast(w.line), text, &buf); const pipes = Pipes.of(text); const delimiter = w.line == table.first + 1; const header = w.line == table.first; @@ -818,11 +829,11 @@ test "tables: a pipe table's columns, alignments and pads, by line" { const fig = "| fig || |"; try testing.expectEqualSlices(Pad, &.{ .{ .at = 6, .cells = 2 }, .{ .at = 7, .cells = 5 }, .{ .at = 8, .cells = 8 }, .{ .at = 9, .cells = 8 }, - }, t.padsOf(&table, fig, &buf)); + }, t.padsOf(&table, 6, fig, &buf)); // `| apple | 3 | red |`: right-aligned 3 pads before its first byte. try testing.expectEqualSlices(Pad, &.{ .{ .at = 9, .cells = 2 }, .{ .at = 13, .cells = 6 }, .{ .at = 18, .cells = 6 }, - }, t.padsOf(&table, "| apple | 3 | red |", &buf)); + }, t.padsOf(&table, 4, "| apple | 3 | red |", &buf)); // Every row laid out is one width: the table's. try testing.expectEqual(@as(u32, 1 + 7 + 1 + 5 + 1 + 17 + 1), table.width); // Not tables: a fence's rows, counts that differ, a delimiter alone. @@ -847,7 +858,7 @@ test "tables: every byte's cell maps back to it, and every pad cell to the byte while (lines.next()) |text| : (n += 1) { if (n < table.first or n >= table.end) continue; var buf: [max_pads]Pad = undefined; - const pads = t.padsOf(table, text, &buf); + const pads = t.padsOf(table, n, text, &buf); var col: usize = 0; for (text, 0..) |_, b| { // The pad before b: each of its cells is b's. @@ -1054,6 +1065,24 @@ test "tables: a big text's tables are read a frame's budget at a time from the t } } +test "tables: a table wider than a wrapping pane is shown as its source, laid out again with Wrap off" { + const p = try Pardes.init(testing.allocator, .{ .tty_only = true, .cols = 34, .rows = 24 }); + defer p.deinit(); + var frame = std.heap.ArenaAllocator.init(testing.allocator); + defer frame.deinit(); + const pane = try testPane(p, sample, frame.allocator()); + try testing.expect(pane.body_wrapped); + try testing.expect(File.wrapWidth(pane, true) < 34); + try testing.expect(File.rowTable(pane, 2) == null); + // Source: a pipe is a pipe, at its source column. + try testing.expectEqual(@as(i32, 7), File.displayOffset(pane, 5, 0, 7)); + try testing.expect(p.executeBuiltinLine(0, "Wrap off")); + _ = frame.reset(.retain_capacity); + _ = try p.render(frame.allocator()); + try testing.expect(!pane.body_wrapped); + try testing.expect(File.rowTable(pane, 2) != null); +} + test "zz tables timings on a big markdown file (TABLES_TIMES=<path>)" { const path_z = std.c.getenv("TABLES_TIMES") orelse return error.SkipZigTest; const path = std.mem.span(path_z); |
