summaryrefslogtreecommitdiff
path: root/docs/next-tables.md
blob: cc9e567ccfe678a969f6ebff7fdf2daa074ca803 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
# Tables in markdown files, as built

> Could we do special rendering for tables in markdown files?

A GFM pipe table in a `.md` file is drawn as a table, in its own rows: every
column's `|` on one screen column whatever the source spacing, the `|` drawn
as rules, the delimiter row (`|:--|--:|`) drawn as one rule with its colons
kept, the header bold on the tag's ground. Every byte stays in its row as the
text it is: B1 selects it, B2/B3 take its words, the chords cut and paste it,
the caret sits on it, and nothing is drawn over anything. Editing a cell
re-aligns the table on the keystroke.

## The one idea: a pad is a tab's extra cells

The rows are not rewritten. A cell narrower than its column is given the
cells it lacks as **virtual width before one of its bytes** (a *pad*): before
the `|` that ends it (left), before its first byte (right), split (centred).
A pad is to its byte what a tab's extra cells are to the tab: drawn blank,
each of its cells maps back to that byte, and the caret before that byte is
drawn after the pad, against its glyph. So there is no parallel map: the
existing pane-aware column functions in `File.zig` add the pads, and so does
every walk that draws a file row.

| where | what changed |
|---|---|
| `File.rowPads` / `rowTable` | the gate: a row of a table of the text as it is now, that fits the pane |
| `File.displayOffset`, `displayEndOffset`, `byteAtRowDisplay` | `+ Tables.between(pads)` / `Tables.byteAt` |
| `File.fillBody` | writes the pads as blanks into the body string (so `paneText`, `expandedSel`, the sweep text see the drawn row) |
| `File.recolorSyntax` | its screen walk steps over the pads |
| `body_layer` | pointer-selection tint and the view-edge checks through the pane-aware functions |
| `edit.capturePointerSelection` | a sweep's columns to bytes through `byteAtRowDisplay` |
| `Text.verticalMove` / `visualMove` | j/k keep the column the table shows (a body of a file pane only) |

Everything else (mouse.zig, exec.zig's B2 cells, look.zig's cursor word, the
caret, multi-cursor tints, `gw` labels) already went through those three
functions and needed no change. That is the evidence the mapping is in one
place: the round-trip test clicks every cell of a row with pads of all three
kinds and checks the caret lands on the byte and is drawn where it is.

## The look (`Tables.paint`)

Glyphs and colours only, on the cells the row has, after the syntax colours:
`|` as `│`, the delimiter row as `├──┼──┤` (its `-`, blanks and pads as `─`,
its `:` in the rule colour), the header bold on `tag_bg` (the compact context
rows' ground: tables reuse a look the editor has). The rule colour is the
theme's `border`, else `lineno`.

## Finding the tables

tree-sitter-markdown here does expose `pipe_table` (header, delimiter row,
cells, alignments). I did not use the tree to find them: the highlighter
parses only the lines around the view (`refreshHighlights`, a fragment from
`scroll - slack`), which starts mid-table as soon as a long table is
scrolled, and a fence is only known from the top. So `Tables.zig` finds
tables by lines with the grammar's own rules (its scanner's
`PIPE_TABLE_START`: a header's cells counted by pipes, a `:?-+:?` delimiter
row of the same count with at least one pipe; rows until a blank line or a
block that interrupts a paragraph; escaped `\|`), and the grammar is the
**oracle in a test** (`tables are the pipe_tables tree-sitter finds`: five
tables including an indented one, a no-pipe continuation row and an escaped
pipe; the fenced one and the short-delimiter one not).

## Cost (measured)

`.scratch/stress-tables.md`: 2.7 MB, 54 508 lines, 2000 tables of 3-6 columns
and 10-30 rows, fences every 50 tables. Debug build, 160x50, each step
`update` + a full render, the same text as `.md` and as `.txt`
(`zz tables timings`, `TABLES_TIMES=<path>`).

| what | .md | .txt |
|---|---|---|
| a whole read of the tables (once, blocking, for comparison) | 127 ms | - |
| the first frame | 27 ms | 18 ms |
| frames until every table is read (4 ms a frame from the top) | 24 | - |
| `j` and its frame, in a table | 1.0 ms (worst 1.5) | 0.6 ms (0.9) |
| a letter typed in a cell and its frame | 3.2 ms (worst 8.5) | 2.0 ms (6.8) |

The first version re-read the whole text on every edit: 119 ms a keystroke.
Two changes the numbers forced, both Places' patterns:

- **An edit reflows**: the tables it did not touch shift by the lines it
  added (`File.State.splice`), and only its block (blank line to blank line,
  the touched tables taken in) is read again. A fence line in the edit, or
  in what it replaced, reads everything (a fence changes every line after
  it). A random 1500-edit test checks every result against a whole read;
  more than 1000 took the short way.
- **Opening reads from the top a frame's budget at a time** (`Tables.frame_ns`,
  `Pardes.nextWake` while `busy`): rows below the read point are drawn as
  source until it gets there (a quarter second on this file). An edit past
  the read point changes nothing read; one before it starts the read again.

## GUI and tty

Both draw the same grid: alignment, rules, header, the mapping are core.
What only the GUI does: the light box-drawing rules (`─│┌┐└┘├┤┬┴┼`) are drawn
by hand edge to edge (`gui.zig` `ruleArms`/`drawRule`, a `rule` bit in
`GlyphKey`, never a slice of a shaped word), so a table's `│` joins the rows
above and below at any line height and the delimiter row is one unbroken
line. In a terminal they are the terminal font's glyphs, which may leave gaps
between rows (the terminal's business). This applies to every box-drawing
cell in the GUI, terminal panes' TUIs too (as Ghostty and kitty do).

Things only a GUI could do next, not done: pixel-thin rules between cells
rather than a cell-wide glyph (needs a cell decoration the surface does not
have); a half-cell inset so a rule does not take a whole column.

## Rough

- A table wider than a wrapping pane is shown as its source, wrapped (a
  padded row must not wrap: wrapping is byte-fitted). `Wrap off` lays it out
  again with hscroll.
- Inline markdown is not coloured inside cells: `syntax.inject` injects
  `markdown_inline` into `inline` nodes, and `pipe_table_cell` is not one.
- A table in a blockquote or list item is not found (the line detector has
  no container prefixes); tree-sitter finds it.
- Mixed leading pipes (`| a |` over `a |`) misalign by one cell; a leading
  pipe's indentation is padded only when the pad is not at byte 0.
- Columns past 64 are not aligned. Zerox twins do not share the splice, so a
  twin re-reads the whole text after an edit (sliced, so it still does not
  hold a frame).
- The sweep-text fallback (`selectionText` without captured rows, a sweep
  begun on the tag) reads the drawn row, so it would carry pad blanks; the
  same is true of the callstack guide today.

## Open questions, and my position on each

1. **Hide redundant source spaces?** Position: no. Every byte stays visible;
   only width is added, never taken. `|  foo|` shows its two spaces. Hiding
   bytes would make a click land on something you cannot see, which is the
   constraint's spirit even if a space is "text".
2. **The glyph substitution: is drawing `|` as `│` drawing over text?**
   Position: no; it is the byte's glyph, like a font ligature (which the GUI
   already does), with the byte selectable, searchable, typed over. The
   alternative, keeping `|` and only padding, is one constant away
   (`rule_v`); I find the plain `|` reads as source, not as a table.
3. **Delimiter colons**: kept, in the rule colour, at the cell edges. The
   alignment is visible anyway, but you edit them, so they must show.
4. **Rule colour**: `border` is faint on doom_one, especially the 1 px
   horizontal rule. `lineno` or a mix toward `fg` would read better; a theme
   field (`table_rule`) is the honest answer if you want it tunable.
5. **Wide tables**: source wrapped (now) versus clipped laid-out rows with
   hscroll per table. Position: source; a clipped row hides text, and the
   wrap is honest. A column could also be squeezed by wrapping a cell inside
   it, but that is a virtual row, and virtual rows break the row = line
   invariant everything here rests on.

## Pushback

- The NEXT brief says "check whether the grammar exposes pipe_table nodes and
  use them if so". It does, and using them for detection would be worse here
  (a view-local fragment parse starts mid-table; a whole-file parse is
  ~seconds in Debug on the stress file, by converge's 4.8 s for 1.2 MB of
  Zig). Using the grammar as the test oracle keeps its rules without its
  cost. If a whole-file markdown tree ever exists (a worker parse, as
  converge does for declarations), the detector could read it instead.
- Re-aligning live means a column widens under the caret as you type in its
  widest cell, so the rest of the row moves right per keystroke. That is the
  point of the feature, but it is motion in text you are not editing; a
  `TableAlign off` setting may be wanted for heavy table editing.
- Making box-drawing rules procedural is a GUI-wide change (terminal TUIs
  change look too). I think it is right (Ghostty, kitty, WezTerm all do it),
  but it is a decision beyond tables.

## Tests

`zig build unit-test -Dplatform=gui -Dtest-filter="tables"`:
`tables: a pipe table's columns, alignments and pads, by line`; `...every
byte's cell maps back to it, and every pad cell to the byte after it`; `...is
drawn aligned, its bytes in order, the caret and a click on each byte's cell`
(also the look); `...typing in a cell re-aligns the table live, and j/k keep
the column shown`; `...an edit's tables, shifted and found again around it,
are the whole text's` (1500 random edits); `...a big text's tables are read
a frame's budget at a time`; `...wider than a wrapping pane is shown as its
source`; `tables are the pipe_tables tree-sitter finds` (syntax.zig);
`a box-drawing rule reaches its cell's edges` (gui.zig); `zz tables timings`
(skipped without `TABLES_TIMES`). No layout change, so no rects invariant.