summaryrefslogtreecommitdiff
path: root/experiments
diff options
context:
space:
mode:
authorGabriel Schneider <[email protected]>2026-08-25 19:29:22 -0300
committerGabriel Schneider <[email protected]>2026-08-25 19:29:22 -0300
commit341a9fe36840ec227284b6fac35e247d5d20ac18 (patch)
treeed12a61b60c18a1793d4b5f70156066afb3ea362 /experiments
parent1935944a0352e9d176f0718063327e686ece1948 (diff)
downloadesp32p4-341a9fe36840ec227284b6fac35e247d5d20ac18.tar.gz
esp32p4-341a9fe36840ec227284b6fac35e247d5d20ac18.zip
Measure all four configurations; the code fix is worth nothing here, the flag is worth 21%
The obvious question after the last commit was what it did on the actual target. The answer is nothing, and the useful part is that the same run says what DOES work. Four builds on the die, 5 document lengths x 7 trials each: configuration fixed per char at 160 vs base ReleaseSmall 16.99 ms 54.3 us 25.56 ms 1.00x ReleaseSmall+lineSpan 17.10 ms 54.0 us 25.63 ms 1.00x ReleaseFast 14.85 ms 34.7 us 20.30 ms 0.79x ReleaseFast+lineSpan 14.88 ms 34.2 us 20.25 ms 0.79x The edit-path change is invisible in BOTH modes, and ReleaseFast+lineSpan is indistinguishable from ReleaseFast alone. The optimisation mode is the whole of the difference, which is what Experiment 3 predicted: the edit path is 220 us of a 17 ms keystroke, so making it cheaper cannot move the total, while the mode makes the RENDERER faster and the renderer is where the time is. `experiments/report.typ` gains that table and the reason the four-way comparison exists rather than four indistinguishable lines on figure 1. The board is now flashed with the default build, which as of the pin next door means ReleaseFast: 809,552 B of the 1,536,000 B partition, 53%.
Diffstat (limited to 'experiments')
-rw-r--r--experiments/length-ReleaseFast-lineSpan.csv36
-rw-r--r--experiments/report.typ57
2 files changed, 86 insertions, 7 deletions
diff --git a/experiments/length-ReleaseFast-lineSpan.csv b/experiments/length-ReleaseFast-lineSpan.csv
new file mode 100644
index 0000000..eec1f79
--- /dev/null
+++ b/experiments/length-ReleaseFast-lineSpan.csv
@@ -0,0 +1,36 @@
+label,experiment,cols,rows,length,op,rep,rtt_us,settle_us,bytes
+ReleaseFast-lineSpan,length,0,0,0,insert,0,15128,26477,141
+ReleaseFast-lineSpan,length,0,0,0,insert,1,14876,20898,80
+ReleaseFast-lineSpan,length,0,0,0,insert,2,14873,21051,81
+ReleaseFast-lineSpan,length,0,0,0,insert,3,14833,20974,81
+ReleaseFast-lineSpan,length,0,0,0,insert,4,14850,21023,81
+ReleaseFast-lineSpan,length,0,0,0,insert,5,14813,20979,81
+ReleaseFast-lineSpan,length,0,0,0,insert,6,14896,21011,81
+ReleaseFast-lineSpan,length,0,0,20,insert,0,15373,21473,81
+ReleaseFast-lineSpan,length,0,0,20,insert,1,15333,21535,81
+ReleaseFast-lineSpan,length,0,0,20,insert,2,15433,21644,81
+ReleaseFast-lineSpan,length,0,0,20,insert,3,15418,21631,81
+ReleaseFast-lineSpan,length,0,0,20,insert,4,15376,21550,81
+ReleaseFast-lineSpan,length,0,0,20,insert,5,15707,28571,158
+ReleaseFast-lineSpan,length,0,0,20,insert,6,15726,21814,80
+ReleaseFast-lineSpan,length,0,0,40,insert,0,16201,22412,81
+ReleaseFast-lineSpan,length,0,0,40,insert,1,16300,22437,81
+ReleaseFast-lineSpan,length,0,0,40,insert,2,16253,22414,81
+ReleaseFast-lineSpan,length,0,0,40,insert,3,16272,22403,81
+ReleaseFast-lineSpan,length,0,0,40,insert,4,16257,22454,81
+ReleaseFast-lineSpan,length,0,0,40,insert,5,16395,22582,81
+ReleaseFast-lineSpan,length,0,0,40,insert,6,16384,22540,81
+ReleaseFast-lineSpan,length,0,0,80,insert,0,17820,24058,81
+ReleaseFast-lineSpan,length,0,0,80,insert,1,17832,24116,81
+ReleaseFast-lineSpan,length,0,0,80,insert,2,17823,24071,81
+ReleaseFast-lineSpan,length,0,0,80,insert,3,17818,23972,81
+ReleaseFast-lineSpan,length,0,0,80,insert,4,17810,24050,81
+ReleaseFast-lineSpan,length,0,0,80,insert,5,17873,24102,81
+ReleaseFast-lineSpan,length,0,0,80,insert,6,17801,24104,81
+ReleaseFast-lineSpan,length,0,0,160,insert,0,20246,26256,81
+ReleaseFast-lineSpan,length,0,0,160,insert,1,20245,26463,81
+ReleaseFast-lineSpan,length,0,0,160,insert,2,20178,26458,81
+ReleaseFast-lineSpan,length,0,0,160,insert,3,20225,26459,81
+ReleaseFast-lineSpan,length,0,0,160,insert,4,20567,33484,158
+ReleaseFast-lineSpan,length,0,0,160,insert,5,20546,26693,80
+ReleaseFast-lineSpan,length,0,0,160,insert,6,20566,26752,81
diff --git a/experiments/report.typ b/experiments/report.typ
index 8ed2b07..8134db2 100644
--- a/experiments/report.typ
+++ b/experiments/report.typ
@@ -42,9 +42,14 @@
}
#let length_rows = rows("length-ReleaseSmall.csv") + rows("length-ReleaseFast.csv")
+ + rows("length-ReleaseSmall-lineSpan.csv") + rows("length-ReleaseFast-lineSpan.csv")
#let ops_rows = rows("ops-ReleaseSmall.csv") + rows("ops-ReleaseFast.csv")
+// Figure 1 compares the two optimisation modes only; the `lineSpan` variants are the SAME source
+// change applied to each, and are tabulated separately in Experiment 3 rather than plotted, because
+// four indistinguishable pairs of lines would be a worse picture than two.
#let builds = ("ReleaseSmall", "ReleaseFast")
+#let all_builds = ("ReleaseSmall", "ReleaseSmall-lineSpan", "ReleaseFast", "ReleaseFast-lineSpan")
#let lengths = (0, 20, 40, 80, 160)
#let med_rtt(rs, pred) = median(rs.filter(pred).map(r => r.rtt))
@@ -419,13 +424,51 @@ independent runs:
1.00×, which is stronger evidence than any single cell.],
)
-And on the board it changed *nothing*: the slope was
-#calc.round(fit(lengths.map(l => l * 1.0), lengths.map(l => med_rtt(length_rows, r => r.label == "ReleaseSmall" and r.length == l))).slope * 1000, digits: 1) µs
-per character before and 54.0 µs after, a ratio of 1.00. That is not a
-contradiction, it is the same fact seen twice: the removed passes are $O(#h(0.1em)$document$)$,
-and this board's document is a few hundred *bytes*, so two scans of it cost nothing
-worth measuring. The identical change is worth 20% on a 19 MB file and 0% on a
-240-character one.
+And on the board it changed *nothing*, in either optimisation mode. All four
+combinations were measured on the die, 5 lengths × 7 trials each:
+
+#figure(
+ table(
+ columns: (auto, auto, auto, auto, auto),
+ align: (left, right, right, right, right),
+ stroke: none,
+ table.hline(),
+ table.header([configuration], [fixed cost], [per character], [at 160 chars], [vs baseline]),
+ table.hline(stroke: 0.5pt),
+ ..all_builds.map(b => {
+ let xs = lengths.map(l => l * 1.0)
+ let ys = lengths.map(l => med_rtt(length_rows, r => r.label == b and r.length == l))
+ let f = fit(xs, ys)
+ let at160 = ys.last()
+ let ref160 = med_rtt(length_rows, r => r.label == "ReleaseSmall" and r.length == 160)
+ (
+ raw(b),
+ [#calc.round(f.intercept, digits: 2) ms],
+ [#calc.round(f.slope * 1000, digits: 1) µs],
+ [#calc.round(at160, digits: 2) ms],
+ [#calc.round(at160 / ref160, digits: 2)×],
+ )
+ }).flatten(),
+ table.hline(),
+ ),
+ caption: [The edit-path change is invisible in both modes; the optimisation mode
+ is the whole of the difference. `ReleaseFast` + `lineSpan` is indistinguishable
+ from `ReleaseFast` alone.],
+)
+
+That is not a contradiction, it is the same fact seen twice: the removed passes are
+$O(#h(0.1em)$document$)$, and this board's document is a few hundred *bytes*, so two
+scans of it cost nothing worth measuring. The identical change is worth 20% on a
+19 MB file and 0% on a 240-character one.
+
+The measurement did change one thing about the board, though, and it is not the
+source: `-Doptimize` defaulted to `Debug`, so a plain `zig build -Dplatform=p4`
+produced an object that *cannot run* — `Debug` wraps every tier in `allocators.zig`
+in a `DebugAllocator` whose metadata is page-granular, and one 4 KiB page per size
+class does not fit in the 384 KiB the board hands over. The p4 target now defaults to
+`ReleaseFast`, which is the mode this experiment chose rather than a preference, and
+an explicit `-Doptimize=` still wins. The 21% is therefore what the default build now
+gives, not something to remember to ask for.
The lesson is the one the instrument exists to enforce. A plausible mechanism, read
off the source and consistent with the shape of the data, was wrong about where the