diff options
Diffstat (limited to 'experiments/report.typ')
| -rw-r--r-- | experiments/report.typ | 57 |
1 files changed, 50 insertions, 7 deletions
diff --git a/experiments/report.typ b/experiments/report.typ index 8ed2b07..8134db2 100644 --- a/experiments/report.typ +++ b/experiments/report.typ @@ -42,9 +42,14 @@ } #let length_rows = rows("length-ReleaseSmall.csv") + rows("length-ReleaseFast.csv") + + rows("length-ReleaseSmall-lineSpan.csv") + rows("length-ReleaseFast-lineSpan.csv") #let ops_rows = rows("ops-ReleaseSmall.csv") + rows("ops-ReleaseFast.csv") +// Figure 1 compares the two optimisation modes only; the `lineSpan` variants are the SAME source +// change applied to each, and are tabulated separately in Experiment 3 rather than plotted, because +// four indistinguishable pairs of lines would be a worse picture than two. #let builds = ("ReleaseSmall", "ReleaseFast") +#let all_builds = ("ReleaseSmall", "ReleaseSmall-lineSpan", "ReleaseFast", "ReleaseFast-lineSpan") #let lengths = (0, 20, 40, 80, 160) #let med_rtt(rs, pred) = median(rs.filter(pred).map(r => r.rtt)) @@ -419,13 +424,51 @@ independent runs: 1.00×, which is stronger evidence than any single cell.], ) -And on the board it changed *nothing*: the slope was -#calc.round(fit(lengths.map(l => l * 1.0), lengths.map(l => med_rtt(length_rows, r => r.label == "ReleaseSmall" and r.length == l))).slope * 1000, digits: 1) µs -per character before and 54.0 µs after, a ratio of 1.00. That is not a -contradiction, it is the same fact seen twice: the removed passes are $O(#h(0.1em)$document$)$, -and this board's document is a few hundred *bytes*, so two scans of it cost nothing -worth measuring. The identical change is worth 20% on a 19 MB file and 0% on a -240-character one. +And on the board it changed *nothing*, in either optimisation mode. All four +combinations were measured on the die, 5 lengths × 7 trials each: + +#figure( + table( + columns: (auto, auto, auto, auto, auto), + align: (left, right, right, right, right), + stroke: none, + table.hline(), + table.header([configuration], [fixed cost], [per character], [at 160 chars], [vs baseline]), + table.hline(stroke: 0.5pt), + ..all_builds.map(b => { + let xs = lengths.map(l => l * 1.0) + let ys = lengths.map(l => med_rtt(length_rows, r => r.label == b and r.length == l)) + let f = fit(xs, ys) + let at160 = ys.last() + let ref160 = med_rtt(length_rows, r => r.label == "ReleaseSmall" and r.length == 160) + ( + raw(b), + [#calc.round(f.intercept, digits: 2) ms], + [#calc.round(f.slope * 1000, digits: 1) µs], + [#calc.round(at160, digits: 2) ms], + [#calc.round(at160 / ref160, digits: 2)×], + ) + }).flatten(), + table.hline(), + ), + caption: [The edit-path change is invisible in both modes; the optimisation mode + is the whole of the difference. `ReleaseFast` + `lineSpan` is indistinguishable + from `ReleaseFast` alone.], +) + +That is not a contradiction, it is the same fact seen twice: the removed passes are +$O(#h(0.1em)$document$)$, and this board's document is a few hundred *bytes*, so two +scans of it cost nothing worth measuring. The identical change is worth 20% on a +19 MB file and 0% on a 240-character one. + +The measurement did change one thing about the board, though, and it is not the +source: `-Doptimize` defaulted to `Debug`, so a plain `zig build -Dplatform=p4` +produced an object that *cannot run* — `Debug` wraps every tier in `allocators.zig` +in a `DebugAllocator` whose metadata is page-granular, and one 4 KiB page per size +class does not fit in the 384 KiB the board hands over. The p4 target now defaults to +`ReleaseFast`, which is the mode this experiment chose rather than a preference, and +an explicit `-Doptimize=` still wins. The 21% is therefore what the default build now +gives, not something to remember to ask for. The lesson is the one the instrument exists to enforce. A plausible mechanism, read off the source and consistent with the shape of the data, was wrong about where the |