Allocate tree-sitter's external scanners from the same Zig allocator as its runtime

syntax.zig points the tree-sitter runtime at a Zig allocator, but the
grammars' external scanners did not follow it. A grammar's
tree_sitter/alloc.h maps ts_malloc to the runtime's current allocator only
when TREE_SITTER_REUSE_ALLOCATOR is defined, and plain malloc otherwise, so
every scanner's state and its arrays came from libc. Five scanners (bash,
markdown, markdown_inline, python, typst) also call malloc, calloc, realloc
and free by name.

build.zig now compiles every grammar with TREE_SITTER_REUSE_ALLOCATOR and
forces in src/tree_sitter_heap.h, which includes stdlib.h and then defines
the four names as calls through ts_current_malloc and friends. A header
forced in from the build rather than a patch to the grammars keeps zig-pkg
pristine. No grammar archive refers to libc's allocator any more; only the
runtime does, for its defaults. A scanner frees only what it allocated
itself, and the syntax tests that create and destroy a parser for every
grammar under a leak-checking allocator pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

3 files changed, 41 insertions, 3 deletions

diff --git a/build.zig b/build.zig
index c70f52b2..f0675a70 100644
--- a/build.zig
+++ b/build.zig
@@ -390,7 +390,17 @@ pub fn build(b: *std.Build) void {
                     .root_module = b.createModule(.{ .target = target, .optimize = c_optimize, .link_libc = !is_web }),
                     .linkage = .static,
                 });
-                const ts_cflags = [_][]const u8{"-fno-sanitize=function"};
+                // Every allocation a grammar's external scanner makes goes to the
+                // runtime's allocator, which syntax.zig points at a Zig one:
+                // TREE_SITTER_REUSE_ALLOCATOR for the scanners that use
+                // ts_malloc, and src/tree_sitter_heap.h for the ones that call
+                // malloc themselves.
+                const ts_cflags = [_][]const u8{
+                    "-fno-sanitize=function",
+                    "-DTREE_SITTER_REUSE_ALLOCATOR",
+                    "-include",
+                    b.path("src/tree_sitter_heap.h").getPath(b),
+                };
                 lib.root_module.addCSourceFile(.{ .file = dep.path(g.src ++ "/parser.c"), .flags = &ts_cflags });
                 if (g.scanner) lib.root_module.addCSourceFile(.{ .file = dep.path(g.src ++ "/scanner.c"), .flags = &ts_cflags });
                 lib.root_module.addIncludePath(dep.path(g.src));
diff --git a/src/syntax.zig b/src/syntax.zig
index 5c8b9c0f..9c1a4f4b 100644
--- a/src/syntax.zig
+++ b/src/syntax.zig
@@ -140,8 +140,9 @@ fn initSpecs() [specCount()]Spec {
 
 var specs = initSpecs();
 
-/// What the tree-sitter runtime allocates from: `start`'s allocator, through
-/// the shared C heap.
+/// What tree-sitter allocates from, the runtime and every grammar's external
+/// scanner (build.zig compiles them to use the runtime's allocator):
+/// `start`'s allocator, through the shared C heap.
 var syntax_allocator: std.mem.Allocator = undefined;
 const syntax_heap = c_heap.Heap(&syntax_allocator);
 var syntax_started = false;
diff --git a/src/tree_sitter_heap.h b/src/tree_sitter_heap.h
new file mode 100644
index 00000000..66083be8
--- /dev/null
+++ b/src/tree_sitter_heap.h
@@ -0,0 +1,27 @@
+/* Forced into every tree-sitter grammar build ahead of its own source
+ * (build.zig passes -include). A few external scanners (bash, markdown,
+ * markdown_inline, python, typst) call malloc, calloc, realloc and free
+ * themselves rather than tree-sitter's ts_malloc and friends, so they would
+ * allocate from libc behind the Zig allocator syntax.zig hands the runtime.
+ * stdlib.h comes first so its declarations are made with the real names;
+ * from here on the four names are the runtime's current allocator functions,
+ * the ones ts_set_allocator sets. The scanners that do use ts_malloc get the
+ * same through tree_sitter/alloc.h and TREE_SITTER_REUSE_ALLOCATOR. A scanner
+ * frees only what it allocated itself, so none of this memory crosses into
+ * the runtime or out of it. */
+#ifndef PARDES_TREE_SITTER_HEAP_H
+#define PARDES_TREE_SITTER_HEAP_H
+
+#include <stdlib.h>
+
+extern void *(*ts_current_malloc)(size_t size);
+extern void *(*ts_current_calloc)(size_t count, size_t size);
+extern void *(*ts_current_realloc)(void *ptr, size_t size);
+extern void (*ts_current_free)(void *ptr);
+
+#define malloc(size) ts_current_malloc(size)
+#define calloc(count, size) ts_current_calloc(count, size)
+#define realloc(ptr, size) ts_current_realloc(ptr, size)
+#define free(ptr) ts_current_free(ptr)
+
+#endif