diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/ghostty-macos-notes.md | 198 | ||||
| -rw-r--r-- | docs/macos.md | 252 |
2 files changed, 450 insertions, 0 deletions
diff --git a/docs/ghostty-macos-notes.md b/docs/ghostty-macos-notes.md new file mode 100644 index 00000000..556831af --- /dev/null +++ b/docs/ghostty-macos-notes.md @@ -0,0 +1,198 @@ +# Notes: how ghostty integrates Zig with Swift + +Research notes taken while designing `docs/macos.md`, kept so the next person +does not have to read ghostty again. Line references are against the checkout at +`~/05-genizah/ghostty` (commit `ba38b493`, "Update to Zig 0.16.0"); they will +drift, the shapes will not. + +Ghostty is the only large Zig program with a shipping native macOS app, so it is +the reference whether or not you copy it. Read this next to `docs/macos.md`, +which records where pardes deliberately went the other way. + +## The seam is small, and it is a hand-written C header + +`include/ghostty.h` is 1207 lines for an application of ghostty's size: about 90 +exported functions, four opaque handles (`typedef void* ghostty_app_t` and +friends, `:56-60`), and one callback struct. Everything is a `void*` on the C +side; the Zig `export fn` signatures carry the real types, so the compiler checks +one end and the other is untyped by construction. + +The header is **not generated**. It is maintained by hand, with `IMPORTANT: Any +changes here update include/ghostty.h` comments scattered through the Zig +(`src/input/key.zig:83`, `src/apprt/action.zig:75`). What keeps it honest is a +test: `build.zig:359-365` runs `addTranslateC` over the header and imports the +result into the unit-test build, and `src/lib/enum.zig:91-141` walks every Zig +enum asserting each tag equals the matching `GHOSTTY_*` constant, failing in both +directions — a Zig tag with no C constant, and a C constant with no Zig tag. + +This is the single most transferable thing in the repository, it costs about ten +lines of build plumbing, and it runs on Linux. pardes now does the same thing in +`src/macos.zig`, extended to compare exported function arity and scalar widths as +well, because a hand-written header drifts in signatures too. + +## Zig→host is one callback, not N + +`ghostty_runtime_config_s` (`include/ghostty.h:999-1027`) holds `userdata` and +six function pointers: `wakeup`, `action`, `read_clipboard`, +`confirm_read_clipboard`, `write_clipboard`, `close_surface`. Only `wakeup` and +the clipboard ones are what they look like. Everything else the core wants from +the host — new window, set title, toggle fullscreen, desktop notification, +sixty-odd cases — travels through the single `action` callback as a tagged union. + +The union is generated at comptime from the action enum +(`src/apprt/action.zig:419-484`) with `@Union(.@"extern", ...)`, and asserted to +a fixed size so adding a case cannot silently move the ABI: + +```zig +comptime { + assert(@sizeOf(CValue) == switch (@sizeOf(usize)) { 4 => 16, 8 => 24, else => unreachable }); +} +``` + +The payoff is that adding a feature adds an enum case, not a struct field: hosts +that do not implement it return `false` and the ABI never changed. Worth copying +the day a fixed callback list starts growing. pardes has two callbacks today and +does not need it yet. + +Two userdata scopes, worth noting: the runtime struct's `userdata` is +app-scoped, and each surface carries its own. Swift round-trips both with +`Unmanaged.passUnretained(self).toOpaque()` — libghostty never retains, so +lifetime stays entirely on the Swift side. + +## Rendering: Zig owns the layer, Swift never draws + +This is the surprising part and the main thing pardes did *not* copy. + +Swift creates an `NSView`, gives it a non-zero frame, and passes the bare +pointer through a tagged platform union (`include/ghostty.h:448-465`): + +```c +typedef struct { void* nsview; } ghostty_platform_macos_s; +``` + +Zig then makes that view layer-hosting and installs a layer *it* owns +(`src/renderer/Metal.zig:108-146`): + +```zig +var layer = try IOSurfaceLayer.init(); +info.view.setProperty("layer", layer.layer.value); +info.view.setProperty("wantsLayer", true); +``` + +Note the order — assigning `.layer` before `wantsLayer = true` is what makes the +view layer-*hosted*, meaning AppKit will never draw into it. And it is not a +`CAMetalLayer` or an `MTKView`: `src/renderer/metal/IOSurfaceLayer.zig` builds a +`CALayer` subclass at runtime with `objc.allocateClassPair`, renders into an +`IOSurface`, and sets it as `layer.contents`. + +The consequence is that Swift is not in the render loop at all. There is no +`ghostty_surface_draw` call anywhere in the Swift sources; the renderer thread +drives its own vsync. Swift pushes only metadata: `set_size` (from a SwiftUI +`GeometryReader`, not `NSView.resize` — macOS 12 did not call it), +`set_content_scale` from `viewDidChangeBackingProperties`, `set_display_id` so +Zig can pick the right refresh rate, and `set_occlusion`. + +The debug inspector is the counter-example and shows both models are supported: +there Swift owns an `MTKView` and passes a command buffer *into* Zig +(`ghostty_inspector_metal_render`, `macos/Sources/Ghostty/Surface View/InspectorView.swift:417`). + +pardes inverts this deliberately — see `docs/macos.md`. The short version: a +terminal grid rendered by a hand-written Metal atlas is what `src/gui/gui.zig` +already spends thousands of lines on, and a cell grid is exactly what CoreText +draws natively. + +## Threading + +Per surface, ghostty spawns two threads (`src/Surface.zig:727-741`): a renderer +thread owning Metal and its own frame clock, and an IO thread doing pty reads and +VT parsing. Both run libxev loops with a `BlockingQueue` mailbox. + +Everything in `ghostty.h` is main-thread-only. The one documented exception is +`wakeup`, and its whole Swift body is a hop: + +```swift +DispatchQueue.main.async { state.appTick() } +``` + +The detail worth stealing: the wakeup is baked into the mailbox push +(`src/App.zig:559-576`), so it is impossible to enqueue without waking. Worker +threads never call host callbacks directly — `action_cb` fires from +`drainMailbox` on the main thread, so although `wakeup` is any-thread, in +practice every other callback is not. + +## Build plumbing, and how much of it is distribution + +Two build systems glued together, and ghostty's own CI never uses the integrated +path — `zig build -Demit-macos-app=false` then `xcodebuild`, because Nix breaks +xcodebuild (`.github/workflows/release-tip.yml:436-441`, and `macos/AGENTS.md` +says the same to humans). + +The Zig half: + +- `src/build/GhosttyLib.zig:19-58` — a static lib rooted at `src/main_c.zig` + with `bundle_compiler_rt = true` and `bundle_ubsan_rt = true`, then every + dependency archive merged into one. The comment is blunt: *"These must be + bundled since we're compiling into a static lib. Otherwise, you get undefined + symbol errors."* pardes's `build.zig` copies exactly this. +- `LibtoolStep.zig` / `CombineArchivesStep.zig` — `libtool -static` on Darwin, + an MRI script through `zig ar -M` elsewhere. Note the workaround at + `LibtoolStep.zig:54-78`: newer Xcode `libtool` drops 64-bit archive members + unless each input is copied and `ranlib`'d first. +- `LipoStep.zig` — 42 lines, `lipo -create`, fed by building the same static lib + twice against retargeted deps (`GhosttyLib.zig:169-199`). +- `XCFrameworkStep.zig` — 77 lines, `xcodebuild -create-xcframework`. It writes + to a fixed repo-relative path outside the Zig cache, which is why it needs + `has_side_effects` and an explicit `rm -rf`; the price of a stable path for + Xcode to reference. +- `GhosttyXCFramework.zig:56-60` stages a headers directory containing *only* + `ghostty.h` plus `module.modulemap`, because pointing at `include/` directly + drags in the separate libghostty-vt headers and Clang complains that the + umbrella header does not include them. + +The Xcode side needs almost nothing: one `PBXFileReference` for the xcframework +with `sourceTree = "<group>"`, added to the frameworks build phase, plus +`OTHER_LDFLAGS = -lstdc++` for the vendored C++. No xcconfig, no search paths, no +bridging header for libghostty — Swift says `import GhosttyKit` and the module +map inside the framework does the rest. (The bridging header that does exist is +for two unrelated Objective-C helpers.) + +Resources are Xcode *folder references* pointing at `../zig-out/share/*`, so Zig +writes the tree and Xcode copies it verbatim. Fonts are not resources at all — +they are `@embedFile`'d (`src/font/embedded.zig`). Metal shaders are compiled to +a `.metallib` by `xcrun` at Zig build time and embedded into the library +(`MetallibStep.zig`, `SharedDeps.zig:504-511`), which is much simpler than +routing shaders through Xcode build rules. + +**What genuinely needs a Mac:** `libtool`, `lipo`, `xcodebuild`, `xcrun metal`, +`dsymutil`, `codesign`, and Swift itself. Everything else — including the +header-drift test — runs on Linux. `flake.nix:63-66` excludes Darwin from +buildable platforms entirely. + +pardes skips lipo, the xcframework, the Xcode project and codesigning. All four +exist to ship a signed universal bundle to someone else's machine; a dev build +needs a `swiftc` invocation and a directory with a plist in it. They are the +named upgrade path in `docs/macos.md`, in that order. + +## Small things worth remembering + +- Enum value 0 is reserved as "invalid" throughout the C API + (`PlatformTag` at `src/apprt/embedded.zig:390-395`), so a zero-initialized + struct is detectable, and tags are validated with `intToEnum` rather than + trusted. +- Zig errors never cross the boundary. Every export is a thin wrapper: + `fn foo_() !T` plus `catch |err| { log; return null; }` + (`src/apprt/embedded.zig:1398-1406`). There is no error channel at all. +- The exports live inside `pub const CAPI = struct {...}` and are only emitted + because `src/main_c.zig:34-47` references the struct in a `comptime` block. +- libghostty defends against host garbage rather than trusting it — + `updateContentScale` rejects NaN and clamps below 1 with the comment *"We are + an embedded API so the caller can send us all sorts of garbage"* + (`src/apprt/embedded.zig:777-793`), and `updateSize` drops no-op resizes + because *"Runtimes sometimes generate superfluous resize events even if the + size did not actually change (SwiftUI)"*. +- Swift keeps a dependency-free `GhosttyPackageMeta.swift` so auxiliary bundles + (the dock tile plugin) can share types without linking the Zig library. +- The Swift wrapper layer compiles unchanged for iOS, which is the real proof + that the C seam is the right width. It manages that by turning Zig→host + actions into `NSNotification`s keyed on the surface, decoupling the callback + from any particular view controller. diff --git a/docs/macos.md b/docs/macos.md new file mode 100644 index 00000000..35fba5f3 --- /dev/null +++ b/docs/macos.md @@ -0,0 +1,252 @@ +# Native macOS backend + +Zig owns the core, the ptys, every effect and the worker threads. Swift owns +`NSApplication`, the window, input translation, and drawing. They meet at a +hand-written C ABI, `src/macos/pardes.h`, built as a static library that the app +links; the Swift half is ordinary AppKit that pumps events in and reads one +packed cell buffer out. There is exactly one core per process, so the ABI has no +handles — the state is a file-scoped singleton and every function names it +implicitly, exactly like the browser backend. + +The research this design came out of is written down separately in +`docs/ghostty-macos-notes.md` — how ghostty wires Zig to Swift, what its build +plumbing actually requires, and which parts of it are distribution machinery +rather than integration. Read that for the alternatives; this file is what was +built and why. + +## Why not ghostty's split + +Ghostty was read carefully before this was written, and this backend +deliberately inverts its division of labour. Ghostty hands Zig a bare `NSView*` +through a tagged platform union (`ghostty_platform_macos_s.nsview`), and Zig +then creates a layer, assigns it as the view's layer, sets `wantsLayer`, and +installs a display callback — `src/renderer/Metal.zig` in that tree. Swift never +renders a glyph; it supplies a rectangle and gets out of the way. + +Pardes goes the other way because its frame is *already* a cell grid. +`pardes.Surface` is `cols × rows` of `Cell`, and CoreText is the native way to +draw one: attributed runs, the system font stack, and Apple's own subpixel and +color-emoji handling, for free. The alternative is a hand-rolled glyph atlas, +and the SDL backend is the measurement of what that costs — roughly three +thousand of `src/gui/gui.zig`'s 4,300 lines are a 2048² stb_truetype R8 atlas, +GPU pipelines, transfer buffers and shaders. Writing a second one, blind, buys +nothing the grid needs. + +Blind is the operative word: this backend was scaffolded on a Linux machine. A +Metal renderer written there would have been untestable code — compiled at best, +never once run — whereas CoreText drawing is a small amount of Swift that only +exists on the machine that can run it. + +The ceiling is accepted and named. Per-cell CoreText drawing is slower than an +atlas, and if it fails to hold a full-screen redraw at the target rate the +escalation is ghostty's model verbatim: Zig owns a `CAMetalLayer` installed into +the view and drives its own frame clock, and the ABI grows a `platform` pointer +field carrying the `NSView*` — one field, because the rest of the seam does not +change. Nothing here is designed to make that harder. + +## Why the ABI mirrors src/web.zig + +The browser and a Cocoa app are the same host, and the browser proved the shape +first. In both, someone else owns the clock and the event loop; events arrive +through flat functions that take scalars and borrowed byte ranges; one call +renders, and the result is one contiguous array of packed cells the host walks +linearly. `pardes_cell_s` is byte-for-byte `WebCell`: seven UTF-8 bytes of +grapheme plus a guaranteed zero, tagged `fg`/`bg` words (`0x01000000` default, +`0x02000000 | index` for the palette, otherwise `0x00RRGGBB`), an attribute +bitfield with the underline style in the high bits, and a `flags` bit meaning +"the core never painted this cell" so a host can draw background only and skip +the glyph. Two hosts spelling the same encoding is not duplication worth +removing; it is the encoding being right. + +The one real difference is IO. The browser has no ptys, so `src/web.zig` +forwards every effect out to JavaScript as a numbered code plus a byte payload, +and JavaScript performs it. macOS has `forkpty`, `read` and `write` right there, +so effects never cross the boundary at all: `pardes_tick` drains the core's +effect queue and performs each one in Zig, the way `drainEffects` in +`src/tty/tty.zig` does. That is why the runtime struct is two callbacks and not +twelve. + +## The seam + +**Frame.** `pardes_frame` renders and returns the cell count; +`pardes_frame_cells` hands back a pointer valid until the next `pardes_frame`, +with `pardes_frame_cols`/`_rows` giving its shape. The cursor rides alongside as +`pardes_cursor_x`/`_y`, each `-1` when it is hidden, plus `pardes_cursor_bar` +asking for a thin insert caret instead of a block. `Surface.images` has no +representation here yet; see the checklist. + +**Input.** `pardes_key` takes a codepoint and the host's already-composed text, +because composition is AppKit's job and the core only ever wants finished +characters. Keys that carry no text are the four ASCII controls (enter, escape, +tab, backspace) and a private-use block starting at `0xF0001` for the arrows and +navigation keys — private-use precisely so a functional key can never be +confused with a real codepoint arriving as text. `pardes_mouse` speaks acme's +vocabulary: three buttons, where 1 selects, 2 executes and 3 looks, with wheel +directions as ordinary buttons rather than a separate axis. Ctrl is the only +modifier the core consults (a left press with ctrl is goto-definition), but the +full mask is passed anyway to keep the signature identical to the web one. +`pardes_scroll` carries a fractional row delta from a trackpad; the Zig side +accumulates it and synthesizes whole `wheel_up`/`wheel_down` presses, because +the core scrolls on button events and its `touch_scroll` event only records the +residual for the debug overlay. `src/gui/gui.zig` (`takeScrollTicks`) and +`src/web/app.mjs` both do exactly this already. A real mouse notch skips the +smoothing and goes through `pardes_mouse`. `pardes_resize` also carries one cell +in *physical* pixels, which only the native PDF placement path reads — pass the +backing-store size, not points. + +**Runtime callbacks.** `pardes_runtime_s` is a `userdata` and two functions, +copied by value during init so the struct need not outlive the call. `wakeup` +means "your state moved, please pump me"; `set_clipboard` fires inside a tick +when the yank register changes, with the text borrowed for the duration of the +call. There is no `open_url` callback because the core already opens links +itself through `/usr/bin/open` (`src/look.zig`), and no file callbacks because +it owns the filesystem side too. + +**Lifecycle.** `pardes_init` returns 0 or an opaque nonzero code, +`pardes_deinit` tears down, `pardes_tick` drains pty output plus the effect +queue and returns whether anything changed, `pardes_should_quit` reports the +Exit builtin or the last pane closing, and `pardes_animating` says a theme +transition wants ~60 Hz ticks until it settles — the one thing in an otherwise +event-driven frontend that redraws on a clock, handled in `tty.zig` by sleeping +the loop and forcing a tick. + +The ordering contract is the part a header cannot enforce. **Init with the real +grid size.** The core defers each shell's greeting until it has seen a resize: +`sync()` in `src/pardes.zig` only emits the opening `ls` once `resize_count > 0` +and the pty has produced its first prompt. Setting `Options.cols`/`rows` alone +never bumps that counter, so init must turn its arguments into an actual resize +event the way `src/web.zig` does immediately after construction — and the size +must be true, because the first `forkpty` takes its winsize from the core's +current grid and a shell booted at a placeholder draws its first prompt at the +wrong width. **Call `pardes_tick` after every input function.** The input calls +only advance the state machine; the writes to the pty, the spawns, the saves all +happen in the drain, so an input without a following tick is an input that +visibly did nothing. **`wakeup` is the only any-thread entry point in the whole +ABI.** Everything else is main-thread only, and the host's `wakeup` must do +nothing but hop — one `DispatchQueue.main.async` that calls `pardes_tick` and +marks the view dirty. + +## Threading + +One core, touched only from the main thread, plus one pty reader task per pane. +A reader blocks in `read(2)`, appends into that pane's mutex-guarded buffer, and +calls `wakeup`; the next tick swaps the buffers under the lock and feeds the +bytes in as `output` events. Sixteen panes is the ceiling (`MAX_PANES`), so it +is sixteen threads worst case, each of which does nothing but move bytes. + +Ghostty again is the contrast: it runs a renderer thread *and* an IO thread per +surface, each with its own mailbox and wakeup, because it owns the frame clock +and must render independently of input. Pardes does not own the clock here — the +host does, through `wakeup` and dirty rects — so there is no third thread to +synchronize and no mailbox protocol to get wrong. + +## Building + +Two commands, because they are two different machines' problems. + +```sh +zig build -Dplatform=macos +``` + +produces `zig-out/lib/libpardes.a` and installs `zig-out/include/pardes.h` +beside it. This is ordinary Zig cross-compilation and runs anywhere, which is +the whole point of the next section. + +```sh +zig build macos-app -Dplatform=macos # on a Mac; or run the script directly +``` + +runs `src/macos/build-app.sh`, which is the whole second half: + +```sh +swiftc -O -import-objc-header src/macos/pardes.h \ + -o zig-out/pardes.app/Contents/MacOS/pardes \ + src/macos/Sources/*.swift \ + zig-out/lib/libpardes.a -lc++ \ + -framework AppKit -framework CoreText -framework CoreGraphics +``` + +plus copying `Contents/Info.plist`, and the bundle is done. The header goes in +through `-import-objc-header` rather than a module map, because the header is +read straight out of the source tree and there is nothing to stage; a module map +is what an *xcframework* needs, and there isn't one. `-lc++` is there because +ghostty-vt pulls in simdutf and highway, which are C++ — the Zig side bundles +`compiler_rt` and `ubsan_rt` into the archive (`bundle_compiler_rt` in +`build.zig`), so the C++ runtime is the only thing left for this link to supply. +Without that bundling the swiftc link ends in undefined symbols, which is the +first thing ghostty's `GhosttyLib.initStatic` does too. + +No Xcode project, no xcframework, no `lipo`, no codesigning — ghostty has all four +(`macos/Ghostty.xcodeproj`, `src/build/GhosttyXCFramework.zig`, the entitlements +files), and every one of them exists for *distribution*: a universal binary for +two architectures, a framework other targets can consume, a signature and +notarization for Gatekeeper. A dev build that runs on the machine that compiled +it needs none of it. They are the named upgrade path, in that order: `lipo` when +a second architecture matters, an xcframework when something other than this app +links the core, codesigning and entitlements the day it is handed to someone +else. + +## The Linux dev loop + +The Zig half of this backend is plain POSIX. `forkpty`, `read`, `write`, +`ioctl(TIOCSWINSZ)` and `/usr/bin/open` differ from the tty backend by a string +constant. So `-Dplatform=macos` compiles on a Linux host, and its tests run +there: + +```sh +zig build unit-test -Dplatform=macos +``` + +Only the Swift app needs a Mac. That is what makes this scaffold verifiable +rather than dead code: the ABI's Zig side, the pty plumbing and the effect drain +are all exercised on the machine they were written on, and the part that cannot +be is small, visible, and made of AppKit calls. + +The header is kept honest with ghostty's trick. `build.zig` runs `translate-C` +over `src/macos/pardes.h` into the unit-test build, and `src/macos.zig` +asserts every constant and every struct layout against the Zig side — the color +tags, the attribute bits, the key codepoints, the mouse and modifier ordinals, +`@sizeOf(pardes_cell_s)` and each field offset. A hand-written header is a +second source of truth, and the only defensible way to keep one is a test that +fails the moment the two disagree. + +A second test compares every exported function's arity and scalar widths against +the header's declaration. It is not a type equality — `translate-C` spells +pointers `[*c]` and mints its own struct types, so nothing would ever match +exactly — but arity and width are what actually break. It earned its place +immediately: `pardes_scroll` grew a cell coordinate after the Swift view had +already been written against the one-argument form, and nothing but a human +reading both files would have caught it. + +## Not implemented + +- **The `lsp` effect.** Needs a worker plus a snapshot of the pane's path and + content taken *before* it starts, as `LspJob` in `tty.zig` does; the core + keeps editing while a query is in flight. Until then language queries are + dropped and nothing waits for an answer. +- **The `pipe` effect.** Selection filters need `pipeRequest(id)` copied into a + job, a worker to run the command, and a `pipe_resp` event back. Same shape as + `lsp`, one more response type. +- **The `watch` effect.** `watchPane` is inotify and returns silently off Linux + (`ponytail:` at `src/tty/tty.zig:1035`). macOS wants the FSEvents half of + `std.Build.Watch`, which the standard library already has as + `Build/Watch/FsEvents.zig`. Without it the core simply never receives + `file_changed`, a state it tolerates because the browser has no filesystem + either. +- **IME and marked text.** Only finished characters reach `pardes_key`. Real + composition means implementing `NSTextInputClient` and giving the core a way + to render an underlined preedit run, which no backend has yet. +- **Tabs and splits at the window level.** One window, one grid. Pardes's own + columns and panes are the layout, and a second window would need a second + core, which the singleton ABI is precisely a decision not to have yet. +- **Native image and PDF placement.** `Surface.images` is ignored. The tty + backend draws these with Kitty graphics and the SDL one with GPU textures; + macOS would be a third path, a `CALayer` or `CGImage` per placement positioned + from the cell rectangle. `pardes_resize` already carries the physical cell + size that path needs. +- **App-bundle resources.** The plist is the minimum that makes a windowed app: + no icon, no bundled fonts, no asset catalog, no localization. +- **argv and file-open handling.** No positional path, no `-l` dump load, and no + `application:openFile:`. The first pane spawns with an empty cwd, so the shell + inherits the process's — which for a bundle launched from Finder is `/`, and + is the first thing worth fixing here. |