From 044da0f75ab5fb5be0213e512741860d95dad8e5 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Thu, 29 Oct 2026 11:14:38 -0300 Subject: tape: every input the core consumes recorded in turn order, replayed into replicas that must reach the same digests; the core's leaks of render, clock and host closed The recorder (src/tape.zig) is null unless PARDES_RECORD= names a tape (.tape); the local path pays a null check a call. It records a tree at the host boundary, both ways: each call the host makes into the core (`Input`: update, serveFs, pump, perform, the shell's acknowledgements and failures, a Restore, ...) with its arguments, and each call the core makes out -- a Host function (the vtable wrapped generically while recording) or a syscall of its own (`tape.site` for every old `turn.yield()`, `tape.value` for a clock or environment read) -- with its result; `done`/`ret` close each, a 9P request served while a step is out lands inside that step's syscall, and fields a host writes between calls (native_images, present_skipped, ...) are found by a diff at every crossing. After each call the host made at the top, the session's digest (src/walk.zig), one per field of the core, its stream and each pane, file and places. `PARDES_REPLAY= pardes` (main.zig) replays it with the same build into three replicas: two at different addresses (the first's memory held while the second runs) and one that never draws. A host call answers what it answered, a site returns what it returned, nothing is performed and no syscall is made; `turn.yield` panics in a replica, so a syscall no site covers is a stack trace; a call the recording did not make, or a digest that differs, is a divergence that names the fields. The headless replica steps over the presents it does not make, applying what the host told the core inside them. What the replicas found, fixed: - Render wrote core state. The view's geometry (wrap map, context rows, body layer rows) is File.geometry, a lazy cache keyed by a hash of what it is made of (revision, scroll, size, tags, tables read, context declarations, the session's view: wrap, layered context, row metrics, rect); the render makes it afresh, a step reading it makes it on demand (wrapAt, wrapRow, ensureCursorVisible). Context declarations, notice bands and the header's reveal are the step's (sync), the tables' reading the tick's (a first look of 4096 lines, a count, wherever it is asked; the rest a frame's clock budget a tick), and a view's paint-ahead is asked of the host as the frame is drawn rather than queued as an effect. - The clock: advance's wakes are the recording session's (a picture decoding or a fade adds no tick to a replica), as are the budgets of Places, Names and Tables (`tape.over`), a message's time of day (the stamp; msg_said_ms and the unattended expiry use the session clock), File.mtime, and the pump reads its clock whether or not it waits. - The host: the session directory (getcwd, asked once at init), $SHELL and its existence, $HOME for `~`, `Shell` naming one that is not there, a missing interpreter, a directory refused above, a deleted file, a pointer seen through a sliding pane (Pardes.pointerCell). - Worker handles (futures, the Io), the 9P server's state, the store file every window shares (a replica keeps its own in memory) and Recent read from it are authority-only; frames, animations and the render's caches local. Query rows' paints count apart from a view's. Terminals replay from their taped pty bytes and are digested as their active screen, cursor and scroll; kitty images are left out until phase 5's screen-state terminals. tagnav.golden: the header's caret reveal now runs every step, as a pane's does, not once per frame drawn, so after `g l B B` typed in one burst the bar stays scrolled as a human typing it sees it. Co-Authored-By: Claude Opus 5.5 --- src/tape.zig | 741 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 741 insertions(+) create mode 100644 src/tape.zig (limited to 'src/tape.zig') diff --git a/src/tape.zig b/src/tape.zig new file mode 100644 index 00000000..eaf1a9fa --- /dev/null +++ b/src/tape.zig @@ -0,0 +1,741 @@ +//! The recorder and its tape (docs/next-daemon.md, phase 1). Recording, +//! everything the core consumes is appended in the order it consumed it, +//! which is the turn's order (a 9P request served while a step is out in a +//! syscall lands inside that syscall's record): +//! +//! - each call the host makes into the core (`Input`), with its arguments; +//! - each call the core makes out into the host -- a Host function, or a +//! syscall of its own (`Site`) -- and what it returned; +//! - each field the host wrote in the core between calls (`watched`); +//! - after every call the host made at the top, the session's digest. +//! +//! Replaying (`PARDES_REPLAY= pardes`, the same build), a core takes +//! its inputs off the tape instead: a host call answers what it answered, a +//! site returns what it returned, nothing is performed and no syscall is +//! made, and each digest is checked against the recorded one -- by two +//! replicas at different addresses and a third that never draws. +//! +//! The local path pays a null check a call: `on` is null unless +//! PARDES_RECORD names where to write, `.tape`. +const std = @import("std"); +const pardes = @import("pardes.zig"); +const walk = @import("walk.zig"); +const Pardes = pardes.Pardes; +const VTable = pardes.Host.VTable; +const libc = std.c; + +pub var on: ?*Tape = null; + +/// Every call into the core a host makes (a 9P connection task's too). +pub const Input = union(enum) { + init: pardes.Options, + init_from_dump: struct { opts: pardes.Options, bytes: []const u8 }, + restore: struct { bytes: []const u8, from: []const u8 }, + /// Whether it was the session's own core, or one a Restore replaced. + deinit: bool, + update: pardes.Event, + serve_fs: pardes.ctlfs.Req, + pump, + perform: pardes.Effect, + next_effect, + advance: u64, + render, + take_restore, + set_cwd: struct { id: usize, cwd: []const u8 }, + acknowledge_shell: struct { id: usize, executable: []const u8, prompt_marks: bool }, + shell_failed: struct { id: u8, err: anyerror }, + restart_failed: struct { id: u8, err: anyerror }, + save_failed: struct { id: u8, path: []const u8, err: anyerror }, + dump_failed: struct { path: []const u8, err: anyerror }, + set_last_dump: []const u8, + set_message: struct { id: usize, text: []const u8 }, + set_status: struct { id: usize, text: []const u8 }, + report_error: struct { id: usize, operation: []const u8, err: anyerror }, + reload_watched_file: struct { id: u8, bytes: []const u8 }, + note_late_failure: usize, + invalidate, + acknowledge_panel_presentation: []const pardes.animation.Track, + run_closed_lines, + file_deleted: u8, +}; + +/// The core's own syscalls (each `turn.yield()`s around itself), numbered +/// after the Host functions on the tape. +pub const Site = enum(u16) { + // Out of the turn (`site`). + os_stat, + os_listing, + os_rw, + make_dirs, + resolve, + read_mount, + read_file, + dir_listing, + kind, + probe, + preview, + // Read in place (`value`). + clock, + budget, + wake, + env, + interpreter, + denied, + shell, + mtime, + cwd, + pointer, +}; + +/// Fields a host writes straight into the core between its calls. +const watched = .{ "native_images", "native_image_patches", "present_skipped", "shell_continuous", "needs_frame", "quit", "windowed", "active" }; + +const Kind = enum(u8) { input, done, host, ret, set, digest }; +const host_fns = @typeInfo(VTable).@"struct".field_names; +/// The `set` that says which Host functions the recording host had. +const set_mask = 0xff; + +pub const Tape = struct { + gpa: std.mem.Allocator, + replay: bool, + /// The core whose digest is taken (a Restore replaces it). + core: ?*Pardes = null, + /// The host has the turn: a call into the core now is an input, and a + /// call out of it is the host's to answer. + host_side: bool = true, + depth: u32 = 0, + vtable: VTable = .{}, + // Recording. + fd: c_int = -1, + buf: std.ArrayList(u8) = .empty, + scratch: std.Io.Writer.Allocating, + real: pardes.Host = .{}, + seen: [watched.len]u64 = @splat(0), + // Replaying. + in: walk.Reader = .{ .bytes = &.{} }, + arena: std.heap.ArenaAllocator, + last_ret: walk.Reader = .{ .bytes = &.{} }, + mask: u64 = 0, + headless: bool = false, + digests: u64 = 0, + diverged: ?[]const u8 = null, + /// The input being applied, for a divergence's report. + applying: []const u8 = "", + /// PARDES_RECORD_EVERY: a digest after every input, a nested one's too + /// (to find the step a divergence starts at), not only the host's last. + every: bool = false, + + fn record(t: *Tape, kind: Kind, comptime T: type, v: *const T) void { + t.scratch.clearRetainingCapacity(); + walk.encode(T, v, &t.scratch.writer) catch @panic("tape: out of memory"); + const payload = t.scratch.written(); + const len: u32 = @intCast(payload.len); + t.buf.append(t.gpa, @intFromEnum(kind)) catch @panic("tape: out of memory"); + t.buf.appendSlice(t.gpa, std.mem.asBytes(&len)) catch @panic("tape: out of memory"); + t.buf.appendSlice(t.gpa, payload) catch @panic("tape: out of memory"); + if (t.buf.items.len > 1 << 20) t.flush(); + } + + fn flush(t: *Tape) void { + var at: usize = 0; + while (at < t.buf.items.len) { + const n = libc.write(t.fd, t.buf.items[at..].ptr, t.buf.items.len - at); + if (n <= 0) break; + at += @intCast(n); + } + t.buf.clearRetainingCapacity(); + } + + /// What the host wrote between calls, said before the core goes on. + fn noteWrites(t: *Tape) void { + const p = t.core orelse return; + inline for (watched, 0..) |name, i| { + const now = walk.digest(@FieldType(Pardes, name), &@field(p, name)); + if (now != t.seen[i]) { + t.seen[i] = now; + const set: struct { u8, @FieldType(Pardes, name) } = .{ i, @field(p, name) }; + t.record(.set, @TypeOf(set), &set); + } + } + } + + fn seeFields(t: *Tape) void { + const p = t.core orelse return; + inline for (watched, 0..) |name, i| t.seen[i] = walk.digest(@FieldType(Pardes, name), &@field(p, name)); + } + + /// The host's functions go through the tape while it records, whoever + /// set `p.host` last. + fn wrap(t: *Tape, p: *Pardes) void { + if (p.host.vtable == &t.vtable) return; + // The same host given again (each pump is given its host): only + // the wrapper put back. + if (p.host.vtable == t.real.vtable and p.host.ctx == t.real.ctx) { + p.host = .{ .ctx = t, .vtable = &t.vtable }; + return; + } + t.real = p.host; + var mask: u64 = 0; + inline for (host_fns, 0..) |f, i| if (@field(t.real.vtable.*, f) != null) { + mask |= @as(u64, 1) << i; + }; + p.host = t.hostOf(mask); + const set: struct { u8, u64 } = .{ set_mask, mask }; + t.record(.set, @TypeOf(set), &set); + } + + /// The tape as a host with these functions. + fn hostOf(t: *Tape, mask: u64) pardes.Host { + t.mask = mask; + t.vtable = .{}; + inline for (host_fns, 0..) |f, i| if (mask & (@as(u64, 1) << i) != 0 or + (t.headless and comptime std.mem.eql(u8, f, "watched"))) + { + @field(t.vtable, f) = &Wrap(i).f; + }; + return .{ .ctx = t, .vtable = &t.vtable }; + } + + fn enterInput(t: *Tape, p: ?*Pardes, input: *const Input) Entered { + if (!t.host_side) return .{}; + t.host_side = false; + t.depth += 1; + if (t.replay) return .{ .outer = true }; + if (p) |core| { + if (t.core == null) t.core = core; + t.noteWrites(); + t.wrap(core); + } + t.record(.input, Input, input); + return .{ .outer = true }; + } + + fn leaveInput(t: *Tape, p: ?*Pardes) void { + t.depth -= 1; + t.host_side = true; + if (t.replay) return; + if (p) |core| if (t.core == null) { + t.core = core; + t.wrap(core); + }; + t.record(.done, void, &{}); + if (t.depth == 0 or t.every) if (t.core) |core| { + var parts: Parts = undefined; + const d = digestParts(core, &parts); + t.record(.digest, []const u64, &d); + }; + if (t.depth == 0) t.flush(); + t.seeFields(); + } + + // ---- replaying ---- + + const Record = struct { kind: Kind, payload: walk.Reader }; + + fn next(t: *Tape) ?Record { + const head = t.in.take(5) catch return null; + const len = std.mem.bytesToValue(u32, head[1..5]); + const payload = t.in.take(len) catch return null; + return .{ .kind = @enumFromInt(head[0]), .payload = .{ .bytes = payload } }; + } + + fn peek(t: *const Tape) ?Kind { + if (t.in.bytes.len - t.in.at < 5) return null; + return @enumFromInt(t.in.bytes[t.in.at]); + } + + fn diverge(t: *Tape, comptime why: []const u8, args: anytype) void { + if (t.diverged != null) return; + t.diverged = std.fmt.allocPrint(t.gpa, why ++ " (tape byte {d}, in {s})", args ++ .{ t.in.at, t.applying }) catch why; + } + + /// A host call the headless replica does not make (it draws no frame, + /// so presents none): what the host told the core inside it (what it + /// presented, `acknowledge_panel_presentation`) is applied all the + /// same, the host's own calls below it stepped over. + fn skipHost(t: *Tape) void { + while (t.diverged == null) { + var rec = t.next() orelse return; + switch (rec.kind) { + .ret => return, + .input => t.apply(&rec.payload), + .set => t.applySet(&rec.payload), + .host => t.skipHost(), + .digest => t.check(&rec.payload), + .done => return t.diverge("a host call's records are out of order", .{}), + } + } + } + + /// Over a call the replica does not make (the headless one's present), + /// what was applied inside it too. + fn skip(t: *Tape) void { + var depth: usize = 1; + while (depth > 0) { + const rec = t.next() orelse return; + switch (rec.kind) { + .host, .input => depth += 1, + .ret, .done => depth -= 1, + else => {}, + } + } + } + + /// The records of an input up to its end: what was applied inside it, + /// as it was; the headless replica steps over the host calls it did + /// not make. + fn finishInput(t: *Tape) void { + while (t.diverged == null) { + const kind = t.peek() orelse return t.diverge("the tape ends inside an input", .{}); + var rec = t.next().?; + switch (kind) { + .done => return, + .set => t.applySet(&rec.payload), + .host => if (t.headless) t.skipHost() else return t.diverge("the core did not make the host call {d} the tape has", .{std.mem.bytesToValue(u16, rec.payload.bytes[0..2])}), + .input => t.apply(&rec.payload), + .digest => t.check(&rec.payload), + .ret => return t.diverge("the core's input ended before the recorded one", .{}), + } + } + } + + /// The records inside a host call up to its return, whose result is + /// left in `last_ret`. + fn finishHost(t: *Tape) bool { + while (t.diverged == null) { + var rec = t.next() orelse { + t.diverge("the tape ends inside a host call", .{}); + return false; + }; + switch (rec.kind) { + .ret => { + t.last_ret = rec.payload; + return true; + }, + .input => t.apply(&rec.payload), + .set => t.applySet(&rec.payload), + .digest => t.check(&rec.payload), + .host, .done => { + t.diverge("a host call's records are out of order", .{}); + return false; + }, + } + } + return false; + } + + fn expectHost(t: *Tape, id: u16, comptime what: []const u8) bool { + if (t.diverged != null) return false; + const rec = t.next() orelse { + t.diverge("the core called " ++ what ++ " past the tape's end", .{}); + return false; + }; + if (rec.kind != .host or std.mem.bytesToValue(u16, rec.payload.bytes[0..2]) != id) { + const other: u16 = if (rec.kind == .host) std.mem.bytesToValue(u16, rec.payload.bytes[0..2]) else 0; + t.diverge("the core called " ++ what ++ " where the tape has a {t} ({d})", .{ rec.kind, other }); + return false; + } + const outer = !t.host_side; + t.host_side = true; + defer t.host_side = !outer; + return t.finishHost(); + } + + /// A recorded digest against the replica's own. + fn check(t: *Tape, r: *walk.Reader) void { + const want = walk.decode([]const u64, r, t.arena.allocator()) catch return; + const p = t.core orelse return; + var parts: Parts = undefined; + const got = digestParts(p, &parts); + if (!std.mem.eql(u64, got, want)) t.diverge("digest {d} differs in {s}", .{ t.digests, partsDiffer(got, want, t.arena.allocator()) }); + t.digests += 1; + } + + fn applySet(t: *Tape, r: *walk.Reader) void { + const which = (r.take(1) catch return)[0]; + if (which == set_mask) { + const host = t.hostOf(std.mem.bytesToValue(u64, r.take(8) catch return)); + if (t.core) |p| p.host = host; + return; + } + const p = t.core orelse return; + inline for (watched, 0..) |name, i| if (i == which) { + @field(p, name) = walk.decode(@FieldType(Pardes, name), r, t.arena.allocator()) catch return; + }; + } + + /// One input, applied to the replica as the host applied it. + fn apply(t: *Tape, r: *walk.Reader) void { + // A session's options are kept by its core (`opts`): theirs for good. + const Tag = std.meta.Tag(Input); + const tag = std.mem.bytesToValue(Tag, (r.take(@sizeOf(Tag)) catch return t.diverge("a short input", .{}))); + r.at -= @sizeOf(Tag); + const kept = tag == .init or tag == .init_from_dump or tag == .restore; + const input = walk.decode(Input, r, if (kept) t.gpa else t.arena.allocator()) catch return t.diverge("a short input", .{}); + const outer = t.applying; + t.applying = switch (input) { + .update => |ev| @tagName(ev), + else => @tagName(input), + }; + defer t.applying = outer; + t.host_side = true; + defer t.host_side = true; + switch (input) { + .init, .init_from_dump => { + var opts = switch (input) { + .init => |o| o, + .init_from_dump => |d| d.opts, + else => unreachable, + }; + opts.image_allocator = t.gpa; + opts.pdf_allocator = t.gpa; + opts.tree_sitter_allocator = t.gpa; + opts.frame_allocator = t.gpa; + t.core = switch (input) { + .init => Pardes.init(t.gpa, opts), + .init_from_dump => |d| pardes.dump.initFromDump(t.gpa, opts, d.bytes), + else => unreachable, + } catch |err| return t.diverge("init failed: {t}", .{err}); + t.core.?.host = t.hostOf(t.mask); + return t.finishInput(); + }, + else => {}, + } + const p = t.core orelse return t.diverge("an input before the core", .{}); + switch (input) { + .init, .init_from_dump => unreachable, + .restore => |rs| { + const nc = pardes.dump.restore(p, rs.bytes, rs.from) catch |err| return t.diverge("restore failed: {t}", .{err}); + t.core = nc; + nc.host = t.hostOf(t.mask); + }, + // The session is over (or the core a Restore replaced is + // freed): nothing done now is compared. + .deinit => |own| { + if (own) t.core = null; + return t.skip(); + }, + .update => |ev| p.update(ev), + .serve_fs => |req| _ = p.serveFs(req), + .pump => p.pump(t.hostOf(t.mask)) catch {}, + .perform => |e| p.perform(e), + .next_effect => _ = p.nextEffect(), + .advance => |now| p.advance(now), + .render => if (t.headless) return t.skip() else { + _ = p.frame_arena.reset(.retain_capacity); + _ = p.render(p.frame_arena.allocator()) catch {}; + }, + .take_restore => _ = p.takeRestore(), + .set_cwd => |a| p.setCwd(a.id, a.cwd), + .acknowledge_shell => |a| p.acknowledgeShell(a.id, a.executable, a.prompt_marks), + .shell_failed => |a| p.shellFailed(a.id, a.err), + .restart_failed => |a| p.restartFailed(a.id, a.err), + .save_failed => |a| p.saveFailed(a.id, a.path, a.err), + .dump_failed => |a| p.dumpFailed(a.path, a.err), + .set_last_dump => |path| p.setLastDump(path), + .set_message => |a| p.setMessage(a.id, a.text), + .set_status => |a| p.setStatus(a.id, a.text), + .report_error => |a| p.reportError(a.id, a.operation, a.err), + .reload_watched_file => |a| _ = p.reloadWatchedFile(a.id, a.bytes), + .note_late_failure => |id| p.noteLateFailure(id), + .invalidate => p.invalidate(), + .acknowledge_panel_presentation => |tracks| p.acknowledgePanelPresentation(tracks), + .run_closed_lines => pardes.ctlfs.runClosedLines(p), + .file_deleted => |id| pardes.panes.File.deleted(p, id), + } + t.finishInput(); + } +}; + +const pardes_fields = walk.fieldNames(Pardes); +const stream_fields = walk.fieldNames(pardes.Stream); +const pane_fields = walk.fieldNames(pardes.Pane); +const file_fields = walk.fieldNames(pardes.panes.File.State); +const places_fields = walk.fieldNames(pardes.panes.Places); +const Parts = [pardes_fields.len + stream_fields.len + pardes.MAX_PANES * (2 + pane_fields.len + file_fields.len + places_fields.len)]u64; + +/// The session's digest as one per field of the core's, its stream's, and +/// each pane's and its file's, so a divergence says where it is. +fn digestParts(p: *const Pardes, parts: *Parts) []const u64 { + var n = walk.fieldDigests(Pardes, p, parts[0..pardes_fields.len]).len; + n += walk.fieldDigests(pardes.Stream, &p.stream, parts[n..][0..stream_fields.len]).len; + for (p.panes, 0..) |slot, id| { + const pane = slot orelse continue; + parts[n] = id; + parts[n + 1] = @intFromBool(pane.file != null); + n += 2; + n += walk.fieldDigests(pardes.Pane, pane, parts[n..][0..pane_fields.len]).len; + if (pane.file) |*f| n += walk.fieldDigests(pardes.panes.File.State, f, parts[n..][0..file_fields.len]).len; + n += walk.fieldDigests(pardes.panes.Places, &pane.places, parts[n..][0..places_fields.len]).len; + } + return parts[0..n]; +} + +/// The fields whose digests differ, named. +fn partsDiffer(got: []const u64, want: []const u64, a: std.mem.Allocator) []const u8 { + var said: std.ArrayList(u8) = .empty; + if (got.len != want.len) said.print(a, "the panes there are ({d} parts, {d} recorded); ", .{ got.len, want.len }) catch {}; + const n = @min(got.len, want.len); + var i: usize = 0; + var count: usize = 0; + while (i < pardes_fields.len + stream_fields.len and i < n) : (i += 1) { + if (got[i] == want[i]) continue; + const name = if (i < pardes_fields.len) pardes_fields[i] else stream_fields[i - pardes_fields.len]; + said.print(a, "{s}{s} ", .{ if (i < pardes_fields.len) "" else "stream.", name }) catch {}; + count += 1; + } + while (i + 2 <= n and count < 16) { + const id = got[i]; + const has_file = got[i + 1] != 0; + i += 2; + for (pane_fields) |name| { + if (i >= n) break; + if (got[i] != want[i]) { + said.print(a, "pane {d}.{s} ", .{ id, name }) catch {}; + count += 1; + } + i += 1; + } + if (has_file) for (file_fields) |name| { + if (i >= n) break; + if (got[i] != want[i]) { + said.print(a, "pane {d}.file.{s} ", .{ id, name }) catch {}; + count += 1; + } + i += 1; + }; + for (places_fields) |name| { + if (i >= n) break; + if (got[i] != want[i]) { + said.print(a, "pane {d}.places.{s} ", .{ id, name }) catch {}; + count += 1; + } + i += 1; + } + } + return said.items; +} + +/// A call into the core: the host's, and recorded, or the core's own. +pub const Entered = struct { outer: bool = false }; + +pub inline fn enter(p: ?*Pardes, input: Input) Entered { + const t = on orelse return .{}; + return t.enterInput(p, &input); +} + +pub inline fn leave(p: ?*Pardes, e: Entered) void { + if (e.outer) on.?.leaveInput(p); +} + +/// The core a Restore made is the session's from here. +pub fn adopt(p: *Pardes) void { + const t = on orelse return; + if (t.replay) return; + t.core = p; + t.wrap(p); +} + +/// Whether `p` is the core being recorded (a Restore's old one is not). +pub fn current(p: *const Pardes) bool { + const t = on orelse return true; + return t.core == null or t.core == p; +} + +/// The session's core is freed: no digest of it after. +pub fn forget(p: *Pardes) void { + const t = on orelse return; + if (t.core == p) t.core = null; +} + +/// The Host function `i`, through the tape. +fn Wrap(comptime i: usize) type { + const Fn = @typeInfo(@typeInfo(@FieldType(VTable, host_fns[i])).optional.child).pointer.child; + const info = @typeInfo(Fn).@"fn"; + const R = info.return_type.?; + const P = info.param_types; + return switch (P.len) { + 1 => struct { + fn f(ctx: ?*anyopaque) R { + return hostCall(i, R, ctx, .{}); + } + }, + 2 => struct { + fn f(ctx: ?*anyopaque, a: P[1].?) R { + return hostCall(i, R, ctx, .{a}); + } + }, + 3 => struct { + fn f(ctx: ?*anyopaque, a: P[1].?, b: P[2].?) R { + return hostCall(i, R, ctx, .{ a, b }); + } + }, + 4 => struct { + fn f(ctx: ?*anyopaque, a: P[1].?, b: P[2].?, c: P[3].?) R { + return hostCall(i, R, ctx, .{ a, b, c }); + } + }, + 5 => struct { + fn f(ctx: ?*anyopaque, a: P[1].?, b: P[2].?, c: P[3].?, d: P[4].?) R { + return hostCall(i, R, ctx, .{ a, b, c, d }); + } + }, + else => @compileError("tape: a host function of more arguments"), + }; +} + +fn hostCall(comptime i: usize, comptime R: type, ctx: ?*anyopaque, args: anytype) R { + const t: *Tape = @ptrCast(@alignCast(ctx.?)); + const name = host_fns[i]; + if (t.replay) { + // A replica that never draws is never watched: it asks no one. + if (comptime std.mem.eql(u8, name, "watched")) if (t.headless) return false; + if (!t.expectHost(i, name)) return std.mem.zeroes(R); + var ret = t.last_ret; + if (R == ?[]const u8) { + // fg_name's answer is written into the caller's buffer. + const text = (walk.decode(R, &ret, t.arena.allocator()) catch return null) orelse return null; + const buf = args[args.len - 1]; + const n = @min(text.len, buf.len); + @memcpy(buf[0..n], text[0..n]); + return buf[0..n]; + } + return walk.decode(R, &ret, t.arena.allocator()) catch std.mem.zeroes(R); + } + const real = @field(t.real.vtable.*, name).?; + const id: u16 = i; + t.record(.host, u16, &id); + t.seeFields(); + const outer = !t.host_side; + t.host_side = true; + const r = @call(.auto, real, .{t.real.ctx} ++ args); + t.host_side = !outer; + t.noteWrites(); + t.record(.ret, R, &r); + return r; +} + +/// A syscall of the core's own, `f(args)`, made out of the turn (a 9P +/// request through a mount of this very session is answered meanwhile): +/// recorded with what it returned, or answered from the tape and never +/// made. A result with slices is decoded into `a` on replay, the allocator +/// `f` would have used. +pub fn site(comptime s: Site, a: ?std.mem.Allocator, comptime f: anytype, args: anytype) @typeInfo(@TypeOf(f)).@"fn".return_type.? { + return call(s, a, f, args, true); +} + +/// A read of the host's that is no syscall out of the turn (the clock, the +/// environment) or yields for itself: as `site`. +pub fn value(comptime s: Site, comptime f: anytype, args: anytype) @typeInfo(@TypeOf(f)).@"fn".return_type.? { + return call(s, null, f, args, false); +} + +fn call(comptime s: Site, a: ?std.mem.Allocator, comptime f: anytype, args: anytype, comptime yields: bool) @typeInfo(@TypeOf(f)).@"fn".return_type.? { + const R = @typeInfo(@TypeOf(f)).@"fn".return_type.?; + const t = on orelse return if (yields) out(f, args) else @call(.auto, f, args); + const id: u16 = @as(u16, host_fns.len) + @intFromEnum(s); + if (t.replay) { + if (!t.expectHost(id, @tagName(s))) { + std.debug.print("replay: {s}\n", .{t.diverged orelse "?"}); + @panic("tape: the replay diverged at a host call of the core's own"); + } + var ret = t.last_ret; + return walk.decode(R, &ret, a orelse t.arena.allocator()) catch @panic("tape: a short result"); + } + // The host's own (a shell's spawn asking the shell), or one inside a + // site's syscall: the outer call is the record. + if (t.host_side) return if (yields) out(f, args) else @call(.auto, f, args); + t.record(.host, u16, &id); + t.seeFields(); + t.host_side = true; + const r = if (yields) out(f, args) else @call(.auto, f, args); + t.host_side = false; + t.noteWrites(); + t.record(.ret, R, &r); + return r; +} + +/// A budget of `budget_ns` from `since_ns` on the monotonic clock is spent: +/// what the clock said is taped, so a replica stops its work where this did. +pub fn over(since_ns: u64, budget_ns: u64) bool { + return value(.budget, overNow, .{ since_ns, budget_ns }); +} + +fn overNow(since_ns: u64, budget_ns: u64) bool { + return pardes.FrameStats.nowNs() -| since_ns > budget_ns; +} + +fn out(comptime f: anytype, args: anytype) @typeInfo(@TypeOf(f)).@"fn".return_type.? { + pardes.turn.yield(); + defer pardes.turn.back(); + return @call(.auto, f, args); +} + +/// Asked by `turn.yield`: a syscall a replica must never make. +pub fn replaying() bool { + return if (on) |t| t.replay else false; +} + +/// PARDES_RECORD set: the tape is written from the session's first call. +pub fn startFromEnv() void { + if (on != null) return; + const prefix = std.mem.span(libc.getenv("PARDES_RECORD") orelse return); + if (prefix.len == 0) return; + const gpa = std.heap.c_allocator; + var path_buf: [4096]u8 = undefined; + const path = std.fmt.bufPrintSentinel(&path_buf, "{s}{d}.tape", .{ prefix, libc.getpid() }, 0) catch return; + const fd = libc.open(path, .{ .ACCMODE = .WRONLY, .CREAT = true, .TRUNC = true, .CLOEXEC = true }, @as(libc.mode_t, 0o644)); + if (fd < 0) return; + const t = gpa.create(Tape) catch return; + t.* = .{ .gpa = gpa, .replay = false, .fd = fd, .scratch = .init(gpa), .arena = .init(gpa), .every = libc.getenv("PARDES_RECORD_EVERY") != null }; + on = t; +} + +/// `PARDES_REPLAY=`: the tape replayed into three replicas, each +/// digest checked; 0 when none diverged. +pub fn replayMain(io: std.Io, path: []const u8) u8 { + const gpa = std.heap.c_allocator; + // What a shell starts before its core (tty.zig's localSession). + pardes.image.start(io, gpa); + if (comptime pardes.pdf_enabled) pardes.pdf.start(gpa); + pardes.syntax.start(gpa); + const bytes = std.Io.Dir.cwd().readFileAlloc(std.Io.Threaded.global_single_threaded.io(), path, gpa, .unlimited) catch |err| { + std.debug.print("replay: cannot read {s}: {t}\n", .{ path, err }); + return 2; + }; + const runs = [_]struct { name: []const u8, headless: bool }{ + .{ .name = "first", .headless = false }, + // The first replica's memory is still held: this one's lies elsewhere. + .{ .name = "second", .headless = false }, + .{ .name = "headless", .headless = true }, + }; + var failed = false; + for (runs) |run| { + var t: Tape = .{ .gpa = gpa, .replay = true, .in = .{ .bytes = bytes }, .headless = run.headless, .scratch = .init(gpa), .arena = .init(gpa) }; + on = &t; + defer on = null; + while (t.diverged == null) { + const kind = t.peek() orelse break; + var rec = t.next().?; + switch (kind) { + .input => { + t.apply(&rec.payload); + _ = t.arena.reset(.retain_capacity); + }, + .set => t.applySet(&rec.payload), + .digest => t.check(&rec.payload), + .host, .ret, .done => t.diverge("a {t} record at the top", .{kind}), + } + } + // A session killed mid-step leaves its last input cut short: the + // end of the tape, not a divergence. + if (t.diverged) |why| if (std.mem.startsWith(u8, why, "the tape ends")) { + t.diverged = null; + }; + if (t.diverged) |why| { + failed = true; + std.debug.print("replay {s}: {s} replica diverged after {d} digests: {s}\n", .{ path, run.name, t.digests, why }); + } else std.debug.print("replay {s}: {s} replica: {d} digests agree\n", .{ path, run.name, t.digests }); + } + return if (failed) 1 else 0; +} -- cgit v1.3 From 2cce341e2c94fb3150dc1f3e7911c26502a70487 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Thu, 29 Oct 2026 17:01:35 -0300 Subject: views: several clients on one session, each its own layout, focus, cursors and scroll over the panes they share; a second `pardes --attach` draws on its own terminal beside the first, and `pardes-gui --attach` opens another window The core still sees one view, the live one, in the fields it always had (src/views.zig): a view's share of the session (`views.fields`: its screen, columns, rects, focus, pointer and gestures, jumps, key prefixes, column tags, presentation and animations, windowed and native_images) and of each pane (`PaneView`: its place in the layout, which text has the keyboard, the body's cursor, selections and mode, its scroll, a document's size, what the step measured of it) is copied out when another view goes live and back when it is live again. One view never parks: the single client pays a test of one byte a step. `toView`, `openView` and `closeView` are taped inputs; the pump draws each other view by going into it and back, which leaves every digested field as it was, so the replica that draws nothing skips it. A parked view is in no digest (its slot is written whenever it goes out); what drawing it made of its own (animations, presentation) is kept, whatever its unpark brought inside a changed text is done again when it is next live. Shared: the panes' text, terminals, tags, prompts, messages, registers (yanked in one view, put in another), the workspace tag and settings. A terminal shown at two sizes takes the size of the view that typed into it last (`Pane.sized_by`), and any view's once that one is gone; the others show it at that size. A text edited in one view moves the other views' cursors and scroll with the splice, as a Zerox twin's (File.ParkedPlaces). A pane closed in one view leaves the others' layouts (by its slot's serial); a pane focused in a view that does not show it is placed in its active column (views.show). tty: each terminal is a Screen of its own (its vaxis, input reader, frame answer, kitty images and shm probe, glide, trail and flash); its input reaches the loop as its view's (a `from` ahead of it when another terminal posted last, so one terminal posts none). A SIGWINCH asks every terminal its size; a SIGHUP asks each whether it hung up or lost its stub. Detach, or a terminal's end, with other views left closes only that view, its stub told `d`; the session's end tells every stub. daemon.zig: an attach names an attached session too (or the only one there is when none is detached). gui: the Gui holds the live window's share of itself (window_fields: the SDL window and its presenting, scroll, glide, scene and post chain, pointer, blink, captures); the others wait in Shell.windows, swapped in (`follow`) as the core's live view changes, and each SDL event first takes its window's view live (`route`). Closing a window while others stay closes its view. Harnesses: `start views=N` in a snapshot script gives the app N-1 more terminals (PARDES_TEST_VIEWS, opened as attached ones), `view ` types into and captures terminal k, every wait reads them all; `pardes-monkey --views=N` switches among N; `zig build replay` runs its seeds at three views too. views.snap pins two terminals: an edit seen in both, a scroll in one, a line yanked in one and put in the other, Newcol in one, Detach in one. Unit tests: own cursor, scroll, mode, layout and focus; edits shared; registers shared; terminal size by last typist; closing views; drawing the others leaves the digest; a view keeps its place in a text another edits. Co-Authored-By: Claude Opus 5.5 --- build/snap.zig | 11 +- src/File.zig | 51 +++- src/daemon.zig | 39 ++- src/gui/gui.zig | 225 ++++++++++++++- src/panes.zig | 16 +- src/pardes.zig | 70 ++++- src/tape.zig | 6 + src/tty/tty.zig | 666 ++++++++++++++++++++++++++++---------------- src/views.zig | 555 ++++++++++++++++++++++++++++++++++++ test/daemon.py | 32 +++ test/e2e_harness.zig | 90 +++++- test/monkey.zig | 34 ++- test/snapshot.zig | 92 ++++-- test/snapshots/views.golden | 123 ++++++++ test/snapshots/views.snap | 57 ++++ 15 files changed, 1763 insertions(+), 304 deletions(-) create mode 100644 src/views.zig create mode 100644 test/snapshots/views.golden create mode 100644 test/snapshots/views.snap (limited to 'src/tape.zig') diff --git a/build/snap.zig b/build/snap.zig index 6982379c..d74767d3 100644 --- a/build/snap.zig +++ b/build/snap.zig @@ -86,7 +86,16 @@ pub fn addTty(b: *std.Build, opts: struct { run_monkey_replay.has_side_effects = true; // One pardes-snap run at a time (its lock): the scripts, then the seeds. run_monkey_replay.step.dependOn(&run_replay.step); - b.step("replay", "record every snapshot script's and -Dreplay-seeds monkey seeds' tapes and require each to replay without diverging").dependOn(&run_monkey_replay.step); + // The same seeds with three terminals on the session, each a view of + // it, typed into in turn (views.zig). + const run_views_replay = b.addRunArtifact(monkey); + run_views_replay.addArtifactArg(snap); + run_views_replay.addArtifactArg(opts.exe); + run_views_replay.addArgs(&.{ b.fmt("{d}", .{seeds}), "70", "--views=3", "--tape=" ++ tapes ++ "/monkey-views", "--out=" ++ tapes ++ "/monkey-views" }); + run_views_replay.setCwd(b.path(".")); + run_views_replay.has_side_effects = true; + run_views_replay.step.dependOn(&run_monkey_replay.step); + b.step("replay", "record every snapshot script's and -Dreplay-seeds monkey seeds' tapes (one terminal and three) and require each to replay without diverging").dependOn(&run_views_replay.step); const monkey_tests = b.addTest(.{ .root_module = monkey.root_module }); b.step("monkey-test", "test the monkey's generator and panic detector").dependOn(&b.addRunArtifact(monkey_tests).step); const run_driver_test = b.addRunArtifact(snap); diff --git a/src/File.zig b/src/File.zig index d795eaa1..dd6b2d34 100644 --- a/src/File.zig +++ b/src/File.zig @@ -1208,7 +1208,12 @@ pub fn setContentSpan(p: *Pardes, f: *State, new: []u8, span_in: ?Span) void { // the caller did not say, is what the line index wants anyway. var twins: [pardes.MAX_PANES]usize = undefined; const ntwins = twinsOf(p, f, &twins); - const span: ?Span = span_in orelse if (ntwins > 0) spanOf(f.content, new) else null; + // The views not live keep their place in it as twins do (views.zig). + const viewed = pardes.views.several(p); + const span: ?Span = span_in orelse if (ntwins > 0 or viewed) spanOf(f.content, new) else null; + var parked: ParkedPlaces = .{}; + if (viewed) parked.before(p, f, span.?, new.len); + defer if (viewed) parked.after(p); var moved: [pardes.MAX_PANES]struct { cursor: usize, scroll: usize } = undefined; for (twins[0..ntwins], 0..) |id, i| { const t = p.panes[id].?; @@ -1301,6 +1306,50 @@ pub fn setContentSpan(p: *Pardes, f: *State, new: []u8, span_in: ?Span) void { } } +/// Each other view's cursor and scroll on a text being changed, as offsets +/// moved by the splice (`spliced`), put back as places in the new text once +/// it is in: a view not live keeps its place, as a twin does. Its +/// selections go, as a twin's do. +const ParkedPlaces = struct { + const max = pardes.MAX_PANES * pardes.views.max; + at: [max]struct { pane: *Pane, view: u8, cursor: usize, scroll: usize } = undefined, + n: usize = 0, + + fn before(pp: *ParkedPlaces, p: *Pardes, f: *State, span: Span, new_len: usize) void { + for (p.panes) |slot| { + const pane = slot orelse continue; + const tf = if (pane.file) |*file| file else continue; + if (tf != f and (f.twin == 0 or tf.twin != f.twin)) continue; + for (0..pardes.views.max) |k| { + if (k == p.view or !pardes.views.isOpen(p, @intCast(k)) or pp.n == max) continue; + const v = &pane.pv[k]; + const at = lineStart(p.gpa, tf, @intCast(@max(0, v.body.cur_row))) + @as(usize, @intCast(@max(0, v.body.cur_col))); + pp.at[pp.n] = .{ + .pane = pane, + .view = @intCast(k), + .cursor = spliced(@min(at, f.content.len), f.content.len, new_len, span), + .scroll = spliced(@min(lineStart(p.gpa, tf, v.scroll), f.content.len), f.content.len, new_len, span), + }; + pp.n += 1; + } + } + } + + fn after(pp: *ParkedPlaces, p: *Pardes) void { + for (pp.at[0..pp.n]) |a| { + const tf = &(a.pane.file orelse continue); + const v = &a.pane.pv[a.view]; + const cursor = positionIn(p.gpa, tf, @min(a.cursor, tf.content.len)); + v.body.cur_row = @intCast(cursor.row); + v.body.cur_col = @intCast(cursor.col); + v.body.vsel.active = false; + v.body.msel.active = false; + v.body.nsel = 0; + v.scroll = positionIn(p.gpa, tf, @min(a.scroll, tf.content.len)).row; + } + } +}; + /// Where an offset into the old text is in the new, after the splice that /// kept `span` at either end: past it, moved by what it added; inside it, /// at its start, as acme moves a clone's q0 (textinsert, textdelete). diff --git a/src/daemon.zig b/src/daemon.zig index 0f770514..ac9e6eb4 100644 --- a/src/daemon.zig +++ b/src/daemon.zig @@ -530,21 +530,32 @@ fn hupOnStubDeath(on: bool) void { pub fn tellStubDetached() void { if (stub < 0) return; hupOnStubDeath(false); + tellDetached(stub); + stub = -1; +} + +/// The stub on `fd` told its terminal is detached (one of a session's +/// several, views.zig), and the pipe let go. +pub fn tellDetached(fd: c_int) void { + if (fd < 0) return; var buf: [160]u8 = undefined; const said = std.fmt.bufPrint(&buf, "d{s}", .{name()}) catch "d"; - _ = libc.write(stub, said.ptr, said.len); - _ = libc.close(stub); - stub = -1; + _ = libc.write(fd, said.ptr, said.len); + _ = libc.close(fd); } /// The session's end, `code` its exit status: the stub is told (and goes). pub fn tellStubExit(code: u8) void { - if (stub < 0) return; + tellExit(stub, code); + stub = -1; +} + +pub fn tellExit(fd: c_int, code: u8) void { + if (fd < 0) return; var buf: [8]u8 = undefined; const said = std.fmt.bufPrint(&buf, "x{d}", .{code}) catch "x1"; - _ = libc.write(stub, said.ptr, said.len); - _ = libc.close(stub); - stub = -1; + _ = libc.write(fd, said.ptr, said.len); + _ = libc.close(fd); } /// A terminal refused (an attach that came too late): its stub is told why. @@ -742,10 +753,7 @@ const Daemon = struct { _ = std.fmt.bufPrint(why, "no '{s}' is {s}={s}`", .{ wanted_name, other, wanted_name }) catch {}; return error.Said; } - if (!s.detached) { - _ = std.fmt.bufPrint(why, "no '{s}' is attached already, to another terminal or window", .{wanted_name}) catch {}; - return error.Said; - } + // Attached already: the session takes another view. return @intCast(i); }; _ = std.fmt.bufPrint(why, "no no session called '{s}'", .{wanted_name}) catch {}; @@ -764,6 +772,15 @@ const Daemon = struct { found = @intCast(i); }; if (count == 1) return found.?; + // None detached: the one session there is, as another view of it. + if (count == 0) { + for (&d.conns, 0..) |*s, i| if (s.fd >= 0 and s.kind == k) { + count += 1; + found = @intCast(i); + }; + if (count == 1) return found.?; + count = 0; + } if (count == 0 and others == 0) { w.writeAll("no no detached session is running") catch {}; return error.Said; diff --git a/src/gui/gui.zig b/src/gui/gui.zig index a1590b48..2d9124cc 100644 --- a/src/gui/gui.zig +++ b/src/gui/gui.zig @@ -1550,6 +1550,68 @@ const Gui = struct { deck: deck.Deck = .{}, }; +/// The Gui's fields that are each window's own, one a view (views.zig): +/// the window and its presenting, its scroll and glide, its scene and post +/// chain, its pointer, its blink, its images as presented, its capture. The +/// device, the atlas, the font and the pipelines are every window's. The +/// Gui holds the live window's; the others wait in `Shell.windows`. +const window_fields = [_][:0]const u8{ + "window", "soft_renderer", "soft_texture", "soft_tex_w", "soft_tex_h", + "window_w", "window_h", "scroll_pane", "scroll_rect", "scroll_body_y", + "scroll_body_h", "scroll_held_px", "scroll_col", "scroll_row", "scroll_delta", + "scroll_lag", "scroll_edge", "scroll_edge_len", "scroll_notch", "program_wheel", + "glide", "pace_ns", "sel_glow", "hover_glow", "frame_arrival_ns", + "blink_due", "cursor_quad_alpha", "scene_tex", "scene_tex_w", "scene_tex_h", + "scene_failures", "scene_target_failed", "post", "pointer_present", "pointer_shape", + "pointer_mapped", "pointer_cell", "presented_images", "prepared_images", "touch", + "live_ctrl", "live_alt", "applied_window_opacity", "capture_tex", "capture_tex_w", + "capture_tex_h", "capture_xfer", "capture_xfer_size", "captured", "capture_dir", +}; + +const GuiWindow = blk: { + @setEvalBranchQuota(100_000); + const st = @typeInfo(Gui).@"struct"; + var types: [window_fields.len]type = undefined; + var attrs: [window_fields.len]std.lang.Type.Struct.FieldAttributes = undefined; + for (window_fields, 0..) |name, i| { + const at = for (st.field_names, 0..) |n, j| { + if (std.mem.eql(u8, n, name)) break j; + } else @compileError("gui: Gui has no field " ++ name); + types[i] = st.field_types[at]; + attrs[i] = .{ .default_value_ptr = st.field_attrs[at].default_value_ptr }; + } + break :blk @Struct(.auto, null, &window_fields, &types, &attrs); +}; + +fn copyWindow(dst: anytype, src: anytype) void { + inline for (window_fields) |name| @field(dst, name) = @field(src, name); +} + +/// The live window's share of `g` as a window starts, but its SDL window +/// (reopenWindow makes that). +fn freshWindow(g: *Gui) void { + @setEvalBranchQuota(100_000); + const st = @typeInfo(Gui).@"struct"; + inline for (window_fields) |name| { + const at = comptime for (st.field_names, 0..) |n, j| { + if (std.mem.eql(u8, n, name)) break j; + } else unreachable; + if (comptime st.field_attrs[at].defaultValue(st.field_types[at])) |d| @field(g, name) = d; + } +} + +/// What a window held of its own let go (closeWindow lets the window go). +fn releaseWindowParts(g: *Gui, gpa: std.mem.Allocator) void { + if (g.scene_tex) |texture| c.SDL_ReleaseGPUTexture(g.device, texture); + if (g.capture_tex) |texture| c.SDL_ReleaseGPUTexture(g.device, texture); + if (g.capture_xfer) |transfer| c.SDL_ReleaseGPUTransferBuffer(g.device, transfer); + g.post.deinit(gpa, g.device); + g.glide.deinit(gpa); + gpa.free(g.scroll_edge); + g.presented_images.deinit(gpa); + g.prepared_images.deinit(gpa); +} + fn setGuiFontName(g: *Gui, fallback: []const u8) void { const name = if (c.ui_font_name(g.font)) |name_z| std.mem.span(name_z) else fallback; const len = @min(name.len, g.font_name.len); @@ -2247,7 +2309,9 @@ fn closeWindow(g: *Gui) void { g.soft_renderer = null; } else c.SDL_ReleaseWindowFromGPUDevice(g.device, g.window); c.SDL_DestroyWindow(g.window); - if (g.window_slot) |slot| slot.* = null; + if (g.window_slot) |slot| if (slot.* == g.window) { + slot.* = null; + }; } /// `pardes-gui --attach`: a window again, as the first was made, at the @@ -2399,6 +2463,14 @@ fn localSession( if (fs) |f| try f.watch(&queue, wakeFs); _ = c.SDL_StartTextInput(g.window); + // PARDES_TEST_VIEWS=N: N more windows from the start, each a view of the + // session (views.zig), captured under view-; view 0 live after. + if (test_mode) if (std.c.getenv("PARDES_TEST_VIEWS")) |n| { + for (0..std.fmt.parseInt(u8, std.mem.span(n), 10) catch 0) |_| shell.openAnotherWindow(); + core.toView(0); + shell.follow(); + }; + defer shell.closeOtherWindows(); if (test_mode) setStdinRaw() catch {}; shell.threads_ok = true; @@ -2431,6 +2503,10 @@ fn localSession( }; pipe_tasks.cancelAll(io); queue.discardCompletions(); + // ponytail: a restored session is one view, the live window's: + // the others close. + shell.closeOtherWindows(); + shell.gview = 0; shell.stopPtys(); for (0..pardes.MAX_PANES) |wid| file_watch.watchPane( inotify_fd, @@ -2635,6 +2711,24 @@ const StdinFeed = struct { // A mouse event at window pixels, built as SDL would and run // through dispatch, so a driver exercises the real pointer path // (pixel to cell, tag and body hits). SGR mouse carries only cells. + // ESC]777;view;BEL: what follows is window k's (k from 1), + // as an event of that window's makes its view live (route). + const view_prefix = "\x1b]777;view;"; + if (std.mem.startsWith(u8, f.buf[seq_start..len], view_prefix)) { + const body = f.buf[seq_start + view_prefix.len .. len]; + const end = std.mem.indexOfScalar(u8, body, 0x07) orelse { + std.mem.copyForwards(u8, f.buf[0 .. len - seq_start], f.buf[seq_start..len]); + f.fill = len - seq_start; + break; + }; + const k = std.fmt.parseInt(u8, body[0..end], 10) catch 0; + if (in.shell) |sh| if (k >= 1 and pardes.views.isOpen(sh.core, k - 1)) { + sh.core.toView(k - 1); + sh.follow(); + }; + seq_start += view_prefix.len + end + 1; + continue; + } const mouse_prefix = "\x1b]777;mouse;"; if (std.mem.startsWith(u8, f.buf[seq_start..len], mouse_prefix)) { const body = f.buf[seq_start + mouse_prefix.len .. len]; @@ -2975,6 +3069,9 @@ test "the window's function and editing keys reach a terminal as its cursor keys const Input = struct { core: ?*pardes.Pardes = null, + /// The shell whose windows these are: a window closed while others + /// stay takes its view with it (views.zig). + shell: ?*Shell = null, quit: bool = false, fn post(in: *Input, ev: pardes.Event) void { @@ -2982,6 +3079,7 @@ const Input = struct { } fn close(in: *Input) void { + if (in.shell) |s| if (pardes.views.count(s.core) > 1) return s.closeView(s.core.view); if (in.core) |core| core.quit = true; in.quit = true; } @@ -3018,7 +3116,11 @@ fn dispatch(g: *Gui, in: *Input, sev: *const c.SDL_Event) void { else => {}, } switch (sev.type) { - c.SDL_EVENT_QUIT, c.SDL_EVENT_WINDOW_CLOSE_REQUESTED => in.close(), + c.SDL_EVENT_QUIT => { + in.shell = null; + in.close(); + }, + c.SDL_EVENT_WINDOW_CLOSE_REQUESTED => in.close(), c.SDL_EVENT_WINDOW_FOCUS_GAINED => in.post(.{ .focus = true }), c.SDL_EVENT_WINDOW_FOCUS_LOST => in.post(.{ .focus = false }), c.SDL_EVENT_WINDOW_MOUSE_LEAVE => { @@ -3677,6 +3779,10 @@ const Shell = struct { /// The tick a paced frame is sampled at (waitInput): the clock the /// core is advanced to while anything moves. paced_ns: ?u64 = null, + /// Each other view's window, while another's is in the Gui (views.zig), + /// and the view whose window the Gui holds (`follow`). + windows: [pardes.views.max]GuiWindow = undefined, + gview: u8 = 0, fn reap(s: *Shell) void { const now = std.Io.Clock.awake.now(s.queue.io).toMilliseconds(); @@ -3766,10 +3872,92 @@ const Shell = struct { fn detachWindow(ctx: ?*anyopaque) void { const s = shellOf(ctx); if (s.gui == null) return s.core.reportFailure(s.core.active, "Detach: no window is open"); + // One of several: its window goes, and the session stays in the rest. + if (pardes.views.count(s.core) > 1) return s.closeView(s.core.view); if (!daemon.kept()) return s.core.reportFailure(s.core.active, "Detach: no pardes daemon keeps this session; closing its window ends it"); s.closeTheWindow(); } + /// The Gui holds the window of the core's live view. + fn follow(s: *Shell) void { + const g = s.gui orelse return; + const k = s.core.view; + if (s.gview == k) return; + if (s.gview < s.windows.len) copyWindow(&s.windows[s.gview], g); + copyWindow(g, &s.windows[k]); + s.gview = k; + } + + /// An event of another view's window: that view goes live. + fn route(s: *Shell, sev: *const c.SDL_Event) void { + if (!pardes.views.several(s.core)) return; + const g = s.gui orelse return; + const w = c.SDL_GetWindowFromEvent(sev) orelse return; + if (w == g.window) return; + for (s.windows, 0..) |win, k| if (k != s.gview and pardes.views.isOpen(s.core, @intCast(k)) and win.window == w) { + s.core.toView(@intCast(k)); + return s.follow(); + }; + } + + /// `pardes-gui --attach` while a window is open: another window, a view + /// of its own (views.zig), opened as the first was. + fn openAnotherWindow(s: *Shell) void { + const g = s.gui orelse return; + s.follow(); + const k = s.core.openView() orelse return log.err("--attach: this session has as many windows as it takes", .{}); + const live = s.gview; + copyWindow(&s.windows[live], g); + freshWindow(g); + if (!reopenWindow(g)) { + log.err("--attach: no window: {s}", .{c.SDL_GetError()}); + copyWindow(g, &s.windows[live]); + return s.core.closeView(k); + } + if (s.test_mode) { + // Its own grid, its captures under view- (PARDES_TEST_VIEWS). + // ponytail: the path is kept for the session, a few bytes. + const cols: u32 = if (std.c.getenv("PARDES_TEST_COLS")) |v| std.fmt.parseInt(u16, std.mem.span(v), 10) catch 80 else 80; + const rows: u32 = if (std.c.getenv("PARDES_TEST_ROWS")) |v| std.fmt.parseInt(u16, std.mem.span(v), 10) catch 24 else 24; + _ = c.SDL_SetWindowSize(g.window, @intCast(cols * g.cell_w + g.capture_pad), @intCast(rows * g.cell_h + g.capture_pad)); + _ = c.SDL_SyncWindow(g.window); + g.capture = true; + g.capture_dir = std.fmt.allocPrint(s.gpa, "{s}/view-{d}", .{ s.windows[live].capture_dir, k }) catch ""; + if (g.capture_dir.len > 0) std.Io.Dir.cwd().createDirPath(s.io, g.capture_dir) catch {}; + } + s.gview = k; + s.core.toView(k); + s.core.needs_frame = true; + } + + /// View `k`'s window closed while others stay: the view goes with it. + fn closeView(s: *Shell, k: u8) void { + const g = s.gui orelse return; + s.core.toView(k); + s.follow(); + closeWindow(g); + releaseWindowParts(g, s.gpa); + s.gview = std.math.maxInt(u8); + s.core.closeView(k); + s.follow(); + s.core.needs_frame = true; + } + + /// The session's end: every window but the live one closed. + fn closeOtherWindows(s: *Shell) void { + const g = s.gui orelse return; + for (0..s.windows.len) |i| { + const k: u8 = @intCast(i); + if (k == s.gview or !pardes.views.isOpen(s.core, k)) continue; + var live: GuiWindow = undefined; + copyWindow(&live, g); + copyWindow(g, &s.windows[k]); + closeWindow(g); + releaseWindowParts(g, s.gpa); + copyWindow(g, &live); + } + } + fn closeTheWindow(s: *Shell) void { const g = s.gui orelse return; closeWindow(g); @@ -3975,7 +4163,7 @@ const Shell = struct { .files_changed => check_files = true, .fs_ready => {}, .window => { - s.openTheWindow(); + if (s.gui != null) s.openAnotherWindow() else s.openTheWindow(); s.saw_event = true; }, .adopted => |a| { @@ -4008,7 +4196,8 @@ fn shellOf(ctx: ?*anyopaque) *Shell { fn waitInput(ctx: ?*anyopaque, timeout_ms: u32) void { const s = shellOf(ctx); const core = s.core; - var in: Input = .{ .core = core }; + var in: Input = .{ .core = core, .shell = s }; + s.follow(); if (s.gui) |g| { var sev = std.mem.zeroes(c.SDL_Event); // Idle is asleep until an event: SDL's own, and the user events the @@ -4100,16 +4289,27 @@ fn waitInput(ctx: ?*anyopaque, timeout_ms: u32) void { // each would repeat hover and selection work the next undoes. var motion: ?c.SDL_Event = null; while (true) { - if (sev.type == c.SDL_EVENT_MOUSE_MOTION) { + if (sev.type == c.SDL_EVENT_MOUSE_MOTION and (motion == null or motion.?.motion.windowID == sev.motion.windowID)) { motion = sev; } else { - if (motion) |*m| dispatch(g, &in, m); + if (motion) |*m| { + s.route(m); + dispatch(g, &in, m); + } motion = null; - dispatch(g, &in, &sev); + if (sev.type == c.SDL_EVENT_MOUSE_MOTION) { + motion = sev; + } else { + s.route(&sev); + dispatch(g, &in, &sev); + } } if (!c.SDL_PollEvent(&sev)) break; } - if (motion) |*m| dispatch(g, &in, m); + if (motion) |*m| { + s.route(m); + dispatch(g, &in, m); + } } runDemo(g, &in, core); // Paced: what came before the tick is in; wait out the rest of it, @@ -4133,8 +4333,12 @@ fn waitInput(ctx: ?*anyopaque, timeout_ms: u32) void { const more = c.SDL_PollEvent(&sev); pardes.turn.wake(); if (more) { + s.route(&sev); dispatch(g, &in, &sev); - while (c.SDL_PollEvent(&sev)) dispatch(g, &in, &sev); + while (c.SDL_PollEvent(&sev)) { + s.route(&sev); + dispatch(g, &in, &sev); + } } else { // The events asked for every 100 µs, the core busy between. const until = c.SDL_GetTicksNS() + 100 * std.time.ns_per_us; @@ -4182,6 +4386,7 @@ fn waitDetached(core: *pardes.Pardes, timeout_ms: u32) void { fn pollFrame(ctx: ?*anyopaque) void { const s = shellOf(ctx); + s.follow(); s.reconcilePtys(); const core = s.core; if (s.fs) |f| if (f.tick().pending) s.queue.push(.fs_ready); @@ -4414,6 +4619,7 @@ test "background opacity acknowledges success and rolls back unsupported surface fn present(ctx: ?*anyopaque, surface: *const pardes.Surface) void { const s = shellOf(ctx); const core = s.core; + s.follow(); const g = s.gui orelse return; frame_started_ns = monoNs(); const frame = @constCast(surface); @@ -4447,6 +4653,7 @@ fn present(ctx: ?*anyopaque, surface: *const pardes.Surface) void { fn postPresent(ctx: ?*anyopaque) void { const s = shellOf(ctx); + s.follow(); const g = s.gui orelse return; // No image (a minimized window) shows no tracks; time still passes and // the pump advances it all the same. diff --git a/src/panes.zig b/src/panes.zig index a782e3e0..71a04464 100644 --- a/src/panes.zig +++ b/src/panes.zig @@ -1,6 +1,7 @@ const std = @import("std"); const vaxis = @import("vaxis"); const pardes = @import("pardes.zig"); +const views = @import("views.zig"); const layout = @import("layout.zig"); const config = @import("config.zig"); const Pardes = pardes.Pardes; @@ -26,8 +27,9 @@ pub const EditText = struct { text: []u8, row0: i32 }; pub const Pane = struct { /// What the replay digest leaves out (src/walk.zig): the view's geometry, /// made again from the rest (File.geometry); the 9P server's record of - /// the pane's opens and readers. + /// the pane's opens and readers; the other views' share of it (views.zig). pub const replica = .{ + .local = .{ "pv", "pv_known" }, .derived = .{ "wrap_line", "wrap_col", "wrap_pad", "wrap_n", "body_wrapped", "context_rows", "body_rows", "body_visible_rows", "context_row_limit", "geometry_key", "geometry_len", "view", "notices" }, .authority = .{"fs"}, .len = .{ .{ "msg", "msg_len" }, .{ "msg_older", "msg_older_len" }, .{ "question", "question_len" }, .{ "shell_tail", "shell_tail_len" }, .{ "sel_snap", "nsel_snap" } }, @@ -95,6 +97,7 @@ pub const Pane = struct { pub const Cwd = union(enum) { none, inherited: *Pane, owned: []u8 }; pub const Focus = enum { body, tag }; + pub const SessionView = struct { wrap: bool = true, layered: bool = true, body_h: u32 = 0, tagline_h: u32 = 0, rect_h: u16 = 0 }; terminal: ?*terminal.State = null, gpa: std.mem.Allocator, serial: u32 = 0, @@ -263,7 +266,7 @@ pub const Pane = struct { geometry_key: u64 = 0, geometry_len: usize = 0, /// The session's part of the geometry, as of the last step (`sync`). - view: struct { wrap: bool = true, layered: bool = true, body_h: u32 = 0, tagline_h: u32 = 0, rect_h: u16 = 0 } = .{}, + view: SessionView = .{}, sel: [3]Sel = @splat(.{}), pointer_selections: [3]?PointerSelection = @splat(null), next_pointer_selection: u64 = 0, @@ -297,6 +300,15 @@ pub const Pane = struct { /// body layer reserves exactly `len` rows for them, the way it already /// reserves rows for sticky context headers. notices: Notices = .{}, + /// Each other view's share of this pane, while it is not live + /// (views.zig): its cursor, scroll and place in that view's layout. + pv: [views.max]views.PaneView = undefined, + /// The views `pv` holds a share for (a pane retired before a view came + /// has none of it). + pv_known: u8 = 0, + /// The view that last typed into it: a terminal shown at two sizes takes + /// that view's (Pardes.sync). + sized_by: u8 = 0, pub const MessagePhase = enum { entering, shown, lingering, leaving }; /// One message line's animation: falling in, shown, lingering after the diff --git a/src/pardes.zig b/src/pardes.zig index 228d97fa..88d75fd1 100644 --- a/src/pardes.zig +++ b/src/pardes.zig @@ -64,6 +64,7 @@ pub const memory = @import("memory.zig"); pub const tape = @import("tape.zig"); pub const walk = @import("walk.zig"); pub const screen_state = if (terminal_panes) @import("screen_state.zig") else struct {}; +pub const views = @import("views.zig"); pub const image = @import("image.zig"); pub const dump = @import("dump.zig"); pub const lsp = @import("lsp/lsp.zig"); @@ -511,6 +512,7 @@ test { _ = @import("Handover.zig"); _ = @import("walk.zig"); _ = screen_state; + _ = @import("views.zig"); } pub const Builtin = builtins.registry.Builtin(); @@ -4699,6 +4701,12 @@ pub const Pardes = struct { restore_buf: [1024]u8 = undefined, surface: Surface = .{}, + /// The view the fields above are of: whose layout, focus, pointer and + /// panes' cursors and scroll (views.zig). `views_open` says which views + /// there are, one a window or terminal; each not live is in `parked`. + view: u8 = 0, + views_open: u8 = 1, + parked: [views.max]?*views.Parked = @splat(null), /// per-update scratch (paneCursorLines, selection text); reset each update scratch: std.heap.ArenaAllocator, /// The arena each frame's Surface text is built in, reset by `pump`. Hosts @@ -4725,6 +4733,9 @@ pub const Pardes = struct { // Drained by every pump before its digest; a replica's queues // are its own. "effects", "in_q", + // The views not live: written whenever one goes out, so the + // live one's slot is stale (views.zig). + "parked", }, .derived = .{ "shell_rows", "diff_output", "locations_cache", "retired_panes", "tty_filter_palette", "body_chrome" }, // The host's handles, the 9P server's fids and log, the store file @@ -5020,6 +5031,7 @@ pub const Pardes = struct { slot.* = null; }; const gpa = p.gpa; + views.deinit(p); for (&p.col_tags) |*t| t.deinit(gpa); p.global_tag.deinit(gpa); p.registers.deinit(gpa); @@ -5167,6 +5179,7 @@ pub const Pardes = struct { if (comptime pdf_enabled) if (pane.pdf) |*pv| panes.Pdf.deinitPane(p, pane, pv); if (pane.file) |*file| panes.File.deinit(p, pane, file); if (pane.ovl) |o| p.gpa.free(o.text); + views.tornDown(p, pane); pane.body.deinit(p.gpa); pane.tag.deinit(p.gpa); pane.input.deinit(p.gpa); @@ -5201,6 +5214,7 @@ pub const Pardes = struct { pane.serial = p.next_serial; p.rects[id] = .{}; p.panes[id] = pane; + views.installed(p, pane); panes.terminal.adoptSettings(p, pane); // /log hears of it once the update that made it ends, when it has // its file, name and place (`ctlfs.events.announce`). @@ -5569,6 +5583,33 @@ pub const Pardes = struct { /// Something changed the core without going through `update` -- a watched /// file reloading under it, a theme file rewritten on disk. `pump` draws /// only when it has a reason to, and this is the reason. + /// A view of its own for a new client (a second window, a terminal + /// `pardes --attach`ed), as the live one is laid out; null at + /// views.max. Not live until `toView`. + pub fn openView(p: *Pardes) ?u8 { + const taped = tape.enter(p, .open_view); + defer tape.leave(p, taped); + p.needs_frame = true; + return views.open(p); + } + + /// The client of view `k` gone: its view goes, the others stay. The + /// last one stays the session's own. + pub fn closeView(p: *Pardes, k: u8) void { + const taped = tape.enter(p, .{ .close_view = k }); + defer tape.leave(p, taped); + views.close(p, k); + p.sync(); + } + + /// Into view `k`, before the input its client sent. + pub fn toView(p: *Pardes, k: u8) void { + if (k == p.view) return; + const taped = tape.enter(p, .{ .to_view = k }); + defer tape.leave(p, taped); + views.to(p, k); + } + pub fn invalidate(p: *Pardes) void { const taped = tape.enter(p, .invalidate); defer tape.leave(p, taped); @@ -6053,6 +6094,8 @@ pub const Pardes = struct { // A pty streaming into an unwatched session spent most of its time // drawing what nobody looked at. if (v.watched) |f| if (!f(h.ctx)) return; + // The other views first, each into its own window or terminal. + if (views.several(p)) try views.drawOthers(p, h); _ = p.frame_arena.reset(.{ .retain_with_limit = memory.limits.frame_kept }); p.frames.mark_update = FrameStats.nowNs(); p.frames.view = if (p.panes[p.active]) |pane| pane.scroll() else 0; @@ -6235,6 +6278,7 @@ pub const Pardes = struct { .key => |key| { p.can_ask = true; defer p.can_ask = false; + p.typedInto(); p.handleKey(key); }, .mouse => |m| { @@ -6254,7 +6298,10 @@ pub const Pardes = struct { } } }, - .paste => |bytes| edit.applyPaste(p, bytes), + .paste => |bytes| { + p.typedInto(); + edit.applyPaste(p, bytes); + }, .command => |line| _ = p.executeBuiltinLine(p.active, line), .pinch => |scale| p.ov_pinch_scale = scale, .touch_scroll => |delta| p.ov_touch_scroll_delta = delta, @@ -7672,6 +7719,7 @@ pub const Pardes = struct { tagline.exitHeader(p); } p.reapPanes(); + if (views.several(p)) views.show(p); layout.compute(p); p.presentation.sync(p); p.trackJump(); @@ -7759,7 +7807,9 @@ pub const Pardes = struct { // A terminal's pty follows its pane down to a row, whatever the // resize; only a folded one keeps the size it had, and one not // placed yet the grid it was made with. - if ((cols != pane.cols or rows != pane.rows) and (r.h > pane.tag_rows or (pane.isTerminal() and !pane.collapsed and r.h > 0))) { + if ((cols != pane.cols or rows != pane.rows) and (r.h > pane.tag_rows or (pane.isTerminal() and !pane.collapsed and r.h > 0)) and + (!pane.isTerminal() or p.sizes(pane))) + { // doc panes have no pty/emulator grid to reflow; just record // the size so bodyText renders the right number of rows if (pane.isTerminal()) { @@ -7798,6 +7848,22 @@ pub const Pardes = struct { tagline.revealHeader(p); } + /// The view typing goes into is the one its terminal is sized for + /// (`sizes`). + fn typedInto(p: *Pardes) void { + if (p.panes[p.active]) |pane| pane.sized_by = p.view; + } + + /// Whether the live view sizes terminal `pane`: a terminal shown at two + /// sizes takes the size of the view that typed into it last, and any + /// view's once that one is gone. The others show it at that size. + fn sizes(p: *Pardes, pane: *Pane) bool { + if (pane.sized_by == p.view) return true; + if (views.isOpen(p, pane.sized_by)) return false; + pane.sized_by = p.view; + return true; + } + pub fn theme(p: *const Pardes) *const Theme { if (p.custom_theme) |*custom| return custom; return &themes[p.settings.theme]; diff --git a/src/tape.zig b/src/tape.zig index eaf1a9fa..ec38deac 100644 --- a/src/tape.zig +++ b/src/tape.zig @@ -57,6 +57,9 @@ pub const Input = union(enum) { acknowledge_panel_presentation: []const pardes.animation.Track, run_closed_lines, file_deleted: u8, + open_view, + close_view: u8, + to_view: u8, }; /// The core's own syscalls (each `turn.yield()`s around itself), numbered @@ -442,6 +445,9 @@ pub const Tape = struct { .acknowledge_panel_presentation => |tracks| p.acknowledgePanelPresentation(tracks), .run_closed_lines => pardes.ctlfs.runClosedLines(p), .file_deleted => |id| pardes.panes.File.deleted(p, id), + .open_view => _ = p.openView(), + .close_view => |k| p.closeView(k), + .to_view => |k| p.toView(k), } t.finishInput(); } diff --git a/src/tty/tty.zig b/src/tty/tty.zig index 760c0692..f949d571 100644 --- a/src/tty/tty.zig +++ b/src/tty/tty.zig @@ -21,15 +21,45 @@ extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: c_int const TIOCSWINSZ: c_int = @bitCast(@as(u32, if (@hasDecl(posix.T, "IOCSWINSZ")) posix.T.IOCSWINSZ else 0x80087467)); -/// The terminal answered the shared-memory probe (`kittyShmProbe`): it can -/// read a raster from a POSIX shm object, so a page costs a name on the wire -/// instead of its whole base64 body. Set by the input reader, which sees the -/// reply before vaxis' parser swallows every graphics APC alike. -var kitty_shm = std.atomic.Value(bool).init(false); -/// Only a reply that arrives while the probe is out counts. The id is one -/// vaxis' own image numbering (1, 2, ...) never reaches, because a terminal -/// answers every transmission `i=;OK` and one of those must not pass. -var kitty_shm_probing = std.atomic.Value(bool).init(false); +/// What a terminal's input reader keeps, one a terminal (views.zig): its +/// graphemes, the frame it waits on the answer to, the shm probe, and the +/// view its input is. +const Feed = struct { + view: u8 = 0, + cache: vaxis.GraphemeCache = .{}, + ack: FrameAck = .{}, + /// The terminal answered the shared-memory probe (`kittyShmProbe`): it + /// can read a raster from a POSIX shm object, so a page costs a name on + /// the wire instead of its whole base64 body. Set by the input reader, + /// which sees the reply before vaxis' parser swallows every graphics APC + /// alike. + shm: std.atomic.Value(bool) = .init(false), + /// Only a reply that arrives while the probe is out counts. The id is + /// one vaxis' own image numbering (1, 2, ...) never reaches, because a + /// terminal answers every transmission `i=;OK` and one of those must + /// not pass. + shm_probing: std.atomic.Value(bool) = .init(false), +}; + +/// A terminal's input onto the loop's queue, as its view's: a `from` goes +/// ahead of it whenever another terminal posted last. With one terminal +/// none ever does. +const Poster = struct { + loop: *Loop, + view: u8, + var lock: std.Io.Mutex = .init; + var last: u8 = 0; + + pub fn postEvent(p: Poster, ev: @TypeOf(Command.value)) !void { + lock.lockUncancelable(p.loop.io); + defer lock.unlock(p.loop.io); + if (last != p.view) { + try p.loop.postEvent(.{ .from = p.view }); + last = p.view; + } + try p.loop.postEvent(ev); + } +}; const kitty_shm_probe_id = 0x7fff_fff1; var kitty_shm_serial: u32 = 0; /// Transmissions an unread shm segment outlives before it is removed. @@ -62,13 +92,13 @@ fn kittyShmWrite(name: [:0]const u8, bytes: []const u8) bool { /// Ask with a 1x1 RGB image in shm whether the terminal reads that medium. A /// remote terminal cannot see our shm and answers an error (or nothing). -fn kittyShmProbe(w: *std.Io.Writer) void { +fn kittyShmProbe(w: *std.Io.Writer, feed: *Feed) void { var buf: [64]u8 = undefined; const name = kittyShmName(&buf, 0) catch return; if (!kittyShmWrite(name, &.{ 0, 0, 0 })) return; var b64: [64]u8 = undefined; const enc = std.base64.standard.Encoder.encode(&b64, name); - kitty_shm_probing.store(true, .release); + feed.shm_probing.store(true, .release); w.print("\x1b_Gi={d},s=1,v=1,a=q,t=s,f=24;{s}\x1b\\", .{ kitty_shm_probe_id, enc }) catch return; w.flush() catch {}; } @@ -86,6 +116,12 @@ pub const Command = struct { adopted: struct { pid: posix.pid_t, fd: posix.fd_t }, /// A timed wait ran out: the core's next wake has come. wake, + /// The input after this is the terminal of this view's (Poster). + from: u8, + /// A SIGWINCH: a terminal changed size (Shell.winched). + winched, + /// A SIGHUP: the session's own terminal hung up. + hangup, key_press: vaxis.Key, pty_read: struct { id: usize, gen: u32, bytes: []u8 }, pty_eof: struct { id: usize, gen: u32 }, @@ -107,36 +143,36 @@ pub const Command = struct { }; const Loop = vaxis.Loop(@TypeOf(Command.value)); -fn startInput(loop: *Loop, cache: *vaxis.GraphemeCache) !void { +fn startInput(loop: *Loop, sc: *Screen) !void { if (comptime builtin.os.tag == .windows) return loop.start(); - if (loop.thread != null) return; - loop.thread = try loop.io.concurrent(inputThread, .{ loop, cache }); + if (sc.thread != null) return; + sc.thread = try loop.io.concurrent(inputThread, .{ loop, sc }); } -fn stopInput(loop: *Loop) void { - if (comptime builtin.os.tag == .windows) return loop.stop(); - if (loop.thread) |*thread| { - thread.cancel(loop.io); - loop.thread = null; +fn stopInput(io: std.Io, thread: *?std.Io.Future(void)) void { + if (thread.*) |*t| { + t.cancel(io); + thread.* = null; } } -fn inputThread(loop: *Loop, cache: *vaxis.GraphemeCache) void { +fn inputThread(loop: *Loop, sc: *Screen) void { // vaxis's test terminal reads nothing off Linux: it has no read at all. if (comptime !@hasDecl(vaxis.Tty, "read")) return; - inputReader(loop, loop.tty, cache); + inputReader(loop, &sc.tty, &sc.vx, &sc.feed); } -fn inputReader(loop: *Loop, tty: anytype, cache: *vaxis.GraphemeCache) void { - readInput(loop, tty, cache) catch |err| { +fn inputReader(loop: *Loop, tty: anytype, vx: *vaxis.Vaxis, feed: *Feed) void { + readInput(loop, tty, vx, feed) catch |err| { if (err == error.Canceled) return; std.log.err("terminal input: {s}", .{@errorName(err)}); }; - loop.postEvent(.quit) catch {}; + (Poster{ .loop = loop, .view = feed.view }).postEvent(.quit) catch {}; } -fn readInput(loop: *Loop, tty: anytype, cache: *vaxis.GraphemeCache) !void { - try loop.postEvent(.{ .winsize = try tty.getWinsize() }); +fn readInput(loop: *Loop, tty: anytype, vx: *vaxis.Vaxis, feed: *Feed) !void { + const post: Poster = .{ .loop = loop, .view = feed.view }; + try post.postEvent(.{ .winsize = try tty.getWinsize() }); var parser: vaxis.Parser = .{}; var buf: [1024]u8 = undefined; var carried: usize = 0; @@ -147,14 +183,14 @@ fn readInput(loop: *Loop, tty: anytype, cache: *vaxis.GraphemeCache) !void { // The terminal's answers to frames (FrameAck) come out first, // wherever they sit: never events, and never glued to a key (an // Escape pressed just before one would read as Alt on it). - const end = FrameAck.take(buf[0 .. carried + received], loop); + const end = feed.ack.take(buf[0 .. carried + received], loop); if (end == 0) { carried = 0; continue; } - if (kitty_shm_probing.load(.acquire)) if (std.mem.indexOf(u8, buf[0..end], std.fmt.comptimePrint("\x1b_Gi={d};", .{kitty_shm_probe_id}))) |at| { - kitty_shm_probing.store(false, .release); - kitty_shm.store(std.mem.startsWith(u8, buf[at..end], std.fmt.comptimePrint("\x1b_Gi={d};OK", .{kitty_shm_probe_id})), .release); + if (feed.shm_probing.load(.acquire)) if (std.mem.indexOf(u8, buf[0..end], std.fmt.comptimePrint("\x1b_Gi={d};", .{kitty_shm_probe_id}))) |at| { + feed.shm_probing.store(false, .release); + feed.shm.store(std.mem.startsWith(u8, buf[at..end], std.fmt.comptimePrint("\x1b_Gi={d};OK", .{kitty_shm_probe_id})), .release); // Answered either way: a terminal that could not read it left it. var name_buf: [64]u8 = undefined; if (kittyShmName(&name_buf, 0)) |name| _ = libc.shm_unlink(name) else |_| {} @@ -187,9 +223,9 @@ fn readInput(loop: *Loop, tty: anytype, cache: *vaxis.GraphemeCache) !void { std.mem.startsWith(u8, FrameAck.answer, rest)) break; if (!waited and parse_end == end and rest[0] == 0x1b and (rest.len == 1 or (rest.len == 2 and rest[1] == 0x1b))) { waited = true; - if (try inputFollows(tty, loop.io, if (loop.vaxis.caps.kitty_keyboard) 1000 else 50)) break; + if (try inputFollows(tty, loop.io, if (vx.caps.kitty_keyboard) 1000 else 50)) break; } - const result = try parser.parse(buf[consumed..parse_end], loop.vaxis.opts.system_clipboard_allocator); + const result = try parser.parse(buf[consumed..parse_end], vx.opts.system_clipboard_allocator); if (result.n == 0) break; consumed += result.n; // A modifier pressed on its own, which the kitty keyboard @@ -197,8 +233,8 @@ fn readInput(loop: *Loop, tty: anytype, cache: *vaxis.GraphemeCache) !void { // chord, and a shell pane was typed its private-use code. if (result.event) |event| if (event == .key_press and event.key_press.isModifier()) continue; if (result.event) |event| - vaxis.loop.handleEventGeneric(loop, loop.vaxis, cache, @TypeOf(Command.value), event, loop.vaxis.opts.system_clipboard_allocator) catch |err| { - if (event == .paste) if (loop.vaxis.opts.system_clipboard_allocator) |gpa| gpa.free(@constCast(event.paste)); + vaxis.loop.handleEventGeneric(post, vx, &feed.cache, @TypeOf(Command.value), event, vx.opts.system_clipboard_allocator) catch |err| { + if (event == .paste) if (vx.opts.system_clipboard_allocator) |gpa| gpa.free(@constCast(event.paste)); return err; }; // The last answer to the startup queries is no event of its @@ -226,14 +262,14 @@ const FrameAck = struct { const request = "\x1b[5n"; const answer = "\x1b[0n"; const timeout_ns: u64 = std.time.ns_per_s; - var waiting = std.atomic.Value(bool).init(false); - var off = std.atomic.Value(bool).init(false); + waiting: std.atomic.Value(bool) = .init(false), + off: std.atomic.Value(bool) = .init(false), /// When the request went out (the loop's thread only). - var sent_ns: u64 = 0; + sent_ns: u64 = 0, /// Take every answer out of `bytes`, closing up the rest; the new /// length. The loop is woken for the frame that waited on one. - fn take(bytes: []u8, loop: *Loop) usize { + fn take(a: *FrameAck, bytes: []u8, loop: *Loop) usize { var len = bytes.len; var found = false; while (std.mem.indexOf(u8, bytes[0..len], answer)) |at| { @@ -241,18 +277,18 @@ const FrameAck = struct { len -= answer.len; found = true; } - if (found and waiting.swap(false, .acq_rel)) loop.postEvent(.nop) catch {}; + if (found and a.waiting.swap(false, .acq_rel)) loop.postEvent(.nop) catch {}; return len; } /// Whether a new frame must wait (and the loop has a wake at the /// timeout); past the timeout the terminal is taken to never answer. - fn holds(now_ns: u64, io: std.Io) bool { - if (off.load(.acquire) or !waiting.load(.acquire)) return false; - const waited_ns = now_ns -| sent_ns; + fn holds(a: *FrameAck, now_ns: u64, io: std.Io) bool { + if (a.off.load(.acquire) or !a.waiting.load(.acquire)) return false; + const waited_ns = now_ns -| a.sent_ns; if (waited_ns >= timeout_ns) { - off.store(true, .release); - waiting.store(false, .release); + a.off.store(true, .release); + a.waiting.store(false, .release); return false; } requestWake(io, @intCast((timeout_ns - waited_ns + std.time.ns_per_ms - 1) / std.time.ns_per_ms)); @@ -260,14 +296,14 @@ const FrameAck = struct { } /// After a frame that wrote something: ask for the answer. - fn ask(w: *std.Io.Writer, now_ns: u64) void { - if (off.load(.acquire)) return; + fn ask(a: *FrameAck, w: *std.Io.Writer, now_ns: u64) void { + if (a.off.load(.acquire)) return; // Waiting before the request goes: a local terminal can answer // before the write returns, and an answer nobody waited for is lost. - sent_ns = now_ns; - waiting.store(true, .release); - w.writeAll(request) catch return waiting.store(false, .release); - w.flush() catch return waiting.store(false, .release); + a.sent_ns = now_ns; + a.waiting.store(true, .release); + w.writeAll(request) catch return a.waiting.store(false, .release); + w.flush() catch return a.waiting.store(false, .release); } }; @@ -299,7 +335,7 @@ test "terminal input preserves fragmented keys queries paste and text after EOF" vx.queries_done.store(false, .unordered); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; const Reader = struct { parts: []const []const u8, next: usize = 0, @@ -321,7 +357,7 @@ test "terminal input preserves fragmented keys queries paste and text after EOF" "plain \x1b[?62;", "4c\x1b[", "A\x1b[200", "~caf\xc3", "\xa9 \xe7", "\x95", "\x8c \xf0\x9f", "\x98\x80", "\x1b[201", "~\x1b]52;c;Y2", "xpcA==\x07tail", "\x1b", } }; - try readInput(&loop, &reader, &cache); + try readInput(&loop, &reader, &vx, &feed); try std.testing.expect(vx.queries_done.load(.unordered)); var text: std.ArrayList(u8) = .empty; defer text.deinit(gpa); @@ -384,7 +420,7 @@ test "terminal replies and malformed sequences never become typed text" { vx.queries_done.store(false, .unordered); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; const Reader = struct { parts: []const []const u8, next: usize = 0, @@ -427,7 +463,7 @@ test "terminal replies and malformed sequences never become typed text" { "c", }, }; - try readInput(&loop, &reader, &cache); + try readInput(&loop, &reader, &vx, &feed); try std.testing.expect(vx.queries_done.load(.unordered)); try std.testing.expect(vx.caps.kitty_graphics); var text: std.ArrayList(u8) = .empty; @@ -466,7 +502,7 @@ test "terminal input split at every byte with a gap reads as it does whole" { defer vx.deinit(gpa, &output.writer); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; // Two reads with `gap_ms` between them, as an ssh link delivers a // sequence cut across two packets. const Reader = struct { @@ -531,7 +567,7 @@ test "terminal input split at every byte with a gap reads as it does whole" { const all = try std.mem.concat(gpa, u8, &.{ "x", case.bytes, "y" }); defer gpa.free(all); var one: Reader = .{ .parts = .{ all, "" }, .gap_ms = 0 }; - try readInput(&loop, &one, &cache); + try readInput(&loop, &one, &vx, &feed); try Reader.events(&loop, &whole); for (1..all.len) |cut| { // A cut just after an ESC is the one that has to wait, and a @@ -539,7 +575,7 @@ test "terminal input split at every byte with a gap reads as it does whole" { // anywhere else is carried however long the gap. const after_esc = all[cut - 1] == 0x1b; var two: Reader = .{ .parts = .{ all[0..cut], all[cut..] }, .gap_ms = if (after_esc and !case.kitty) 40 else 60 }; - try readInput(&loop, &two, &cache); + try readInput(&loop, &two, &vx, &feed); try Reader.events(&loop, &split); std.testing.expectEqualStrings(whole.items, split.items) catch |err| { std.debug.print("case {f} cut at {d}\n", .{ std.ascii.hexEscape(case.bytes, .lower), cut }); @@ -562,7 +598,7 @@ test "an ESC read by itself is the Escape key at once" { defer vx.deinit(gpa, &output.writer); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; // More always follows at once: a reader that waited on the ESC would // take `j` for the rest of its sequence (Alt-j). const Reader = struct { @@ -587,7 +623,7 @@ test "an ESC read by itself is the Escape key at once" { for ([_]bool{ false, true }) |kitty| { vx.caps.kitty_keyboard = kitty; var reader: Reader = .{ .parts = &.{ "\x1b", "j" } }; - try readInput(&loop, &reader, &cache); + try readInput(&loop, &reader, &vx, &feed); try std.testing.expect(!reader.asked); try std.testing.expectEqual(.winsize, std.meta.activeTag((try loop.tryEvent()).?)); const escape = (try loop.tryEvent()).?.key_press; @@ -613,7 +649,7 @@ test "a frame's answer is taken wherever the reads cut it, and never typed" { defer vx.deinit(gpa, &output.writer); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; const Reader = struct { parts: [2][]const u8, next: usize = 0, @@ -635,12 +671,12 @@ test "a frame's answer is taken wherever the reads cut it, and never typed" { } }; const all = "x" ++ FrameAck.answer ++ "y"; - defer FrameAck.waiting.store(false, .release); + defer feed.ack.waiting.store(false, .release); for (0..all.len) |cut| { - FrameAck.waiting.store(true, .release); + feed.ack.waiting.store(true, .release); var reader: Reader = .{ .parts = .{ all[0..cut], all[cut..] } }; - try readInput(&loop, &reader, &cache); - try std.testing.expect(!FrameAck.waiting.load(.acquire)); + try readInput(&loop, &reader, &vx, &feed); + try std.testing.expect(!feed.ack.waiting.load(.acquire)); var typed: std.ArrayList(u8) = .empty; defer typed.deinit(gpa); var woke = false; @@ -658,9 +694,9 @@ test "a frame's answer is taken wherever the reads cut it, and never typed" { } // An Escape pressed just before an answer, in the same read, is the // Escape key, not Alt on the answer. - FrameAck.waiting.store(true, .release); + feed.ack.waiting.store(true, .release); var glued: Reader = .{ .parts = .{ "\x1b" ++ FrameAck.answer, "j" } }; - try readInput(&loop, &glued, &cache); + try readInput(&loop, &glued, &vx, &feed); try std.testing.expectEqual(.winsize, std.meta.activeTag((try loop.tryEvent()).?)); try std.testing.expectEqual(.nop, std.meta.activeTag((try loop.tryEvent()).?)); const escape = (try loop.tryEvent()).?.key_press; @@ -672,25 +708,21 @@ test "a frame's answer is taken wherever the reads cut it, and never typed" { test "a terminal that never answers a frame stops being asked after one timeout" { const io = std.testing.io; - defer { - FrameAck.off.store(false, .release); - FrameAck.waiting.store(false, .release); - FrameAck.sent_ns = 0; - } + var ack: FrameAck = .{}; var out: std.Io.Writer.Allocating = .init(std.testing.allocator); defer out.deinit(); const t0: u64 = 5 * std.time.ns_per_s; - FrameAck.ask(&out.writer, t0); + ack.ask(&out.writer, t0); try std.testing.expectEqualStrings(FrameAck.request, out.written()); // Unanswered: the next frame waits, until the timeout. - try std.testing.expect(FrameAck.holds(t0 + 10 * std.time.ns_per_ms, io)); - try std.testing.expect(FrameAck.holds(t0 + 999 * std.time.ns_per_ms, io)); - try std.testing.expect(!FrameAck.holds(t0 + FrameAck.timeout_ns, io)); + try std.testing.expect(ack.holds(t0 + 10 * std.time.ns_per_ms, io)); + try std.testing.expect(ack.holds(t0 + 999 * std.time.ns_per_ms, io)); + try std.testing.expect(!ack.holds(t0 + FrameAck.timeout_ns, io)); // Then never again: no request, no wait. out.clearRetainingCapacity(); - FrameAck.ask(&out.writer, t0 + 2 * std.time.ns_per_s); + ack.ask(&out.writer, t0 + 2 * std.time.ns_per_s); try std.testing.expectEqual(@as(usize, 0), out.written().len); - try std.testing.expect(!FrameAck.holds(t0 + 2 * std.time.ns_per_s, io)); + try std.testing.expect(!ack.holds(t0 + 2 * std.time.ns_per_s, io)); } test "terminal input cancellation joins blocked reads and queued EOF" { @@ -724,8 +756,8 @@ test "terminal input cancellation joins blocked reads and queued EOF" { } return self.file.readStreaming(std.testing.io, &.{buf}); } - fn run(loop: *Loop, reader: *@This(), cache: *vaxis.GraphemeCache) void { - inputReader(loop, reader, cache); + fn run(loop: *Loop, reader: *@This(), vx_: *vaxis.Vaxis, feed: *Feed) void { + inputReader(loop, reader, vx_, feed); } }; const Case = enum { blocked_read, cancel_eof, deliver_eof }; @@ -738,9 +770,9 @@ test "terminal input cancellation joins blocked reads and queued EOF" { var reader: Reader = .{ .file = .{ .handle = fds[0], .flags = .{ .nonblocking = false } }, .eof = eof }; var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; - loop.thread = try io.concurrent(Reader.run, .{ &loop, &reader, &cache }); - defer stopInput(&loop); + var feed: Feed = .{}; + var thread: ?std.Io.Future(void) = try io.concurrent(Reader.run, .{ &loop, &reader, &vx, &feed }); + defer stopInput(io, &thread); try std.testing.expectEqual(.winsize, std.meta.activeTag(try loop.nextEvent())); try reader.entered.wait(io); if (eof) { @@ -750,10 +782,10 @@ test "terminal input cancellation joins blocked reads and queued EOF" { } if (case == .deliver_eof) { try std.testing.expectEqual(.nop, std.meta.activeTag(try loop.nextEvent())); - loop.thread.?.await(io); + thread.?.await(io); } - stopInput(&loop); - try std.testing.expect(loop.thread == null); + stopInput(io, &thread); + try std.testing.expect(thread == null); var count: usize = 0; while (try loop.tryEvent()) |event| { const expected: std.meta.Tag(@TypeOf(Command.value)) = if (case == .deliver_eof and count == 511) .quit else .nop; @@ -1194,7 +1226,8 @@ pub fn run(init: std.process.Init, opts: pardes.Options, term: Term) !void { posix.sigprocmask(posix.SIG.BLOCK, &winch_set, null); var tty_buf: [0x10000]u8 = undefined; - var screen: Screen = .{ .io = io, .gpa = gpa, .env = init.environ_map, .buf = &tty_buf }; + var screen: Screen = .{ .io = io, .gpa = gpa, .env = init.environ_map, .buf = &tty_buf, .kitty = .init(gpa), .primary = true }; + defer screen.kitty.deinit(); switch (term) { .open => try screen.enter(try vaxis.Tty.init(io, &tty_buf)), .fd => |fd| try screen.enter(try Screen.ttyOn(io, &tty_buf, fd)), @@ -1202,27 +1235,19 @@ pub fn run(init: std.process.Init, opts: pardes.Options, term: Term) !void { } defer screen.leave(); - var kitty_handles = std.AutoHashMap(pardes.ImageCacheKey, vaxis.Image).init(gpa); - defer { - if (screen.on) clearNativeImages(&kitty_handles, &screen.vx, &screen.tty); - kitty_handles.deinit(); - } var paste_buf: std.Io.Writer.Allocating = .init(gpa); defer paste_buf.deinit(); var loop: Loop = .init(io, &screen.tty, &screen.vx); - var input_cache: vaxis.GraphemeCache = .{}; - try localSession(init, opts, &screen, &loop, &input_cache, &kitty_handles, &paste_buf); + try localSession(init, opts, &screen, &loop, &paste_buf); } -/// Whether a terminal is there to ask its size (winchWatch): none while -/// detached, when the descriptor it had may be another's. -var terminal_on = std.atomic.Value(bool).init(false); - -/// The terminal a session draws on, while it has one: entered (raw, the +/// A terminal a session draws on, while it has one: entered (raw, the /// alternate screen, mouse and paste reports) and left (all of it put back, /// the terminal closed) at the session's start and end, and at a detach and -/// a reattach in between (daemon.zig). +/// a reattach in between (daemon.zig). One a view (views.zig): the session's +/// own (`primary`, its stub `daemon.stub`), and one more for each terminal +/// `pardes --attach`ed while it is attached, which keeps its own. const Screen = struct { io: std.Io, gpa: std.mem.Allocator, @@ -1231,6 +1256,24 @@ const Screen = struct { tty: vaxis.Tty = undefined, vx: vaxis.Vaxis = undefined, on: bool = false, + primary: bool = false, + /// The pipe to its stub, an attached one's (the primary's is daemon.stub). + stub: posix.fd_t = -1, + /// Its slot in Killed. + killed: ?usize = null, + feed: Feed = .{}, + thread: ?std.Io.Future(void) = null, + kitty: std.AutoHashMap(pardes.ImageCacheKey, vaxis.Image), + caps_pending: bool = true, + /// Its size as last taken (`Shell.resized`). + size: ?vaxis.Winsize = null, + tracks: []const pardes.animation.Track = &.{}, + /// T2: the cursor's jump trail (JumpTrail). + trail: JumpTrail = .{}, + /// The view's glide, the GUI's motion drawn at whole rows (anim 12). + glide: pardes.glide.Glide = .{}, + /// T7: when each pane's view last moved (ThumbFlash). + flash: ThumbFlash = .{}, /// `fd` made raw, as vaxis.Tty.init makes /dev/tty: the terminal of a /// forked session is no controlling terminal of its, which /dev/tty is. @@ -1250,56 +1293,88 @@ const Screen = struct { try sc.vx.setMouseMode(sc.tty.writer(), true); try sc.vx.setBracketedPaste(sc.tty.writer(), true); if (comptime !builtin.is_test) { - Killed.arm(sc.tty.fd.handle, sc.tty.termios); + sc.killed = Killed.arm(sc.tty.fd.handle, sc.tty.termios); // What a panic puts back (vaxis.recover). - vaxis.tty.global_tty = sc.tty; + if (sc.primary) vaxis.tty.global_tty = sc.tty; } + const view = sc.feed.view; + sc.feed = .{ .view = view }; + sc.caps_pending = true; + sc.size = null; sc.on = true; - terminal_on.store(true, .release); } fn leave(sc: *Screen) void { if (!sc.on) return; sc.on = false; - terminal_on.store(false, .release); - Killed.disarm(); + if (sc.killed) |slot| Killed.disarm(slot); + sc.killed = null; + clearNativeImages(&sc.kitty, &sc.vx, &sc.tty); sc.vx.exitAltScreen(sc.tty.writer()) catch {}; sc.vx.deinit(sc.gpa, sc.tty.writer()); - if (comptime !builtin.is_test) vaxis.tty.global_tty = null; + if (comptime !builtin.is_test) if (sc.primary) { + vaxis.tty.global_tty = null; + }; sc.tty.deinit(); // restore cooked termios LAST, after vx flushed its resets } + + /// Its stub told the terminal is detached (it ends, and its shell has + /// the terminal back). + fn sayDetached(sc: *Screen) void { + if (sc.primary) return daemon.tellStubDetached(); + daemon.tellDetached(sc.stub); + sc.stub = -1; + } + + /// Its terminal hung up, or the stub that waits on it is gone. + fn hungUp(sc: *const Screen) bool { + const stub = if (sc.primary) daemon.stub else sc.stub; + var fds = [_]posix.pollfd{ + .{ .fd = sc.tty.fd.handle, .events = 0, .revents = 0 }, + .{ .fd = stub, .events = 0, .revents = 0 }, + }; + _ = posix.poll(fds[0..if (stub >= 0) 2 else 1], 0) catch return false; + const gone = posix.POLL.HUP | posix.POLL.ERR; + return fds[0].revents & gone != 0 or (stub >= 0 and fds[1].revents & gone != 0); + } + + fn setStub(sc: *Screen, fd: posix.fd_t) void { + if (sc.primary) daemon.stub = fd else sc.stub = fd; + } }; -/// SIGTERM or SIGHUP: the terminal put back as a clean exit leaves it -- +/// SIGTERM or SIGHUP: each terminal put back as a clean exit leaves it -- /// the main screen, no mouse, paste or keyboard modes, the cursor shown, /// cooked termios -- and then the signal's own death. A nested editor /// killed in a pane must not leave that pane's terminal wedged. /// Async-signal-safe: write, tcsetattr, sigaction, raise. const Killed = struct { - var fd: posix.fd_t = -1; - var cooked: posix.termios = undefined; + var fds: [pardes.views.max]posix.fd_t = @splat(-1); + var cooked: [pardes.views.max]posix.termios = undefined; const reset = "\x1b[= 0) { + for (fds, &cooked) |fd, *termios| if (fd >= 0) { _ = std.c.write(fd, reset, reset.len); - _ = std.c.tcsetattr(fd, .FLUSH, &cooked); - } + _ = std.c.tcsetattr(fd, .FLUSH, termios); + }; // The 9P socket and private prompt files go too: a killed editor // leaves none behind. Its shells too, the daemon's dups let go. ninep_io.unlinkOwnSocket(); @@ -1315,14 +1390,10 @@ fn localSession( opts: pardes.Options, screen: *Screen, loop: *Loop, - input_cache: *vaxis.GraphemeCache, - kitty_handles: *std.AutoHashMap(pardes.ImageCacheKey, vaxis.Image), paste_buf: *std.Io.Writer.Allocating, ) !void { const io = init.io; const gpa = init.gpa; - const vx = &screen.vx; - const tty = &screen.tty; const allocs = pardes.memory.init(gpa); defer pardes.memory.deinit(); @@ -1367,20 +1438,23 @@ fn localSession( .core = core, .prompt_rcs = &prompt_rcs, .loop = loop, - .vx = vx, - .tty = tty, - .screen = screen, - .input_cache = input_cache, - .kitty = kitty_handles, .frame = &frame_arena, .paste_buf = paste_buf, .inotify_fd = file_watch.init(false), .fs = fs, .test_clock = host_io.testClock(), }; + sh.terms[0] = screen; pardes.lsp.setStatusSink(&sh, lspStatusSink); host_io.setExitWake(loop, wakeForExit); - defer sh.glide.deinit(gpa); + defer screen.glide.deinit(gpa); + // The terminals of the other views end with the session: each put back, + // its stub told. + defer for (&sh.terms) |*slot| if (slot.*) |sc| if (!sc.primary) { + stopInput(io, &sc.thread); + sh.dropScreen(sc, .exit); + slot.* = null; + }; defer { host_io.setExitWake(null, null); pardes.lsp.setStatusSink(null, null); @@ -1421,7 +1495,7 @@ fn localSession( .search_rows => |b| b.deinit(gpa), .attach => |a| { _ = libc.close(a.tty); - _ = libc.close(a.stub); + if (a.stub >= 0) _ = libc.close(a.stub); }, .adopted => |a| _ = libc.close(a.fd), else => {}, @@ -1434,14 +1508,26 @@ fn localSession( core.unattended = !screen.on; while (core.nextEffect()) |effect| core.perform(effect); - if (screen.on) try startInput(loop, input_cache); - defer stopInput(loop); - (try std.Thread.spawn(.{}, winchWatch, .{ loop, vx, tty })).detach(); + if (screen.on) try startInput(loop, screen); + defer stopInput(io, &screen.thread); + (try std.Thread.spawn(.{}, winchWatch, .{loop})).detach(); (try std.Thread.spawn(.{}, wakeWatch, .{ io, loop })).detach(); if (fs) |f| try f.watch(loop, wakeFs); - if (screen.on) try vx.queryTerminalSend(tty.writer()); + if (screen.on) try screen.vx.queryTerminalSend(screen.tty.writer()); // Detached from the start, nothing else may wake the first wait. if (!screen.on) daemon.watch(loop, deliverNote); + // PARDES_TEST_VIEWS: the terminals at these paths are views of the + // session too, from its start, as `pardes --attach`ed ones are (the + // snapshot harness's `start ... views=N`). + if (init.environ_map.get("PARDES_TEST_VIEWS")) |paths| { + var it = std.mem.tokenizeScalar(u8, paths, ' '); + while (it.next()) |path| { + var path_buf: [256]u8 = undefined; + const z = std.fmt.bufPrintSentinel(&path_buf, "{s}", .{path}, 0) catch continue; + const fd = libc.open(z, .{ .ACCMODE = .RDWR, .NOCTTY = true, .CLOEXEC = true }, @as(libc.mode_t, 0)); + if (fd >= 0) sh.attachTerminal(fd, -1); + } + } sh.threads_ok = true; for (&sh.ptys, 0..) |*slot, id| if (slot.*) |*pt| { @@ -1497,15 +1583,29 @@ fn localSession( .{ .text = 0 }, ); _ = file_watch.applyThemeEffect(core, gpa, sh.inotify_fd, &sh.watches, 0, false, false); - if (screen.on) clearNativeImages(kitty_handles, vx, tty); - nc.native_images = screen.on and vx.caps.kitty_graphics; - nc.unattended = !screen.on; + // ponytail: a restored session is one view, the live one's + // terminal: the others are let go, as a detach of each. + const live = sh.term().?; + for (&sh.terms) |*slot| if (slot.*) |sc| if (sc != live) { + stopInput(io, &sc.thread); + sh.dropScreen(sc, .detach); + slot.* = null; + }; + sh.terms = @splat(null); + sh.terms[0] = live; + stopInput(io, &live.thread); + live.feed.view = 0; + sh.from = 0; + if (live.on) startInput(loop, live) catch {}; + if (live.on) clearNativeImages(&live.kitty, &live.vx, &live.tty); + nc.native_images = live.on and live.vx.caps.kitty_graphics; + nc.unattended = !live.on; if (fs) |f| f.reset(nc); core.deinit(); core = nc; sh.core = nc; - if (comptime pardes.pdf_enabled) if (screen.on) { - updateCoreTerminalSize(core, vx.screen.width, vx.screen.height, vx.screen.width_pix, vx.screen.height_pix); + if (comptime pardes.pdf_enabled) if (live.on) { + updateCoreTerminalSize(core, live.vx.screen.width, live.vx.screen.height, live.vx.screen.width_pix, live.vx.screen.height_pix); }; } } @@ -1554,12 +1654,11 @@ const Shell = struct { core: *pardes.Pardes, prompt_rcs: *const host_io.Shell.PromptFiles, loop: *Loop, - vx: *vaxis.Vaxis, - tty: *vaxis.Tty, - /// The terminal, while there is one (Screen.on); null in tests. - screen: ?*Screen = null, - input_cache: ?*vaxis.GraphemeCache = null, - kitty: *std.AutoHashMap(pardes.ImageCacheKey, vaxis.Image), + /// Each view's terminal, by view (views.zig): the session's own first; + /// none in tests. + terms: [pardes.views.max]?*Screen = @splat(null), + /// The view the input taken now is from (`Command.value.from`). + from: u8 = 0, frame: *std.heap.ArenaAllocator, paste_buf: *std.Io.Writer.Allocating, inotify_fd: c_int, @@ -1576,16 +1675,8 @@ const Shell = struct { lsp_task: ?host_io.Lsp.Task = null, pipe_tasks: PipeTasks = .{}, threads_ok: bool = false, - caps_pending: bool = true, check_files: bool = false, in_paste: bool = false, - tracks: []const pardes.animation.Track = &.{}, - /// T2: the cursor's jump trail (JumpTrail). - trail: JumpTrail = .{}, - /// The view's glide, the GUI's motion drawn at whole rows (anim 12). - glide: pardes.glide.Glide = .{}, - /// T7: when each pane's view last moved (ThumbFlash). - flash: ThumbFlash = .{}, /// PARDES_TEST_CLOCK's virtual time (host_io.testClock), and where the /// running-out wait moves it: the core's next wake. test_clock: ?u64 = null, @@ -1602,6 +1693,8 @@ const Shell = struct { input_ns: u64 = 0, focused_ns: u64 = 0, + const Leaving = enum { detach, exit }; + fn of(ctx: ?*anyopaque) *Shell { return @ptrCast(@alignCast(ctx.?)); } @@ -1653,8 +1746,13 @@ const Shell = struct { return s.test_clock orelse @intCast(std.Io.Clock.awake.now(s.io).nanoseconds); } + /// The live view's terminal (null in tests). + fn term(s: *const Shell) ?*Screen { + return s.terms[s.core.view]; + } + fn attached(s: *const Shell) bool { - const screen = s.screen orelse return true; + const screen = s.term() orelse return true; return screen.on; } @@ -1668,14 +1766,46 @@ const Shell = struct { fn detach(ctx: ?*anyopaque) void { const s = of(ctx); if (!s.attached()) return s.core.reportFailure(s.core.active, "Detach: no terminal is attached"); + // One of several: its view goes, and the session stays on the rest. + if (pardes.views.count(s.core) > 1) return s.dropView(s.core.view, .detach); if (!daemon.kept()) return s.core.reportFailure(s.core.active, "Detach: no pardes daemon keeps this session (it is its terminal's job)"); s.leaveTerminal(); } + /// View `k`'s terminal let go (detached, or gone) while others stay: the + /// view goes with it. + fn dropView(s: *Shell, k: u8, how: Leaving) void { + const sc = s.terms[k] orelse return; + stopInput(s.io, &sc.thread); + s.terms[k] = null; + s.core.closeView(k); + s.dropScreen(sc, how); + } + + /// A terminal put back and its stub told; an attached one's Screen freed + /// (the session's own stays, a view of none). + fn dropScreen(s: *Shell, sc: *Screen, how: Leaving) void { + if (sc.primary) { + sc.leave(); + return sc.sayDetached(); + } + sc.leave(); + switch (how) { + .detach => sc.sayDetached(), + .exit => { + daemon.tellExit(sc.stub, 0); + sc.stub = -1; + }, + } + sc.glide.deinit(s.gpa); + sc.kitty.deinit(); + s.gpa.free(sc.buf); + s.gpa.destroy(sc); + } + fn leaveTerminal(s: *Shell) void { - const screen = s.screen orelse return; - stopInput(s.loop); - clearNativeImages(s.kitty, s.vx, s.tty); + const screen = s.term() orelse return; + stopInput(s.io, &screen.thread); screen.leave(); s.core.native_images = false; s.core.unattended = true; @@ -1685,39 +1815,65 @@ const Shell = struct { for (0..3) |i| _ = libc.dup2(null_fd, @intCast(i)); if (null_fd > 2) _ = libc.close(null_fd); } - daemon.tellStubDetached(); + screen.sayDetached(); daemon.setState(true); } - /// `pardes --attach`'s terminal, for this detached session: entered as - /// the first one was, sized, asked what it can do, and drawn whole. + /// `pardes --attach`'s terminal: for a detached session its own again, + /// entered as the first one was; for one attached already, a view of + /// its own (views.zig). Sized, asked what it can do, and drawn whole. fn attachTerminal(s: *Shell, fd: posix.fd_t, stub: posix.fd_t) void { - const screen = s.screen orelse return; - if (screen.on) { + const live = s.term() orelse return; + const screen = if (!live.on) live else extra: { + const sc = s.gpa.create(Screen) catch break :extra null; + const buf = s.gpa.alloc(u8, 0x10000) catch { + s.gpa.destroy(sc); + break :extra null; + }; + sc.* = .{ .io = s.io, .gpa = s.gpa, .env = live.env, .buf = buf, .kitty = .init(s.gpa) }; + break :extra sc; + } orelse { _ = libc.close(fd); - return daemon.refuseStub(stub, "that session has a terminal already"); - } + return refuse(stub, "that session has no room for another terminal"); + }; + const extra = screen != live; const tty = Screen.ttyOn(s.io, screen.buf, fd) catch { _ = libc.close(fd); - daemon.setState(true); - return daemon.refuseStub(stub, "that terminal would not go raw"); + if (extra) s.freeScreen(screen) else daemon.setState(true); + return refuse(stub, "that terminal would not go raw"); }; screen.enter(tty) catch { - daemon.setState(true); - return daemon.refuseStub(stub, "that terminal would not take the screen"); + if (extra) s.freeScreen(screen) else daemon.setState(true); + return refuse(stub, "that terminal would not take the screen"); }; - daemon.stub = stub; - FrameAck.waiting.store(false, .release); - FrameAck.off.store(false, .release); - kitty_shm.store(false, .release); - s.caps_pending = true; + if (extra) { + const k = s.core.openView() orelse { + screen.leave(); + s.freeScreen(screen); + return refuse(stub, "that session has as many terminals as it takes"); + }; + screen.feed.view = k; + s.terms[k] = screen; + } + screen.setStub(stub); s.core.unattended = false; - startInput(s.loop, s.input_cache.?) catch {}; - s.vx.queryTerminalSend(s.tty.writer()) catch {}; + startInput(s.loop, screen) catch {}; + screen.vx.queryTerminalSend(screen.tty.writer()) catch {}; s.core.needs_frame = true; daemon.setState(false); } + fn freeScreen(s: *Shell, sc: *Screen) void { + sc.kitty.deinit(); + s.gpa.free(sc.buf); + s.gpa.destroy(sc); + } + + /// A terminal refused: its stub told why (a test's has none). + fn refuse(stub: posix.fd_t, why: []const u8) void { + if (stub >= 0) daemon.refuseStub(stub, why); + } + /// Adopt: the daemon is asked for the shells of sessions that died /// holding them; each comes back (`adopted`) as a terminal pane here. fn adopt(ctx: ?*anyopaque, pane: u8) void { @@ -1880,12 +2036,38 @@ const Shell = struct { else => {}, } if (s.input_seen) s.input_ns = @intCast(std.Io.Clock.awake.now(s.io).nanoseconds); + // A terminal's input goes into its view; a closed one's last is + // dropped. + switch (event) { + .key_press, .mouse, .paste, .paste_start, .paste_end, .winsize, .focus_in, .focus_out, .quit => { + if (s.terms[s.from] == null and s.term() != null) { + if (event == .paste) s.gpa.free(@constCast(event.paste)); + return false; + } + if (s.term() != null) core.toView(s.from); + }, + else => {}, + } switch (event) { .nop => {}, + .from => |k| s.from = k, + .winched => s.winched(), + // A terminal hung up (its stub passes the hangup on) or its stub + // died (a forked session's death signal): that one's input has + // ended. A HUP says not whose: each is asked. + .hangup => for (s.terms, 0..) |slot, k| if (slot) |sc| if (sc.on and sc.hungUp()) { + s.from = @intCast(k); + if (s.apply(.quit)) return true; + }, .wake => if (s.test_clock) |*virtual| { virtual.* = @max(virtual.*, s.wake_to); }, .quit => { + // One of several: its view goes with it. + if (pardes.views.count(core) > 1) { + s.dropView(core.view, .detach); + return true; + } // The terminal went: a session the daemon keeps carries on // without it, to be attached again. Gone before the first // frame, the reader that takes the daemon's `ok` (kept @@ -1905,13 +2087,7 @@ const Shell = struct { core.update(.pointer_leave); core.update(.{ .focus = false }); }, - .winsize => |ws| if (s.attached()) { - s.vx.resize(s.gpa, s.tty.writer(), ws) catch {}; - if (comptime pardes.pdf_enabled) - updateCoreTerminalSize(core, ws.cols, ws.rows, ws.x_pixel, ws.y_pixel) - else - core.update(.{ .resize = .{ .cols = ws.cols, .rows = ws.rows } }); - }, + .winsize => |ws| if (s.attached()) s.resized(ws), .pty_read => |pr| { tookRead(s.io, pr.id); s.output_seen = true; @@ -2008,6 +2184,32 @@ const Shell = struct { return false; } + /// The live view's terminal is `ws` big. + fn resized(s: *Shell, ws: vaxis.Winsize) void { + const core = s.core; + if (s.term()) |sc| { + sc.size = ws; + sc.vx.resize(s.gpa, sc.tty.writer(), ws) catch {}; + } + if (comptime pardes.pdf_enabled) + updateCoreTerminalSize(core, ws.cols, ws.rows, ws.x_pixel, ws.y_pixel) + else + core.update(.{ .resize = .{ .cols = ws.cols, .rows = ws.rows } }); + } + + /// A SIGWINCH: each terminal whose size it changed is resized, in its + /// view (a terminal reporting in band says so as input). + fn winched(s: *Shell) void { + for (s.terms, 0..) |slot, k| { + const sc = slot orelse continue; + if (!sc.on or sc.vx.state.in_band_resize) continue; + const ws = sc.tty.getWinsize() catch continue; + if (sc.size) |was| if (std.meta.eql(was, ws)) continue; + s.core.toView(@intCast(k)); + s.resized(ws); + } + } + fn reloadWatched(s: *Shell) void { if (!s.check_files) return; s.check_files = false; @@ -2020,13 +2222,12 @@ const Shell = struct { /// notches are); `wheeled` from the mouse event if it should glide. fn stepGlide(ctx: ?*anyopaque) void { const s = of(ctx); - s.glide.step(s.core, s.gpa, null, null); + if (s.term()) |sc| sc.glide.step(s.core, s.gpa, null, null); } fn pollFrame(ctx: ?*anyopaque) void { const s = of(ctx); stepGlide(ctx); - s.core.native_image_patches = kitty_shm.load(.acquire); if (s.fs) |f| if (f.tick().pending) { _ = s.loop.tryPostEvent(.fs_ready) catch {}; }; @@ -2035,23 +2236,26 @@ const Shell = struct { if (host_io.ttyCwd(pt.pid, pt.file.handle, s.core.settings.shell.effective.get(), &lbuf)) |cwd| s.core.setCwd(id, cwd); }; - if (s.caps_pending and s.attached() and s.vx.queries_done.load(.unordered)) { - s.caps_pending = false; - s.vx.enableDetectedFeatures(s.tty.writer()) catch {}; - s.core.native_images = s.vx.caps.kitty_graphics; - if (s.vx.caps.kitty_graphics) kittyShmProbe(s.tty.writer()); - s.vx.queueRefresh(); + const sc = s.term() orelse return; + s.core.native_image_patches = sc.feed.shm.load(.acquire); + if (sc.caps_pending and sc.on and sc.vx.queries_done.load(.unordered)) { + sc.caps_pending = false; + sc.vx.enableDetectedFeatures(sc.tty.writer()) catch {}; + s.core.native_images = sc.vx.caps.kitty_graphics; + if (sc.vx.caps.kitty_graphics) kittyShmProbe(sc.tty.writer(), &sc.feed); + sc.vx.queueRefresh(); s.core.needs_frame = true; } } fn present(ctx: ?*anyopaque, canonical: *const pardes.Surface) void { const s = of(ctx); - const vx = s.vx; - s.tracks = canonical.panelTracks(); + const sc = s.term() orelse return; + const vx = &sc.vx; + sc.tracks = canonical.panelTracks(); // The last frame is not answered yet: this one stays owed, and the // answer's wake draws the state as it is then. - if (FrameAck.holds(@intCast(std.Io.Clock.awake.now(s.io).nanoseconds), s.io)) { + if (sc.feed.ack.holds(@intCast(std.Io.Clock.awake.now(s.io).nanoseconds), s.io)) { s.core.present_skipped = true; return; } @@ -2059,21 +2263,21 @@ const Shell = struct { const surface = panel_compositor.compose( s.frame.allocator(), @constCast(canonical), - s.tracks, + sc.tracks, s.core.theme().bg, ) catch return; const tz_cells = tracy.zone(@src(), "surface->vaxis"); const win = vx.window(); const truecolor = vx.caps.rgb or colortermTruecolor(); // Each of the shell's own motions below asks for frames while it runs. - if (s.glide.pending) s.glide.measure(s.gpa, canonical); - if (s.glide.moving()) s.glide.advance(s.core.now_ns, s.glide.motion); - s.core.shell_continuous = s.glide.moving(); - s.core.frames.scroll = @round(s.glide.follow.offset); - const glided = pardes.glide.shifted(&s.glide, s.frame.allocator(), surface) catch surface; + if (sc.glide.pending) sc.glide.measure(s.gpa, canonical); + if (sc.glide.moving()) sc.glide.advance(s.core.now_ns, sc.glide.motion); + s.core.shell_continuous = sc.glide.moving(); + s.core.frames.scroll = @round(sc.glide.follow.offset); + const glided = pardes.glide.shifted(&sc.glide, s.frame.allocator(), surface) catch surface; const shadowed = chipShadows(s.frame.allocator(), glided, s.core, truecolor) catch glided; - const flashed = s.flash.apply(s.frame.allocator(), shadowed, s.core, truecolor) catch shadowed; - const trailed = s.trail.apply(s.frame.allocator(), flashed, s.core, truecolor) catch flashed; + const flashed = sc.flash.apply(s.frame.allocator(), shadowed, s.core, truecolor) catch shadowed; + const trailed = sc.trail.apply(s.frame.allocator(), flashed, s.core, truecolor) catch flashed; paintCells(win, trailed.cells, trailed.cols, trailed.rows); tz_cells.end(); for (surface.images[0..surface.nimages]) |maybe| { @@ -2083,7 +2287,7 @@ const Shell = struct { if (!kittyImageRepresentable(place)) continue; } const key = place.cacheKey(); - if (!s.kitty.contains(key) and vx.caps.kitty_graphics and kitty_shm.load(.acquire)) shm: { + if (!sc.kitty.contains(key) and vx.caps.kitty_graphics and sc.feed.shm.load(.acquire)) shm: { var buf: [64]u8 = undefined; kitty_shm_serial +%= 1; // Bounded: a segment this many transmissions old was never @@ -2095,21 +2299,21 @@ const Shell = struct { } const name = kittyShmName(&buf, kitty_shm_serial) catch break :shm; if (!kittyShmWrite(name, place.rgba)) break :shm; - if (vx.transmitLocalImagePath(s.gpa, s.tty.writer(), name, @intCast(place.iw), @intCast(place.ih), .shared_mem, .rgba) catch null) |handle| - s.kitty.put(key, handle) catch vx.freeImage(s.tty.writer(), handle.id) + if (vx.transmitLocalImagePath(s.gpa, sc.tty.writer(), name, @intCast(place.iw), @intCast(place.ih), .shared_mem, .rgba) catch null) |handle| + sc.kitty.put(key, handle) catch vx.freeImage(sc.tty.writer(), handle.id) else _ = libc.shm_unlink(name); } - if (!s.kitty.contains(key) and vx.caps.kitty_graphics) { + if (!sc.kitty.contains(key) and vx.caps.kitty_graphics) { const enc = std.base64.standard.Encoder; if (s.gpa.alloc(u8, enc.calcSize(place.rgba.len))) |b64| { defer s.gpa.free(b64); _ = enc.encode(b64, place.rgba); - if (vx.transmitPreEncodedImage(s.tty.writer(), b64, @intCast(place.iw), @intCast(place.ih), .rgba) catch null) |handle| - s.kitty.put(key, handle) catch vx.freeImage(s.tty.writer(), handle.id); + if (vx.transmitPreEncodedImage(sc.tty.writer(), b64, @intCast(place.iw), @intCast(place.ih), .rgba) catch null) |handle| + sc.kitty.put(key, handle) catch vx.freeImage(sc.tty.writer(), handle.id); } else |_| {} } - if (s.kitty.get(key)) |cached| { + if (sc.kitty.get(key)) |cached| { if (comptime !pardes.pdf_enabled) { const child = win.child(.{ .x_off = place.x, .y_off = place.y, .width = place.w, .height = place.h }); cached.draw(child, .{ .scale = .contain }) catch {}; @@ -2150,32 +2354,32 @@ const Shell = struct { while (true) { var stale: [pardes.MAX_PANES]pardes.ImageCacheKey = undefined; var stale_len: usize = 0; - var image_iterator = s.kitty.iterator(); + var image_iterator = sc.kitty.iterator(); while (image_iterator.next()) |entry| { if (surfaceHasKittyKey(surface, entry.key_ptr.*)) continue; stale[stale_len] = entry.key_ptr.*; stale_len += 1; if (stale_len == stale.len) break; } - for (stale[0..stale_len]) |key| if (s.kitty.fetchRemove(key)) |removed| - vx.freeImage(s.tty.writer(), removed.value.id); + for (stale[0..stale_len]) |key| if (sc.kitty.fetchRemove(key)) |removed| + vx.freeImage(sc.tty.writer(), removed.value.id); if (stale_len < stale.len) break; } if (glided.cursor) |cur| paintCursor(win, cur.x, cur.y, cur.bar); const tz_render = tracy.zone(@src(), "vx.render"); - const before = s.tty.tty_writer.pos + s.tty.writer().end; - vx.render(s.tty.writer()) catch {}; + const before = sc.tty.tty_writer.pos + sc.tty.writer().end; + vx.render(sc.tty.writer()) catch {}; // The frame's cost to the terminal (FrameStats.Frame.bytes): what // vaxis's diff of the cells wrote. - s.core.frames.bytes +|= @intCast(@min(std.math.maxInt(u32), s.tty.tty_writer.pos + s.tty.writer().end -| before)); - if (s.tty.tty_writer.pos + s.tty.writer().end != before) FrameAck.ask(s.tty.writer(), @intCast(std.Io.Clock.awake.now(s.io).nanoseconds)); + s.core.frames.bytes +|= @intCast(@min(std.math.maxInt(u32), sc.tty.tty_writer.pos + sc.tty.writer().end -| before)); + if (sc.tty.tty_writer.pos + sc.tty.writer().end != before) sc.feed.ack.ask(sc.tty.writer(), @intCast(std.Io.Clock.awake.now(s.io).nanoseconds)); s.drawn_ns = @intCast(std.Io.Clock.awake.now(s.io).nanoseconds); tz_render.end(); } fn postPresent(ctx: ?*anyopaque) void { const s = of(ctx); - s.core.acknowledgePanelPresentation(s.tracks); + if (s.term()) |sc| s.core.acknowledgePanelPresentation(sc.tracks); tracy.frameMark(); } @@ -2320,14 +2524,16 @@ const Shell = struct { fn setClipboard(ctx: ?*anyopaque, text: []const u8) void { const s = of(ctx); // Detached, the session keeps it (a 9P read, a paste here). - if (!s.attached()) return s.core.fallback.setClipboard(text); - copyToClipboard(s.vx, s.tty, s.gpa, text); + const sc = s.term() orelse return s.core.fallback.setClipboard(text); + if (!sc.on) return s.core.fallback.setClipboard(text); + copyToClipboard(&sc.vx, &sc.tty, s.gpa, text); } fn readClipboard(ctx: ?*anyopaque) void { const s = of(ctx); - if (!s.attached()) return s.core.update(.{ .paste = s.core.fallback.clipboard.items }); - requestClipboard(s.vx, s.tty); + const sc = s.term() orelse return s.core.update(.{ .paste = s.core.fallback.clipboard.items }); + if (!sc.on) return s.core.update(.{ .paste = s.core.fallback.clipboard.items }); + requestClipboard(&sc.vx, &sc.tty); } fn openLink(_: ?*anyopaque, url: []const u8) void { @@ -2391,9 +2597,6 @@ test "TTY queued results reject old PTY generations and LSP request IDs" { .core = core, .prompt_rcs = undefined, .loop = undefined, - .vx = undefined, - .tty = undefined, - .kitty = undefined, .frame = undefined, .paste_buf = undefined, .inotify_fd = -1, @@ -2439,9 +2642,6 @@ test "TTY worker setup failures finish matching LSP and pipe requests" { .core = core, .prompt_rcs = undefined, .loop = undefined, - .vx = undefined, - .tty = undefined, - .kitty = undefined, .frame = undefined, .paste_buf = undefined, .inotify_fd = -1, @@ -2507,24 +2707,16 @@ fn watchFiles(io: std.Io, fd: c_int, loop: *Loop) anyerror!void { fn keepSignal(_: posix.SIG) callconv(.c) void {} -fn winchWatch(loop: *Loop, vx: *vaxis.Vaxis, tty: *vaxis.Tty) void { +fn winchWatch(loop: *Loop) void { var set = posix.sigemptyset(); posix.sigaddset(&set, posix.SIG.WINCH); if (daemon.link >= 0) posix.sigaddset(&set, posix.SIG.HUP); while (true) { var sig: c_int = 0; if (libc.sigwait(&set, &sig) != 0) continue; - // The terminal hung up (its stub passes the hangup on): as its - // input's end. - if (sig == @backingInt(posix.SIG.HUP)) { - loop.postEvent(.quit) catch {}; - continue; - } - // Detached, the descriptor the terminal had may be another's now. - if (!terminal_on.load(.acquire)) continue; - if (vx.state.in_band_resize) continue; // terminal reports via CSI 48 - const ws = tty.getWinsize() catch continue; - loop.postEvent(.{ .winsize = ws }) catch {}; + // The terminals are the loop's: it asks each its size, and takes a + // hangup as the end of the one that hung up. + loop.postEvent(if (sig == @backingInt(posix.SIG.HUP)) .hangup else .winched) catch {}; } } @@ -2665,7 +2857,7 @@ test "a terminal pane gets function keys as their sequences and lone modifiers n vx.queries_done.store(false, .unordered); var tty: vaxis.Tty = undefined; var loop: Loop = .init(io, &tty, &vx); - var cache: vaxis.GraphemeCache = .{}; + var feed: Feed = .{}; const Reader = struct { parts: []const []const u8, next: usize = 0, @@ -2698,7 +2890,7 @@ test "a terminal pane gets function keys as their sequences and lone modifiers n "\x1b[104;;104u", }, }; - try readInput(&loop, &reader, &cache); + try readInput(&loop, &reader, &vx, &feed); const core = try pardes.Pardes.init(gpa, .{ .tty_only = true }); defer core.deinit(); core.panes[0].?.body.mode = .tty; diff --git a/src/views.zig b/src/views.zig new file mode 100644 index 00000000..ae2b338d --- /dev/null +++ b/src/views.zig @@ -0,0 +1,555 @@ +//! Views: several clients on one session, kakoune's way (docs/next-daemon.md, +//! "Phase 4 notes"). Each view has its own layout, focus, selections and +//! scroll over the panes every view shares: their text, terminals, tags, +//! registers and messages. +//! +//! The core still sees one view, the live one, in the fields it always had: +//! a view's share of the session (`fields`) and of each pane (`PaneView`) is +//! copied out when another view goes live (`park`) and back when it is live +//! again (`unpark`). No line of the core reads a view through an +//! indirection, and one view never parks: today's single client runs +//! exactly as before. +//! +//! A parked view is in no digest (Pardes.replica's `parked`, Pane's `pv`): +//! a slot is written whenever its view goes out, so the live view's slot is +//! stale by design, and a parked view that diverged shows when it is live +//! again. The pump draws the other views by going into each and back +//! (`drawOthers`), which leaves every digested field as it was; a replica +//! that draws nothing skips it. +const std = @import("std"); +const pardes = @import("pardes.zig"); +const Pardes = pardes.Pardes; +const Pane = pardes.Pane; +const Text = pardes.panes.Text; +const File = pardes.panes.File; +const layout = pardes.layout; +const MAX_PANES = pardes.MAX_PANES; + +/// Views at once: a window or terminal each. +pub const max = 4; + +/// The session's fields that are each view's own: its screen and layout, +/// its focus, its pointer and gestures, its jumps, its key prefixes, and +/// what drawing it keeps (animations, the presentation of its panels). +pub const fields = [_][:0]const u8{ + "screen_w", "screen_h", "cell_pixels", "row_metrics", + "ncol", "col_weight", "col_panes", "col_n", + "col_tags", "col_serial", "finder", "rects", + "col_x", "col_w", "presentation", "active", + "jumps", "njumps", "jcur", "jump_force", + "active_column", "lifts", "lift_serials", "cursor_glide", + "cursor_view", "cursor_crossing", "cursor_held", "shell_continuous", + "input_ns", "native_images", "native_image_patches", "windowed", + "drag", "mouse_grab", "window_focused", "focus_told", + "hover_col", "hover_row", "pointer_raw_col", "pointer_raw_row", + "pointer_body_hit", "pointer_tag_hit", "raw_hover_intent", "pointer_inside", + "last_click", "click_as", "chord_hold", "wheel_guard", + "ctrl_w_pending", "register", "leader_on", "window_pick", + "leader_keys", "leader_n", "header_focus", "header_column", + "header_drag", "header_sweep", "header_scroll", "header_top", + "ov_pinch_scale", "ov_touch_scroll_delta", "chord_arg", "clip_pending", +}; + +/// A pane's fields that are each view's own: where it is in the view's +/// layout, which of its texts has the keyboard, how far it is scrolled, and +/// what the view's step measured of it (its tag's rows, its bands). +const pane_fields = [_][:0]const u8{ "focus", "vweight", "collapsed", "attached_to", "hscroll", "tag_scroll", "tag_top", "tag_rows", "view", "notices" }; + +/// The body's: its cursor, selections and mode. Its undo stays the pane's. +const body_fields = [_][:0]const u8{ "mode", "cur_pinned", "cur_row", "cur_col", "msel", "vsel", "sels", "nsel", "select", "sticky_col", "restore_cursor", "last_edit", "reg_wait", "normal", "find_op", "find_ch" }; + +pub const View = Part(Pardes, &fields); + +/// Named field by field, not taken from Pane's type: a pane holds its own. +pub const PaneView = struct { + focus: Pane.Focus = .body, + vweight: f32 = 1, + collapsed: bool = false, + attached_to: ?u32 = null, + hscroll: i32 = 0, + tag_scroll: u16 = 0, + tag_top: u16 = 0, + tag_rows: u16 = 1, + view: Pane.SessionView = .{}, + notices: Pane.Notices = .{}, + body: Part(Text, &body_fields) = .{}, + /// A document's scroll and size; a terminal's size is its pty's, which + /// every view shows (sized by its last typist, `Pardes.sync`). + scroll: usize = 0, + cols: u16 = 1, + rows: u16 = 1, +}; + +/// A view not live: its share of the session, and the serial of each pane +/// slot when it went out, by which the panes closed since leave its layout. +pub const Parked = struct { + view: View, + serials: [MAX_PANES]u32 = @splat(0), +}; + +/// The fields of `T` named, as a struct of their own, each of its type and +/// default. +fn Part(comptime T: type, comptime names: []const [:0]const u8) type { + @setEvalBranchQuota(100_000); + const s = @typeInfo(T).@"struct"; + var types: [names.len]type = undefined; + var attrs: [names.len]std.lang.Type.Struct.FieldAttributes = undefined; + for (names, 0..) |name, i| { + const at = for (s.field_names, 0..) |n, j| { + if (std.mem.eql(u8, n, name)) break j; + } else @compileError("views: " ++ @typeName(T) ++ " has no field " ++ name); + types[i] = s.field_types[at]; + attrs[i] = .{ .default_value_ptr = s.field_attrs[at].default_value_ptr }; + } + return @Struct(.auto, null, names, &types, &attrs); +} + +fn copy(comptime names: []const [:0]const u8, dst: anytype, src: anytype) void { + inline for (names) |name| @field(dst, name) = @field(src, name); +} + +pub fn isOpen(p: *const Pardes, k: u8) bool { + return k < max and p.views_open & (@as(u8, 1) << @intCast(k)) != 0; +} + +pub fn count(p: *const Pardes) usize { + return @popCount(p.views_open); +} + +/// More than one view: the one test the single client's path pays. +pub fn several(p: *const Pardes) bool { + return p.views_open & (p.views_open -% 1) != 0; +} + +/// Pane `pane`'s share of the live view. +fn extract(pane: *const Pane) PaneView { + var v: PaneView = .{ .cols = pane.cols, .rows = pane.rows }; + copy(&pane_fields, &v, pane); + copy(&body_fields, &v.body, &pane.body); + if (pane.file) |f| v.scroll = f.scroll; + return v; +} + +/// Its share of view `v` put back. +fn apply(pane: *Pane, v: *const PaneView) void { + copy(&pane_fields, pane, v); + copy(&body_fields, &pane.body, &v.body); + if (!pane.isTerminal()) { + pane.cols = v.cols; + pane.rows = v.rows; + } + const f = if (pane.file) |*file| file else return; + // The text may have changed while the view was out: what it held is + // brought back inside it. + const lines = File.nlines(pane.gpa, f); + f.scroll = @min(v.scroll, lines -| 1); + if (pane.body.cur_row >= lines) { + pane.body.vsel.active = false; + pane.body.msel.active = false; + pane.body.nsel = 0; + } + _ = pardes.edit.clampFileCursor(&pane.body); +} + +/// A copy of the live view's share for another view, owning nothing of the +/// live one's (its extra selections stay behind). +fn fresh(pane: *const Pane) PaneView { + var v = extract(pane); + v.body.sels = &.{}; + v.body.nsel = 0; + return v; +} + +/// A new pane is in each other view as it was made, there to be shown. +pub fn installed(p: *Pardes, pane: *Pane) void { + pane.sized_by = p.view; + if (!several(p)) return; + for (0..max) |k| if (k != p.view and isOpen(p, @intCast(k))) { + pane.pv[k] = fresh(pane); + }; + pane.pv_known = p.views_open; +} + +/// A pane going: the other views' selections of it go too. +pub fn tornDown(p: *Pardes, pane: *Pane) void { + for (0..max) |k| if (k != p.view and pane.pv_known & (@as(u8, 1) << @intCast(k)) != 0) { + const sels = pane.pv[k].body.sels; + if (sels.len > 0) p.gpa.free(sels); + pane.pv[k].body.sels = &.{}; + }; +} + +fn park(p: *Pardes) void { + const slot = p.parked[p.view].?; + copy(&fields, &slot.view, p); + for (p.panes, 0..) |s, id| { + const pane = s orelse { + slot.serials[id] = 0; + continue; + }; + slot.serials[id] = pane.serial; + pane.pv[p.view] = extract(pane); + } +} + +fn unpark(p: *Pardes, k: u8) void { + const slot = p.parked[k].?; + copy(&fields, p, &slot.view); + p.view = k; + for (p.panes) |s| if (s) |pane| apply(pane, &pane.pv[k]); + // The panes closed while it was out (a slot empty or another pane's + // now) leave its layout, and its keyboard leaves them. + for (0..p.ncol) |c| { + var n: usize = 0; + for (p.col_panes[c][0..p.col_n[c]]) |id| { + const pane = p.panes[id] orelse continue; + if (pane.serial != slot.serials[id]) continue; + p.col_panes[c][n] = id; + n += 1; + } + p.col_n[c] = n; + } + if (layout.findPane(p, p.active) == null) { + for (0..p.ncol) |c| if (p.col_n[c] > 0) { + p.active = p.col_panes[c][0]; + break; + }; + } + layout.compute(p); +} + +/// The pane with the keyboard is one the live view shows: focused where its +/// layout has it not (a Look at a file another view opened, a jump, a 9P +/// focus), it goes into the view's active column, as kakoune shows a buffer +/// in the window that asked for it. +pub fn show(p: *Pardes) void { + if (p.panes[p.active] == null or layout.findPane(p, p.active) != null) return; + const c = layout.firstColumn(p) orelse layout.columnBySerial(p, p.active_column) orelse p.ncol - 1; + layout.acmePlace(p, c, null, p.active); +} + +/// Into view `k`: the live one parked, `k`'s put in its place. +pub fn to(p: *Pardes, k: u8) void { + std.debug.assert(isOpen(p, k)); + if (k == p.view) return; + park(p); + unpark(p, k); +} + +/// A new view, as the live one is laid out and focused, with its panes +/// where they are scrolled and its cursors where they are; the rest of it +/// (column tags, gestures, animations) as a session starts. Null when there +/// are `max` already. +pub fn open(p: *Pardes) ?u8 { + const k: u8 = for (0..max) |i| { + if (!isOpen(p, @intCast(i))) break @intCast(i); + } else return null; + // Each live view parks too: its slot is made with the first other. + for ([_]u8{ p.view, k }) |i| if (p.parked[i] == null) { + const slot = p.gpa.create(Parked) catch return null; + slot.* = .{ .view = .{ .screen_w = p.screen_w, .screen_h = p.screen_h } }; + p.parked[i] = slot; + }; + const slot = p.parked[k].?; + slot.* = .{ .view = .{ .screen_w = p.screen_w, .screen_h = p.screen_h } }; + inline for (.{ "screen_w", "screen_h", "cell_pixels", "row_metrics", "ncol", "col_weight", "col_panes", "col_n", "col_serial", "finder", "active", "active_column", "jumps", "njumps", "jcur", "windowed", "native_images", "native_image_patches" }) |name| + @field(slot.view, name) = @field(p, name); + slot.view.col_tags = @splat(.{ .what = .column, .gpa = p.gpa }); + for (p.panes, 0..) |s, id| { + const pane = s orelse continue; + slot.serials[id] = pane.serial; + pane.pv[k] = fresh(pane); + pane.pv_known |= @as(u8, 1) << @intCast(k); + } + p.views_open |= @as(u8, 1) << @intCast(k); + return k; +} + +/// View `k` gone, and all it held; the live one moves to another first. +/// The last view stays: a session without a client keeps its own. +pub fn close(p: *Pardes, k: u8) void { + if (!isOpen(p, k) or count(p) == 1) return; + if (k == p.view) { + const other: u8 = for (0..max) |i| { + if (i != k and isOpen(p, @intCast(i))) break @intCast(i); + } else unreachable; + park(p); + unpark(p, other); + } + const v = &p.parked[k].?.view; + for (&v.col_tags) |*t| t.deinit(p.gpa); + if (v.chord_arg) |a| p.gpa.free(a); + v.presentation.deinit(p.gpa); + for (p.panes) |s| if (s) |pane| { + const sels = pane.pv[k].body.sels; + if (sels.len > 0) p.gpa.free(sels); + pane.pv[k].body.sels = &.{}; + pane.pv_known &= ~(@as(u8, 1) << @intCast(k)); + }; + p.views_open &= ~(@as(u8, 1) << @intCast(k)); +} + +/// The session's end: what each view not live holds. +pub fn deinit(p: *Pardes) void { + for (0..max) |i| { + const k: u8 = @intCast(i); + const slot = p.parked[k] orelse continue; + if (k != p.view and isOpen(p, k)) { + const v = &slot.view; + for (&v.col_tags) |*t| t.deinit(p.gpa); + if (v.chord_arg) |a| p.gpa.free(a); + v.presentation.deinit(p.gpa); + } + p.gpa.destroy(slot); + } +} + +/// What drawing a view keeps of its own (its animations, its panels as +/// presented), kept from a frame drawn while it was out. +const drawn_fields = [_][:0]const u8{ "presentation", "lifts", "lift_serials", "cursor_glide", "cursor_view", "cursor_crossing", "cursor_held", "shell_continuous" }; + +/// Each view but the live one drawn and presented, the live one put back +/// after exactly as it was: no digested field moves, so a replica that +/// draws nothing (and skips this) stays the same. A view drawn keeps only +/// what its drawing made (`drawn_fields`); whatever its unpark brought +/// inside a changed text is not kept, and is done again when it is next +/// live. Before the live one's frame: the surface after the pump is its. +pub fn drawOthers(p: *Pardes, h: pardes.Host) !void { + const live = p.view; + const v = h.vtable; + park(p); + defer { + copy(&fields, p, &p.parked[live].?.view); + p.view = live; + for (p.panes) |s| if (s) |pane| { + const pv = &pane.pv[live]; + copy(&pane_fields, pane, pv); + copy(&body_fields, &pane.body, &pv.body); + if (!pane.isTerminal()) { + pane.cols = pv.cols; + pane.rows = pv.rows; + } + if (pane.file) |*f| f.scroll = pv.scroll; + }; + } + for (0..max) |i| { + const k: u8 = @intCast(i); + if (k == live or !isOpen(p, k)) continue; + unpark(p, k); + _ = p.frame_arena.reset(.{ .retain_with_limit = pardes.memory.limits.frame_kept }); + const surface = try p.render(p.frame_arena.allocator()); + if (v.present) |f| f(h.ctx, surface); + if (v.post_present) |f| f(h.ctx); + copy(&drawn_fields, &p.parked[k].?.view, p); + } +} + +// ---- tests ---- + +const testing = std.testing; +const Key = pardes.Key; + +fn typeKeys(p: *Pardes, text: []const u8) void { + for (text) |c| p.update(.{ .key = .{ .cp = c, .text = &[_]u8{c} } }); +} + +fn escape(p: *Pardes) void { + p.update(.{ .key = .{ .cp = Key.escape } }); +} + +fn numbered(gpa: std.mem.Allocator, n: usize) ![]u8 { + var out: std.ArrayList(u8) = .empty; + for (1..n + 1) |i| try out.print(gpa, "line {d}\n", .{i}); + return out.toOwnedSlice(gpa); +} + +/// The cursor put on `row`, as a jump puts it, and the step after. +fn cursorTo(p: *Pardes, pane: *Pane, row: i32) void { + pane.body.vsel.active = false; + pane.body.nsel = 0; + pane.body.cur_row = row; + pane.body.cur_col = 0; + pane.body.cur_pinned = true; + pane.ensureCursorVisible(); + p.sync(); +} + +test "each view has its own cursor and scroll, and an edit in one is in the other" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 80, .rows = 24 }); + defer p.deinit(); + const text = try numbered(gpa, 200); + defer gpa.free(text); + const pane = try p.setTestFile(text); + try testing.expectEqual(@as(?u8, 1), p.openView()); + cursorTo(p, pane, 119); + const scrolled = pane.file.?.scroll; + try testing.expect(scrolled > 0); + // The other view is where the session was when it came. + p.toView(1); + try testing.expectEqual(@as(i32, 0), pane.body.cur_row); + try testing.expectEqual(@as(usize, 0), pane.file.?.scroll); + typeKeys(p, "ix"); + escape(p); + try testing.expectEqual(Text.Mode.normal, pane.body.mode); + p.toView(0); + try testing.expectEqual(@as(i32, 119), pane.body.cur_row); + try testing.expectEqual(scrolled, pane.file.?.scroll); + try testing.expect(std.mem.startsWith(u8, pane.file.?.content, "xline 1\n")); + // Its mode is its own: insert here leaves the other in normal. + typeKeys(p, "i"); + try testing.expectEqual(Text.Mode.insert, pane.body.mode); + p.toView(1); + try testing.expectEqual(Text.Mode.normal, pane.body.mode); +} + +test "each view has its own layout and focus over the same panes" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 100, .rows = 30 }); + defer p.deinit(); + _ = try p.setTestFile("one\ntwo\n"); + const k = p.openView().?; + const ncol = p.ncol; + p.toView(k); + p.update(.{ .command = "Newcol" }); + p.update(.{ .command = "New" }); + const made = p.active; + try testing.expect(made != 0); + try testing.expectEqual(ncol + 1, p.ncol); + p.toView(0); + try testing.expectEqual(ncol, p.ncol); + try testing.expectEqual(@as(usize, 0), p.active); + try testing.expect(layout.findPane(p, made) == null); + try testing.expectEqual(pardes.Rect{}, p.rects[made]); + // The pane is the session's: focused here, this view shows it too. + p.active = made; + p.sync(); + try testing.expect(layout.findPane(p, made) != null); + try testing.expect(p.rects[made].h > 0); + // Closed here, it leaves the other view's layout, and its keyboard. + try p.removePane(made, null); + p.sync(); + p.toView(k); + try testing.expect(layout.findPane(p, made) == null); + try testing.expect(p.panes[p.active] != null); + try testing.expect(layout.findPane(p, p.active) != null); +} + +test "registers are the session's: yanked in one view, put in another" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 80, .rows = 24 }); + defer p.deinit(); + const pane = try p.setTestFile("alpha\nbeta\n"); + const k = p.openView().?; + typeKeys(p, "xy"); + p.toView(k); + cursorTo(p, pane, 1); + typeKeys(p, "p"); + try testing.expectEqualStrings("alpha\nbeta\nalpha\n", pane.file.?.content); +} + +test "a terminal shown at two sizes takes the size of the view that typed into it last" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 80, .rows = 24 }); + defer p.deinit(); + const term = p.panes[p.active].?; + try testing.expect(term.isTerminal()); + const small = .{ term.cols, term.rows }; + const k = p.openView().?; + p.toView(k); + p.update(.{ .resize = .{ .cols = 120, .rows = 40 } }); + // Shown bigger here, it keeps the size of the view that typed last. + try testing.expect(p.rects[p.active].w > 80); + try testing.expectEqual(small, .{ term.cols, term.rows }); + p.update(.{ .key = .{ .cp = Key.right } }); + try testing.expect(term.cols > small[0] and term.rows > small[1]); + const big = .{ term.cols, term.rows }; + p.toView(0); + p.sync(); + try testing.expectEqual(big, .{ term.cols, term.rows }); + p.update(.{ .key = .{ .cp = Key.right } }); + try testing.expectEqual(small, .{ term.cols, term.rows }); + // That view gone, the one left sizes it. + p.toView(k); + p.closeView(0); + try testing.expectEqual(k, p.view); + try testing.expectEqual(big, .{ term.cols, term.rows }); +} + +test "a view closing leaves the others as they were" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 100, .rows = 30 }); + defer p.deinit(); + const text = try numbered(gpa, 100); + defer gpa.free(text); + const pane = try p.setTestFile(text); + const a = p.openView().?; + const b = p.openView().?; + try testing.expectEqual(@as(usize, 3), count(p)); + p.toView(b); + p.update(.{ .command = "Newcol" }); + cursorTo(p, pane, 40); + typeKeys(p, "xy"); + p.toView(a); + cursorTo(p, pane, 70); + p.toView(0); + cursorTo(p, pane, 10); + const ncol = p.ncol; + // The live one closing: the session goes into another. + p.toView(b); + p.closeView(b); + try testing.expect(p.view != b); + try testing.expectEqual(@as(usize, 2), count(p)); + p.toView(a); + try testing.expectEqual(@as(i32, 70), pane.body.cur_row); + p.closeView(0); + try testing.expectEqual(a, p.view); + try testing.expectEqual(ncol, p.ncol); + // The last stays. + p.closeView(a); + try testing.expectEqual(@as(usize, 1), count(p)); + try testing.expectEqual(@as(i32, 70), pane.body.cur_row); +} + +test "drawing the other views leaves the live one as it was, to the digest" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 80, .rows = 24 }); + defer p.deinit(); + const text = try numbered(gpa, 100); + defer gpa.free(text); + const pane = try p.setTestFile(text); + const k = p.openView().?; + p.toView(k); + cursorTo(p, pane, 60); + typeKeys(p, "dd"); + p.toView(0); + cursorTo(p, pane, 95); + const before = pardes.walk.digest(Pardes, p); + try drawOthers(p, .{}); + try testing.expectEqual(@as(u8, 0), p.view); + try testing.expectEqual(before, pardes.walk.digest(Pardes, p)); + p.toView(k); + try testing.expectEqual(@as(i32, 60), pane.body.cur_row); +} + +test "a view not live keeps its place in a text another view edits" { + const gpa = testing.allocator; + const p = try Pardes.init(gpa, .{ .tty_only = true, .cols = 80, .rows = 24 }); + defer p.deinit(); + const text = try numbered(gpa, 200); + defer gpa.free(text); + const pane = try p.setTestFile(text); + const k = p.openView().?; + p.toView(k); + cursorTo(p, pane, 100); + const scrolled = pane.file.?.scroll; + p.toView(0); + // Three lines put above it, one taken below it. + typeKeys(p, "xyppp"); + cursorTo(p, pane, 150); + typeKeys(p, "xd"); + p.toView(k); + try testing.expectEqual(@as(i32, 103), pane.body.cur_row); + try testing.expectEqual(scrolled + 3, pane.file.?.scroll); + try testing.expect(std.mem.startsWith(u8, pardes.panes.File.sourceLine(pane, 103), "line 101")); +} diff --git a/test/daemon.py b/test/daemon.py index 9e9ea600..04455f0f 100644 --- a/test/daemon.py +++ b/test/daemon.py @@ -208,6 +208,38 @@ def main(binary, cycles): assert now == baseline, (cycle, now, baseline) print(f'daemon: {cycles} detach/attach cycles, fds/sockets/processes level: {baseline}') + # Another terminal while one is attached: a view of its own + # (src/views.zig), drawn beside the first. Detached from inside, its + # stub ends and the first draws on; closed outright, the same. + for how in ('detach', 'close'): + term2 = Term(20, 60) + stub2 = term2.spawn([binary, '--attach=work'], env) + started.append(stub2.pid) + wait_for('the second terminal\'s first frame', lambda: b'\x1b[?1049h' in term2.text()) + wait_for('the second terminal\'s view live', lambda: client.screen()['cols'] == 60) + seen = len(term.text()) + run(client, shell, f'echo both-drawn-{how}\n'.encode()) + wait_for('the first terminal drawn', lambda: f'both-drawn-{how}'.encode() in term.text()[seen:]) + wait_for('the second terminal drawn', lambda: f'both-drawn-{how}'.encode() in term2.text()) + assert stub.poll() is None and stub2.poll() is None + if how == 'detach': + # Its view is the live one (its terminal spoke last). + client.write(f'/pane/{shell}/exec', b'Detach\n') + assert stub2.wait(timeout=10) == 0, stub2.returncode + assert b'detached from \'work\'' in term2.text(), term2.text()[-400:] + term2.close() + else: + term2.close() + assert stub2.wait(timeout=10) != 0 or True + wait_for('the first terminal\'s view live again', lambda: client.screen()['cols'] == 100) + seen = len(term.text()) + run(client, shell, f'echo first-on-{how}\n'.encode()) + wait_for('the first terminal drawn on', lambda: f'first-on-{how}'.encode() in term.text()[seen:]) + assert stub.poll() is None, 'the first terminal stays attached' + now = (fds(session), fds(daemon), len(list(root.iterdir())), len([p for p in processes(root) if alive(p)])) + assert now[:3] == baseline[:3], (how, now, baseline) + print('daemon: a second terminal on an attached session drawn beside the first, then detached and closed; the first drew on') + # Killed outright: its shells live on, their masters the daemon's. os.kill(session, signal.SIGKILL) stub.wait(timeout=10) diff --git a/test/e2e_harness.zig b/test/e2e_harness.zig index 7e29476b..6011ba16 100644 --- a/test/e2e_harness.zig +++ b/test/e2e_harness.zig @@ -12,6 +12,14 @@ extern "c" fn forkpty( termp: ?*const anyopaque, winp: ?*const posix.winsize, ) c_int; +extern "c" fn openpty( + amaster: *c_int, + aslave: *c_int, + name: ?[*]u8, + termp: ?*const anyopaque, + winp: ?*const posix.winsize, +) c_int; +extern "c" fn ttyname(fd: c_int) ?[*:0]const u8; extern "c" fn execvp(file: [*:0]const u8, argv: [*:null]const ?[*:0]const u8) c_int; extern "c" fn _exit(status: c_int) noreturn; extern "c" fn unsetenv(name: [*:0]const u8) c_int; @@ -72,6 +80,12 @@ pub const Harness = struct { /// clipboard writes). Capture is bounded explicitly so a runaway child cannot /// consume unbounded test-runner memory. raw: *RawCapture, + /// Another view's terminal of the app this harness started (`view`): + /// it neither started the app nor kills it. + view: bool = false, + /// Every terminal of the app's, this one's too: each wait reads them + /// all, or the app would block writing a frame to one nobody reads. + views: []const *Harness = &.{}, pub fn init(gpa: std.mem.Allocator, exe: [*:0]const u8, rows: u16, cols: u16) !Harness { return initArgs(gpa, exe, rows, cols, null); @@ -129,8 +143,59 @@ pub const Harness = struct { }; } + /// A terminal for another view of the app `pid` (views.zig): a pty of + /// its own, whose path the app is given (PARDES_TEST_VIEWS) and opens. + pub fn initView(gpa: std.mem.Allocator, rows: u16, cols: u16, path: *[64]u8) !Harness { + var master: c_int = undefined; + var slave: c_int = undefined; + const ws = posix.winsize{ .row = rows, .col = cols, .xpixel = 0, .ypixel = 0 }; + if (openpty(&master, &slave, null, null, &ws) != 0) return error.NoPty; + defer _ = libc.close(slave); + const name = std.mem.span(ttyname(slave) orelse return error.NoPty); + if (name.len >= path.len) return error.NoPty; + @memcpy(path[0..name.len], name); + path[name.len] = 0; + const term = try gpa.create(ghostty_vt.Terminal); + errdefer gpa.destroy(term); + term.* = try ghostty_vt.Terminal.init(std.Io.Threaded.global_single_threaded.io(), gpa, .{ + .cols = cols, + .rows = rows, + .max_scrollback_bytes = 1024 * 1024, + }); + errdefer term.deinit(gpa); + const raw = try gpa.create(RawCapture); + raw.* = .{}; + return .{ .gpa = gpa, .master = master, .pid = 0, .term = term, .stream = term.vtStream(), .rows = rows, .cols = cols, .raw = raw, .view = true }; + } + + /// What the app wrote to its other terminals, read without waiting. + fn drainOthers(self: *Harness) void { + for (self.views) |other| if (other != self) while (other.pumpRead(0)) {}; + } + + fn pumpRead(self: *Harness, ms: i64) bool { + var fds = [_]posix.pollfd{.{ .fd = self.master, .events = posix.POLL.IN, .revents = 0 }}; + const ready = posix.poll(&fds, @intCast(@max(0, ms))) catch 0; + if (ready == 0 or (fds[0].revents & posix.POLL.IN) == 0) return false; + var buf: [4096]u8 = undefined; + const n = posix.read(self.master, &buf) catch return false; + if (n == 0) return false; + self.recordRaw(buf[0..n]); + self.feed(buf[0..n]); + return true; + } + + /// Bytes in every terminal of the app's (`pending`). + pub fn pendingAll(self: *Harness) usize { + var n = self.pending(); + for (self.views) |other| if (other != self) { + n += other.pending(); + }; + return n; + } + pub fn deinit(self: *Harness) void { - posix.kill(self.pid, posix.SIG.KILL) catch {}; + if (!self.view) posix.kill(self.pid, posix.SIG.KILL) catch {}; _ = libc.close(self.master); self.stream.deinit(); self.gpa.destroy(self.raw); @@ -175,8 +240,9 @@ pub const Harness = struct { const deadline = nowMs() + ms; var buf: [4096]u8 = undefined; while (nowMs() < deadline) { + self.drainOthers(); var fds = [_]posix.pollfd{.{ .fd = self.master, .events = posix.POLL.IN, .revents = 0 }}; - _ = posix.poll(&fds, 50) catch {}; + _ = posix.poll(&fds, if (self.views.len > 1) 5 else 50) catch {}; if ((fds[0].revents & posix.POLL.IN) != 0) { const n = posix.read(self.master, &buf) catch break; if (n == 0) break; @@ -193,15 +259,14 @@ pub const Harness = struct { /// SETTLED grid sleeps in poll until the app actually writes, instead of /// burning a fixed window per check (see snapshot.zig's waitStable). pub fn pumpOnce(self: *Harness, ms: i64) !bool { - var fds = [_]posix.pollfd{.{ .fd = self.master, .events = posix.POLL.IN, .revents = 0 }}; - const ready = posix.poll(&fds, @intCast(@max(0, ms))) catch 0; - if (ready == 0 or (fds[0].revents & posix.POLL.IN) == 0) return false; - var buf: [4096]u8 = undefined; - const n = posix.read(self.master, &buf) catch return false; - if (n == 0) return false; - self.recordRaw(buf[0..n]); - self.feed(buf[0..n]); - return true; + if (self.views.len <= 1) return self.pumpRead(ms); + // Several terminals: each looked at every few milliseconds. + const deadline = nowMs() + ms; + while (true) { + self.drainOthers(); + if (self.pumpRead(@min(5, @max(0, deadline - nowMs())))) return true; + if (nowMs() >= deadline) return false; + } } /// Bytes sitting in the pty that nobody has picked up — keystrokes the app @@ -261,8 +326,9 @@ pub const Harness = struct { const deadline = nowMs() + ms; var buf: [4096]u8 = undefined; while (nowMs() < deadline) { + self.drainOthers(); var fds = [_]posix.pollfd{.{ .fd = self.master, .events = posix.POLL.IN, .revents = 0 }}; - _ = posix.poll(&fds, 50) catch {}; + _ = posix.poll(&fds, if (self.views.len > 1) 5 else 50) catch {}; if ((fds[0].revents & posix.POLL.IN) != 0) { const n = posix.read(self.master, &buf) catch break; if (n == 0) break; diff --git a/test/monkey.zig b/test/monkey.zig index 7c9761ff..d35599ff 100644 --- a/test/monkey.zig +++ b/test/monkey.zig @@ -1,6 +1,6 @@ // pardes-monkey: random snapshot scripts against the tty, hunting panics. // -// zig build monkey -Dplatform=tty -- [steps] [--from=N] [--out=DIR] [--keep] +// zig build monkey -Dplatform=tty -- [steps] [--from=N] [--out=DIR] [--keep] [--views=N] // // Each seed makes one script (a fixed seed makes the same script, so a hit // replays): keys and leader bursts, typing, mouse presses, drags and wheel, @@ -9,7 +9,9 @@ // /tmp/pardes-monkey), and a panic is a record in the app's crash file // (src/crash.zig, in the script's fake home: found even when the crash cut // the script short) or a `.zig:N:N: 0x` stack frame in the report or the -// grid it captured (a trace renders INTO the screen). A hit +// grid it captured (a trace renders INTO the screen). --views=N gives the +// session N terminals, each a view of it (views.zig), and types into and +// sizes each in turn, a `view` switch among the steps. A hit // keeps DIR/seed-N.snap, its captures and its report; the rest are deleted. // A panic's trace can scroll off a small grid: replay the kept script, or // run the app in a larger pty. Every hit gets a fix and a pinned regression @@ -26,6 +28,9 @@ const Script = struct { gpa: std.mem.Allocator, rows: u16 = 24, cols: u16 = 80, + /// Each terminal's size, and the one typed into (`--views`). + sizes: [4][2]u16 = @splat(.{ 24, 80 }), + view: usize = 0, fn line(s: *Script, comptime fmt: []const u8, args: anytype) !void { try s.out.print(s.gpa, fmt ++ "\n", args); @@ -38,18 +43,24 @@ const commands = [_][]const u8{ "Newcol", "Find", "Grep", "Joincol", "Help", "Du const leader_keys = "abcdefghijklmnopqrstuvwxyz0123456789$^"; const typed = "abcdefghijklmnopqrstuvwxyz0123456789 -_=+,:;()"; -fn generate(gpa: std.mem.Allocator, seed: u64, steps: usize) ![]u8 { +fn generate(gpa: std.mem.Allocator, seed: u64, steps: usize, views: usize) ![]u8 { var prng = std.Random.DefaultPrng.init(seed); const random = prng.random(); var s: Script = .{ .gpa = gpa }; - try s.line("# pardes-monkey seed {d}, {d} steps", .{ seed, steps }); + try s.line("# pardes-monkey seed {d}, {d} steps, {d} views", .{ seed, steps, views }); try s.line("file bad.txt hello \\xff\\xfe world\\nlatin-1 caf\\xe9 tail\\ntruncated \\xe6\\x97 here\\nend\\n", .{}); try s.line("file wide.txt {s}tail\\nsecond line\\n", .{std.mem.asBytes(&@as([8][37]u8, @splat("\\xe6\\x97\\xa5\\xe6\\x9c\\xac\\xe8\\xaa\\x9e ".*)))}); try s.line("lines long.txt 400 some words after the number, long enough to wrap in a narrow pane", .{}); - try s.line("start 24 80 bad.txt", .{}); + if (views > 1) try s.line("start 24 80 views={d} bad.txt", .{views}) else try s.line("start 24 80 bad.txt", .{}); try s.line("wait 8000 Newcol Joincol", .{}); try s.line("stable 300 5000", .{}); for (0..steps) |step| { + if (views > 1 and random.uintLessThan(u8, 4) == 0) { + s.sizes[s.view] = .{ s.rows, s.cols }; + s.view = random.uintLessThan(usize, views); + s.rows, s.cols = s.sizes[s.view]; + try s.line("view {d}", .{s.view + 1}); + } const wide = s.cols >= 70 and s.rows >= 6; switch (random.uintLessThan(u8, 12)) { // A workspace word, run with a middle click: new columns make @@ -150,9 +161,9 @@ test "a panic trace is found, and ordinary colons are not" { } test "a seed makes the same script every time" { - const a = try generate(std.testing.allocator, 42, 50); + const a = try generate(std.testing.allocator, 42, 50, 3); defer std.testing.allocator.free(a); - const b = try generate(std.testing.allocator, 42, 50); + const b = try generate(std.testing.allocator, 42, 50, 3); defer std.testing.allocator.free(b); try std.testing.expectEqualStrings(a, b); } @@ -166,7 +177,7 @@ pub fn main(init: std.process.Init) !void { const io = init.io; const args = try init.minimal.args.toSlice(init.arena.allocator()); if (args.len < 4) { - std.debug.print("usage: pardes-monkey [steps] [--from=N] [--out=DIR] [--keep] [--tape=DIR]\n", .{}); + std.debug.print("usage: pardes-monkey [steps] [--from=N] [--out=DIR] [--keep] [--tape=DIR] [--views=N]\n", .{}); std.process.exit(2); } const snap = args[1]; @@ -180,9 +191,12 @@ pub fn main(init: std.process.Init) !void { // --tape=DIR: each seed's sessions record their tapes there and the // seed is a hit if one does not replay (pardes-snap --tape). var tape: ?[]const u8 = null; + var views: usize = 1; var positional: usize = 0; for (args[3..]) |a| { - if (std.mem.startsWith(u8, a, "--from=")) { + if (std.mem.startsWith(u8, a, "--views=")) { + views = std.math.clamp(try std.fmt.parseInt(usize, a["--views=".len..], 10), 1, 4); + } else if (std.mem.startsWith(u8, a, "--from=")) { from = try std.fmt.parseInt(u64, a["--from=".len..], 10); } else if (std.mem.eql(u8, a, "--keep")) { keep = true; @@ -203,7 +217,7 @@ pub fn main(init: std.process.Init) !void { var hits: usize = 0; var seed = from; while (seed < from + seeds) : (seed += 1) { - const script = try generate(gpa, seed, steps); + const script = try generate(gpa, seed, steps, views); defer gpa.free(script); var path_buf: [4096]u8 = undefined; const path = try std.fmt.bufPrint(&path_buf, "{s}/seed-{d}.snap", .{ out, seed }); diff --git a/test/snapshot.zig b/test/snapshot.zig index 255f55bf..8a0583e3 100644 --- a/test/snapshot.zig +++ b/test/snapshot.zig @@ -29,7 +29,11 @@ // lines [tail] create file with n numbered lines, tail on each // dirmk create a subdirectory // run run a command in the script's cwd (NOT in a pane) -// start [arg] fork the app in a pty (optional extra CLI arg) +// start [views=N] [arg] fork the app in a pty (optional +// extra CLI arg); views=N gives it N-1 more +// terminals, each a view of the session +// view type into and capture terminal k (1 is the +// app's own; a resize there SIGWINCHes the app) // wait pump until needle appears on the grid (fails hard) // settle pump for a fixed duration // stable pump until the grid stops changing @@ -74,6 +78,7 @@ const Client = @import("9p_io").Client; pub const std_options: std.Options = .{ .log_level = .err }; extern "c" fn setenv(name: [*:0]const u8, value: [*:0]const u8, overwrite: c_int) c_int; +extern "c" fn unsetenv(name: [*:0]const u8) c_int; extern "c" fn execvp(file: [*:0]const u8, argv: [*:null]const ?[*:0]const u8) c_int; extern "c" fn system(cmd: [*:0]const u8) c_int; extern "c" fn mkdtemp(template: [*:0]u8) ?[*:0]u8; @@ -652,6 +657,14 @@ fn runScript(arena: std.mem.Allocator, exe_z: [:0]const u8, script_path: []const var chunk_len: usize = 0; var h: ?eh.Harness = null; defer if (h) |*hp| hp.deinit(); + // `views=N`: the app's other terminals, and the one driven now. + var others: [3]eh.Harness = undefined; + var nothers: usize = 0; + var group: [4]*eh.Harness = undefined; + var cur: usize = 0; + defer for (others[0..nothers]) |*o| o.deinit(); + // A capture is a delta against the previous one of its terminal's. + var prevs: [4]Prev = @splat(.{}); // captures are deltas against the previous one of the same kind (see Prev) var prev: Prev = .{}; @@ -717,15 +730,46 @@ fn runScript(arena: std.mem.Allocator, exe_z: [:0]const u8, script_path: []const const cols = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); var extra: [6]?[*:0]const u8 = @splat(null); var nextra: usize = 0; - while (tok.next()) |a| : (nextra += 1) { + var nviews: usize = 1; + while (tok.next()) |a| { + if (std.mem.startsWith(u8, a, "views=")) { + nviews = try std.fmt.parseInt(usize, a["views=".len..], 10); + if (nviews < 1 or nviews > 4) return error.BadScript; + continue; + } if (nextra >= extra.len) return error.BadScript; extra[nextra] = (try arena.dupeSentinel(u8, a, 0)).ptr; + nextra += 1; } if (h) |*hp| hp.deinit(); // scripts may restart (e.g. dump then -l) + for (others[0..nothers]) |*o| o.deinit(); + nothers = 0; + cur = 0; // a new process is a new screen: capture it whole, or its first // frame would be a delta against a dead instance's rows prev = .{}; + prevs = @splat(.{}); + var paths: std.ArrayList(u8) = .empty; + while (nothers + 1 < nviews) : (nothers += 1) { + var path: [64]u8 = undefined; + others[nothers] = try eh.Harness.initView(gpa, rows, cols, &path); + try paths.print(arena, "{s}{s}", .{ if (paths.items.len > 0) " " else "", std.mem.sliceTo(&path, 0) }); + } + if (nothers > 0) _ = setenv("PARDES_TEST_VIEWS", try arena.dupeSentinel(u8, paths.items, 0), 1); h = try eh.Harness.initArgv(gpa, exe_z.ptr, rows, cols, extra[0..nextra]); + _ = unsetenv("PARDES_TEST_VIEWS"); + group[0] = &h.?; + for (others[0..nothers], 1..) |*o, i| { + o.pid = h.?.pid; + group[i] = o; + } + for (group[0 .. nothers + 1]) |v| v.views = group[0 .. nothers + 1]; + } else if (std.mem.eql(u8, cmd, "view")) { + const k = try std.fmt.parseInt(usize, tok.next() orelse return error.BadScript, 10); + if (k < 1 or k > nothers + 1) return error.BadScript; + prevs[cur] = prev; + cur = k - 1; + prev = prevs[cur]; } else if (std.mem.eql(u8, cmd, "snapfile")) { const label = tok.next() orelse return error.BadScript; const contents = try readFileAlloc(arena, tok.rest()); @@ -735,49 +779,52 @@ fn runScript(arena: std.mem.Allocator, exe_z: [:0]const u8, script_path: []const try appendChunk(chunks, &chunk_len, try snapFile(arena, label, contents)); } else if (std.mem.eql(u8, cmd, "wait")) { const ms = try std.fmt.parseInt(i64, tok.next() orelse return error.BadScript, 10); - try live(&h).expectWaitFor(tok.rest(), ms, "snapshot wait"); + try on(&h, &group, nothers, cur).expectWaitFor(tok.rest(), ms, "snapshot wait"); } else if (std.mem.eql(u8, cmd, "settle")) { - try live(&h).pump(try std.fmt.parseInt(i64, tok.next() orelse return error.BadScript, 10)); + try on(&h, &group, nothers, cur).pump(try std.fmt.parseInt(i64, tok.next() orelse return error.BadScript, 10)); } else if (std.mem.eql(u8, cmd, "stable")) { const quiet = try std.fmt.parseInt(i64, tok.next() orelse return error.BadScript, 10); const timeout = try std.fmt.parseInt(i64, tok.next() orelse return error.BadScript, 10); - try waitStable(live(&h), quiet, timeout); + try waitStable(on(&h, &group, nothers, cur), quiet, timeout); } else if (std.mem.eql(u8, cmd, "text")) { - try live(&h).send(tok.rest()); + try on(&h, &group, nothers, cur).send(tok.rest()); } else if (std.mem.eql(u8, cmd, "send")) { - try live(&h).send(try unescape(arena, tok.rest())); + try on(&h, &group, nothers, cur).send(try unescape(arena, tok.rest())); } else if (std.mem.eql(u8, cmd, "key")) { - while (tok.next()) |name| try live(&h).send(keyBytes(name) orelse return error.BadKey); + while (tok.next()) |name| try on(&h, &group, nothers, cur).send(keyBytes(name) orelse return error.BadKey); } else if (std.mem.eql(u8, cmd, "press") or std.mem.eql(u8, cmd, "release") or std.mem.eql(u8, cmd, "drag")) { const btn = buttonCode(tok.next() orelse return error.BadScript) orelse return error.BadScript; const col_spec = tok.next() orelse return error.BadScript; const row = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); - const col = try resolveCol(live(&h), col_spec, row); + const col = try resolveCol(on(&h, &group, nothers, cur), col_spec, row); const b: u16 = if (std.mem.eql(u8, cmd, "drag")) btn + 32 else btn; const fin: u8 = if (std.mem.eql(u8, cmd, "release")) 'm' else 'M'; - try live(&h).send(try std.fmt.allocPrint(arena, "\x1b[<{d};{d};{d}{c}", .{ b, col, row, fin })); + try on(&h, &group, nothers, cur).send(try std.fmt.allocPrint(arena, "\x1b[<{d};{d};{d}{c}", .{ b, col, row, fin })); } else if (std.mem.eql(u8, cmd, "motion")) { const col_spec = tok.next() orelse return error.BadScript; const row = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); - const col = try resolveCol(live(&h), col_spec, row); - try live(&h).send(try std.fmt.allocPrint(arena, "\x1b[<35;{d};{d}M", .{ col, row })); + const col = try resolveCol(on(&h, &group, nothers, cur), col_spec, row); + try on(&h, &group, nothers, cur).send(try std.fmt.allocPrint(arena, "\x1b[<35;{d};{d}M", .{ col, row })); } else if (std.mem.eql(u8, cmd, "wheel")) { const dir = tok.next() orelse return error.BadScript; const b: u16 = if (std.mem.eql(u8, dir, "up")) 64 else 65; const col_spec = tok.next() orelse return error.BadScript; const row = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); - const col = try resolveCol(live(&h), col_spec, row); - try live(&h).send(try std.fmt.allocPrint(arena, "\x1b[<{d};{d};{d}M", .{ b, col, row })); + const col = try resolveCol(on(&h, &group, nothers, cur), col_spec, row); + try on(&h, &group, nothers, cur).send(try std.fmt.allocPrint(arena, "\x1b[<{d};{d};{d}M", .{ b, col, row })); } else if (std.mem.eql(u8, cmd, "resize")) { const rows = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); const cols = try std.fmt.parseInt(u16, tok.next() orelse return error.BadScript, 10); - try live(&h).resize(rows, cols); + try on(&h, &group, nothers, cur).resize(rows, cols); + // Another view's terminal is no one's controlling one: the app + // is told it changed, as a stub tells it of its own. + if (cur > 0) std.posix.kill(live(&h).pid, std.posix.SIG.WINCH) catch {}; } else if (std.mem.eql(u8, cmd, "snap")) { - try appendChunk(chunks, &chunk_len, try snapText(arena, live(&h), tok.rest(), &prev)); + try appendChunk(chunks, &chunk_len, try snapText(arena, on(&h, &group, nothers, cur), tok.rest(), &prev)); } else if (std.mem.eql(u8, cmd, "snapstyle")) { - try appendChunk(chunks, &chunk_len, try snapStyle(arena, live(&h), tok.rest(), &prev)); + try appendChunk(chunks, &chunk_len, try snapStyle(arena, on(&h, &group, nothers, cur), tok.rest(), &prev)); } else if (std.mem.eql(u8, cmd, "snap9p")) { - try appendChunk(chunks, &chunk_len, try snap9p(arena, live(&h), tok.rest(), &prev)); + try appendChunk(chunks, &chunk_len, try snap9p(arena, on(&h, &group, nothers, cur), tok.rest(), &prev)); } else { std.debug.print("unknown command {s}\n", .{cmd}); return error.BadScript; @@ -891,6 +938,13 @@ fn snapFile(arena: std.mem.Allocator, label: []const u8, contents: []const u8) ! return out; } +/// The terminal the script drives now (`view`). +fn on(h: *?eh.Harness, group: *[4]*eh.Harness, nothers: usize, cur: usize) *eh.Harness { + const app = live(h); + if (cur == 0 or cur > nothers) return app; + return group[cur]; +} + fn live(h: *?eh.Harness) *eh.Harness { if (h.*) |*v| return v; fatal("script used the app before `start`", .{}); @@ -928,7 +982,7 @@ fn waitStable(h: *eh.Harness, quiet: i64, timeout: i64) !void { // and nothing in its process tree is runnable. Held across two probes, // because a pane's shell can write and sleep a scheduler tick before // the app wakes to read it. - if (since >= probe_after_ms and h.pending() == 0 and treeAsleep(h.pid)) { + if (since >= probe_after_ms and h.pendingAll() == 0 and treeAsleep(h.pid)) { if (asleep_since == 0) { asleep_since = now; } else if (now - asleep_since >= probe_hold_ms) { diff --git a/test/snapshots/views.golden b/test/snapshots/views.golden new file mode 100644 index 00000000..40f6a294 --- /dev/null +++ b/test/snapshots/views.golden @@ -0,0 +1,123 @@ +== snap one grid=80x24 cursor=7,3 +| Newcol Joincol Find Grep Help Changelog Tutor Dump Themes Config Debug Exit +| New Tty Find Grep Joincol Delcol +| /tmp/pardes-snap/views/cwd/v.txt Save Tty Collapse Del +| 1 line 1 +| 2 line 2 +| 3 line 3 +| 4 line 4 +| 5 line 5 +| 6 line 6 +| 7 line 7 +| 8 line 8 +| 9 line 9 +| 10 line 10 +| 11 line 11 +| 12 line 12 +| 13 line 13 +| 14 line 14 +| 15 line 15 +| 16 line 16 +| 17 line 17 +| 18 line 18 +| 19 line 19 +| 20 line 20 +| 21 line 21 +== snap two grid=80x24 cursor=7,3 +| Newcol Joincol Find Grep Help Changelog Tutor Dump Themes Config Debug Exit +| New Tty Find Grep Joincol Delcol +| /tmp/pardes-snap/views/cwd/v.txt Save Tty Collapse Del +| 1 line 1 +| 2 line 2 +| 3 line 3 +| 4 line 4 +| 5 line 5 +| 6 line 6 +| 7 line 7 +| 8 line 8 +| 9 line 9 +| 10 line 10 +| 11 line 11 +| 12 line 12 +| 13 line 13 +| 14 line 14 +| 15 line 15 +| 16 line 16 +| 17 line 17 +| 18 line 18 +| 19 line 19 +| 20 line 20 +| 21 line 21 +== snap one-edit grid=80x24 cursor=15,3 +|2: */tmp/pardes-snap/views/cwd/v.txt Save Tty Collapse Del +|3: 1 EDIT-ONEline 1 +== snap two-sees grid=80x24 cursor=7,3 +|2: */tmp/pardes-snap/views/cwd/v.txt Save Tty Collapse Del +|3: 1 EDIT-ONEline 1 +== snap two-down grid=80x24 cursor=7,6 +|3: 22 line 22 +|4: 23 line 23 +|5: 24 line 24 +|6: 25 line 25 +|7: 26 line 26 +|8: 27 line 27 +|9: 28 line 28 +|10: 29 line 29 +|11: 30 line 30 +|12: 31 line 31 +|13: 32 line 32 +|14: 33 line 33 +|15: 34 line 34 +|16: 35 line 35 +|17: 36 line 36 +|18: 37 line 37 +|19: 38 line 38 +|20: 39 line 39 +|21: 40 line 40 +|22: 41 line 41 +|23: 42 line 42 +== snap one-top grid=80x24 cursor=15,3 +== snap two-put grid=80x24 cursor=21,7 +|7: 26 EDIT-ONEline 1 +|8: 27 line 26 +|9: 28 line 27 +|10: 29 line 28 +|11: 30 line 29 +|12: 31 line 30 +|13: 32 line 31 +|14: 33 line 32 +|15: 34 line 33 +|16: 35 line 34 +|17: 36 line 35 +|18: 37 line 36 +|19: 38 line 37 +|20: 39 line 38 +|21: 40 line 39 +|22: 41 line 40 +|23: 42 line 41 +== snap two-col grid=80x24 cursor=42,1 +|1: New Tty Find Grep Joincol Delcol New Tty Find Grep Joincol Delcol +|2: */tmp/pardes-snap/views/cwd/v.txt Save +|3: Tty Collapse Del +|4: 22 line 22 +|5: 23 line 23 +|6: 24 line 24 +|7: 25 line 25 +|8: 26 EDIT-ONEline 1 +|9: 27 line 26 +|10: 28 line 27 +|11: 29 line 28 +|12: 30 line 29 +|13: 31 line 30 +|14: 32 line 31 +|15: 33 line 32 +|16: 34 line 33 +|17: 35 line 34 +|18: 36 line 35 +|19: 37 line 36 +|20: 38 line 37 +|21: 39 line 38 +|22: 40 line 39 +|23: 41 line 40 +== snap one-col grid=80x24 cursor=21,3 +== snap one-after grid=80x24 cursor=13,4 diff --git a/test/snapshots/views.snap b/test/snapshots/views.snap new file mode 100644 index 00000000..eba26bae --- /dev/null +++ b/test/snapshots/views.snap @@ -0,0 +1,57 @@ +# Two terminals on one session, each a view of it (src/views.zig): each its +# own layout, focus, cursor and scroll over the same panes. An edit typed in +# one is in the other; a line yanked in one is put in the other (registers +# are the session's); Newcol in one leaves the other's columns; Detach in +# one lets that terminal go and leaves the other drawing. +config Verbose off +lines v.txt 120 +start 24 80 views=2 v.txt +wait 8000 line 1 +stable 500 10000 +snap one +view 2 +wait 8000 line 1 +stable 500 10000 +snap two +# Typed in view 1: in view 2 too. +view 1 +key i +text EDIT-ONE +key esc +settle 100 +stable 400 8000 +snap one-edit +view 2 +stable 400 8000 +snap two-sees +# Scrolled in view 2: view 1 stays at the top. +key pgdn +stable 400 8000 +snap two-down +view 1 +stable 400 8000 +snap one-top +# Yanked in view 1, put in view 2. +key x y +stable 400 8000 +view 2 +key p +stable 400 8000 +snap two-put +# A column in view 2 only. +key space +text cn +stable 400 8000 +snap two-col +view 1 +stable 400 8000 +snap one-col +# Detached in view 2: its terminal let go, view 1 drawing on. +view 2 +key space +text sD +stable 400 8000 +view 1 +key j +stable 400 8000 +snap one-after -- cgit v1.3 From 3b57a72becd6dd6700d996c9c5c000e1cb53e076 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Thu, 29 Oct 2026 18:19:13 -0300 Subject: phase 5 part A: the speculation and rollback engine, in one process: a walker copy put back where it was, replicas that apply their own input at once and take the authority's tape as the truth, and a harness of typists over delayed links src/rollback.zig: the replica core's allocator (`Epoch`) knows every allocation, and while a copy is held puts off frees of what was there before and refuses to grow it in place, so what the core held stays where it was. The copy (`Copy`) is the digest's walk saving the bytes of every allocation it reaches, field by field (the derived caches and this process's own too: a safe build's free writes over what it frees, and the frame arena points at what the epoch then frees), but the queues a step drains and the frame statistics; a file's text, its line index and its undo snapshots are pinned instead (made anew at each edit, never written; now freed raw everywhere), checked unchanged in a safe build. Put back, each allocation's bytes go back over it, the epoch's allocations are freed and its put-off frees forgotten; let go, they are made. Tree-sitter's trees, the C heap's, are held the same way (syntax.held_trees). src/replica.zig: an `Authority` records its core's every input to a tape in memory and hands each client's input back as a unit (origin, sequence number, its tape); a `Replica` applies its own input at once, as the authority will (update, then pump), recorded by its own tape (tape.zig's speculation mode), and for each unit in order drops its pending head when the authority recorded the same (inputs, host calls and answers: the fast path), else puts the copy back, replays the units since and this one (every digest checked) and applies its pending inputs again. A host call or a syscall stops a speculation: it is answered harmlessly, undone when it returns, and the input waits for the authority's answer; a clock or environment read is a guess the comparison checks; an emulator is the authority's (terminal.zig's writers stop a speculation; a safe build asserts none changed). The view's events (a resize, the pointer) wait. Messages are bytes: no transport yet. Each client has a view of its own (src/views.zig): the authority goes into it before the client's input (`toView`, taped), the replica's speculation the same, and the copy walks the views not live too (`parked`, each pane's `pv`). test/replicas.zig (`zig build replicas`): one authority and N replicas, a view each (`--shared` puts them in one), each client typing monkey.zig's mix at 50-200 ms over its own link (half the RTT plus 10% jitter, in order), the authority's pty echoing 2 ms later; gates: no replica diverges, every replica's digest is the authority's at quiescence; reported per RTT: inputs applied at once, stopped, waiting, fast paths, rollbacks (mispredicted, foreign), and what a speculative apply, a copy and a rollback cost. 3 clients x 100 inputs at 30, 80 and 207 ms: no divergence over 12 Debug seeds and 20 ReleaseFast ones with a filled heap; ReleaseFast p99 a speculative apply under 0.2 ms, a rollback under 1.6 ms. `zig build rollback-bench` times the real copy now (take, release, an edit, restore). What the replicas found, fixed: a release build reads what the heap had where Debug's fill made two cores agree: `col_panes` and `restore_buf` digested undefined (zeroed now), `terminal.State.reply_dropped` never set at create, `Text.sels` and `PendingPipe.ranges` digested past their length (a walker `len` cuts a slice too). PARDES_REPLICA_FILL fills a replica's allocations to show what is left. Co-Authored-By: Claude Opus 5.5 --- build.zig | 10 + src/File.zig | 14 +- src/Pipe.zig | 2 + src/Text.zig | 6 +- src/memory.zig | 9 + src/pardes.zig | 11 +- src/replica.zig | 578 ++++++++++++++++++++++++++++++++++++++++++++++++ src/rollback.zig | 496 +++++++++++++++++++++++++++++++++++++++++ src/syntax.zig | 24 ++ src/tape.zig | 236 ++++++++++++++++++-- src/terminal.zig | 17 ++ src/walk.zig | 24 +- test/replicas.zig | 327 +++++++++++++++++++++++++++ test/rollback_bench.zig | 120 ++++------ 14 files changed, 1763 insertions(+), 111 deletions(-) create mode 100644 src/replica.zig create mode 100644 src/rollback.zig create mode 100644 test/replicas.zig (limited to 'src/tape.zig') diff --git a/build.zig b/build.zig index 13cb8ebc..1a5dd200 100644 --- a/build.zig +++ b/build.zig @@ -1504,6 +1504,16 @@ pub fn build(b: *std.Build) void { run_rollback_bench.has_side_effects = true; b.step("rollback-bench", "a replica's rollback copy (walked state, pinned blobs, walk time) for a typical and a big session, and a fork of the big one (-Doptimize=ReleaseFast)").dependOn(&run_rollback_bench.step); + const replicas = b.addExecutable(.{ + .name = "pardes-replicas", + .root_module = b.createModule(.{ .target = target, .optimize = optimize, .root_source_file = b.path("test/replicas.zig"), .link_libc = true }), + }); + replicas.root_module.addImport("pardes", core_module); + const run_replicas = b.addRunArtifact(replicas); + run_replicas.addPassthruArgs(); + run_replicas.has_side_effects = true; + b.step("replicas", "one authority and several replicas in process, each client typing over a delayed link: digests at quiescence, rollback rate and costs per RTT (-- [--seeds N] [--steps M] [--clients C] [--rtt 30,80,207])").dependOn(&run_replicas.step); + const locations_bench_step = b.step("locations-bench", "measure cold/warm result formatting (-Doptimize=ReleaseFast; CSV on stderr)"); if (optimize != .fast) { locations_bench_step.dependOn(&b.addFail("locations-bench requires -Doptimize=ReleaseFast").step); diff --git a/src/File.zig b/src/File.zig index dd6b2d34..84222191 100644 --- a/src/File.zig +++ b/src/File.zig @@ -1159,10 +1159,12 @@ pub fn deinit(p: *Pardes, pane: *Pane, file: *State) void { if (p.panes[id].?.file.?.line_starts.ptr == file.line_starts.ptr) break true; } else false; p.gpa.free(file.path); - if (ntwins == 0) p.gpa.free(file.content); + // Raw, as every text is freed (a replica's rollback copy pins them, + // src/rollback.zig, and a safe build's free writes over what it frees). + if (ntwins == 0) pardes.memory.freeUnfilled(p.gpa, file.content); if (file.listing) |listing| p.gpa.free(listing.names); if (file.mini) |*mini| mini.deinit(p.gpa); - if (file.line_starts.len > 0 and !index_shared) p.gpa.free(file.line_starts); + if (file.line_starts.len > 0 and !index_shared) pardes.memory.freeRaw(p.gpa, file.line_starts); if (file.highlights.len > 0) p.tree_sitter_gpa.free(file.highlights); p.gpa.free(file.pager_spans); locations.freeRows(p.gpa, file.location_rows); @@ -1264,7 +1266,7 @@ pub fn setContentSpan(p: *Pardes, f: *State, new: []u8, span_in: ?Span) void { }; // The clock the session that recorded read (src/tape.zig). f.mtime = pardes.tape.value(.mtime, pardes.ctlfs.events.now, .{}); - if (f.line_starts.len > 0) p.gpa.free(f.line_starts); + if (f.line_starts.len > 0) pardes.memory.freeRaw(p.gpa, f.line_starts); f.line_starts = starts; if (f.highlights.len > 0) p.tree_sitter_gpa.free(f.highlights); f.highlights = &.{}; @@ -1276,7 +1278,7 @@ pub fn setContentSpan(p: *Pardes, f: *State, new: []u8, span_in: ?Span) void { const tf = &t.file.?; // Its own index, built while the shared one was empty, goes; the // shared one, freed above, is replaced. - if (tf.line_starts.len > 0 and tf.line_starts.ptr != old_starts) p.gpa.free(tf.line_starts); + if (tf.line_starts.len > 0 and tf.line_starts.ptr != old_starts) pardes.memory.freeRaw(p.gpa, tf.line_starts); tf.content = f.content; tf.line_starts = f.line_starts; tf.revision = f.revision; @@ -1498,8 +1500,8 @@ pub fn relink(p: *Pardes, of: usize, id: usize, dot: ?[2]u32) void { sf.twin = p.next_twin; p.next_twin +%= 1; } - p.gpa.free(f.content); - if (f.line_starts.len > 0) p.gpa.free(f.line_starts); + pardes.memory.freeUnfilled(p.gpa, f.content); + if (f.line_starts.len > 0) pardes.memory.freeRaw(p.gpa, f.line_starts); f.history.destroy(p.gpa); f.content = sf.content; f.line_starts = sf.line_starts; diff --git a/src/Pipe.zig b/src/Pipe.zig index 850f9c64..239a94e4 100644 --- a/src/Pipe.zig +++ b/src/Pipe.zig @@ -52,6 +52,8 @@ pub fn pipeMarker(how: modal.Normal.PipeBehavior) []const u8 { } pub const PendingPipe = struct { + /// The digest's walk (src/walk.zig) of the ranges, as far as they go. + pub const replica = .{ .len = .{.{ "ranges", "nranges" }} }; id: u32, pane: usize, serial: u32, diff --git a/src/Text.zig b/src/Text.zig index 3f107542..da4be590 100644 --- a/src/Text.zig +++ b/src/Text.zig @@ -49,9 +49,9 @@ pub const SelRange = struct { sticky: i32 = -1, }; -/// The replay digest's walk (src/walk.zig) of the histories: as far as -/// they go. -pub const replica = .{ .len = .{ .{ "ed_undo", "ed_undo_len" }, .{ "ed_redo", "ed_redo_len" } } }; +/// The replay digest's walk (src/walk.zig) of the histories and the ranges: +/// as far as they go (the ranges' room is made whole and used `nsel` deep). +pub const replica = .{ .len = .{ .{ "ed_undo", "ed_undo_len" }, .{ "ed_redo", "ed_redo_len" }, .{ "sels", "nsel" } } }; what: What = .body, /// The characters of a text that holds its own (gpa-owned): everything but diff --git a/src/memory.zig b/src/memory.zig index 47bd755d..1d082964 100644 --- a/src/memory.zig +++ b/src/memory.zig @@ -421,6 +421,15 @@ pub fn freeUnfilled(allocator: Allocator, bytes: []const u8) void { if (bytes.len > 0) allocator.rawFree(@constCast(bytes), .of(u8), @returnAddress()); } +/// `freeUnfilled` for a slice of any element: a text's line index, made +/// anew at every edit too (and pinned by a replica's rollback copy, +/// src/rollback.zig, which a safe build's overwrite would spoil). +pub fn freeRaw(allocator: Allocator, items: anytype) void { + const E = @typeInfo(@TypeOf(items)).pointer.child; + const bytes = std.mem.sliceAsBytes(items); + if (bytes.len > 0) allocator.rawFree(@constCast(bytes), .of(E), @returnAddress()); +} + /// Gives free memory back to the system, the arenas' and libc's, inside its /// heap too: after a big result is formatted or a big pane torn down, the /// tree-sitter arena kept 73 MiB of emptied buckets resident and glibc what diff --git a/src/pardes.zig b/src/pardes.zig index 75e91ee9..5d889c86 100644 --- a/src/pardes.zig +++ b/src/pardes.zig @@ -65,6 +65,8 @@ pub const tape = @import("tape.zig"); pub const walk = @import("walk.zig"); pub const screen_state = if (terminal_panes) @import("screen_state.zig") else struct {}; pub const views = @import("views.zig"); +pub const rollback = @import("rollback.zig"); +pub const replica = @import("replica.zig"); pub const image = @import("image.zig"); pub const dump = @import("dump.zig"); pub const lsp = @import("lsp/lsp.zig"); @@ -474,6 +476,8 @@ pub const Loc = struct { }; test { + _ = @import("rollback.zig"); + _ = @import("replica.zig"); _ = @import("modal.zig"); _ = @import("tagline.zig"); _ = @import("hunks.zig"); @@ -4412,7 +4416,10 @@ pub const Pardes = struct { // layout: columns own x by weight; panes own y by vweight within a column. ncol: usize = 0, col_weight: [MAX_COLS]u64 = @splat(column_weight_unit), - col_panes: [MAX_COLS][MAX_PANES]usize = undefined, + // Zeroed, not undefined: a digest takes the whole of it, a release + // build's heap leaves its unused slots whatever was there before, and + // two replicas must agree on them (src/replica.zig). + col_panes: [MAX_COLS][MAX_PANES]usize = @splat(@splat(0)), col_n: [MAX_COLS]usize = @splat(0), /// Each column's tag and the workspace's: Texts like a pane's tag, with /// no prefix, showing a default until someone edits them. @@ -4698,7 +4705,7 @@ pub const Pardes = struct { /// a Restore builtin wants this dump loaded into the live session; the /// shell consumes it via takeRestore each frame (restore contents stay host-fed) restore_req: ?[]const u8 = null, - restore_buf: [1024]u8 = undefined, + restore_buf: [1024]u8 = @splat(0), surface: Surface = .{}, /// The view the fields above are of: whose layout, focus, pointer and diff --git a/src/replica.zig b/src/replica.zig new file mode 100644 index 00000000..42d94503 --- /dev/null +++ b/src/replica.zig @@ -0,0 +1,578 @@ +//! A session's replicas (docs/next-daemon.md, phase 5 part A): the authority +//! orders every input and records what its core consumed (src/tape.zig); +//! each replica runs a core of its own, applies its own inputs at once, and +//! takes the authority's stream as the truth. +//! +//! A replica's state is the confirmed position plus its own pending inputs, +//! applied on top (a speculation, recorded as the authority would record +//! it). For each unit of the authority's stream, in order: +//! - its own pending head, recorded the same (inputs, host calls and +//! answers): the head goes, nothing is redone (the fast path); +//! - else the rollback copy is put back (src/rollback.zig), the confirmed +//! units since it are replayed, then this one, and the pending inputs +//! are applied again. +//! A speculation that reaches a host call or a syscall (a Host function, +//! a `tape.site`) or an emulator (the authority's) stops: the copy is put +//! back and the input waits for the authority's answer, one RTT late but +//! never wrong. Clock and environment reads (`tape.value`) are made here, +//! and the comparison catches a guess the authority did not share. +//! +//! No transport: messages are bytes in and out. A client's input is its +//! id, a sequence number and the event as detached/wire.zig frames it; a +//! unit is the input's origin (0 for the authority's own: a pty's output, a +//! tick), its sequence number and the authority's tape of it. +const std = @import("std"); +const builtin = @import("builtin"); +const pardes = @import("pardes.zig"); +const tape = @import("tape.zig"); +const rollback = @import("rollback.zig"); +const walk = @import("walk.zig"); +const wire = @import("detached/wire.zig"); +const Allocator = std.mem.Allocator; +const Pardes = pardes.Pardes; + +const checked = builtin.mode == .debug or builtin.mode == .safe; +const head_len = 6; // origin or client: u16le, seq: u32le +/// Confirmed units fast-pathed under one copy before it is taken afresh: +/// a rollback replays them all. +const rebase_after = 32; + +fn header(buf: []u8, who: u16, seq: u32) void { + std.mem.writeInt(u16, buf[0..2], who, .little); + std.mem.writeInt(u32, buf[2..6], seq, .little); +} + +pub const Unit = struct { origin: u16, seq: u32, tape: []const u8 }; + +pub fn parseUnit(msg: []const u8) ?Unit { + if (msg.len < head_len) return null; + return .{ .origin = std.mem.readInt(u16, msg[0..2], .little), .seq = std.mem.readInt(u32, msg[2..6], .little), .tape = msg[head_len..] }; +} + +pub const Input = struct { client: u16, seq: u32, event: pardes.Event }; + +/// A client's input message; its event borrows `msg`. +pub fn parseInput(msg: []const u8) ?Input { + if (msg.len < head_len) return null; + const f = (wire.framed(msg[head_len..]) catch return null) orelse return null; + const ev = switch (wire.decodeClient(f.tag, f.payload) catch return null) { + .event => |ev| ev, + else => return null, + }; + return .{ .client = std.mem.readInt(u16, msg[0..2], .little), .seq = std.mem.readInt(u32, msg[2..6], .little), .event = ev }; +} + +/// The core whose tape is `t`, stepped: tape.on is process-wide, and so +/// are syntax.zig's held trees, so one process can step several. +const Stepping = struct { + tape: ?*tape.Tape, + trees: ?*pardes.syntax.HeldTrees, + + fn begin(t: *tape.Tape, epoch: ?*rollback.Epoch) Stepping { + const was: Stepping = .{ .tape = tape.on, .trees = pardes.syntax.held_trees }; + tape.on = t; + rollback.Epoch.select(epoch); + return was; + } + + fn end(was: Stepping) void { + tape.on = was.tape; + pardes.syntax.held_trees = was.trees; + } +}; + +fn nowNs() u64 { + return pardes.FrameStats.nowNs(); +} + +// ---- the authority ---- + +pub const Authority = struct { + gpa: Allocator, + t: tape.Tape, + core: *Pardes, + host: pardes.Host, + out: std.ArrayListUnmanaged(u8) = .empty, + seq: u32 = 0, + /// Each client's view (src/views.zig): its own layout, focus and + /// cursors over the session's text. The first client has the session's. + view_of: [64]u8 = @splat(0), + + /// A session booted with `opts`; `unit` is its first, which a replica + /// is made from. + pub fn init(gpa: Allocator, options: pardes.Options, host: pardes.Host) !*Authority { + // A replay gives its core every allocator (tape.zig), and a digest + // tells an allocator from none. + var opts = options; + opts.image_allocator = opts.image_allocator orelse gpa; + opts.pdf_allocator = opts.pdf_allocator orelse gpa; + opts.tree_sitter_allocator = opts.tree_sitter_allocator orelse gpa; + opts.frame_allocator = opts.frame_allocator orelse gpa; + const a = try gpa.create(Authority); + errdefer gpa.destroy(a); + a.* = .{ .gpa = gpa, .t = tape.memory(gpa), .core = undefined, .host = host }; + const was = Stepping.begin(&a.t, null); + defer was.end(); + a.core = try Pardes.init(gpa, opts); + a.core.host = host; + a.core.pump(host) catch {}; + return a; + } + + pub fn deinit(a: *Authority) void { + const was = Stepping.begin(&a.t, null); + a.core.deinit(); + was.end(); + tape.release(&a.t); + a.out.deinit(a.gpa); + a.gpa.destroy(a); + } + + /// What was recorded since the last unit, as one (the boot's, a setup's + /// done straight on the core). + pub fn unit(a: *Authority, origin: u16, seq: u32) ![]const u8 { + a.out.clearRetainingCapacity(); + try a.out.appendNTimes(a.gpa, 0, head_len); + header(a.out.items, origin, seq); + try a.out.appendSlice(a.gpa, tape.taken(&a.t)); + tape.clearTaken(&a.t); + return a.out.items; + } + + /// An input of the authority's own (a pty's output, a tick, a setup's + /// command), recorded into the unit being made. + pub fn apply(a: *Authority, ev: pardes.Event) void { + const was = Stepping.begin(&a.t, null); + a.core.update(ev); + a.core.pump(a.host) catch {}; + was.end(); + } + + /// One of the authority's own inputs, as its unit. + pub fn local(a: *Authority, ev: pardes.Event) ![]const u8 { + a.apply(ev); + a.seq += 1; + return a.unit(0, a.seq); + } + + /// A view of its own for `client`, as its unit; which it is. + pub fn openView(a: *Authority, client: u16) !struct { view: u8, unit: []const u8 } { + const was = Stepping.begin(&a.t, null); + const k = a.core.openView() orelse { + was.end(); + return error.TooManyViews; + }; + a.core.pump(a.host) catch {}; + was.end(); + a.view_of[client] = k; + a.seq += 1; + return .{ .view = k, .unit = try a.unit(0, a.seq) }; + } + + /// One client's input, applied in the order it came, in its view: the + /// unit every replica takes. + pub fn take(a: *Authority, msg: []const u8) ![]const u8 { + const in = parseInput(msg) orelse return error.BadInput; + const was = Stepping.begin(&a.t, null); + a.core.toView(a.view_of[in.client]); + a.core.update(in.event); + a.core.pump(a.host) catch {}; + was.end(); + return a.unit(in.client, in.seq); + } +}; + +// ---- a replica ---- + +pub const Stats = struct { + inputs: u32 = 0, + /// Applied at once and clean. + speculated: u32 = 0, + /// Stopped at a host call, a syscall or an emulator: waits one RTT. + stopped: u32 = 0, + /// Not tried: an event of the view's (a resize, the pointer), phase 4's. + unspeculated: u32 = 0, + /// The authority's unit was the pending head, recorded the same. + fast: u32 = 0, + /// The copy put back: the head recorded otherwise (a guess of the clock + /// the authority did not share), or another's unit first. + mispredicted: u32 = 0, + foreign: u32 = 0, + rebases: u32 = 0, + units: u32 = 0, + /// Nanoseconds: one speculative apply, a copy taken, a rollback (the + /// copy put back, the confirmed units replayed, the pending reapplied). + spec_ns: std.ArrayListUnmanaged(u64) = .empty, + copy_ns: std.ArrayListUnmanaged(u64) = .empty, + rollback_ns: std.ArrayListUnmanaged(u64) = .empty, + copy_bytes: u64 = 0, + copies: u32 = 0, + + pub fn deinit(s: *Stats, gpa: Allocator) void { + s.spec_ns.deinit(gpa); + s.copy_ns.deinit(gpa); + s.rollback_ns.deinit(gpa); + } +}; + +const Pending = struct { seq: u32, msg: []u8, spec: ?[]u8 = null }; + +pub const Replica = struct { + gpa: Allocator, + id: u16, + /// Its own view of the session (Authority.openView's). + view: u8 = 0, + /// The core's allocator; its tapes and the engine's own lists are not. + epoch: rollback.Epoch = .{}, + t: tape.Tape, + copy: rollback.Copy = .{}, + pending: std.ArrayListUnmanaged(Pending) = .empty, + /// pending[0..applied] are applied on top of the confirmed position. + applied: usize = 0, + /// The pending input that waits for the authority (it stopped at a host + /// call, or is the view's), and those after it with it. + blocked: ?u32 = null, + /// Confirmed units fast-pathed since the copy was taken. + since: std.ArrayListUnmanaged([]u8) = .empty, + /// Pending inputs to apply again (after a rollback). + dirty: bool = false, + next_seq: u32 = 1, + stats: Stats = .{}, + diverged: ?[]const u8 = null, + out: std.ArrayListUnmanaged(u8) = .empty, + /// Nanoseconds of the rollback being made, until its reapply. + recovering: ?u64 = null, + + /// A replica of the session whose first unit is `first`. + pub fn init(gpa: Allocator, id: u16, first: []const u8) !*Replica { + const r = try gpa.create(Replica); + r.* = .{ .gpa = gpa, .id = id, .t = tape.memory(gpa) }; + r.t.core_gpa = r.epoch.allocator(); + r.confirm(first); + if (r.t.core == null) { + r.deinit(); + return error.NoCore; + } + return r; + } + + pub fn deinit(r: *Replica) void { + if (r.t.core) |c| { + const was = Stepping.begin(&r.t, &r.epoch); + if (r.copy.held) r.copy.restore(); + c.deinit(); + was.end(); + } + r.copy.deinit(); + for (r.pending.items) |p| r.freePending(p); + r.pending.deinit(r.gpa); + for (r.since.items) |u| r.gpa.free(u); + r.since.deinit(r.gpa); + r.out.deinit(r.gpa); + r.stats.deinit(r.gpa); + tape.release(&r.t); + if (r.diverged) |why| r.gpa.free(why); + r.epoch.deinit(); + r.gpa.destroy(r); + } + + pub fn core(r: *const Replica) *Pardes { + return r.t.core.?; + } + + fn freePending(r: *Replica, p: Pending) void { + r.gpa.free(p.msg); + if (p.spec) |s| r.gpa.free(s); + } + + /// The replica's own input: applied at once when it can be, and the + /// message for the authority returned (valid until the next call). + pub fn input(r: *Replica, ev: pardes.Event) ![]const u8 { + r.stats.inputs += 1; + const seq = r.next_seq; + r.next_seq += 1; + const room = try r.gpa.alloc(u8, head_len + wire.clientBound(.{ .event = ev })); + defer r.gpa.free(room); + const n = (try wire.encodeClient(room[head_len..], .{ .event = ev })).len; + header(room, r.id, seq); + const msg = try r.gpa.dupe(u8, room[0 .. head_len + n]); + try r.pending.append(r.gpa, .{ .seq = seq, .msg = msg }); + r.dirty = true; + r.settle(); + r.out.clearRetainingCapacity(); + try r.out.appendSlice(r.gpa, msg); + return r.out.items; + } + + /// One unit of the authority's stream, in its order. + pub fn confirm(r: *Replica, msg: []const u8) void { + if (r.diverged != null) return; + const u = parseUnit(msg) orelse return r.fail("a unit too short to read"); + r.stats.units += 1; + const was = Stepping.begin(&r.t, &r.epoch); + defer was.end(); + if (u.origin == r.id and r.pending.items.len > 0 and r.pending.items[0].seq == u.seq and r.applied > 0 and + tape.sameRecords(r.pending.items[0].spec.?, u.tape)) + { + r.stats.fast += 1; + r.freePending(r.pending.orderedRemove(0)); + r.applied -= 1; + if (r.applied == 0) { + // Nothing speculative on top any more: the state is the + // confirmed one. + r.copy.release(); + rollback.Epoch.select(null); + r.clearSince(); + return; + } + const kept = r.gpa.dupe(u8, u.tape) catch return r.fail("out of memory"); + r.since.append(r.gpa, kept) catch return r.fail("out of memory"); + if (r.since.items.len >= rebase_after) { + r.stats.rebases += 1; + r.toConfirmed(); + r.dirty = true; + } + return; + } + if (r.applied > 0) { + if (u.origin == r.id) r.stats.mispredicted += 1 else r.stats.foreign += 1; + if (u.origin == r.id and std.c.getenv("PARDES_REPLICA_TRACE") != null) { + var w: std.Io.Writer.Allocating = .init(r.gpa); + defer w.deinit(); + w.writer.print("replica {d}: its input {d} was recorded otherwise; it guessed:\n", .{ r.id, u.seq }) catch {}; + tape.describe(r.pending.items[0].spec.?, &w.writer); + w.writer.print("and the authority recorded:\n", .{}) catch {}; + tape.describe(u.tape, &w.writer); + std.debug.print("{s}", .{w.written()}); + } + r.recovering = nowNs(); + r.toConfirmed(); + } + if (tape.replayUnit(&r.t, u.tape)) |why| { + if (std.c.getenv("PARDES_REPLICA_TRACE") != null) { + var w: std.Io.Writer.Allocating = .init(r.gpa); + defer w.deinit(); + w.writer.print("replica {d}: the unit of {d}/{d} diverged: {s}\n", .{ r.id, u.origin, u.seq, why }) catch {}; + tape.describe(u.tape, &w.writer); + std.debug.print("{s}", .{w.written()}); + } + return r.fail(why); + } + if (u.origin == r.id) { + if (r.pending.items.len == 0 or r.pending.items[0].seq != u.seq) return r.fail("the authority confirmed an input this replica did not send next"); + if (r.blocked == u.seq) r.blocked = null; + r.freePending(r.pending.orderedRemove(0)); + } + r.dirty = true; + } + + fn fail(r: *Replica, why: []const u8) void { + if (r.diverged == null) r.diverged = r.gpa.dupe(u8, why) catch return; + } + + fn clearSince(r: *Replica) void { + for (r.since.items) |u| r.gpa.free(u); + r.since.clearRetainingCapacity(); + } + + /// Back to the confirmed position: the copy put back, the units fast- + /// pathed since replayed. + fn toConfirmed(r: *Replica) void { + if (r.copy.held) { + r.copy.restore(); + rollback.Epoch.select(null); + } + r.applied = 0; + for (r.since.items) |u| if (tape.replayUnit(&r.t, u)) |why| return r.fail(why); + r.clearSince(); + for (r.pending.items) |*p| if (p.spec) |s| { + r.gpa.free(s); + p.spec = null; + }; + } + + /// The pending inputs applied again, as far as they go. + pub fn settle(r: *Replica) void { + if (!r.dirty or r.diverged != null) return; + r.dirty = false; + const was = Stepping.begin(&r.t, &r.epoch); + defer was.end(); + while (r.applied < r.pending.items.len and r.blocked != r.pending.items[r.applied].seq) if (!r.speculate()) break; + if (r.recovering) |t0| { + r.stats.rollback_ns.append(r.gpa, nowNs() - t0) catch {}; + r.recovering = null; + } + } + + /// pending[applied] applied on top: whether it was. + fn speculate(r: *Replica) bool { + const p = &r.pending.items[r.applied]; + const in = parseInput(p.msg) orelse return false; + switch (in.event) { + .key, .paste, .command => {}, + else => { + // The view's (a resize, the pointer): the authority's for + // now, phase 4's views to come. + r.stats.unspeculated += 1; + r.blocked = p.seq; + return false; + }, + } + const c = r.core(); + if (!r.copy.held) { + const t0 = nowNs(); + r.copy.take(&r.epoch, c) catch return false; + rollback.Epoch.select(&r.epoch); + r.stats.copy_ns.append(r.gpa, nowNs() - t0) catch {}; + r.stats.copy_bytes += r.copy.savedBytes(); + r.stats.copies += 1; + } + var terminals: [pardes.MAX_PANES]u64 = undefined; + if (checked) terminalDigests(c, &terminals); + const t0 = nowNs(); + tape.beginSpeculation(&r.t); + c.toView(r.view); + c.update(in.event); + c.pump(c.host) catch {}; + const clean = tape.endSpeculation(&r.t); + const dt = nowNs() - t0; + // Where a blob was written, should one be: the input that did. + if (checked and std.c.getenv("PARDES_REPLICA_TRACE") != null) r.copy.check(); + if (!clean) { + // Undone, and the earlier ones applied again on the confirmed + // position: this one waits for the authority. + r.stats.stopped += 1; + r.blocked = p.seq; + const upto = r.applied; + r.toConfirmed(); + while (r.applied < upto) if (!r.speculate()) break; + if (r.applied == 0 and r.copy.held) { + r.copy.release(); + rollback.Epoch.select(null); + } + return false; + } + if (checked) { + var after: [pardes.MAX_PANES]u64 = undefined; + terminalDigests(c, &after); + if (!std.mem.eql(u64, &terminals, &after)) @panic("replica: a speculation wrote a terminal's emulator"); + } + r.stats.speculated += 1; + r.stats.spec_ns.append(r.gpa, dt) catch {}; + p.spec = r.gpa.dupe(u8, tape.taken(&r.t)) catch return false; + tape.clearTaken(&r.t); + r.applied += 1; + return true; + } + + /// Nothing pending and nothing applied: the replica is the authority's + /// confirmed state, which `digest` says. + pub fn quiet(r: *const Replica) bool { + return r.pending.items.len == 0 and r.applied == 0 and !r.copy.held; + } + + pub fn digest(r: *Replica) u64 { + const was = Stepping.begin(&r.t, &r.epoch); + defer was.end(); + return tape.digestOf(r.core()); + } +}; + +fn terminalDigests(p: *const Pardes, out: *[pardes.MAX_PANES]u64) void { + for (p.panes, out) |slot, *d| { + d.* = 0; + const pane = slot orelse continue; + const state = pane.terminal orelse continue; + var h = walk.Hasher.init(0); + state.digest(&h); + d.* = h.final(); + } +} + +pub fn digestOf(a: *Authority) u64 { + const was = Stepping.begin(&a.t, null); + defer was.end(); + return tape.digestOf(a.core); +} + +const TestHost = struct { + fn watched(_: ?*anyopaque) bool { + return false; + } + const vtable: pardes.Host.VTable = .{ .watched = watched }; + fn host() pardes.Host { + return .{ .vtable = &vtable }; + } +}; + +fn key(r: *Replica, cp: u21, sent: *std.ArrayListUnmanaged([]u8)) !void { + const text = [_]u8{@intCast(cp & 0x7f)}; + const msg = try r.input(.{ .key = .{ .cp = cp, .text = if (cp < 0x80 and cp >= 0x20) &text else "" } }); + try sent.append(std.testing.allocator, try std.testing.allocator.dupe(u8, msg)); +} + +test "a replica's own keys apply at once, confirm on the fast path, and end at the authority's digest" { + const gpa = std.testing.allocator; + const a = try Authority.init(gpa, .{ .cols = 80, .rows = 24, .file = "mise.toml" }, TestHost.host()); + defer a.deinit(); + const r = try Replica.init(gpa, 1, try a.unit(0, 0)); + defer r.deinit(); + try std.testing.expectEqual(digestOf(a), r.digest()); + var sent: std.ArrayListUnmanaged([]u8) = .empty; + defer { + for (sent.items) |m| gpa.free(m); + sent.deinit(gpa); + } + for ("oabc") |k| try key(r, k, &sent); + try key(r, pardes.Key.escape, &sent); + try std.testing.expectEqual(@as(usize, 5), r.applied); + for (sent.items) |m| { + r.confirm(try a.take(m)); + r.settle(); + } + if (r.diverged) |why| std.debug.print("diverged: {s}\n", .{why}); + try std.testing.expect(r.diverged == null); + try std.testing.expectEqual(@as(u32, 5), r.stats.fast); + try std.testing.expect(r.quiet()); + try std.testing.expectEqual(digestOf(a), r.digest()); +} + +test "another client's input first: a replica rolls back, replays it and applies its own again" { + const gpa = std.testing.allocator; + const a = try Authority.init(gpa, .{ .cols = 80, .rows = 24, .file = "mise.toml" }, TestHost.host()); + defer a.deinit(); + const first = try gpa.dupe(u8, try a.unit(0, 0)); + defer gpa.free(first); + const r1 = try Replica.init(gpa, 1, first); + defer r1.deinit(); + const r2 = try Replica.init(gpa, 2, first); + defer r2.deinit(); + var s1: std.ArrayListUnmanaged([]u8) = .empty; + var s2: std.ArrayListUnmanaged([]u8) = .empty; + defer { + for (s1.items) |m| gpa.free(m); + for (s2.items) |m| gpa.free(m); + s1.deinit(gpa); + s2.deinit(gpa); + } + for ("oone") |k| try key(r1, k, &s1); + for ("Otwo") |k| try key(r2, k, &s2); + // The authority hears r2 first, then r1; both hear the units in that + // order. + for ([_]*std.ArrayListUnmanaged([]u8){ &s2, &s1 }) |list| for (list.items) |m| { + const u = try gpa.dupe(u8, try a.take(m)); + defer gpa.free(u); + r1.confirm(u); + r1.settle(); + r2.confirm(u); + r2.settle(); + }; + for ([_]*Replica{ r1, r2 }) |r| { + if (r.diverged) |why| std.debug.print("diverged: {s}\n", .{why}); + try std.testing.expect(r.diverged == null); + try std.testing.expect(r.quiet()); + try std.testing.expectEqual(digestOf(a), r.digest()); + } + try std.testing.expect(r1.stats.foreign >= 1); + try std.testing.expectEqual(@as(u32, 4), r2.stats.fast); +} diff --git a/src/rollback.zig b/src/rollback.zig new file mode 100644 index 00000000..9137b6f6 --- /dev/null +++ b/src/rollback.zig @@ -0,0 +1,496 @@ +//! A replica's rollback copy (docs/next-daemon.md, phase 5): the state a +//! speculation can write, saved before it writes, and put back where it was +//! when the authority's order differs. +//! +//! Two halves: +//! - `Epoch`, the replica core's allocator. It knows every allocation, and +//! while a copy is held it frees nothing that was there before (a free +//! is put off) and grows nothing of then in place. What is allocated +//! meanwhile is the epoch's. So whatever the core held when the copy was +//! taken is still where it was when the copy is put back. +//! - `Copy`, the walker's walk of the core's state (src/walk.zig: what a +//! digest covers, so views, buffers, registers, counters and terminal +//! metadata), saving the bytes of every allocation it reaches, the core +//! struct's own included. A file's text, its line index and its undo +//! snapshots are pinned instead: an edit puts a new buffer in their +//! place (File.setContentSpan) and never writes one, so the copy keeps +//! a reference and the put-off free keeps it alive. +//! +//! Putting the copy back writes each saved allocation's bytes over it, so +//! every pointer in them points where it did, at what was there; then the +//! epoch's allocations are freed and its put-off frees forgotten. Letting +//! the copy go (the speculation was right) makes the put-off frees. +//! +//! ponytail: an allocation reached only through a field the walk leaves +//! out (a derived cache's own heap, say) is not saved; that cache is the +//! epoch's as it was written, which is right for one remade from the text +//! it was made of and wrong for one changed in place. The replica harness +//! (test/replicas.zig) checks a digest at every confirmed step. +const std = @import("std"); +const builtin = @import("builtin"); +const pardes = @import("pardes.zig"); +const walk = @import("walk.zig"); +const Allocator = std.mem.Allocator; +const Alignment = std.mem.Alignment; +const Pardes = pardes.Pardes; + +/// Blobs this long or longer (`pinned`) are pinned, not copied. Checked +/// unchanged in a safe build when the copy goes. +const blob_min = 4096; +const checked = builtin.mode == .debug or builtin.mode == .safe; + +pub const Epoch = struct { + child: Allocator = std.heap.c_allocator, + /// Its own bookkeeping, never the epoch's. + meta: Allocator = std.heap.c_allocator, + live: std.AutoHashMapUnmanaged(usize, Entry) = .empty, + on: bool = false, + id: u32 = 0, + born: std.ArrayListUnmanaged(usize) = .empty, + deferred: std.ArrayListUnmanaged(usize) = .empty, + trees: pardes.syntax.HeldTrees = .{ .meta = std.heap.c_allocator }, + + const Entry = struct { len: usize, alignment: Alignment, epoch: u32, freed: bool = false }; + + pub fn allocator(e: *Epoch) Allocator { + return .{ .ptr = e, .vtable = &.{ .alloc = alloc, .resize = resize, .remap = remap, .free = free } }; + } + + pub fn deinit(e: *Epoch) void { + var it = e.live.iterator(); + while (it.next()) |kv| e.child.rawFree(@as([*]u8, @ptrFromInt(kv.key_ptr.*))[0..kv.value_ptr.len], kv.value_ptr.alignment, @returnAddress()); + e.live.deinit(e.meta); + e.born.deinit(e.meta); + e.deferred.deinit(e.meta); + e.trees.born.deinit(e.meta); + e.trees.freed.deinit(e.meta); + } + + fn mine(e: *const Epoch, entry: Entry) bool { + return e.on and entry.epoch == e.id; + } + + /// The live allocation that starts at `addr`. + pub fn owns(e: *const Epoch, addr: usize) ?Entry { + const entry = e.live.get(addr) orelse return null; + return if (entry.freed) null else entry; + } + + fn note(e: *Epoch, addr: usize, len: usize, alignment: Alignment) bool { + e.live.put(e.meta, addr, .{ .len = len, .alignment = alignment, .epoch = if (e.on) e.id else 0 }) catch return false; + if (e.on) e.born.append(e.meta, addr) catch {}; + return true; + } + + /// Every new allocation filled with this: a core reading memory it never + /// wrote shows as a replica's divergence (test/replicas.zig, a release + /// build's heap is otherwise whatever it was). + pub var fill: ?u8 = null; + + fn alloc(ctx: *anyopaque, len: usize, alignment: Alignment, ra: usize) ?[*]u8 { + const e: *Epoch = @ptrCast(@alignCast(ctx)); + const ptr = e.child.rawAlloc(len, alignment, ra) orelse return null; + if (fill) |b| @memset(ptr[0..len], b); + if (!e.note(@intFromPtr(ptr), len, alignment)) { + e.child.rawFree(ptr[0..len], alignment, ra); + return null; + } + return ptr; + } + + fn resize(ctx: *anyopaque, memory: []u8, alignment: Alignment, new_len: usize, ra: usize) bool { + const e: *Epoch = @ptrCast(@alignCast(ctx)); + const entry = e.live.getPtr(@intFromPtr(memory.ptr)) orelse return false; + // What the copy saved stays its size: the core moves instead. + if (e.on and !e.mine(entry.*)) return false; + if (!e.child.rawResize(memory, alignment, new_len, ra)) return false; + entry.len = new_len; + return true; + } + + fn remap(ctx: *anyopaque, memory: []u8, alignment: Alignment, new_len: usize, ra: usize) ?[*]u8 { + const e: *Epoch = @ptrCast(@alignCast(ctx)); + const old = @intFromPtr(memory.ptr); + const entry = e.live.get(old) orelse return null; + if (e.on and !e.mine(entry)) return null; + const ptr = e.child.rawRemap(memory, alignment, new_len, ra) orelse return null; + _ = e.live.remove(old); + e.live.put(e.meta, @intFromPtr(ptr), .{ .len = new_len, .alignment = alignment, .epoch = entry.epoch }) catch {}; + if (e.mine(entry) and @intFromPtr(ptr) != old) e.born.append(e.meta, @intFromPtr(ptr)) catch {}; + return ptr; + } + + fn free(ctx: *anyopaque, memory: []u8, alignment: Alignment, ra: usize) void { + const e: *Epoch = @ptrCast(@alignCast(ctx)); + const addr = @intFromPtr(memory.ptr); + const entry = e.live.getPtr(addr) orelse return e.child.rawFree(memory, alignment, ra); + if (e.on and !e.mine(entry.*)) { + // Held by the copy until it is put back or let go. + if (!entry.freed) e.deferred.append(e.meta, addr) catch {}; + entry.freed = true; + return; + } + e.child.rawFree(memory.ptr[0..entry.len], entry.alignment, ra); + _ = e.live.remove(addr); + } + + fn reallyFree(e: *Epoch, addr: usize) void { + const entry = e.live.get(addr) orelse return; + e.child.rawFree(@as([*]u8, @ptrFromInt(addr))[0..entry.len], entry.alignment, @returnAddress()); + _ = e.live.remove(addr); + } + + fn begin(e: *Epoch) void { + std.debug.assert(!e.on); + e.id +%= 1; + if (e.id == 0) e.id = 1; + e.on = true; + e.born.clearRetainingCapacity(); + e.deferred.clearRetainingCapacity(); + e.trees.born.clearRetainingCapacity(); + e.trees.freed.clearRetainingCapacity(); + pardes.syntax.held_trees = &e.trees; + } + + /// The copy put back: what the epoch made goes, what it freed stays. + fn discard(e: *Epoch) void { + for (e.born.items) |addr| if (e.live.get(addr)) |entry| if (e.mine(entry)) e.reallyFree(addr); + for (e.deferred.items) |addr| if (e.live.getPtr(addr)) |entry| { + entry.freed = false; + }; + for (e.trees.born.items) |t| pardes.syntax.destroyTree(t); + e.end(); + } + + /// The copy let go: what the epoch freed goes now. + fn keep(e: *Epoch) void { + for (e.deferred.items) |addr| e.reallyFree(addr); + for (e.trees.freed.items) |t| pardes.syntax.destroyTree(t); + e.end(); + } + + fn end(e: *Epoch) void { + e.on = false; + e.born.clearRetainingCapacity(); + e.deferred.clearRetainingCapacity(); + e.trees.born.clearRetainingCapacity(); + e.trees.freed.clearRetainingCapacity(); + if (pardes.syntax.held_trees == &e.trees) pardes.syntax.held_trees = null; + } + + /// The trees of whichever core is stepped next: one in-process harness + /// steps several (syntax.held_trees is process-wide). + pub fn select(e: ?*Epoch) void { + pardes.syntax.held_trees = if (e) |x| (if (x.on) &x.trees else null) else null; + } +}; + +pub const Copy = struct { + meta: Allocator = std.heap.c_allocator, + regions: std.ArrayListUnmanaged(Region) = .empty, + bytes: std.ArrayListUnmanaged(u8) = .empty, + seen: std.AutoHashMapUnmanaged(Seen, void) = .empty, + pins: std.ArrayListUnmanaged(Pin) = .empty, + held: bool = false, + epoch: ?*Epoch = null, + /// The field being walked, for a pin's report, and whether it is one + /// of the blobs: a file's text or an undo snapshot's. + where: []const u8 = "", + pinnable: bool = false, + + const Region = struct { addr: usize, off: usize, len: usize }; + const Pin = struct { addr: usize, len: usize, hash: u64, where: []const u8 }; + const Seen = struct { addr: usize, ty: usize }; + pub const Error = Allocator.Error; + + pub fn deinit(c: *Copy) void { + c.regions.deinit(c.meta); + c.bytes.deinit(c.meta); + c.seen.deinit(c.meta); + c.pins.deinit(c.meta); + } + + pub fn savedBytes(c: *const Copy) usize { + return c.bytes.items.len; + } + + pub fn pinnedBytes(c: *const Copy) usize { + var n: usize = 0; + for (c.pins.items) |pin| n += pin.len; + return n; + } + + /// The copy of `p`, whose allocator is `e`'s, taken: from now on what + /// it held stays where it is until the copy is put back or let go. + pub fn take(c: *Copy, e: *Epoch, p: *const Pardes) Error!void { + std.debug.assert(!c.held); + c.regions.clearRetainingCapacity(); + c.bytes.clearRetainingCapacity(); + c.seen.clearRetainingCapacity(); + c.pins.clearRetainingCapacity(); + c.epoch = e; + // A step drains these before it ends: between steps they hold + // nothing, and a copy is taken between steps. + std.debug.assert(p.effects_len == 0 and p.in_len == 0); + if (c.epoch.?.owns(@intFromPtr(p)) == null) @panic("rollback: the core is not its epoch's"); + try c.object(Pardes, p); + e.begin(); + c.held = true; + } + + /// The core as it was when the copy was taken. + pub fn restore(c: *Copy) void { + std.debug.assert(c.held); + c.check(); + for (c.regions.items) |r| @memcpy(@as([*]u8, @ptrFromInt(r.addr))[0..r.len], c.bytes.items[r.off..][0..r.len]); + c.epoch.?.discard(); + c.held = false; + } + + /// The speculation was right: the copy is not wanted. + pub fn release(c: *Copy) void { + std.debug.assert(c.held); + c.check(); + c.epoch.?.keep(); + c.held = false; + } + + pub fn check(c: *const Copy) void { + if (comptime !checked) return; + for (c.pins.items) |pin| if (std.hash.Wyhash.hash(0, @as([*]const u8, @ptrFromInt(pin.addr))[0..pin.len]) != pin.hash) { + const now = @as([*]const u8, @ptrFromInt(pin.addr))[0..pin.len]; + const entry = c.epoch.?.live.get(pin.addr); + std.debug.print("rollback: {s}, {d} bytes at {x}, pinned, was written in place: now {any}; entry {any}, epoch {d}\n", .{ pin.where, pin.len, pin.addr, now[0..32], entry, c.epoch.?.id }); + @panic("rollback: a pinned blob was written in place"); + }; + } + + fn save(c: *Copy, addr: usize, len: usize) Error!void { + if (len == 0) return; + const off = c.bytes.items.len; + try c.bytes.appendSlice(c.meta, @as([*]const u8, @ptrFromInt(addr))[0..len]); + try c.regions.append(c.meta, .{ .addr = addr, .off = off, .len = len }); + } + + /// Whether the value at `addr` of type `T` was reached before. + fn reached(c: *Copy, comptime T: type, addr: usize) Error!bool { + const key: Seen = .{ .addr = addr, .ty = @intFromPtr(@typeName(T).ptr) }; + const got = try c.seen.getOrPut(c.meta, key); + return got.found_existing; + } + + /// The allocation at `addr`, saved once, whatever reaches it. + fn allocation(c: *Copy, addr: usize) Error!?usize { + const entry = c.epoch.?.owns(addr) orelse return null; + if (try c.reached(void, addr)) return entry.len; + try c.save(addr, entry.len); + return entry.len; + } + + fn visit(c: *Copy, comptime T: type, v: *const T) Error!void { + if (comptime !heapIn(T, 0)) return; + switch (@typeInfo(T)) { + .optional => |o| if (v.*) |*payload| try c.visit(o.child, payload), + .array => |a| if (comptime heapIn(a.child, 0)) for (v) |*e| try c.visit(a.child, e), + .pointer => |p| switch (p.size) { + .one => try c.object(p.child, v.*), + .slice => try c.slice(p.child, v.*), + // No length to walk by: its keeper's (a C handle's buffer). + else => {}, + }, + .@"struct" => |s| { + if (comptime walk.machinery(T)) return; + // A terminal: the authority's (its emulator is never written + // by a speculation: terminal.zig's guards). + if (comptime @hasDecl(T, "digest") and @TypeOf(T.digest) == fn (*const T, *walk.Hasher) void) return; + if (s.layout == .@"packed") return; + if (comptime @hasField(T, "items") and @hasField(T, "capacity") and s.field_names.len == 2) { + if (v.capacity == 0) return; + _ = try c.allocation(@intFromPtr(v.items.ptr)); + const E = @typeInfo(@FieldType(T, "items")).pointer.child; + if (comptime heapIn(E, 0)) for (v.items) |*e| try c.visit(E, e); + return; + } + if (comptime @hasDecl(T, "KV") and @hasDecl(T, "Hash") and @hasField(T, "metadata")) { + const m = v.metadata orelse return; + // std's header sits before the metadata, at the start + // of the one allocation. + const header = std.mem.alignForward(usize, 2 * @sizeOf(usize) + @sizeOf(u32), @alignOf(usize)); + _ = try c.allocation(@intFromPtr(m) - header); + var it = v.iterator(); + while (it.next()) |entry| { + try c.visit(@TypeOf(entry.key_ptr.*), entry.key_ptr); + try c.visit(@TypeOf(entry.value_ptr.*), entry.value_ptr); + } + return; + } + if (comptime @hasField(T, "raw") and s.field_names.len == 1 and @hasDecl(T, "load")) return; + if (comptime @hasField(T, "any_future")) return; + if (comptime @hasField(T, "bit_length") and @hasField(T, "masks") and s.field_names.len == 2) { + if (v.bit_length == 0) return; + _ = try c.allocation(@intFromPtr(v.masks) - @sizeOf(@typeInfo(@FieldType(T, "masks")).pointer.child)); + return; + } + inline for (s.field_names, s.field_types, s.field_attrs) |name, F, attr| { + // What a digest covers, the caches made from it and this + // process's own (the views not live are among it): a + // safe build's free writes over what it frees, so what + // the speculation freed would come back as that. Not the + // queues a step drains (undefined between steps), nor the + // authority's. + if (attr.@"comptime" or comptime (annotated(T, "authority", name) or (annotated(T, "local", name) and unsaved(name)))) continue; + if (comptime opaqueTo(F)) continue; + c.where = @typeName(T) ++ "." ++ name; + c.pinnable = comptime pinned(T, name); + if (comptime walk.lengthOf(T, name) != null and @typeInfo(F) == .array) { + const len = comptime walk.lengthOf(T, name).?; + if (comptime heapIn(@typeInfo(F).array.child, 0)) for (@field(v, name)[0..walk.lengthIn(v, len)]) |*e| try c.visit(@typeInfo(F).array.child, e); + } else try c.visit(F, &@field(v, name)); + } + }, + .@"union" => switch (v.*) { + inline else => |*payload| try c.visit(@TypeOf(payload.*), payload), + }, + .error_union => |e| if (v.*) |*payload| try c.visit(e.payload, payload) else |_| {}, + else => {}, + } + } + + fn object(c: *Copy, comptime P: type, ptr: *const P) Error!void { + if (comptime walk.machinery(P)) return; + const addr = @intFromPtr(ptr); + if (try c.reached(P, addr)) return; + if (comptime @hasDecl(P, "digest") and @TypeOf(P.digest) == fn (*const P, *walk.Hasher) void) return; + if (c.epoch.?.owns(addr) != null) { + _ = try c.reached(void, addr); + try c.fields(P, ptr); + } + try c.visit(P, ptr); + } + + /// A struct's bytes, field by field where it says what its fields are: + /// all of them (this process's own too, which the epoch's frees would + /// leave pointing at nothing), but the queues a step drains and the + /// frame statistics (numbers, megabytes of them), and an array only as + /// far as its length says. + fn fields(c: *Copy, comptime T: type, v: *const T) Error!void { + if (comptime @typeInfo(T) != .@"struct" or !@hasDecl(T, "replica")) return c.save(@intFromPtr(v), @sizeOf(T)); + const s = @typeInfo(T).@"struct"; + inline for (s.field_names, s.field_types, s.field_attrs) |name, F, attr| { + if (attr.@"comptime") continue; + if (comptime annotated(T, "local", name) and unsaved(name)) continue; + const at = @intFromPtr(&@field(v, name)); + if (comptime walk.lengthOf(T, name) != null and @typeInfo(F) == .array) { + try c.save(at, walk.lengthIn(v, comptime walk.lengthOf(T, name).?) * @sizeOf(@typeInfo(F).array.child)); + } else if (comptime annotatedStruct(F)) { + try c.fields(F, &@field(v, name)); + } else if (comptime @typeInfo(F) == .array and annotatedStruct(@typeInfo(F).array.child)) { + for (&@field(v, name)) |*e| try c.fields(@typeInfo(F).array.child, e); + } else try c.save(at, @sizeOf(F)); + } + } + + fn slice(c: *Copy, comptime E: type, s: []const E) Error!void { + if (s.len == 0) return; + const addr = @intFromPtr(s.ptr); + if (try c.reached([]const E, addr)) return; + if (comptime !heapIn(E, 0)) if (c.pinnable and s.len * @sizeOf(E) >= blob_min and c.epoch.?.owns(addr) != null) { + const bytes = std.mem.sliceAsBytes(s); + try c.pins.append(c.meta, .{ .addr = addr, .len = bytes.len, .hash = if (checked) std.hash.Wyhash.hash(0, bytes) else 0, .where = c.where }); + return; + }; + _ = try c.allocation(addr); + if (comptime heapIn(E, 0)) for (s) |*e| try c.visit(E, e); + } +}; + +/// The blobs: what an edit puts in place of its predecessor, never writes +/// (File.setContentSpan), and frees raw (memory.freeUnfilled, freeRaw). +fn pinned(comptime T: type, comptime name: []const u8) bool { + const File = pardes.panes.File; + return (T == File.State and (std.mem.eql(u8, name, "content") or std.mem.eql(u8, name, "line_starts"))) or + (T == File.Snapshot and std.mem.eql(u8, name, "content")); +} + +fn annotatedStruct(comptime T: type) bool { + return @typeInfo(T) == .@"struct" and @hasDecl(T, "replica"); +} + +/// The queues every step drains (empty between steps, asserted when a copy +/// is taken: what is in their slots is nothing), and the frame statistics +/// (numbers, and a static name). +fn unsaved(comptime name: []const u8) bool { + return std.mem.eql(u8, name, "effects") or std.mem.eql(u8, name, "in_q") or std.mem.eql(u8, name, "frames"); +} + +fn annotated(comptime T: type, comptime kind: []const u8, comptime name: []const u8) bool { + @setEvalBranchQuota(100_000); + if (!@hasDecl(T, "replica") or !@hasField(@TypeOf(T.replica), kind)) return false; + inline for (@field(T.replica, kind)) |n| if (comptime std.mem.eql(u8, n, name)) return true; + return false; +} + +/// A field no walk sees into (a C handle, a function): its own keeper's. +fn opaqueTo(comptime F: type) bool { + return switch (@typeInfo(F)) { + .optional => |o| opaqueTo(o.child), + .pointer => |p| p.size == .many or p.size == .c or switch (@typeInfo(p.child)) { + .@"opaque", .@"fn" => true, + else => false, + }, + else => false, + }; +} + +/// Whether a `T` can hold a pointer to the heap at all (else a walk of it +/// finds nothing to save). Deep types count as yes. +fn heapIn(comptime T: type, comptime depth: u32) bool { + if (depth > 12) return true; + return switch (@typeInfo(T)) { + .void, .null, .undefined, .bool, .int, .float, .@"enum", .error_set, .comptime_int, .comptime_float, .enum_literal, .type => false, + .optional => |o| heapIn(o.child, depth + 1), + .array => |a| a.len > 0 and heapIn(a.child, depth + 1), + .pointer => true, + .error_union => |e| heapIn(e.payload, depth + 1), + .@"struct" => |s| blk: { + if (s.layout == .@"packed") break :blk false; + for (s.field_types, s.field_attrs) |F, attr| if (!attr.@"comptime" and heapIn(F, depth + 1)) break :blk true; + break :blk false; + }, + .@"union" => |u| blk: { + for (u.field_types) |F| if (heapIn(F, depth + 1)) break :blk true; + break :blk false; + }, + else => true, + }; +} + +test "a copy put back is the core as it was; one let go keeps what was done" { + var e: Epoch = .{ .child = std.testing.allocator }; + defer e.deinit(); + const gpa = e.allocator(); + const p = try Pardes.init(gpa, .{ .cols = 80, .rows = 24, .file = "mise.toml", .image_allocator = gpa, .pdf_allocator = gpa, .tree_sitter_allocator = gpa, .frame_allocator = gpa }); + p.pump(.{}) catch {}; + const before = walk.digest(Pardes, p); + var c: Copy = .{}; + defer c.deinit(); + const typed = "ox y z"; + try c.take(&e, p); + for (typed) |k| p.update(.{ .key = .{ .cp = k, .text = &.{k} } }); + p.update(.{ .key = .{ .cp = pardes.Key.escape } }); + p.pump(.{}) catch {}; + const edited = walk.digest(Pardes, p); + try std.testing.expect(edited != before); + c.restore(); + try std.testing.expectEqual(before, walk.digest(Pardes, p)); + // The same again, let go this time: what was typed stays. + try c.take(&e, p); + for (typed) |k| p.update(.{ .key = .{ .cp = k, .text = &.{k} } }); + p.update(.{ .key = .{ .cp = pardes.Key.escape } }); + p.pump(.{}) catch {}; + try std.testing.expectEqual(edited, walk.digest(Pardes, p)); + c.release(); + try std.testing.expectEqual(edited, walk.digest(Pardes, p)); + p.deinit(); + try std.testing.expectEqual(@as(usize, 0), e.live.count()); +} diff --git a/src/syntax.zig b/src/syntax.zig index 985a9888..858ef6a2 100644 --- a/src/syntax.zig +++ b/src/syntax.zig @@ -381,10 +381,34 @@ pub fn parseTree(path: []const u8, content: []const u8) ?*anyopaque { if (!enabled) return null; const selected = (forExt(std.fs.path.extension(path)) catch return null) orelse return null; const tree = selected.parser.parseString(content, null) orelse return null; + if (held_trees) |h| h.born.append(h.meta, @ptrCast(tree)) catch {}; return @ptrCast(tree); } +/// While a replica's rollback copy is held (src/rollback.zig) a tree parsed +/// is the speculation's, and one freed is put off: trees live on the C +/// heap, which the core's allocator cannot see. Null otherwise. +pub var held_trees: ?*HeldTrees = null; +pub const HeldTrees = struct { + meta: std.mem.Allocator, + born: std.ArrayListUnmanaged(*anyopaque) = .empty, + freed: std.ArrayListUnmanaged(*anyopaque) = .empty, +}; + pub fn freeTree(tree: *anyopaque) void { + if (!enabled) return; + if (held_trees) |h| { + for (h.born.items, 0..) |t, i| if (t == tree) { + _ = h.born.swapRemove(i); + return destroyTree(tree); + }; + h.freed.append(h.meta, tree) catch destroyTree(tree); + return; + } + destroyTree(tree); +} + +pub fn destroyTree(tree: *anyopaque) void { if (!enabled) return; const t: *ts.Tree = @ptrCast(@alignCast(tree)); t.destroy(); diff --git a/src/tape.zig b/src/tape.zig index ec38deac..70609858 100644 --- a/src/tape.zig +++ b/src/tape.zig @@ -127,19 +127,40 @@ pub const Tape = struct { /// PARDES_RECORD_EVERY: a digest after every input, a nested one's too /// (to find the step a divergence starts at), not only the host's last. every: bool = false, + /// A replica applying its own input before the authority has ordered it + /// (src/replica.zig): recorded as the authority would record it, so the + /// two can be compared, but nothing is performed. A host call or a + /// syscall stops it (`poisoned`): what the call returns is the + /// authority's to say, and the input waits for it. A read of the clock + /// or the environment (`value`) is made here, a guess the comparison + /// checks. + speculating: bool = false, + poisoned: bool = false, + /// The allocator of the core a replay makes (a replica's rollback epoch, + /// src/rollback.zig), when it is not the tape's own. + core_gpa: ?std.mem.Allocator = null, + /// Where a replay keeps what a core keeps of an input (its options), + /// for a tape that is let go; else the tape's allocator, for good. + kept_arena: ?std.heap.ArenaAllocator = null, + /// No digests recorded (a speculation's: a replica compares what it + /// was given and what it asked, from a state the authority shares). + no_digest: bool = false, fn record(t: *Tape, kind: Kind, comptime T: type, v: *const T) void { t.scratch.clearRetainingCapacity(); walk.encode(T, v, &t.scratch.writer) catch @panic("tape: out of memory"); const payload = t.scratch.written(); const len: u32 = @intCast(payload.len); - t.buf.append(t.gpa, @intFromEnum(kind)) catch @panic("tape: out of memory"); + t.buf.append(t.gpa, @backingInt(kind)) catch @panic("tape: out of memory"); t.buf.appendSlice(t.gpa, std.mem.asBytes(&len)) catch @panic("tape: out of memory"); t.buf.appendSlice(t.gpa, payload) catch @panic("tape: out of memory"); if (t.buf.items.len > 1 << 20) t.flush(); } fn flush(t: *Tape) void { + // A tape in memory (a replica's, an authority's stream) is taken + // by whoever holds it. + if (t.fd < 0) return; var at: usize = 0; while (at < t.buf.items.len) { const n = libc.write(t.fd, t.buf.items[at..].ptr, t.buf.items.len - at); @@ -222,7 +243,7 @@ pub const Tape = struct { t.wrap(core); }; t.record(.done, void, &{}); - if (t.depth == 0 or t.every) if (t.core) |core| { + if ((t.depth == 0 or t.every) and !t.no_digest) if (t.core) |core| { var parts: Parts = undefined; const d = digestParts(core, &parts); t.record(.digest, []const u64, &d); @@ -239,12 +260,12 @@ pub const Tape = struct { const head = t.in.take(5) catch return null; const len = std.mem.bytesToValue(u32, head[1..5]); const payload = t.in.take(len) catch return null; - return .{ .kind = @enumFromInt(head[0]), .payload = .{ .bytes = payload } }; + return .{ .kind = @fromBackingInt(@intCast(head[0])), .payload = .{ .bytes = payload } }; } fn peek(t: *const Tape) ?Kind { if (t.in.bytes.len - t.in.at < 5) return null; - return @enumFromInt(t.in.bytes[t.in.at]); + return @fromBackingInt(@intCast(t.in.bytes[t.in.at])); } fn diverge(t: *Tape, comptime why: []const u8, args: anytype) void { @@ -374,7 +395,8 @@ pub const Tape = struct { const tag = std.mem.bytesToValue(Tag, (r.take(@sizeOf(Tag)) catch return t.diverge("a short input", .{}))); r.at -= @sizeOf(Tag); const kept = tag == .init or tag == .init_from_dump or tag == .restore; - const input = walk.decode(Input, r, if (kept) t.gpa else t.arena.allocator()) catch return t.diverge("a short input", .{}); + const keep = if (t.kept_arena) |*k| k.allocator() else t.gpa; + const input = walk.decode(Input, r, if (kept) keep else t.arena.allocator()) catch return t.diverge("a short input", .{}); const outer = t.applying; t.applying = switch (input) { .update => |ev| @tagName(ev), @@ -390,13 +412,14 @@ pub const Tape = struct { .init_from_dump => |d| d.opts, else => unreachable, }; - opts.image_allocator = t.gpa; - opts.pdf_allocator = t.gpa; - opts.tree_sitter_allocator = t.gpa; - opts.frame_allocator = t.gpa; + const gpa = t.core_gpa orelse t.gpa; + opts.image_allocator = gpa; + opts.pdf_allocator = gpa; + opts.tree_sitter_allocator = gpa; + opts.frame_allocator = gpa; t.core = switch (input) { - .init => Pardes.init(t.gpa, opts), - .init_from_dump => |d| pardes.dump.initFromDump(t.gpa, opts, d.bytes), + .init => Pardes.init(gpa, opts), + .init_from_dump => |d| pardes.dump.initFromDump(gpa, opts, d.bytes), else => unreachable, } catch |err| return t.diverge("init failed: {t}", .{err}); t.core.?.host = t.hostOf(t.mask); @@ -608,8 +631,21 @@ fn hostCall(comptime i: usize, comptime R: type, ctx: ?*anyopaque, args: anytype } return walk.decode(R, &ret, t.arena.allocator()) catch std.mem.zeroes(R); } - const real = @field(t.real.vtable.*, name).?; const id: u16 = i; + if (t.speculating) { + // Nobody watches a replica's step: its frames are its own, drawn + // outside it (and the authority's host says so too, or the + // comparison fails). + if (comptime std.mem.eql(u8, name, "watched")) { + const r = false; + t.record(.host, u16, &id); + t.record(.ret, R, &r); + return r; + } + t.poisoned = true; + return poisonOf(R); + } + const real = @field(t.real.vtable.*, name).?; t.record(.host, u16, &id); t.seeFields(); const outer = !t.host_side; @@ -639,7 +675,7 @@ pub fn value(comptime s: Site, comptime f: anytype, args: anytype) @typeInfo(@Ty fn call(comptime s: Site, a: ?std.mem.Allocator, comptime f: anytype, args: anytype, comptime yields: bool) @typeInfo(@TypeOf(f)).@"fn".return_type.? { const R = @typeInfo(@TypeOf(f)).@"fn".return_type.?; const t = on orelse return if (yields) out(f, args) else @call(.auto, f, args); - const id: u16 = @as(u16, host_fns.len) + @intFromEnum(s); + const id: u16 = @as(u16, host_fns.len) + @backingInt(s); if (t.replay) { if (!t.expectHost(id, @tagName(s))) { std.debug.print("replay: {s}\n", .{t.diverged orelse "?"}); @@ -651,6 +687,12 @@ fn call(comptime s: Site, a: ?std.mem.Allocator, comptime f: anytype, args: anyt // The host's own (a shell's spawn asking the shell), or one inside a // site's syscall: the outer call is the record. if (t.host_side) return if (yields) out(f, args) else @call(.auto, f, args); + // A syscall a speculating replica must not make: its input waits for + // the authority's answer. + if (yields and t.speculating) { + t.poisoned = true; + return poisonOf(R); + } t.record(.host, u16, &id); t.seeFields(); t.host_side = true; @@ -679,7 +721,173 @@ fn out(comptime f: anytype, args: anytype) @typeInfo(@TypeOf(f)).@"fn".return_ty /// Asked by `turn.yield`: a syscall a replica must never make. pub fn replaying() bool { - return if (on) |t| t.replay else false; + return if (on) |t| t.replay or t.speculating else false; +} + +/// A replica speculating (src/replica.zig): state only the authority +/// changes (a terminal's emulator) is left alone, and the input waits. +pub fn speculating() bool { + return if (on) |t| t.speculating else false; +} + +/// What a speculation reached is the authority's: the input is undone and +/// waits for the authority's order. +pub fn poison() void { + if (on) |t| if (t.speculating) { + t.poisoned = true; + }; +} + +/// A harmless stand-in for what a host call or a syscall would have +/// returned to a speculation that is thrown away once it returns: an +/// error where one can be said, else zeroes. +fn poisonOf(comptime R: type) R { + return switch (@typeInfo(R)) { + .error_union => |e| if (@typeInfo(e.error_set).error_set.error_names) |errors| + (if (errors.len > 0) @field(e.error_set, errors[0]) else std.mem.zeroes(R)) + else + error.Unexpected, + .optional => null, + .void => {}, + .@"union" => |u| inline for (u.field_names, u.field_types) |name, F| { + if (comptime zeroable(F)) break @unionInit(R, name, std.mem.zeroes(F)); + } else @compileError("tape: no harmless value of " ++ @typeName(R)), + else => std.mem.zeroes(R), + }; +} + +fn zeroable(comptime T: type) bool { + return switch (@typeInfo(T)) { + .int, .float, .bool, .optional, .void => true, + .pointer => |p| p.size == .slice, + .array => |a| zeroable(a.child), + .@"struct" => |s| blk: { + for (s.field_types) |F| if (!zeroable(F)) break :blk false; + break :blk true; + }, + else => false, + }; +} + +// ---- a replica's tapes in memory (src/replica.zig) ---- + +/// A tape recorded in memory: an authority's stream, taken unit by unit. +pub fn memory(gpa: std.mem.Allocator) Tape { + return .{ .gpa = gpa, .replay = false, .scratch = .init(gpa), .arena = .init(gpa), .kept_arena = .init(gpa) }; +} + +/// What a tape in memory holds, given back (its core is gone already). +pub fn release(t: *Tape) void { + t.buf.deinit(t.gpa); + t.scratch.deinit(); + t.arena.deinit(); + if (t.kept_arena) |*k| k.deinit(); +} + +/// What was recorded since the last take; the tape's to reuse after. +pub fn taken(t: *Tape) []const u8 { + return t.buf.items; +} + +pub fn clearTaken(t: *Tape) void { + t.buf.clearRetainingCapacity(); +} + +/// The records of `bytes`, top-level inputs and all, applied to the +/// tape's core as the authority applied them, each digest checked; why +/// they diverged, or null. +pub fn replayUnit(t: *Tape, bytes: []const u8) ?[]const u8 { + t.replay = true; + t.speculating = false; + defer t.replay = false; + t.in = .{ .bytes = bytes }; + while (t.diverged == null) { + const kind = t.peek() orelse break; + var rec = t.next().?; + switch (kind) { + .input => { + t.apply(&rec.payload); + _ = t.arena.reset(.retain_capacity); + }, + .set => t.applySet(&rec.payload), + .digest => t.check(&rec.payload), + .host, .ret, .done => t.diverge("a {t} record at the top", .{kind}), + } + } + return t.diverged; +} + +/// Starts recording `t`'s core as a speculation: the host's watched +/// fields as they are now, nothing recorded yet. +pub fn beginSpeculation(t: *Tape) void { + t.replay = false; + t.speculating = true; + t.poisoned = false; + t.no_digest = true; + t.buf.clearRetainingCapacity(); + t.seeFields(); +} + +/// Whether it stayed clean (no host call, no syscall, no authority state). +pub fn endSpeculation(t: *Tape) bool { + t.speculating = false; + t.no_digest = false; + return !t.poisoned; +} + +/// Whether two tapes record the same inputs, host calls and answers, +/// their digests aside. +pub fn sameRecords(a: []const u8, b: []const u8) bool { + var x: walk.Reader = .{ .bytes = a }; + var y: walk.Reader = .{ .bytes = b }; + while (true) { + const ra = nextRecord(&x); + const rb = nextRecord(&y); + if (ra == null or rb == null) return ra == null and rb == null; + if (ra.?.kind != rb.?.kind or !std.mem.eql(u8, ra.?.payload, rb.?.payload)) return false; + } +} + +fn nextRecord(r: *walk.Reader) ?struct { kind: Kind, payload: []const u8 } { + while (true) { + const head = r.take(5) catch return null; + const payload = r.take(std.mem.bytesToValue(u32, head[1..5])) catch return null; + const kind: Kind = @fromBackingInt(@intCast(head[0])); + if (kind != .digest) return .{ .kind = kind, .payload = payload }; + } +} + +/// A tape's records, one a line (kind; an input's or a host call's name), +/// for a divergence's report. +pub fn describe(bytes: []const u8, w: *std.Io.Writer) void { + var r: walk.Reader = .{ .bytes = bytes }; + var depth: usize = 0; + while (true) { + const head = r.take(5) catch return; + const payload = r.take(std.mem.bytesToValue(u32, head[1..5])) catch return; + const kind: Kind = @fromBackingInt(@intCast(head[0])); + if (kind == .done or kind == .ret) depth -|= 1; + w.splatByteAll(' ', depth * 2) catch {}; + switch (kind) { + .input => { + const tag = std.mem.bytesToValue(std.meta.Tag(Input), payload[0..@sizeOf(std.meta.Tag(Input))]); + w.print("input {t}", .{tag}) catch {}; + if (tag == .update) w.print(" {t}", .{std.mem.bytesToValue(std.meta.Tag(pardes.Event), payload[@sizeOf(std.meta.Tag(Input))..][0..@sizeOf(std.meta.Tag(pardes.Event))])}) catch {}; + }, + .host => { + const id = std.mem.bytesToValue(u16, payload[0..2]); + w.print("host {s}", .{if (id < host_fns.len) host_fns[id] else @tagName(@as(Site, @fromBackingInt(@intCast(id - host_fns.len))))}) catch {}; + }, + else => w.print("{t} ({d} bytes)", .{ kind, payload.len }) catch {}, + } + w.writeByte('\n') catch {}; + if (kind == .input or kind == .host) depth += 1; + } +} + +/// The digest a replica and its authority must agree on. +pub fn digestOf(p: *const Pardes) u64 { + return walk.digest(Pardes, p); } /// PARDES_RECORD set: the tape is written from the session's first call. diff --git a/src/terminal.zig b/src/terminal.zig index e737a831..81fae5db 100644 --- a/src/terminal.zig +++ b/src/terminal.zig @@ -417,9 +417,18 @@ pub fn panFor(p: *const Pardes, pane: *const Pane, r: pardes.Rect) u16 { return @intCast(@min(pane.rows - shown, (at + 1) -| shown)); } +/// A replica's speculation leaves an emulator be: it is the authority's +/// (docs/next-daemon.md, phase 5), so the input waits for its order. +fn authorityOnly() bool { + if (!pardes.tape.speculating()) return false; + pardes.tape.poison(); + return true; +} + /// Move the emulator's viewport by `delta` shell rows (negative scrolls back). pub fn scrollGrid(pane: *Pane, delta: i32) void { if (comptime !enabled) return; + if (authorityOnly()) return; const state = pane.terminal orelse return; state.vt.screens.active.scroll(.{ .delta_row = delta }); } @@ -434,6 +443,7 @@ pub fn atBottom(pane: *const Pane) bool { /// Snap the viewport back onto live output. pub fn followOutput(pane: *Pane) void { if (comptime !enabled) return; + if (authorityOnly()) return; const state = pane.terminal orelse return; state.vt.screens.active.scroll(.active); } @@ -442,6 +452,7 @@ pub fn followOutput(pane: *Pane) void { /// a scrollback; the next resize retries with the same numbers. pub fn resizeGrid(pane: *Pane, gpa: std.mem.Allocator, cols: u16, rows: u16) void { if (comptime !enabled) return; + if (authorityOnly()) return; const state = pane.terminal orelse return; if (cols != pane.cols) for (0..pane.sel.len) |button| { pane.clearPointerSelection(button); @@ -643,6 +654,7 @@ pub fn screenTextLen(pane: *Pane) usize { /// Release the emulator's heap. The Pane allocation itself is the core's. pub fn deinitEmulator(pane: *Pane, gpa: std.mem.Allocator) void { if (comptime !enabled) return; + if (authorityOnly()) return; const state = pane.terminal orelse return; if (comptime images_enabled) state.images.deinit(gpa); state.stream.deinit(); @@ -654,6 +666,7 @@ pub fn deinitEmulator(pane: *Pane, gpa: std.mem.Allocator) void { /// Allocate the live emulator half of a terminal pane. Slot ownership, serial /// assignment, and spawn effects remain core lifecycle invariants. pub fn create(gpa: std.mem.Allocator, cols: u16, rows: u16) !*Pane { + if (authorityOnly()) return error.OutOfMemory; const pane = try createDoc(gpa, cols, rows); errdefer gpa.destroy(pane); if (comptime !enabled) return pane; @@ -674,6 +687,9 @@ pub fn create(gpa: std.mem.Allocator, cols: u16, rows: u16) !*Pane { }); state.stream = .init(.{ .handler = .{ .inner = state.vt.vtHandler() }, .allocator = gpa }); state.reply_len = 0; + // Every field its default (create leaves them what the heap had, which + // a digest then reads: two replicas must agree, src/replica.zig). + state.reply_dropped = 0; state.mouse_cell = null; state.images = if (images_enabled) .{} else {}; if (comptime images_enabled) TerminalImages.installPngDecoder(); @@ -900,6 +916,7 @@ pub fn feedOutput(p: *Pardes, pane: *Pane, bytes: []const u8) void { // There are no pty reads at all without an emulator to parse them into. if (comptime !enabled) return; if (pane.terminal == null) return; + if (authorityOnly()) return; const has_positions = pane.ovl != null or pane.body.cur_pinned or pane.body.vsel.active or pane.body.msel.active or pane.body.nsel > 0 or pane.body.last_edit != null or pane.look_at != null or pane.body.ed_undo_len > 0 or pane.body.ed_redo_len > 0 or pointerRowCount(pane) > 0; diff --git a/src/walk.zig b/src/walk.zig index dc1664cb..3e482e9a 100644 --- a/src/walk.zig +++ b/src/walk.zig @@ -42,7 +42,7 @@ pub fn fieldDigests(comptime T: type, v: *const T, out: []u64) []u64 { if (attr.@"comptime" or comptime excluded(T, name)) continue; var h = Hasher.init(0); if (comptime lengthOf(T, name)) |len| { - hashSlice(@typeInfo(F).array.child, @field(v, name)[0..lengthIn(v, len)], &h); + hashSlice(Element(F), @field(v, name)[0..lengthIn(v, len)], &h); } else hash(F, &@field(v, name), &h); out[n] = h.final(); n += 1; @@ -63,7 +63,7 @@ pub fn fieldNames(comptime T: type) []const []const u8 { } /// Types that are machinery, never state: an allocator, the Io, a lock. -fn machinery(comptime T: type) bool { +pub fn machinery(comptime T: type) bool { inline for (.{ std.mem.Allocator, std.Io, std.Io.Mutex, std.Io.Condition, std.Thread, std.heap.ArenaAllocator }) |M| if (T == M) return true; return false; } @@ -92,7 +92,7 @@ fn opaqueTo(comptime F: type) bool { }; } -fn excluded(comptime T: type, comptime name: []const u8) bool { +pub fn excluded(comptime T: type, comptime name: []const u8) bool { @setEvalBranchQuota(100_000); if (!@hasDecl(T, "replica")) return false; inline for (.{ "local", "derived", "authority" }) |kind| { @@ -104,12 +104,22 @@ fn excluded(comptime T: type, comptime name: []const u8) bool { } /// How far the field `len` names says its array is written (null: none). -fn lengthIn(v: anytype, comptime len: []const u8) usize { +/// What a `len`-cut field holds: a fixed array's, or a slice's room (a +/// selection's ranges, made once and used `nsel` deep). +fn Element(comptime F: type) type { + return switch (@typeInfo(F)) { + .array => |a| a.child, + .pointer => |p| p.child, + else => @compileError("walk: a `len` cuts an array or a slice, not " ++ @typeName(F)), + }; +} + +pub fn lengthIn(v: anytype, comptime len: []const u8) usize { const n = @field(v, len); return if (@typeInfo(@TypeOf(n)) == .optional) (if (n) |k| k else 0) else n; } -fn lengthOf(comptime T: type, comptime name: []const u8) ?[]const u8 { +pub fn lengthOf(comptime T: type, comptime name: []const u8) ?[]const u8 { @setEvalBranchQuota(100_000); if (!@hasDecl(T, "replica") or !@hasField(@TypeOf(T.replica), "len")) return null; inline for (T.replica.len) |pair| if (comptime std.mem.eql(u8, pair[0], name)) return pair[1]; @@ -134,7 +144,7 @@ pub fn hash(comptime T: type, v: *const T, h: *Hasher) void { .bool => h.update(&.{@intFromBool(v.*)}), .int => hashInt(T, v.*, h), .float => h.update(std.mem.asBytes(v)), - .@"enum" => |e| hashInt(e.tag_type, @intFromEnum(v.*), h), + .@"enum" => |e| hashInt(e.tag_type, @backingInt(v.*), h), .optional => |o| if (v.*) |*payload| { h.update(&.{1}); hash(o.child, payload, h); @@ -167,7 +177,7 @@ pub fn hash(comptime T: type, v: *const T, h: *Hasher) void { if (attr.@"comptime" or comptime excluded(T, name)) continue; if (comptime opaqueTo(F)) @compileError("walk: " ++ @typeName(T) ++ "." ++ name ++ " is " ++ @typeName(F) ++ ", which no walk sees into: say what it is (`replica`)"); if (comptime lengthOf(T, name)) |len| { - hashSlice(@typeInfo(F).array.child, @field(v, name)[0..lengthIn(v, len)], h); + hashSlice(Element(F), @field(v, name)[0..lengthIn(v, len)], h); } else hash(F, &@field(v, name), h); } }, diff --git a/test/replicas.zig b/test/replicas.zig new file mode 100644 index 00000000..cdb161e8 --- /dev/null +++ b/test/replicas.zig @@ -0,0 +1,327 @@ +//! pardes-replicas: one authority and several replicas of a session in one +//! process (src/replica.zig), each client typing at once over its own +//! delayed, jittered link, as monkey.zig's scripts type at a tty, each in a +//! view of its own (src/views.zig) over the text they share; `--shared` +//! puts them all in one, where they clash at every key: a stress of +//! correctness, not a picture of a session. +//! +//! zig build replicas -Dplatform=tty -- [--seeds N] [--steps M] [--clients C] [--rtt 30,80,207] [--first K] [--shared] +//! +//! Gates, per seed and RTT: no replica diverges (every confirmed unit's +//! digests agree as it is replayed), and at quiescence every replica's +//! digest is the authority's. Reported per RTT: how inputs went (applied at +//! once, stopped at a host call, left to the authority), how units were +//! taken (fast path, rollback for a misprediction or another's input), and +//! what a speculative apply, a copy and a rollback cost against a frame. +const std = @import("std"); +const libc = std.c; +const pardes = @import("pardes"); +const replica = pardes.replica; +const Key = pardes.Key; + +const gpa = std.heap.c_allocator; +var trace = false; +const frame_us = 16_667; + +/// The authority's host: no frames drawn (a replica draws its own), and a +/// pty that echoes what it is written, 2 ms later, as a shell's. +const Host = struct { + echo: std.ArrayListUnmanaged(struct { pane: u8, bytes: []u8 }) = .empty, + + fn watched(_: ?*anyopaque) bool { + return false; + } + + fn ptyWrite(ctx: ?*anyopaque, pane: u8, bytes: []const u8) void { + const h: *Host = @ptrCast(@alignCast(ctx.?)); + h.echo.append(gpa, .{ .pane = pane, .bytes = gpa.dupe(u8, bytes) catch return }) catch {}; + } + + fn ignore(_: ?*anyopaque, _: []const u8) void {} + + const vtable: pardes.Host.VTable = .{ .watched = watched, .pty_write = ptyWrite, .set_clipboard = ignore }; + + fn host(h: *Host) pardes.Host { + return .{ .ctx = h, .vtable = &vtable }; + } +}; + +const Kind = enum { input, to_authority, to_replica, echo }; + +const Msg = struct { + at: u64, + n: u64, + kind: Kind, + who: u16, + bytes: []u8, + + fn order(_: void, a: Msg, b: Msg) std.math.Order { + if (a.at != b.at) return std.math.order(a.at, b.at); + return std.math.order(a.n, b.n); + } +}; + +const Link = struct { + last: u64 = 0, + /// When a message sent at `now` arrives: half the RTT, jitter on top, + /// and never before the one sent ahead of it (a stream keeps order). + fn arrive(l: *Link, random: std.Random, now: u64, one_way: u64, jitter: u64) u64 { + const at = @max(l.last + 1, now + one_way + random.uintLessThan(u64, jitter + 1)); + l.last = at; + return at; + } +}; + +const Totals = struct { + inputs: u64 = 0, + speculated: u64 = 0, + stopped: u64 = 0, + unspeculated: u64 = 0, + units: u64 = 0, + fast: u64 = 0, + mispredicted: u64 = 0, + foreign: u64 = 0, + rebases: u64 = 0, + copies: u64 = 0, + copy_bytes: u64 = 0, + divergences: u64 = 0, + unequal: u64 = 0, + spec_ns: std.ArrayListUnmanaged(u64) = .empty, + copy_ns: std.ArrayListUnmanaged(u64) = .empty, + rollback_ns: std.ArrayListUnmanaged(u64) = .empty, + + fn add(t: *Totals, s: *const replica.Stats) !void { + t.inputs += s.inputs; + t.speculated += s.speculated; + t.stopped += s.stopped; + t.unspeculated += s.unspeculated; + t.units += s.units; + t.fast += s.fast; + t.mispredicted += s.mispredicted; + t.foreign += s.foreign; + t.rebases += s.rebases; + t.copies += s.copies; + t.copy_bytes += s.copy_bytes; + try t.spec_ns.appendSlice(gpa, s.spec_ns.items); + try t.copy_ns.appendSlice(gpa, s.copy_ns.items); + try t.rollback_ns.appendSlice(gpa, s.rollback_ns.items); + } +}; + +/// One client's next input: monkey.zig's mix, as events. +fn nextEvent(random: std.Random, buf: *[64]u8) pardes.Event { + const typed = "abcdefghijklmnopqrstuvwxyz0123456789 -_=+,:;()"; + const normal = "hjklwbe0$xpuyoOiaAIdcv.nN%sC"; + switch (random.uintLessThan(u8, 40)) { + 0...15 => { + buf[0] = typed[random.uintLessThan(usize, typed.len)]; + return .{ .key = .{ .cp = buf[0], .text = buf[0..1] } }; + }, + 16...27 => { + buf[0] = normal[random.uintLessThan(usize, normal.len)]; + return .{ .key = .{ .cp = buf[0], .text = buf[0..1] } }; + }, + 28...31 => return .{ .key = .{ .cp = Key.escape } }, + 32 => return .{ .key = .{ .cp = Key.enter } }, + 33 => return .{ .key = .{ .cp = Key.backspace } }, + 34 => return .{ .key = .{ .cp = switch (random.uintLessThan(u8, 4)) { + 0 => Key.up, + 1 => Key.down, + 2 => Key.left, + else => Key.right, + } } }, + 35 => { + const n = random.intRangeAtMost(usize, 1, 24); + for (buf[0..n]) |*c| c.* = typed[random.uintLessThan(usize, typed.len)]; + return .{ .paste = buf[0..n] }; + }, + 36 => return .{ .command = switch (random.uintLessThan(u8, 4)) { + 0 => "Newcol", + 1 => "Zerox", + 2 => "Undo", + else => "Redo", + } }, + 37 => return .{ .resize = .{ .cols = @intCast(random.intRangeAtMost(u16, 60, 120)), .rows = @intCast(random.intRangeAtMost(u16, 16, 40)) } }, + 38 => return .{ .mouse = .{ .button = .left, .kind = .press, .col = random.uintLessThan(u16, 80), .row = random.uintLessThan(u16, 24) } }, + else => return .{ .mouse = .{ .button = .left, .kind = .release, .col = random.uintLessThan(u16, 80), .row = random.uintLessThan(u16, 24) } }, + } +} + +fn writeFile(path: [:0]const u8, lines: usize) !void { + const fd = libc.open(path, .{ .ACCMODE = .WRONLY, .CREAT = true, .TRUNC = true }, @as(libc.mode_t, 0o644)); + if (fd < 0) return error.OpenFailed; + defer _ = libc.close(fd); + var line: [80]u8 = undefined; + for (0..lines) |n| { + const text = try std.fmt.bufPrint(&line, "const value_{d:0>4} = {d}; // a line to edit\n", .{ n, n *% 7919 }); + if (libc.write(fd, text.ptr, text.len) != @as(isize, @intCast(text.len))) return error.WriteFailed; + } +} + +const Run = struct { seed: u64, rtt_ms: u64, steps: usize, clients: u16, shared: bool }; + +fn run(r: Run, dir: []const u8, totals: *Totals) !void { + if (trace) std.debug.print("seed {d} rtt {d} ms\n", .{ r.seed, r.rtt_ms }); + var prng = std.Random.DefaultPrng.init(r.seed *% 0x9e3779b97f4a7c15 +% r.rtt_ms); + const random = prng.random(); + var host: Host = .{}; + defer { + for (host.echo.items) |e| gpa.free(e.bytes); + host.echo.deinit(gpa); + } + var paths: [3][256:0]u8 = undefined; + for (&paths, 0..) |*p, i| { + const path = try std.fmt.bufPrintSentinel(p, "{s}/f{d}.zig", .{ dir, i }, 0); + try writeFile(path, 60 + 40 * i); + } + // The tty's own boot (a raw shell, a text pane under it), then three + // files looked at: keys go to the last, a pane's shell echoes. + const a = try replica.Authority.init(gpa, .{ .cols = 100, .rows = 30, .boot = .tty }, host.host()); + defer a.deinit(); + const first = try gpa.dupe(u8, try a.unit(0, 0)); + defer gpa.free(first); + var reps: [8]*replica.Replica = undefined; + for (reps[0..r.clients], 1..) |*rp, id| rp.* = try replica.Replica.init(gpa, @intCast(id), first); + defer for (reps[0..r.clients]) |rp| rp.deinit(); + for (&paths) |*path| { + var line: [300]u8 = undefined; + const unit = try a.local(.{ .command = try std.fmt.bufPrint(&line, "Look {s}", .{std.mem.sliceTo(path, 0)}) }); + for (reps[0..r.clients]) |rp| rp.confirm(unit); + } + // Each client a view of its own (src/views.zig), the first the + // session's; or, --shared, every client in the one. + if (!r.shared) for (reps[1..r.clients], 2..) |rp, id| { + const opened = try a.openView(@intCast(id)); + rp.view = opened.view; + for (reps[0..r.clients]) |each| each.confirm(opened.unit); + }; + var queue: std.PriorityQueue(Msg, void, Msg.order) = .empty; + defer queue.deinit(gpa); + var n: u64 = 0; + const one_way = r.rtt_ms * 1000 / 2; + const jitter = one_way / 10; + var up: [8]Link = @splat(.{}); + var down: [8]Link = @splat(.{}); + var left: [8]usize = @splat(r.steps); + // Each client's first input, soon. + for (0..r.clients) |c| { + try queue.push(gpa, .{ .at = random.uintLessThan(u64, 200_000), .n = n, .kind = .input, .who = @intCast(c + 1), .bytes = &.{} }); + n += 1; + } + while (queue.pop()) |m| { + defer if (m.bytes.len > 0) gpa.free(m.bytes); + switch (m.kind) { + .input => { + const c = m.who - 1; + var buf: [64]u8 = undefined; + const ev = nextEvent(random, &buf); + const before = reps[c].stats; + const msg = try gpa.dupe(u8, try reps[c].input(ev)); + if (trace) std.debug.print("{d} ms client {d}: {t} {s} -> {s}\n", .{ m.at / 1000, m.who, ev, switch (ev) { + .key => |k| k.text, + .paste => |t| t, + .command => |t| t, + else => "", + }, if (reps[c].stats.speculated > before.speculated) "applied" else if (reps[c].stats.stopped > before.stopped) "STOPPED" else if (reps[c].stats.unspeculated > before.unspeculated) "view" else "waits" }); + try queue.push(gpa, .{ .at = up[c].arrive(random, m.at, one_way, jitter), .n = n, .kind = .to_authority, .who = m.who, .bytes = msg }); + n += 1; + left[c] -= 1; + if (left[c] > 0) { + // A typist: keys 50-200 ms apart, a pause now and then. + const gap: u64 = if (random.uintLessThan(u8, 12) == 0) random.intRangeAtMost(u64, 300_000, 1_500_000) else random.intRangeAtMost(u64, 50_000, 200_000); + try queue.push(gpa, .{ .at = m.at + gap, .n = n, .kind = .input, .who = m.who, .bytes = &.{} }); + n += 1; + } + }, + .to_authority, .echo => { + const unit = if (m.kind == .echo) + try a.local(.{ .output = .{ .pane = @intCast(m.who), .bytes = m.bytes } }) + else + try a.take(m.bytes); + for (0..r.clients) |c| { + const copy = try gpa.dupe(u8, unit); + try queue.push(gpa, .{ .at = down[c].arrive(random, m.at, one_way, jitter), .n = n, .kind = .to_replica, .who = @intCast(c + 1), .bytes = copy }); + n += 1; + } + for (host.echo.items) |e| { + try queue.push(gpa, .{ .at = m.at + 2_000, .n = n, .kind = .echo, .who = e.pane, .bytes = e.bytes }); + n += 1; + } + host.echo.clearRetainingCapacity(); + }, + .to_replica => { + const rp = reps[m.who - 1]; + rp.confirm(m.bytes); + rp.settle(); + }, + } + } + const want = replica.digestOf(a); + for (reps[0..r.clients], 1..) |rp, id| { + try totals.add(&rp.stats); + if (rp.diverged) |why| { + totals.divergences += 1; + std.debug.print("seed {d} rtt {d} ms: replica {d} diverged: {s}\n", .{ r.seed, r.rtt_ms, id, why }); + } else if (!rp.quiet() or rp.digest() != want) { + totals.unequal += 1; + std.debug.print("seed {d} rtt {d} ms: replica {d} at quiescence: {s}\n", .{ r.seed, r.rtt_ms, id, if (rp.quiet()) "another digest" else "inputs still pending" }); + } + } +} + +fn pct(samples: []u64, p: usize) u64 { + if (samples.len == 0) return 0; + std.mem.sort(u64, samples, {}, std.sort.asc(u64)); + return samples[@min(samples.len - 1, samples.len * p / 100)]; +} + +pub fn main(init: std.process.Init) !void { + const args = try init.minimal.args.toSlice(init.arena.allocator()); + var seeds: u64 = 4; + var first: u64 = 0; + var steps: usize = 120; + var clients: u16 = 3; + var rtts: []const u64 = &.{ 30, 80, 207 }; + var shared = false; + var rtt_buf: [8]u64 = undefined; + var i: usize = 1; + while (i < args.len and std.mem.eql(u8, args[i], "--shared")) : (i += 1) shared = true; + while (i + 1 < args.len) : (i += 2) { + while (i < args.len and std.mem.eql(u8, args[i], "--shared")) : (i += 1) shared = true; + if (i + 1 >= args.len) break; + const name = args[i]; + const value = args[i + 1]; + if (std.mem.eql(u8, name, "--seeds")) seeds = try std.fmt.parseInt(u64, value, 10) else if (std.mem.eql(u8, name, "--first")) first = try std.fmt.parseInt(u64, value, 10) else if (std.mem.eql(u8, name, "--steps")) steps = try std.fmt.parseInt(usize, value, 10) else if (std.mem.eql(u8, name, "--clients")) clients = @min(8, try std.fmt.parseInt(u16, value, 10)) else if (std.mem.eql(u8, name, "--rtt")) { + var it = std.mem.tokenizeScalar(u8, value, ','); + var k: usize = 0; + while (it.next()) |v| : (k += 1) rtt_buf[k] = try std.fmt.parseInt(u64, v, 10); + rtts = rtt_buf[0..k]; + } else return error.UnknownOption; + } + trace = libc.getenv("PARDES_REPLICA_TRACE") != null; + if (libc.getenv("PARDES_REPLICA_FILL")) |b| pardes.rollback.Epoch.fill = std.fmt.parseInt(u8, std.mem.span(b), 0) catch 0xff; + var dir_buf: [256]u8 = undefined; + const tmp = std.mem.span(libc.getenv("TMPDIR") orelse "/tmp"); + const dir = try std.fmt.bufPrint(&dir_buf, "{s}/replicas-{d}", .{ tmp, @as(u32, @intCast(libc.getpid())) }); + try std.Io.Dir.createDirAbsolute(init.io, dir, .default_dir); + defer std.Io.Dir.cwd().deleteTree(init.io, dir) catch {}; + var failed = false; + std.debug.print("{d} seeds x {d} inputs from each of {d} clients, {s}; costs in us (frame {d})\n", .{ seeds, steps, clients, if (shared) "sharing one view" else "a view each", frame_us }); + std.debug.print(" rtt inputs at-once stopped view units fast rollback(mis+foreign) rate rebase spec p50/p99 copy p50/p99 (KB) rollback p50/p99 diverged unequal\n", .{}); + for (rtts) |rtt| { + var t: Totals = .{}; + for (first..first + seeds) |seed| try run(.{ .seed = seed, .rtt_ms = rtt, .steps = steps, .clients = clients, .shared = shared }, dir, &t); + const rollbacks = t.mispredicted + t.foreign; + std.debug.print("{d:>4} {d:>6} {d:>7} {d:>7} {d:>4} {d:>6} {d:>6} {d:>6} ({d}+{d}) {d:>5.1}% {d:>6} {d:>5}/{d:<6} {d:>5}/{d:<5} ({d}) {d:>7}/{d:<7} {d:>8} {d:>7}\n", .{ + rtt, t.inputs, t.speculated, + t.stopped, t.unspeculated, t.units, + t.fast, rollbacks, t.mispredicted, + t.foreign, @as(f64, @floatFromInt(rollbacks)) * 100 / @as(f64, @floatFromInt(@max(t.inputs, 1))), t.rebases, + pct(t.spec_ns.items, 50) / 1000, pct(t.spec_ns.items, 99) / 1000, pct(t.copy_ns.items, 50) / 1000, + pct(t.copy_ns.items, 99) / 1000, t.copy_bytes / @max(t.copies, 1) / 1024, pct(t.rollback_ns.items, 50) / 1000, + pct(t.rollback_ns.items, 99) / 1000, t.divergences, t.unequal, + }); + if (t.divergences > 0 or t.unequal > 0) failed = true; + } + if (failed) std.process.exit(1); +} diff --git a/test/rollback_bench.zig b/test/rollback_bench.zig index 721c6f13..38fe011d 100644 --- a/test/rollback_bench.zig +++ b/test/rollback_bench.zig @@ -1,14 +1,11 @@ -//! What a replica's rollback copy would cost (docs/next-daemon.md, open -//! point 3), and what a fork of the authority stalls it for (open point 5). -//! A typical session and a big one are built in process; for each: -//! - the state the walker visits (src/walk.zig's digest walk, which is -//! what speculation can write: views, buffers, registers, counters, -//! terminal metadata), in bytes; -//! - the part of it that is pinned blobs: a file's text and its undo -//! snapshots, which an edit replaces rather than changes -//! (File.setContentSpan), so a copy keeps a reference; -//! - the walk's time, the blobs' share of it, and a memcpy of the rest: -//! a walker copy costs about the walk plus that write. +//! What a replica's rollback copy costs (docs/next-daemon.md, open point +//! 3, src/rollback.zig), and what a fork of the authority stalls it for +//! (open point 5). A typical session and a big one are built in process, on +//! a rollback epoch's allocator as a replica's core is; for each: +//! - the copy taken and let go: its time, the bytes it saves, the bytes +//! it pins (a file's text, its undo snapshots); +//! - a key that edits a file (`x`) applied on top, as a speculation, and +//! the copy put back: the apply's time and the restore's. //! Then a fork of the big one, child gone at once, and the authority's //! copy-on-write after it. //! @@ -69,8 +66,8 @@ fn writeFile(path: []const u8, lines: usize) !void { } } -fn build(s: Session, dir: []const u8) !*pardes.Pardes { - const core = try pardes.Pardes.init(gpa, .{ .tty_only = true }); +fn build(s: Session, dir: []const u8, a: std.mem.Allocator) !*pardes.Pardes { + const core = try pardes.Pardes.init(a, .{ .tty_only = true, .image_allocator = a, .pdf_allocator = a, .tree_sitter_allocator = a, .frame_allocator = a }); core.update(.{ .resize = .{ .cols = 200, .rows = 60 } }); pump(core); // The shell: pane 0 of a tty boot, its output fed as a pty's. @@ -112,36 +109,6 @@ fn build(s: Session, dir: []const u8) !*pardes.Pardes { return core; } -/// Where a file's text and its undo snapshots are held: what a copy pins. -fn blobs(core: *pardes.Pardes, out: *std.ArrayList(*[]u8)) !usize { - var total: usize = 0; - for (core.panes) |slot| if (slot) |pane| if (pane.file) |*f| { - try out.append(gpa, &f.content); - for (f.history.undo[0..f.history.undo_len]) |*snap| try out.append(gpa, &snap.content); - for (f.history.redo[0..f.history.redo_len]) |*snap| try out.append(gpa, &snap.content); - }; - for (out.items) |b| total += b.len; - return total; -} - -fn walkNs(core: *const pardes.Pardes) u64 { - var t: [reps]u64 = undefined; - var sink: u64 = 0; - for (&t) |*x| { - const t0 = nowNs(); - sink +%= walk.digest(pardes.Pardes, core); - x.* = nowNs() - t0; - } - std.mem.doNotOptimizeAway(sink); - return median(&t); -} - -fn walked(core: *const pardes.Pardes) usize { - var h = walk.Hasher.init(0); - walk.hash(pardes.Pardes, core, &h); - return h.total_len; -} - fn median(samples: []u64) u64 { std.mem.sort(u64, samples, {}, std.sort.asc(u64)); return samples[samples.len / 2]; @@ -161,49 +128,44 @@ pub fn main(init: std.process.Init) !void { const dir = try std.fmt.bufPrint(&dir_buf, "{s}/rollback-bench-{d}", .{ tmp, @as(u32, @intCast(libc.getpid())) }); try std.Io.Dir.createDirAbsolute(init.io, dir, .default_dir); defer std.Io.Dir.cwd().deleteTree(init.io, dir) catch {}; - std.debug.print("session panes walked MB blobs MB copy KB walk ms walk ms, blobs out memcpy ms copy ms (est.)\n", .{}); + std.debug.print("session panes copy KB pinned MB take ms release ms edit ms restore ms\n", .{}); var big: ?*pardes.Pardes = null; - for (sessions) |s| { - const core = try build(s, dir); - var list: std.ArrayList(*[]u8) = .empty; - defer list.deinit(gpa); - const blob_bytes = try blobs(core, &list); - const all = walked(core); - const walk_ns = walkNs(core); - // The blobs taken out, as a copy pins them: the walk of what a copy - // writes, then put back. - const kept = try gpa.alloc([]u8, list.items.len); - defer gpa.free(kept); - for (list.items, kept) |b, *k| { - k.* = b.*; - b.* = &.{}; + var epochs: [sessions.len]*pardes.rollback.Epoch = undefined; + for (sessions, &epochs) |s, *e| { + e.* = try gpa.create(pardes.rollback.Epoch); + e.*.* = .{}; + const core = try build(s, dir, e.*.allocator()); + var c: pardes.rollback.Copy = .{}; + defer c.deinit(); + var take: [reps]u64 = undefined; + var release: [reps]u64 = undefined; + for (&take, &release) |*x, *y| { + const t0 = nowNs(); + try c.take(e.*, core); + const t1 = nowNs(); + c.release(); + x.* = t1 - t0; + y.* = nowNs() - t1; } - const copy_bytes = walked(core); - const copy_walk_ns = walkNs(core); - for (list.items, kept) |b, k| b.* = k; - const src = try gpa.alloc(u8, @max(copy_bytes, 1)); - defer gpa.free(src); - const dst = try gpa.alloc(u8, @max(copy_bytes, 1)); - defer gpa.free(dst); - @memset(src, 1); - @memset(dst, 0); - var t: [reps]u64 = undefined; - for (&t) |*x| { + var edit: [reps]u64 = undefined; + var restore: [reps]u64 = undefined; + for (&edit, &restore) |*x, *y| { + try c.take(e.*, core); const t0 = nowNs(); - @memcpy(dst, src); - std.mem.doNotOptimizeAway(dst.ptr); - x.* = nowNs() - t0; + key(core, 'x', "x"); + const t1 = nowNs(); + c.restore(); + x.* = t1 - t0; + y.* = nowNs() - t1; } - const memcpy_ns = median(&t); var panes: usize = 0; for (core.panes) |slot| panes += @intFromBool(slot != null); const ms = 1e-6; - std.debug.print("{s:<8} {d:>5} {d:>9.1} {d:>8.1} {d:>7.1} {d:>7.3} {d:>18.3} {d:>9.3} {d:>14.3}\n", .{ - s.name, panes, - @as(f64, @floatFromInt(all)) / 1e6, @as(f64, @floatFromInt(blob_bytes)) / 1e6, - @as(f64, @floatFromInt(copy_bytes)) / 1e3, @as(f64, @floatFromInt(walk_ns)) * ms, - @as(f64, @floatFromInt(copy_walk_ns)) * ms, @as(f64, @floatFromInt(memcpy_ns)) * ms, - @as(f64, @floatFromInt(copy_walk_ns + memcpy_ns)) * ms, + std.debug.print("{s:<8} {d:>5} {d:>7.1} {d:>9.1} {d:>7.3} {d:>10.3} {d:>7.3} {d:>10.3}\n", .{ + s.name, panes, + @as(f64, @floatFromInt(c.savedBytes())) / 1e3, @as(f64, @floatFromInt(c.pinnedBytes())) / 1e6, + @as(f64, @floatFromInt(median(&take))) * ms, @as(f64, @floatFromInt(median(&release))) * ms, + @as(f64, @floatFromInt(median(&edit))) * ms, @as(f64, @floatFromInt(median(&restore))) * ms, }); if (big) |b| b.deinit(); big = core; -- cgit v1.3 From 70de5ec5ff3dad01afcc45244ad5e5a481486da6 Mon Sep 17 00:00:00 2001 From: Gabriel Schneider Date: Thu, 29 Oct 2026 17:01:35 -0300 Subject: dump: one walker dump of the whole session, by name across builds, for Restore, -l, Handover and the store; dump.zig's hand-written schema gone A dump is walk.zig's walk of the core with a table of every type it reaches (fields, enum and union names, number widths), read back by name; terminals are ghostty's snapshot (screens, history, kitty images); a failed load frees what it made. Co-Authored-By: Claude Opus 5.5 --- build.zig | 3 + build/snap.zig | 10 +- docs/typ/building.typ | 4 +- docs/typ/reference.typ | 35 +- src/File.zig | 387 --------------- src/Handover.zig | 112 +++-- src/Output.zig | 34 +- src/Pipe.zig | 4 + src/Query.zig | 9 +- src/Store.zig | 57 +-- src/builtins.zig | 2 +- src/dump.zig | 1176 +++++++++++----------------------------------- src/file_watch.zig | 57 +-- src/image.zig | 63 --- src/ninep/ctl.zig | 64 +-- src/ninep/events.zig | 24 +- src/panes.zig | 4 +- src/pardes.zig | 55 +-- src/pdf.zig | 4 +- src/pdf_view.zig | 14 + src/tagline.zig | 8 - src/tape.zig | 40 +- src/terminal.zig | 316 ++++++------- src/walk.zig | 1120 ++++++++++++++++++++++++++++++++++++++++++- test/fs.py | 5 +- test/fs_namespace.zig | 44 +- test/kitty.zig | 6 +- test/monkey.zig | 9 +- test/output.zig | 17 - test/panes.zig | 61 +-- test/pdf.zig | 39 -- test/snapshot.zig | 9 +- test/snapshots/dump.snap | 9 +- 33 files changed, 1851 insertions(+), 1950 deletions(-) (limited to 'src/tape.zig') diff --git a/build.zig b/build.zig index 1a5dd200..962b6322 100644 --- a/build.zig +++ b/build.zig @@ -1950,6 +1950,9 @@ pub fn build(b: *std.Build) void { run_shell_test.has_side_effects = true; unit_step.dependOn(&run_shell_test.step); if (unit_match) |check| check.addArtifactArg(shell_test); + // The native shell's tests alone, without the perf gate's + // ReleaseFast build: what a -Dtest-filter iteration wants. + b.step("shell-test", "run the native shell's unit tests (src/main.zig's)").dependOn(&run_shell_test.step); const run_profile = b.addRunArtifact(unit_profile); run_profile.addArtifactArg(shell_test); run_profile.addPassthruArgs(); diff --git a/build/snap.zig b/build/snap.zig index d74767d3..c1a26e64 100644 --- a/build/snap.zig +++ b/build/snap.zig @@ -69,19 +69,21 @@ pub fn addTty(b: *std.Build, opts: struct { // The replay-divergence gate (src/tape.zig, docs/next-daemon.md): every // snapshot script and `replay-seeds` monkey seeds with each session's // tape recorded, each replayed into two replicas at different - // addresses and one that never draws, every digest checked. + // addresses and one that never draws, every digest checked; and at each + // session's end its dump loaded, which must be that session, digest for + // digest, and dump again to the same bytes (phase 2). const tapes = ".zig-cache/tapes"; const run_replay = b.addRunArtifact(snap); run_replay.addArtifactArg(opts.exe); run_replay.addPrefixedFileArg("--lspmock=", lspmock.getEmittedBin()); - run_replay.addArg("--tape=" ++ tapes); + run_replay.addArgs(&.{ "--tape=" ++ tapes, "--dump" }); run_replay.setCwd(b.path(".")); run_replay.has_side_effects = true; const seeds = b.option(u32, "replay-seeds", "monkey seeds `zig build replay` records and replays (default 20)") orelse 20; const run_monkey_replay = b.addRunArtifact(monkey); run_monkey_replay.addArtifactArg(snap); run_monkey_replay.addArtifactArg(opts.exe); - run_monkey_replay.addArgs(&.{ b.fmt("{d}", .{seeds}), "70", "--tape=" ++ tapes ++ "/monkey", "--out=" ++ tapes ++ "/monkey" }); + run_monkey_replay.addArgs(&.{ b.fmt("{d}", .{seeds}), "70", "--tape=" ++ tapes ++ "/monkey", "--dump", "--out=" ++ tapes ++ "/monkey" }); run_monkey_replay.setCwd(b.path(".")); run_monkey_replay.has_side_effects = true; // One pardes-snap run at a time (its lock): the scripts, then the seeds. @@ -95,7 +97,7 @@ pub fn addTty(b: *std.Build, opts: struct { run_views_replay.setCwd(b.path(".")); run_views_replay.has_side_effects = true; run_views_replay.step.dependOn(&run_monkey_replay.step); - b.step("replay", "record every snapshot script's and -Dreplay-seeds monkey seeds' tapes (one terminal and three) and require each to replay without diverging").dependOn(&run_views_replay.step); + b.step("replay", "record every snapshot script's and -Dreplay-seeds monkey seeds' tapes (one terminal and three) and require each to replay without diverging, and each session's dump to load to its digest").dependOn(&run_views_replay.step); const monkey_tests = b.addTest(.{ .root_module = monkey.root_module }); b.step("monkey-test", "test the monkey's generator and panic detector").dependOn(&b.addRunArtifact(monkey_tests).step); const run_driver_test = b.addRunArtifact(snap); diff --git a/docs/typ/building.typ b/docs/typ/building.typ index 00088d7b..e42702c6 100644 --- a/docs/typ/building.typ +++ b/docs/typ/building.typ @@ -19,7 +19,7 @@ builds MuPDF and is on by default (`-Dmupdf=false` drops it). 9P over QUIC [`zig build`], [the terminal shell and the SDL window together, *installed into `~/.local/bin`* (`pardes`, `pardes-gui`, and `pardes-v9fs`, the #word("Tty9p") helper)], [`zig build -Dplatform=tty`], [the terminal shell alone], [`zig build -Dplatform=gui`], [the SDL3 window alone], - [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)], + [`zig build web -Dplatform=web -Dtarget=wasm32-freestanding -Ddump=`], [a freestanding wasm core plus vanilla JavaScript (`docs/web.md`)], [`zig build -Dplatform=macos`], [an AppKit and CoreText app over a static `libpardes.a` (`docs/macos.md`)], [`zig build -Dplatform=esp32p4`], [a freestanding riscv32 editor object for the ESP32-P4, with a 384 KiB heap], ) @@ -59,7 +59,7 @@ there; its namespace is in `src/esp32p4_gpio.zig`. [`-Dprebuilt-shaders`], [bool: embed the committed SPIR-V; on for a bare `zig build`, off with `-Dplatform`; `zig build shaders` refreshes it with glslc], [`-Dtracy`], [path to a Tracy checkout; off], [`-Dmacos-identity`], [codesigning identity for `pardes.app`; `-` (ad hoc)], - [`-Ddump`], [a `dump.zon` to embed in the web shell], + [`-Ddump`], [a Dump to embed in the web shell], [`-Dtest-filter`], [run only tests whose name contains it; a filter that matches nothing fails], [`-Dtest-rebuild`], [bool: fresh Zig test compilation], [`-Dhelix-harness`], [reference executable for live differential tests; `HX_HARNESS`, else `hx-harness` on `PATH`], diff --git a/docs/typ/reference.typ b/docs/typ/reference.typ index ce2a45f1..2e395cc6 100644 --- a/docs/typ/reference.typ +++ b/docs/typ/reference.typ @@ -368,22 +368,20 @@ Writes take settings and session builtins (`scope = .session` in #word("Delcol") and a pane's `get` refuse the same way with their own word, except that #word("Delcol") opens no `+Unsaved` pane: it names the panes only in its `unsaved` records and its notice. -- #word("Dump") writes `pardes--