diff options
| author | Gabriel Schneider <[email protected]> | 2026-09-27 17:56:32 -0300 |
|---|---|---|
| committer | Gabriel Schneider <[email protected]> | 2026-09-27 18:18:09 -0300 |
| commit | 5beb0167c3b9dc1a788e08b9c1652aaba8eb0a40 (patch) | |
| tree | e17d2c4f87a99f7c42ff2b5b47d9f206e0b8c169 /9ns/src/bridge.zig | |
| parent | 1c1b192d4ef59199a7196229d56901b1f6512678 (diff) | |
| download | cloud9-5beb0167c3b9dc1a788e08b9c1652aaba8eb0a40.tar.gz cloud9-5beb0167c3b9dc1a788e08b9c1652aaba8eb0a40.zip | |
9ns: keep many 9P requests in flight, so a parked read no longer freezes the mount
9ns served one FUSE request at a time per 9P server, so a read the server
parks (pardes's log, a pane's event) held up every other operation on the
mount: with 'cat log' parked, 'cat status' took 3.5-4.5 s instead of ~1 ms.
nine.zig: the session is shared by threads and multiplexes by tag the way
Plan 9's mount driver does (devmnt.c mountio/mountmux/mntgate/
mntflushalloc). Each thread calls through its own nine.Caller (reply buffer,
wake-up eventfd, interrupt source, ename). A call writes its T-message under
wlock; the first waiter with nobody reading takes the reading baton, reads
for everyone and hands each R-message to the caller its tag names, then
passes the baton on. Tflush is per tag: a reply that beats the Rflush is
honoured, the flushed tag is not reused before the Rflush, the baton is let
go before the flush is written. With every tag taken a call waits for one
(starved list) instead of failing.
bridge.zig: one thread (the loop) still owns the inode/handle/qid tables and
serves everything that touches them. READ and WRITE are resolved to their
fid on the loop and handed to transfer workers (up to 31 per bridge,
started on demand), which make the 9P calls on their own Caller and reply
themselves. FUSE_INTERRUPT for a worker's request: still queued -> EINTR on
the spot; in flight -> its unique down the worker's pipe -> Tflush of exactly
that tag. Fids are clunked only on RELEASE. mntgen mounts get the same pool
(their dispatcher routes interrupts to it); the mount interrupt pipe moved
into Bridge. main.zig attaches through a Caller.
Tests: two multiplexing tests in nine.zig (a parked read holds up only its
caller and its interrupt flushes exactly its tag; with every tag parked a
call waits for a tag), and a serve()-level test driving a fake /dev/fuse (a
parked READ holds up neither READs nor GETATTR; INTERRUPT -> one Tflush for
the parked tag -> EINTR; RELEASE). Also de-flaked the pre-existing
'short read' test (it raced its own server reply under load).
Measured (own 9ns mount on the live pardes socket, read-only): with log
parked, cat status 4498 ms before, 7 ms after.
Tag window: client.zig gains ClientWith(n) (Client = ClientWith(16) as
before; request/result types hoisted so all instantiations share them).
9ns uses ClientWith(32): 32 is fs.zig's default slot_capacity, so one 9ns
mount can never have more parked on a cloud9 fs server than it parks, and
never meets its "Resource temporarily unavailable" refusal. fs.zig is
unchanged (its default is also the ESP32 board's, held to 24 KiB by its
budget tests). Full window: the loop keeps its tag (metadata unaffected);
the 32nd READ/WRITE waits in the pool queue (blocks, interruptible, never
EAGAIN/EIO).
More tests (bridge.zig, serve() over a fake /dev/fuse): one open is one fid
(two opens of an inode read through distinct fids, concurrently, one of
them parked); FLUSH on a handle with a parked read clunks nothing and the
read later completes on that fid; only RELEASE clunks; a flushed READ whose
reply beats the Rflush returns the data, not EINTR.
Diffstat (limited to '9ns/src/bridge.zig')
| -rw-r--r-- | 9ns/src/bridge.zig | 846 |
1 files changed, 703 insertions, 143 deletions
diff --git a/9ns/src/bridge.zig b/9ns/src/bridge.zig index 327cde1..2226658 100644 --- a/9ns/src/bridge.zig +++ b/9ns/src/bridge.zig @@ -1,16 +1,29 @@ //! FUSE ↔ 9P2000 translation: the request loop that turns kernel FUSE requests //! into synchronous 9P calls on a `nine.Session` and sends the replies back. //! -//! Everything here is single-threaded and one request at a time. State is three -//! tables: inodes (nodeid → fid/qid, deduplicated by qid.path), open handles -//! (fh → fid plus a cached directory listing), and the reverse qid map. +//! State is three tables: inodes (nodeid → fid/qid, deduplicated by +//! qid.path), open handles (fh → fid plus a cached directory listing), and +//! the reverse qid map. One thread owns them, the loop, and serves every +//! request that reads or changes them one at a time. //! -//! One request at a time does not mean deaf: while a 9P reply is outstanding -//! the session polls the FUSE descriptor too (`nine.Interrupt`). A -//! FUSE_INTERRUPT for the request being served becomes a Tflush, and if the -//! server honours it the request fails with EINTR; anything else the kernel -//! sends meanwhile is copied into a queue and served next, so the FUSE fd -//! is always being read and no INTERRUPT waits behind a parked request. +//! READ and WRITE are the exception: the loop resolves the handle to its fid +//! and hands the transfer to a worker thread (`Pool`), which makes the 9P +//! calls on its own `nine.Caller` and replies itself. The session multiplexes +//! them by tag, so a read the server parks (a blocking `log` or `event` +//! file) holds up only the process that issued it, not the mount. The +//! workers touch no table; the fid they use stays bound until FUSE_RELEASE, +//! which the kernel sends only once no read or write on that file is still +//! in progress. +//! +//! Waiting on the server does not make the loop deaf: while one of its own +//! 9P replies is outstanding the session polls the FUSE descriptor too +//! (`nine.Interrupt`). A FUSE_INTERRUPT for the request being served becomes +//! a Tflush, and if the server honours it the request fails with EINTR; one +//! for a worker's request is passed to that worker, whose session does the +//! same; READ and WRITE are handed to workers on the spot; anything else the +//! kernel sends meanwhile is copied into a queue and served next, so the +//! FUSE fd is always being read and no INTERRUPT waits behind a parked +//! request. const std = @import("std"); const cloud9 = @import("cloud9"); const post = cloud9.post; @@ -88,10 +101,12 @@ pub fn serve(gpa: std.mem.Allocator, fuse_fd: i32, session: *nine.Session, root_ // (often zero) size and truncate reads: 9P sizes are authoritative and change // under us. direct_io ignores the cached size, so the cache is safe only there. if (!effective.direct_io) effective.attr_timeout_ns = 0; + var caller = try nine.Caller.init(session); + defer caller.deinit(); var b: Bridge = .{ .gpa = gpa, .fuse_fd = fuse_fd, - .nine = session, + .nine = &caller, .opts = effective, }; defer b.deinit(); @@ -111,8 +126,7 @@ pub fn serve(gpa: std.mem.Allocator, fuse_fd: i32, session: *nine.Session, root_ defer session.stop_fd = -1; // And watch the FUSE fd meanwhile: INTERRUPTs become Tflush, other // requests (INIT arrives during the root stat) wait in the stash. - session.interrupt = b.interruptSource(); - defer session.interrupt = null; + caller.interrupt = b.interruptSource(); // Node 1 is the root; its qid comes from a stat so lookups resolving back to // it (e.g. via a walk) dedupe onto node 1. @@ -171,7 +185,8 @@ pub fn serve(gpa: std.mem.Allocator, fuse_fd: i32, session: *nine.Session, root_ const Bridge = struct { gpa: std.mem.Allocator, fuse_fd: i32, - nine: *nine.Session, + /// This thread's side of the 9P session (a worker's bridge has its own). + nine: *nine.Caller, opts: Options, /// Node id of this bridge's root inode. `fuse.root_id` (1) in /// single-connection mode; in mntgen mode the mount's `root_node`, @@ -198,8 +213,15 @@ const Bridge = struct { stash: std.ArrayList([]align(8) u8) = .empty, /// `unique` of the FUSE request being served, if any (0 = none): the only /// one an INTERRUPT may cancel. Atomic: in mntgen mode the dispatcher - /// thread scans it to route FUSE_INTERRUPTs to the right server. + /// thread scans it to route FUSE_INTERRUPTs to the right server, and + /// the loop scans its workers' to route theirs. cur_unique: std.atomic.Value(u64) = .init(0), + /// Where an INTERRUPT is delivered when this bridge's thread does not read + /// the FUSE fd itself (a worker, a mntgen mount): whoever does writes the + /// target's unique into `int_pipe[1]`, and `pipeSource` polls `[0]`. + int_pipe: [2]i32 = .{ -1, -1 }, + /// The workers READ and WRITE run on (the table-owning bridge's only). + pool: Pool = .{}, /// An INTERRUPT for `cur_unique` was consumed: chunked loops stop early /// even when the flushed reply won the race. Reset per request. interrupted: bool = false, @@ -219,6 +241,10 @@ const Bridge = struct { last_err: linux.E = .IO, fn deinit(b: *Bridge) void { + b.stopPool(); + if (b.int_pipe[0] >= 0) _ = linux.close(b.int_pipe[0]); + if (b.int_pipe[1] >= 0) _ = linux.close(b.int_pipe[1]); + b.int_pipe = .{ -1, -1 }; var it = b.handles.valueIterator(); while (it.next()) |h| if (h.dir) |*d| d.deinit(b.gpa); b.handles.deinit(b.gpa); @@ -274,9 +300,30 @@ const Bridge = struct { b.interrupted = true; return true; } + if (b.interruptWorker(in.unique)) return false; + // Waiting its turn in the stash: it has not started, and the + // kernel sends an INTERRUPT once, so it ends here. + for (b.stash.items, 0..) |raw, i| { + if (uniqueOf(raw) != in.unique or !wantsReply(raw)) continue; + _ = b.stash.orderedRemove(i); + b.gpa.free(raw); + b.trace("<- interrupt for unique={d} (stashed): answered EINTR", .{in.unique}); + b.replyError(in.unique, .INTR) catch {}; + return false; + } b.trace("<- interrupt for unique={d} (not in flight; ignored)", .{in.unique}); return false; } + // A transfer goes to a worker right away rather than wait behind + // the loop's own 9P reply. + if (h.op() == .read or h.op() == .write) { + if (b.ioFid(req)) |fid| { + if (b.offload(req, fid)) { + b.trace("<- {s} unique={d} handed to a worker while a 9P reply is outstanding", .{ opName(h.op()), h.unique }); + return false; + } + } else |_| {} + } b.trace("<- {s} unique={d} nodeid={d} stashed while a 9P reply is outstanding", .{ opName(h.op()), h.unique, h.nodeid }); const raw = b.spare_buf[0..h.len]; const copy = b.gpa.alignedAlloc(u8, .@"8", raw.len) catch { @@ -307,7 +354,7 @@ const Bridge = struct { fn dispatch(b: *Bridge, req: fuse.Request) !bool { const h = req.header; const op = h.op(); - b.trace("<- {s} unique={d} nodeid={d} len={d} (fids={d} inodes={d} handles={d})", .{ opName(op), h.unique, h.nodeid, h.len, b.nine.fidsInUse(), b.inodes.count(), b.handles.count() }); + b.trace("<- {s} unique={d} nodeid={d} len={d} (fids={d} inodes={d} handles={d})", .{ opName(op), h.unique, h.nodeid, h.len, b.nine.session.fidsInUse(), b.inodes.count(), b.handles.count() }); const wants_reply = switch (op) { .forget, .batch_forget, .interrupt => false, else => true, @@ -322,23 +369,8 @@ const Bridge = struct { b.handle(req) catch |e| { if (e == error.Stopped and b.fuse_gone) return false; if (e == error.Stopped and b.fuse_fail != null) return b.fuse_fail.?; - const code: linux.E = switch (e) { - error.Nine => b.last_err, - error.BadRequest => .INVAL, - error.NoEntry => .NOENT, - error.BadHandle => .BADF, - error.Exdev => .XDEV, - error.Perm => .PERM, - error.NotSup => .NOSYS, - error.OutOfMemory => .NOMEM, - error.TooLarge => .NAMETOOLONG, - error.BadDir => .IO, - error.Closed, error.Io => if (b.stale_on_death) .STALE else .IO, - error.Protocol, error.Stopped => .IO, - error.Interrupted => .INTR, - error.FuseIo => return error.FuseIo, - }; - if (wants_reply) try b.replyError(h.unique, code); + if (e == error.FuseIo) return error.FuseIo; + if (wants_reply) try b.replyError(h.unique, b.errnoOf(e)); switch (e) { error.Closed, error.Protocol, error.Io => return error.Closed, error.Stopped => return false, // the child is gone; the mount is being torn down @@ -346,12 +378,31 @@ const Bridge = struct { } }; // An EINTR that came from the server going mute (a Tflush unanswered - // past its grace) is the last thing this session says: the - // connection is gone with it. - if (b.nine.wedged) return error.Closed; + // past its grace, here or on a worker) is the last thing this session + // says: the connection is gone with it. + if (b.nine.session.over()) return error.Closed; return true; } + /// The errno a failed request is answered with. + fn errnoOf(b: *const Bridge, e: HandlerError) linux.E { + return switch (e) { + error.Nine => b.last_err, + error.BadRequest => .INVAL, + error.NoEntry => .NOENT, + error.BadHandle => .BADF, + error.Exdev => .XDEV, + error.Perm => .PERM, + error.NotSup => .NOSYS, + error.OutOfMemory => .NOMEM, + error.TooLarge => .NAMETOOLONG, + error.BadDir => .IO, + error.Closed, error.Io => if (b.stale_on_death) .STALE else .IO, + error.Protocol, error.Stopped, error.FuseIo => .IO, + error.Interrupted => .INTR, + }; + } + fn handle(b: *Bridge, req: fuse.Request) HandlerError!void { const u = req.header.unique; switch (req.header.op()) { @@ -388,21 +439,11 @@ const Bridge = struct { .setattr => try b.setattr(req), .open => try b.openFile(req, false), .opendir => try b.openFile(req, true), - .read => { - const in = try body(fuse.ReadIn, req); - const h = b.handles.get(in.fh) orelse return error.BadHandle; - const want: usize = @min(in.size, max_write); - const n = try b.read(h.fid, in.offset, b.data_buf[0..want]); - try b.reply(u, &.{b.data_buf[0..n]}); - }, - .write => { - const in = try body(fuse.WriteIn, req); - const h = b.handles.get(in.fh) orelse return error.BadHandle; - const rest = req.body[@sizeOf(fuse.WriteIn)..]; - if (rest.len < in.size) return error.BadRequest; - const n = try b.write(h.fid, in.offset, rest[0..in.size]); - const out = fuse.WriteOut{ .size = @intCast(n) }; - try b.reply(u, &.{std.mem.asBytes(&out)}); + .read, .write => { + const fid = try b.ioFid(req); + if (b.offload(req, fid)) return; + // No worker to be had: the loop does it itself, as it used to. + try b.transfer(req, fid); }, .readdir => try b.readdir(req), .release, .releasedir => { @@ -453,7 +494,10 @@ const Bridge = struct { const out = fuse.StatfsOut{ .st = .{ .bsize = 4096, .namelen = 255, .frsize = 4096 } }; try b.reply(u, &.{std.mem.asBytes(&out)}); }, - .interrupt => {}, + .interrupt => { + const in = try body(fuse.InterruptIn, req); + if (!b.interruptWorker(in.unique)) b.trace(" interrupt for unique={d} (not in flight; ignored)", .{in.unique}); + }, .destroy => unreachable, // handled in dispatch .access => return error.NotSup, else => return error.NotSup, @@ -716,6 +760,226 @@ const Bridge = struct { return list; } + // -- READ and WRITE on workers ------------------------------------------------------ + + /// The fid behind a READ's or WRITE's file handle (loop thread: the + /// handle table is the loop's). + fn ioFid(b: *Bridge, req: fuse.Request) HandlerError!u32 { + const fh = switch (req.header.op()) { + .read => (try body(fuse.ReadIn, req)).fh, + .write => (try body(fuse.WriteIn, req)).fh, + else => unreachable, + }; + const h = b.handles.get(fh) orelse return error.BadHandle; + return h.fid; + } + + /// The 9P half of a READ or WRITE on `fid`, and the reply. Touches no + /// table, so it runs on a worker as well as on the loop. + fn transfer(b: *Bridge, req: fuse.Request, fid: u32) HandlerError!void { + const u = req.header.unique; + switch (req.header.op()) { + .read => { + const in = try body(fuse.ReadIn, req); + const want: usize = @min(in.size, max_write); + const n = try b.read(fid, in.offset, b.data_buf[0..want]); + try b.reply(u, &.{b.data_buf[0..n]}); + }, + .write => { + const in = try body(fuse.WriteIn, req); + const rest = req.body[@sizeOf(fuse.WriteIn)..]; + if (rest.len < in.size) return error.BadRequest; + const n = try b.write(fid, in.offset, rest[0..in.size]); + const out = fuse.WriteOut{ .size = @intCast(n) }; + try b.reply(u, &.{std.mem.asBytes(&out)}); + }, + else => unreachable, + } + } + + /// Queues a READ or WRITE (copied: the request buffer is reused) for a + /// worker, starting one when every worker is taken and the pool is not + /// full. False when there is no worker to be had (no memory, no thread, + /// the pool stopping): the caller serves the request itself. + fn offload(b: *Bridge, req: fuse.Request, fid: u32) bool { + const raw = b.gpa.alignedAlloc(u8, .@"8", req.header.len) catch return false; + @memcpy(raw[0..@sizeOf(fuse.InHeader)], std.mem.asBytes(&req.header)); + @memcpy(raw[@sizeOf(fuse.InHeader)..], req.body); + const io = nine.lockIo(); + const p = &b.pool; + p.mutex.lockUncancelable(io); + defer p.mutex.unlock(io); + if (!p.stopping) { + // Every queued request needs an idle worker of its own, or it + // would wait behind a read that may park. + if (p.jobs.items.len + 1 > p.idle) b.addWorker(); + if (p.count != 0) { + if (p.jobs.append(b.gpa, .{ .raw = raw, .fid = fid })) |_| { + p.cond.signal(io); + return true; + } else |_| {} + } + } + b.gpa.free(raw); + return false; + } + + /// Starts a worker if the pool has room (pool mutex held). Best effort: + /// without one the queued request waits for a busy worker. + fn addWorker(b: *Bridge) void { + const p = &b.pool; + const slot = for (p.workers, 0..) |w, i| { + if (w == null) break i; + } else return; + const w = b.gpa.create(Worker) catch return; + w.* = .{ + .owner = b, + .b = .{ .gpa = b.gpa, .fuse_fd = b.fuse_fd, .nine = &w.caller, .opts = b.opts, .stale_on_death = b.stale_on_death }, + .caller = nine.Caller.init(b.nine.session) catch { + b.gpa.destroy(w); + return; + }, + .thread = undefined, + }; + // From here on `w.b.deinit` frees what the worker holds. + const ready = blk: { + w.b.data_buf = b.gpa.alloc(u8, max_write) catch break :blk false; + if (linux.errno(linux.pipe2(&w.b.int_pipe, .{ .CLOEXEC = true, .NONBLOCK = true })) != .SUCCESS) { + w.b.int_pipe = .{ -1, -1 }; + break :blk false; + } + w.caller.interrupt = w.b.pipeSource(); + w.thread = std.Thread.spawn(.{}, poolWorker, .{w}) catch break :blk false; + break :blk true; + }; + if (!ready) { + w.caller.deinit(); + w.b.deinit(); + b.gpa.destroy(w); + return; + } + p.workers[slot] = w; + p.count += 1; + b.trace(" worker {d} started ({d} in the pool)", .{ slot, p.count }); + } + + /// An INTERRUPT for a request a worker holds. Still queued, it has not + /// started: it is taken out and answered EINTR here (the kernel sends an + /// INTERRUPT once). In flight, its unique goes down the worker's pipe and + /// the worker's session flushes the 9P request. False when no worker has + /// it. Safe from any thread (mntgen's dispatcher routes here too). + fn interruptWorker(b: *Bridge, target: u64) bool { + const io = nine.lockIo(); + const p = &b.pool; + p.mutex.lockUncancelable(io); + for (p.jobs.items, 0..) |job, i| { + if (uniqueOf(job.raw) != target) continue; + _ = p.jobs.orderedRemove(i); + p.mutex.unlock(io); + b.trace(" interrupt for unique={d}: still queued for a worker; answered EINTR", .{target}); + b.replyError(target, .INTR) catch {}; + b.gpa.free(job.raw); + return true; + } + for (p.workers) |slot| { + const w = slot orelse continue; + if (w.b.cur_unique.load(.seq_cst) != target) continue; + // Under the pool mutex, which is where a worker takes a request + // on (sets cur_unique) and where its pipe is closed. A dropped + // packet (full pipe) only means one interrupt missed its window. + var packet: [8]u8 = undefined; + std.mem.writeInt(u64, &packet, target, .little); + _ = linux.write(w.b.int_pipe[1], &packet, packet.len); + p.mutex.unlock(io); + b.trace(" interrupt for unique={d}: forwarded to a worker", .{target}); + return true; + } + p.mutex.unlock(io); + return false; + } + + /// Stops and joins the workers. Requests still queued are answered as a + /// lost session's are; a worker parked in a 9P wait is released by + /// hanging up the socket (the program's exit releases it through + /// `stop_fd` anyway, but an unmount or DESTROY can come with the program + /// alive, and so can a mntgen mount's death). The session must still be + /// open. + fn stopPool(b: *Bridge) void { + const io = nine.lockIo(); + const p = &b.pool; + p.mutex.lockUncancelable(io); + p.stopping = true; + for (p.jobs.items) |job| { + b.replyError(uniqueOf(job.raw), if (b.stale_on_death) .STALE else .IO) catch {}; + b.gpa.free(job.raw); + } + p.jobs.clearAndFree(b.gpa); + var busy = false; + for (p.workers) |slot| { + if (slot) |w| busy = busy or w.b.cur_unique.load(.seq_cst) != 0; + } + p.mutex.unlock(io); + if (p.count == 0) return; + p.cond.broadcast(io); + if (busy) _ = linux.shutdown(b.nine.session.fd, linux.SHUT.RDWR); + for (&p.workers) |*slot| { + const w = slot.* orelse continue; + w.thread.join(); + p.mutex.lockUncancelable(io); + slot.* = null; + p.count -= 1; + p.mutex.unlock(io); + w.caller.deinit(); + w.b.deinit(); + b.gpa.destroy(w); + } + } + + // -- an interrupt pipe as the interrupt source ------------------------------------------ + // + // For a bridge whose thread does not read /dev/fuse (a worker; a mntgen + // mount, whose dispatcher owns the fd): whoever reads the INTERRUPT + // writes the target's unique into `int_pipe[1]`. + + fn pipeSource(b: *Bridge) nine.Interrupt { + return .{ .ctx = b, .watch = pipeWatch, .onReadable = pipeReadable, .armed = interruptArmed }; + } + + fn pipeWatch(ctx: *anyopaque) i32 { + const b: *Bridge = @ptrCast(@alignCast(ctx)); + return b.int_pipe[0]; + } + + fn pipeReadable(ctx: *anyopaque) nine.Session.Error!bool { + const b: *Bridge = @ptrCast(@alignCast(ctx)); + return b.pipeHit(); + } + + /// Drains the interrupt pipe; true when one of the packets named the + /// request in flight, which is then marked interrupted. + fn pipeHit(b: *Bridge) nine.Session.Error!bool { + var packet: [8]u8 = undefined; + var hit = false; + while (true) { + const rc = linux.read(b.int_pipe[0], &packet, packet.len); + switch (linux.errno(rc)) { + .SUCCESS => { + // Pipe writes of 8 bytes are atomic; a short read cannot happen. + const target = std.mem.readInt(u64, &packet, .little); + const cur = b.cur_unique.load(.seq_cst); + if (cur != 0 and target == cur) hit = true; + }, + .INTR => continue, + .AGAIN => break, // drained + else => return error.Io, + } + } + if (!hit) return false; + b.trace("<- interrupt for unique={d} (in flight): cancelling, sending Tflush", .{b.cur_unique.load(.seq_cst)}); + b.interrupted = true; + return true; + } + // -- 9P wrappers (tracing) ----------------------------------------------------------- fn stat(b: *Bridge, fid: u32) nine.Session.Error!cloud9.Stat { @@ -740,9 +1004,9 @@ const Bridge = struct { /// interruption the server may or may not have bound it (the Rflush /// tells us only that no reply is coming), so it is clunked to be sure. fn walkTo(b: *Bridge, fid: u32, names: []const []const u8) nine.Session.Error!u32 { - const newfid = b.nine.allocFid(); + const newfid = b.nine.session.allocFid(); _ = b.nine.walk(fid, newfid, names) catch |e| { - if (e == error.Interrupted) b.clunkQuiet(newfid) else b.nine.freeFid(newfid); + if (e == error.Interrupted) b.clunkQuiet(newfid) else b.nine.session.freeFid(newfid); return b.nineErr("walk", fid, e); }; return newfid; @@ -850,15 +1114,94 @@ const Bridge = struct { }; // =========================================================================== +// Workers: READ and WRITE off the loop +// =========================================================================== +// +// Plan 9's mount driver needs no pool: every process blocked in a read is +// its own waiter on the mount (devmnt.c `mountio`). A FUSE server receives +// the kernel's requests on one descriptor instead, so the waiting has to be +// given threads here. Only READ and WRITE get them: those are the requests +// a synthetic server parks (a blocking `log`, `event` or `run` file) and +// they touch no table, needing just the fid their handle resolved to on +// the loop. Everything else stays on the loop, so the inode, handle and +// qid tables, and the reconciliation of lookups with forgets, keep a +// single owner. + +/// Workers per bridge. With the loop they make as many callers as the 9P +/// client has request tags (`nine.max_tags`), so parked transfers cannot +/// starve the loop's lookups of a tag. A transfer that finds every worker +/// parked waits in the queue for one (interruptible there too). +pub const max_workers: usize = nine.max_tags - 1; + +const Pool = struct { + /// Guards everything here, and each worker's move of a request from + /// the queue to its `cur_unique` and back (see `interruptWorker`). + mutex: std.Io.Mutex = .init, + cond: std.Io.Condition = .init, + /// Transfers waiting for a worker, in arrival order. + jobs: std.ArrayList(Job) = .empty, + workers: [max_workers]?*Worker = @splat(null), + count: usize = 0, + /// Workers waiting on `cond` for a job. + idle: usize = 0, + stopping: bool = false, +}; + +/// A READ or WRITE, copied out of the request buffer, and the fid its file +/// handle named when the loop looked it up. +const Job = struct { raw: []align(8) u8, fid: u32 }; + +/// A worker thread and a bridge of its own that owns no table: its caller +/// on the session, its data buffer, the request in flight, the errno of the +/// last Rerror, its interrupt pipe. +const Worker = struct { + owner: *Bridge, + b: Bridge, + caller: nine.Caller, + thread: std.Thread, +}; + +fn poolWorker(w: *Worker) void { + const io = nine.lockIo(); + const p = &w.owner.pool; + p.mutex.lockUncancelable(io); + while (true) { + if (p.jobs.items.len != 0) { + const job = p.jobs.orderedRemove(0); + // In flight from the moment it leaves the queue, under the same + // lock: an INTERRUPT finds it in one place or the other. + w.b.cur_unique.store(uniqueOf(job.raw), .seq_cst); + p.mutex.unlock(io); + const req = requestOf(job.raw); + w.b.interrupted = false; + w.b.trace(" worker: {s} unique={d} fid={d}", .{ opName(req.header.op()), req.header.unique, job.fid }); + w.b.transfer(req, job.fid) catch |e| { + if (e != error.FuseIo) w.b.replyError(req.header.unique, w.b.errnoOf(e)) catch {}; + }; + w.owner.gpa.free(job.raw); + p.mutex.lockUncancelable(io); + w.b.cur_unique.store(0, .seq_cst); + continue; + } + if (p.stopping) break; + p.idle += 1; + p.cond.waitUncancelable(io, &p.mutex); + p.idle -= 1; + } + p.mutex.unlock(io); +} + +// =========================================================================== // mntgen: many servers behind one FUSE mount // =========================================================================== // // `9ns --mntgen` serves one FUSE mount whose synthetic root lists the posted // 9P services in `$XDG_RUNTIME_DIR/9p` (cloud9.post's registry; no connection // is made to list). A walk into a name makes a mount for it and hands the -// walk to the mount's worker thread, which dials the server (the only -// thread that ever talks to it) and then runs the ordinary bridge -// translation above for everything under the name. The dispatcher thread +// walk to the mount's worker thread, which dials the server and then runs +// the ordinary bridge translation above for everything under the name +// (READ and WRITE on that mount's own transfer workers, on the same +// session). The dispatcher thread // reads /dev/fuse, routes, and serves the registry's own directories; it // never waits on a server, so one server that never answers holds up only // the walks into its own name — and those, being ordinary requests on a @@ -991,12 +1334,17 @@ const Mount = struct { /// the worker under `mutex`, so the dispatcher can shut the socket down /// at teardown; `root_attr` and the bridge's root inode exist with it. session: ?nine.Session = null, + /// The worker's caller on `session` (the bridge's `nine`), made and + /// dropped with it. + caller: ?nine.Caller = null, /// Attr of the server's 9P root from the dial-time stat: what a /// LOOKUP of the name answers. root_attr: fuse.Attr = undefined, b: *Bridge, - /// Guards `queue`, `stopping`, `session`'s existence, and the moment a - /// request leaves the queue for `b.cur_unique` (see `routeInterrupt`). + /// Guards `queue`, `stopping`, `session`'s existence, the moment a + /// request leaves the queue for `b.cur_unique` (see `routeInterrupt`), + /// and `b.int_pipe`, where the dispatcher drops a FUSE_INTERRUPT's + /// target unique for the worker's session to poll. mutex: std.Io.Mutex = .init, cond: std.Io.Condition = .init, queue: std.ArrayList([]align(8) u8) = .empty, @@ -1005,9 +1353,6 @@ const Mount = struct { dead: std.atomic.Value(bool) = .init(false), /// Draining and exiting (child gone, FUSE device gone or DESTROY). stopping: bool = false, - /// The dispatcher drops a FUSE_INTERRUPT's target unique in here; the - /// worker's session polls it while a 9P reply is outstanding. - int_pipe: [2]i32, thread: std.Thread, fn sockPath(m: *const Mount) [:0]const u8 { @@ -1117,9 +1462,8 @@ const Mntgen = struct { m.queue.deinit(mg.gpa); // A mount that died released all of this itself (`retire`). if (!m.dead.load(.seq_cst)) { - _ = linux.close(m.int_pipe[0]); - _ = linux.close(m.int_pipe[1]); - m.b.deinit(); + m.b.deinit(); // its workers first: they use the session + if (m.caller) |*c| c.deinit(); if (m.session) |*sess| sess.deinit(); } mg.gpa.free(m.name); @@ -1356,7 +1700,7 @@ const Mntgen = struct { return; } const in_flight = m.b.cur_unique.load(.seq_cst) == target; - if (in_flight and m.int_pipe[1] >= 0) { + if (in_flight and m.b.int_pipe[1] >= 0) { // Under the mutex, because the worker closes the pipe there // when the mount dies. The pipe is small and nonblocking; a // dropped packet only means one interrupt missed its window @@ -1364,13 +1708,15 @@ const Mntgen = struct { // exit ends the session through stop_fd regardless). var packet: [8]u8 = undefined; std.mem.writeInt(u64, &packet, target, .little); - _ = linux.write(m.int_pipe[1], &packet, packet.len); + _ = linux.write(m.b.int_pipe[1], &packet, packet.len); } m.mutex.unlock(mg.io); if (in_flight) { mg.trace(" interrupt for unique={d}: forwarded to '{s}'", .{ target, m.name }); return; } + // A READ or WRITE on one of the mount's transfer workers. + if (m.b.interruptWorker(target)) return; } mg.trace(" interrupt for unique={d} (not in flight; ignored)", .{target}); } @@ -1750,14 +2096,10 @@ const Mntgen = struct { errdefer b.deinit(); b.data_buf = try mg.gpa.alloc(u8, max_write); - var pipes: [2]i32 = undefined; - if (linux.errno(linux.pipe2(&pipes, .{ .CLOEXEC = true, .NONBLOCK = true })) != .SUCCESS) { + if (linux.errno(linux.pipe2(&b.int_pipe, .{ .CLOEXEC = true, .NONBLOCK = true })) != .SUCCESS) { + b.int_pipe = .{ -1, -1 }; return error.SystemResources; } - errdefer { - _ = linux.close(pipes[0]); - _ = linux.close(pipes[1]); - } const m = try mg.gpa.create(Mount); errdefer mg.gpa.destroy(m); @@ -1773,7 +2115,6 @@ const Mntgen = struct { .entry_name = undefined, // a slice of `name`, set below .stop_fd = mg.stop_fd, .b = b, - .int_pipe = pipes, .thread = undefined, }; errdefer mg.gpa.free(m.name); @@ -1921,7 +2262,7 @@ const DialError = error{ NotPosted, Stale, Busy, SystemResources, Io } || nine.S /// no session yet to flush a request out of (`abort_on_cancel`). fn dial(m: *Mount) DialError!void { const fd = try connectSocket(m); - var fresh = nine.Session.dial(m.gpa, .{ .fd = fd }, m.mo.msize, m.stop_fd) catch |e| { + const fresh = nine.Session.dial(m.gpa, .{ .fd = fd }, m.mo.msize, m.stop_fd) catch |e| { _ = linux.close(fd); // With an `.fd` address the only thing left to fail is the buffers. return switch (e) { @@ -1929,18 +2270,19 @@ fn dial(m: *Mount) DialError!void { else => error.Io, }; }; - fresh.interrupt = mountInterrupt(m); - fresh.abort_on_cancel = true; m.mutex.lockUncancelable(m.io); m.session = fresh; m.mutex.unlock(m.io); errdefer dropSession(m); - const sess = &m.session.?; - m.b.nine = sess; - try sess.version(); - _ = try sess.attach(0, m.mo.uname, m.mo.aname); - const st = try sess.stat(0); - sess.abort_on_cancel = false; + m.caller = try nine.Caller.init(&m.session.?); + const c = &m.caller.?; + c.interrupt = m.b.pipeSource(); + c.abort_on_cancel = true; + m.b.nine = c; + try c.version(); + _ = try c.attach(0, m.mo.uname, m.mo.aname); + const st = try c.stat(0); + c.abort_on_cancel = false; // The kernel-visible root of this server's subtree: the mount's root // node id, fid 0, its qid from the stat. From here on the bridge is an @@ -1957,6 +2299,8 @@ fn dial(m: *Mount) DialError!void { /// Closes the session and its socket; the mount is back to having none. fn dropSession(m: *Mount) void { m.mutex.lockUncancelable(m.io); + if (m.caller) |*c| c.deinit(); + m.caller = null; if (m.session) |*sess| sess.deinit(); m.session = null; m.mutex.unlock(m.io); @@ -1973,11 +2317,12 @@ fn dropSession(m: *Mount) void { fn retire(m: *Mount) void { m.dead.store(true, .seq_cst); retireEntry(m); + m.b.stopPool(); // before the session goes: the workers use it dropSession(m); m.mutex.lockUncancelable(m.io); - _ = linux.close(m.int_pipe[0]); - _ = linux.close(m.int_pipe[1]); - m.int_pipe = .{ -1, -1 }; + _ = linux.close(m.b.int_pipe[0]); + _ = linux.close(m.b.int_pipe[1]); + m.b.int_pipe = .{ -1, -1 }; m.mutex.unlock(m.io); m.b.deinit(); } @@ -2010,14 +2355,14 @@ fn connectSocket(m: *Mount) DialError!i32 { if (waited >= connect_grace_ms) return error.Busy; var pfds = [_]linux.pollfd{ .{ .fd = m.stop_fd, .events = linux.POLL.IN, .revents = 0 }, - .{ .fd = m.int_pipe[0], .events = linux.POLL.IN, .revents = 0 }, + .{ .fd = m.b.int_pipe[0], .events = linux.POLL.IN, .revents = 0 }, }; switch (linux.errno(linux.poll(&pfds, pfds.len, connect_retry_ms))) { .SUCCESS, .INTR, .AGAIN => {}, else => return error.Io, } if (pfds[0].revents != 0) return error.Stopped; - if (pfds[1].revents != 0 and try interruptHit(m)) return error.Interrupted; + if (pfds[1].revents != 0 and try m.b.pipeHit()) return error.Interrupted; waited += connect_retry_ms; } _ = linux.fcntl(fd, linux.F.SETFL, 0); @@ -2059,60 +2404,6 @@ fn retireEntry(m: *Mount) void { }; } -// -- a mount's interrupt source ------------------------------------------------- -// -// The worker's session polls `int_pipe[0]` while a 9P reply is outstanding -// (and `connectSocket` while it waits on a full backlog); the dispatcher -// writes the interrupted request's unique into it. Unlike the -// single-connection source, no FUSE reading happens here — the dispatcher -// owns /dev/fuse. - -fn mountInterrupt(m: *Mount) nine.Interrupt { - return .{ .ctx = m, .watch = mountWatch, .onReadable = mountOnReadable, .armed = mountArmed }; -} - -fn mountWatch(ctx: *anyopaque) i32 { - const m: *Mount = @ptrCast(@alignCast(ctx)); - return m.int_pipe[0]; -} - -fn mountOnReadable(ctx: *anyopaque) nine.Session.Error!bool { - const m: *Mount = @ptrCast(@alignCast(ctx)); - return interruptHit(m); -} - -/// Drains the interrupt pipe; true when one of the packets named the -/// request in flight, which is then marked interrupted. -fn interruptHit(m: *Mount) nine.Session.Error!bool { - var packet: [8]u8 = undefined; - var hit = false; - while (true) { - const rc = linux.read(m.int_pipe[0], &packet, packet.len); - switch (linux.errno(rc)) { - .SUCCESS => { - // Pipe writes of 8 bytes are atomic; a short read cannot happen. - const target = std.mem.readInt(u64, &packet, .little); - const cur = m.b.cur_unique.load(.seq_cst); - if (cur != 0 and target == cur) hit = true; - }, - .INTR => continue, - .AGAIN => break, // drained - else => return error.Io, - } - } - if (hit) { - if (m.debug) std.debug.print("9ns: [{s}] interrupt for unique={d} (in flight): cancelling\n", .{ m.name, m.b.cur_unique.load(.seq_cst) }); - m.b.interrupted = true; - return true; - } - return false; -} - -fn mountArmed(ctx: *anyopaque) bool { - const m: *Mount = @ptrCast(@alignCast(ctx)); - return m.b.interrupted; -} - // -- pure helpers (unit-tested) ------------------------------------------------------------ /// Attr from a 9P Stat: DMDIR → S_IFDIR else S_IFREG, low 9 permission bits kept. @@ -2391,7 +2682,7 @@ test "interrupt source: INTERRUPT in flight → Tflush, other INTERRUPTs ignored var fs: nine.FakeServer = .{ .fd = -1, .msize = 8192, .file_len = 50, .hang_offset = 0, .on_flush = .rflush, .on_hang_inject = &pi }; var pair = try fs.start(); defer pair.close(); - const s = &pair.session; + const s = pair.caller; var b: Bridge = .{ .gpa = testing.allocator, .fuse_fd = pi.read_end, .nine = s, .opts = .{ .uid = 0, .gid = 0 } }; defer b.deinit(); b.spare_buf = try testing.allocator.alignedAlloc(u8, .@"8", request_buf_len); @@ -2414,7 +2705,7 @@ test "interrupt source: INTERRUPT in flight → Tflush, other INTERRUPTs ignored b.interrupted = false; b.cur_unique.store(8, .seq_cst); try testing.expectEqual(@as(usize, 40), try b.read(1, 10, buf[0..40])); - try testing.expectEqual(@as(usize, 0), s.client.pending()); + try testing.expectEqual(@as(usize, 0), s.session.client.pending()); // A FORGET arriving during a wait is copied into the stash, and the fd // stays watched: an INTERRUPT can still arrive behind it. @@ -2508,3 +2799,272 @@ test "mntgen: ino_xor keeps two servers' identical qid.paths distinct" { var b0: Bridge = .{ .gpa = testing.allocator, .fuse_fd = -1, .nine = undefined, .opts = .{ .uid = 0, .gid = 0 } }; try testing.expectEqual(inoFromPath(qid.path), b0.inoOf(1, qid)); } + +/// The kernel's end of a fake /dev/fuse: a SOCK_SEQPACKET pair keeps one +/// request per read and one reply per write, like the device. +const FakeKernel = struct { + fd: i32, + buf: [8192]u8 align(8) = undefined, + + const Reply = struct { unique: u64, err: i32, payload: []const u8 }; + + fn send(k: *FakeKernel, op: fuse.Opcode, unique: u64, nodeid: u64, parts: []const []const u8) !void { + var len: usize = @sizeOf(fuse.InHeader); + for (parts) |p| len += p.len; + const header = fuse.InHeader{ .len = @intCast(len), .opcode = @intFromEnum(op), .unique = unique, .nodeid = nodeid, .uid = 0, .gid = 0, .pid = 0, .total_extlen = 0, .padding = 0 }; + var wire: [512]u8 = undefined; + @memcpy(wire[0..@sizeOf(fuse.InHeader)], std.mem.asBytes(&header)); + var at: usize = @sizeOf(fuse.InHeader); + for (parts) |p| { + @memcpy(wire[at..][0..p.len], p); + at += p.len; + } + if (linux.write(k.fd, &wire, len) != len) return error.Io; + } + + /// The next reply, or error.Timeout after `ms` milliseconds without one. + fn recv(k: *FakeKernel, ms: i32) !Reply { + var pfd = [_]linux.pollfd{.{ .fd = k.fd, .events = linux.POLL.IN, .revents = 0 }}; + if (linux.poll(&pfd, 1, ms) != 1) return error.Timeout; + const rc = linux.read(k.fd, &k.buf, k.buf.len); + if (linux.errno(rc) != .SUCCESS or rc < @sizeOf(fuse.OutHeader)) return error.Io; + const h = std.mem.bytesToValue(fuse.OutHeader, k.buf[0..@sizeOf(fuse.OutHeader)]); + return .{ .unique = h.unique, .err = h.@"error", .payload = k.buf[@sizeOf(fuse.OutHeader)..rc] }; + } +}; + +const ServeThread = struct { + session: *nine.Session, + fuse_fd: i32, + stop_fd: i32, + result: anyerror!void = {}, + + fn run(t: *ServeThread) void { + t.result = serve(testing.allocator, t.fuse_fd, t.session, 0, t.stop_fd, .{ .uid = 0, .gid = 0 }); + } +}; + +/// `serve` on its own thread, fed through a fake /dev/fuse, against a +/// FakeServer; `lookupOpen` does the LOOKUP + OPEN a process's open(2) does. +const ServeRig = struct { + pair: nine.FakeServer.Pair, + dev: [2]i32, + stop: [2]i32, + st: ServeThread, + thread: std.Thread, + k: FakeKernel, + + fn start(r: *ServeRig, fs: *nine.FakeServer) !void { + r.pair = try fs.start(); + errdefer r.pair.close(); + try testing.expectEqual(linux.E.SUCCESS, linux.errno(linux.socketpair(linux.AF.UNIX, linux.SOCK.SEQPACKET | linux.SOCK.CLOEXEC, 0, &r.dev))); + try testing.expectEqual(linux.E.SUCCESS, linux.errno(linux.pipe2(&r.stop, .{ .CLOEXEC = true }))); + r.st = .{ .session = r.pair.session, .fuse_fd = r.dev[0], .stop_fd = r.stop[0] }; + r.thread = try std.Thread.spawn(.{}, ServeThread.run, .{&r.st}); + r.k = .{ .fd = r.dev[1] }; + } + + /// Whatever happened, the loop must end: DESTROY, and stop_fd for a + /// loop stuck in a 9P wait (which is what head-of-line blocking looks + /// like). Then everything is closed. + fn stopAll(r: *ServeRig) void { + r.k.send(.destroy, 999, 0, &.{}) catch {}; + _ = linux.write(r.stop[1], "x", 1); + r.thread.join(); + for (r.dev ++ r.stop) |fd| _ = linux.close(fd); + r.pair.close(); + } + + fn lookupOpen(r: *ServeRig, unique: u64) !struct { node: u64, fh: u64 } { + try r.k.send(.lookup, unique, fuse.root_id, &.{"file\x00"}); + var rep = try r.k.recv(3000); + try testing.expectEqual(@as(i32, 0), rep.err); + const node = std.mem.bytesToValue(fuse.EntryOut, rep.payload[0..@sizeOf(fuse.EntryOut)]).nodeid; + try r.k.send(.open, unique + 1, node, &.{std.mem.asBytes(&fuse.OpenIn{ .flags = 0, .open_flags = 0 })}); + rep = try r.k.recv(3000); + try testing.expectEqual(@as(i32, 0), rep.err); + return .{ .node = node, .fh = std.mem.bytesToValue(fuse.OpenOut, rep.payload[0..@sizeOf(fuse.OpenOut)]).fh }; + } + + fn read(r: *ServeRig, unique: u64, node: u64, fh: u64, offset: u64, size: u32) !void { + const in = fuse.ReadIn{ .fh = fh, .offset = offset, .size = size, .read_flags = 0, .lock_owner = 0, .flags = 0, .padding = 0 }; + try r.k.send(.read, unique, node, &.{std.mem.asBytes(&in)}); + } + + fn getattr(r: *ServeRig, unique: u64, node: u64) !void { + try r.k.send(.getattr, unique, node, &.{std.mem.asBytes(&fuse.GetattrIn{ .getattr_flags = 0, .dummy = 0, .fh = 0 })}); + } + + fn waitHung(fs: *nine.FakeServer, n: u32) !void { + var waited: usize = 0; + while (fs.hung_now.load(.seq_cst) < n and waited < 3000) : (waited += 1) { + const ts: linux.timespec = .{ .sec = 0, .nsec = std.time.ns_per_ms }; + _ = linux.nanosleep(&ts, null); + } + try testing.expect(fs.hung_now.load(.seq_cst) >= n); + } +}; + +test "serve: a parked READ holds up neither other READs nor metadata; its INTERRUPT flushes it" { + var fs: nine.FakeServer = .{ .fd = -1, .msize = 8192, .file_len = 50, .hang_offset = 0, .on_flush = .rflush }; + var rig: ServeRig = undefined; + try rig.start(&fs); + defer rig.stopAll(); + const k = &rig.k; + const f = try rig.lookupOpen(1); + + // The READ at offset 0 parks on the server. + try rig.read(10, f.node, f.fh, 0, 100); + try ServeRig.waitHung(&fs, 1); + + // Meanwhile another READ on the same file and a GETATTR are answered, + // promptly and in any order; the parked READ is not. + for (0..20) |i| { + const u: u64 = 100 + 2 * i; + try rig.read(u, f.node, f.fh, 10, 40); + try rig.getattr(u + 1, f.node); + var seen: u2 = 0; + for (0..2) |_| { + const r = try k.recv(1000); + try testing.expectEqual(@as(i32, 0), r.err); + if (r.unique == u) { + try testing.expectEqual(@as(usize, 40), r.payload.len); + try testing.expectEqual(@as(u8, 10), r.payload[0]); + seen |= 1; + } else { + try testing.expectEqual(u + 1, r.unique); + const attr = std.mem.bytesToValue(fuse.AttrOut, r.payload[0..@sizeOf(fuse.AttrOut)]); + try testing.expectEqual(@as(u64, 50), attr.attr.size); + seen |= 2; + } + } + try testing.expectEqual(@as(u2, 3), seen); + } + try testing.expectEqual(@as(u32, 0), fs.flushes.load(.seq_cst)); + + // INTERRUPT for the parked READ: exactly one Tflush, for its tag, and + // the READ comes back EINTR. + const hung_tag = fs.hung_tag.load(.seq_cst); + try k.send(.interrupt, 11, 0, &.{std.mem.asBytes(&fuse.InterruptIn{ .unique = 10 })}); + var r = try k.recv(3000); + try testing.expectEqual(@as(u64, 10), r.unique); + try testing.expectEqual(-@as(i32, @intFromEnum(linux.E.INTR)), r.err); + try testing.expectEqual(@as(u32, 1), fs.flushes.load(.seq_cst)); + try testing.expectEqual(hung_tag, fs.flush_oldtag.load(.seq_cst)); + + // RELEASE clunks the fid; the mount carries on. + try k.send(.release, 12, f.node, &.{std.mem.asBytes(&fuse.ReleaseIn{ .fh = f.fh, .flags = 0, .release_flags = 0, .lock_owner = 0 })}); + r = try k.recv(3000); + try testing.expectEqual(@as(u64, 12), r.unique); + try testing.expectEqual(@as(i32, 0), r.err); + try rig.getattr(13, f.node); + r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + try testing.expectEqual(@as(usize, 0), rig.pair.session.client.pending()); +} + +test "serve: one open is one fid; FLUSH clunks nothing, and a READ parked on the handle completes on its fid" { + // pardes keeps state per open (a pty's or run's answer, a log cursor, + // an event reader): two opens of one file must be two fids, and each + // handle's reads must go to its own. + var fs: nine.FakeServer = .{ .fd = -1, .msize = 8192, .file_len = 50, .hang_offset = 0, .on_flush = .rflush, .fid_in_data = true, .release_on_stat = true }; + var rig: ServeRig = undefined; + try rig.start(&fs); + defer rig.stopAll(); + const k = &rig.k; + const a = try rig.lookupOpen(1); + const b = try rig.lookupOpen(3); + try testing.expectEqual(a.node, b.node); // one inode... + try testing.expect(a.fh != b.fh); // ...two opens + + // Each handle reads through its own fid (the server stamps it in byte 0). + try rig.read(10, a.node, a.fh, 10, 40); + var r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + const fid_a = r.payload[0]; + try rig.read(11, b.node, b.fh, 10, 40); + r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + const fid_b = r.payload[0]; + try testing.expect(fid_a != fid_b); + + // A read parks on `a`; `b` keeps answering from its own fid meanwhile, + // concurrently, as two processes would see it. + try rig.read(20, a.node, a.fh, 0, 100); + try ServeRig.waitHung(&fs, 1); + for (0..5) |i| { + try rig.read(30 + i, b.node, b.fh, 10, 40); + r = try k.recv(1000); + try testing.expectEqual(30 + i, r.unique); + try testing.expectEqual(fid_b, r.payload[0]); + } + + // close(2) of `a` in another thread sends FLUSH while the read is still + // parked: it is answered and clunks nothing. + const clunks = fs.clunks.load(.seq_cst); + try k.send(.flush, 21, a.node, &.{std.mem.asBytes(&fuse.FlushIn{ .fh = a.fh, .unused = 0, .padding = 0, .lock_owner = 0 })}); + r = try k.recv(3000); + try testing.expectEqual(@as(u64, 21), r.unique); + try testing.expectEqual(@as(i32, 0), r.err); + try testing.expectEqual(clunks, fs.clunks.load(.seq_cst)); + try testing.expectEqual(@as(u32, 1), fs.hung_now.load(.seq_cst)); + + // The server answers the parked read (release_on_stat, triggered by a + // GETATTR): it arrives on `a`'s fid, which was never clunked. + try rig.getattr(22, a.node); + var seen: u2 = 0; + for (0..2) |_| { + r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + if (r.unique == 20) { + try testing.expectEqual(@as(usize, 50), r.payload.len); + try testing.expectEqual(fid_a, r.payload[0]); + seen |= 1; + } else { + try testing.expectEqual(@as(u64, 22), r.unique); + seen |= 2; + } + } + try testing.expectEqual(@as(u2, 3), seen); + + // Only RELEASE clunks: one per open. + for ([_]u64{ a.fh, b.fh }, 0..) |fh, i| { + try k.send(.release, 40 + i, a.node, &.{std.mem.asBytes(&fuse.ReleaseIn{ .fh = fh, .flags = 0, .release_flags = 0, .lock_owner = 0 })}); + r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + } + try testing.expectEqual(clunks + 2, fs.clunks.load(.seq_cst)); +} + +test "serve: a flushed READ whose reply beats the Rflush returns that reply, not EINTR (flush(5))" { + var fs: nine.FakeServer = .{ .fd = -1, .msize = 8192, .file_len = 50, .hang_offset = 0, .on_flush = .reply_then_rflush }; + var rig: ServeRig = undefined; + try rig.start(&fs); + defer rig.stopAll(); + const k = &rig.k; + const f = try rig.lookupOpen(1); + try rig.read(10, f.node, f.fh, 0, 100); + try ServeRig.waitHung(&fs, 1); + + // The server answers the Tflush with the Rread first, then the Rflush: + // the read completed on the server, so its data is the answer. + const hung_tag = fs.hung_tag.load(.seq_cst); + try k.send(.interrupt, 11, 0, &.{std.mem.asBytes(&fuse.InterruptIn{ .unique = 10 })}); + var r = try k.recv(3000); + try testing.expectEqual(@as(u64, 10), r.unique); + try testing.expectEqual(@as(i32, 0), r.err); + try testing.expectEqual(@as(usize, 50), r.payload.len); + for (r.payload, 0..) |byte, i| try testing.expectEqual(@as(u8, @truncate(i)), byte); + try testing.expectEqual(@as(u32, 1), fs.flushes.load(.seq_cst)); + try testing.expectEqual(hung_tag, fs.flush_oldtag.load(.seq_cst)); + + // The late Rflush is dropped by whoever reads next; nothing is left + // pending and the mount carries on. + try rig.getattr(12, f.node); + r = try k.recv(3000); + try testing.expectEqual(@as(i32, 0), r.err); + try rig.read(13, f.node, f.fh, 10, 40); + r = try k.recv(3000); + try testing.expectEqual(@as(usize, 40), r.payload.len); + try testing.expectEqual(@as(usize, 0), rig.pair.session.client.pending()); +} |
