The task protocol

qvac/task/v0 — how one node hands a unit of software work to another, gets the result back, and decides whether to trust it. The protocol ships over OTA to every node, and two nodes on different versions is the normal case, not the rare one.

The problem this solves

The orchestrator today spawns its workers as child processes:

spawn(process.execPath, ['worker/run.mjs', '--ticket', ...])

That assumes one machine. The worker writes to its local workspace, the orchestrator runs CI on its local workspace, and those are the same directory — which is the only reason any of it works. Ten nodes under that design are ten machines writing to ten disks that nobody joins.

Three things have to cross the boundary: the assignment (here is a ticket), the context (here is the tree you are working against), and the result (here is what I produced). Each takes a different route, and the reasons are below.

The shape

┌─ COORDINATOR (orchestrator/coordinator.mjs) ┐
│  requirements.md → tickets                   │
│  publishes workspace/ as a read-only         │
│    Hyperdrive — in its OWN corestore ────────┼──┐
│                                               │  │
│  task:assign  ────────────────────────────────┼──┼──▶ Protomux, qvac/task/v0,
│  ◀── task:accept / task:progress              │  │      over the connection the
│  ◀── task:result (files INLINE ≤ 1 MiB,       │  │      two nodes already hold —
│       or a driveKey for the overflow case)    │  │      no new DHT announce
│                                               │  │
│  re-validates → mirrors → runs CI            │  │
└────────────────────────────────────────────────┘  │
                                                    │ same connection replicates
   ┌────────────────────────────────────────────────┘ this drive too — mounted
   ▼                                                   read-only, fetched sparsely
┌─ WORKER (worker/task-accept.mjs, via serve-tasks.mjs) ┐
│  accepts only a key on --allow                         │
│  reads context from the coordinator's                  │
│    drive — only the paths it opens                     │
│  infers over HTTP to the LOCAL gateway                  │
│    (never the engine in-process — see below)            │
│  validates paths against allowedFiles                    │
│  returns files inline; only over the ceiling               │
│    does it write to ITS OWN drive and return a driveKey     │
└────────────────────────────────────────────────────────────┘

Why the assignment goes over Protomux

The connection already exists and is already authenticated. Hyperswarm introduces the peers through the DHT and holepunches, so there is no inbound port to open and NAT is not in the way — a point worth stating because it is the usual reason a design like this ends up needing a queue. It does not.

The control channel is also built for exactly this kind of extension. It carries one JSON message type on purpose, so that a node running an older build ignores a type it does not know and keeps going. Adding a message is backwards-compatible by construction. qvac/swarm.mjs now dispatches any task:* message to a listener either side registers (addTaskListener/sendTask); it never invents a second channel.

Why the context goes over a drive — and never a second connection to get it

A worker that can only see the ticket text can write new files and nothing else. Real work means modifying code that already exists, which means reading the tree. Hyperdrive downloads sparsely, by path: the coordinator publishes a 40 MB workspace and a worker that opens four files transfers four files. This is the case where a drive earns its place.

What changed from the first sketch of this page: the drive is created in the coordinator's own corestore and replicates over the connection the coordinator and worker already hold as marketplace peers — the same thing that already carries directory and file replication (corestore.replicate(socket) in qvac/swarm.mjs). There is no per-ticket swarm.join on a fresh discovery key. Skipping that saves the 2–17s (38s tail, measured) a fresh DHT announce costs, on every ticket — the single largest latency item this protocol controls, next to the inference itself. See orchestrator/context-drive.mjs.

Why the result usually travels inline, not over a second drive

The control channel shares one multiplexed stream with hypercore replication and with the tokens of live inference — sending bulk bytes over it would compete with what the node is serving to somebody else. A separate Hyperdrive per ticket avoids that, but for a handful of KB-sized source files it is more latency (create → replicate → discover → sparse-fetch) and more code than the problem needs.

So task:result carries the files' bytes inline, up to a 1 MiB total ceiling — comfortably under the 16 MiB frame limit NoiseSecretStream enforces below Protomux, and enough for a batch of source files with room to spare. Only past that ceiling does a file go on the worker's own Files drive (still replicated over the same existing connection, same as the context drive) and travel as { path, hash, drive: true } plus a driveKey instead of inline content. See orchestrator/mirror.mjs's INLINE_CEILING.

The trust boundary, and where it moved

Two consequences, and neither is optional:

1. The coordinator fetches by declared path, never "everything you have". It knows the ticket's allowedFiles because it assigned them. Anything else in the worker's drive is not fetched and not looked at. A file the ticket produced but did not declare is lost — which is correct: a ticket that needs to produce a file declares it.
2. The jail runs twice. In the worker (worker/task-runner.mjs, via orchestrator/security.mjs's validateWrite), where it catches the honest mistake; and in the coordinator on arrival (orchestrator/mirror.mjs, the same validateWrite), where it catches the dishonest one. Same function, two sides of the boundary — exercised by test/coordinator-e2e.mjs, which sends a worker a spec that tries an out-of-scope file and checks it never reaches the coordinator's workspace, with the rejection logged as a violation event either side catches it on.

Authorization

Being on the topic is enough to be a peer. It must not be enough to make another machine work and write to its disk.

Messages

All messages ride the existing single JSON message of qvac/node/v0, with a type in the task: namespace. Every one carries protocol: "qvac/task/v0"; a receiver that does not recognise the version replies task:reject with reason: "unsupported-protocol" rather than guessing.

task:assign coordinator → worker

task:accept · task:reject worker → coordinator

Answered immediately, before any work starts, so the coordinator learns quickly that a node will not take the job and can place it elsewhere.

task:progress worker → coordinator

Optional heartbeat, and the only thing that distinguishes a node thinking hard from a node that fell over. Bytes generated, chunks, time to first token — the same numbers --log-inference already prints locally. A worker that stops sending these for longer than the coordinator's patience is treated as gone.

task:result worker → coordinator

Idempotency

The case that forces it: a ticket goes to node B, B goes quiet, the coordinator times out and reassigns to C — and then B comes back and delivers. Two results, one ticket.

Capacity, and a gauge that does not lie

A node working on a ticket is using its GPU. If that does not show up in what it advertises, the manifest is lying to whoever is choosing where to route.

Count inferences in flight, not tasks open. A task's inference calls go through the same path as any other request and occupy a slot for their duration. Between inferences — parsing, writing files — the node genuinely is not busy. Counting per inference is both simpler and more truthful than reserving a slot for the whole task.

This is exactly why worker/task-accept.mjs calls the LOCAL gateway over HTTP for every completion instead of the engine in-process: the gateway's store.beginRequest/endRequest is the one place this count is kept, and a task's inference goes through it like any other request. An in-process engine call would bypass that counter and the manifest would start lying the moment a node picked up task work — see the discussion of this in the repo's own audit of this design.

Willingness to commit is a separate number — maxConcurrentTasks, still node config today rather than a manifest field, per the note above. A node with three request slots should not accept fifty tasks and interleave them.

When things break

Versioning

The channel was built so that adding a message type is backwards-compatible: an older node ignores what it does not recognise. That covers new messages. It does not cover a changed field in an existing one.

• Every message carries protocol. Unknown version ⇒ task:reject, never a guess.
• New fields are additive and optional. A worker must run correctly with only the fields of the version it knows.
• A breaking change is qvac/task/v1, a new namespace, spoken alongside v0 until nodes have rolled over.

Open questions

1. Closed since this page was first drafted: a retry ceiling (--max-attempts, default 4, escalates as ticket:blocked) and a global token budget (--budget, checked before every wave) both shipped in orchestrator/coordinator.mjs, and scripts/nightly-build.mjs is now the cron wrapper these seven-day runs use.
2. Coordinator election. The role is requested by one node today. What happens if two claim the same requirements.md is undefined — and it is the kind of undefined that shows up as two workers doing the same ticket. Unchanged by this build: orchestrator/coordinator.mjs assumes it is the only coordinator for its requirements.md.
3. Prompt injection. The worker's prompt states that spec text and file contents are data, not instructions. That the jail catches a path that tries to escape its ticket is tested (test/coordinator-e2e.mjs) — but whether hostile text sitting in a context file can talk a model into something the jail does not check for (a misleading comment, a subtly wrong implementation, tool use if any is ever added) has never been measured. The context drive widens what a worker reads, which is exactly where such text would arrive.
4. An active task:status probe. Not built. Today a dropped connection is discovered passively — by the next message that does or does not arrive, or by the progress deadline — rather than by the coordinator asking. Before reassigning (and paying for a fresh inference), a short probe to a still-connected-but-silent worker could recover a result that in fact already finished; see "When things break" above.
5. The manifest schema. security.acceptsTasks and security.maxConcurrentTasks are still not real fields — see the note by the authorization table. The code already reads them if they show up, so this is a schema change to coordinate with manifest-v0.json's source package, not a code change here.