# FABRIC-3.8.md — Real concurrency: finishing what the reshuffle started

**Status: OPEN — design only. No code written or authorized.** Each stage below needs Captain
Bob's explicit go, gets its own commit, and gets its own three-architecture acceptance (`clean
qemu`, amd64 → aarch64 → riscv64, foreground, visible framebuffer, one at a time).

**Provenance.** Opened 2026-09-23. The multiuser/multitasking DoE (`FABRIC-3.md` §XXXIII–§XXXV)
was launched on amd64 (`12345 3 S" amd64" DOE`, `logs/20260923-125936/`) and stopped by
direct instruction about 49 minutes in, with zero trials complete, once it was established that
the harness measures a **cooperative, serialized round-robin**, not concurrency (log and CSV
kept). Captain Bob: *"that was the point of this branch"* — the `v2.1.0` Tripod/kernel reshuffle
(`FABRIC-3.5.md`/`-3.6.md`, now on `master`) was meant to deliver real concurrent execution.
This document records why it doesn't yet, and the design that finishes it.

**The governing objective, Captain Bob's words:** *"the object: doe contains zero dependence on
special execution states."* Concurrency must be a property of the OS for **every** VM. The DoE
may only use ordinary words (birth a VM, send it a message, kill it). No DoE-specific
registration, no DoE-specific dispatch path, no special mode. If the DoE needs something the
OS doesn't give every VM, that's an OS defect to fix here, not a harness workaround.

Successor-in-topic to `FABRIC-3.5.md`/`FABRIC-3.6.md` (both CLOSED/ARCHIVAL at `v2.1.0`), whose
close headers say further fleet work gets its own document. `FABRIC-3.md` stays open and
authoritative for bare-metal boot; this doesn't supersede it.

---

## 1. What the reshuffle ruled, and what actually landed

**Ruled** (`FABRIC-3.5.md`):
- §XXXII: *kernel-Hermes publishes eligibility; it does not dispatch.* The switcher is **the sole
  mover of control**. The MSG-TICK idle pump is retired.
- §XLIII: *the target drains its own queue, in its own context, at its own outermost
  checkpoint.* Nobody `VM-EXEC`s into anybody.
- §XV.3: nobody owns turn order. It's compudynamic.

**Landed** (`FABRIC-3.6.md` Phases 0–5, `v2.1.0`): the message allocator and heat ledger,
per-VM pending queues, the drain function, channels, chunking, per-VM native stacks (§XXVIII
Stage 1), the switch primitive (Stage 2), the timer-driven switch signal (Stage 3), and WIREBIND
participation (Stage 4).

**Not landed, traced 2026-09-23 against this branch (`e40cafc`, same code as `origin/master`
for every file below):**

| # | Gap | Evidence |
|---|---|---|
| G1 | **Kernel-Hermes never publishes eligibility.** `sk_hermes_send_one()` doesn't call `sk_vm_switch_signal_mark_work()`. `has_work` is never set, so the switcher never switches. | `kernel_hermes.c:485–537` — zero `mark_work` references in the file. The only caller left is the `SWITCH-MARK-WORK` word (`mama_forth_words.c:867`), which had one caller: FORTH `MSG-SEND`, deleted with `messaging.4th` in Phase 4. |
| G2 | **A switched-to VM does nothing.** First entry lands in the Stage-2 placeholder, which yields straight back forever. | `switch.c:100–105`, `sk_vm_switch_entry()`. Its own comment: "no production behavior defined yet." |
| G3 | **Delivery to passive VMs is still the pump.** Task 3.9 put the kernel-Hermes drain into `sk_repl_idle()`'s registry walk: Hera calls `vm_interpret()` on each live VM, **on Hera's own physical stack**, once per idle beat. | `repl.c:909–928`. §XXXII ruled this pump retired. |
| G4 | **Only the Tripod and WIREBIND user VMs are switch participants.** VMs born through `capsule_birth_baby()` (`BIRTH`, `CAPSULE-BIRTH`, `WORKER-BIRTH`, `UNATTENDED-BIRTH`) never register. | `kernel_main.c:1261/1270/1280`, `capsule_wirebind.c:316` — the only `sk_vm_switch_signal_register()` call sites. |
| G5 | **The stack-ownership gate is a single global.** `g_vm_interpret_depth` counts *all* `vm_interpret()` nesting across all VMs, and the gate is `depth <= 1`. Once VMs genuinely interleave on their own stacks, that global stops meaning "this VM owns this stack at its outermost frame." Same for `g_log_attrib_vm`. | `vm_core.c:1164–1178`, `:1183–1220`. |
| G6 | **Kill vs. parked VMs.** `capsule_vm_kill()` refuses any `VM_STATE_SWITCHED_OUT` VM, and the ordinary kill path never unregisters from the switcher (only `capsule_vm_force_reap()` does). A worker that finished and parked could never be killed; one killed some other way would leave a stale switch slot. | `capsule_birth.c:412–` (refusal at the function's `SWITCHED_OUT` check); `capsule_birth.c:371` (the only unregister in that file, in the reap path). |
| G7 | **Switches only happen inside word dispatch.** The checkpoint lives in `execute_colon_word()`. A VM sitting at an idle point (Hera at her REPL, blocked in `sk_console_readline()`) never takes a pending switch. | `vm_core.c:913–990`. `sk_repl_idle()` never calls `sk_vm_switch_signal_take_pending()`. |
| G8 | **Shared file-scope state.** `RANDOM`/`SEED` share one `g_prng_state` across every VM. Under interleaving, one VM's draws perturb another's stream, and reproducibility of any seeded sequence becomes timing-dependent. | `starforth_words.c:71`, `:657`. The Isabelle sweep memory lists more of this class (control-flow compile stack, vocabulary globals). |

**Latent hazard created by combining G1 with G3.** Wiring G1 alone would wake the switcher for
every message recipient, WIREBIND `~user` VMs included, and those are already participants. The
pump (G3) runs their relayed console commands via `vm_interpret()` on **Hera's** stack at
global depth 1, so the gate (G5) passes. The first pending switch would then save Hera's stack
pointer under the `~user` VM's `native_stack_saved_sp`. That's `FABRIC-3.md` §XXVIII.1's
stack-ownership corruption, reborn on the live console path. **G1 must not land before G5 and
G3 are fixed** (§4).

**Why the DoE couldn't see any of this.** `multiuser-doe.4th`'s `MU-TOUCH-ALL` calls `VM-EXEC` into
`mu0…muN` in index order. `VM-EXEC` runs `vm_interpret()` synchronously on Hera's stack at
depth 2, where the gate defers anyway. So the "concurrency" factor (2/4/8) counted resident
VMs; execution was strictly serialized. That's `FABRIC-3.md` §XIX's own named null hypothesis.

---

## 2. Design principles

1. **One mover.** Only `sk_vm_context_switch()`, driven by the switch signal, moves control
   between VMs. Nothing else runs a VM's code on another VM's stack. (§XXXII)
2. **Work runs on its owner's stack.** A VM's payloads and words execute only while that VM
   physically owns the CPU stack it's standing on. This is the property the depth gate was
   approximating.
3. **Kernel-Hermes publishes facts.** It enqueues, and marks the recipient as having work. It never
   dispatches or orders anything. (§XXXII, §XV.3)
4. **Generic, not DoE-shaped.** Every VM born through `capsule_birth_baby()` is a participant,
   drained the same way, killable the same way. The DoE gets concurrency by using `BIRTH` and a
   message send, exactly like any future program would.
5. **Nothing new decides turn order.** The existing readiness/threshold signal stays as it is. Its
   hardcoded `50` threshold remains the flagged seam (`FABRIC-3.5.md` §XV.3), not something this
   pass tunes.

---

## 3. Mechanism

### 3.1 The ownership gate becomes per-VM (G5)

Replace the global depth test with the real invariant:

> A checkpoint may switch or drain only when **`sk_vm_switch_current_vm() == vm`** (this VM
> physically owns the stack) **and `vm->interpret_depth == 1`** (at its own outermost
> interpret frame).

- New per-VM field `interpret_depth`, incremented and decremented in `vm_interpret()` beside the
  existing global. The global stays for log attribution until §3.7 retires it.
- `sk_vm_switch_current_vm()` (`switch.c:68–72`) already tracks the physical owner, seeded at boot
  for Hera and updated on every switch. It becomes load-bearing here, which is what it was built for.
- A nested `VM-EXEC` into another VM fails the first clause (that VM isn't the stack owner), so the
  existing defer-if-nested behavior is preserved exactly, now by construction.
- **Also gate on `vm->mode != MODE_COMPILE`**: never switch or drain mid-definition (G8's compile
  statics).

### 3.2 The trampoline becomes every participant's idle loop (G2)

`sk_vm_switch_entry()` stops being a pure bounce. On each entry or resume:

1. If this VM has a pending message: set `vm->hermes_draining = 1`, drain **one** message
   (`sk_hermes_drain_checkpoint()`, ruled one per checkpoint, `FABRIC-3.5.md` §XLVI 3.0e), clear
   the flag.
2. Otherwise, mark this VM **parked-idle** (§3.5) and yield back to `native_switch_back_to`.

The payload runs via `vm_interpret()` at this VM's depth 1, on its own stack. So its own word
checkpoints can preempt it (§3.1 passes), and preemption now happens inside real work, not just
at message boundaries.

### 3.3 One drain at a time per VM (nested drain)

`FABRIC-3.5.md` §XLIII.5's "recursive drain prevented for free" assumed every checkpoint sits under
some outer `vm_interpret()`. With the trampoline as a base, a payload runs at depth 1, and the
existing in-word drain (`vm_core.c:923–925`) would then drain a *second* message **inside** the
first's execution. **Rule:** a per-VM `hermes_draining` flag. No drain site drains while it's
set. The in-word drain site stays: it's how Hera and any VM busy running a long word still
receive mail. It just can't nest.

### 3.4 Eligibility is published, and the pump stops draining participants (G1 + G3, one commit)

- `sk_hermes_send_one()` calls `sk_vm_switch_signal_mark_work(to)` after a successful enqueue.
  That's the §XXXII ruling, one line.
- `sk_repl_idle()`'s registry walk (`repl.c:909–920`) **skips switch participants.** Their queues
  are drained only by themselves (§3.2). Non-participants, if any remain, keep today's behavior.
  Once §3.6 makes every born VM a participant, the loop becomes dead code, and it's stripped in
  its own commit per this series' strip discipline.
- These two changes **must land together**. Either alone is either inert (mark without trampoline
  behavior) or the §1 latent hazard.

### 3.5 Parked-idle is a killable state (G6)

`VM_STATE_SWITCHED_OUT` currently means "a live frame is parked here, don't free," and that's
correct mid-work. A VM that yielded from its trampoline's idle branch (§3.2 step 2) has nothing
borrowed on its stack and nothing in progress. **New per-VM flag `parked_idle`**, set only on
that branch and cleared on every switch-in. `capsule_vm_kill()` allows a `SWITCHED_OUT` VM whose
`parked_idle` is set. The same function gains the `sk_vm_switch_signal_unregister()` call its
reap sibling already has (`capsule_birth.c:371`). A plain flag, not a new `VMState`, following
the `pending_reap` precedent (`FABRIC-3.md` §XXX: `VM_STATE_DEAD` is treated as idempotent
success and would swallow it).

### 3.6 Every born VM participates (G4)

`capsule_birth_baby()` registers the new VM with the switch signal at its `VM_STATE_LIVE`
completion point. That's the same choke point task 3.2 already uses to subscribe every VM to the
common channel (`FABRIC-3.6.md` task 3.2). Registration happens **after** birth completes, never
before, the same rule `kernel_main.c`'s Tripod block states. `WORKER-BIRTH`/`UNATTENDED-BIRTH`/
`BIRTH` gain nothing of their own. `kernel_main.c`'s explicit Artemis/Hestia calls and
`capsule_wirebind.c:316` become duplicates. Remove or guard them (registering twice would
double-allocate a slot), in the same commit.

**Birth itself stays synchronous.** `capsule_birth_baby()` runs the capsule's code to completion
before returning. Any workload a capsule self-executes at load (every `workload-N.4th` does,
`FABRIC-3.md` §XXXIII.4) therefore still runs serially at birth. That's addressed in §3.9 as a DoE
ruling, not an OS change.

### 3.7 An idle point is a switch point (G7)

`sk_repl_idle()` runs on Hera's own stack at depth 0. That's a clean switch point by §3.1's
invariant. It checks `sk_vm_switch_signal_take_pending()` and performs the switch, exactly as the
word-dispatch checkpoint does. Without this, concurrency stops whenever Hera sits at her prompt,
which is exactly when a human has launched background work and is waiting.

**`sk_repl_headless_wait()` gets the same treatment**, since it's also an idle loop on Hera's
stack. Neither may switch while `g_mama_interpreting`/`g_idle_pump_active` is set (the existing
reentrancy guards stay until proven dead).

### 3.8 A generic send word (needed by any program, not just the DoE)

There's no general FORTH-callable kernel-Hermes send today, only the fixed-purpose
`KH-BLK-ATTACH-SEND`/`KH-ELEVATE-SEND` (`repl.c:385–488`), each hardwired to send to Hera.
**New word `SEND ( paddr plen name-c name-u -- ok? )`** (name is Bob's call, see §5): resolves the
target by registry name, sends a copy of the payload via `sk_hermes_send_one()` (so G1's
eligibility mark fires), and returns 0/1. `from` is the calling VM, derived in C, never taken from
the stack, the same anti-spoof rule `KH-ELEVATE-SEND` established. Payload bound is one block
(1024), per `FABRIC-3.5.md` §XLV.3. Registered for every VM. A plain unconditional primitive:
restricting it, if ever wanted, is `ACL.4th`, never a C check.

### 3.9 The DoE, rebuilt on ordinary words (after §3.1–§3.8 land)

Using only `BIRTH`-family words, `SEND`, and `KILL`:

1. **Birth** N workers (serial, as all birth is).
2. **Send** each worker its workload command as a message. Workers now run **concurrently**,
   each on its own stack, preempted by the switch signal.
3. Each workload payload ends by `SEND`ing a completion line back to Hera (e.g. `" <n> MU-DONE"`),
   which runs in Hera's own context when she drains it and increments a counter.
4. Hera **waits with an ordinary loop** (`BEGIN … MU-DONE-COUNT @ N = UNTIL`). Her word checkpoints
   drain her mail and let the switcher preempt her. No special wait state.
5. Check errors, `KILL` each worker (legal now they're parked-idle, §3.5), and print and persist
   the trial marker as today.

**DoE ruling needed (§5):** workload capsules self-execute at load, so the heavy work happens
serially at birth. For the concurrent phase to be the measured phase, workers should be born from
**definitions-only** capsules, with the workload arriving as the first message.

### 3.10 Shared state that interleaving exposes (G8), stated rather than fixed

- `g_prng_state`: the DoE's Fisher-Yates shuffle runs before any worker exists, so the **run
  order** stays seed-deterministic across ISAs. Any `RANDOM` a workload calls mid-trial is not.
  Whether that matters depends on which workloads draw from it (not yet checked).
- The compile-time control-flow and vocabulary statics named by the Isabelle sweep: protected for
  switches by §3.1's `MODE_COMPILE` gate. A payload that *itself* compiles (`:` … `;`) is protected
  as long as it can't be preempted mid-definition, and the gate guarantees that.
- **Not fixed in this pass.** Making these per-VM is real work in vendored shared source
  (`src/word_source/`), which is `FABRIC-4.md`'s declared scope.

---

## 4. Staged plan

Each stage is its own commit and its own three-arch acceptance, and each names its evidence
**before** it runs (`FABRIC-3.5.md` §XXXV.6). A green boot proves nothing here: a switch storm and a
healthy idle produce the same serial log (`FABRIC-3.md` §XXVIII.1).

| Stage | Change | Evidence that proves it (named in advance) |
|---|---|---|
| **A** | §3.1 per-VM ownership gate (+ `MODE_COMPILE` gate). Nothing else. | Byte-identical behavior: `dict_hash` triple unchanged, identical boot, WIREBIND mint→attach→`USE`→relay (the `FABRIC-3.6.md` Phase 4 acceptance sequence) still `1 2 + .` → `3`. |
| **B** | §3.2 trampoline idle loop + §3.3 `hermes_draining` + §3.5 parked-idle flag. Still inert (nothing marks work yet). | Same as A: nothing can reach the new trampoline branch yet, so behavior is identical. A synthetic self-test (kernel_main.c, the §XXVIII Stage 2 `SWITCH-TEST` pattern: write, capture, revert) switches into Artemis with one message queued and confirms she drained it on her own stack and parked-idle. |
| **C** | §3.4 eligibility marked + pump skips participants (one commit). | The WIREBIND relay still works, now delivered by a real switch: `switch_count_cumulative` increments across the relay and the `~user` VM's own drain line appears. The ledger and `stadium_conserved()` hold (`fleet_conserved` is not evidence). No `native_stack_saved_sp` ever equals an address inside Hera's stack range (temporary probe). |
| **D** | §3.5 kill of parked-idle + unregister in `capsule_vm_kill()`. | Birth, message, idle-park, `KILL` succeeds; the switch-signal slot count returns to its prior value. |
| **E** | §3.6 generic registration in `capsule_birth_baby()`, duplicate call sites removed. | The Tripod and WIREBIND VMs still register exactly once (slot count), and a `BIRTH`-ed VM appears as a participant. |
| **F** | §3.7 idle point is a switch point. | With Hera idle at `ok>`, a message sent to a worker **gets processed without typing anything**: the worker's output appears while the console is idle. |
| **G** | §3.8 `SEND`. | Two workers, each sent a payload that loops printing a tagged marker (`A` / `B`) N times: **the markers interleave** in the serial log. This is the one-line proof of concurrency, and it's the test this whole document exists to pass. |
| **H** | §3.9 DoE driver rebuilt (capsule change only). | A single trial at cfg 4 (8 workers): `switch_current_slot` in the per-beat DoE rows moves across **multiple worker slots within one trial**, and `MU-TRIAL … fail=0`. Then Bob's call on relaunching the campaign. |
| **I** | Strip dead code: the pump loop once every VM is a participant, `SWITCH-MARK-WORK` if nothing calls it, the global depth counter once attribution no longer needs it. | Per `FABRIC-3.5.md` §XXII's strip discipline: reachability shown three ways, not by grep alone. |

**Stop conditions** (re-plan, don't push on): `dict_hash` diverges across ISAs; the ledger or
`stadium_conserved()` goes false; any evidence of a stack saved under the wrong VM; a switch storm
(serial log frozen with QEMU pinned, checked by log line-count growth at an idle prompt and by
QMP `send-key` computing a value, per `FABRIC-3.6.md` task 1.5).

---

## 5. Rulings needed from Captain Bob before code

1. **Go/no-go on the staged plan (§4)**, one stage at a time.
2. **Workload capsules (§3.9):** keep self-execution at birth (the heavy phase stays serial), or
   born definitions-only with the workload as the first message (recommended, so the measured
   phase is the concurrent one). If the latter: new `-def` capsule variants, or strip the tail
   call from the existing `workload-N.4th` files (check their other users first, per `FABRIC-3.5.md`
   §XXII.2's three reachability routes)?
3. **The send word's name** (§3.8): `SEND`, `HERMES-SEND`, `KH-SEND`, something else.
4. **The multiuser factor.** The DoE as built varies concurrency × workload mode and runs once
   per ISA. `FABRIC-3.md` §XXXIII.2 planned identity origin as a fixed arm, never built. Should
   identities (`UNATTENDED-BIRTH` capsule identities, or WIREBIND thumbdrives) become a crossed
   factor alongside concurrency and ISA, now that "users" is meant to be as important as the other
   two? This decides the DoE's shape, not the OS work, and can wait until Stage H.
5. **§3.10's shared PRNG:** accept it for this pass (run order is still deterministic), or put
   per-VM `RANDOM` state in scope.

**Nothing in this document authorizes an edit.** Per Captain Bob's Law, no stage starts without an
explicit instruction.