LithosAnanake/FABRIC-3.8.md

FABRIC-3.8.md — Real concurrency: finishing what the reshuffle started

Status: OPEN — design only. No code written or authorized. Each stage below needs Captain Bob's explicit go, gets its own commit, and gets its own three-architecture acceptance (clean qemu, amd64 → aarch64 → riscv64, foreground, visible framebuffer, one at a time).

Provenance. Opened 2026-09-23. The multiuser/multitasking DoE (FABRIC-3.md §XXXIII–§XXXV) was launched on amd64 (12345 3 S" amd64" DOE, logs/20260923-125936/) and stopped by direct instruction about 49 minutes in, with zero trials complete, once it was established that the harness measures a cooperative, serialized round-robin, not concurrency (log and CSV kept). Captain Bob: "that was the point of this branch" — the v2.1.0 Tripod/kernel reshuffle (FABRIC-3.5.md/-3.6.md, now on master) was meant to deliver real concurrent execution. This document records why it doesn't yet, and the design that finishes it.

The governing objective, Captain Bob's words: "the object: doe contains zero dependence on special execution states." Concurrency must be a property of the OS for every VM. The DoE may only use ordinary words (birth a VM, send it a message, kill it). No DoE-specific registration, no DoE-specific dispatch path, no special mode. If the DoE needs something the OS doesn't give every VM, that's an OS defect to fix here, not a harness workaround.

Successor-in-topic to FABRIC-3.5.md/FABRIC-3.6.md (both CLOSED/ARCHIVAL at v2.1.0), whose close headers say further fleet work gets its own document. FABRIC-3.md stays open and authoritative for bare-metal boot; this doesn't supersede it.

---

1. What the reshuffle ruled, and what actually landed

Ruled (FABRIC-3.5.md):

• §XXXII: kernel-Hermes publishes eligibility; it does not dispatch. The switcher is the sole mover of control. The MSG-TICK idle pump is retired.
• §XLIII: the target drains its own queue, in its own context, at its own outermost checkpoint. Nobody VM-EXECs into anybody.
• §XV.3: nobody owns turn order. It's compudynamic.

Landed (FABRIC-3.6.md Phases 0–5, v2.1.0): the message allocator and heat ledger, per-VM pending queues, the drain function, channels, chunking, per-VM native stacks (§XXVIII Stage 1), the switch primitive (Stage 2), the timer-driven switch signal (Stage 3), and WIREBIND participation (Stage 4).

Not landed, traced 2026-09-23 against this branch (e40cafc, same code as origin/master for every file below):

#	Gap	Evidence
G1	Kernel-Hermes never publishes eligibility. sk_hermes_send_one() doesn't call sk_vm_switch_signal_mark_work(). has_work is never set, so the switcher never switches.	kernel_hermes.c:485–537 — zero mark_work references in the file. The only caller left is the SWITCH-MARK-WORK word (mama_forth_words.c:867), which had one caller: FORTH MSG-SEND, deleted with messaging.4th in Phase 4.
G2	A switched-to VM does nothing. First entry lands in the Stage-2 placeholder, which yields straight back forever.	switch.c:100–105, sk_vm_switch_entry(). Its own comment: "no production behavior defined yet."
G3	Delivery to passive VMs is still the pump. Task 3.9 put the kernel-Hermes drain into sk_repl_idle()'s registry walk: Hera calls vm_interpret() on each live VM, on Hera's own physical stack, once per idle beat.	repl.c:909–928. §XXXII ruled this pump retired.
G4	Only the Tripod and WIREBIND user VMs are switch participants. VMs born through capsule_birth_baby() (BIRTH, CAPSULE-BIRTH, WORKER-BIRTH, UNATTENDED-BIRTH) never register.	kernel_main.c:1261/1270/1280, capsule_wirebind.c:316 — the only sk_vm_switch_signal_register() call sites.
G5	The stack-ownership gate is a single global. g_vm_interpret_depth counts all vm_interpret() nesting across all VMs, and the gate is depth <= 1. Once VMs genuinely interleave on their own stacks, that global stops meaning "this VM owns this stack at its outermost frame." Same for g_log_attrib_vm.	vm_core.c:1164–1178, :1183–1220.
G6	Kill vs. parked VMs. capsule_vm_kill() refuses any VM_STATE_SWITCHED_OUT VM, and the ordinary kill path never unregisters from the switcher (only capsule_vm_force_reap() does). A worker that finished and parked could never be killed; one killed some other way would leave a stale switch slot.	capsule_birth.c:412– (refusal at the function's SWITCHED_OUT check); capsule_birth.c:371 (the only unregister in that file, in the reap path).
G7	Switches only happen inside word dispatch. The checkpoint lives in execute_colon_word(). A VM sitting at an idle point (Hera at her REPL, blocked in sk_console_readline()) never takes a pending switch.	vm_core.c:913–990. sk_repl_idle() never calls sk_vm_switch_signal_take_pending().
G8	Shared file-scope state. RANDOM/SEED share one g_prng_state across every VM. Under interleaving, one VM's draws perturb another's stream, and reproducibility of any seeded sequence becomes timing-dependent.	starforth_words.c:71, :657. The Isabelle sweep memory lists more of this class (control-flow compile stack, vocabulary globals).

Latent hazard created by combining G1 with G3. Wiring G1 alone would wake the switcher for every message recipient, WIREBIND ~user VMs included, and those are already participants. The pump (G3) runs their relayed console commands via vm_interpret() on Hera's stack at global depth 1, so the gate (G5) passes. The first pending switch would then save Hera's stack pointer under the ~user VM's native_stack_saved_sp. That's FABRIC-3.md §XXVIII.1's stack-ownership corruption, reborn on the live console path. G1 must not land before G5 and G3 are fixed (§4).

Why the DoE couldn't see any of this. multiuser-doe.4th's MU-TOUCH-ALL calls VM-EXEC into mu0…muN in index order. VM-EXEC runs vm_interpret() synchronously on Hera's stack at depth 2, where the gate defers anyway. So the "concurrency" factor (2/4/8) counted resident VMs; execution was strictly serialized. That's FABRIC-3.md §XIX's own named null hypothesis.

---

2. Design principles

1. One mover. Only sk_vm_context_switch(), driven by the switch signal, moves control between VMs. Nothing else runs a VM's code on another VM's stack. (§XXXII)
2. Work runs on its owner's stack. A VM's payloads and words execute only while that VM physically owns the CPU stack it's standing on. This is the property the depth gate was approximating.
3. Kernel-Hermes publishes facts. It enqueues, and marks the recipient as having work. It never dispatches or orders anything. (§XXXII, §XV.3)
4. Generic, not DoE-shaped. Every VM born through capsule_birth_baby() is a participant, drained the same way, killable the same way. The DoE gets concurrency by using BIRTH and a message send, exactly like any future program would.
5. Nothing new decides turn order. The existing readiness/threshold signal stays as it is. Its hardcoded 50 threshold remains the flagged seam (FABRIC-3.5.md §XV.3), not something this pass tunes.

---

3. Mechanism

3.1 The ownership gate becomes per-VM (G5)

Replace the global depth test with the real invariant:

A checkpoint may switch or drain only when sk_vm_switch_current_vm() == vm (this VM physically owns the stack) and vm->interpret_depth == 1 (at its own outermost interpret frame).

• New per-VM field interpret_depth, incremented and decremented in vm_interpret() beside the existing global. The global stays for log attribution until §3.7 retires it.
• sk_vm_switch_current_vm() (switch.c:68–72) already tracks the physical owner, seeded at boot for Hera and updated on every switch. It becomes load-bearing here, which is what it was built for.
• A nested VM-EXEC into another VM fails the first clause (that VM isn't the stack owner), so the existing defer-if-nested behavior is preserved exactly, now by construction.
• Also gate on vm->mode != MODE_COMPILE: never switch or drain mid-definition (G8's compile statics).

3.2 The trampoline becomes every participant's idle loop (G2)

sk_vm_switch_entry() stops being a pure bounce. On each entry or resume:

1. If this VM has a pending message: set vm->hermes_draining = 1, drain one message (sk_hermes_drain_checkpoint(), ruled one per checkpoint, FABRIC-3.5.md §XLVI 3.0e), clear the flag.
2. Otherwise, mark this VM parked-idle (§3.5) and yield back to native_switch_back_to.

The payload runs via vm_interpret() at this VM's depth 1, on its own stack. So its own word checkpoints can preempt it (§3.1 passes), and preemption now happens inside real work, not just at message boundaries.

3.3 One drain at a time per VM (nested drain)

FABRIC-3.5.md §XLIII.5's "recursive drain prevented for free" assumed every checkpoint sits under some outer vm_interpret(). With the trampoline as a base, a payload runs at depth 1, and the existing in-word drain (vm_core.c:923–925) would then drain a second message inside the first's execution. Rule: a per-VM hermes_draining flag. No drain site drains while it's set. The in-word drain site stays: it's how Hera and any VM busy running a long word still receive mail. It just can't nest.

3.4 Eligibility is published, and the pump stops draining participants (G1 + G3, one commit)

• sk_hermes_send_one() calls sk_vm_switch_signal_mark_work(to) after a successful enqueue. That's the §XXXII ruling, one line.
• sk_repl_idle()'s registry walk (repl.c:909–920) skips switch participants. Their queues are drained only by themselves (§3.2). Non-participants, if any remain, keep today's behavior. Once §3.6 makes every born VM a participant, the loop becomes dead code, and it's stripped in its own commit per this series' strip discipline.
• These two changes must land together. Either alone is either inert (mark without trampoline behavior) or the §1 latent hazard.

3.5 Parked-idle is a killable state (G6)

VM_STATE_SWITCHED_OUT currently means "a live frame is parked here, don't free," and that's correct mid-work. A VM that yielded from its trampoline's idle branch (§3.2 step 2) has nothing borrowed on its stack and nothing in progress. New per-VM flag parked_idle, set only on that branch and cleared on every switch-in. capsule_vm_kill() allows a SWITCHED_OUT VM whose parked_idle is set. The same function gains the sk_vm_switch_signal_unregister() call its reap sibling already has (capsule_birth.c:371). A plain flag, not a new VMState, following the pending_reap precedent (FABRIC-3.md §XXX: VM_STATE_DEAD is treated as idempotent success and would swallow it).

3.6 Every born VM participates (G4)

capsule_birth_baby() registers the new VM with the switch signal at its VM_STATE_LIVE completion point. That's the same choke point task 3.2 already uses to subscribe every VM to the common channel (FABRIC-3.6.md task 3.2). Registration happens after birth completes, never before, the same rule kernel_main.c's Tripod block states. WORKER-BIRTH/UNATTENDED-BIRTH/ BIRTH gain nothing of their own. kernel_main.c's explicit Artemis/Hestia calls and capsule_wirebind.c:316 become duplicates. Remove or guard them (registering twice would double-allocate a slot), in the same commit.

Birth itself stays synchronous. capsule_birth_baby() runs the capsule's code to completion before returning. Any workload a capsule self-executes at load (every workload-N.4th does, FABRIC-3.md §XXXIII.4) therefore still runs serially at birth. That's addressed in §3.9 as a DoE ruling, not an OS change.

3.7 An idle point is a switch point (G7)

sk_repl_idle() runs on Hera's own stack at depth 0. That's a clean switch point by §3.1's invariant. It checks sk_vm_switch_signal_take_pending() and performs the switch, exactly as the word-dispatch checkpoint does. Without this, concurrency stops whenever Hera sits at her prompt, which is exactly when a human has launched background work and is waiting.

sk_repl_headless_wait() gets the same treatment, since it's also an idle loop on Hera's stack. Neither may switch while g_mama_interpreting/g_idle_pump_active is set (the existing reentrancy guards stay until proven dead).

3.8 A generic send word (needed by any program, not just the DoE)

There's no general FORTH-callable kernel-Hermes send today, only the fixed-purpose KH-BLK-ATTACH-SEND/KH-ELEVATE-SEND (repl.c:385–488), each hardwired to send to Hera. New word SEND ( paddr plen name-c name-u -- ok? ) (name is Bob's call, see §5): resolves the target by registry name, sends a copy of the payload via sk_hermes_send_one() (so G1's eligibility mark fires), and returns 0/1. from is the calling VM, derived in C, never taken from the stack, the same anti-spoof rule KH-ELEVATE-SEND established. Payload bound is one block (1024), per FABRIC-3.5.md §XLV.3. Registered for every VM. A plain unconditional primitive: restricting it, if ever wanted, is ACL.4th, never a C check.

3.9 The DoE, rebuilt on ordinary words (after §3.1–§3.8 land)

Using only BIRTH-family words, SEND, and KILL:

1. Birth N workers (serial, as all birth is).
2. Send each worker its workload command as a message. Workers now run concurrently, each on its own stack, preempted by the switch signal.
3. Each workload payload ends by SENDing a completion line back to Hera (e.g. " <n> MU-DONE"), which runs in Hera's own context when she drains it and increments a counter.
4. Hera waits with an ordinary loop (BEGIN … MU-DONE-COUNT @ N = UNTIL). Her word checkpoints drain her mail and let the switcher preempt her. No special wait state.
5. Check errors, KILL each worker (legal now they're parked-idle, §3.5), and print and persist the trial marker as today.

DoE ruling needed (§5): workload capsules self-execute at load, so the heavy work happens serially at birth. For the concurrent phase to be the measured phase, workers should be born from definitions-only capsules, with the workload arriving as the first message.

3.10 Shared state that interleaving exposes (G8), stated rather than fixed

• g_prng_state: the DoE's Fisher-Yates shuffle runs before any worker exists, so the run order stays seed-deterministic across ISAs. Any RANDOM a workload calls mid-trial is not. Whether that matters depends on which workloads draw from it (not yet checked).
• The compile-time control-flow and vocabulary statics named by the Isabelle sweep: protected for switches by §3.1's MODE_COMPILE gate. A payload that itself compiles (: … ;) is protected as long as it can't be preempted mid-definition, and the gate guarantees that.
• Not fixed in this pass. Making these per-VM is real work in vendored shared source (src/word_source/), which is FABRIC-4.md's declared scope.

---

4. Staged plan

Each stage is its own commit and its own three-arch acceptance, and each names its evidence before it runs (FABRIC-3.5.md §XXXV.6). A green boot proves nothing here: a switch storm and a healthy idle produce the same serial log (FABRIC-3.md §XXVIII.1).

Stage	Change	Evidence that proves it (named in advance)
A	§3.1 per-VM ownership gate (+ MODE_COMPILE gate). Nothing else.	Byte-identical behavior: dict_hash triple unchanged, identical boot, WIREBIND mint→attach→USE→relay (the FABRIC-3.6.md Phase 4 acceptance sequence) still 1 2 + . → 3.
B	§3.2 trampoline idle loop + §3.3 hermes_draining + §3.5 parked-idle flag. Still inert (nothing marks work yet).	Same as A: nothing can reach the new trampoline branch yet, so behavior is identical. A synthetic self-test (kernel_main.c, the §XXVIII Stage 2 SWITCH-TEST pattern: write, capture, revert) switches into Artemis with one message queued and confirms she drained it on her own stack and parked-idle.
C	§3.4 eligibility marked + pump skips participants (one commit).	The WIREBIND relay still works, now delivered by a real switch: switch_count_cumulative increments across the relay and the ~user VM's own drain line appears. The ledger and stadium_conserved() hold (fleet_conserved is not evidence). No native_stack_saved_sp ever equals an address inside Hera's stack range (temporary probe).
D	§3.5 kill of parked-idle + unregister in capsule_vm_kill().	Birth, message, idle-park, KILL succeeds; the switch-signal slot count returns to its prior value.
E	§3.6 generic registration in capsule_birth_baby(), duplicate call sites removed.	The Tripod and WIREBIND VMs still register exactly once (slot count), and a BIRTH-ed VM appears as a participant.
F	§3.7 idle point is a switch point.	With Hera idle at ok>, a message sent to a worker gets processed without typing anything: the worker's output appears while the console is idle.
G	§3.8 SEND.	Two workers, each sent a payload that loops printing a tagged marker (A / B) N times: the markers interleave in the serial log. This is the one-line proof of concurrency, and it's the test this whole document exists to pass.
H	§3.9 DoE driver rebuilt (capsule change only).	A single trial at cfg 4 (8 workers): switch_current_slot in the per-beat DoE rows moves across multiple worker slots within one trial, and MU-TRIAL … fail=0. Then Bob's call on relaunching the campaign.
I	Strip dead code: the pump loop once every VM is a participant, SWITCH-MARK-WORK if nothing calls it, the global depth counter once attribution no longer needs it.	Per FABRIC-3.5.md §XXII's strip discipline: reachability shown three ways, not by grep alone.

Stop conditions (re-plan, don't push on): dict_hash diverges across ISAs; the ledger or stadium_conserved() goes false; any evidence of a stack saved under the wrong VM; a switch storm (serial log frozen with QEMU pinned, checked by log line-count growth at an idle prompt and by QMP send-key computing a value, per FABRIC-3.6.md task 1.5).

---

5. Rulings needed from Captain Bob before code

1. Go/no-go on the staged plan (§4), one stage at a time.
2. Workload capsules (§3.9): keep self-execution at birth (the heavy phase stays serial), or born definitions-only with the workload as the first message (recommended, so the measured phase is the concurrent one). If the latter: new -def capsule variants, or strip the tail call from the existing workload-N.4th files (check their other users first, per FABRIC-3.5.md §XXII.2's three reachability routes)?
3. The send word's name (§3.8): SEND, HERMES-SEND, KH-SEND, something else.
4. The multiuser factor. The DoE as built varies concurrency × workload mode and runs once per ISA. FABRIC-3.md §XXXIII.2 planned identity origin as a fixed arm, never built. Should identities (UNATTENDED-BIRTH capsule identities, or WIREBIND thumbdrives) become a crossed factor alongside concurrency and ISA, now that "users" is meant to be as important as the other two? This decides the DoE's shape, not the OS work, and can wait until Stage H.
5. §3.10's shared PRNG: accept it for this pass (run order is still deterministic), or put per-VM RANDOM state in scope.

Nothing in this document authorizes an edit. Per Captain Bob's Law, no stage starts without an explicit instruction.