FABRIC-3.8.md Stage B: kernel-side idle loop in the switch trampoline

sk_vm_switch_entry() stops being the Stage-2 placeholder bounce and becomes
every switch participant's own idle loop: it drains one of its own
kernel-Hermes messages per pass, on its own native stack (driven by the drain's
return value, so a non-consuming drain cannot spin), and when nothing is
drained it marks the VM parked_idle and yields back. It is entirely OS-driven,
with no VM-side yield word. sk_hermes_drain_checkpoint() gains a per-VM
hermes_draining guard so a payload's own in-word checkpoint cannot drain a
second message inside the first. Still inert in normal operation: nothing
marks work until Stage C.

Finding fixed here: amd64 sk_vm_switch_prime built the first-entry frame at
top-56, entering with rsp 16-aligned instead of the SysV-required 8 mod 16, so
the first aligned SSE spill in a callee took #GP (logs/20260923-142526, kept).
It had been latent since SXXVIII Stage 2 because the placeholder never called
anything. The frame is now at top-64 with a zeroed pad slot. aarch64 and
riscv64 were already correct.

Evidence: a temporary kernel_main probe (reverted, not in this commit) queued
two messages to Hestia, the first calling a colon word, and switched into her.
It passed on all three ISAs: 0 pending, parked_idle, depth 0, context saved on
her own stack, SB-SEQ=12 (drained one at a time, not nested), ledger and
stadium_conserved restored. Formal acceptance without the probe on all three
ISAs: dict_hash identical and unchanged, zero UNKNOWN WORD or faults, and the
live WIREBIND mint -> USE -> 1 2 + . -> 3 relay.

FABRIC-3.8.md also records Captain Bob's rulings that no VM-side word ever gives
up control and that multitasking is a compudynamic hybrid, and opens SS3.12
(VM frames for per-VM execution state).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-23 14:57:51 -04:00
co-authored by Claude Opus 5.5
parent 2aa338483a
commit 56da55aba9
20 changed files with 64228 additions and 5 deletions
+85 -2
View File
@@ -1,6 +1,6 @@
# FABRIC-3.8.md — Real concurrency: finishing what the reshuffle started # FABRIC-3.8.md — Real concurrency: finishing what the reshuffle started
**Status: OPEN — Stage A done (§6); Stages B–I not started. §3.11 (compudynamic preemption) needs **Status: OPEN — Stages A and B done (§6); Stages C–I not started. §3.11 (compudynamic preemption) needs
a ruling before Stage C.** Each stage below needs Captain a ruling before Stage C.** Each stage below needs Captain
Bob's explicit go, gets its own commit, and gets its own three-architecture acceptance (`clean Bob's explicit go, gets its own commit, and gets its own three-architecture acceptance (`clean
qemu`, amd64 → aarch64 → riscv64, foreground, visible framebuffer, one at a time). qemu`, amd64 → aarch64 → riscv64, foreground, visible framebuffer, one at a time).
@@ -251,9 +251,61 @@ one (e.g. cooler per §XIX), and never merely because N timer ticks have elapsed
would then keep only what physics needs from it, and nothing that decides turns. The fixed `50` would then keep only what physics needs from it, and nothing that decides turns. The fixed `50`
threshold and the per-tick readiness accrual go away. threshold and the per-tick readiness accrual go away.
**Clarified 2026-09-23, Captain Bob:** *"this is kind of like a mixture of preemptive and
cooperative multitasking… It's the compudynamics that drives the whole thing. Not necessarily a
preemptive timer. Not necessarily cooperatively, as in 'when I finish my task, I move on'. But
it's a mixture of both."* Read as:
- **Where a switch may happen is cooperative in shape.** Only at a VM's own clean boundary, the
ownership-gated word checkpoint (§3.1): stack owned, own outermost frame, not compiling. Never
at an arbitrary instruction, never because an interrupt fired.
- **Whether, and to whom, is preemptive in effect.** A running VM doesn't hold the CPU until it
finishes. At any such boundary it can be displaced mid-task when the compudynamics favors
another VM with pending work.
- **Running out of work parks a VM.** That's the plain cooperative case (§3.2's idle loop).
- **Neither a clock nor task completion decides. The physics does.**
**Constraint, Captain Bob 2026-09-23:** *"make sure nothing inside of a given execution state or
VM controls this. There are no special words saying I'm giving up control or anything like
that. This is strictly all operating system and strictly all compudynamically driven."* No yield
word, no `PAUSE`, no "I'm done" word, anywhere. Parking happens in the kernel's trampoline when
the OS sees an empty queue (§3.2), and every switch decision is kernel-side.
**Also a Stage C input:** the trampoline's point between two messages is a clean switch point
(own stack, depth 0). The compudynamic decision has to be consulted there too, or a VM whose queue
holds only checkpoint-free payloads (primitive words, no colon calls) never yields. Noted in
`sk_vm_switch_entry()`'s own comment; not built in Stage B.
**This is a design question for Captain Bob before Stage C** (Stage C is the first stage where **This is a design question for Captain Bob before Stage C** (Stage C is the first stage where
anything actually switches). Stages A and B don't depend on it. anything actually switches). Stages A and B don't depend on it.
### 3.12 VM frames: per-VM execution state that travels with the VM (OPEN, raised by Captain Bob)
**Captain Bob, 2026-09-23:** *"I'm thinking we may need VM frames for state."* Once VMs genuinely
interleave, any execution context that isn't already inside the `VM` struct or on the VM's own
native stack is shared across VMs, and it goes wrong at the first switch that lands inside it.
The Stage B probe hit an instance of this class on its first run (§6, the amd64 alignment finding
is the same family: state the first-entry frame should have carried and didn't).
**Known members of that class today, traced:**
| State | Where | Why it breaks under interleaving |
|---|---|---|
| `g_log_attrib_vm` | `vm_core.c` (save/restore in `vm_interpret()`) | Save/restore assumes last-in-first-out nesting. Hera switches out mid-interpret; Hestia's `vm_interpret()` saves `prev=Hera`; Hestia switches back mid-payload → attribution stays "Hestia" while Hera runs. `(LOG-APPEND-RAW)` takes its source from this, so persisted log records name the wrong VM (defeating `FABRIC-3.md` §XXVII's anti-spoof rule). |
| `g_vm_interpret_depth` | `vm_core.c` | Same LIFO assumption; superseded by per-VM `interpret_depth` (Stage A) and retired in Stage I. |
| `g_switch_entry_vm` | `switch.c` | Read once at first entry; safe only while first entries can't overlap. |
| `g_prng_state` | `starforth_words.c:71` | One `RANDOM` stream for every VM (G8). |
| Compile-time control-flow and vocabulary statics | vendored `src/word_source/` (Isabelle sweep memory) | Shared across VMs; kept safe for switches by the Stage A `MODE_COMPILE` gate, not by ownership. |
**Shape, for ruling (not decided):** a `VMFrame` struct in its own header (per
`feedback_dedicated_headers_not_inline_fields`: related state gets a header, not loose `VM`
fields), embedded in `VM`, holding the per-VM execution context that is global today. The
switch primitive then needs to swap nothing extra: whatever lives in the frame moves with the VM
by construction. Log attribution becomes "the frame of `sk_vm_switch_current_vm()`" rather than a
saved global. The vendored-source statics are `FABRIC-4.md`'s scope, but the frame is where they'd
go. **Should land before or with Stage C**, since Stage C is the first stage where switches
happen for real.
--- ---
## 4. Staged plan ## 4. Staged plan
@@ -265,7 +317,7 @@ healthy idle produce the same serial log (`FABRIC-3.md` §XXVIII.1).
| Stage | Change | Evidence that proves it (named in advance) | | Stage | Change | Evidence that proves it (named in advance) |
|---|---|---| |---|---|---|
| **A** | §3.1 per-VM ownership gate (+ `MODE_COMPILE` gate). Nothing else. | Byte-identical behavior: `dict_hash` triple unchanged, identical boot, WIREBIND mint→attach→`USE`→relay (the `FABRIC-3.6.md` Phase 4 acceptance sequence) still `1 2 + .` → `3`. | | **A** | §3.1 per-VM ownership gate (+ `MODE_COMPILE` gate). Nothing else. | Byte-identical behavior: `dict_hash` triple unchanged, identical boot, WIREBIND mint→attach→`USE`→relay (the `FABRIC-3.6.md` Phase 4 acceptance sequence) still `1 2 + .` → `3`. |
| **B** | §3.2 trampoline idle loop + §3.3 `hermes_draining` + §3.5 parked-idle flag. Still inert (nothing marks work yet). | Same as A: nothing can reach the new trampoline branch yet, so behavior is identical. A synthetic self-test (kernel_main.c, the §XXVIII Stage 2 `SWITCH-TEST` pattern: write, capture, revert) switches into Artemis with one message queued and confirms she drained it on her own stack and parked-idle. | | **B** | §3.2 trampoline idle loop + §3.3 `hermes_draining` + §3.5 parked-idle flag. Still inert (nothing marks work yet). | Same as A: nothing can reach the new trampoline branch yet, so behavior is identical. A synthetic self-test (kernel_main.c, the §XXVIII Stage 2 `SWITCH-TEST` pattern: write, capture, revert) switches into **Hestia** (not Artemis, whom the Zuse attach flow `VM-EXEC`s; a parked VM refuses `VM-EXEC`) with **two** messages queued, the first calling a colon word so the in-word drain checkpoint is actually reached. The ordering witness `SB-SEQ` must equal 12 (one-at-a-time drain), not 21 (nested). She must end parked-idle at depth 0 with her context saved on her own stack, current VM back to Hera, and the ledger and `stadium_conserved()` restored. |
| **C** | §3.4 eligibility marked + pump skips participants (one commit). | The WIREBIND relay still works, now delivered by a real switch: `switch_count_cumulative` increments across the relay and the `~user` VM's own drain line appears. The ledger and `stadium_conserved()` hold (`fleet_conserved` is not evidence). No `native_stack_saved_sp` ever equals an address inside Hera's stack range (temporary probe). | | **C** | §3.4 eligibility marked + pump skips participants (one commit). | The WIREBIND relay still works, now delivered by a real switch: `switch_count_cumulative` increments across the relay and the `~user` VM's own drain line appears. The ledger and `stadium_conserved()` hold (`fleet_conserved` is not evidence). No `native_stack_saved_sp` ever equals an address inside Hera's stack range (temporary probe). |
| **D** | §3.5 kill of parked-idle + unregister in `capsule_vm_kill()`. | Birth, message, idle-park, `KILL` succeeds; the switch-signal slot count returns to its prior value. | | **D** | §3.5 kill of parked-idle + unregister in `capsule_vm_kill()`. | Birth, message, idle-park, `KILL` succeeds; the switch-signal slot count returns to its prior value. |
| **E** | §3.6 generic registration in `capsule_birth_baby()`, duplicate call sites removed. | The Tripod and WIREBIND VMs still register exactly once (slot count), and a `BIRTH`-ed VM appears as a participant. | | **E** | §3.6 generic registration in `capsule_birth_baby()`, duplicate call sites removed. | The Tripod and WIREBIND VMs still register exactly once (slot count), and a `BIRTH`-ed VM appears as a participant. |
@@ -327,3 +379,34 @@ explicit instruction.
Stage C). With the new gate, checkpoints inside that relayed payload now correctly **fail** Stage C). With the new gate, checkpoints inside that relayed payload now correctly **fail**
the ownership test (Hera owns the stack, not `rajames~user`). Before Stage A they passed on the ownership test (Hera owns the stack, not `rajames~user`). Before Stage A they passed on
the global depth count, which was the latent §1 hazard. the global depth count, which was the latent §1 hazard.
- [x] **Stage B — trampoline idle loop, one drain at a time, parked-idle (§3.2/§3.3/§3.5).**
2026-09-23. `include/vm.h`: `VM.hermes_draining`, `VM.parked_idle`. `kernel_hermes.c`:
`sk_hermes_drain_checkpoint()` refuses while `hermes_draining` is set, and sets and clears it
around its `vm_interpret()` (set only once a message is in hand; single exit). `switch.c`:
`sk_vm_switch_entry()` is now each participant's kernel-side idle loop. It drains one of its
own messages per pass, driven by the drain's return value, not a separate pending probe
(so a non-consuming drain can't spin with no checkpoint to break it). When nothing is
drained, it sets `parked_idle` and yields to `native_switch_back_to`; a failed yield panics.
Inert in normal operation: nothing marks work yet (Stage C), so no real boot enters it.
**Finding, fixed in this stage: amd64 first-entry stack misalignment.** The first probe boot
(`logs/20260923-142526/amd64/`, kept) took **#GP (vector 13)** the moment Hestia's trampoline
called into `vm_interpret()`. `sk_vm_switch_prime` (amd64 `switch.S`) built its frame at
`top − 56`, so the entry function ran with `rsp == top` (16-aligned). SysV needs
`rsp ≡ 8 (mod 16)` at entry, so every compiler-emitted aligned SSE spill faulted. Latent since
§XXVIII Stage 2, because the old placeholder never called anything that spilled. Fixed:
frame at `top − 64` with a zeroed pad slot. aarch64 and riscv64 return through the link
register and already land 16-aligned; no change there.
**Evidence, as named in §4:** a temporary probe (kernel_main.c, written, captured,
**reverted**; `git diff` afterward shows only `vm.h`, `kernel_hermes.c`, `switch.c` and amd64
`switch.S`) queued two messages to Hestia, the first calling a colon word, and switched
Hera→Hestia. `STAGE-B PROBE: PASS` on all three: `logs/20260923-143121/amd64/`,
`-143435/aarch64/`, `-143838/riscv64/`. Sends ok, switch ok, 0 pending, `parked_idle=1`,
depth 0, context saved on her own stack, current VM back to Hera, **`SB-SEQ = 12`** (one at a
time; a nested drain gives 21), ledger `held` restored, `stadium_conserved(hera)` true. The
probe's printed line reads `seq=ld …` because this kernel's freestanding `snprintf` has no
`%lld` and shifted the arguments after it; the PASS test compares values directly and is
unaffected. Formal acceptance with the probe gone, coupled images reset before each:
`logs/20260923-144234/amd64/`, `-144701/aarch64/`, `-145224/riscv64/`. All reach
`[zuse@Hera] ok>` with zero `UNKNOWN WORD`/failures/faults, the `dict_hash` set identical
across ISAs and unchanged from Stage A, and the full mint → WIREBIND → `USE` → `1 2 + .` →
`3` relay with `stadium_conserved(self)=true`.
+1 -1
View File
@@ -1,5 +1,5 @@
# Capsule Block Manifest — Auto-generated # Capsule Block Manifest — Auto-generated
<!-- Generated by mkcapsule --manifest 2026-09-23T18:12:28Z --> <!-- Generated by mkcapsule --manifest 2026-09-23T18:51:49Z -->
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. --> <!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
<!-- Hand-written justifications and immutability notes live --> <!-- Hand-written justifications and immutability notes live -->
<!-- in MANIFEST.md alongside this auto-generated index. --> <!-- in MANIFEST.md alongside this auto-generated index. -->
+13
View File
@@ -677,6 +677,19 @@ typedef struct VM
* count across all VMs stops meaning "this VM is * count across all VMs stops meaning "this VM is
* at its own outermost frame." 0 = not * at its own outermost frame." 0 = not
* interpreting; 1 = outermost frame. */ * interpreting; 1 = outermost frame. */
int hermes_draining; /**< FABRIC-3.8.md §3.3 (Stage B): nonzero while
* sk_hermes_drain_checkpoint() is interpreting one
* of this VM's own messages. No drain site drains
* again while it is set -- a payload running at the
* VM's depth 1 would otherwise let its own in-word
* checkpoint drain a SECOND message inside the
* first one's execution. */
int parked_idle; /**< FABRIC-3.8.md §3.5 (Stage B): nonzero only while
* this VM is parked in its switch trampoline's idle
* branch -- nothing borrowed on its stack, no work
* in progress. Distinct from VM_STATE_SWITCHED_OUT,
* which is also true of a VM parked mid-work.
* Consumed by the kill path in Stage D. */
/** @} */ /** @} */
/** @name Stadium Identity (item 4.2, FABRIC-0.md §25.5) /** @name Stadium Identity (item 4.2, FABRIC-0.md §25.5)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+10 -1
View File
@@ -68,7 +68,16 @@ sk_vm_switch_to:
.global sk_vm_switch_prime .global sk_vm_switch_prime
sk_vm_switch_prime: sk_vm_switch_prime:
mov rax, rdi mov rax, rdi
sub rax, 56 /* FABRIC-3.8.md Stage B finding (2026-09-23): was `sub rax, 56`, which
* entered `entry` with rsp == stack_top (16-byte aligned). SysV requires
* rsp == 8 (mod 16) at function entry (as if a `call` had just pushed a
* return address); off by 8, any compiler-emitted aligned SSE spill in
* the entry function's callees #GPs. Latent through Stage 2 because the
* placeholder trampoline never called anything that spilled; found live
* the moment the trampoline started draining (vm_interpret()). 64 leaves
* a zeroed pad slot above the entry address so entry sees rsp = top-8. */
sub rax, 64
mov qword ptr [rax+56], 0 /* pad: the fake caller's slot, never used */
mov qword ptr [rax+0], 0 /* r15 */ mov qword ptr [rax+0], 0 /* r15 */
mov qword ptr [rax+8], 0 /* r14 */ mov qword ptr [rax+8], 0 /* r14 */
mov qword ptr [rax+16], 0 /* r13 */ mov qword ptr [rax+16], 0 /* r13 */
+10
View File
@@ -589,6 +589,11 @@ int sk_hermes_drain_checkpoint(VM *vm) {
if (!vm) return -1; if (!vm) return -1;
if (sk_hermes_pending_total == 0) return 0; /* fast path -- see this if (sk_hermes_pending_total == 0) return 0; /* fast path -- see this
* counter's own doc comment */ * counter's own doc comment */
/* FABRIC-3.8.md §3.3 (Stage B): one drain at a time per VM. A payload
* this function is already interpreting for `vm` runs at vm's own
* depth 1, so its in-word checkpoint (vm_core.c) would otherwise drain
* a SECOND message inside the first one's execution. */
if (vm->hermes_draining) return 0;
self = vm->stadium_vm_id; self = vm->stadium_vm_id;
msg = sk_hermes_pending_peek(self); msg = sk_hermes_pending_peek(self);
@@ -651,7 +656,12 @@ int sk_hermes_drain_checkpoint(VM *vm) {
vm->mode = MODE_INTERPRET; /* a payload is never compiled into whatever vm->mode = MODE_INTERPRET; /* a payload is never compiled into whatever
* the enclosing context was mid-defining */ * the enclosing context was mid-defining */
vm->hermes_draining = 1; /* FABRIC-3.8.md §3.3 -- set only once a message
* is actually in hand (every earlier return
* above leaves it untouched), cleared on the
* single path out below */
vm_interpret(vm, (const char *)msg->payload_addr); vm_interpret(vm, (const char *)msg->payload_addr);
vm->hermes_draining = 0;
drain_error = vm->error; drain_error = vm->error;
memcpy(vm->input_buffer, saved_input, sizeof(saved_input)); memcpy(vm->input_buffer, saved_input, sizeof(saved_input));
+39 -1
View File
@@ -30,6 +30,8 @@
#include "starkernel/vm/switch.h" #include "starkernel/vm/switch.h"
#include "vm.h" #include "vm.h"
#include "starkernel/capsule_birth.h" /* capsule_vm_set_state */ #include "starkernel/capsule_birth.h" /* capsule_vm_set_state */
#include "starkernel/vm/kernel_hermes.h" /* FABRIC-3.8.md §3.2 -- sk_hermes_drain_checkpoint() */
#include "starkernel/hal/hal.h" /* sk_hal_panic() */
#include <stdint.h> #include <stdint.h>
/* Arch-specific (switch.S in each arch dir). sk_vm_switch_prime() builds /* Arch-specific (switch.S in each arch dir). sk_vm_switch_prime() builds
@@ -97,10 +99,46 @@ void sk_vm_switch_set_current(VM *vm) { g_switch_current_vm = vm; }
* itself, independent of vm_interpret()'s own attribution bookkeeping. * itself, independent of vm_interpret()'s own attribution bookkeeping.
* This placeholder's behavior is therefore unchanged from Stage 2. * This placeholder's behavior is therefore unchanged from Stage 2.
*/ */
/*
* FABRIC-3.8.md §3.2 (Stage B, 2026-09-23): the placeholder above is
* superseded -- this trampoline is now every switch participant's own
* kernel-side idle loop, the base of its native stack. Entirely OS-driven:
* nothing a VM executes decides to give up control (Captain Bob: "no
* special words saying I'm giving up control... strictly all operating
* system and strictly all compudynamically driven"). Each pass:
*
* - Drain one of this VM's own kernel-Hermes messages, in its own
* context, on its own stack (sk_hermes_drain_checkpoint() at depth 0
* here, so the payload runs at this VM's depth 1 and its own word
* checkpoints pass the Stage A ownership gate).
* - If nothing was drained, this VM has no work: mark it parked-idle
* (§3.5 -- nothing borrowed on its stack, nothing in progress) and
* yield back to whoever switched in.
*
* Driven off the drain's return value, never a separate pending-count
* probe: any path where a drain returns without consuming a message would
* otherwise spin here forever with no word checkpoint to break it --
* indistinguishable from a switch storm. A failed yield is a kernel
* invariant violation, not something to retry, so it panics loudly.
*
* Deliberately NOT here yet (FABRIC-3.8.md §3.11, a Stage C design
* question): this point between messages is also a clean switch point
* (own stack, depth 0), and the compudynamic switch decision should be
* consulted here too, so a VM with a queue of checkpoint-free payloads
* cannot hold the CPU indefinitely.
*/
void sk_vm_switch_entry(void) { void sk_vm_switch_entry(void) {
VM *self = g_switch_entry_vm; VM *self = g_switch_entry_vm;
for (;;) { for (;;) {
sk_vm_context_switch(self, self->native_switch_back_to); if (sk_hermes_drain_checkpoint(self) != 0) {
continue; /* consumed one message (1 = ok, -1 = payload erred) */
}
self->parked_idle = 1;
if (!self->native_switch_back_to ||
sk_vm_context_switch(self, self->native_switch_back_to) != 0) {
sk_hal_panic("switch trampoline: idle yield failed");
}
self->parked_idle = 0; /* resumed: someone switched back in */
} }
} }