Files
LithosAnanake/FABRIC-3.10.8.md
T
Robert Allan JamesandClaude Opus 5.5 02c4c7e620 Stage MC-0d: Stadium partitioned by quota slot, one lock per slot (FABRIC-3.10.8 §3.5)
- Ruling (Captain Bob): per-slot locks, not "own path needs no lock" --
  a lock only on the cross-slot side can't exclude an unlocked owner
  (Hermes release evicting a sender's cell while the sender admits).
  FABRIC-3.10.0 §3.6 amended.
- StadiumVMQuota.lock covers the slot's lists, reservoir, consumed, word
  row and its cells. Word dispatch holds only its own slot lock for the
  whole dispatch (*_in_slot forms); peek-then-pull is one step.
- g_lock_stadium: grant slot allocation (new slot published with release
  in_use) and the shared block table (ruled: the one shared-lock Stadium
  path). Taken before a slot lock.
- Bitmap bytes set/cleared atomically (8 cells, possibly 8 slots).
- Cross-slot writers moved under the owner's lock: Hermes decay
  (stadium_cell_decay), Hermes release ledger (stadium_evict_credit),
  session pin (stadium_cell_set_pin). stadium_conserved() one hold.
- Word and block cell lookups check the cell is still this slot's.
- Probe (removed), identical on all three ISAs: 10,746,599 of 10,746,616
  word dispatches took exactly one lock (own slot), the 17 others all row
  growths; every live VM conserved; Hermes audit failures 0.
- Findings reported: births/USE run a VM's words while another VM is the
  core's current VM (MC-3 claim must cover it); quota slots never
  released; owner admission can evict a live message cell (latent, 0).
- Standard sequence on amd64/aarch64/riscv64: dict_hash set and G
  byte-identical, MINT/WIREBIND/USE ok, conservation true, 0 idle growth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 22:40:33 -04:00

32 KiB
Raw Blame History

FABRIC-3.10.8.md — Multi-core: execution plan and log

Status: IN PROGRESS. Design and rulings are in FABRIC-3.10.0.md (ruled 2026-09-25, §7). Multi-core runs ahead of O-D (MC-R6 as ruled); O-D relaunches afterwards with row path A.

Each stage:

  • needs Captain Bob's explicit go;
  • gets its own commit;
  • gets its own three-architecture acceptance (clean qemu; amd64, then aarch64, then riscv64; foreground; visible framebuffer; one at a time; coupled images reset before each).

Evidence is named before each stage runs.

The standing acceptance sequence, unchanged from FABRIC-3.9.8.md:

  • a boot to [zuse@Hera] ok> with zero UNKNOWN WORD, faults and frame corruption;
  • the dict_hash set identical across ISAs;
  • the Stage G interleave (A/B), byte-identical across ISAs while there is one worker core; by counts (A = 40, B = 40) once there are more (MC-R4);
  • the O-A checks (observer precedence, console responsive under load);
  • mint → WIREBIND → USE → 1 2 + . → 3;
  • 0 bytes of idle log growth over 20 s.

No DoE execution until everything is closed (Captain Bob, 2026-09-25: "until we close everything, do not do ant DoE eecution."). MC-4's campaigns wait for that too. Stage evidence is the standard sequence plus probes, never a campaign.

Never make clean an ISA while its QEMU runs: it deletes that QEMU's QMP and serial sockets (found 2026-09-25).

1. What MC-0 must cover (traced 2026-09-25)

Everything that would break if two cores ran kernel or VM code at the same moment.

Kernel subsystems (src/starkernel/), one lock each (MC-R2), except where noted:

Subsystem Shared state Treatment
kmalloc, pmm, vmm free lists, bitmaps, page tables heap lock; PMM lock; VMM lock
kernel-Hermes arena directory and segments, queue table, channel table, ledger, event counters one lock
Stadium reservoirs, cells, quotas, word rows per-dispatch path partitioned by quota slot (a VM's own row and cells need no lock while it runs on its one core); reservoir arithmetic atomic; cross-slot operations (admission evicting another VM's cell, grants, audits) take the Stadium lock
VM registry, birth, kill, reap registry list, names registry lock; KILL refuses a VM running on another core
switch-signal table slots, stamps, decision-due flag compare-and-swap for claim/release (MC-3); lock for register/unregister
current VM, frame stack g_switch_current_vm and friends move to the per-core block
console UART, framebuffer VT100 state, line prefix state one line lock
log region, DoE region, block subsystem rings, block chain, caches a lock each
heartbeat tick counters per-core counters (MC-3); one core until then

Shared VM code (src/, vendored; runs inside every VM):

File Global Treatment
dictionary_management.c sf_fc_list/sf_fc_count/sf_fc_cap/sf_cached_latest: one dictionary lookup index for all VMs, rebuilt whenever a different VM's dictionary is searched move into VM (per-VM index). Also stops the rebuild thrash on every switch today.
word_source/starforth_words.c g_prng_state: one PRNG stream for all VMs (accepted at Stage H) an atomic step, so the stream stays one stream (order is nondeterministic across cores anyway, MC-R4)
physics_runtime.c g_runtime inventory in MC-0; per-VM or atomic
word_source/dictionary_manipulation_words.c state_variable (STATE) inventory in MC-0; per-VM
physics_hotwords_cache.c g_hotwords_cache retired under __STARKERNEL__ (item 4.1); confirm unused
word_source/ttf_words.c font and raster cache core 0 only (Hestia's framebuffer)

2. Staged plan

Stage Change Evidence (named in advance)
MC-0a The per-core block, on one core. A per-core structure reached through each ISA's per-CPU register (GS base / TPIDR_EL1 / tp), set up for core 0 at boot. g_switch_current_vm moves into it. Standard sequence unchanged. A probe reads the per-core pointer from a normal kernel path and from an interrupt handler, and gets core 0's block both times on all three ISAs.
MC-0b Real spinlocks. A kernel spinlock primitive (atomic exchange, interrupts masked while held on the owning core). sf_mutex_* stop being no-ops. The §1 kernel locks are added. Standard sequence unchanged. A probe counts every lock's acquisitions and releases through a boot and the standard sequence: counts balance, and no lock is held across a context switch or on return to the idle point.
MC-0b2 The block subsystem's lock (split from MC-0b, Captain Bob 2026-09-25). The vendored block_subsystem.c calls its own public functions in cycles (blk_set_meta → migration check → relocate → blk_set_meta), so it is refactored into locking public functions over unlocked internals, under __STARKERNEL__. Standard sequence unchanged. The MC-0b probe, extended to the block lock: counts balance, 0 recursions.
MC-0c Shared VM code made per-VM or atomic (§1, second table). The dictionary lookup index moves into VM. The dict_hash set unchanged (the index is not hashed). A probe: two VMs searched alternately never rebuild each other's index. Standard sequence.
MC-0d Stadium's dispatch path partitioned (§1). Standard sequence; conservation holds (stadium_conserved() on every VM, the ledger audit). A probe proves the dispatch path takes no shared lock.
MC-1 Bring up the extra cores and park them halted, per ISA: amd64 (MADT, INIT-SIPI-SIPI, a position-independent trampoline below 1 MiB), aarch64 (DTB /cpus, PSCI CPU_ON), riscv64 (DTB harts, SBI HSM hart_start). QEMU gets -smp 2. The boot reports each core online with its own per-core block; the extra cores sit halted (0 idle growth, no spin); the standard sequence unchanged.
MC-2 Observers get core-0 affinity. With one worker core, behaviour identical to today. A probe shows no observer ever claimed off core 0.
MC-3 Worker cores claim through the shared decision: running_on with compare-and-swap, per-core idle contexts, per-core beats, wake by inter-processor interrupt. Two workers genuinely overlapping in time on two cores. A probe counts double claims: 0. Conservation holds. G by counts. Idle worker cores halted (0 idle growth).
MC-4 DoE with cores as a factor (after O-D): per-ISA campaigns at 1, 2 and 4 cores (MC-R5). Per ISA and core count: 18 MU-TRIAL markers, fail=0, MU-CAMPAIGN: complete.

Stop conditions (re-plan rather than push on):

  • dict_hash diverges across ISAs;
  • conservation goes false;
  • a lock held across a switch, or an unbalanced lock count;
  • a double claim;
  • an idle core spinning.

3. Execution log

3.1 MC-0a — the per-core block, on one core (2026-09-25)

Change.

  • include/starkernel/percpu.h (new): sk_percpu_t { self, core_id, current_vm }, with self first.
    • sk_percpu() reads this core's block: movq %gs:0 on amd64, mrs tpidr_el1 on aarch64, mv tp on riscv64.
    • sk_percpu_install() writes the register: IA32_GS_BASE via wrmsr, TPIDR_EL1, or tp.
  • src/starkernel/hal/percpu.c (new): the boot core's block is one static object, not a table sized to a core count. Secondary cores get theirs as they come online (MC-1).
  • kernel_main.c: sk_percpu_boot_init() runs straight after arch_early_init(), because on amd64 that call reloads GS, which would reset the base.
  • switch.c: the global g_switch_current_vm is gone.
    • sk_vm_switch_current_vm(), the setter, the idle point and sk_vm_context_switch() use sk_percpu()->current_vm.
    • After resume, the context switch re-reads the block rather than trusting a pointer cached before the switch.

Probe (temporary; removed before commit).

  • kernel_main.c checked, from the normal path after the Tripod came up, that:
    • sk_percpu()->self == sk_percpu();
    • core_id == 0;
    • current_vm is Hera;
    • sk_vm_switch_current_vm() agrees.
  • heartbeat_tick() ran the same self-check from interrupt context on every beat.
ISA log normal path core_id current_vm is Hera switch.c agrees interrupt path ok / bad
amd64 logs/20260925-130335 ok 0 1 1 889 / 0
aarch64 logs/20260925-130905 ok 0 1 1 1877 / 0
riscv64 logs/20260925-131337 ok 0 1 1 1625 / 0

Standard sequence (clean build, with the probe reverted; the tree was checked byte-for-byte against the saved stage diff).

ISA log O-A (G, PREC, LOAD) M7.1a hash MINT / WIREBIND / USE / 1 2 + . idle growth UNKNOWN / faults
amd64 logs/20260925-131805 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
aarch64 logs/20260925-133049 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
riscv64 logs/20260925-134357 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
  • The dict_hash values (Hera 0x00340a8b2c842eb7, Hestia 0x78bf28439e4a643f, Artemis 0x66886d174d5f3b5a) are identical across ISAs and unchanged from O-C.
  • The ledger line matches across ISAs: held=2048, pulled=258048, returned=255263, consumed=737, conserved=true.
  • No stop condition was hit. MC-0a done.

3.2 MC-0b — real spinlocks (2026-09-25)

The primitive (include/starkernel/spinlock.h, src/starkernel/hal/spinlock.c, new).

  • One lock word per lock: 0 = free, core_id + 1 = held.

    • Taken by compare-and-swap. The atomics are inline on every ISA (nm -u is empty on all three ELFs).
    • Interrupts stay masked on the owning core while it holds any lock.
  • The mask state lives in the per-core block, not in the lock (lock_depth, saved_irq in sk_percpu_t):

    • the first lock taken saves the interrupt state and masks;
    • the last lock released restores it;
    • so release order doesn't matter.
  • The rule: no lock crosses a switch or reaches the idle point. Each of these sites checks it and counts any violation:

    • sk_vm_context_switch(), when leaving and on resume;
    • sk_vm_idle_point().
  • Failures are counted and logged at the idle point (LOCK: lines, LOG_ERROR):

    • recursion (a core takes a lock it already holds): let through, so the core doesn't hang, and counted. It is never legitimate, because on one core "the owner is me" can't tell a nested call from a second VM on the same core;
    • release of a lock not held;
    • a lock held at a switch or idle point;
    • a VM mutex destroyed while held.
  • The locks are no-ops until the boot core's block is installed (sk_percpu_ready). This was found during the probe runs:

    • riscv64 is built monolithic, and its arch_early_init() prints through the console before sk_percpu_boot_init();
    • the first lock therefore read tp while it still held the firmware's value, and the load hit OpenSBI memory (stval = tp+8, a firmware load access fault);
    • early boot is one core with interrupts off, so skipping the locks there is correct.

    That crashed run's log was deleted as flawed.

  • sf_mutex_* (the vendored VM's dict_lock and tuning_lock, shim.c) now use the same lock word. They are no longer no-ops.

The kernel locks, in acquisition order (spinlock.h; MC-3's deadlock freedom rests on this order; checked by reading the code, not yet by a counted check; MC-0b2 inserts block between log and console, §3.3):

  1. hermes
  2. switch
  3. registry
  4. doe
  5. log
  6. console
  7. heap
  8. vmm
  9. pmm

Stadium's lock comes with MC-0d, below Hermes. Stadium today runs inside dict_lock (vm_core.c's heat bump calls stadium_word_dispatch()); MC-0d's order must account for that.

Lock Covers Shape
heap kmalloc/kfree, stats, censuses kmalloc() delegates to kmalloc_aligned(), and only the latter locks
vmm page-table walks *_locked bodies. The map-conflict print moved outside the lock (the console lock sits above it). Takes pmm for table pages
pmm bitmap pmm_alloc_page()/pmm_free_page() delegate to the contiguous forms
console output path and line state (name, line start, pending line close, the framebuffer terminal) A string goes out under one hold. Unlocked on purpose: console_get_vm_name() (the prefix provider calls it under the lock), the boot-time console_fb_init()/console_fb_enable_ttf() (they log), and input
log, doe control-block read-modify-write plus the slot write *_locked bodies
registry node list, entry copies, single-field writes Never held while calling out. KILL and reap take it only for their reads and final writes. A birth fills its embryo unlocked (private until LIVE). kill_all_nonmama (cold-reset path) walks unlocked
switch slot table, stamps, decision-due flag, counters Never held while calling out. The decision, the Stage H floor and the reap sweep copy one slot under the lock (slot_copy()) and judge it outside
hermes arena, ledger, channels, queues, event counters Public functions lock over *_locked statics. Never held across vm_interpret_kind() (the drain) or a policy word. The switch notifications after a send go out once the lock is dropped (the floor reads Hermes queues back). Publish and event walk members one copied member at a time. sk_hermes_event()'s static payload moved onto the stack

The block subsystem is split out as MC-0b2 (Captain Bob, 2026-09-25), because its internal call cycles need a locked/unlocked refactor of vendored code. See §2.

Probe (temporary, removed before commit). A MC0B-PROBE word in Hera's and every child's vocabulary printed every counter, snapshotted before printing:

  • each lock's acquisitions and releases, and its owner;
  • the VM mutexes (the core total minus the kernel locks);
  • the core-0 total and its depth;
  • the number of switch/idle checks and violations;
  • recursions, bad releases and destroyed-while-held.

It ran at Hera's prompt (after O-A's G, PREC and LOAD), and again on rajames after MINT, WIREBIND, USE and 1 2 + .. The table shows the final (rajames) reading:

ISA log total acquired = released depth switch/idle checks held at switch recursions / bad releases / destroyed held
amd64 logs/20260925-150735 1,503,458,645 0 51,137,748 0 0 / 0 / 0
aarch64 logs/20260925-152109 235,854,532 0 7,565,646 0 0 / 0 / 0
riscv64 logs/20260925-155048 566,894,806 0 19,029,507 0 0 / 0 / 0
  • Every lock balances individually on every ISA, at both readings.
  • The amd64 and aarch64 probe runs predate the sk_percpu_ready guard. That guard only changes behaviour before sk_percpu_boot_init(), and the acceptance runs below are on the final code.
  • The riscv64 vblk_io timed out count, watched because known defect 3 sits in code that now runs with interrupts masked, is 2 in the probe run, the same as MC-0a's acceptance run. It is 0 in this stage's acceptance run.

Finding, for MC-3. Hera's input-poll idle loop runs the switch decision without pause, and every pass takes the registry, switch and Hermes locks once per slot. That is the hundreds of millions of acquisitions above (amd64: registry 379 M, switch 858 M, Hermes 266 M). It is harmless on one core. With several cores, core 0 spinning through these same locks would contend with every worker's claim. MC-3 should settle how that loop waits before cores share these locks.

Standard sequence (clean build, with the probe reverted; checked against the saved stage diff, which differs only by the guard added after it).

ISA log O-A (G, PREC, LOAD) M7.1a hash MINT / WIREBIND / USE / 1 2 + . idle growth UNKNOWN / faults
amd64 logs/20260925-160434 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
aarch64 logs/20260925-161720 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
riscv64 logs/20260925-163024 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
  • The G string is byte-identical to MC-0a on all three ISAs, so masking interrupts under the locks didn't move the interleave.
  • The dict_hash set and the ledger line are unchanged from MC-0a.
  • No stop condition was hit. MC-0b done (the block subsystem follows as MC-0b2).

3.3 MC-0b2 — the block subsystem's lock (2026-09-25)

Change. src/block_subsystem.c (vendored) is split into bodies and wrappers:

  • Every public function's body is now a static *_locked function.

  • Bodies call each other only through the _locked forms. That breaks the internal cycles:

    • relocate → get buffer / update / mark free;
    • meta relocate → relocate;
    • migration scan → meta relocate;
    • firsttouch → get meta / set meta / mark allocated;
    • the six meta accessors.
  • A block of thin public wrappers at the end of the file takes the lock around each body.

    • Under __STARKERNEL__ that lock is g_lock_block.
    • The hosted build has no lock, and the vendored hosted make still builds clean.
  • The lock covers the device chain, every slot's cache and BAM, the relocation table, and the device I/O issued on their behalf.

  • Its place in the order is below log and doe (both regions write through the block subsystem) and above console (block logs), heap (calloc/free), vmm and pmm:

    1. hermes
    2. switch
    3. registry
    4. doe
    5. log
    6. block
    7. console
    8. heap
    9. vmm
    10. pmm

    Stadium (MC-0d) sits above block, because STADIUM_BEHAVIOUR_MIGRATE calls blk_flush().

Not covered, and not coverable by this lock: blk_get_buffer() returns a pointer into RAM, a ramdrive, or a device cache slot. The caller (BLOCK/BUFFER, Artemis) reads and writes through that pointer after the lock is released. On one core nothing can evict the slot in between; with worker cores (MC-3) another core's block access can. Block-buffer ownership across cores is an open item for MC-3.

The masking risk, measured. virtio_blk.c records (2026-09-09) that masking interrupts around vblk_io() once made missed completions worse. All block I/O now runs with interrupts masked (the MC-0b primitive), so the probe runs counted vblk_io timed out on every ISA:

Where amd64 aarch64 riscv64
This stage's probe runs 0 0 4
This stage's acceptance runs 0 0 2

The riscv64 counts sit inside the pre-lock baseline: the 24 most recent riscv64 logs, from 2026-09-24 (no locks at all) through this stage, range from 0 to 6, with 4, 5 and 6 all seen before MC-0b. No regression is visible at this sample size. The masking stays as MC-0b ruled.

Probe (temporary, removed before commit). This is MC-0b's MC0B-PROBE word, extended with the block lock. Same sequence: O-A, then Hera's prompt, then MINT, WIREBIND, USE and 1 2 + ., then rajames. The table shows the final (rajames) reading:

ISA log block acquired = released total acquired = released depth switch/idle checks held at switch recursions / bad releases / destroyed held
amd64 logs/20260925-173901 2,379 1,387,861,111 0 46,795,001 0 0 / 0 / 0
aarch64 logs/20260925-175215 2,057 247,761,816 0 8,014,413 0 0 / 0 / 0
riscv64 logs/20260925-180546 2,300 555,650,314 0 18,447,619 0 0 / 0 / 0

Every lock balances individually on every ISA, at both readings. 0 recursions means no public-to-public call inside block_subsystem.c was missed.

Standard sequence (clean build, with the probe reverted; the tree is byte-identical to the saved stage diff).

ISA log O-A (G, PREC, LOAD) M7.1a hash MINT / WIREBIND / USE / 1 2 + . idle growth UNKNOWN / faults
amd64 logs/20260925-181908 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
aarch64 logs/20260925-183151 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
riscv64 logs/20260925-184454 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
  • The G string is byte-identical to MC-0b and MC-0a on all three ISAs.
  • The dict_hash set and the ledger line are unchanged.
  • No stop condition was hit. MC-0b2 done.

3.4 MC-0c — shared VM code made per-VM or atomic (2026-09-25)

Scope: §1's second table, plus five further items found by a sweep of every file-scope and function-local mutable variable in the vendored VM sources the kernel compiles. Captain Bob ruled all of it into this stage (2026-09-25).

§1's table:

Item Finding Change
dictionary lookup index (sf_fc_list/count/cap, sf_cached_latest) One index for the whole system, holding whichever VM's dictionary was searched last: every switch between VMs that look words up rebuilt it. dict_reorganize_buckets_by_heat() reordered that shared index, which could be another VM's. Per VM: struct SfFcIndex (dictionary_heat_optimization.h), vm->fc_index, allocated on first lookup and freed by both vm_cleanup()s. An allocation failure is counted and logged, and the lookup walks the dictionary newest-first instead (still correct). The heat-aware lookup and the heat reorganization use the VM's own index. The index's unused global reset is gone, as are the benchmark word's stale externs. The 256 buckets are one per possible first byte (the whole domain, not a capacity).
g_prng_state (starforth_words.c) One stream for all VMs, as accepted at Stage H; plain read-modify-write. Still one stream. The step is a compare-and-swap loop, and SEED is a single atomic store.
g_runtime (physics_runtime.c) The kernel build (STARFORTH_MINIMAL) compiles stubs; g_runtime exists only hosted. None.
state_variable (dictionary_manipulation_words.c) Written by INTERPRET, never read. STATE is vm->mode, already per VM. Removed (dead).
g_hotwords_cache (physics_hotwords_cache.c) Defined, never referenced; the live cache is per VM (vm->hotwords_cache). None.
ttf_words.c font and raster cache Hestia's framebuffer. None; core 0 only, per plan.

Found by the sweep:

Item Finding Change
heartbeat_export.c, three function-local statics heartbeat_capture_tick_snapshot() (every beat, vm_runtime.c) took each VM's deltas (beats, words executed, ANOVA early exits) against whichever VM was sampled last. Wrong across VMs on one core today. The snapshot feeds doe_log_tick_row() only, not any decision. The previous values move into the VM's HeartbeatState (export_last_*).
TOP-WORDS (starforth_words.c) A static DictEntry *[1000]: shared by every VM, and silently cut at 1000 words. Sized to the heated words and allocated per call. A refusal sets vm->error and is logged.
inference_outputs_to_string() Returned a static 256-byte buffer. No callers. Takes a caller's buffer.
Failure counters: word-id refusals, transition-grow failures, WORD too long Plain ++. Atomic increments and loads.
block_subsystem.c crc64 table Lazy init with no publish ordering; compute_crc64() also runs outside the block lock. Published with release/acquire.

Left as system-wide by design: system_running and forth_79_standard (system_words.c, system flags). profiler_state and vm_debug's g_vm are hosted/diagnostic; vm_debug is stubbed in the kernel.

The vendored hosted make builds clean.

Probe (temporary, removed before commit). An MC0C-PROBE word in Hera's and every child's vocabulary:

  • listed every VM's index, and how many times it has ever been rebuilt;
  • then looked up DUP, SWAP, EMIT, BLOCK and USE in two VMs alternately, 1,000 rounds each (5,000 lookups per VM);
  • checked that every result belongs to the dictionary of the VM that was searched.

It ran at Hera's prompt after O-A, and again on rajames after the standard sequence. The results are identical on all three ISAs:

ISA log every VM has its own index Hera rebuilds (her own defines/forgets) every other VM's rebuilds alternating 2×5000: rebuilds / found / wrong-VM alloc failures
amd64 logs/20260925-194705 yes (7 VMs) 85 2 0+0 / 10000 / 0 0
aarch64 logs/20260925-200019 yes (7 VMs) 85 2 0+0 / 10000 / 0 0
riscv64 logs/20260925-201350 yes (7 VMs) 85 2 0+0 / 10000 / 0 0

These runs' dict_hash values differ from the table below only because the probe adds a word.

Standard sequence (clean build, with the probe reverted; the tree is byte-identical to the saved stage diff).

ISA log O-A (G, PREC, LOAD) M7.1a hash MINT / WIREBIND / USE / 1 2 + . idle growth UNKNOWN / faults
amd64 logs/20260925-202657 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
aarch64 logs/20260925-203935 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
riscv64 logs/20260925-205238 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
  • The dict_hash set is unchanged, as the plan's evidence requires (the index is not hashed): Hera 0x00340a8b2c842eb7, Hestia 0x78bf28439e4a643f, Artemis 0x66886d174d5f3b5a.
  • The G string is byte-identical to MC-0b2, MC-0b and MC-0a on all three ISAs.
  • The ledger line is unchanged.
  • No stop condition was hit. MC-0c done.

3.5 MC-0d — Stadium partitioned by quota slot (2026-09-25)

Ruling (Captain Bob, 2026-09-25): one lock per quota slot. FABRIC-3.10.0 §3.6 said a VM's own Stadium row and cells need no lock, with a Stadium lock only for cross-slot operations. That can't be made sound: a lock taken only on the cross-slot side can't exclude an owner that takes none. Concrete race:

  • a receiver's drain runs sk_hermes_release() → stadium_evict() on the sender's message cell, which rewrites the sender's free list, resident list and reservoir;
  • meanwhile the sender, running on another core, is popping that same free list in stadium_admit().

stadium_grant_quota() splits a donor's free list the same way. The amendment is recorded at FABRIC-3.10.0 §3.6.

Second ruling: the block table takes the shared Stadium lock. stadium_blocks.c keeps one open-addressed table for every VM, keyed by (slot, LBN). stadium_block_dispatch() therefore takes g_lock_stadium. It is the one shared-lock Stadium path, recorded for MC-3 to measure; block access is rare next to word dispatch.

Change.

  • The slot lock.

    • StadiumVMQuota gains lock, a spinlock.h lock word.
    • It covers the slot's free list, resident list, reservoir, consumed, word row and statistics, and the slot's cells: their headers, owner entries and resident-list links.
    • Every public Stadium function locks the slot it touches. A cell operation locks the cell's owning slot (lock_cell_owner() re-reads the owner once locked).
  • Word dispatch (stadium_word_dispatch()) holds its own VM's slot lock for the whole dispatch: row, cell, cooling, pull, and admission including its eviction fallback. It uses the new *_in_slot forms, so it takes no other lock (a row growth's kmalloc excepted).

    • The reservoir peek-then-pull is now one step under that lock.
  • Global part: g_lock_stadium.

    • Quota-slot allocation in grants. Each new slot is filled first, then published with a release store of in_use; quota_slot_for_vm() reads in_use with acquire.
    • The shared block table.
    • It is taken before a slot lock, never after.
  • The residency bitmap: a byte covers 8 cells, which can belong to different slots, so set and clear are atomic OR/AND.

  • Cross-slot writers moved into Stadium, under the owner's lock:

    • kernel-Hermes's decay: stadium_cell_decay(), the same q48_mul arithmetic, consumed recorded in the same step;
    • Hermes's release ledger: stadium_evict_credit() returns the heat actually credited, read in the same step as the eviction, where it used to be read from the cell beforehand;
    • session_set_pinned()'s pin bit: stadium_cell_set_pin().
  • stadium_conserved() reads its three terms under one hold, so the sum is of one instant.

  • Owner check added to both dispatch paths' cell lookups: the cell must still be this slot's.

    • Without it, a word or block cell evicted to this slot's free list could move to another VM in a grant and be admitted there with a matching identity. Word ids and LBNs are shared across VMs, so the old row would then heat the other VM's cell.
    • This is latent on one core.
  • Unpartitioned reads that remain: sk_hermes_scan_held() (reads message cells under the Hermes lock), session.c's bounds check, and kernel_main.c's decay self-test. None writes.

  • Lock order (spinlock.h):

    1. hermes
    2. switch
    3. registry
    4. stadium
    5. a slot lock
    6. doe
    7. log
    8. block
    9. console
    10. heap
    11. vmm
    12. pmm

    A VM's dict_lock sits above Stadium: vm_core.c's heat bump dispatches the word under it. Nothing in the block path calls into Stadium, so MIGRATE's blk_flush() edge doesn't close a cycle.

Probe (temporary, removed before commit). It counted:

  • the per-core lock acquisitions made during each stadium_word_dispatch();
  • slot-lock acquisitions made for a slot other than the running VM's own;
  • admission-fallback evictions of message (DELIVER) cells;
  • quota slots in use.

An MC0D-PROBE word in every VM printed these, with stadium_conserved() on every live VM and the Hermes audit-failure count. The results are identical on all three ISAs, count for count:

ISA log word dispatches exactly one lock (own slot) more than one …all at a row growth conserved Hermes audit failures DELIVER-cell fallback evictions
amd64 logs/20260925-212159 10,746,616 10,746,599 17 17 7/7 0 0
aarch64 logs/20260925-213527 10,746,616 10,746,599 17 17 7/7 0 0
riscv64 logs/20260925-214855 10,746,616 10,746,599 17 17 7/7 0 0

The table shows the final (rajames) reading. At Hera's prompt, before MINT, the readings were 9,687,103 dispatches with 14 exceptions (all row growths), 5/5 conserved, and 268 cross-slot slot locks.

Findings, reported and not fixed:

  1. The running VM isn't always the core's current VM. Cross-slot slot locks went from 268 to 1,059,738 across MINT, WIREBIND, USE and 1 2 + ., the same growth as word dispatches in that window.
    • A birth runs the child's capsule on the parent's stack (capsule_birth.c), and USE runs another VM's lines, so these dispatches are for a VM other than sk_percpu()->current_vm.
    • The Stadium locking is correct either way: it locks the dispatching VM's own slot.
    • But MC-3's claim (running_on) must cover a VM executed this way, not only one switched to.
  2. Quota slots are never released. stadium.c has no release path: in_use is set by grants and cleared only at boot.
    • The probe counted 13 slots in use against 7 registry entries. Boot self-tests grant quotas to test ids (kernel_main.c), and a dead VM keeps its slot and cells.
    • A long-running system eventually exhausts the RAM-sized slot table.
  3. An owner's word admission can evict a live message cell. The admission fallback evicts the owner's least-dense resident, skipping only pinned and containing cells. Hermes admits messages unpinned into the sender's quota.
    • After such an eviction, msg->stadium_cell is stale: the release would evict whatever occupies that cell next.
    • This run counted 0 such evictions (the quotas never ran dry of free cells), so it is latent.

Standard sequence (clean build, with the probe reverted; the tree is byte-identical to the saved stage diff).

ISA log O-A (G, PREC, LOAD) M7.1a hash MINT / WIREBIND / USE / 1 2 + . idle growth UNKNOWN / faults
amd64 logs/20260925-220202 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
aarch64 logs/20260925-221440 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
riscv64 logs/20260925-222740 A=40 B=40, HHH, ok 0x99b01134c3d3049c all ok, 3 0 B / 20 s 0 / 0
  • The G string is byte-identical to MC-0c and earlier on all three ISAs, so the O(ncells) audit scan now running under a slot lock didn't move it.
  • The dict_hash set is unchanged, and the ledger line reads stadium_conserved(self)=true.
  • No stop condition was hit. MC-0d done, and with it MC-0: one core, fully locked, ready for MC-1.