FABRIC-3.10.0: Stadium's per-dispatch path must be partitioned by quota slot, not locked (the multi-core speedup depends on it); expected-gain note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-25 12:44:26 -04:00
co-authored by Claude Opus 5.5
parent 85b97ea467
commit 1777ac59a1
+26 -3
View File
@@ -119,12 +119,29 @@ by line):
| `sf_mutex_*` | Real spinlocks (with interrupts masked while held on the owning core). |
| `kmalloc` | A heap lock. |
| kernel-Hermes (arena, queues, channels, ledger, events) | A lock (one to start; per-queue later if contention shows). |
| Stadium (reservoirs, cells, quotas, word rows) | A lock, plus atomic reservoir arithmetic. |
| Stadium (reservoirs, cells, quotas, word rows) | **Not one lock — the per-dispatch path must not contend** (see below). |
| VM registry, birth, kill | A registry lock. `KILL` of a VM whose `running_on` is another core is refused, the multi-core form of today's "it is the VM running" refusal. |
| Switch-signal table | Compare-and-swap for claim and release; a lock for register and unregister. |
| Console output | A line lock, so lines from different cores never interleave mid-line. |
| Log region, block subsystem | A lock each. Artemis's storage stays on core 0, but workers can use `BLOCK` words. |
**The Stadium hot path decides whether multi-core speeds anything up** (added 2026-09-25, after
Captain Bob asked whether this should give a significant performance boost).
- `stadium_word_dispatch()` runs on **every word dispatch** of every VM (`vm_core.c`), for word
heat: resolve the word's cell, cool it, pull admission heat from the VM's reservoir, and admit
or evict a cell.
- Behind a single Stadium lock, every worker core would contend on every word, and the speedup
would collapse towards 1×. `kmalloc` and kernel-Hermes are touched far less often (on
definitions and messages), so one lock each is fine there.
- **Requirement: Stadium's per-dispatch state is partitioned by the VM's quota slot,** which is
already how the word rows are keyed (`word_slots[slot][word_id]`, S-1b).
- A VM runs on one core at a time, so its own row and cells need no lock while it runs.
- Its reservoir takes atomic arithmetic.
- Only operations that cross slots (admission evicting another VM's cell, quota grants,
conservation audits) take the Stadium lock.
- This is part of MC-0, and MC-0's evidence includes it: a per-dispatch path that takes no
shared lock.
### 3.7 Bringing up the cores, per ISA
- **amd64:** enumerate cores from the MADT's LAPIC entries. Start each with INIT-SIPI-SIPI through
@@ -160,7 +177,7 @@ Each step keeps the system shippable and gets its own 3-ISA acceptance.
| Stage | Change | Evidence (to be named in full in 3.10.8) |
|---|---|---|
| **MC-0** | Per-core infrastructure on one core: the per-CPU pointer and block; real spinlocks behind `sf_mutex`; the §3.6 locking applied. | Standard sequence unchanged on one core. A probe shows every lock taken and released, never held across a switch. |
| **MC-0** | Per-core infrastructure on one core: the per-CPU pointer and block; real spinlocks behind `sf_mutex`; the §3.6 locking applied, including Stadium's per-dispatch path partitioned by quota slot. | Standard sequence unchanged on one core. A probe shows every lock taken and released, never held across a switch, and **no shared lock taken on the word-dispatch path**. |
| **MC-1** | Bring up the extra cores and park them halted; QEMU `-smp 2`. | The boot reports N cores online; the standard sequence is unchanged, with the extra cores idle. |
| **MC-2** | Observers get system-core affinity (core 0). | With one worker core, behaviour identical to today; observers are never claimed off core 0. |
| **MC-3** | Worker cores claim through the shared decision, with per-core idle contexts and wake by interrupt. | Two workers genuinely overlapping in time on two cores. A probe proves the claim never lets two cores run one worker. Conservation holds throughout. G by counts. Idle cores halted, not spinning (0 idle log growth). |
@@ -172,7 +189,13 @@ Each step keeps the system shippable and gets its own 3-ISA acceptance.
- **Worker cores and devices:** only core 0 services USB. A worker core never touches xHCI.
- **Locking granularity:** one lock per subsystem to start (MC-R2) may contend under many
workers. It is measured before it is refined.
workers. It is measured before it is refined. Stadium's per-dispatch path is the exception,
partitioned from the start (§3.6).
- **Expected gain.** Close to linear throughput for many independent compute-bound workers on
real hardware, once the dispatch path is partitioned. None for a single worker (a VM is
single-threaded). Little for anything bound by serial output, the current DoE's limit: every
core's heartbeat rows funnel through one console. MC-3/MC-4 measure it: throughput for a fixed
batch of workers at 1, 2 and 4 cores.
- **Interrupt routing:** device IRQs stay on core 0 (the I/O APIC, GIC SPIs and PLIC contexts
route there). Only inter-processor wakes reach worker cores.