admin/LithosAnanake
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FABRIC-3.10.8 §7: MC-3 design -- claim, per-core idle, wake by IPI; rulings needed

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-26 14:27:52 -04:00
co-authored by Claude Opus 5.5
parent c8b754fd80
commit 076622abb7
1 changed files with 54 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
FABRIC-3.10.8.md
+54
View File
@@ -985,3 +985,57 @@ The boot hart HSM chose (0) is the one OpenSBI names as `Boot HART ID`.
| riscv64 | `logs/20260926-141508` | same; 0 `timed out` |
**MC-2 is complete.** Next: MC-3, where worker cores claim through the shared decision. It is the stage that makes the extra cores do work, and it needs its own design section first: the claim by compare-and-swap, per-core idle contexts and beats, wake by IPI with each core's interrupt controller, and the §5.3 findings.
## 7. MC-3 — worker cores run participants (design, 2026-09-26)
### 7.1 How it works today, on one core (traced 2026-09-26)
- **One chain of control.** A participant runs at its trampoline (`sk_vm_switch_entry()`), draining one message at a time. At each step it asks `sk_vm_switch_decide()` who should run.
- With no work anywhere, it switches to **Hera**, the root. Hera's idle point polls the console.
- Mid-word, a VM re-decides at its own checkpoint (`sk_vm_switch_checkpoint()`), only when `g_decision_due` is set.
- **Beats come from execution, not a timer.** `vm_heartbeat_run_cycle()` runs every `HEARTBEAT_CHECK_FREQUENCY` words of the VM being interpreted. It charges the running participant (`vm_physics_charge()`) and sets `g_decision_due`. So a VM gets beats on whatever core runs it; no per-core timer is needed for the compudynamics.
- **What is still one-core-only:**
- **(a)** `g_decision_due` and the wait-stamp counter `g_stamp_next` are single globals, written without atomics.
- **(b)** `capsule_vm_physics.c` (fleet heat transfer, the charge, the touch window) has **no lock**; its header says "no locking". MC-0 did not cover it.
- **(c)** Nothing records which core runs a VM, so two cores could both pick the same one.
- **(d)** The fallback root is Hera, who lives on core 0.
- **(e)** A worker core, once parked, has no way to be woken.
- **(f)** `KILL` refuses only "the VM running", meaning on this core.
### 7.2 Design
1. **The claim.**
- Every VM gets `running_on`: the core running it, or none.
- `sk_vm_context_switch()` claims `to` with a compare-and-swap from none to this core. If the claim fails, another core already has `to`; the switch returns "not taken" and the caller re-decides. It is counted, not an error.
- `from` is released only once its context is saved. The core records `from` as "release after switch", and whatever code runs next on that core releases it: the resumed VM, a first-entry trampoline, or the core's idle context. Until then no other core can take a half-saved context.
- **A double claim** (two cores running one VM) is checked for at every switch-in and counted. It must be 0.
2. **Per-core idle context** (cores 1..N).
- Each worker core runs an idle loop on its own stack: decide over participants only (MC-2), claim and switch if something has work, otherwise halt with interrupts enabled until woken.
- A participant on a worker core with nothing left to do switches back to **its core's idle context**, not to Hera. The fallback root is per core: Hera on core 0, the idle context elsewhere.
- The context switch gains a VM-less form for this (the idle context's saved stack pointer lives in the per-core block).
3. **Wake.**
- When a participant gains work (mail admitted: `note_work_gained`, `note_mail`) and a worker core is halted, one halted core gets an inter-processor interrupt. Its handler does nothing; the core simply returns from the halt into its idle loop and decides.
- Each worker core first initialises its own interrupt controller, masked except for the IPI:
- amd64: the LAPIC, with one IPI vector;
- aarch64: the GIC CPU interface, with SGI 0;
- riscv64: `sie.SSIE`, sent by the SBI IPI extension.
- Core 0 keeps its timer and devices. Worker cores get no timer, and never touch USB or xHCI (FABRIC-3.10.0 §6).
4. **Per-core decision state.**
- "Decision due" moves into the per-core block: a beat or new mail sets it on the core that saw it, and mail for a halted core's candidates sets it there along with the IPI.
- `g_stamp_next` becomes an atomic fetch-add, so stamps stay unique across cores and ties still have one answer.
5. **Fleet physics lock.** It is one lock over `capsule_vm_physics.c`'s node list, window and slope. It ranks **between switch and registry**, because physics calls `capsule_vm_registry_get()` (the root walk) and the registry never calls out.
6. **`KILL`** refuses a VM whose `running_on` is any core, not only this one (the FABRIC-3.10.0 §3.6 rule).
7. **Affinity** (MC-R3): among equally cool candidates, a worker core prefers a VM it ran last.
### 7.3 Evidence (for MC-3's last stage, from the plan, made concrete)
- **Two workers genuinely overlapping.** A probe records each worker's (core, first beat, last beat) on the raw counter. The two workers' intervals overlap, on different cores.
- **0 double claims.** Conservation holds throughout (`stadium_conserved()`, the ledger, fleet K = 65536).
- **G by counts:** A = 40, B = 40. The string is no longer byte-identical once the cores truly interleave (MC-R4).
- **Idle worker cores halted:** 0 park wakes except by IPI, each IPI wake counted; host `ps -L` shows the worker vCPU near 0% when idle.
- **The standard sequence** otherwise: dict_hash unchanged, all ok `3`, 0 B idle growth.
### 7.4 Rulings needed
1. **Stage split.** Proposed: **MC-3a** is the claim and release, `running_on`, the physics lock, the atomic stamps and the per-core decision flag. It changes nothing observable with one worker core, and a probe shows 0 double claims. **MC-3b** is each worker core's interrupt controller and the IPI: core 0 wakes core 1, which counts it and parks again. **MC-3c** is the per-core idle context and wake on work, where workers actually run on core 1. Or all of it as one stage.
2. **Core 0 when idle** (finding §5.2): it polls at about 64% of a host CPU. Proposed: **leave it** in MC-3, since core 0 is the console, USB and timer core and polls by design. Or: core 0 halts until its timer or a device interrupt (a change to the idle point's input path).