LithosAnanake/FABRIC-3.10.8.md

FABRIC-3.10.8.md — Multi-core: execution plan and log

Status: IN PROGRESS. Design and rulings are in FABRIC-3.10.0.md (ruled 2026-09-25, §7). Multi-core runs ahead of O-D (MC-R6 as ruled); O-D relaunches afterwards with row path A.

Each stage:

• needs Captain Bob's explicit go;
• gets its own commit;
• gets its own three-architecture acceptance (clean qemu; amd64, then aarch64, then riscv64; foreground; visible framebuffer; one at a time; coupled images reset before each).

Evidence is named before each stage runs.

The standing acceptance sequence, unchanged from FABRIC-3.9.8.md:

• a boot to [zuse@Hera] ok> with zero UNKNOWN WORD, faults and frame corruption;
• the dict_hash set identical across ISAs;
• the Stage G interleave (A/B), byte-identical across ISAs while there is one worker core; by counts (A = 40, B = 40) once there are more (MC-R4);
• the O-A checks (observer precedence, console responsive under load);
• mint → WIREBIND → USE → 1 2 + . → 3;
• 0 bytes of idle log growth over 20 s.

No DoE execution until everything is closed (Captain Bob, 2026-09-25: "until we close everything, do not do ant DoE eecution."). MC-4's campaigns wait for that too. Stage evidence is the standard sequence plus probes, never a campaign.

Never make clean an ISA while its QEMU runs: it deletes that QEMU's QMP and serial sockets (found 2026-09-25).

1. What MC-0 must cover (traced 2026-09-25)

Everything that would break if two cores ran kernel or VM code at the same moment.

Kernel subsystems (src/starkernel/), one lock each (MC-R2), except where noted:

Subsystem	Shared state	Treatment
kmalloc, pmm, vmm	free lists, bitmaps, page tables	heap lock; PMM lock; VMM lock
kernel-Hermes	arena directory and segments, queue table, channel table, ledger, event counters	one lock
Stadium	reservoirs, cells, quotas, word rows	per-dispatch path partitioned by quota slot (a VM's own row and cells need no lock while it runs on its one core); reservoir arithmetic atomic; cross-slot operations (admission evicting another VM's cell, grants, audits) take the Stadium lock
VM registry, birth, kill, reap	registry list, names	registry lock; KILL refuses a VM running on another core
switch-signal table	slots, stamps, decision-due flag	compare-and-swap for claim/release (MC-3); lock for register/unregister
current VM, frame stack	g_switch_current_vm and friends	move to the per-core block
console	UART, framebuffer VT100 state, line prefix state	one line lock
log region, DoE region, block subsystem	rings, block chain, caches	a lock each
heartbeat	tick counters	per-core counters (MC-3); one core until then

Shared VM code (src/, vendored; runs inside every VM):

File	Global	Treatment
dictionary_management.c	sf_fc_list/sf_fc_count/sf_fc_cap/sf_cached_latest: one dictionary lookup index for all VMs, rebuilt whenever a different VM's dictionary is searched	move into VM (per-VM index). Also stops the rebuild thrash on every switch today.
word_source/starforth_words.c	g_prng_state: one PRNG stream for all VMs (accepted at Stage H)	an atomic step, so the stream stays one stream (order is nondeterministic across cores anyway, MC-R4)
physics_runtime.c	g_runtime	inventory in MC-0; per-VM or atomic
word_source/dictionary_manipulation_words.c	state_variable (STATE)	inventory in MC-0; per-VM
physics_hotwords_cache.c	g_hotwords_cache	retired under __STARKERNEL__ (item 4.1); confirm unused
word_source/ttf_words.c	font and raster cache	core 0 only (Hestia's framebuffer)

2. Staged plan

Stage	Change	Evidence (named in advance)
MC-0a	The per-core block, on one core. A per-core structure reached through each ISA's per-CPU register (GS base / TPIDR_EL1 / tp), set up for core 0 at boot. g_switch_current_vm moves into it.	Standard sequence unchanged. A probe reads the per-core pointer from a normal kernel path and from an interrupt handler, and gets core 0's block both times on all three ISAs.
MC-0b	Real spinlocks. A kernel spinlock primitive (atomic exchange, interrupts masked while held on the owning core). sf_mutex_* stop being no-ops. The §1 kernel locks are added.	Standard sequence unchanged. A probe counts every lock's acquisitions and releases through a boot and the standard sequence: counts balance, and no lock is held across a context switch or on return to the idle point.
MC-0b2	The block subsystem's lock (split from MC-0b, Captain Bob 2026-09-25). The vendored block_subsystem.c calls its own public functions in cycles (blk_set_meta → migration check → relocate → blk_set_meta), so it is refactored into locking public functions over unlocked internals, under __STARKERNEL__.	Standard sequence unchanged. The MC-0b probe, extended to the block lock: counts balance, 0 recursions.
MC-0c	Shared VM code made per-VM or atomic (§1, second table). The dictionary lookup index moves into VM.	The dict_hash set unchanged (the index is not hashed). A probe: two VMs searched alternately never rebuild each other's index. Standard sequence.
MC-0d	Stadium's dispatch path partitioned (§1).	Standard sequence; conservation holds (stadium_conserved() on every VM, the ledger audit). A probe proves the dispatch path takes no shared lock.
MC-1	Bring up the extra cores and park them halted, per ISA: amd64 (MADT, INIT-SIPI-SIPI, a position-independent trampoline below 1 MiB), aarch64 (DTB /cpus, PSCI CPU_ON), riscv64 (DTB harts, SBI HSM hart_start). QEMU gets -smp 2.	The boot reports each core online with its own per-core block; the extra cores sit halted (0 idle growth, no spin); the standard sequence unchanged.
MC-2	Observers get core-0 affinity.	With one worker core, behaviour identical to today. A probe shows no observer ever claimed off core 0.
MC-3	Worker cores claim through the shared decision: running_on with compare-and-swap, per-core idle contexts, per-core beats, wake by inter-processor interrupt.	Two workers genuinely overlapping in time on two cores. A probe counts double claims: 0. Conservation holds. G by counts. Idle worker cores halted (0 idle growth).
MC-4	DoE with cores as a factor (after O-D): per-ISA campaigns at 1, 2 and 4 cores (MC-R5).	Per ISA and core count: 18 MU-TRIAL markers, fail=0, MU-CAMPAIGN: complete.

Stop conditions (re-plan rather than push on):

• dict_hash diverges across ISAs;
• conservation goes false;
• a lock held across a switch, or an unbalanced lock count;
• a double claim;
• an idle core spinning.

3. Execution log

3.1 MC-0a — the per-core block, on one core (2026-09-25)

Change.

• include/starkernel/percpu.h (new): sk_percpu_t { self, core_id, current_vm }, with self first.
  • sk_percpu() reads this core's block: movq %gs:0 on amd64, mrs tpidr_el1 on aarch64, mv tp on riscv64.
  • sk_percpu_install() writes the register: IA32_GS_BASE via wrmsr, TPIDR_EL1, or tp.
• src/starkernel/hal/percpu.c (new): the boot core's block is one static object, not a table sized to a core count. Secondary cores get theirs as they come online (MC-1).
• kernel_main.c: sk_percpu_boot_init() runs straight after arch_early_init(), because on amd64 that call reloads GS, which would reset the base.
• switch.c: the global g_switch_current_vm is gone.
  • sk_vm_switch_current_vm(), the setter, the idle point and sk_vm_context_switch() use sk_percpu()->current_vm.
  • After resume, the context switch re-reads the block rather than trusting a pointer cached before the switch.

Probe (temporary; removed before commit).

• kernel_main.c checked, from the normal path after the Tripod came up, that:
  • sk_percpu()->self == sk_percpu();
  • core_id == 0;
  • current_vm is Hera;
  • sk_vm_switch_current_vm() agrees.
• heartbeat_tick() ran the same self-check from interrupt context on every beat.

ISA	log	normal path	core_id	current_vm is Hera	switch.c agrees	interrupt path ok / bad
amd64	logs/20260925-130335	ok	0	1	1	889 / 0
aarch64	logs/20260925-130905	ok	0	1	1	1877 / 0
riscv64	logs/20260925-131337	ok	0	1	1	1625 / 0

Standard sequence (clean build, with the probe reverted; the tree was checked byte-for-byte against the saved stage diff).

ISA	log	O-A (G, PREC, LOAD)	M7.1a hash	MINT / WIREBIND / USE / 1 2 + .	idle growth	UNKNOWN / faults
amd64	logs/20260925-131805	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0
aarch64	logs/20260925-133049	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0
riscv64	logs/20260925-134357	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0

• The dict_hash values (Hera 0x00340a8b2c842eb7, Hestia 0x78bf28439e4a643f, Artemis 0x66886d174d5f3b5a) are identical across ISAs and unchanged from O-C.
• The ledger line matches across ISAs: held=2048, pulled=258048, returned=255263, consumed=737, conserved=true.
• No stop condition was hit. MC-0a done.

3.2 MC-0b — real spinlocks (2026-09-25)

The primitive (include/starkernel/spinlock.h, src/starkernel/hal/spinlock.c, new).

• One lock word per lock: 0 = free, core_id + 1 = held.

  • Taken by compare-and-swap. The atomics are inline on every ISA (nm -u is empty on all three ELFs).
  • Interrupts stay masked on the owning core while it holds any lock.

• The mask state lives in the per-core block, not in the lock (lock_depth, saved_irq in sk_percpu_t):

  • the first lock taken saves the interrupt state and masks;
  • the last lock released restores it;
  • so release order doesn't matter.

• The rule: no lock crosses a switch or reaches the idle point. Each of these sites checks it and counts any violation:

  • sk_vm_context_switch(), when leaving and on resume;
  • sk_vm_idle_point().

• Failures are counted and logged at the idle point (LOCK: lines, LOG_ERROR):

  • recursion (a core takes a lock it already holds): let through, so the core doesn't hang, and counted. It is never legitimate, because on one core "the owner is me" can't tell a nested call from a second VM on the same core;
  • release of a lock not held;
  • a lock held at a switch or idle point;
  • a VM mutex destroyed while held.

• The locks are no-ops until the boot core's block is installed (sk_percpu_ready). This was found during the probe runs:

  • riscv64 is built monolithic, and its arch_early_init() prints through the console before sk_percpu_boot_init();
  • the first lock therefore read tp while it still held the firmware's value, and the load hit OpenSBI memory (stval = tp+8, a firmware load access fault);
  • early boot is one core with interrupts off, so skipping the locks there is correct.

  That crashed run's log was deleted as flawed.

• sf_mutex_* (the vendored VM's dict_lock and tuning_lock, shim.c) now use the same lock word. They are no longer no-ops.

The kernel locks, in acquisition order (spinlock.h; MC-3's deadlock freedom rests on this order):

1. hermes
2. switch
3. registry
4. doe
5. log
6. console
7. heap
8. vmm
9. pmm

Stadium's lock comes with MC-0d, below Hermes. Stadium today runs inside dict_lock (vm_core.c's heat bump calls stadium_word_dispatch()); MC-0d's order must account for that.

Lock	Covers	Shape
heap	kmalloc/kfree, stats, censuses	kmalloc() delegates to kmalloc_aligned(), and only the latter locks
vmm	page-table walks	*_locked bodies. The map-conflict print moved outside the lock (the console lock sits above it). Takes pmm for table pages
pmm	bitmap	pmm_alloc_page()/pmm_free_page() delegate to the contiguous forms
console	output path and line state (name, line start, pending line close, the framebuffer terminal)	A string goes out under one hold. Unlocked on purpose: console_get_vm_name() (the prefix provider calls it under the lock), the boot-time console_fb_init()/console_fb_enable_ttf() (they log), and input
log, doe	control-block read-modify-write plus the slot write	*_locked bodies
registry	node list, entry copies, single-field writes	Never held while calling out. KILL and reap take it only for their reads and final writes. A birth fills its embryo unlocked (private until LIVE). kill_all_nonmama (cold-reset path) walks unlocked
switch	slot table, stamps, decision-due flag, counters	Never held while calling out. The decision, the Stage H floor and the reap sweep copy one slot under the lock (slot_copy()) and judge it outside
hermes	arena, ledger, channels, queues, event counters	Public functions lock over *_locked statics. Never held across vm_interpret_kind() (the drain) or a policy word. The switch notifications after a send go out once the lock is dropped (the floor reads Hermes queues back). Publish and event walk members one copied member at a time. sk_hermes_event()'s static payload moved onto the stack

The block subsystem is split out as MC-0b2 (Captain Bob, 2026-09-25), because its internal call cycles need a locked/unlocked refactor of vendored code. See §2.

Probe (temporary, removed before commit). A MC0B-PROBE word in Hera's and every child's vocabulary printed every counter, snapshotted before printing:

• each lock's acquisitions and releases, and its owner;
• the VM mutexes (the core total minus the kernel locks);
• the core-0 total and its depth;
• the number of switch/idle checks and violations;
• recursions, bad releases and destroyed-while-held.

It ran at Hera's prompt (after O-A's G, PREC and LOAD), and again on rajames after MINT, WIREBIND, USE and 1 2 + .. The table shows the final (rajames) reading:

ISA	log	total acquired = released	depth	switch/idle checks	held at switch	recursions / bad releases / destroyed held
amd64	logs/20260925-150735	1,503,458,645	0	51,137,748	0	0 / 0 / 0
aarch64	logs/20260925-152109	235,854,532	0	7,565,646	0	0 / 0 / 0
riscv64	logs/20260925-155048	566,894,806	0	19,029,507	0	0 / 0 / 0

• Every lock balances individually on every ISA, at both readings.
• The amd64 and aarch64 probe runs predate the sk_percpu_ready guard. That guard only changes behaviour before sk_percpu_boot_init(), and the acceptance runs below are on the final code.
• The riscv64 vblk_io timed out count, which the advisor flagged (defect 3 sits in the region now masked), is 2 in the probe run, the same as MC-0a's acceptance run. It is 0 in this stage's acceptance run.

Finding, for MC-3. Hera's input-poll idle loop runs the switch decision without pause, and every pass takes the registry, switch and Hermes locks once per slot. That is the hundreds of millions of acquisitions above (amd64: registry 379 M, switch 858 M, Hermes 266 M). It is harmless on one core. With several cores, core 0 spinning through these same locks would contend with every worker's claim. MC-3 should settle how that loop waits before cores share these locks.

Standard sequence (clean build, with the probe reverted; checked against the saved stage diff, which differs only by the guard added after it).

ISA	log	O-A (G, PREC, LOAD)	M7.1a hash	MINT / WIREBIND / USE / 1 2 + .	idle growth	UNKNOWN / faults
amd64	logs/20260925-160434	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0
aarch64	logs/20260925-161720	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0
riscv64	logs/20260925-163024	A=40 B=40, HHH, ok	0x99b01134c3d3049c	all ok, 3	0 B / 20 s	0 / 0

• The G string is byte-identical to MC-0a on all three ISAs, so masking interrupts under the locks didn't move the interleave.
• The dict_hash set and the ledger line are unchanged from MC-0a.
• No stop condition was hit. MC-0b done (the block subsystem follows as MC-0b2).