admin/LithosAnanake
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Stage E: Category B strip -- remove Hermes, messaging.4th, old routing (FABRIC-3.6.md task 4.1-4.4)

Browse Source 
All FORTH-owned message types were cut over to kernel-Hermes in Phase 3
(tasks 3.8-3.10). This removes the now-dead FORTH messaging layer and
the Hermes VM itself: capsules/common/messaging.4th, capsules/hermes/init.4th,
the slot-3 VM-NAME-REG pairing convention, and every load-site/birth-site
reference across capsules/init.4th, artemis/init.4th, hestia/init.4th,
doe-campaign.4th (Artemis-only now), capsule_console.c, capsule_mint.c,
capsule_wirebind.c, capsule_birth.c, and kernel_main.c.

Verified on all three architectures: clean boot, mkcapsule --lint clean
(36 files, 0 violations), zero UNKNOWN WORD, identical dict_hash across
amd64/aarch64/riscv64 for every VM, and a full mint -> WIREBIND-attach ->
USE -> relay round-trip exercising the two highest-risk edits
(capsule_console.c/capsule_mint.c).

Found, not fixed: deleting messaging.4th removes SEND-ELEVATE-REQUEST,
which was the only caller of KH-ELEVATE-SEND and the only path to
ELEVATE-GRANT (zuse-eligibility.4th, still loaded at boot) -- Phase 8
PKI's own elevation entrypoint. Needs a decision before Phase 5 close-out.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-22 18:29:51 -04:00
co-authored by Claude Sonnet 5
parent cab9b5a31f
commit 3e201c82a5
23 changed files with 27825 additions and 886 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/starkernel/repl.c
+86 -134
View File
@@ -444,19 +444,22 @@ static void sk_word_kh_blk_attach_send(VM *vm) {
#define SK_KH_ELEVATE_BUF_SIZE 256
static char g_kh_elevate_buf[SK_KH_ELEVATE_BUF_SIZE];
/* KH-ELEVATE-SEND ( paddr plen -- ok? ): messaging.4th's own
* SEND-ELEVATE-REQUEST builds its payload (a FORTH command string
* calling ELEVATE-GRANT, block 5040) and hands it here instead of
* CH-REQUEST's old COMMON-CH/MSG-SEND path. `from`/`to` are derived the
* same way KH-BLK-ATTACH-SEND's own already does (the calling VM and
* sk_get_mama_vm() -- ELEVATE-GRANT always runs on Hera, per
* zuse-eligibility.4th's own header comment) -- the caller never
/* KH-ELEVATE-SEND ( paddr plen -- ok? ): built for messaging.4th's own
* SEND-ELEVATE-REQUEST (a FORTH command string calling ELEVATE-GRANT,
* zuse-eligibility.4th block 4022), which handed its payload here
* instead of CH-REQUEST's old COMMON-CH/MSG-SEND path. `from`/`to` are
* derived the same way KH-BLK-ATTACH-SEND's own already does (the
* calling VM and sk_get_mama_vm() -- ELEVATE-GRANT always runs on Hera,
* per zuse-eligibility.4th's own header comment) -- the caller never
* supplies either, which is a real correctness improvement over the
* FORTH path it replaces: CH-REQUEST's own "initiator-only gate...
* FORTH path it replaced: CH-REQUEST's own "initiator-only gate...
* refuses if from doesn't match MY-CH-ID" existed only because a caller
* COULD claim to be a different VM by passing the wrong stack value;
* deriving `from` from the C-level calling VM's own identity makes that
* spoof structurally impossible rather than merely gated. */
* spoof structurally impossible rather than merely gated. FABRIC-3.6.md
* Phase 4, Stage E deleted messaging.4th -- SEND-ELEVATE-REQUEST no
* longer exists anywhere, so this word currently has no FORTH caller.
* Found, not fixed; see FABRIC-3.6.md's own Phase 4 entry. */
static void sk_word_kh_elevate_send(VM *vm) {
if (vm->dsp < 1) {
log_message(LOG_ERROR, "KH-ELEVATE-SEND: stack underflow");
@@ -527,49 +530,42 @@ static void sk_cursor_show(void)
console_fb_draw_cursor();
}
/* Reentrancy guards for the MSG-TICK pump inside sk_repl_idle().
/* Reentrancy guards for the kernel-Hermes drain pump inside sk_repl_idle().
*
* sk_repl_idle() runs vm_interpret(mama, ...) (below) to VM-EXEC MSG-TICK
* into every live child VM (FABRIC-2.md Phase C). But sk_repl_idle() is
* itself called from the blocking KEY/EXPECT/QUERY reads (sk_console_getkey()
* / sk_console_readline(), which run *from inside* the executing VM's own
* vm_interpret once a FORTH word reads input mid-line). vm_interpret() is
* not reentrant -- it resets the VM's single input_buffer/input_pos/
* input_length (vm_core.c) on entry. Calling vm_interpret(mama, ...) at that
* point would re-enter mama's interpreter mid-parse and silently truncate
* the rest of the line; if the pump's own MSG-TICK work then triggers another
* blocking read, it would also recurse unboundedly. Two flags keep the pump
* off every path that could re-enter an interpreter:
* FABRIC-3.6.md Phase 4 (Stage E), 2026-09-22: this used to guard a
* VM-EXEC "MSG-TICK" dispatch into every live child VM (FABRIC-2.md
* Phase C, common:messaging.4th, now removed). The mechanism changed --
* sk_repl_idle() now calls sk_hermes_drain_checkpoint() directly per VM,
* not VM-EXEC -- but the reentrancy hazard these two flags exist for is
* unchanged: drain_checkpoint() still calls vm_interpret() internally
* (on the drained message's own payload), and sk_repl_idle() is itself
* called from the blocking KEY/EXPECT/QUERY reads (sk_console_getkey()/
* sk_console_readline(), which run *from inside* the executing VM's own
* vm_interpret once a FORTH word reads input mid-line). vm_interpret()
* is not reentrant -- it resets the VM's single input_buffer/input_pos/
* input_length (vm_core.c) on entry. Calling it on mama at that point
* would re-enter her interpreter mid-parse and silently truncate the
* rest of the line. Two flags keep the pump off every path that could
* re-enter an interpreter:
* - g_mama_interpreting: set while Hera is executing a dispatched line, so
* the pump defers to the next safe (prompt) boundary.
* - g_idle_pump_active: belt-and-suspenders; stops recursive re-entry
* from inside the pump's own vm_interpret call.
* from inside the pump's own drain call.
*/
static int g_mama_interpreting; /* zero-initialized (BSS) */
static int g_idle_pump_active; /* zero-initialized (BSS) */
/* FABRIC-3.md SXXII (2026-09-12): the messaging pump below used to walk
* the ENTIRE live-VM registry every idle beat and dispatch a full
* VM-EXEC "MSG-TICK" into every one of them -- O(N) per tick, forever,
* with no cap. Fine at Tripod's original 3-VM scale; live-caught as a
* real bottleneck at just 9 VMs on riscv64 (a std79 turn-attractor
* campaign that completed in ~200-340s on amd64/aarch64 never finished
* on riscv64 at the same VM count, while 8 VMs was fine there too --
* the qemu-instance progress check (utime climbing steadily, ~66-75%
* CPU) ruled out a hang; it was genuinely just this loop, at riscv64's
* slower per-dispatch cost, dominating wall time). The real fleet is
* headed toward hundreds of VMs, at which point this isn't a riscv64
* quirk -- unconditional O(N) per second is a wall on every
* architecture. Fixed by round-robin batching: SK_MSG_PUMP_BATCH live
* VMs get dispatched per beat, not all of them, with a persistent
* cursor picking up where the previous beat left off (wrapping around
* the registry). Bounds both the scan and the dispatch cost to O(K)
* regardless of total VM count -- the tradeoff is that any single VM's
* own queue now drains roughly every ceil(N/K) beats instead of every
* beat, which is still bounded and still matches the pump's own
* existing "best-effort, resumes next beat" contract. */
#define SK_MSG_PUMP_BATCH 4u /* max VM-EXEC MSG-TICK dispatches per idle beat */
static uint32_t g_msg_pump_cursor; /* zero-initialized (BSS) */
/* FABRIC-3.md SXXII (2026-09-12) recorded a real O(N) bottleneck here --
* the messaging pump below used to walk the entire live-VM registry
* every idle beat and dispatch a full VM-EXEC "MSG-TICK" into every one
* of them, live-caught as a wall-time-dominating cost at just 9 VMs on
* riscv64 -- and fixed it with round-robin batching (a persistent
* cursor, a fixed number of dispatches per beat). FABRIC-3.6.md Phase 4
* (Stage E), 2026-09-22: that whole mechanism (MSG-TICK, the batching
* cap, and this comment's own prior form) is gone along with
* common:messaging.4th itself -- see sk_repl_idle()'s own current
* comment, just below, for why the replacement (kernel-Hermes drain)
* does not carry the same O(N) cost and needs no cap. */
static void sk_repl_idle(VM *active_vm)
{
@@ -857,18 +853,25 @@ static void sk_repl_idle(VM *active_vm)
* comment. */
capsule_wirebind_reap_idle_check();
/* FABRIC-2.md Phase C (2026-08-28): distributed messaging pump. Every
* live VM except Hera herself now owns its own MSG-ARENA/CH-ARENA and
* MSG-TICK word (see capsules/common/messaging.4th) instead of only
* Hermes having one -- "fully distributed, Hera pumps each VM's
* drain" was the confirmed design. Hera is excluded: she never loads
* common:messaging.4th (kernel_main.c's own comment at the Hermes-
* birth call site explains why -- the STADIUM-* primitives it needs
* are deliberately never registered in her own dictionary, to keep
* her dict_hash off item 4.1's baseline), so MSG-TICK is genuinely
* absent there, not just untried. She is the only VM with a
* persistent idle tick, so she walks the registry once per idle beat
* and VM-EXECs MSG-TICK into every OTHER live VM's own dictionary. */
/* FABRIC-3.6.md Phase 4 (Stage E, Category B strip), 2026-09-22:
* this used to be a distributed FORTH messaging pump -- FABRIC-2.md
* Phase C's own design, VM-EXEC'ing MSG-TICK into every live VM's
* own dictionary once per idle beat (common:messaging.4th, now
* removed entirely). That mechanism, and the comment that used to
* sit here explaining it, are both gone: kernel-Hermes's own drain
* (below) has needed no FORTH word since task 3.9, and every
* messaging.4th-owned message type had a real cutover by task 3.10
* (FABRIC-3.6.md task 3.11, Phase 3 gate). The prior version of this
* comment also claimed "Hera never loads common:messaging.4th" --
* that was already wrong before this strip (capsules/init.4th line
* 21 EXECs it directly; task 3.10's own live FIND-based check
* confirmed it), not just made moot by removing the file. She still
* has the only persistent idle tick, so she is still the one that
* walks the registry once per idle beat -- now purely to give every
* live VM's own kernel-Hermes queue a chance to drain, batched the
* same round-robin way the old MSG-TICK dispatch was (SK_MSG_PUMP_
* BATCH below), kept for the same reason: bounding this to O(K)
* regardless of total VM count, not O(N) every beat. */
VM *mama = (VM *)sk_get_mama_vm();
/* Reentrancy guard: never run the pump while mama is mid-interpret
@@ -887,93 +890,42 @@ static void sk_repl_idle(VM *active_vm)
g_idle_pump_active = 1;
{
uint32_t count = capsule_vm_registry_count();
uint32_t dispatched = 0;
uint32_t scanned = 0;
uint32_t i = (count > 0) ? (g_msg_pump_cursor % count) : 0;
/* FABRIC-3.6.md Phase 4 (Stage E), 2026-09-22: the SK_MSG_PUMP_
* BATCH cap this loop used to enforce existed to bound the cost
* of the old MSG-TICK VM-EXEC dispatch -- a genuinely expensive
* per-VM vm_interpret() call, confirmed live as a real O(N)
* bottleneck at just 9 VMs on riscv64 (see this section's own
* history above). sk_hermes_drain_checkpoint() (task 3.4) is not
* that: its own fast path is a single global-counter read
* (sk_hermes_pending_total == 0 -> return immediately,
* kernel_hermes.c's own doc comment), so walking every live VM
* every beat costs one cheap integer read per VM in the common
* case (nothing in flight), not an O(N) wall -- the cap this
* loop's own comment history worried about does not apply to
* this call. Removed: every live VM (except mama, handled
* separately below) is now visited every idle beat, matching
* what task 3.9's own testing already exercised, without the
* ceil(N/SK_MSG_PUMP_BATCH)-beat delivery latency the cap was
* accidentally imposing on kernel-Hermes drain specifically. */
uint32_t count = capsule_vm_registry_count();
uint32_t i;
while (count > 0 && dispatched < SK_MSG_PUMP_BATCH && scanned < count) {
for (i = 0; i < count; i++) {
VMRegistryEntry ent;
scanned++;
if (capsule_vm_registry_get_by_index(i, &ent) == 0 &&
ent.state == VM_STATE_LIVE &&
ent.vm_ptr != (void *)mama) {
/* FABRIC-3.6.md task 3.9, found live 2026-09-22: kernel-
* Hermes's own drain (sk_hermes_drain_checkpoint(), task
* 3.4) only ever runs as a side effect of vm_interpret()
* being called ON the target VM (vm_core.c's own outermost-
* interpret checkpoint hook) -- it has nothing to do with
* the old FORTH MSG-TICK word this loop was built around.
* A WIREBIND identity minted with the STD79-lockdown
* personality (MINT_PERSONALITY_STD79_LOCKDOWN) never loads
* common:messaging.4th, so the MSG-TICK gate just below
* always skips it -- meaning it never got an interpret
* tick at all, meaning a CONSOLE-CMD-EVENT queued for it
* sat in its pending queue forever, silently. Confirmed
* live: sk_hermes_send_one() returned 0 (queued OK) but no
* drain ever ran. Called unconditionally here, independent
* of the MSG-TICK check below -- kernel-Hermes doesn't need
* a FORTH word to exist, it just needs its target VM to get
* an outermost interpret call periodically. Direct C call,
* not routed through VM-EXEC/vm_interpret: drain_checkpoint
* already saves/restores the target's own interpret state
* internally (kernel_main.c's own self-test calls it the
* same standalone way), so no vm_interpret() wrapper is
* needed to reach it from here. */
(void)sk_hermes_drain_checkpoint((VM *)ent.vm_ptr);
/* FABRIC-3.md, 2026-09-09: a per-VM flag (e.g. "this one is
* std79-locked") doesn't generalize -- any VM whose own
* dictionary lacks MSG-TICK, for whatever reason (never
* loaded common:messaging.4th, a future personality that
* drops it, ACL denial), hits the identical failure. So
* check the actual target VM's own dictionary fresh every
* tick, the same source of truth the interpreter's own ACL
* enforcement uses (vm_core.c), rather than pre-flag specific
* personalities. Skip silently rather than spam "VM-EXEC:
* ERROR in <name>" every idle tick forever for a VM that
* plain doesn't have -- or isn't allowed -- MSG-TICK
* (live-caught 2026-09-07/09). */
DictEntry *msgtick = vm_find_word((VM *)ent.vm_ptr, "MSG-TICK", 8);
if (msgtick && msgtick->acl_allow) {
static const char PFX[] = "S\" MSG-TICK\" S\" ";
static const char SFX[] = "\" VM-EXEC";
char cmd[128];
size_t p = 0;
size_t name_len = strlen(ent.name);
if (name_len > VM_NAME_MAX - 1u) name_len = VM_NAME_MAX - 1u;
memcpy(cmd + p, PFX, sizeof(PFX) - 1u); p += sizeof(PFX) - 1u;
memcpy(cmd + p, ent.name, name_len); p += name_len;
memcpy(cmd + p, SFX, sizeof(SFX) - 1u); p += sizeof(SFX) - 1u;
cmd[p] = '\0';
vm_interpret(mama, cmd);
dispatched++;
}
}
i = (i + 1u) % count;
}
g_msg_pump_cursor = i;
}
/* FABRIC-3.md SXX: Hera now genuinely has her own MSG-ARENA/CH-ARENA
* (the messaging-load root cause above is fixed -- see kernel_main.c's
* own updated comment at the Hermes-birth call site), so her own
* queued messages need draining too, same as every other VM's. She's
* excluded from the loop above not because she lacks MSG-TICK now, but
* because she IS the pump -- self-targeting VM-EXEC would hit the same
* reentrancy class the guard above exists for. No such problem calling
* MSG-TICK directly here, though: it's a plain word in her own
* dictionary, not a VM-EXEC dispatch into anyone else's input buffer.
* Guarded the same defensive way as the loop above, not assumed. */
{
DictEntry *msgtick = vm_find_word(mama, "MSG-TICK", 8);
if (msgtick && msgtick->acl_allow) {
vm_interpret(mama, "MSG-TICK");
}
}
/* Hera's own kernel-Hermes queue needs draining too, same as every
* other VM's -- she is excluded from the loop above not because she
* has nothing to drain, but because she IS the pump: a self-
* targeting call here runs directly, no VM-EXEC/reentrancy concern
* (sk_hermes_drain_checkpoint() is a plain C call, not a dispatch
* into anyone else's input buffer). */
(void)sk_hermes_drain_checkpoint(mama);
g_idle_pump_active = 0;
if (console_tx_count() == tx_before_idle) {