Robert Allan JamesandClaude Opus 5.5
316be9fc4b
- The static word_scratch_addr shared by every VM is gone: WORD builds in vm->word_buf (257 bytes: count + 255 + NUL). A token over 255 chars is refused whole (delimiter restored, >IN unconsumed, vm->error), logged at LOG_ERROR and counted -- no more silent cut to 62 chars. - Pre-existing corruption found by the probe: TIB/>IN/SPAN (and WORD's scratch) were allotted lazily at HERE; POST's restore_dict_state(), FORGET and COLD rolled HERE below them and compiled the dictionary over them (Hera's TIB lay under ACL-RECHECK's body; QUERY/WORD wrote into code). Per Captain Bob, all four are now allotted first in both vm_init()s (offsets 0-1305); COLD's clamp never goes below them. Probe (reverted), identical on amd64/aarch64/riscv64: 0 colon bodies inside the buffers; per-VM isolation; 62/100/255-char tokens full length; 300 refused and counted; interpreter sane after full-width TIB writes. 3-ISA acceptance passes; M7.1a hash and dict_hash set unchanged (here 0xd8 -> 0x5f8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
416 lines
31 KiB
Markdown
416 lines
31 KiB
Markdown
# FABRIC-3.9.8.md — Observers and participants: execution plan and log
|
||
|
||
**Status: IN PROGRESS.** Done: O-A, the stack-convention fix, S-1a, S-1b and S-1c (§2). The design and
|
||
its rulings (O-1 to O-4, the discard, and §7/§8's M- and S- rulings) are in `FABRIC-3.9.0.md`.
|
||
|
||
Each stage:
|
||
- needs Captain Bob's explicit go;
|
||
- gets its own commit;
|
||
- gets its own three-architecture acceptance (`clean qemu`; amd64, then aarch64, then riscv64;
|
||
foreground; visible framebuffer; one at a time; coupled images reset before each).
|
||
|
||
Evidence is named **before** each stage runs.
|
||
|
||
**The standing acceptance sequence for every stage**, unchanged from `FABRIC-3.8.md`:
|
||
- a boot to `[zuse@Hera] ok>` with zero `UNKNOWN WORD`, faults and frame corruption;
|
||
- the `dict_hash` set identical across ISAs;
|
||
- the Stage G interleave (two workers, `A`/`B`), byte-identical across ISAs;
|
||
- mint → WIREBIND → `USE` → `1 2 + .` → `3`;
|
||
- 0 bytes of idle log growth over 20 s.
|
||
|
||
## 1. Staged plan
|
||
|
||
| Stage | Change | Evidence (named in advance) |
|
||
|---|---|---|
|
||
| **O-A** | Observer role (`FABRIC-3.9.0.md` §2.1). The absolute-heat-share charge is discarded (ruling 5). Observers are left out of the decision's candidates, the work-heat charge and the floor, and (O-2) the absolute charge. Observers with events go ahead at clean boundaries, among themselves by wait stamp (§2.4, per O-1). | *(Corrected before starting.)* The standard sequence, with the G interleave unchanged. **Console responsive under load:** `1 2 + .` typed while `wa`/`wb` run prints `3` in the middle of the A/B stream. **Observer precedence:** a message to Hestia sent in the same line as the two worker jobs runs before any `A`/`B`. **Not run: the busy-wait DoE driver.** Under O-1, Hera running `DOE` is an observer acting on an event, and nothing displaces her, so `MU-WAIT-ALL` would wait forever. The originally named evidence ("Hera's share falls on the unchanged busy-wait driver") contradicts the design. `DOE` stays unrun until O-C replaces the driver. |
|
||
| **M-0** *(first; a defect against the standing no-fixed-bounds rule)* | Kernel-Hermes arena, per-VM queues and channel member lists grow on demand; `Q_SLOT` decoupled from arena size (`FABRIC-3.9.0.md` §7.0). | The standard sequence. The existing kernel-Hermes boot self-tests (allocator, publish, drain, chunk, negotiation) still PASS, with `expected_n` now set by the reservoir, not the arena. A probe (reverted) admits **more than 32** messages system-wide, spread over senders with reservoir to pay, and more than 32 to one queue, with conservation and the ledger holding throughout. A channel takes more than 64 members. Nothing moves or corrupts across a grow: messages queued before a grow drain intact after it. |
|
||
| **M-A** *(ahead of O-B, per Captain Bob: "close the gap first")* | Manual messaging words over the existing C API (`FABRIC-3.9.0.md` §7.2, names per M-1). | The standard sequence. At the console: `CHANNEL`, then `JOIN-VM` of `wa` and `wb`, then `PUBLISH` of a marker payload delivers to both (`n = 2`, both print). `LEAVE-VM` then `PUBLISH` delivers to one. `CHANNEL-CLOSE` succeeds, and refuses channel 0. `INJECT` lands in a queue that `MAIL?`/`.MAIL` show before the recipient runs. `.CHANNELS` lists what was made. The `dict_hash` set changes by the new words and is identical across ISAs. |
|
||
| **M-B** | Autonomous events (§7.3): event channels created at boot; `EV-DONE`/`EV-BORN`/`EV-DIED` fired from the existing kernel points, for participants, funded per M-5, dropped and counted when full (M-6). | The standard sequence. A VM that `JOIN`s `EV-BORN`/`EV-DONE`/`EV-DIED` and defines the `ON-…` words sees one event for each `WORKER-BIRTH`, each job finished, and each `KILL`, with **no `SEND` anywhere in the job text**. A VM that didn't join sees nothing. The dropped-event counter reads 0 in normal use; a probe that floods the arena shows events dropped and counted, never blocking. |
|
||
| **O-B** | Hera's own mail drained at `sk_vm_idle_point()`, with the readline `n > 0` gate; the timer-paced pump drain removed (§2.5). | The standard sequence, plus: a message sent to Hera while she sits idle at the prompt is acted on **without waiting for the idle beat**. A probe records the delay in beats from admission to drain; it must be well under `SK_IDLE_BEAT_INTERVAL`. The relay and BLK-ATTACH flows are unchanged. |
|
||
| **O-C** | An event-driven DoE driver (§3, as revised by §7.4 once M-B lands): `DOE` starts trial 1 and returns to the prompt; Hera's `ON-DONE` (fed by `EV-DONE`) counts, ends trials and starts the next; `MU-WAIT-ALL` and the job text's own completion `SEND` are deleted. Capsule only. | A 1-rep smoke campaign on amd64 runs all 6 cells to `MU-CAMPAIGN: complete`, with every trial `fail=0` and N/N `KILL … dead`. Hera's rows during trials are only event handling. The prompt answers typed input while trials run. Then the standard sequence on all three ISAs. |
|
||
| **O-D** | The DoE relaunch: `12345 3 S" <isa>" DOE`, amd64, then aarch64, then riscv64, with the ring extracted between ISAs and `artemis.img` **not** reset in between. | A fresh time estimate from O-C's measured row rate, **before** launch. Then per ISA: 18 `MU-TRIAL` markers, `fail=0`, `MU-CAMPAIGN: complete`. |
|
||
| **Merge** | This branch to `master`, once all three ISAs have completed O-D. | Per this project's own merge practice, at Captain Bob's word. |
|
||
|
||
**Stop conditions** (re-plan rather than push on):
|
||
- `dict_hash` diverges across ISAs;
|
||
- conservation (`vm_physics_conserved()`, `stadium_conserved()`) goes false;
|
||
- an observer is ever returned as a heat candidate;
|
||
- a switch storm: log growth at an idle prompt, with QEMU pinned;
|
||
- a trial that never produces its marker.
|
||
|
||
## 2. Execution log
|
||
|
||
- [x] **Stage O-A — the observer role (`FABRIC-3.9.0.md` §2).** 2026-09-24. Captain Bob: *"all as
|
||
proposed, go ahead with O-A"* (O-1 to O-4 and the discard, all as proposed).
|
||
- **Discarded:** the uncommitted absolute-heat-share charge (`git checkout` of
|
||
`capsule_vm_physics.c/.h`), per ruling 5.
|
||
- **`capsule_vm_switch_signal.c/.h`:**
|
||
- Each slot carries an `observer` flag, with `sk_vm_switch_signal_set_observer()` and
|
||
`is_observer()`.
|
||
- `sk_vm_switch_decide()` now works in tiers:
|
||
1. An observer that is itself in progress (a live non-IDLE frame) or holding an event
|
||
keeps the CPU.
|
||
2. Otherwise, an observer with an event, meaning mail (or console input, for Hera),
|
||
goes ahead of every participant. Among observers, the oldest wait stamp wins.
|
||
3. Otherwise, participants are compared by work heat exactly as in FABRIC-3.8.
|
||
Observers are never heat candidates.
|
||
- `note_work_gained()` never floors an observer and leaves observers out of the floor
|
||
set. The running VM counts only if it is a participant.
|
||
- **`capsule_birth.c`:** `is_fleet_foundation` is hoisted to function scope. Registration
|
||
marks the Tripod as observers from that same birth-time fact.
|
||
- **`kernel_main.c`:** Hera is marked an observer where she registers.
|
||
- **`vm_runtime.c`:** observers are not charged at all (O-2).
|
||
- Nothing a VM executes can set or clear the role.
|
||
|
||
**Evidence, as corrected before starting (§1).** Ordinary words only, typed at
|
||
`[zuse@Hera] ok>`, byte-identical on all three ISAs (`logs/20260924-084904/amd64/`,
|
||
`-090107/aarch64/`, `-091424/riscv64/`):
|
||
- **G interleave unchanged:** A = 40, B = 40, 48 runs.
|
||
- **Observer precedence:** `S" LA" S" wa" SEND . S" LB" S" wb" SEND . S" 72 EMIT 72 EMIT
|
||
72 EMIT" S" Hestia" SEND .` gives `HHHABBAABBAA…`. Hestia's mail ran before any
|
||
participant output once Hera went idle.
|
||
- **Console responsive under load:** with `wa`/`wb` on 200 markers × 20,000-iteration
|
||
loops, `1 2 + .` typed mid-run appears as `…ABABAB1 2 + .` / `3 ok` / `ok> ABABBA…`.
|
||
Hera, an observer with console input, got the CPU at the next worker boundary, ran the
|
||
line, and the workers resumed. This is the C-4 (α) starvation recorded at C-b, fixed.
|
||
*(The first amd64 attempt at this check used loops that finished before the line was
|
||
typed, so it proved nothing. It was re-run with the heavy loops, and the script was fixed
|
||
for the other two ISAs.)*
|
||
|
||
**Acceptance** (same boots):
|
||
- All reach `[zuse@Hera] ok>` with zero `UNKNOWN WORD`, faults and frame corruption.
|
||
- The `dict_hash` set is unchanged from Stage I (word_count 531, M7.1a
|
||
`0x572daac83b797332`), since no words were added or removed.
|
||
- Mint → WIREBIND → `USE` → `1 2 + .` → `3`.
|
||
- Idle log growth: 0 bytes.
|
||
- Clean build on all three plus hosted, no new warnings.
|
||
|
||
**Not run, by design:** the busy-wait DoE driver, which would deadlock under O-1 (§1).
|
||
Don't run `DOE` until O-C.
|
||
- [x] **Stack-convention fix: out-of-bounds stack writes in four vendored word files** (found
|
||
during S-1a, 2026-09-24). Captain Bob: *"Fix now, own commit."*
|
||
|
||
**The defect.** The VM's `dsp` is the index of the top item (−1 = empty):
|
||
`vm_pop` reads `data_stack[dsp--]`. Four vendored files instead treated `dsp` as a count,
|
||
pushing with `data_stack[vm->dsp++] = x` (11 sites) and popping with
|
||
`data_stack[--vm->dsp]` (17 sites):
|
||
- `physics_freeze_words.c`: 7 pushes, 12 pops;
|
||
- `dictionary_heat_diagnostic_words.c`: 4 pushes, 2 pops;
|
||
- `physics_diagnostic_words.c`: 1 pop;
|
||
- `physics_benchmark_words.c`: 1 pop.
|
||
|
||
**The consequences:**
|
||
- On an empty stack a push wrote `data_stack[-1]`, **8 bytes before the VM struct**. POST's
|
||
own `DECAY-RATE@.basic` test does exactly that on every boot.
|
||
- Otherwise a push overwrote the top item and left `dsp` one slot off.
|
||
- The pops read the item *below* the top, and on the last pop, `data_stack[-1]`.
|
||
- Four one-item words also guarded with `dsp < 1`, which demands two items, so
|
||
`LOOKUP-STRATEGY!`, `COMPARE-LOOKUPS`, `PHYSICS-BURN` and `BENCH-DICT-LOOKUP` always
|
||
errored on valid input.
|
||
|
||
**How it was found.** The S-1a parity probe's incomplete counter read one too many on
|
||
aarch64 only.
|
||
- A per-test hook in the test runner named `DECAY-RATE@.basic` as the test after which
|
||
the counter moved (`logs/20260924-104424/aarch64/`).
|
||
- Writing a marker into `mama->data_stack[-1]` changed the counter to the marker
|
||
(`-105120`).
|
||
- Printing runtime addresses (`-105729`) showed why aarch64 only: the aarch64 kernel runs
|
||
inside the `lld-link` PE, whose `.bss` puts `parity.c`'s counter directly before
|
||
`sk_mama_vm`.
|
||
- On amd64 and riscv64 the same write lands in the last cell of the POST block-RAM buffer,
|
||
silently.
|
||
- An interim hypothesis of memory aliasing, drawn from ELF symbol addresses that don't
|
||
describe the aarch64 runtime image, was wrong and is withdrawn.
|
||
|
||
**The fix:**
|
||
- Every site now uses the VM's own `vm_pop(vm)` / `vm_push(vm, x)`, which check bounds.
|
||
- The four one-item guards are now `dsp < 0`.
|
||
- The two- and three-item guards (`dsp < 1`, `dsp < 2`) were already correct for the
|
||
top-index convention and are unchanged.
|
||
|
||
**Evidence.** A probe (reverted) compared the 8 bytes before the VM struct across POST,
|
||
plus console checks, on all three ISAs (`logs/20260924-111124/amd64/`, `-112508/aarch64/`,
|
||
`-113949/riscv64/`):
|
||
- `STACKFIX PROBE: … UNCHANGED across POST`;
|
||
- `DECAY-RATE@ DEPTH . .` gives `1 1`;
|
||
- `S" DUP" FROZEN? DEPTH . .` gives `1 0`;
|
||
- `HEAT@` leaves 1 item, and `HEAT-PERCENTILES` leaves 3;
|
||
- `1 LOOKUP-STRATEGY! LOOKUP-STRATEGY@ .` gives `1`, depth 0, where it used to error;
|
||
and `0 …` gives `0`.
|
||
|
||
Acceptance in the same boots:
|
||
- the O-A checks: G interleave A = 40/B = 40/48 runs, `HHH` precedence, and the typed line
|
||
running mid-stream;
|
||
- relay `3`, idle growth 0;
|
||
- `dict_hash` unchanged (531, `0x572daac83b797332`);
|
||
- POST passes;
|
||
- clean kernel and hosted builds.
|
||
- [x] **Stage S-1a — parity hashing covers the whole dictionary (`FABRIC-3.9.0.md` §8.1, class C).**
|
||
2026-09-24. Captain Bob: *"all as proposed, commit and start S-1."*
|
||
|
||
**The change** (`src/starkernel/vm/parity.c`, `include/starkernel/vm/parity.h`):
|
||
- **`MAX_DICT_ENTRIES` (2048) removed.** The walk covers the whole link chain. The entry list
|
||
is allocated to the dictionary's actual size (`kmalloc` in the kernel, `malloc` hosted) and
|
||
freed after.
|
||
- **Cycle check replaces the cap.** The only thing the cap guarded against, a corrupt cyclic
|
||
chain, is now detected directly (Floyd: a second pointer moving two links per step).
|
||
- **The colon-body walk is bounded by `vm->here`,** not a 1024-cell cap. A body that reaches
|
||
`HERE` without `EXIT` or a NULL cell is marked incomplete, not quietly truncated.
|
||
- **Failures follow §8.1's rules.** Every failure (cyclic chain, entry list not allocated,
|
||
malformed body) is:
|
||
- printed in full to the console, `PARITY:INCOMPLETE #n -- <why>`. `log_message`'s line
|
||
buffer is 64 bytes and cut every reason off mid-sentence.
|
||
- logged at `LOG_ERROR` for the persisted record;
|
||
- counted (`sk_parity_incomplete_count()`).
|
||
- **An incomplete hash is marked as such.** The record prints `hash=INCOMPLETE` (new
|
||
`ParityPacket.hash_complete`) and `PARITY:FAIL code=4`
|
||
(`SK_BOOTSTRAP_PARITY_INCOMPLETE`), never a value that looks valid.
|
||
- `sk_dict_word_count` returns 0 on a cyclic chain, logged and counted.
|
||
|
||
**Two pre-existing defects found by the probe, both fixed in the walker:**
|
||
1. **The colon-body walker dereferenced inline data.** It treated every cell of a body as a
|
||
`DictEntry*` and followed it. Only `LIT`'s operand was known to be data. `S"` string
|
||
bytes, branch offsets and other inline operands were followed too.
|
||
- This was harmless while parity only ever ran at early boot, before any colon bodies
|
||
existed.
|
||
- The first hash of a mature dictionary took a #GP (vector 13) and halted the kernel. This
|
||
is the "interrupt exception" Captain Bob reported (`logs/20260924-094537/amd64/`,
|
||
`-095154/amd64/`).
|
||
- The **original** code faults the same way (`logs/20260924-095730/amd64/`), so the
|
||
defect predates S-1a.
|
||
- **Fix:** a cell is dereferenced only if it is a known dictionary entry, found by binary
|
||
search in a sorted copy of the entry list. Any other cell is hashed as its raw value and
|
||
never followed.
|
||
2. **A `LIT` operand holding an entry pointer was hashed by address.** `['] WORD` compiles
|
||
`LIT <DictEntry*>`, a native address that differs per ISA (seen in `ACL-BOOT`'s
|
||
`ACL-PIN` sequences). So the mature-dictionary hash differed across ISAs:
|
||
- amd64 `0x945a76bc6ace1fdc`, aarch64 `0x8cc1255e6a40c12e` (`-100252`, `-100732`).
|
||
- **Fix:** rule 2 (word_id, never address) now applies to such an operand. The walker
|
||
hashes a tag byte (`0xE7`) plus the entry's `word_id`, so it can't collide with a plain
|
||
number. Any other literal is still hashed as its value.
|
||
|
||
A third defect, an incomplete counter one too high on aarch64 only (`-101803`, `-102504`,
|
||
`-103207`, `-103820`), was not in parity. It was the out-of-bounds stack write fixed in its
|
||
own commit (the entry above).
|
||
|
||
**Evidence.** A probe (reverted) at the end of boot:
|
||
1. hashed Hera's full dictionary;
|
||
2. registered 2,000 more words (`PZ0000`…) and hashed again;
|
||
3. made the chain cyclic and asked for both the word count and the hash.
|
||
|
||
The results are byte-identical on all three ISAs (`logs/20260924-115524/amd64/`,
|
||
`-115954/aarch64/`, `-120533/riscv64/`):
|
||
|
||
| Probe step | Result |
|
||
|---|---|
|
||
| before | `word_count=563 hash=0x224007f9af328111`, `PARITY:OK` |
|
||
| after +2000 | `word_count=2563 hash=0xcf85876337e75a84`, `PARITY:OK`. Past the old 2048 cap: the whole dictionary is hashed. |
|
||
| cyclic | `PARITY:INCOMPLETE #1`/`#2 -- dictionary link chain is cyclic (corrupt)`, two `[HADES][ERROR]` lines, `word_count=0 hash=INCOMPLETE`, `PARITY:FAIL code=4` |
|
||
| counter | `incomplete_count=2` (word count + hash) |
|
||
|
||
The "after" record shows `latest_id=4294967295`. That is `WORD_ID_INVALID`: the 2,000
|
||
probe words ran past the 1024 stable word ids, and each logged "out of stable IDs, disabling
|
||
metrics". That is S-1b's defect, not S-1a's. The §8.1 row now notes that this path does log
|
||
a warning, but neither counts nor refuses.
|
||
|
||
**Acceptance** on the clean tree, with the probe reverted (`logs/20260924-121207/amd64/`,
|
||
`-122500/aarch64/`, `-123759/riscv64/`):
|
||
- Boot M7.1a is unchanged: `word_count=531 hash=0x572daac83b797332`. At boot there are no
|
||
colon bodies or entry-pointer literals yet, so neither fix changes the boot hash.
|
||
- The `dict_hash` set is unchanged: Hera `0xda9742ce9c3376a2`, Hestia `0x413e7d086f342d8b`,
|
||
Artemis `0xdfd68316d5f5ba85`.
|
||
- The O-A checks pass: G interleave A = 40/B = 40/48 runs, `HHH` precedence, and a typed
|
||
`1 2 + .` gives `3` mid-stream.
|
||
- Mint → WIREBIND → `USE` → `1 2 + .` → `3`. Idle log growth 0. Zero `UNKNOWN WORD` or
|
||
faults. QEMU exits cleanly.
|
||
- [x] **Stage S-1b — dictionary word ids grow with the dictionary (`FABRIC-3.9.0.md` §8.1, class C).**
|
||
2026-09-24. Captain Bob: *"go ahead with S-1b."*
|
||
|
||
**The defect.** Stable word ids came from fixed `word_id_map[1024]` /
|
||
`recycled_word_ids[1024]` tables in the `VM` struct. Word 1025 onward got `WORD_ID_INVALID`
|
||
and a `LOG_WARN` ("out of stable IDs, disabling metrics"), but was still defined. It then ran
|
||
outside the physics:
|
||
- the rolling window and the pipelining hooks skipped it;
|
||
- Stadium never admitted it;
|
||
- parity hashed it as `0xFFFFFFFF`.
|
||
|
||
Nothing counted it and nothing refused it.
|
||
|
||
**The change:**
|
||
- **`vm.h` / `dictionary_management.c`: the core.**
|
||
- `word_id_map` and `recycled_word_ids` are heap tables sharing one `word_id_capacity`.
|
||
- They start at 64 and double when a fresh id reaches capacity.
|
||
- Growth is all-or-nothing: both new tables are allocated before either old one is touched.
|
||
It uses allocate-copy-free, because the kernel's `sf_realloc()` does not copy.
|
||
- Numbering is unchanged (a recycled id first, else `next_word_id++`). That is why boot
|
||
parity is unchanged.
|
||
- Both `vm_cleanup()`s free the tables.
|
||
- **Refusal is whole.**
|
||
- `vm_dictionary_track_entry()` now returns a status.
|
||
- `vm_create_word()` takes the id *before* linking the entry. On failure it frees the entry
|
||
and its transition metrics, sets `vm->error`, and returns NULL.
|
||
- The failure is logged at `LOG_ERROR` ("word id table can't grow: '…' refused") and counted
|
||
(`vm_dictionary_word_id_refusals()`).
|
||
- Every caller already handled NULL from the existing malloc-failure and pinned-shadow paths.
|
||
- **The bound for every reader of the id space is `vm->next_word_id`,** not `DICTIONARY_SIZE`.
|
||
Every issued id, live or recycled, is below it:
|
||
- `lookup_by_word_id`, `untrack`;
|
||
- both interpreters' physics hooks (`physics_execution_hooks.c`, `vm_core.c`);
|
||
- the rolling-window seed (its frequency array is now sized to the bound);
|
||
- `inference_engine.c`;
|
||
- the pipelining diagnostics.
|
||
- **`transition_heat` (`physics_pipelining_metrics.c`).** Each word's array was sized once, at
|
||
whatever size its first record passed in. With a dynamic bound, that would have been a heap
|
||
overflow on the first transition to a newer id.
|
||
- It now has its own `transition_heat_cap` and grows on record (to the bound, at least
|
||
doubling).
|
||
- Every reader bounds by that field. `get_probability` had no bounds check at all.
|
||
- A failed grow drops that one transition (the word already ran; nothing is left to refuse).
|
||
It is logged and counted (`transition_metrics_grow_failures()`).
|
||
- **Untested at runtime.** Pipelining is compiled out of the kernel *and* the default
|
||
hosted build (`ENABLE_PIPELINING=0` in both), so this grow path is dead in every shipped
|
||
configuration. No run exercised it. It is corrected, not proven.
|
||
- **Stadium's id→cell rows (`stadium_words.c`).** These were `[slot][DICTIONARY_SIZE]`.
|
||
- Each row now starts empty and grows on dispatch (`row_cap[slot]`), not at definition,
|
||
because a quota slot can be granted after a VM's words exist, and slots are reused across
|
||
VMs.
|
||
- A failed grow is logged and counted (`stadium_words_row_grow_failures()`). The word is not
|
||
admitted that dispatch, which is the same, already-conserving outcome as a full Stadium.
|
||
- This is the id→cell map, not the cell pool, so **S-2 stays open**.
|
||
- `DICTIONARY_SIZE` stays defined for its one remaining user, reported below.
|
||
|
||
**Evidence, named before the run.** A probe (reverted) at the end of boot, byte-identical on
|
||
all three ISAs (38 lines; `logs/20260924-130702/amd64/`, `-131140/aarch64/`,
|
||
`-131624/riscv64/`):
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| Start | `next_word_id=738`, capacity 1024 (grown 64 → 1024 during boot) |
|
||
| +2000 words | `next_word_id=2738`, capacity 4096, `PZ1999 word_id=2737`, refusals 0, and no "out of stable IDs" line. Parity OK, `word_count=2563`, `latest_id=2737` (was `4294967295` at S-1a), `hash=0xc873c20962d728ed`. The hash differs from S-1a's `0xcf85…` as expected: 1,539 words now hash real ids instead of `0xFFFFFFFF`. |
|
||
| Physics past id 1024 | `: PZHOT 1 DROP ; : PZRUN 200 0 DO PZHOT LOOP ; PZRUN` gives PZHOT id 2738 and PZRUN id 2739; `lookup(2738)==PZHOT`; **200** PZHOT entries in the rolling window (the hooks used to skip any id ≥ 1024); Stadium promotions +2 (both words admitted); row-grow failures 0. |
|
||
| FORGET/recycle | `FORGET PZHOT` leaves 2 recycled ids and `lookup(2738)==NULL`. `: PZNEW ;` reuses id 2738, and `lookup` returns PZNEW. |
|
||
| Forced grow failure | After filling the tables to capacity (4096), one allocation failure was injected into the grow. `PZFAIL` then gives NULL, `vm->error=1`, refusals 1, and `[HADES][ERROR] word id table can't grow: 'PZFAIL' refused`. `latest`, `word_count`, capacity and `next_word_id` are all unchanged, and `PZFAIL` is not findable. |
|
||
| Retry | With the injection cleared, `PZOK` is defined: id 4096, capacity 8192. |
|
||
| Final parity | `word_count=3922`, `hash=0x5af2d7e820d21624`, `incomplete_count=0` |
|
||
|
||
**Acceptance** on the clean tree, with the probe reverted (`logs/20260924-132055/amd64/`,
|
||
`-133339/aarch64/`, `-134638/riscv64/`):
|
||
- Boot M7.1a is unchanged (`word_count=531`, `latest_id=700`, `0x572daac83b797332`), and so is
|
||
the `dict_hash` set (Hera `0xda9742ce9c3376a2`, Hestia `0x413e7d086f342d8b`, Artemis
|
||
`0xdfd68316d5f5ba85`).
|
||
- The O-A checks pass: G interleave A = 40/B = 40/48 runs, `HHH` precedence, and a typed
|
||
`1 2 + .` gives `3` mid-stream.
|
||
- Mint → WIREBIND → `USE` → `3`. Idle log growth 0. Zero `UNKNOWN WORD` or faults. QEMU exits
|
||
cleanly.
|
||
- Clean kernel builds on all three ISAs, plus the hosted build.
|
||
|
||
**Found, reported, not fixed:**
|
||
1. `dictionary_heat_optimization.c:123`: `cell_t heats[DICTIONARY_SIZE]` snapshots at most
|
||
1024 words' heat and silently ignores the rest. That is class C, keyed on the dictionary
|
||
count, not on ids.
|
||
2. `vm_create_word()`: a failed transition-metrics malloc logs a WARN and defines the word
|
||
anyway, with no pipelining metrics. That is a partial.
|
||
3. Neither `vm_cleanup()` frees `transition_heat` (or `context_window`) inside each word's
|
||
metrics, only the metrics struct itself. That is a leak on every VM kill with pipelining on.
|
||
4. `transition_metrics_update_cache()` scans the whole id space on every recorded transition.
|
||
The hot-path cost grows with the dictionary, now that the dictionary can grow.
|
||
5. The kernel's FC-bucket grow (`sf_fc_reserve`) copies after `sf_realloc()`, which is correct,
|
||
but never frees the old block. Each bucket grow leaks it.
|
||
6. §8.1 asks that each failure counter be readable by a FORTH word or DoE row. S-1a and S-1b
|
||
expose C accessors only (`sk_parity_incomplete_count()`,
|
||
`vm_dictionary_word_id_refusals()`, `transition_metrics_grow_failures()`,
|
||
`stadium_words_row_grow_failures()`). Adding a word changes the `dict_hash` set, so this
|
||
awaits Captain Bob's ruling.
|
||
|
||
**Follow-up review (same day, own commit; acceptance `logs/20260924-140359/amd64/`, `-141643/aarch64/`, `-142946/riscv64/`: boot parity and `dict_hash` set unchanged, O-A checks, relay `3`, idle growth 0, no faults):**
|
||
- **The refusal path leaked each metrics' `context_window`.** It freed the
|
||
`WordTransitionMetrics` struct without `transition_metrics_cleanup()`. That call is now made
|
||
first.
|
||
- **Moving tables were checked for concurrent readers.** Before S-1b the id map never moved;
|
||
now a grow frees the old map, and likewise the Stadium rows. Every id lookup off the
|
||
interpreter path runs under `dict_lock`: the heartbeat decay cursor (`vm_time.c`,
|
||
`vm_runtime.c`), the rolling-window seed, and inference. `vm_create_word()` holds that
|
||
lock across the grow. In the kernel, the heartbeat is mainline and execution-paced, not an
|
||
interrupt, and Stadium rows are touched only from the interpreter and
|
||
`stadium_words_resident_heat`'s FORTH word. No interrupt reads either.
|
||
- **Sweep for other arrays indexed by word ids** (not only those named `DICTIONARY_SIZE`):
|
||
- `rolling_window_find_hottest_word()` sizes by its own `dict_size` parameter and has no
|
||
callers.
|
||
- No word-id field narrower than 32 bits.
|
||
- Nothing else found.
|
||
- [x] **Stage S-1c — `WORD`'s buffer is per VM, and the input buffers sit below all compiled
|
||
code (`FABRIC-3.9.0.md` §8.1, class C).** 2026-09-24. Captain Bob: *"go ahead with S-1c"*,
|
||
then, on the defect below, *"Allot at VM init"*.
|
||
|
||
**The defect as scoped.** `WORD` built its counted string in one `static vaddr_t
|
||
word_scratch_addr` shared by every VM:
|
||
- It was `vm_allot`'d once, in whichever VM called first, at wherever that VM's `HERE` stood.
|
||
- A second VM got an address meant for someone else's space.
|
||
- Tokens were silently cut to 62 characters (a 64-byte buffer).
|
||
|
||
**The deeper defect, found by the first probe.** The input buffers were allotted lazily by
|
||
`vm_input_ensure()` at the current `HERE`: the TIB (`INPUT_BUFFER_SIZE`), `>IN`, `SPAN`, and
|
||
`WORD`'s scratch.
|
||
- Three things later roll `HERE` back below them and compile the dictionary over them: POST's
|
||
test harness (`restore_dict_state()`), `FORGET`, and `COLD` (which clamps `HERE` to 1024).
|
||
- On Hera, POST allotted the TIB at offset 120 and then restored `HERE`. **`ACL-RECHECK`'s
|
||
colon body lived at offset 824, inside the TIB.**
|
||
- The first S-1c probe wrote the TIB, as `QUERY` does, which zeroed that body. Hera hit
|
||
`execute_colon_word: NULL cell in 'ACL-RECHECK'` and never reached her prompt
|
||
(`logs/20260924-151033/amd64/`; layout diagnostic `-152047/amd64/`).
|
||
- So before this stage, `QUERY` wrote into Hera's compiled code, and so did `WORD` through
|
||
the old static scratch. Nothing in a normal boot calls either afterwards, which is why
|
||
acceptance never saw it. The first draft of the per-VM buffer, allotted the same lazy way,
|
||
inherited the defect.
|
||
|
||
**The change:**
|
||
- **`vm.h`:**
|
||
- `VM.word_buf`, one per VM.
|
||
- `WORD_BUFFER_SIZE 257`: a count byte, 255 characters (the most a counted string can
|
||
hold, a format limit, not a chosen bound), and a NUL.
|
||
- **`vm_api.c`:** `vm_input_ensure()` also allots `word_buf`, after `SPAN`, so TIB/`>IN`/
|
||
`SPAN` keep their relative layout.
|
||
- **Both `vm_init()`s** (`src/vm_bootstrap.c`, `src/starkernel/vm/vm_bootstrap.c`) call
|
||
`vm_input_ensure()` first, before POST, capsules or any definition.
|
||
- The buffers take offsets 0–1305 in every VM.
|
||
- Every later `HERE` rollback (test restore, `FORGET`) lands above them.
|
||
- If the allotment fails, `vm_init` fails with an error.
|
||
- **`COLD`** (`system_words.c`) still clamps `HERE` to 1024, but never below the end of the
|
||
input buffers.
|
||
- **`string_words.c`:** the static is gone. `WORD` builds into `vm->word_buf`.
|
||
- A token over 255 characters is **refused whole**: the delimiter goes back on the stack,
|
||
`>IN` is not consumed, and `vm->error` is set.
|
||
- The refusal is logged at `LOG_ERROR` ("WORD: token over 255 chars refused") and counted
|
||
(`string_word_word_refusals()`).
|
||
|
||
**Evidence** (probe, reverted; byte-identical on all three ISAs, 37 lines;
|
||
`logs/20260924-152935/amd64/`, `-153409/aarch64/`, `-153842/riscv64/`):
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| Layout, Hera and Hestia | TIB offset 0, `>IN` 1032, `SPAN` 1040, `word_buf` 1048, end 1305. Colon bodies inside the buffers: **0** (Hera 26 checked, Hestia 179). |
|
||
| Isolation | Hera's `WORD` returns her own `word_buf`, in her own memory. Hestia's returns hers, in hers, and the two are distinct. Hera's `HERAWORD` is intact after Hestia parses `HESTIAWORD`. |
|
||
| Length | 62, 100 and 255 characters come back at full length with the correct last character. It used to be 62 at most. |
|
||
| 300-char token | Refused, `vm->error=1`, refusals 1, the ERROR line logged, the delimiter restored, `>IN` 0. |
|
||
| After the probe | The probe wrote the TIB full-width, and `1 2 + .` still prints `3`. Hera reaches `[zuse@Hera] ok>`. |
|
||
|
||
**Acceptance** on the clean tree (`logs/20260924-154256/amd64/`, `-155542/aarch64/`,
|
||
`-160845/riscv64/`):
|
||
- Boot M7.1a `word_count=531`, hash `0x572daac83b797332`, unchanged. Only its `here` moves,
|
||
from `0xd8` to `0x5f8`, the 1,305 buffer bytes now allotted first.
|
||
- The `dict_hash` set is unchanged (Hera `0xda9742ce9c3376a2`, Hestia `0x413e7d086f342d8b`,
|
||
Artemis `0xdfd68316d5f5ba85`). The canonical hash walks word ids and body cells, and no
|
||
colon body carries an absolute address that the shift would change.
|
||
- The O-A checks pass: G interleave 40/40/48, `HHH` precedence, and a typed `1 2 + .` gives
|
||
`3` mid-stream.
|
||
- Mint → WIREBIND → `USE` → `3`. Idle log growth 0. No faults. QEMU exits cleanly.
|
||
- Clean kernel builds on all three ISAs, plus the hosted build.
|
||
|
||
**Found, reported, not fixed:**
|
||
1. **`COLD` rewinds `HERE` to 1024 without rewinding `latest`.** Every word defined past 1024
|
||
stays linked while new definitions are compiled over its body. S-1c only keeps the clamp
|
||
off the input buffers.
|
||
2. **`PAD` overlaps the block I/O window.** It returns `VM_MEMORY_SIZE - 512`, which is inside
|
||
the top block slot (`BLK_VM_WINDOW_BASE` = `VM_MEMORY_SIZE - 4096`), so `PAD` and the
|
||
4th `BLOCK` buffer share bytes.
|