Files
LithosAnanke/docs/v4.0.0/plans/2026-11-13-engine-step4-phase7-the-loops-and-parity.md
T
Claude 5cac792627 docs(v4): whole-POST parity ruled -- v3's table, not v3's bugs (#189)
Captain Bob, 2026-11-14: where v3 does not do what its own table says, v4 is held to the table; where a
case's output names v3's internals, to the same form with v4's values; both listed. POST-PARITY.md, the
Phase 7 plan's Task 8. Not pushed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
2026-11-14 02:19:49 +00:00

17 KiB

ENGINE Step 4, Phase 7: the physics loops on the word records, and v3's whole POST -- Plan

For agentic workers: steps use checkbox (- [ ]) syntax. One QEMU at a time, in the foreground, every boot under timeout 570 (.claude/CLAUDE.md, .claude/memory/rules.md); every log and CSV is committed. Test first: each change starts with a failing test.

Goal (#189): every loop v3 runs before its prompt runs on v4, on the kernel's word records (Phase 2), on every node; and v3's whole POST passes on v4.

Parent plan: 2026-10-20-engine-step4-node-in-the-vms-place.md, Phase 7; after Phase 2 (2026-11-13-engine-step4-phase2-word-accounts.md, protocol PASS on e8e7ec8b).

Rulings (Captain Bob, 2026-11-13)

  1. Every call, as v3. The per-call loops (decay on touch, the rolling window, pipelining) run at every call, from the hook, as v3's run at every execution -- not replayed from a trace each tick. Measured, kept or reverted on the numbers (as HEAT-OP was).

  2. Rewrite for v4. The loops are written afresh for v4's records, with v3's code (v3/src/rolling_window_of_truth.c, physics_pipelining_metrics.c, inference_engine.c, physics_metadata.c, physics_execution_hooks.c, compudynamics.c, ssm_jacquard.c, and the kernel's vm_runtime.c tick) and v3's POST as the specification.

  3. Accepted on v3's whole POST. Not the loops' own modules alone: every module of v3's POST passes on v4.

  4. Words in F18 and StarForth, not C (2026-11-13, on the Mama module: "do not compose words in C99, use F18 and StarForth"). Every word this phase adds -- the loops' (FREEZE-WORD, HEAT@, the inference words, the entropy words), Mama's, the StarForth words -- is written in the system's F18 or in StarForth. What only the kernel holds (a word's record, the window, the registry) is asked for by a request on port 0, as KERNEL-WORD words do. The loops themselves run in the kernel, on its records (Q4).

  5. Mama's 20 cases run as they are, on v4 words with v3's names mapped to v4's equivalents; the C modules (ACL, break_me) are rewritten for v4. Inventory and rulings: docs/v4.0.0/POST-PARITY.md.

  6. Pipelining and the hot-words cache: "Built, off as v3" (2026-11-13). Asked first "Build a hot-words cache" (in front of FIND, in F18/StarForth, as v3 promotes into its own); then it was found that v3 as built has both off (ENABLE_PIPELINING, ENABLE_HOTWORDS_CACHE: n in Kconfig.physics and kernel/Makefile), so v3's boot records no transitions and uses no cache. Ruled: both written and tested, behind the same two build options, off by default as in v3.

  7. Compudynamics on the F18 opcodes as well as the composed words (2026-11-14: "we're supposed to be working on f18 opcode as well as our composed f18 dictionary"; ruled "#1 and an easy revert if opcodes are too granular"). Every loop runs on both: the 32 opcodes are a second set of records on each node, fed at every retired instruction (a hook at retire, as Part A's at call) -- heat, decay, temperature, an opcode window, opcode transitions (under L4, as the words'), background decay and inference in the cycle. L8 scores from the words, as v3, and its card's gates govern both. The tick stays paced by word calls, as v3's. The revert: one build option, OPCODE_PHYSICS (Kconfig.physics, default y); off, the opcodes are counted and charged to the Stadium as in Phase 1, and nothing more. Where the arithmetic lives: compudynamics is the Stadium/fabric's (2026-11-14: "The compudynamics is stadium/fabric"): the kernel computes; a word that reads or sets it is F18 that asks the kernel.

What ruling 3 brings in

v3's POST is 26 modules (v3/src/test_runner/modules/, about 1,050 cases; v3's bare-metal boot says 1050). v4's POST is v3's cases for the FORTH-79 Required Word Set, 539 (v4/tools/mkpost.py). The rest need words v4 does not have yet, which the parent plan put in other phases:

v3's modules Needs Parent plan's phase
physics_freeze, inference the loops, and the words that read and set them (FREEZE-WORD, ...) 7
starforth, q48, format, double (beyond the Required set), string, system, vocabulary, ... the StarForth words: init.4th, starforth.4th 3
acl ACL.4th on v4; the word card (the four ACL fields) on the record; the countdown at call 4
block the blocks the kernel serves; writes 5 (in part)
mama_forth, integration, break_me, stress to be read case by case (Task 0): some name v3's VMs and fleet --

So, by ruling 3, Phase 7 is accepted only when Phases 3 and 4 (and the block cases of 5) are in. The ruled order (Part A, Phase 1, Phase 2, Phase 7) stands for the loops; the words of 3 and 4 come in under this plan, as tasks of it.

The design

The records grow v3's physics. A word's record (v4_mesh_account, Phase 2) gains what a DictEntry carries for the loops: temperature, the last decay tick, mass, the state flags (frozen, pinned), and its transition metrics (by next word ID). Per node: one rolling window, one pipelining context, one L8 mode, one inference state.

At every call (the hook; ruling 1), in v3's order (V3-PARITY.md 2.1): decay the word by the ticks since it was touched (Loop #3, unless frozen or L8 has it off); its heat +1 (Loop #1, done); the Stadium (Phase 2 charges per tick: kept); the rolling window records its ID (Loop #2); the transition from the word before is recorded and the likely next worked out (Loop #4); the ACL countdown (with Phase 4).

At every heartbeat tick, for every node: the window tuner (Loop #5), the slope validator (Loop #6), background decay, L8's mode update, the inference engine, the snapshot -- as vm_runtime.c does for v3's VM.

Tasks

Task 0: the inventory

  • Every case of v3's POST, by module, in v3's own order: kept as it is, kept with v4's address units (D-1), needs a word of Phase 3, 4, 5 or 7, or names v3's VMs (Mama, the fleet) -- the last listed for Captain Bob's ruling, case by case, before anything is written for them.
  • docs/v4.0.0/POST-PARITY.md: the table, and the count v4 must reach (about 1,050; 539 today). Mama and the C modules ruled (above).

Task 1: the records' physics fields, and the per-node state

  • Test first (test_mesh_boot.c): a new word's heat, temperature, last decay tick and state flags are 0; each node starts with v3's decay slope (21845, Q48.16) and Loop #3 on; a word's state flags as v3 derives them (IMMEDIATE read from its header).
  • v4_mesh_account gains heat (v3's execution_heat), last_decay_tick, temperature_q8, state_flags (the kernel's own: pinned, frozen); v4_mesh_physics per node (tick, decay slope, L8's Loop #3 gate); v4_mesh_state_flags. Immediate and hidden are read from the node's header when asked, so they are never stale. v3's mass_bytes is only ever set, never read, in v3: not kept until a word reads it.

Task 2a: the per-call loops at call -- decay on touch, heat, temperature, the rolling window -- measured

  • Test first (test_mesh_boot.c, 56 checks): heat +1 a call; decay by the ticks since at v3's slope and arithmetic (6 ticks: 1; 3 ticks: 0); none for a frozen word, none while L8 has Loop #3 off; the temperature; the window.
  • mesh.c's call hook, in v3's order; the kernel's heartbeat read at each call (ticks). Until Task 3 brings L8, Loop #2 and #3 are on (v3's boot sets both off and lets its card decide).
  • Measured: amd64 boot 146 s, against Phase 2's 142 s (about +3%).

Task 2b: pipelining (transitions, the likely next word, speculation) and the hot-words cache in front of FIND, behind ENABLE_PIPELINING and ENABLE_HOTWORDS_CACHE, off as v3

  • Test first: test_mesh_boot.c built twice -- as the build is (off: no transitions, an empty cache; 57 checks) and with both on (make -C v4 mesh-boot-on, 65 checks): PA's 12 transitions to PB, its likeliest next; speculation after 10 puts PB in the node's cache; hits; FIND through the cache; PB defined again is taken out; a word past heat 50 goes in; FORGET takes the rest out.
  • The kernel (mesh.c), at every call: v3's hit check, the transition, the likeliest next (kept as each count rises, not a scan of 1024 a call: the same word but in a tie v3's Q48.16 rounding makes, recorded in the code), speculation at v3's thresholds (1/2, 10 samples). Counters from the owner's heap on a word's first transition; given back when it goes.
  • The node (dict.f18, (LOOKUP)): (HOT)'s xts tried first, in F18, calling only C@ as before; the kernel fills it as v3's promote, and takes out entries that go or whose name is defined again. Empty unless the kernel's build has ENABLE_HOTWORDS_CACHE.
  • One departure, ruled by default (Captain Bob did not choose; recorded): v3 promotes a hot word when a lookup finds it in the dictionary; v4's heat is the kernel's, so it is promoted at the call where it is hot.
  • Measured: amd64 174 s and 181 s; 2a's own commit booted again just after, 175 s (it was 146 s an hour before). The host slowed; 2b costs nothing measurable, as the host mesh boot also shows.
  • Reported, not changed (v3's own): with the cache full, v3's promote takes the next slot in turn without looking for the word already there, so a few hot words churn through all 32 slots (hotwords_cache_promote, v3/src/physics_hotwords_cache.c). v4 does the same. And v3 never calls transition_metrics_record_prefetch_miss: its misses are always 0, as v4's.

Task 3: the per-tick loops (tuner, slope, background decay, L8, inference, snapshot)

  • The tick, by execution (Captain Bob, 2026-11-13: "By execution, as v3"). v3's kernel runs a VM's heartbeat cycle every 256 word executions (HEARTBEAT_CHECK_FREQUENCY, kernel/src/vm/vm_core.c:912 and :1210), and its tick is the count of cycles; the timer heartbeat is not the VM's tick. Task 2a had used the kernel's timer: corrected. Each node counts its word calls; every 256th runs its cycle at the call. Test first (test_mesh_boot.c, 59 and 67 checks).
  • Background decay (v3's vm_tick_apply_background_decay): 64 words a cycle from a cursor down the dictionary, so a word not called cools. The window's adaptive pass (v3's rolling_window_run_adaptive_pass): diversity against the last; halve to 256, double to 4096. Test first (test_mesh_boot.c: 62 checks; 70 on).
  • Inference every 1000 ticks (v3's vm_tick_inference_engine): v4/system/inference.c, v3's inference_engine.c and Q48.16 arithmetic ported to the bit -- v4/tests/test_inference.c links v3's own and compares them on POST's cases, 20000 random multiply and log pairs, and 400 trajectories (make -C v4 inference). The trajectory from the window's newest words' heat; the window width and slope applied unless it exits early (as v3, the last inferred slope re-applied on a full run even with L6 off). Loop #7: the heartrate 25% slower when stable, 20% faster when not, within 1/4..4x; Hera's re-arms the kernel's heartbeat. Test first (72 checks; 80 on). amd64 PASS, 163 s.
  • L8's update (v3's vm_heartbeat_update_l8 with its table, ssm_jacquard.c, compudynamics.c): the regime (window entropy >= 0.75, prediction CV >= 0.15, locality >= 1/2, as exact integer comparisons), the card scored (ANOVA from inference; joint convergence when L5 or L6), the next by UCB1, its gates applied. Every gate off and card 8 at the start, as v3's kernel boot. Applied through card_bits -- v3 applies the card's number, one bit off (#196; ruled "v4 correct, v3 as is"). v4_mesh_l8_force as v3's ssm_l8_force_config. Test first (67 checks; 75 on). amd64 PASS, 156 s.
  • The snapshot (v3's heartbeat_publish_snapshot): each cycle, the tick, window, slope and the dictionary's heat as inference counted it (v3's *_at_check). One copy: v3 double-buffers for its heartbeat thread, which v4 has no counterpart of. Test first (73 checks; 81 on).

Task 3b: the loops on the 32 opcodes (ruling 7), behind OPCODE_PHYSICS -- measured

  • The engine: a hook at retire (exec.h, on_retire), as Part A's at call; test first (test_exec.c).
  • The mesh: 32 opcode records per node beside the words', an opcode window, opcode transitions (L4); at every retire decay at the opcodes' own slope, heat, temperature, the window, the transition; in the cycle all 32 cooled, the opcode window's service, inference on its heat (their own width and slope). L8 scores from the words; its gates govern both. The window refactored to one type for both. Test first, three builds: as built (77 checks), pipelining on (86), the revert OPCODE_PHYSICS=0 (74).
  • OPCODE_PHYSICS in Kconfig.physics (default y) and the kernel's flags; OPCODE_PHYSICS=0 checked to reach the build (the revert's kernel has none of the opcode code).
  • Measured, back to back on amd64: 219 s on, 196 s off -- about +12% under TCG. Kept, as ruled; the revert is the one switch.

Task 4: the words that read and set them (FREEZE-WORD, the inference words, ...), and their POST

  • capsules/v4/physics.4th (blocks 8200-8201), a kernel capsule on Hera: v3's physics-freeze words (FREEZE-WORD UNFREEZE-WORD FROZEN? HEAT! HEAT@ DECAY-RATE@) and inference words (Q.VARIANCE INFER-DECAY-SLOPE INFER-WINDOW-WIDTH INFER-RUN INFER-WINDOW@ INFER-DECAY@ INFER-VARIANCE@ INFER-FIT@ INFER-EARLY-EXIT@), each KERNEL-WORD -- requests -21 to -34, which the kernel serves on any node ("compudynamics is stadium/fabric"): a word named as v3's, by the newest visible entry of CONTEXT then FORTH; the arithmetic v4/system/inference.c's. A Q48.16 result is v4's double (qmath.f18: low cell, then high). HEAT! starts the word's decay from now (a word never touched would otherwise decay at once by every tick since its definition). Test first, v3's POST cases with v4's address units (test_mesh_boot.c: 86 checks; 95 on; 83 without opcode physics).
  • The tick, counted as v3's (Captain Bob, 2026-11-14: "v3"; "we can always change it late"). v3 counts one execution per token its interpreter takes and one per word its definitions run; its interpreter is C. v4's is F18, and counting every call made ~700 counts to v3's one on a typed line (measured on 8 lines) but 1:1 in compiled code, so no single HEARTBEAT_CHECK_FREQUENCY could match (the knob ruling, withdrawn on that measurement). Now: INTERPRET counts its tokens in (TOKENS) (one F18 store); the kernel adds them in at each call, and counts a call -- and gives it physics -- only when it comes from code defined at run time (dictionary space), or is INTERPRET's or EXECUTE's dispatch; the system's own calls among its words, none. 3 4 + . counts 4, as v3. Test first (90 checks; 99 on; 87 without opcode physics). amd64 PASS: promotions=136 (fewer: the interpreter's own calls are no longer words run).
  • v3's StarForth module's physics words, in physics.4th (blocks 8202-8204): ENTROPY@ ENTROPY! (a word's heat by the address ' leaves; an error for what is not a word, as v3's) and RESET-ENTROPY -- requests -35..-37; TOP-WORDS and WORD-ENTROPY in StarForth, printing what the kernel gives them (-38..-40: how many have heat, the i-th hottest in v3's stable order, a word's heat), in v3's form; (- in StarForth, skipping to the matching close paren with nesting as v3's code does. Test first (101 checks; 110 on; 98 without opcode physics). Listing: 403 live words. For Task 8: WORD-ENTROPY prints "Total words", and v4's dictionary is not v3's size -- that line cannot print v3's number; TOP-WORDS' hottest words are v4's system words, not v3's.

Task 5: the StarForth words (Phase 3's: init.4th, starforth.4th), and their POST

Task 6: the ACL (Phase 4's: ACL.4th, the word card, the countdown), and its POST

Task 7: the block cases (Phase 5's part)

Task 8: mkpost.py beyond the Required set

  • mkpost.py --all --out FILE: the whole POST as a table, measured on Hera in the fabric -- 1020 cases, 849 pass, 171 fail (POST-PARITY.md section 5).
  • Ruled (2026-11-14): "v3's table, not v3's bugs" -- v3's self-disagreements held to its table, cases naming v3's internals held to their form with v4's values; both listed.
  • The 171, by kind: address units (rules); what v4 lacks (SP@ SP! CASE SKIP SCAN SEARCH COMPARE ORDER THRU FLUSH ...); v3's self-disagreements and v3-internal output (by the ruling); the rest case by case. Then the whole table becomes v4's POST and its floor.: v3's whole POST as v4's; the floor raised

Task 9: acceptance -- the full protocol; ENGINE.md, V3-PARITY.md, FABRIC-3

Each task gets its steps when the one before it is accepted, test first, as Phase 2's did.

Review focus

  1. Cost at call (ruling 1): the boot's time under TCG before and after each per-call loop.
  2. Order: the window and the transitions see calls in the order they happen, across nodes kept apart.
  3. Recycled IDs: a word ID FORGET gave back starts with no physics (decay, window, transitions).
  4. The 32-bit build: the same results at 32-bit cells.
  5. What names v3's VMs (Task 0): ruled, not guessed.