docs(v4): ENGINE step 4 under MACHINE.md; Q1-Q5 and the identity gate ruled
MACHINE.md stands: v3 goes after parity; Hera, Hermes, Artemis (and Hestia) are kernel functions. ENGINE.md section 2 amended with the rulings, section 6's step-3 rows struck (R2). The step 4 plan rewritten: Part A (hook at call, called in-line words) and Phases 1-8 to parity. No identity thumbdrive, headless; work goes ahead only with Zuse attached. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
d6eab090b4
commit
eef6c15c28
@@ -0,0 +1,176 @@
|
||||
# ENGINE Step 4, revised under MACHINE.md: v4 in the kernel's boot — Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (Captain Bob prefers native execution). Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** On bare metal, v4 boots *in the kernel's boot*, not before it: the kernel brings up its
|
||||
own services, then the v4 system, then does what v3 does before its first prompt, through the
|
||||
kernel and the nodes, never through a v3 VM. When v4 is shown at parity, v3 leaves the build
|
||||
(`MACHINE.md` §5.1, §6 step 6, §7 item 3).
|
||||
|
||||
**What changed (RULED 2026-10-20, Captain Bob): "MACHINE.md stands; v3 goes after parity; Hera,
|
||||
Hermes, Artemis are now all kernel."** The first version of this plan followed `ENGINE.md` §2: a
|
||||
node behind each v3 `VM` record, a third interpreter file answering v3's engine functions, v3's C
|
||||
words served to nodes, the Tripod and kernel-Hermes kept. That is withdrawn. Nothing is put behind a
|
||||
v3 `VM`; v3 is kept only as the reference until parity.
|
||||
|
||||
**Spec:** `docs/v4.0.0/MACHINE.md` (governs), `ENGINE.md` §3, 3a, 3b, 3d and §5 step 3 (stand where
|
||||
they do not conflict), `V3-PARITY.md` §1 to §1j (what v3 does before its prompt; the yardstick),
|
||||
`MESH.md` §9 and §9a (the kernel's registry, births, KILL).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Branch `StarForth-v4.0.0`, main checkout. Never create a branch, never stash, never `--no-verify`.
|
||||
- Commit with `G="git -c filter.lfs.process= -c filter.lfs.clean=cat -c filter.lfs.smudge=cat -c filter.lfs.required=false"`; push with `PATH="$PATH:/usr/share/gitkraken/resources/app.asar.unpacked/git/libexec/git-core" $G push -q origin StarForth-v4.0.0`. Messages end `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
|
||||
- v4 = v3 up to the first prompt; nothing stubbed, simulated or stood in for (`ENGINE.md` 1).
|
||||
- The engine in `v4/src` stays free of anything specific to StarForth or the kernel (`ENGINE.md` 3d).
|
||||
- A node may be stopped between any two instruction words (`ENGINE.md` 3a).
|
||||
- v3 stays in the tree and `system=v3` still boots until parity is shown; nothing in v3's own path
|
||||
changes (`MACHINE.md` §7 item 3). v3 is not part of acceptance.
|
||||
- Compose in FORTH first; a kernel request only for what only the kernel can do.
|
||||
- Never apply a fix that is not in this plan; report it.
|
||||
- Suite per task: `make -C v4 test test-32 sanitize hosted-check`.
|
||||
- Acceptance per kernel-affecting increment: `make ARCH=<isa> clean qemu` on amd64, aarch64,
|
||||
riscv64, one at a time, foreground, to `ok>`, `PARITY:V4_POST ... fail=0`, `POST: PASSED`, equal
|
||||
`dict_hash`; `make -C v4 test-32` 0 failures; logs committed.
|
||||
|
||||
## What becomes of the first plan's rulings
|
||||
|
||||
| | First plan | Now |
|
||||
|---|---|---|
|
||||
| R1 | MESH inc 2, then this | Done (6cb8f128). |
|
||||
| R2 | Step 3: hook at `call`; `EXECUTE` calls; all in-line words but `>R R> R@ I J LEAVE` called | **Stands** (Part A). |
|
||||
| R3 | Third interpreter file replaces `vm_core.c`; `system=` goes | **Gone.** No v3 function is answered by a node, so both systems link as today; `system=` and `STARFORTH_V4=0` go with v3 at the end (§5.1). |
|
||||
| R4 | v3 C words on a node by stack copy and copy helpers | **Gone.** v3's C words are not served to nodes. |
|
||||
| R5 | Every v3 C word a node lacks becomes a `KERNEL-WORD` served by v3's function | **Replaced by Q3.** |
|
||||
| R6 | v3's POST run on the node, failures listed | **Replaced by Q5.** |
|
||||
| R7 | A node behind each VM the boot births | **Gone.** The fleet is the nodes' (MESH 8 and 9); Hera, Hermes and Artemis are the kernel's. |
|
||||
|
||||
## Rulings (Captain Bob)
|
||||
|
||||
**RULED 2026-10-20: "yes, Q1-Q5 as recommended"**, recorded in `ENGINE.md` §2 and `MACHINE.md` §5.1
|
||||
(Task 0). And with it: **"go ahead only if zuse thumbdrive is attached so you have zuse. no identity
|
||||
thumb drive equals headless. when a thumb drive is inserted, only then is there an identity and a
|
||||
console."** So no task's acceptance boot is run without Zuse: the Zuse thumbdrive is attached and
|
||||
minted, and the boot recognises it. Today it does not: the drive QEMU attaches by default,
|
||||
`disk/thumbdrives/zuse-thumb-ident.img`, is all zeros (bleached, `e56974e0`), and the v4 boot reads
|
||||
no thumbdrive and prints its prompt with no identity. See "The identity gate" below.
|
||||
|
||||
**Q1. What "Hera, Hermes, Artemis are kernel" means in v4.**
|
||||
*Recommend:* each is a function of the kernel, not a VM or node with a capsule of its own:
|
||||
- **Hera** — process management: the registry, births, `NODE-KILL`/`SLEEP`/`WAKE` and who may ask
|
||||
(`v4/src/manage.c`, already the kernel's, MESH 9a.4). The first node, which the kernel makes and
|
||||
POSTs and whose prompt the console shows, keeps the name Hera, as `[zuse@Hera] ok>` needs, and
|
||||
is the root of the birth graph; `hera.4th`'s words stay the commands she types.
|
||||
- **Hermes** — messaging: the wires and the kernel's serving of them (`v4/src/wire.c`,
|
||||
`message.c`, the GONE the kernel tells). Not v3's `kernel_hermes.c`.
|
||||
- **Artemis** — storage: the kernel serving blocks (`v4/system/blocks.c` on the block subsystem),
|
||||
the disk's genesis signature, and the owner's word that the disk may be formatted (`ENGINE.md`
|
||||
step 7). No Artemis capsule.
|
||||
|
||||
**Q2. Hestia.** Not named in the ruling. v3's Hestia is a VM owning console policy (`V3-PARITY.md`
|
||||
1b). *Recommend:* the same as the other three — the console is the kernel's (HAL, REPL), with no
|
||||
Hestia node.
|
||||
|
||||
**Q3. Which of v3's kernel code survives v3.** `MACHINE.md` §5.1 sends `kernel/src/vm/`,
|
||||
`kernel/src/capsule/` and `v3/src` out of the kernel. Some of what lives there does not touch a v3
|
||||
`VM` at all: the Stadium (`stadium.c`, `stadium_words.c`, `stadium_blocks.c`, keyed by `VMUuid`),
|
||||
the heartbeat (`kernel/src/heartbeat.c`), the block subsystem (`v3/src/block_subsystem.c`, which
|
||||
v4 already uses), the capsule directory and its signatures, the disk signature, and the identity code
|
||||
(Zuse, mint, `WIREBIND`, `kernel/src/crypto`).
|
||||
*Recommend:* these are the kernel's and stay, moved out of v3's directories when v3 goes, keyed by
|
||||
node number where they are keyed by VM today. What goes is what is a v3 VM: `vm_core.c`, `arena.c`,
|
||||
`capsule_birth.c`, `mama_forth_words.c`, `kernel_hermes.c`, sessions and switcher as they hold
|
||||
`VM *`, the C ACL fields, `v3/src`'s interpreter and C words. *If not:* each of those services is
|
||||
rewritten for v4 before parity, which is most of the kernel.
|
||||
|
||||
**Q4. Where a v4 word's accounts are kept** (ruling "A", `V3-PARITY.md` 1j: code the node's,
|
||||
accounts the kernel's, joined by word ID). "A" named v3's `DictEntry`, which goes with v3.
|
||||
*Recommend:* the kernel keeps, per node and word ID, a record of its own with the account fields
|
||||
`DictEntry` has (`execution_heat`, the physics record, the four ACL fields, the transition metrics,
|
||||
the word ID), and the Stadium's word layer is fed from it (`stadium_word_dispatch` takes a VM ID and
|
||||
a word ID, not a `DictEntry`). The physics loops (`v3/src/physics_*`, `rolling_window_of_truth.c`,
|
||||
`inference_engine.c`, `ssm_jacquard.c`) walk `DictEntry` lists today; each is ported to the record
|
||||
when its turn comes (Phase 7).
|
||||
|
||||
**Q5. The words v3 has and a node lacks, and the yardstick.** v3 has 526 words before its prompt;
|
||||
a node has 317. *Recommend:* each missing word is written in FORTH in the layer `MACHINE.md` §3
|
||||
gives it (`forth79.4th`, `starforth.4th`, a system capsule), or is a kernel request where only the
|
||||
kernel can do it (blocks, births, the Stadium, identity, the framebuffer). v3's POST is the measure:
|
||||
each module's cases are added to `v4/system/post_cases.c` (by `mkpost.py`, from the v3 binary) when
|
||||
its words land, and must pass. Parity is v3's whole POST passing on v4 plus the boot of
|
||||
`V3-PARITY.md` §1, rows 1 to 12, done by v4's means.
|
||||
|
||||
## The identity gate (2026-10-20)
|
||||
|
||||
v3 as built: with no identity drive the kernel waits headless (`sk_repl_headless_wait`,
|
||||
`kernel_main.c`), servicing the heartbeat and watching USB; `capsule_zuse_boot_try_attach`
|
||||
recognises Zuse's drive when it arrives, and only then is there a prompt (`[zuse@Hera] ok>`).
|
||||
The v4 boot does none of this. Until it does, the condition for going ahead cannot be met by any
|
||||
boot. Two things are needed, and are put to Captain Bob:
|
||||
1. **A minted Zuse drive.** Re-minting the committed images is his call; a scratch copy can be
|
||||
minted by a v3 boot (`ZUSEDISK=<scratch>/zuse.img`, genesis minted on first boot) without
|
||||
touching the committed one.
|
||||
2. **The gate on v4:** the headless wait and Zuse's recognition, done by the kernel for the v4
|
||||
boot, ahead of Part A, moved forward from Phase 6.
|
||||
|
||||
## Phases
|
||||
|
||||
Each phase is its own plan, written when the one before it is accepted. Each ends with full
|
||||
acceptance. Only Part A is written in tasks below; it does not depend on Q1 to Q5.
|
||||
|
||||
| Phase | What | `V3-PARITY.md` §1 rows |
|
||||
|---|---|---|
|
||||
| A | `ENGINE.md` step 3, engine side: the hook at `call`; `EXECUTE` and the in-line words called | (3, 8: the hook they need) |
|
||||
| 1 | **v4 in the kernel's boot.** `sk_v4_run()` before everything goes. The kernel's order runs as v3's does: hardware, the Stadium, the heartbeat, then the v4 system on the first node (Hera) with POST, then the devices (PCI, the disk), then the kernel's REPL hands lines to Hera. The registry and births (`manage.c`) come into the kernel build: MESH step 9. | 1, 2, 5, 6, 9 |
|
||||
| 2 | **Word accounts at `call`** (Q4): the kernel's record per word; at each call, `execution_heat`, `stadium_word_dispatch`, the ACL countdown. | 3 (part), 8 (part) |
|
||||
| 3 | **`init.4th`** (`MACHINE.md` §6 step 3) and `starforth.4th`: the StarForth words, with their POST modules. | 4 (part), 7 |
|
||||
| 4 | **ACL**: `ACL.4th` on v4, the word card from Phase 2, `BIRTH` and `CAPSULE-BIRTH` pinned; the ACL module of POST. | 8 |
|
||||
| 5 | **Artemis, the kernel's**: the disk's genesis signature, formatting by the owner's word, writes. | 6, 10 (Artemis) |
|
||||
| 6 | **Identity and the prompt**: Zuse, the thumbdrive, `WIREBIND`, `EJECT`; the banner; `[zuse@Hera] ok>`. | 11, 12 |
|
||||
| 7 | **Physics** on the word records: decay, rolling window, pipelining, inference, L8; the physics-freeze and inference POST modules. v3's whole POST passes: parity. | 3, 4 |
|
||||
| 8 | **v3 leaves the build** (`MACHINE.md` §6 step 6). | — |
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **A word called through `EXECUTE` from inside a definition** must be seen by the hook exactly once, and the definition must go on after it. Pinned in A2.
|
||||
2. **`>R ... R>` around a call** (the six in-line words beside called ones): a called word must not disturb a return-stack value the caller pushed. Pinned in A2.
|
||||
3. **The 32-bit build**: the call hook and the called in-line words at 32-bit cells give the same results. Covered by `test-32` in every task.
|
||||
4. **A definition that runs past the end of its text while the hook is set** (an abandoned definition): the hook sees no call into it. Pinned in A2.
|
||||
5. **Speed**: every in-line word becoming a call slows the node; POST and the depth sweep must still finish under QEMU TCG in the time the acceptance boots allow. Measured in A2 (boot time before and after recorded in the commit message).
|
||||
|
||||
---
|
||||
|
||||
## Part A — `ENGINE.md` step 3, engine side
|
||||
|
||||
### Task 0: Record the rulings
|
||||
|
||||
**Files:** `docs/v4.0.0/ENGINE.md` (§2, §3c, §5 steps 3 and 4, §6), `docs/v4.0.0/MACHINE.md` (§5.1).
|
||||
|
||||
- [ ] **Step 1:** Write the 2026-10-20 ruling and the approved Q1 to Q5 into `ENGINE.md` and `MACHINE.md` §5.1, with Captain Bob's words and the date; strike §6's three rows settled by R2.
|
||||
- [ ] **Step 2:** Commit `docs(v4): ENGINE step 4 under MACHINE.md`.
|
||||
|
||||
### Task A1: A generic hook at `call`
|
||||
|
||||
**Files:** Modify `v4/src/exec.c` (`V4_OP_CALL`), `v4/include/v4/exec.h`; `v4/src/heat.c`, `v4/include/v4/heat.h` (the per-call-target count goes, `ENGINE.md` 4, `V3-PARITY.md` 2.7). Test: `v4/tests/test_exec.c`.
|
||||
|
||||
**Interfaces:** Produces `typedef void (*v4_call_hook)(void *ctx, v4_node *n, v4_cell target);` and, in `v4_exec_state`, `v4_call_hook on_call; void *on_call_ctx;`. Called at `V4_OP_CALL` after the return address is pushed, before `n->p` moves. NULL = no hook.
|
||||
|
||||
- [ ] **Step 1: Failing test** `test_call_hook_sees_every_call`: a programme calling three routines, one twice; the hook records the four targets in order; with `on_call = NULL` the result is the same.
|
||||
- [ ] **Step 2:** `make -C v4 test` — FAIL (no `on_call`).
|
||||
- [ ] **Step 3:** Add the hook; remove `v4_heat_on_call`, `call[]`, the freeze mask and `v4_heat_call_*`, and their tests.
|
||||
- [ ] **Step 4:** Suite passes at both widths and under sanitizers; `hosted-check` unchanged.
|
||||
- [ ] **Step 5:** Commit `feat(v4): a hook at call, for whoever runs the node`. (No boot change: nothing sets the hook on bare metal yet. One amd64 boot to confirm POST 538/538 and the same `dict_hash`.)
|
||||
|
||||
### Task A2: Words that are checked are called
|
||||
|
||||
**Files:** Modify `capsules/v4/sys/compile.f18` (`(CALL,)` and the in-line table), the `.f18` file that defines `EXECUTE`. Test: `v4/tests/test_host_compile.c`; the 538 POST cases; `test_host_depth.c`.
|
||||
|
||||
**Interfaces:** Consumes A1's hook (the test sets it to observe).
|
||||
|
||||
- [ ] **Step 1: Failing tests** `inline_words_are_called` (for each callable in-line word of `V3-PARITY.md` 1f, a definition using it produces a call the hook sees), `residue_stays_inline` (`>R R> R@ I J LEAVE` compile in line), `execute_passes_the_hook` (`' DUP EXECUTE` seen once; Review Focus 1, from inside a definition that goes on after), `rstack_kept_across_call` (Review Focus 2), `abandoned_definition_not_called` (Review Focus 4).
|
||||
- [ ] **Step 2:** Run — FAIL.
|
||||
- [ ] **Step 3:** The compiler's in-line table becomes the six; `EXECUTE` calls the xt. Regenerate the maps (`make -C v4 sysmap`).
|
||||
- [ ] **Step 4:** Suite passes; POST 538/538; record the new `dict_hash` (equal at both widths' products as before).
|
||||
- [ ] **Step 5:** Full acceptance (three boots, test-32; boot times before and after, Review Focus 5); commit `feat(v4): every word but the six is called, so it can be checked`, then the logs.
|
||||
|
||||
Phase 1's plan is written when Part A is accepted and Q1 to Q5 are ruled.
|
||||
Reference in New Issue
Block a user