Files
LithosAnanke/docs/v4.0.0/plans/2026-11-18-capsule-blocks.md
T
Claude 6dcfbe6740 Capsule blocks steps 3+4 for v4: capsules are placed in RAM and load their lowest block
Captain Bob ruled: Block n lands at n-R; the lowest block is
interpreted; chaining is the developer's; together, v4 first.
- v4_blocks_place_capsule copies each Block n into RAM block n - R as 16
  lines of 64, space-padded, in any file order. A later block of the
  same number overwrites. It refuses a block below R.
- The mesh's load(), its capsule reader (CAPSULE-OPEN/(LINES), so a
  child's forth79.4th via BIRTH too), boot.c and test_host_unit's model
  each place a capsule, then interpret "<lowest> LOAD".
- mkdictionary.py skips --> when it types capsules at the hosted prompt.
- The v4 capsules are renumbered into R..2R-1, landing at RAM 1900 and
  up, away from POST's blocks 0-5. Each is chained with -->, and
  centre.4th's gap at 8103 is closed.

Test first:
- test_host_blocks.c gained 6 placement checks; it failed to build
  before.
- test_mesh_boot.c checks forth79's lowest block in RAM block 1900; it
  failed before.

After: POST is 990/990; all 52 tests at 64 bits, test-32, hosted-check
and ASan/UBSan pass; the amd64 kernel builds; the dictionary has 404
words.

Logs are in logs/20261118-capsule-place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
2026-11-18 20:33:21 +00:00

11 KiB
Raw Blame History

Capsules and their blocks -- the talk, and the bug it found

Written 2026-11-18, from the capsules talk with Captain Bob. It records the rulings as they are given. Nothing here is built.

1. The design (Captain Bob)

  • A capsule is ephemeral. "Capsules load into their respective block by the header for that block, Block <n>. Then relocated into fast ram for execution. Other than the blocks used as actual held data, everything else should be free for reuse."

  • Block <n> "loads 1024 bytes from the defined block number and relocated". A page shorter than 1024 bytes is padded; zeroes or spaces, "doesn't matter".

  • The order in the file is irrelevant. His example, as the pages appear in the file:

    Block 4098   \ stuff
    Block 4255   \ stuff
    Block 4096   \ program for stuff
    Block 5077   \ crappy stuff
    

    Each page lands by its own number, then is relocated into the VM's RAM and interpreted there.

  • The offset is not a constant (ruled: "That's THE BUG! #1 needs to determine that from the runtime and build knobs"). Where the VM's RAM ends, and so how far a page moves, comes from the runtime and the build's knobs, never from a fixed 2048.

  • The order of interpretation (ruled): "Start with lowest block loaded. The current block will determine the next block interpreted." The lowest page runs first. What runs next is decided by the page that is running, through -->, LOAD or THRU by number.

2. What the tree does today (read 2026-11-18)

Where What it does Against the design
tools/mkcapsule.c:414, :1094 Refuses a page below the literal 2048, "the VM's fast RAM (0..2047)" The fixed number is the bug (ruled)
v3/include/block_subsystem.h:78-88 BLK_RAM_BLOCKS 2080 and BLK_FORTH_SYS_RESERVED 32 give 2048 user blocks The knobs exist, but nothing derives the capsule offset from them
v3's INIT (v3/src/word_source/starforth_words.c:300-490) Renumbers the pages 1, 2, 3 ... in file order and runs them Places by file order, not by number; no offset from the knobs
v4's loader (v4/system/mesh.c:102-117, load()) Drops the number and sends each page's lines to Hera in file order; writes no block The pages are never in blocks, so a capsule cannot LOAD, THRU or --> its own pages; file order is execution order
experiments/bare_metal/README.md rule 3 "Blocks are loaded in file order and executed top-to-bottom" Contradicts "order is irrelevant"
mkcapsule's collision check (:846-879), BLOCK_MAP.md, the range tables (experiments/bare_metal/README.md, .claude/CLAUDE.md, capsules/README.md) Treat each page number as a permanent claim across all capsules Open: see section 3

3. Open, one at a time

  1. The Block <n> header: ruled above.

  2. The collision check across capsules Ruled: "Overwrite. That's it. Deterministic and final." A page written to a block number already in use overwrites it; what was there is gone. Which write wins follows from the load order, which is fixed, so it is deterministic. A shared number is not an error, so mkcapsule's cross-capsule collision check (:846-879) goes.

  3. The knobs Ruled: the RAM block count. The offset is the VM's user RAM block count, BLK_RAM_BLOCKS - BLK_FORTH_SYS_RESERVED (2048 today, v3/include/block_subsystem.h:78-88).

    • It becomes a Kconfig symbol.
    • The kernel reads it.
    • mkcapsule is given it at build time in place of its literal 2048 (:414, :1094).

    v4's node window (V4_BLOCK_SLOTS) is not the source of the offset.

  4. Where the pages land Ruled: at n, then copied down.

    • Each page is written to the block store at n, then copied to n - R for interpreting, where R is the RAM block count (ruling 3).
    • Block n is then free again.
    • Asked about pages whose n - R falls outside the RAM (Block 4096 lands at 2048 when R is 2048), Captain Bob ruled: "Just copy down the specified blocks. Where they land will be developers responsibility." The loader does not check where a page lands. The one rule left is that n is at least R (mkcapsule's floor, from the knob).
  5. The range tables and BLOCK_MAP.md Ruled: "Map stays, tables go".

    • BLOCK_MAP.md stays, generated by mkcapsule, as a record of the numbers each capsule uses ("Blocks used", not "claimed"). Captain Bob: "Block map is a convenience": an aid for developers, with no authority and nothing checked against it.
    • The range tables go from experiments/bare_metal/README.md, .claude/CLAUDE.md and capsules/README.md. In their place: n is at least R, a later write overwrites, and where a page lands is the developer's responsibility.
  6. Chaining. Under "the lowest block first; the current block determines the next", only a capsule's lowest page runs unless it chains. No capsule in the tree chains its pages today (forth79.4th has 3 pages, centre.4th 7, v3's init.4th 3, none with -->, LOAD or THRU). Ruled: "No 'chaining' again developer responsible."

    • The loader interprets the capsule's lowest page and nothing more.
    • A capsule that needs its other pages chains them itself (-->, LOAD, THRU).
    • So each capsule in the tree is given its chaining when the loader changes, or only its lowest page would run.

4. The work, for ruling (nothing built)

Each step is test first, then POST and the protocol:

  1. The knob. Built 2026-11-18:

    • BLK_RAM_USER_BLOCKS is in Kconfig.kernel, default 2048, defined in v3/include/blk_layout.h.
    • block_subsystem.h derives BLK_RAM_BLOCKS (2080) and BLK_DISK_START from it.
    • kernel/Makefile passes it to the kernel's C code, mkcapsule and mkartemis, and mkcapsule's two literal 2048s are the knob.
    • Tested by scripts/test_capsule_ram_blocks.sh, 8 checks, in logs/20261118-ram-blocks-knob/.

    The plan as written: block_subsystem.h and the kernel read it, and mkcapsule is given it at build time in place of its literal 2048.

  2. mkcapsule. Built 2026-11-18 (rulings: overwrite within a capsule too, "consistent with VOCABULARY FORGET & FENCE philosophy, not implementation"; of what only prevents collisions, "All of it goes"):

    • the cross-capsule collision gate is gone;
    • so are tools/capsule-reserved.txt with its check, and the interactive resolver with its claim registry;
    • the map says "Blocks used", and marks a shared number "the later write overwrites";
    • an unknown option is refused with the usage, so a removed mode is never taken for the capsules directory.

    A repeat within one capsule had no check to remove. Tested by scripts/test_mkcapsule_overwrite.sh, 9 checks, in logs/20261118-mkcapsule-overwrite/.

    The plan as written:

    • Its floor comes from the knob.
    • Its cross-capsule collision check goes, since a later write overwrites.
    • BLOCK_MAP.md says "Blocks used".
    • Open: whether a number repeated within one capsule is still refused, or overwrites by the same rule.
  3. The capsules chain their own pages. Each multi-page capsule ends each page with -->, or has its lowest page THRU the rest. That keeps it working under step 4.

  4. The loaders, v4's load() and v3's INIT:

    • each page is written to block n, then copied down to n - R;
    • the lowest page is interpreted;
    • nothing else is done.
  5. The documents. The range tables go, and the rules take their place. experiments/bare_metal/ README.md rule 3 ("file order") goes.

5. (- marks a system blob (Captain Bob's idea, 2026-11-18; nothing built)

The idea, as given:

  • (- is a special comment word, terminated by ) like any ( comment.
  • It also "leaves behind the equivalent of a magic number".
  • "The capsule loader can differentiate a 'system capsule' from any other capsule and it can be diverted into 'please include in a kernel rom image'."

What exists already:

  • v3: (- is a C word (v3/src/word_source/starforth_words.c:274) that skips to the matching ) and nests. Its own comment says it is "used for marking blocks that should be extracted to init.4th", so v3 already meant it as a marker.
  • v4: the same nesting comment, in capsules/v4/physics.4th block 8204.
  • POST: v3's 4 cases for it pass in v4.

As Captain Bob put it next, which supersedes the readings below ("(- is more of a build marker"):

  • It is per block. "this particular block number": a block that opens with (- is marked.
  • At run time it loads verbatim, like any block, "because it might be part of ongoing development or a required dependency". (- is only the nesting comment there; there is no run-time magic number.
  • At build it is a request to CI: "asking the CI to consider this particular block number be cleared for inclusion in the next iso image that qemu runs with".
  • What CI does with the request (ruled): "Open ended stubs to be hooked into after we begin the SDK later." So the stub is the record of the request, with nothing to act on it until the SDK.

Kept: (- nests, as now. v3's 4 POST cases are unchanged.

Superseded (kept as the record of how it was first understood):

  • the magic number "both" at build and at run;
  • the reading "per named blob";
  • the targets (1) a new kernel ROM image and (3) the capsule directory of system blobs only.

The "kernel ROM image" is now "the next iso image that qemu runs with".

Nothing is built. When the SDK begins, the hook is to be designed against these lines.

6. Steps 3 and 4 for v4, built together (2026-11-18)

Why together (ruled "Together, v4 first"): v4's --> is error 14 outside a block load, so capsule chaining added under the old line-streaming loader would have broken every boot.

What is built:

  • v4_blocks_place_capsule (v4/system/blocks.c, blocks.h). Each Block n is copied to RAM block n - R as 16 lines of 64 characters, padded with spaces, in any file order. A later block of the same number overwrites. The lowest placed is given back, and a block below R is refused.

  • Every v4 loader places a capsule, then interprets <lowest> LOAD:

    • the mesh's load() (Hera's boot);
    • the mesh's capsule reader (CAPSULE-OPEN/(LINES), which is how BIRTH gives a child forth79.4th);
    • the hosted product's boot.c;
    • test_host_unit's model of the kernel.

    mkdictionary.py types the capsules at the hosted prompt, and skips --> there.

  • The v4 capsules are renumbered into R..2R-1 and chained with -->. They land at RAM 1900 and up, away from POST's blocks 0–5:

    Capsule Blocks
    forth79.4th 3948–3950
    starshipos.4th 3956–3957
    hera.4th 3960–3961
    centre.4th 3964–3970 (it had a gap at 8103)
    physics.4th 3972–3976

Test first:

  • test_host_blocks.c gained 6 placement checks (54 now). It failed to build before, with no v4_blocks_place_capsule.
  • test_mesh_boot.c: forth79.4th's lowest block is in RAM block 1900. It failed before, with block 1900 empty, and passes after (110 checks).

Verified (logs/20261118-capsule-place/):

  • POST is 990/990;
  • all 52 tests at 64 bits, test-32, hosted-check (three ISAs identical), and ASan/UBSan pass;
  • the amd64 kernel builds;
  • the dictionary check has 404 words, as before.

v3's INIT is the next step, as ruled.