Files
LithosAnanke/docs/v4.0.0/plans/2026-11-19-f18-artifact-build.md
T
Claude f7769c0e93 v4: the host placer, and a small train placed on Hera at her HERE (ADR step 3, task 4)
place.c places a train's artifact as ABI.md section 5 has it, the
code split at each export with its entry before it. test_mesh_boot
places tests/train-place, built by mkf18a with an import from the
nucleus: its words run by name, the nucleus's ABS is reached, the
kernel records each export, and six refusals leave Hera unchanged.
mkf18a refuses a word that runs on into an export. Failed first;
gates pass, POST 990/990. Plan section 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
2026-11-20 12:44:50 +00:00

26 KiB
Raw Blame History

Artifacts in the build -- design, for ruling

Written 2026-11-19. Step 3 of ADR-2026-11-19-F18-ARTIFACTS.md §7, "Integrate artifact production into the existing build system". It builds on:

  • ABI.md, ABI v0 as ruled, in particular choice 1, "A host tool";
  • ARTIFACT.md, the layout as ruled.

Nothing here is built. The questions in §6 are ruled (2026-11-19).

1. What is there already

  • The host assembler exists. v4/src/text.c assembles .f18 text onto a node. v4/tools/mkrom.c uses it on the build machine to build the ROM, "as a bitstream is built". The tests use it to build the system.

  • The ROM and text.c are proved to agree. test_host_rom_nucleus assembles the system's source twice:

    • once off the node with text.c, the oracle;
    • once on the machine, by the ROM fed through the console.

    Every word of RAM must be the same, apart from the ROM's own state. This is ABI choice 1's cell-for-cell check already, for the one train that has .f18 today.

  • Only one train has .f18 today: the nucleus. Its source is capsules/v4/sys. The map comes first, then the fourteen files in sys_files.h's order, then end.f18. The boot feeds them to the ROM as the blobs v4:sys:*.f18 (boot.c).

  • The nucleus is not relocatable. It is assembled at a fixed origin against the map's fixed addresses. end.f18 stores addresses into the map's variables, and its dictionary holds absolute links. Ruling 1 keeps it as it is: the ROM still assembles the system at boot (ABI.md §8).

2. The tool: v4/tools/mkf18a.c

A C tool beside mkrom.c, built the same way, one per cell width:

mkf18a OUT.f18a MAP.f18 DIR
  1. It assembles every .f18 in DIR with text.c, at origin 0, in name order (ruled §6.3), after MAP.
  2. It writes the artifact, laid out as ARTIFACT.md gives:
    • the header;
    • the provenance, with its four SHA-512 hashes;
    • the exports, the imports and the relocations;
    • an empty verification record;
    • the string pool and the code, low byte first.
  3. It fails the build if anything cannot be written as the ABI requires:
    • a relocation whose patched address is beyond its slot's reach;
    • a capability it cannot derive.

The relocations come from text.c. text.c gains a record of each cell it lays down that holds an address: every branch it resolves to a word or a label, and every &NAME. That is one hook, and test-first: test_text checks the record against the cells.

The stack effects come from a new analysis, v4/src/effect.c. It walks each word's instruction words from its entry to every ; and counts what each opcode takes and leaves on both stacks:

  • the opcode table is iword.h's;
  • a call takes the effect of the word it calls, derived first;
  • a branch that joins two paths with different depths is "no fixed effect";
  • so are a loop whose count is data, EXECUTE and a DEFERred word.

It is a module of its own, so the proofs can state it on its own (ADR step 5).

3. Where it runs

  • v4/Makefile: make -C v4 artifacts builds the artifacts of each width into v4/build/. make -C v4 test runs it, so a .f18 the tool refuses fails the test gate.
  • The check against the ROM: the nucleus is not an artifact (ruled §6.2a), so no file is written for it. test_host_rom_nucleus assembles it with the tool's assembler, the address record turned on, and the ROM must still lay down every word the same. That proves the record changes no cell. The records themselves are proved by the small train (§4, test 5).
  • The kernel build: nothing changes in step 3. The boot still feeds the ROM the source. Carrying the artifact in the capsule, and placing it, is steps 4 and 6.
  • CI: build.yml's v4-tests job runs make -C v4 test already, so it gains the gate with no change.

4. Test first, seen to fail

  1. test_text: the address record for a branch, a forward branch, a label, and an &NAME.
  2. test_effect, new: words of known effect, written in test_text's notation:
    • dup (1 → 2);
    • drop (1 → 0);
    • a call;
    • a for/next loop of fixed count;
    • two branches that agree;
    • two branches that disagree (no fixed effect).
  3. test_mkf18a, new: a small text's artifact read back unit by unit against ARTIFACT.md. It checks the magic, the counts, each record, every reserved unit 0, and the code low byte first.
  4. test_host_rom_nucleus, with the address record on: the ROM's assembly equal to the tool's, word for word (ruled §6.2a: no artifact file for the nucleus).
  5. Then a small train, a few words of .f18 with one import from the nucleus, is placed from its artifact at a base other than 0. Its words run, and its calls into the nucleus reach the nucleus (ruled §6.2).

POST, the protocol and the dictionary gate are unchanged, since the boot is unchanged.

5. Cost

The build gains one assembly of the nucleus per width, which takes a second or less on the host (test_host_rom_nucleus reports its time today), plus the analysis. That is measured when it is built, as ADR step 8 asks. The boot gains nothing.

6. For ruling, one at a time

  1. Address literals Ruled 2026-11-19: "Slot 7 = whole cell", recorded in ARTIFACT.md §5 and §7. As asked: ARTIFACT.md §5's relocation covers branch fields only (a slot, 0–3). The system's source also holds addresses in whole cells: the &NAME literals (31 in capsules/v4/sys), and the dictionary's links. One fix is a slot value of 7 meaning "the whole cell". The top 3 bits allow 0–7, so the record keeps its size.

  2. Which train first Ruled 2026-11-19: "Both, nucleus first". The nucleus is built and checked against the ROM; then a small train is placed at a base other than 0, to prove the relocations (§4, test 5). As asked: The nucleus is the only .f18 train, but it is not relocatable, and the ROM keeps assembling it (§1). Its artifact can be built and checked (§3), but it is never placed by the loader.

    2a, ruled 2026-11-19: "Nucleus not an artifact". Asked how an artifact would say it is fixed at an origin, since ABI.md §4 has its code "as if placed at address 0": only relocatable trains are artifacts. The nucleus is checked as the tool's assembly against the ROM's, and no artifact file is written for it. ABI.md §4 stands. Is the nucleus the first artifact, or is a small relocatable train written first?

  3. The order of the sources Ruled 2026-11-19: "All it finds, name order". The tool takes every .f18 in a train's directory, sorted by name, so the file names set the order (00-core.f18, ...) and there is no list to keep. As asked: Captain Bob's image is "the .f18 assembler assembles all .f18 files it finds", but the order matters: the map first, sys_files.h's list, end.f18 last. Does each train name its order in a list, as sys_files.h does, or does the tool take the files in name order? 3a, ruled 2026-11-19: "Nucleus too: rename". Asked whether name order applies to the nucleus, which is read in sys_files.h's order: it does. capsules/v4/sys is renamed so that its name order is today's order, the map first and end.f18 last (00-map, 01-core, ... 99-end), and sys_files.h goes. The boot, the tests and the ROM's feed read the directory in name order. The blob names and the capsule hashes change, so the protocol runs again. This is a task of its own, designed and tested first. One point for its design: the directory holds both widths' maps, map-32.f18 and map-64.f18, and name order would take both.

  4. Which words are exports Ruled 2026-11-19: "Every fixed-effect word". Every word the tool derives a fixed effect for is exported; the rest stay internal, and the build does not fail for them. As asked: The nucleus defines hundreds of words, and some have no fixed effect, which 2c refuses as exports. Is every word with a fixed effect exported, or only those a list names? With a list, the words outside it stay internal.

  5. The tool's identity Ruled 2026-11-19: "Its source". The provenance's tool hash is the SHA-512 of the tool's source: mkf18a.c, text.c, effect.c and the v4 headers they include, in a fixed order. The same source gives the same hash on every host. As asked: The provenance holds "the host tool's hash". The hash of the binary differs with the host's compiler. Should it be the hash of the tool's source (mkf18a.c, text.c, effect.c and their headers), or of the binary?

  6. The name Ruled 2026-11-19: "mkf18a / .f18a", after the magic: v4/tools/mkf18a.c, and each train's artifact <train>.f18a. As asked: mkf18a and .f18a, after the magic, or something else?

7. Status, and the order of the work

All six questions are ruled, plus 2a and 3a. Each task below is built test-first, seen to fail, and accepted before the next:

  1. text.c's address record (§4, test 1), with test_host_rom_nucleus unchanged (test 4).
  2. effect.c, the stack-effect analysis (test 2).
  3. mkf18a and make -C v4 artifacts (test 3).
  4. The small train, placed at a base other than 0 (test 5).
  5. The nucleus renamed to name order (3a), designed first: it changes the boot, so the protocol runs.

8. Task 1 built (2026-11-19): text.c's address record

What changed:

  • text.h and text.c: v4_text_record(t, buf, cap) turns the record on, and v4_text_addresses(t) says how many were recorded. Each entry is a cell and its slot, 0–3 for a branch field or 7 (V4_TEXT_WHOLE_CELL) for the whole cell. What is recorded:

    • every branch, to a word or a label, back or forward, including NEXT's;
    • an &NAME literal in code, and #, &NAME;
    • here, and a set variable that holds an address, wherever it is laid down (code, #,, store);
    • a header's name cell, and its link when there is an entry before it.

    The map's constants and plain numbers are not recorded. A full record is an error. With the record off, nothing changes.

  • Found while building: the nucleus lays its headers down with set HDR-NFA here ... #, HDR-LINK, not with header. So a set variable carries whether it holds an address, or those links would be missed.

Test first (logs/20261119-text-record/):

  • test_text: 15 new checks. Before the change, 13 failed: every one that expects an address recorded. The two that expect nothing passed, as they should. Two errors in the tests themselves were fixed before the change: source needs its N-1 constant, and the notation is FOR ... NEXT, upper case.
  • test_host_rom_nucleus, with the record on (test 4). On the system's source it records 1825 branches and 664 whole cells. Every whole cell holds an address of the assembly, and the ROM's assembly is still equal to it word for word, at both widths. This check could not be seen to fail before: the record did not exist. It guards that the record changes no cell.

Verified:

  • make -C v4 test passes: all tests at both widths, and the three mesh boots with POST 990/990.
  • make -C v4 test-32 passes.
  • ASan+UBSan passes for every test at both widths, and for the mesh boot (POST 990/990). The run hit my one-hour background limit at the 64-bit test_host_mesh, so the rest was run again from there. v4-gates.txt and v4-sanitize-rest.txt hold the two parts.
  • The boot does not use the record, so the kernel and the protocol are unchanged.

9. Task 2 built (2026-11-19): effect.c, the stack effects

What it is: v4/include/v4/effect.h and v4/src/effect.c, a build-machine analysis, not part of the engine or the kernel. v4_effect_of(e, addr, how) walks a word's code from its entry and gives what it takes and leaves on both stacks, or why it has none, and where. The opcode counts are the engine's own table (exec.c, D-16). Effects of called words are kept, so each is walked once. A callback gives the effects of code outside the range analysed: the ROM's, or another train's.

Two ways a word is reached, from the header's flag 8:

  • Called: it returns with ; at its own entry depth on the return stack, so its return-stack effect is always 0 → 0. A ; with its own value on the return stack is a computed jump (EXECUTE, a DEFERred word), and has no fixed effect. So has touching its caller's return address, and ex.
  • In line: the body runs to the first ; wherever the return stack is. push in line is data 1 → 0 and return stack 0 → 1.

No fixed effect: paths that meet with different depths (a loop that leaves a count of values), returns that disagree, recursion, a word that never returns, and a call to a word with none.

Test first (logs/20261119-effect/): test_effect, 34 checks.

  • Against a stub that derived nothing, the 23 checks that expect a fixed effect failed. The 10 that expect none passed vacuously.
  • Three of my expectations were wrong, and the analysis was right. I corrected them:
    • a! @ ! is 1 → 0, not 2 → 0: the value fetched is stored back;
    • drop jump A, with A dup, is 2 → 2: dup needs a second cell;
    • 3 FOR dup drop NEXT is 1 → 1: the dup needs a cell already there. The fixed-loop check now uses 5 drop, and the 1 → 1 case is a check of its own.
  • Five real words agree with their stack comments: SWAP', UM/MOD, C@, C! and ROUNDTRIP (2 → 1, through its jump C@).

On the nucleus (nucleus-dictionary.txt, a one-off measurement): of 315 headers, 65 of them in line, 214 have a fixed effect and 101 have none.

  • 78 call a word with no fixed effect;
  • 11 have a computed jump;
  • 8 have returns that disagree;
  • 4 touch their caller's return address.

Under ruling §6.4, the 214 would be the nucleus's exports. The nucleus is not an artifact (§6.2a), so this is a measure of the analysis, not an interface.

Verified:

  • make -C v4 test passes, with all three mesh boots at POST 990/990.
  • make -C v4 test-32 passes.
  • make -C v4 sanitize passes: every test at both widths, and the mesh boot at POST 990/990, under ASan+UBSan.

10. Task 3, mkf18a: rulings while designing it

  1. Exports: ": NAME words, no headers" (2026-11-19). Every : word with a fixed effect is exported, named by its : name. A train's source lays no dictionary entry: placement makes the entries (ABI.md §5, step 4, as written). Asked because a header in the code would make each entry twice.
  2. Flags: "After the name, like header" (2026-11-19). The flags follow the : name, as they follow header today: : PLUS inline + ;, : [ immediate .... The ACL defaults come the same way, as pinned and mode N. text.c records them and lays nothing in the code. In-line also decides how the tool derives the word's effect (§9).
  3. Imports: "An import line" (2026-11-19). import TRAIN NAME declares NAME as from the train TRAIN, before it is used. The tool takes TRAIN's capsule hash, and the word's derived effects, from that train's build, so the build fails if TRAIN does not export NAME. Each call to NAME is a relocation to its import.
  4. A train's capsule hash: "Its .f18a's capsule ID" (2026-11-19). A train is identified by its engine: the capsule ID mkcapsule gives its .f18a. The nucleus has none (§6.2a), so a word imported from it is named with hash 0 and resolved by name in the node's dictionary at placement. Exact-hash imports (ABI.md choice 4) hold between artifact trains only. Asked because a capsule today is one file, and the nucleus is 16 blobs.
  5. Capabilities: "Declared, then checked" (2026-11-19). v4 has no port opcode: a port is an address, reached through A, B or P, whose values are data, so which words touch ports cannot be derived. A word declares what it needs after its name, like its flags: ports, request N. The tool records the declaration. Where it can see a port or a request, it fails the build if the declaration is missing: a literal port address put into A or B, or a kernel request with a literal number. What it cannot see is the declaration's. This changes ARTIFACT.md choice 4's "the tool derives it from the code".

11. Task 3 built (2026-11-19): mkf18a and make -C v4 artifacts

What changed.

  • text.c (part 1, committed first): the attributes after a : name (immediate data inline compile-only pinned mode N ports request N), recorded and laid nowhere in the code; import TRAIN NAME, whose uses are calls to a placeholder V4_NODE_WORDS-1-k and are recorded as relocations to import k. A use before the declaration fails, and so does code that reaches the placeholders.
  • include/v4/artifact.h, src/artifact.c: the layout of ARTIFACT.md and its check (§7), shared by the tool and, later, by placement.
  • tools/f18a.c, tools/f18a.h, tools/mkf18a.c: the tool, mkf18a OUT.f18a MAP.f18 ROM DIR [TRAIN=PATH.f18a ...]. It assembles the map, then DIR's .f18 files in name order, at 0, with the record on (§6.3). It refuses a header, a start, and a store; resolves each import from the trains it is given; derives the effects (§9) and exports every : word that has one (§6.4, ruling 1); checks the declared capabilities (ruling 5); and writes the relocations, the pool and the four hashes.
  • Makefile: mkf18a-32, mkf18a-64 and their tests; artifacts builds the tool at both widths. test runs mkf18a-32/64, sanitize their ASan+UBSan builds.

Test first, seen to fail. test_text had 11 failures of 274 checks before part 1 (logs/20261119-mkf18a/text-before-64.txt), none after (text-after.txt). test_mkf18a failed at its first check against a stub that built nothing (mkf18a-before-64.txt), and passes 83 checks at both widths after (mkf18a-after-64.txt, mkf18a-after.txt). It builds a library train and a train that imports from it, reads every unit back against text.c, builds both twice for the same bytes, and checks ten refusals: of the checker, and of the tool (r1-r9 refused, r10 built).

The test's own error. A name meant to be 32 characters, to be refused, was 31 and was taken; the test was corrected, not the tool.

Choices made in drafting. Ruled 2026-11-20, each "Keep as drafted", one at a time:

  1. The provenance hashes frame each input as its name, a NUL, its length in 8 bytes low first, then its bytes (f18a.h).
  2. The tool's hash covers tools/mkf18a.c, tools/f18a.c, tools/f18a.h, src/artifact.c, asm.c, effect.c, iword.c, text.c, then every include/v4/*.h, in that order. The configuration hash covers the text V4_CELL_BITS=.. V4_NODE_WORDS=.. V4_DATA_RING=.. V4_RET_RING=.. V4_ROM_WORDS=.., framed as config, then the map. The ROM's is the ROM file.
  3. The pool's unit 0 is always 0, so an offset of 0 means none.
  4. A store of a literal number to (PORT) counts as a request, not as touching the ports.
  5. Capabilities pass through calls: a word that calls one that declares ports must declare it too.
  6. The imports' placeholders are the node's top words, V4_NODE_WORDS-1-k.

Recorded, not fixed: a store into the code below here is not caught by the store check, which looks only above here.

Verified (logs/20261119-mkf18a/v4-gates.txt, exit 0):

  • make -C v4 test passes, with the mesh boots at POST 990/990, and mkf18a-64 and mkf18a-32 at 83 checks.
  • make -C v4 test-32 passes.
  • make -C v4 sanitize passes, mkf18a included at both widths.

12. Task 4, the small train placed: design, for ruling

Written 2026-11-20. Test 5 of §4: a few words of .f18 with one import from the nucleus, built by mkf18a into a .f18a, placed at a base other than 0. Its words run, and its call into the nucleus reaches the nucleus. Nothing here is built. The kernel and the boot are unchanged: carrying artifacts in capsules and placing them on a node are the ADR's steps 4 and 6 (§3).

What it needs that task 3 left out:

  • Imports from the nucleus. Ruling 4 (§10) names them with hash 0, resolved by name in the node's dictionary. To check the import's effect at build time, the tool needs the nucleus's effects. Task 2 derived them once, by the dictionary's names (logs/20261119-effect/nucleus-dictionary.txt); task 4 makes that a part of the tool.
  • Placement, ABI.md §5: check, place at the base, fix each relocation, enter each export.

The test, test_place, written first:

  1. A train of two files: : SQ dup + ; style words, one of them calling a nucleus word with a fixed effect (OVER or the like, picked from the 214), one calling the other.
  2. mkf18a builds it, with the nucleus import's hash 0 and its effect from the nucleus.
  3. The host system is booted as test_mesh_boot boots it. The train is placed into Hera at her HERE, which is not 0. HERE moves past it.
  4. Console lines run the words and print their results. Each must be as the same source gives when typed at the prompt. Each relocated cell must hold the base plus its offset, or the nucleus word's address.
  5. Refusals, each leaving the node unchanged: a nucleus import the dictionary lacks; an import whose effect disagrees; an artifact the checker refuses; a relocation out of its slot's reach at that base.

Ruled 2026-11-20, one at a time:

  1. Placement: "Host placer in C". v4/src/place.c checks, copies, patches and enters into a host node's memory, in the booted host system. It is the oracle for the loader on the node (step 6). Asked: a host placer in C, or the loader in FORTH on the node now, pulled forward from step 6.
  2. Entries: "Placer lays entries". The placer lays each export's entry as (HEADER) lays one: the ID, the flags with the ACL defaults, the name, the link, the code. It tells the kernel, as (WORD-DEFINED) does. The test runs the words by name at the console. Asked: entries now, or calls by address and entries left to step 6. 2a, ruled 2026-11-20: "Split at each export". An entry is laid right before its code (dict.f18: the four cells at xt-4 to xt-1, the code at xt), but the artifact's code is one block with no room between its words (ruling 1). So the placer lays each export's entry, then that export's code, up to the next export's entry offset. Code before the first export goes first, with no entry. Every cell moves by the entries laid before it, so every relocation is mapped through the moved offsets, and each one's reach is checked at its final address. Calls go straight to the code. A word whose code runs on into the next export's, with no ; or jump between, cannot be split, and the placer refuses it. Corrected 2026-11-20, ruled "In the tool": the tool refuses it, at build time, and the placer trusts the artifact for it. The placer cannot see it: the artifact names only the exports, and the word before an export may be internal, with no fixed effect. The tool knows every : word, and walks each with its range ending at the next. Asked: that, or the whole block at the base, with an entry after it whose xt cell jumps to the word.
  3. The nucleus's effects: "Tool assembles it". mkf18a is given the nucleus's directory, assembles it with its map, and derives each header's effect by name, as task 2's measurement did. Nothing is generated in between. Asked: that, or an effects file generated once per build.
  4. The import's name: "import nucleus NAME". nucleus is a reserved train name: the import has hash 0, its effect from the nucleus, and placement resolves it by name in the dictionary. Asked: that, or a directive of its own.

13. Task 4 built (2026-11-20): the small train placed

What changed.

  • tools/f18a.c, mkf18a.c: import nucleus NAME (ruling 4). Given nucleus=SYSDIR, the tool assembles the nucleus as the ROM does and looks NAME up by its dictionary name. The import has hash 0 and the effect derived there. A word in line there, or one with no fixed effect, is refused (ruling 3). The tool also refuses a word that runs on into the next export (2a, as corrected).
  • include/v4/place.h, src/place.c: the host placer, v4_train_place (rulings 1, 2, 2a). It checks everything before it writes anything. Then it places the code at HERE, split at each export with the export's entry laid before its code, fixes each relocation at its moved address, enters each export into CURRENT's vocabulary, moves DP, and tells the kernel of each xt.
  • tests/train-place/: the small train, two files. DIST calls the nucleus's ABS; DIST2 calls DIST; CLAMP0 has a forward branch; ADDR holds &DIST. The Makefile builds its .f18a with mkf18a-64 itself.
  • tests/test_mesh_boot.c: the system is booted, and the train is placed on Hera at her HERE (9782).
  • Makefile: the train's rule. The mesh boot's four builds link the placer. test_mkf18a is built with the host node's size, as the tool is: the nucleus did not fit a mesh node's.

Test first, seen to fail.

  • test_mkf18a, nucleus imports: 3 of 88 checks failed against a stub (logs/20261120-place/nucleus-before-64.txt). 93 passed after.
  • The run-on refusal failed first (runon-before-64.txt). Now 95 pass at both widths.
  • test_mesh_boot: 16 new checks failed against a stub placer (place-before.txt). Now 141 pass (place-after.txt).

What the placement checks:

  • the words run by name: 7 3 DIST . and 3 7 DIST . give 4, DIST2 8, CLAMP0 0 and 5;
  • ADDR equals ' DIST;
  • a word compiled at the prompt calls a placed one;
  • the kernel keeps a record of each export;
  • six refusals each leave Hera unchanged: a wrong magic, another node size, a name the dictionary lacks, an effect not the one expected, an import from a train, and no room below DLIMIT.

The test's own error. Its defined callback first only kept the xts, and did not tell the kernel. The test was corrected to run the word in (WORD-DEFINED) with each xt, as (DEFINED) does. The placer was not changed.

Choices made in building, for ruling:

  1. An import from the nucleus declares no capabilities: the nucleus declares none.
  2. The placer looks an import up in FORTH, from (LATEST), skipping hidden entries, and enters the exports into CURRENT's vocabulary.
  3. An import from an artifact train (a hash that is not 0) is refused for now, "not placed yet".
  4. The run-on check walks the word before each export with its range ending at that export. A word that has no fixed effect for another reason, found first, is not seen to run on.

Verified (logs/20261120-place/v4-gates.txt, exit 0):

  • make -C v4 test: the mesh boots at POST 990/990, the placement among them, and mkf18a at 95 checks at both widths.
  • make -C v4 test-32 passes.
  • make -C v4 sanitize passes, the mesh boot with the placement included.