v4: mkf18a, the host tool that builds a train's F18 artifact (ADR step 3, task 3)
Build / Image RASPI (push) Successful in 3m59s
Build / Image PC (push) Successful in 3m58s
Build / Image MILKV (push) Successful in 4m7s
Build / v4 tests (32- and 64-bit cells) (push) Successful in 16m5s

The artifact's layout and its check (artifact.h, artifact.c), the tool
(tools/f18a.c, mkf18a.c) and make -C v4 artifacts. test_mkf18a, 83
checks at both widths, failed against a stub first. Gates: test,
test-32 and sanitize pass, POST 990/990. Plan section 11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
Claude
2026-11-19 21:09:22 +00:00
parent 84f119a1b3
commit dc3803ab47
12 changed files with 2924 additions and 3 deletions
@@ -247,3 +247,50 @@ this is a measure of the analysis, not an interface.
declaration. Where it can see a port or a request, it fails the build if the declaration is missing:
a literal port address put into A or B, or a kernel request with a literal number. What it cannot see
is the declaration's. This changes `ARTIFACT.md` choice 4's "the tool derives it from the code".
## 11. Task 3 built (2026-11-19): `mkf18a` and `make -C v4 artifacts`
**What changed.**
- `text.c` (part 1, committed first): the attributes after a `:` name (`immediate data inline compile-only
pinned mode N ports request N`), recorded and laid nowhere in the code; `import TRAIN NAME`, whose uses are
calls to a placeholder `V4_NODE_WORDS-1-k` and are recorded as relocations to import k. A use before the
declaration fails, and so does code that reaches the placeholders.
- `include/v4/artifact.h`, `src/artifact.c`: the layout of `ARTIFACT.md` and its check (§7), shared by the
tool and, later, by placement.
- `tools/f18a.c`, `tools/f18a.h`, `tools/mkf18a.c`: the tool, `mkf18a OUT.f18a MAP.f18 ROM DIR
[TRAIN=PATH.f18a ...]`. It assembles the map, then DIR's `.f18` files in name order, at 0, with the record
on (§6.3). It refuses a header, a start, and a store; resolves each import from the trains it is given;
derives the effects (§9) and exports every `:` word that has one (§6.4, ruling 1); checks the declared
capabilities (ruling 5); and writes the relocations, the pool and the four hashes.
- `Makefile`: `mkf18a-32`, `mkf18a-64` and their tests; `artifacts` builds the tool at both widths. `test`
runs `mkf18a-32/64`, `sanitize` their ASan+UBSan builds.
**Test first, seen to fail.** `test_text` had 11 failures of 274 checks before part 1
(`logs/20261119-mkf18a/text-before-64.txt`), none after (`text-after.txt`). `test_mkf18a` failed at its
first check against a stub that built nothing (`mkf18a-before-64.txt`), and passes 83 checks at both widths
after (`mkf18a-after-64.txt`, `mkf18a-after.txt`). It builds a library train and a train that imports from
it, reads every unit back against `text.c`, builds both twice for the same bytes, and checks ten refusals:
of the checker, and of the tool (r1-r9 refused, r10 built).
**The test's own error.** A name meant to be 32 characters, to be refused, was 31 and was taken; the test
was corrected, not the tool.
**Choices made in drafting, for ruling:**
1. The provenance hashes frame each input as its name, a NUL, its length in 8 bytes low first, then its bytes
(`f18a.h`).
2. The tool's hash covers `tools/mkf18a.c`, `tools/f18a.c`, `tools/f18a.h`, `src/artifact.c`, `asm.c`,
`effect.c`, `iword.c`, `text.c`, then every `include/v4/*.h`, in that order. The configuration hash covers
the text `V4_CELL_BITS=.. V4_NODE_WORDS=.. V4_DATA_RING=.. V4_RET_RING=.. V4_ROM_WORDS=..`, framed as
`config`, then the map. The ROM's is the ROM file.
3. The pool's unit 0 is always 0, so an offset of 0 means none.
4. A store of a literal number to `(PORT)` counts as a request, not as touching the ports.
5. Capabilities pass through calls: a word that calls one that declares `ports` must declare it too.
6. The imports' placeholders are the node's top words, `V4_NODE_WORDS-1-k`.
**Recorded, not fixed:** a store into the code below `here` is not caught by the store check, which looks
only above `here`.
**Verified** (`logs/20261119-mkf18a/v4-gates.txt`, exit 0):
- `make -C v4 test` passes, with the mesh boots at POST 990/990, and `mkf18a-64` and `mkf18a-32` at 83 checks.
- `make -C v4 test-32` passes.
- `make -C v4 sanitize` passes, `mkf18a` included at both widths.
+8
View File
@@ -0,0 +1,8 @@
make: Entering directory '/home/user/LithosAnanke/v4'
[V4_CELL_BITS=64 mkf18a]
v4 mkf18a: V4_CELL_BITS=64
built, not refused
FAIL /home/user/LithosAnanke/v4/tests/test_mkf18a.c:317: an export's name longer than the dictionary's 31
83 checks, 1 failures
make: *** [Makefile:462: mkf18a-64] Error 1
make: Leaving directory '/home/user/LithosAnanke/v4'
+8
View File
@@ -0,0 +1,8 @@
make: Entering directory '/home/user/LithosAnanke/v4'
[V4_CELL_BITS=64 mkf18a]
v4 mkf18a: V4_CELL_BITS=64
83 checks, 0 failures
[V4_CELL_BITS=32 mkf18a]
v4 mkf18a: V4_CELL_BITS=32
83 checks, 0 failures
make: Leaving directory '/home/user/LithosAnanke/v4'
@@ -0,0 +1,7 @@
make: Entering directory '/home/user/LithosAnanke/v4'
[V4_CELL_BITS=64 mkf18a]
v4 mkf18a: V4_CELL_BITS=64
FAIL /home/user/LithosAnanke/v4/tests/test_mkf18a.c:165: LIB builds: not built yet
1 checks, 1 failures
make: *** [Makefile:462: mkf18a-64] Error 1
make: Leaving directory '/home/user/LithosAnanke/v4'
File diff suppressed because it is too large Load Diff
+44 -3
View File
@@ -31,7 +31,7 @@ HERE := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
SRCS := $(wildcard $(HERE)/src/*.c)
# test_mesh_boot.c is the kernel's boot of the fabric, linked as the hosted product is (its own rule,
# `mesh-boot`, below), not one of the per-width tests.
TESTS := $(filter-out %/test_mesh_boot.c %/test_inference.c,$(wildcard $(HERE)/tests/test_*.c))
TESTS := $(filter-out %/test_mesh_boot.c %/test_inference.c %/test_mkf18a.c,$(wildcard $(HERE)/tests/test_*.c))
# Cell widths to build and verify. 32 is the F18/Zynq case, 64 the aarch64
# host case; see DECOMPOSITION.md D-5.
@@ -357,7 +357,7 @@ $(foreach w,$(WIDTHS),$(foreach t,$(TESTS),$(eval $(call TEST_RULE,$(w),$(t)))))
$(foreach w,$(WIDTHS),$(foreach t,$(TESTS),$(eval $(call RUN_RULE,$(w),$(t)))))
$(foreach w,$(WIDTHS),$(foreach t,$(TESTS),$(eval $(call SAN_RULE,$(w),$(t)))))
sanitize: $(foreach w,$(WIDTHS),$(foreach t,$(TESTS),san-$(w)-$(notdir $(t)))) san-mesh-boot
sanitize: $(foreach w,$(WIDTHS),$(foreach t,$(TESTS),san-$(w)-$(notdir $(t)))) san-mesh-boot $(foreach w,$(WIDTHS),san-mkf18a-$(w))
@echo "all v4 tests passed under ASan+UBSan"
.PHONY: $(foreach w,$(WIDTHS),$(foreach t,$(TESTS),run-$(w)-$(notdir $(t)) san-$(w)-$(notdir $(t))))
@@ -424,7 +424,48 @@ $(BINDIR)/test_inference: $(HERE)/tests/test_inference.c $(HERE)/system/inferenc
inference: $(BINDIR)/test_inference
@$(BINDIR)/test_inference
test: $(addprefix test-,$(WIDTHS)) mesh-boot mesh-boot-on mesh-boot-noop inference
# THE HOST TOOL (docs/v4.0.0/plans/2026-11-19-f18-artifact-build.md section 2): mkf18a builds a train's
# engine, its F18 artifact, off the node, as mkrom builds the ROM -- one per cell width, with the host
# node's size. It hashes with the kernel's SHA-512 and names trains with its XXHash64, as mkcapsule does.
# test_mkf18a (section 4, test 3) builds two small trains with the same code and reads them back.
F18A_SRCS := $(HERE)/tools/f18a.c $(ROOT_DIR)/kernel/src/crypto/sha512.c $(ROOT_DIR)/kernel/src/hash/xxhash64.c
F18A_INC := -isystem $(ROOT_DIR)/kernel/include
F18A_DEPS := $(HERE)/tools/f18a.c $(HERE)/tools/f18a.h $(SRCS) $(wildcard $(HERE)/include/v4/*.h) $(HERE)/Makefile
define F18A_RULE
$(BINDIR)/mkf18a-$(1): $(HERE)/tools/mkf18a.c $$(F18A_DEPS)
@mkdir -p $(BINDIR)
$$(CC) $$(CFLAGS) -I$(HERE)/include $(F18A_INC) -DV4_CELL_BITS=$(1) $(HOST_DEFS) -DV4_ROOT='"$(HERE)"' \
$(HERE)/tools/mkf18a.c $(F18A_SRCS) $$(SRCS) -o $$@
$(BINDIR)/$(1)-test_mkf18a: $(HERE)/tests/test_mkf18a.c $$(F18A_DEPS)
@mkdir -p $(BINDIR)
$$(CC) $$(CFLAGS) -I$(HERE)/include $(F18A_INC) -DV4_CELL_BITS=$(1) -DSCRATCH='"$(BINDIR)/scratch-mkf18a-$(1)"' -DV4_ROOT='"$(HERE)"' \
$(HERE)/tests/test_mkf18a.c $(F18A_SRCS) $$(SRCS) -o $$@
$(BINDIR)/san-$(1)-test_mkf18a: $(HERE)/tests/test_mkf18a.c $$(F18A_DEPS)
@mkdir -p $(BINDIR)
$$(CC) $$(CSTD) $$(WARN) -O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined -fno-sanitize-recover=all \
-I$(HERE)/include $(F18A_INC) -DV4_CELL_BITS=$(1) -DSCRATCH='"$(BINDIR)/scratch-san-mkf18a-$(1)"' -DV4_ROOT='"$(HERE)"' \
$(HERE)/tests/test_mkf18a.c $(F18A_SRCS) $$(SRCS) -o $$@
.PHONY: mkf18a-$(1) san-mkf18a-$(1)
mkf18a-$(1): $(BINDIR)/$(1)-test_mkf18a
@echo " [V4_CELL_BITS=$(1) mkf18a]"
@$(BINDIR)/$(1)-test_mkf18a
san-mkf18a-$(1): $(BINDIR)/san-$(1)-test_mkf18a
@echo " [ASan+UBSan V4_CELL_BITS=$(1) mkf18a]"
@$(BINDIR)/san-$(1)-test_mkf18a
endef
$(foreach w,$(WIDTHS),$(eval $(call F18A_RULE,$(w))))
# The artifacts of each width (section 3). No train has a .f18 directory of its own yet, so this is the
# tool; the first train comes with task 4.
.PHONY: artifacts
artifacts: $(foreach w,$(WIDTHS),$(BINDIR)/mkf18a-$(w))
test: $(addprefix test-,$(WIDTHS)) mesh-boot mesh-boot-on mesh-boot-noop inference artifacts $(foreach w,$(WIDTHS),mkf18a-$(w))
@echo "all v4 tests passed"
clean:
+46
View File
@@ -0,0 +1,46 @@
/* SPDX-License-Identifier: LicenseRef-Starship-1.0
* Copyright (c) 2023-2026 Robert A. James
*/
/* artifact.h -- a train's engine, as an F18 artifact: its layout, and the check of it.
*
* docs/v4.0.0/ARTIFACT.md, as ruled. The records are 32-bit units, low byte first; the code is cells of
* the build's width, each low byte first. In order:
* header 16, provenance 64, exports E x 8, imports I x 6, relocations R x 2, verification 18,
* string pool S, code C cells
* The string pool's unit 0 is always 0, so an offset of 0 means "none" (ARTIFACT.md section 5).
*
* v4_f18a_check is section 7, "what makes a valid artifact": what the tool writes, and what placement
* checks before it places anything.
*/
#ifndef V4_ARTIFACT_H
#define V4_ARTIFACT_H
#include <stddef.h>
#include <stdint.h>
#define V4_F18A_MAGIC 0x41383146u /* "F18A", low byte first */
#define V4_F18A_ABI 0u
#define V4_F18A_HEADER 16u
#define V4_F18A_PROV 64u
#define V4_F18A_EXPORT 8u
#define V4_F18A_IMPORT 6u
#define V4_F18A_RELOC 2u
#define V4_F18A_VERIFY 18u
#define V4_F18A_INTERNAL 0xFFFFFFFFu /* a relocation within the code: the base is added */
#define V4_F18A_WHOLE 7u /* a relocation's slot: the whole cell holds the address */
#define V4_F18A_PORTS 0x80000000u /* an export's capabilities: it touches ports */
/* Where each part is, in units from the start, and the counts. */
typedef struct {
uint32_t cell_bits, node_words;
uint32_t e, i, r, s, c;
size_t exports, imports, relocs, verify, pool, code;
} v4_f18a_layout;
/* The unit at unit offset `u`. */
uint32_t v4_f18a_unit(const unsigned char *a, size_t u);
/* 1 and *lay if the `len` bytes at `a` are a valid artifact of `cell_bits`-bit cells; else 0 and *why. */
int v4_f18a_check(const unsigned char *a, size_t len, unsigned cell_bits, v4_f18a_layout *lay, const char **why);
#endif /* V4_ARTIFACT_H */
+90
View File
@@ -0,0 +1,90 @@
/* SPDX-License-Identifier: LicenseRef-Starship-1.0
* Copyright (c) 2023-2026 Robert A. James
*/
/* artifact.c -- the check of an F18 artifact (artifact.h, ARTIFACT.md section 7). */
#include "v4/artifact.h"
#include "v4/iword.h"
#include "v4/opcode.h"
#define LIMIT 0x01000000u /* no count is larger: it keeps every sum below in range */
uint32_t v4_f18a_unit(const unsigned char *a, size_t u)
{
const unsigned char *p = a + u * 4u;
return (uint32_t)p[0] | ((uint32_t)p[1] << 8) | ((uint32_t)p[2] << 16) | ((uint32_t)p[3] << 24);
}
static int bad(const char **why, const char *what)
{
*why = what;
return 0;
}
/* a counted string at pool offset `off`, of 1 to `max` characters */
static int string_ok(const unsigned char *a, const v4_f18a_layout *l, uint32_t off, unsigned max)
{
unsigned len;
if (off == 0 || off >= l->s) return 0;
len = a[(l->pool + off) * 4u];
if (len == 0 || len > max) return 0;
return off + (1u + len + 3u) / 4u <= l->s;
}
int v4_f18a_check(const unsigned char *a, size_t len, unsigned cell_bits, v4_f18a_layout *l, const char **why)
{
size_t u, cell_bytes = cell_bits / 8u;
uint32_t k;
*why = NULL;
if (len < V4_F18A_HEADER * 4u) return bad(why, "shorter than a header");
if (v4_f18a_unit(a, 0) != V4_F18A_MAGIC) return bad(why, "not an F18 artifact: the magic");
if (v4_f18a_unit(a, 1) != V4_F18A_ABI) return bad(why, "an ABI version this does not take");
l->cell_bits = v4_f18a_unit(a, 2);
if (l->cell_bits != cell_bits || (cell_bits != 32u && cell_bits != 64u)) return bad(why, "another cell width");
l->node_words = v4_f18a_unit(a, 3);
l->e = v4_f18a_unit(a, 4); l->i = v4_f18a_unit(a, 5); l->r = v4_f18a_unit(a, 6);
l->s = v4_f18a_unit(a, 7); l->c = v4_f18a_unit(a, 8);
if (l->e >= LIMIT || l->i >= LIMIT || l->r >= LIMIT || l->s >= LIMIT || l->c >= LIMIT) return bad(why, "a count out of range");
if (v4_f18a_unit(a, 9) != V4_F18A_HEADER + V4_F18A_PROV) return bad(why, "the exports are not after the provenance");
for (u = 10; u < V4_F18A_HEADER; u++) if (v4_f18a_unit(a, u) != 0) return bad(why, "a reserved unit of the header is not 0");
l->exports = V4_F18A_HEADER + V4_F18A_PROV;
l->imports = l->exports + (size_t)V4_F18A_EXPORT * l->e;
l->relocs = l->imports + (size_t)V4_F18A_IMPORT * l->i;
l->verify = l->relocs + (size_t)V4_F18A_RELOC * l->r;
l->pool = l->verify + V4_F18A_VERIFY;
l->code = l->pool + l->s;
if (len != l->code * 4u + (size_t)l->c * cell_bytes) return bad(why, "its length is not what its counts make");
if (l->s == 0 || v4_f18a_unit(a, l->pool) != 0) return bad(why, "the string pool does not start with 0");
for (k = 0; k < l->e; k++) {
size_t x = l->exports + (size_t)V4_F18A_EXPORT * k;
uint32_t caps = v4_f18a_unit(a, x + 5) & ~V4_F18A_PORTS;
if (!string_ok(a, l, v4_f18a_unit(a, x), 31u)) return bad(why, "an export's name is not a counted string of the pool");
if (v4_f18a_unit(a, x + 1) >= l->c) return bad(why, "an export's entry is outside the code");
if (caps != 0 && (caps >= l->s || caps + 1u + v4_f18a_unit(a, l->pool + caps) > l->s))
return bad(why, "an export's requests are not a list of the pool");
if (v4_f18a_unit(a, x + 6) != 0 || v4_f18a_unit(a, x + 7) != 0) return bad(why, "a reserved unit of an export is not 0");
}
for (k = 0; k < l->i; k++) {
size_t x = l->imports + (size_t)V4_F18A_IMPORT * k;
if (!string_ok(a, l, v4_f18a_unit(a, x + 2), 31u)) return bad(why, "an import's name is not a counted string of the pool");
if (v4_f18a_unit(a, x + 5) != 0) return bad(why, "a reserved unit of an import is not 0");
}
for (k = 0; k < l->r; k++) {
size_t x = l->relocs + (size_t)V4_F18A_RELOC * k;
uint32_t u0 = v4_f18a_unit(a, x), to = v4_f18a_unit(a, x + 1);
uint32_t cell = u0 & 0x1FFFFFFFu, slot = u0 >> 29;
if (cell >= l->c) return bad(why, "a relocation's cell is outside the code");
if (slot < 4u) {
/* an instruction word is the cell's low 32 bits, its first unit */
v4_iword w = (v4_iword)v4_f18a_unit(a, l->code + (size_t)cell * (cell_bytes / 4u));
if (!v4_op_is_branch(v4_iword_op(w, slot))) return bad(why, "a relocation's slot holds no branch");
} else if (slot != V4_F18A_WHOLE) return bad(why, "a relocation's slot is not 0-3 or 7");
if (to != V4_F18A_INTERNAL && to >= l->i) return bad(why, "a relocation's import is beyond the imports");
}
if (v4_f18a_unit(a, l->verify + 16) != 0 && !string_ok(a, l, v4_f18a_unit(a, l->verify + 16), 255u))
return bad(why, "the proof's assumptions are not a counted string of the pool");
if (v4_f18a_unit(a, l->verify + 17) != 0) return bad(why, "a reserved unit of the verification is not 0");
return 1;
}
File diff suppressed because it is too large Load Diff
+540
View File
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
/* SPDX-License-Identifier: LicenseRef-Starship-1.0
* Copyright (c) 2023-2026 Robert A. James
*/
/* f18a.h -- build a train's engine, its F18 artifact, on the build machine (mkf18a).
*
* docs/v4.0.0/plans/2026-11-19-f18-artifact-build.md section 2 and section 10. Every .f18 in the train's
* directory is assembled by text.c, at 0, in name order, after the map; then:
* - every : word with a fixed stack effect (effect.c) is an export, with the flags and the capabilities
* written after its name;
* - every `import TRAIN NAME` is an import of TRAIN's export NAME: TRAIN is given as an artifact, whose
* capsule ID (the XXHash64 of its bytes, as mkcapsule gives it) names it, and whose export gives the
* effects expected;
* - every address text.c records is a relocation;
* - the provenance's four SHA-512 hashes, each over files framed as: the file's name, a NUL, its length
* in 8 bytes low first, its bytes. The source: the train's .f18, in name order. The tool: its own
* source (tools/mkf18a.c, tools/f18a.c, tools/f18a.h, src/artifact.c, src/asm.c, src/effect.c,
* src/iword.c, src/text.c, then every .h of include/v4 in name order), named from the v4 directory.
* The configuration: the text "V4_CELL_BITS=.. V4_NODE_WORDS=.. V4_DATA_RING=.. V4_RET_RING=..
* V4_ROM_WORDS=..", framed as "config", then the map. The ROM: the ROM's file as given.
* It refuses what the ABI cannot carry: a header, a store or a start in the source; an export's name
* over 31 characters; a port or a request it can see that the word does not declare, or one declared by
* a word it calls.
*/
#ifndef V4_TOOLS_F18A_H
#define V4_TOOLS_F18A_H
#include <stddef.h>
typedef struct {
const char *name; /* as `import` names it */
const char *path; /* its .f18a */
} f18a_train;
typedef struct {
const char *map; /* the memory map's constants: map-<width>.f18 */
const char *dir; /* the train's .f18 */
const char *rom; /* the ROM it is built against */
const f18a_train *trains; /* the trains it may import from */
unsigned ntrains;
const char *tool_root; /* the v4 directory, for the tool's hash */
} f18a_args;
/* 1 and the artifact in *out (malloc'd, *len bytes), or 0 and why in err. */
int f18a_build(const f18a_args *a, unsigned char **out, size_t *len, char *err, size_t errlen);
#endif /* V4_TOOLS_F18A_H */
+50
View File
@@ -0,0 +1,50 @@
/* SPDX-License-Identifier: LicenseRef-Starship-1.0
* Copyright (c) 2023-2026 Robert A. James
*/
/* mkf18a.c -- build a train's engine, its F18 artifact, off the node (f18a.h).
*
* mkf18a OUT.f18a MAP.f18 ROM DIR [TRAIN=PATH.f18a ...]
*
* docs/v4.0.0/plans/2026-11-19-f18-artifact-build.md section 2. Built as mkrom is, one per cell width,
* with the node's size. DIR holds the train's .f18; each TRAIN=PATH is a train it may import from.
*/
#include "f18a.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef V4_ROOT
#error "V4_ROOT must name the v4 directory"
#endif
int main(int argc, char **argv)
{
f18a_args a;
f18a_train trains[64];
unsigned char *out = NULL;
size_t len = 0;
char err[512];
int i;
FILE *f;
if (argc < 5) { fprintf(stderr, "usage: mkf18a OUT.f18a MAP.f18 ROM DIR [TRAIN=PATH.f18a ...]\n"); return 2; }
a.map = argv[2];
a.rom = argv[3];
a.dir = argv[4];
a.ntrains = 0;
for (i = 5; i < argc; i++) {
char *eq = strchr(argv[i], '=');
if (!eq || a.ntrains == 64) { fprintf(stderr, "mkf18a: not TRAIN=PATH: %s\n", argv[i]); return 2; }
*eq = 0;
trains[a.ntrains].name = argv[i];
trains[a.ntrains].path = eq + 1;
a.ntrains++;
}
a.trains = trains;
a.tool_root = V4_ROOT;
if (!f18a_build(&a, &out, &len, err, sizeof err)) { fprintf(stderr, "mkf18a: %s: %s\n", a.dir, err); return 1; }
f = fopen(argv[1], "wb");
if (!f || fwrite(out, 1, len, f) != len || fclose(f) != 0) { fprintf(stderr, "mkf18a: cannot write %s\n", argv[1]); return 1; }
free(out);
return 0;
}