From a4221d8b8c6d9addab3ed3ef7c659af522924e37 Mon Sep 17 00:00:00 2001 From: rajames Date: Tue, 27 Oct 2026 12:29:45 -0400 Subject: [PATCH] docs(v4): the plan for the stick's increment 3 -- the images Ruled (Captain Bob, 2026-10-27): the data stick is built and wired behind DATASTICK=1, off by default; increment 5 turns it on. Co-Authored-By: Claude Opus 5.5 --- .../v4.0.0/plans/2026-10-27-stick-3-images.md | 174 ++++++++++++++++++ 1 file changed, 174 insertions(+) create mode 100644 docs/v4.0.0/plans/2026-10-27-stick-3-images.md diff --git a/docs/v4.0.0/plans/2026-10-27-stick-3-images.md b/docs/v4.0.0/plans/2026-10-27-stick-3-images.md new file mode 100644 index 00000000..5229670f --- /dev/null +++ b/docs/v4.0.0/plans/2026-10-27-stick-3-images.md @@ -0,0 +1,174 @@ +# The stick, increment 3: the images — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (Captain Bob prefers native execution). Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** `make boot_image TARGET=PC` writes the stick (ESP + StarshipOS identity + StarshipOS Artemis, +Artemis formatted and signed at build); `make ... datastick` writes the data stick (identity + Artemis, +no ESP); QEMU boots the stick as a USB stick on amd64, and the `clean qemu` targets can attach the data +stick over USB on every ISA. + +**Architecture:** A host tool, `tools/mkartemis.c`, formats an Artemis partition inside a disk image +with the kernel's own code -- v3's `block_subsystem.c` (`blk_subsys_confirm_format`) and +`kernel/src/artemis_sig.c` (`artemis_sig_genesis_stamp`) -- through a small `blkio` backend over a +byte range of the image file, and checks one the same way. `scripts/mkdiskimage.sh` gains the two +StarshipOS partitions and calls it. The kernel is not changed: it reads these images with increment 2's +GPT reader as they are. + +**Tech Stack:** host C99 (`cc`, as `mkcapsule`), `sfdisk`/`mkfs.fat`/`mtools`, QEMU q35 + OVMF with +`usb-storage` on `qemu-xhci`; `scripts/qemu_session.py`. + +**Spec:** `docs/v4.0.0/STICK.md` §4 (layout, sizes, "the Artemis partition is formatted when the image +is built"), §9 (QEMU boots the stick image itself; the data stick), §11 increment 3, §12 (GUIDs in +`mkdiskimage.sh`). + +## Ruled (Captain Bob, 2026-10-27: "yes" -- the recommendation below, and the plan) + +**When the `clean qemu` boots switch to the data stick.** §11 puts "every `clean qemu` boot attaches the +data stick" in this increment, and §9 says it replaces the virtio disk and the scratch identity drive +and is "fresh for each boot, so the first boot mints". The kernel cannot use it that way until +increment 4 (Artemis from the stick, virtio Artemis off v4) and increment 5 (genesis, so a fresh +stick's first boot mints Zuse). Switched now, every `clean qemu` boot would be headless and acceptance +(`[zuse@Hera] ok> `) would fail until increment 5. Attached *beside* today's disks, it would put a blank +500 MiB identity partition into every acceptance boot's block chain ahead of nothing it serves, and +renumber the chain's LBNs, for no test it passes. + +**Recommended:** this increment builds the data stick and wires it into all three `qemu` targets behind +`DATASTICK=1` (off by default), tested by a session on each ISA; increment 5, which makes the first boot +mint, turns it on by default and retires the scratch pair (D4) and the virtio disk in the same commit. +STICK.md §11 is amended to say so. The plan below is written that way. + +## Global Constraints + +- As increment 2's plan: branch `StarForth-v4.0.0`, commits, strict C99, `-Werror` for kernel code, + v3 unchanged, QEMU/bare-metal parity, no fix outside the plan, full acceptance on every task + (three `clean qemu` boots to `[zuse@Hera] ok> `, POST 538/538, `dict_hash` `0xfec8c47cfc6f437f` + equal on all three, `make -C v4 test-32` 0 failures, logs committed). +- Partition type GUIDs (fixed in increment 2, `kernel/src/gpt.c`): + - StarshipOS identity `23BEBFE1-4817-4CF3-A537-9069BF159788` + - StarshipOS Artemis `BCD7B586-1C8E-4EEF-86A3-273A8D8EE624` + - EFI System `C12A7328-F81F-11D2-BA4B-00A0C93EC93B` +- Sizes (STICK.md §4), set once in `kernel/Makefile`: + `STICK_IMAGE_MIB := 2048`, `STICK_ESP_MIB := 64`, `STICK_IDENT_MIB := 500`. + Data stick (my call: identity as the stick's, Artemis 1 GiB as today's `disk/artemis.img`): + `DATASTICK_MIB := 1536`, identity `STICK_IDENT_MIB`, Artemis the rest. +- Layout: partition 1 at LBA 2048; each partition starts where the previous ends; every start and every + size a whole MiB (so even LBAs and even sector counts, as the GPT reader requires); the last + partition (Artemis) ends on the last whole MiB before the backup GPT's 33 sectors. Partition names + `EFI System`, `StarshipOS identity`, `StarshipOS Artemis`. +- The identity partition is all zeros in both images (sparse). +- Artemis is formatted exactly as the kernel formats a disk -- the same `block_subsystem.c` code, the + volume header `STFR` v2 at the partition's start, its BAM and relocation table -- and stamped with + an `artemis_sig_t` (`artemis_sig_genesis_stamp`, `disk_uuid` from the host's `getrandom(2)`). Each + built image has its own `disk_uuid`. Zuse's genesis marker is not written (increment 5 writes it on + first boot). +- The stick's ESP is today's PC image's ESP: `EFI/BOOT/BOOTX64.EFI`, `startup.nsh`, `starforth.cfg` + (`system=v4 $(KERNEL_ARGS)`). Only `BOARD=pc` gains the StarshipOS partitions; Milk-V's `uefi-esp` + image and the Pi's are unchanged. +- Images are sparse files; every copy is `cp --sparse=always`. + +## Review Focus + +1. **A partition that is not a whole number of 1 KiB blocks** -- an odd sector count from + "the rest of the disk" -- would be refused by the GPT reader on the stick. Pinned by + `check_images.sh` (every start and size even, Artemis ending on a MiB boundary). +2. **mkartemis writing outside its partition** -- the BAM or the signature landing in the identity + partition or the backup GPT. Pinned by `check_images.sh`: after formatting, the identity partition + is still all zeros and `sfdisk --verify` reports the GPT (both headers, both CRCs) clean. +3. **A `blkio` range backend that reads past its range** returns `BLKIO_EINVAL`, never the next + partition's bytes. Pinned by `mkartemis --check` reading the signature at the partition's top + (the last devblocks), which fails if the range's end is wrong. +4. **The stick booted twice** behaves the same (nothing written yet): pinned by `stick_boot` run on a + fresh copy; the second-boot case is increment 5's, when the first boot writes. +5. **A DATASTICK boot that loses Zuse** -- the data stick enumerated where Zuse's drive should be, or + its blank identity taken for hers -- pinned by `datastick` on all three ISAs: Zuse still logs in. + +--- + +### Task 1: mkartemis, and the two images + +**Files:** +- Create: `tools/mkartemis.c` +- Create: `scripts/check_images.sh` (the test) +- Modify: `scripts/mkdiskimage.sh` (StarshipOS partitions) +- Modify: `kernel/Makefile` (sizes; `MKARTEMIS_BIN` rule beside `MKCAPSULE_BIN`; `boot_image` for + `BOARD_STICK=1`; new `datastick` target), `boards/pc/board.mk` (`BOARD_STICK := 1`) +- Modify: `disk/README.md` (the two images, one paragraph) + +**Interfaces (produces):** +- `mkartemis --format IMG --first-lba L --sectors N` — formats and stamps the Artemis partition at + `[L, L+N)` 512-byte sectors of `IMG`; prints `mkartemis: formatted blocks, STFR v2, signature + stamped`; exit 0, else a message and exit 1. +- `mkartemis --check IMG --first-lba L --sectors N` — attaches the range with `blk_subsys_attach_device` + and reports `mkartemis: STFR v2, user blocks, signature ok` (exit 0) only if + `blk_get_volume_meta` gives magic `0x53544652` version 2 and `artemis_sig_check` gives + `ARTEMIS_SIG_OK`; otherwise names what failed, exit 1. +- The range backend: a `blkio_vtable_t` whose block n is the file's bytes `L*512 + n*1024`, `pread`/ + `pwrite`, refusing `n >= N/2` with `BLKIO_EINVAL`. `rng_get_bytes(uint8_t *buf, size_t n)` (the + signature `kernel/include/starkernel/rng.h` declares) is defined in `mkartemis.c` with `getrandom(2)`. +- Built from `tools/mkartemis.c`, v3's `V3_BLOCK_SRCS` as `v4/Makefile` lists them (`block_subsystem.c + blkio_ram.c blkio_file.c log.c platform/platform_init.c platform/linux/time.c`) and + `kernel/src/artemis_sig.c`, `-std=gnu99` as `v4/Makefile`'s `V3_CFLAGS`, into + `$(BUILD_DIR)/tools/mkartemis`. +- `mkdiskimage.sh ... --starship IDENT_MIB --mkartemis TOOL [--esp-mib E]`: with `--esp-mib E`, an ESP + of E MiB holding the SRC:DEST files, then identity, then Artemis; with `--esp-mib 0` and no files, + identity and Artemis only. Without `--starship`, exactly today's single-ESP image. +- `make boot_image TARGET=PC` → `build/boards/pc/lithos-pc.img`, `STICK_IMAGE_MIB`. +- `make -f kernel/Makefile ARCH= datastick` → `$(BUILD_DIR)/datastick.img`, `DATASTICK_MIB`, + rebuilt every time it is made (fresh, as §9 asks). + +- [ ] **Step 1: Failing test** — `scripts/check_images.sh` builds both images + (`make boot_image TARGET=PC`, `make -f kernel/Makefile ARCH=amd64 datastick`) and checks, with + `sfdisk --json`, `sfdisk --verify`, `mtools` (`-i IMG@@OFFSET`) and `cmp -n` against `/dev/zero`: + - stick: size 2048 MiB; three partitions; types EFI System / identity / Artemis in that order; starts + 2048, 2048+131072, 2048+131072+1024000; ESP 131072 sectors, identity 1024000; every start and size + even and a multiple of 2048; Artemis ends at or before the last usable LBA and within 2048 sectors + of it; `sfdisk --verify` clean; the ESP holds `EFI/BOOT/BOOTX64.EFI`, `startup.nsh`, and + `starforth.cfg` whose first word is `system=v4`; the identity partition is all zeros; + `mkartemis --check` on the Artemis partition passes. + - data stick: size 1536 MiB; two partitions, identity then Artemis, identity at 2048 of 1024000 + sectors; the same evenness, verify, zeros and `--check`. + - two data sticks built in a row have different `disk_uuid`s (`mkartemis --check` prints it). + Prints `PASS:`/`FAIL:` per line and `RESULT:`; exit 0 only on all PASS. +- [ ] **Step 2:** Run it: FAIL (no `datastick` target, the PC image has one partition). +- [ ] **Step 3:** Implement `mkartemis.c`, the `mkdiskimage.sh` options, the Makefile rules. +- [ ] **Step 4:** `check_images.sh` passes. A mutation check, once, not committed: shift the range + backend's base by one block and see `--check` fail. +- [ ] **Step 5:** Suite; full acceptance; commit `feat(build): the stick and the data stick, Artemis + formatted and signed at build` with the logs. + +### Task 2: QEMU boots the stick, and the data stick over USB + +**Files:** +- Modify: `kernel/Makefile` (`qemu-stick` target, amd64; `DATASTICK` in the three `qemu` targets) +- Modify: `scripts/qemu_session.py` (modes `stick_boot`, `datastick`; a `--target` option, default + `qemu`) +- Modify: `docs/v4.0.0/STICK.md` §11 (the amendment, as ruled), `boards/BRINGUP.md` §3.2 (how the + stick is rehearsed in QEMU, one paragraph) + +**Interfaces:** +- `make -f kernel/Makefile ARCH=amd64 clean qemu-stick [KERNEL_ARGS=...]`: builds the PC stick + (`$(MAKE) ... BOARD=pc boot_image`), copies it to `$(BUILD_DIR)/stick-run.img`, boots q35 + OVMF + (`-cpu qemu64`, as `qemu`) with the copy as `usb-storage,bus=xhci0.0,port=1,bootindex=0`; no + `-cdrom`, no virtio disk, no `ZUSEDISK`; serial, QMP and `logs//amd64/` exactly as `qemu`. + `ARCH` other than amd64: a message and exit 1 (ARM and RISC-V sticks are a non-goal, §10). +- `DATASTICK=1` on `qemu` (all three ISAs): makes `datastick`, then adds + `-drive if=none,id=datadrv,file=$(BUILD_DIR)/datastick.img,format=raw + -device usb-storage,bus=xhci0.0,port=2,drive=datadrv`. Default empty: the command line is unchanged. + +- [ ] **Step 1: Failing tests** (harness): + - `stick_boot` (amd64, `--target qemu-stick`, `KERNEL_ARGS=--log-level=info`): the loader runs from + the USB stick (`System: v4` and `POST: PASSED`); a `blk: disk` line whose user-block count is that + of the 500 MiB identity partition (between 300000 and 512000 -- the whole 2 GiB disk would be over + 1.5 million); no `ok> ` for 30 s (the identity partition is blank: headless, as increment 2 left + it); no fault. + - `datastick` (amd64, aarch64, riscv64; `--make "DATASTICK=1 KERNEL_ARGS=--log-level=info"`, Zuse's + scratch drive attached as usual): `[zuse@Hera] ok> `, `3 4 + .` gives `7 ok`; the log shows two + USB drives found; a `blk: disk` line for the data stick's identity partition (the same count as + `stick_boot`'s). + Today: FAIL (no `qemu-stick` target; `DATASTICK` does nothing, one drive found). +- [ ] **Step 2:** Run each; record the failures. +- [ ] **Step 3:** Implement the target and the `DATASTICK` arguments. +- [ ] **Step 4:** `stick_boot` passes; `datastick` passes on all three ISAs; the identity gate's + sessions, increment 1's and increment 2's GPT sessions still pass. +- [ ] **Step 5:** Suite; full acceptance; commit `feat(build): QEMU boots the stick; the data stick + over USB` with the logs.