docs(v4): the plan for the stick's increment 3 -- the images
Ruled (Captain Bob, 2026-10-27): the data stick is built and wired behind DATASTICK=1, off by default; increment 5 turns it on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
2e71f7aa0d
commit
a4221d8b8c
@@ -0,0 +1,174 @@
|
|||||||
|
# The stick, increment 3: the images — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (Captain Bob prefers native execution). Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** `make boot_image TARGET=PC` writes the stick (ESP + StarshipOS identity + StarshipOS Artemis,
|
||||||
|
Artemis formatted and signed at build); `make ... datastick` writes the data stick (identity + Artemis,
|
||||||
|
no ESP); QEMU boots the stick as a USB stick on amd64, and the `clean qemu` targets can attach the data
|
||||||
|
stick over USB on every ISA.
|
||||||
|
|
||||||
|
**Architecture:** A host tool, `tools/mkartemis.c`, formats an Artemis partition inside a disk image
|
||||||
|
with the kernel's own code -- v3's `block_subsystem.c` (`blk_subsys_confirm_format`) and
|
||||||
|
`kernel/src/artemis_sig.c` (`artemis_sig_genesis_stamp`) -- through a small `blkio` backend over a
|
||||||
|
byte range of the image file, and checks one the same way. `scripts/mkdiskimage.sh` gains the two
|
||||||
|
StarshipOS partitions and calls it. The kernel is not changed: it reads these images with increment 2's
|
||||||
|
GPT reader as they are.
|
||||||
|
|
||||||
|
**Tech Stack:** host C99 (`cc`, as `mkcapsule`), `sfdisk`/`mkfs.fat`/`mtools`, QEMU q35 + OVMF with
|
||||||
|
`usb-storage` on `qemu-xhci`; `scripts/qemu_session.py`.
|
||||||
|
|
||||||
|
**Spec:** `docs/v4.0.0/STICK.md` §4 (layout, sizes, "the Artemis partition is formatted when the image
|
||||||
|
is built"), §9 (QEMU boots the stick image itself; the data stick), §11 increment 3, §12 (GUIDs in
|
||||||
|
`mkdiskimage.sh`).
|
||||||
|
|
||||||
|
## Ruled (Captain Bob, 2026-10-27: "yes" -- the recommendation below, and the plan)
|
||||||
|
|
||||||
|
**When the `clean qemu` boots switch to the data stick.** §11 puts "every `clean qemu` boot attaches the
|
||||||
|
data stick" in this increment, and §9 says it replaces the virtio disk and the scratch identity drive
|
||||||
|
and is "fresh for each boot, so the first boot mints". The kernel cannot use it that way until
|
||||||
|
increment 4 (Artemis from the stick, virtio Artemis off v4) and increment 5 (genesis, so a fresh
|
||||||
|
stick's first boot mints Zuse). Switched now, every `clean qemu` boot would be headless and acceptance
|
||||||
|
(`[zuse@Hera] ok> `) would fail until increment 5. Attached *beside* today's disks, it would put a blank
|
||||||
|
500 MiB identity partition into every acceptance boot's block chain ahead of nothing it serves, and
|
||||||
|
renumber the chain's LBNs, for no test it passes.
|
||||||
|
|
||||||
|
**Recommended:** this increment builds the data stick and wires it into all three `qemu` targets behind
|
||||||
|
`DATASTICK=1` (off by default), tested by a session on each ISA; increment 5, which makes the first boot
|
||||||
|
mint, turns it on by default and retires the scratch pair (D4) and the virtio disk in the same commit.
|
||||||
|
STICK.md §11 is amended to say so. The plan below is written that way.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- As increment 2's plan: branch `StarForth-v4.0.0`, commits, strict C99, `-Werror` for kernel code,
|
||||||
|
v3 unchanged, QEMU/bare-metal parity, no fix outside the plan, full acceptance on every task
|
||||||
|
(three `clean qemu` boots to `[zuse@Hera] ok> `, POST 538/538, `dict_hash` `0xfec8c47cfc6f437f`
|
||||||
|
equal on all three, `make -C v4 test-32` 0 failures, logs committed).
|
||||||
|
- Partition type GUIDs (fixed in increment 2, `kernel/src/gpt.c`):
|
||||||
|
- StarshipOS identity `23BEBFE1-4817-4CF3-A537-9069BF159788`
|
||||||
|
- StarshipOS Artemis `BCD7B586-1C8E-4EEF-86A3-273A8D8EE624`
|
||||||
|
- EFI System `C12A7328-F81F-11D2-BA4B-00A0C93EC93B`
|
||||||
|
- Sizes (STICK.md §4), set once in `kernel/Makefile`:
|
||||||
|
`STICK_IMAGE_MIB := 2048`, `STICK_ESP_MIB := 64`, `STICK_IDENT_MIB := 500`.
|
||||||
|
Data stick (my call: identity as the stick's, Artemis 1 GiB as today's `disk/artemis.img`):
|
||||||
|
`DATASTICK_MIB := 1536`, identity `STICK_IDENT_MIB`, Artemis the rest.
|
||||||
|
- Layout: partition 1 at LBA 2048; each partition starts where the previous ends; every start and every
|
||||||
|
size a whole MiB (so even LBAs and even sector counts, as the GPT reader requires); the last
|
||||||
|
partition (Artemis) ends on the last whole MiB before the backup GPT's 33 sectors. Partition names
|
||||||
|
`EFI System`, `StarshipOS identity`, `StarshipOS Artemis`.
|
||||||
|
- The identity partition is all zeros in both images (sparse).
|
||||||
|
- Artemis is formatted exactly as the kernel formats a disk -- the same `block_subsystem.c` code, the
|
||||||
|
volume header `STFR` v2 at the partition's start, its BAM and relocation table -- and stamped with
|
||||||
|
an `artemis_sig_t` (`artemis_sig_genesis_stamp`, `disk_uuid` from the host's `getrandom(2)`). Each
|
||||||
|
built image has its own `disk_uuid`. Zuse's genesis marker is not written (increment 5 writes it on
|
||||||
|
first boot).
|
||||||
|
- The stick's ESP is today's PC image's ESP: `EFI/BOOT/BOOTX64.EFI`, `startup.nsh`, `starforth.cfg`
|
||||||
|
(`system=v4 $(KERNEL_ARGS)`). Only `BOARD=pc` gains the StarshipOS partitions; Milk-V's `uefi-esp`
|
||||||
|
image and the Pi's are unchanged.
|
||||||
|
- Images are sparse files; every copy is `cp --sparse=always`.
|
||||||
|
|
||||||
|
## Review Focus
|
||||||
|
|
||||||
|
1. **A partition that is not a whole number of 1 KiB blocks** -- an odd sector count from
|
||||||
|
"the rest of the disk" -- would be refused by the GPT reader on the stick. Pinned by
|
||||||
|
`check_images.sh` (every start and size even, Artemis ending on a MiB boundary).
|
||||||
|
2. **mkartemis writing outside its partition** -- the BAM or the signature landing in the identity
|
||||||
|
partition or the backup GPT. Pinned by `check_images.sh`: after formatting, the identity partition
|
||||||
|
is still all zeros and `sfdisk --verify` reports the GPT (both headers, both CRCs) clean.
|
||||||
|
3. **A `blkio` range backend that reads past its range** returns `BLKIO_EINVAL`, never the next
|
||||||
|
partition's bytes. Pinned by `mkartemis --check` reading the signature at the partition's top
|
||||||
|
(the last devblocks), which fails if the range's end is wrong.
|
||||||
|
4. **The stick booted twice** behaves the same (nothing written yet): pinned by `stick_boot` run on a
|
||||||
|
fresh copy; the second-boot case is increment 5's, when the first boot writes.
|
||||||
|
5. **A DATASTICK boot that loses Zuse** -- the data stick enumerated where Zuse's drive should be, or
|
||||||
|
its blank identity taken for hers -- pinned by `datastick` on all three ISAs: Zuse still logs in.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: mkartemis, and the two images
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `tools/mkartemis.c`
|
||||||
|
- Create: `scripts/check_images.sh` (the test)
|
||||||
|
- Modify: `scripts/mkdiskimage.sh` (StarshipOS partitions)
|
||||||
|
- Modify: `kernel/Makefile` (sizes; `MKARTEMIS_BIN` rule beside `MKCAPSULE_BIN`; `boot_image` for
|
||||||
|
`BOARD_STICK=1`; new `datastick` target), `boards/pc/board.mk` (`BOARD_STICK := 1`)
|
||||||
|
- Modify: `disk/README.md` (the two images, one paragraph)
|
||||||
|
|
||||||
|
**Interfaces (produces):**
|
||||||
|
- `mkartemis --format IMG --first-lba L --sectors N` — formats and stamps the Artemis partition at
|
||||||
|
`[L, L+N)` 512-byte sectors of `IMG`; prints `mkartemis: formatted <N/2> blocks, STFR v2, signature
|
||||||
|
stamped`; exit 0, else a message and exit 1.
|
||||||
|
- `mkartemis --check IMG --first-lba L --sectors N` — attaches the range with `blk_subsys_attach_device`
|
||||||
|
and reports `mkartemis: STFR v2, <user> user blocks, signature ok` (exit 0) only if
|
||||||
|
`blk_get_volume_meta` gives magic `0x53544652` version 2 and `artemis_sig_check` gives
|
||||||
|
`ARTEMIS_SIG_OK`; otherwise names what failed, exit 1.
|
||||||
|
- The range backend: a `blkio_vtable_t` whose block n is the file's bytes `L*512 + n*1024`, `pread`/
|
||||||
|
`pwrite`, refusing `n >= N/2` with `BLKIO_EINVAL`. `rng_get_bytes(uint8_t *buf, size_t n)` (the
|
||||||
|
signature `kernel/include/starkernel/rng.h` declares) is defined in `mkartemis.c` with `getrandom(2)`.
|
||||||
|
- Built from `tools/mkartemis.c`, v3's `V3_BLOCK_SRCS` as `v4/Makefile` lists them (`block_subsystem.c
|
||||||
|
blkio_ram.c blkio_file.c log.c platform/platform_init.c platform/linux/time.c`) and
|
||||||
|
`kernel/src/artemis_sig.c`, `-std=gnu99` as `v4/Makefile`'s `V3_CFLAGS`, into
|
||||||
|
`$(BUILD_DIR)/tools/mkartemis`.
|
||||||
|
- `mkdiskimage.sh ... --starship IDENT_MIB --mkartemis TOOL [--esp-mib E]`: with `--esp-mib E`, an ESP
|
||||||
|
of E MiB holding the SRC:DEST files, then identity, then Artemis; with `--esp-mib 0` and no files,
|
||||||
|
identity and Artemis only. Without `--starship`, exactly today's single-ESP image.
|
||||||
|
- `make boot_image TARGET=PC` → `build/boards/pc/lithos-pc.img`, `STICK_IMAGE_MIB`.
|
||||||
|
- `make -f kernel/Makefile ARCH=<arch> datastick` → `$(BUILD_DIR)/datastick.img`, `DATASTICK_MIB`,
|
||||||
|
rebuilt every time it is made (fresh, as §9 asks).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Failing test** — `scripts/check_images.sh` builds both images
|
||||||
|
(`make boot_image TARGET=PC`, `make -f kernel/Makefile ARCH=amd64 datastick`) and checks, with
|
||||||
|
`sfdisk --json`, `sfdisk --verify`, `mtools` (`-i IMG@@OFFSET`) and `cmp -n` against `/dev/zero`:
|
||||||
|
- stick: size 2048 MiB; three partitions; types EFI System / identity / Artemis in that order; starts
|
||||||
|
2048, 2048+131072, 2048+131072+1024000; ESP 131072 sectors, identity 1024000; every start and size
|
||||||
|
even and a multiple of 2048; Artemis ends at or before the last usable LBA and within 2048 sectors
|
||||||
|
of it; `sfdisk --verify` clean; the ESP holds `EFI/BOOT/BOOTX64.EFI`, `startup.nsh`, and
|
||||||
|
`starforth.cfg` whose first word is `system=v4`; the identity partition is all zeros;
|
||||||
|
`mkartemis --check` on the Artemis partition passes.
|
||||||
|
- data stick: size 1536 MiB; two partitions, identity then Artemis, identity at 2048 of 1024000
|
||||||
|
sectors; the same evenness, verify, zeros and `--check`.
|
||||||
|
- two data sticks built in a row have different `disk_uuid`s (`mkartemis --check` prints it).
|
||||||
|
Prints `PASS:`/`FAIL:` per line and `RESULT:`; exit 0 only on all PASS.
|
||||||
|
- [ ] **Step 2:** Run it: FAIL (no `datastick` target, the PC image has one partition).
|
||||||
|
- [ ] **Step 3:** Implement `mkartemis.c`, the `mkdiskimage.sh` options, the Makefile rules.
|
||||||
|
- [ ] **Step 4:** `check_images.sh` passes. A mutation check, once, not committed: shift the range
|
||||||
|
backend's base by one block and see `--check` fail.
|
||||||
|
- [ ] **Step 5:** Suite; full acceptance; commit `feat(build): the stick and the data stick, Artemis
|
||||||
|
formatted and signed at build` with the logs.
|
||||||
|
|
||||||
|
### Task 2: QEMU boots the stick, and the data stick over USB
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `kernel/Makefile` (`qemu-stick` target, amd64; `DATASTICK` in the three `qemu` targets)
|
||||||
|
- Modify: `scripts/qemu_session.py` (modes `stick_boot`, `datastick`; a `--target` option, default
|
||||||
|
`qemu`)
|
||||||
|
- Modify: `docs/v4.0.0/STICK.md` §11 (the amendment, as ruled), `boards/BRINGUP.md` §3.2 (how the
|
||||||
|
stick is rehearsed in QEMU, one paragraph)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- `make -f kernel/Makefile ARCH=amd64 clean qemu-stick [KERNEL_ARGS=...]`: builds the PC stick
|
||||||
|
(`$(MAKE) ... BOARD=pc boot_image`), copies it to `$(BUILD_DIR)/stick-run.img`, boots q35 + OVMF
|
||||||
|
(`-cpu qemu64`, as `qemu`) with the copy as `usb-storage,bus=xhci0.0,port=1,bootindex=0`; no
|
||||||
|
`-cdrom`, no virtio disk, no `ZUSEDISK`; serial, QMP and `logs/<TS>/amd64/` exactly as `qemu`.
|
||||||
|
`ARCH` other than amd64: a message and exit 1 (ARM and RISC-V sticks are a non-goal, §10).
|
||||||
|
- `DATASTICK=1` on `qemu` (all three ISAs): makes `datastick`, then adds
|
||||||
|
`-drive if=none,id=datadrv,file=$(BUILD_DIR)/datastick.img,format=raw
|
||||||
|
-device usb-storage,bus=xhci0.0,port=2,drive=datadrv`. Default empty: the command line is unchanged.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Failing tests** (harness):
|
||||||
|
- `stick_boot` (amd64, `--target qemu-stick`, `KERNEL_ARGS=--log-level=info`): the loader runs from
|
||||||
|
the USB stick (`System: v4` and `POST: PASSED`); a `blk: disk` line whose user-block count is that
|
||||||
|
of the 500 MiB identity partition (between 300000 and 512000 -- the whole 2 GiB disk would be over
|
||||||
|
1.5 million); no `ok> ` for 30 s (the identity partition is blank: headless, as increment 2 left
|
||||||
|
it); no fault.
|
||||||
|
- `datastick` (amd64, aarch64, riscv64; `--make "DATASTICK=1 KERNEL_ARGS=--log-level=info"`, Zuse's
|
||||||
|
scratch drive attached as usual): `[zuse@Hera] ok> `, `3 4 + .` gives `7 ok`; the log shows two
|
||||||
|
USB drives found; a `blk: disk` line for the data stick's identity partition (the same count as
|
||||||
|
`stick_boot`'s).
|
||||||
|
Today: FAIL (no `qemu-stick` target; `DATASTICK` does nothing, one drive found).
|
||||||
|
- [ ] **Step 2:** Run each; record the failures.
|
||||||
|
- [ ] **Step 3:** Implement the target and the `DATASTICK` arguments.
|
||||||
|
- [ ] **Step 4:** `stick_boot` passes; `datastick` passes on all three ISAs; the identity gate's
|
||||||
|
sessions, increment 1's and increment 2's GPT sessions still pass.
|
||||||
|
- [ ] **Step 5:** Suite; full acceptance; commit `feat(build): QEMU boots the stick; the data stick
|
||||||
|
over USB` with the logs.
|
||||||
Reference in New Issue
Block a user