Files
LithosAnanake/include/starkernel/doe_region.h
T
Robert Allan JamesandClaude Sonnet 5 2d773ca923 Port DoE trial-summary persistence (doe_region.c) onto the reshuffled branch
Local master had 9 unpushed commits building block-backed DoE persistence on
Artemis's own disk (FABRIC-3.md SXXXV.3/.4/.12, designed and built 2026-09-17,
never pushed) that this branch never inherited after the Tripod/kernel
reshuffle diverged history. Ports the mechanism cleanly onto the current
Hera/Artemis/Hestia shape:

- doe_region.h/.c: growable, CRC-protected trial-summary ring in Artemis's
  top-of-device metadata fence, immediately past log_region.c's own ceiling
  (devblock_from_top 98-102). Same accessor/CRC discipline as log_region.c.
- log_words.c: DOE-PERSIST-TRIAL and DOE-RESUME-COUNT kernel-only FORTH words.
- multiuser-doe.4th: per-ISA tagging (MU-ISA-TAG!) and campaign resume support
  (MU-EXEC-CAMPAIGN now takes a start-idx).
- per-isa-doe.4th (new): RUN-AMD64-CAMPAIGN/RUN-AARCH64-CAMPAIGN/
  RUN-RISCV64-CAMPAIGN entry points, resuming from DOE-RESUME-COUNT.
- extract_doe_region.py (new): host-side reader, CRC-verified, no filesystem
  assumptions -- reads the ring directly off a raw disk image.

Deliberately not ported: the R/LaTeX report-generation pipeline
(analyse_multiuser_doe.R, multiuser_doe_report.tex, build_multiuser_doe_report.sh)
-- not needed to prove persistence works, and unscoped for this pass.

Verified live on all three architectures: clean boot, word_count 530->532,
zero UNKNOWN WORD. amd64 and riscv64 both live-exercised end-to-end (write via
DOE-PERSIST-TRIAL, count via DOE-RESUME-COUNT, clean QEMU exit, then read back
correctly with extract_doe_region.py against the raw disk image -- no host
serial dependency, matching the real bare-metal requirement this campaign is
rehearsing for).

riscv64 finding (not a defect in this port, no fix applied): DOE-PERSIST-TRIAL
failed 3/3 on a first attempt against a fresh disk image with
"virtio_blk: vblk_io timed out". Root cause: disk/artemis.img is a sparse file
(180KB allocated of a 1GB apparent size); the new DOE region (devblock_from_top
98-102) is genuinely virgin territory on a fresh image, and riscv64's slower
TCG emulation missed the existing virtio_blk spin-bound budget (already
documented as fragile, FABRIC-3.md 2026-09-09) on that first cold touch.
Confirmed by isolation: a plain BLOCK/UPDATE at an already-warm LBN succeeded
instantly, the pre-existing LOG-ERROR" (same meta-zone mechanism, an
already-warmed offset range) also succeeded, and a reboot reusing the
now-partially-warmed disk let DOE-PERSIST-TRIAL succeed immediately and
verify correctly on read-back. Pre-existing environmental fragility, not a
logic bug in the ported address computation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 11:01:26 -04:00

201 lines
9.7 KiB
C
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* doe_region.h - Growable per-trial DoE-campaign-summary ring on Artemis's
* own disk (FABRIC-3.md §XXXV.3/.4, built 2026-09-17)
*
* Records one summary per completed multiuser-doe.4th/per-isa-doe.4th trial
* so a campaign's pass/fail and top-line physics picture survives past the
* QEMU serial log -- §XXXV.1 identified that doe_log.c's per-tick rows
* (console_puts()-only) cannot survive real bare-metal boot at all, and
* raw per-tick mirroring to block storage was ruled out entirely on size
* grounds (§XXXV.2: 516 MB for 8 trials vs. a ~1 GiB device). This region
* persists trial SUMMARIES only (v1, §XXXV.4) -- exactly the fields
* MU-EMIT-TRIAL-MARKER already computes, plus three trial-end aggregate
* physics reads (vm_physics_fleet_heat_sum(), vm_physics_conserved(),
* sk_vm_switch_signal_switch_count()) -- not a raw tick mirror.
*
* Same fence/accessor discipline log_region.h already established: lives
* in Artemis's top-of-device system-metadata fence, reached via
* block_subsystem.h's blk_meta_zone_read()/write() (devblock_from_top
* addressing), same growable-ring control-header shape (magic/version/CRC,
* devblocks/head_slot/tail_slot/record_count). A SEPARATE region from
* log_region.c's own -- different shape (structured numeric fields, not
* free text), different growth ceiling, and isolation so a runaway DoE
* region can never crowd out real log persistence (§XXXV.3's own stated
* reasoning, reused verbatim here).
*
* Fixed devblock_from_top allocation, immediately past log_region.c's own
* ceiling (which occupies [65, 65+1+LOG_REGION_MAX_DEVBLOCKS-1] =
* [65, 97] at full growth):
* 98 -- this region's control header (DOE_REGION_DEVBLOCK_FROM_TOP_BASE)
* 99-102 -- this region's slot devblocks, growable up to DOE_REGION_MAX_DEVBLOCKS
*
* Slot granularity is small (DOE_SLOT_SIZE, 64 bytes) relative to
* log_slot_t's 1024 -- a trial summary is a handful of fixed numeric
* fields plus an 8-byte ISA tag, nothing free-text-sized. 64 slots/devblock
* x DOE_REGION_INITIAL_DEVBLOCKS(1) = 64 already exceeds one full 3-ISA
* campaign's 54 trials (§XXXV.4); the growth ceiling (4 devblocks = 256
* slots) is headroom for repeated campaigns, not a sizing risk.
*
* No priority-eviction logic (unlike log_region.c's LOG_REGION_PROTECTED_
* MAX_LEVEL, §XXXII.4) -- there is no level concept for a trial summary;
* plain FIFO eviction of the oldest record when the ring is full AND
* already at the growth ceiling, which in practice will not be reached by
* any campaign shape this project has run or ratified.
*/
#ifndef STARKERNEL_DOE_REGION_H
#define STARKERNEL_DOE_REGION_H
#include <stdint.h>
#include "starkernel/log_region.h" /* LOG_REGION_DEVBLOCK_FROM_TOP_BASE, LOG_REGION_MAX_DEVBLOCKS -- fence math only */
#ifdef __cplusplus
extern "C" {
#endif
/*===========================================================================
* Fence allocation
*===========================================================================*/
#define DOE_REGION_DEVBLOCK_FROM_TOP_BASE \
(LOG_REGION_DEVBLOCK_FROM_TOP_BASE + 1u + LOG_REGION_MAX_DEVBLOCKS) /* 98 */
#define DOE_REGION_INITIAL_DEVBLOCKS 1u /* slot devblocks at first use, excludes control header */
#define DOE_REGION_GROWTH_INCREMENT 1u
#define DOE_REGION_MAX_DEVBLOCKS 4u /* ceiling -- devblock_from_top stays within [99,102] */
#define DOE_SLOT_SIZE 64u
#define DOE_SLOTS_PER_DEVBLOCK (4096u / DOE_SLOT_SIZE) /* 64 */
/*===========================================================================
* doe_region_ctrl_t - ring control header, one devblock at
* DOE_REGION_DEVBLOCK_FROM_TOP_BASE. Same shape as log_region_ctrl_t.
*===========================================================================*/
#define DOE_REGION_MAGIC 0x44454F44ull /* 'DOED' */
#define DOE_REGION_VERSION_0 0
#define DOE_REGION_PACK(ver) \
(DOE_REGION_MAGIC | ((uint64_t)(ver) << 32))
#define DOE_REGION_GET_MAGIC(m) ((uint32_t)((m) & 0xFFFFFFFFull))
#define DOE_REGION_GET_VERSION(m) ((uint8_t)(((m) >> 32) & 0xFF))
typedef struct {
uint64_t magic; /* DOE_REGION_PACK(...) */
uint32_t devblocks; /* current slot-area size, in devblocks (excludes this header) */
uint32_t head_slot; /* index of the oldest live record */
uint32_t tail_slot; /* index where the NEXT record will be written */
uint32_t record_count; /* live records, <= devblocks * DOE_SLOTS_PER_DEVBLOCK */
uint64_t hdr_crc; /* covers every field above this one */
uint8_t _pad[4096 - (8 + 4 + 4 + 4 + 4 + 8)];
} doe_region_ctrl_t;
typedef char doe_region_ctrl_size_check[(sizeof(doe_region_ctrl_t) == 4096) ? 1 : -1];
/*===========================================================================
* doe_trial_slot_t - one trial summary record, exactly DOE_SLOT_SIZE bytes.
* Field order: uint64_t pair first (natural 8-byte alignment at offsets
* 0/8), then uint32_t fields (4-byte aligned from offset 16 on), then the
* ISA tag, then an EXPLICIT alignment field (_align_pad) so slot_crc lands
* on an offset the compiler needs no implicit padding to reach -- same
* "every field lands on its natural alignment with zero compiler-inserted
* padding" discipline log_slot_t's own doc comment states. Leaving this to
* an implicit trailing _pad array (as an earlier draft did) put slot_crc
* right after an 8-byte char array at a non-8-aligned offset, so the
* compiler silently inserted its own padding there instead and the
* hand-computed trailing pad size no longer matched sizeof() -- caught by
* this file's own static size-check failing to compile.
*===========================================================================*/
#define DOE_SLOT_ISA_MAX 8u /* NUL-padded, e.g. "amd64", "aarch64", "riscv64" */
typedef struct {
uint64_t fleet_k_q48; /* vm_physics_fleet_heat_sum() at trial end */
uint64_t switch_count_cumulative; /* sk_vm_switch_signal_switch_count() at trial end */
uint32_t run_id;
uint32_t cfg;
uint32_t rep;
uint32_t nw;
uint32_t mode;
uint32_t fail_count;
uint32_t fleet_conserved; /* vm_physics_conserved() at trial end, 0/1 */
char isa[DOE_SLOT_ISA_MAX];
uint32_t _align_pad; /* explicit -- brings the next field to offset 56 (8-aligned) */
uint64_t slot_crc; /* covers every field above this one (§XXXV.12) --
* a devblock holds DOE_SLOTS_PER_DEVBLOCK=64 slots via
* one read-modify-write, so a crash mid-write to ANY
* slot in a devblock can corrupt its siblings; this is
* how a reader tells a genuine record from corruption. */
} doe_trial_slot_t;
typedef char doe_trial_slot_size_check[(sizeof(doe_trial_slot_t) == DOE_SLOT_SIZE) ? 1 : -1];
/*===========================================================================
* API
*===========================================================================*/
/*
* doe_region_append - Write one trial-summary record to the ring, growing
* it (within DOE_REGION_MAX_DEVBLOCKS) or evicting the oldest record (ring
* full and already at the growth ceiling) as needed. Initializes the ring
* on first use (control header blank).
*
* @param run_id, cfg, rep, nw, mode, fail_count Same fields
* MU-EMIT-TRIAL-MARKER already prints (multiuser-doe.4th Block 5051).
* @param fleet_k_q48, fleet_conserved, switch_count_cumulative Trial-end
* aggregate physics reads (§XXXV.4).
* @param isa ISA tag, e.g. "amd64" (may be empty for the pre-per-isa-
* doe.4th single-ISA campaign shape).
* @param isa_len Length of isa (truncated to DOE_SLOT_ISA_MAX-1).
* @return 0 on success, -1 on any read/write failure (ring left however
* the failed operation left it -- blk_meta_zone_write() itself
* never partially writes a devblock).
*/
int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
uint32_t mode, uint32_t fail_count,
uint64_t fleet_k_q48, uint32_t fleet_conserved,
uint64_t switch_count_cumulative,
const char *isa, uint32_t isa_len);
/*
* doe_region_count_for_isa - Count CRC-valid, live records in the ring
* whose isa tag matches the given string (§XXXV.12, campaign crash-resume).
* A record that fails its slot_crc check does not count -- the caller
* (DOE-RESUME-COUNT) treats a corrupted slot as "not actually done," so
* that trial safely re-runs rather than being silently skipped. Returns 0
* if the ring has never been initialized (matching doe_region_append()'s
* own "blank == not yet initialized" discipline), never negative.
*
* @param isa ISA tag to match, e.g. "amd64".
* @param isa_len Length of isa (truncated to DOE_SLOT_ISA_MAX-1).
* @return Count of matching, CRC-valid records (0 or more).
*/
uint32_t doe_region_count_for_isa(const char *isa, uint32_t isa_len);
#ifdef __cplusplus
}
#endif
#endif /* STARKERNEL_DOE_REGION_H */