Port DoE trial-summary persistence (doe_region.c) onto the reshuffled branch
Local master had 9 unpushed commits building block-backed DoE persistence on Artemis's own disk (FABRIC-3.md SXXXV.3/.4/.12, designed and built 2026-09-17, never pushed) that this branch never inherited after the Tripod/kernel reshuffle diverged history. Ports the mechanism cleanly onto the current Hera/Artemis/Hestia shape: - doe_region.h/.c: growable, CRC-protected trial-summary ring in Artemis's top-of-device metadata fence, immediately past log_region.c's own ceiling (devblock_from_top 98-102). Same accessor/CRC discipline as log_region.c. - log_words.c: DOE-PERSIST-TRIAL and DOE-RESUME-COUNT kernel-only FORTH words. - multiuser-doe.4th: per-ISA tagging (MU-ISA-TAG!) and campaign resume support (MU-EXEC-CAMPAIGN now takes a start-idx). - per-isa-doe.4th (new): RUN-AMD64-CAMPAIGN/RUN-AARCH64-CAMPAIGN/ RUN-RISCV64-CAMPAIGN entry points, resuming from DOE-RESUME-COUNT. - extract_doe_region.py (new): host-side reader, CRC-verified, no filesystem assumptions -- reads the ring directly off a raw disk image. Deliberately not ported: the R/LaTeX report-generation pipeline (analyse_multiuser_doe.R, multiuser_doe_report.tex, build_multiuser_doe_report.sh) -- not needed to prove persistence works, and unscoped for this pass. Verified live on all three architectures: clean boot, word_count 530->532, zero UNKNOWN WORD. amd64 and riscv64 both live-exercised end-to-end (write via DOE-PERSIST-TRIAL, count via DOE-RESUME-COUNT, clean QEMU exit, then read back correctly with extract_doe_region.py against the raw disk image -- no host serial dependency, matching the real bare-metal requirement this campaign is rehearsing for). riscv64 finding (not a defect in this port, no fix applied): DOE-PERSIST-TRIAL failed 3/3 on a first attempt against a fresh disk image with "virtio_blk: vblk_io timed out". Root cause: disk/artemis.img is a sparse file (180KB allocated of a 1GB apparent size); the new DOE region (devblock_from_top 98-102) is genuinely virgin territory on a fresh image, and riscv64's slower TCG emulation missed the existing virtio_blk spin-bound budget (already documented as fragile, FABRIC-3.md 2026-09-09) on that first cold touch. Confirmed by isolation: a plain BLOCK/UPDATE at an already-warm LBN succeeded instantly, the pre-existing LOG-ERROR" (same meta-zone mechanism, an already-warmed offset range) also succeeded, and a reboot reusing the now-partially-warmed disk let DOE-PERSIST-TRIAL succeed immediately and verify correctly on read-back. Pre-existing environmental fragility, not a logic bug in the ported address computation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
6302dcb50e
commit
2d773ca923
@@ -0,0 +1,200 @@
|
||||
/*
|
||||
StarForth — Steady-State Virtual Machine Runtime
|
||||
|
||||
Copyright (c) 2023–2025 Robert A. James
|
||||
All rights reserved.
|
||||
|
||||
This file is part of the StarForth project.
|
||||
|
||||
Licensed under the StarForth License, Version 1.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
|
||||
You may obtain a copy of the License at:
|
||||
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
|
||||
|
||||
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
express or implied, including but not limited to the warranties of
|
||||
merchantability, fitness for a particular purpose, and noninfringement.
|
||||
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* doe_region.h - Growable per-trial DoE-campaign-summary ring on Artemis's
|
||||
* own disk (FABRIC-3.md §XXXV.3/.4, built 2026-09-17)
|
||||
*
|
||||
* Records one summary per completed multiuser-doe.4th/per-isa-doe.4th trial
|
||||
* so a campaign's pass/fail and top-line physics picture survives past the
|
||||
* QEMU serial log -- §XXXV.1 identified that doe_log.c's per-tick rows
|
||||
* (console_puts()-only) cannot survive real bare-metal boot at all, and
|
||||
* raw per-tick mirroring to block storage was ruled out entirely on size
|
||||
* grounds (§XXXV.2: 516 MB for 8 trials vs. a ~1 GiB device). This region
|
||||
* persists trial SUMMARIES only (v1, §XXXV.4) -- exactly the fields
|
||||
* MU-EMIT-TRIAL-MARKER already computes, plus three trial-end aggregate
|
||||
* physics reads (vm_physics_fleet_heat_sum(), vm_physics_conserved(),
|
||||
* sk_vm_switch_signal_switch_count()) -- not a raw tick mirror.
|
||||
*
|
||||
* Same fence/accessor discipline log_region.h already established: lives
|
||||
* in Artemis's top-of-device system-metadata fence, reached via
|
||||
* block_subsystem.h's blk_meta_zone_read()/write() (devblock_from_top
|
||||
* addressing), same growable-ring control-header shape (magic/version/CRC,
|
||||
* devblocks/head_slot/tail_slot/record_count). A SEPARATE region from
|
||||
* log_region.c's own -- different shape (structured numeric fields, not
|
||||
* free text), different growth ceiling, and isolation so a runaway DoE
|
||||
* region can never crowd out real log persistence (§XXXV.3's own stated
|
||||
* reasoning, reused verbatim here).
|
||||
*
|
||||
* Fixed devblock_from_top allocation, immediately past log_region.c's own
|
||||
* ceiling (which occupies [65, 65+1+LOG_REGION_MAX_DEVBLOCKS-1] =
|
||||
* [65, 97] at full growth):
|
||||
* 98 -- this region's control header (DOE_REGION_DEVBLOCK_FROM_TOP_BASE)
|
||||
* 99-102 -- this region's slot devblocks, growable up to DOE_REGION_MAX_DEVBLOCKS
|
||||
*
|
||||
* Slot granularity is small (DOE_SLOT_SIZE, 64 bytes) relative to
|
||||
* log_slot_t's 1024 -- a trial summary is a handful of fixed numeric
|
||||
* fields plus an 8-byte ISA tag, nothing free-text-sized. 64 slots/devblock
|
||||
* x DOE_REGION_INITIAL_DEVBLOCKS(1) = 64 already exceeds one full 3-ISA
|
||||
* campaign's 54 trials (§XXXV.4); the growth ceiling (4 devblocks = 256
|
||||
* slots) is headroom for repeated campaigns, not a sizing risk.
|
||||
*
|
||||
* No priority-eviction logic (unlike log_region.c's LOG_REGION_PROTECTED_
|
||||
* MAX_LEVEL, §XXXII.4) -- there is no level concept for a trial summary;
|
||||
* plain FIFO eviction of the oldest record when the ring is full AND
|
||||
* already at the growth ceiling, which in practice will not be reached by
|
||||
* any campaign shape this project has run or ratified.
|
||||
*/
|
||||
|
||||
#ifndef STARKERNEL_DOE_REGION_H
|
||||
#define STARKERNEL_DOE_REGION_H
|
||||
|
||||
#include <stdint.h>
|
||||
#include "starkernel/log_region.h" /* LOG_REGION_DEVBLOCK_FROM_TOP_BASE, LOG_REGION_MAX_DEVBLOCKS -- fence math only */
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/*===========================================================================
|
||||
* Fence allocation
|
||||
*===========================================================================*/
|
||||
|
||||
#define DOE_REGION_DEVBLOCK_FROM_TOP_BASE \
|
||||
(LOG_REGION_DEVBLOCK_FROM_TOP_BASE + 1u + LOG_REGION_MAX_DEVBLOCKS) /* 98 */
|
||||
#define DOE_REGION_INITIAL_DEVBLOCKS 1u /* slot devblocks at first use, excludes control header */
|
||||
#define DOE_REGION_GROWTH_INCREMENT 1u
|
||||
#define DOE_REGION_MAX_DEVBLOCKS 4u /* ceiling -- devblock_from_top stays within [99,102] */
|
||||
|
||||
#define DOE_SLOT_SIZE 64u
|
||||
#define DOE_SLOTS_PER_DEVBLOCK (4096u / DOE_SLOT_SIZE) /* 64 */
|
||||
|
||||
/*===========================================================================
|
||||
* doe_region_ctrl_t - ring control header, one devblock at
|
||||
* DOE_REGION_DEVBLOCK_FROM_TOP_BASE. Same shape as log_region_ctrl_t.
|
||||
*===========================================================================*/
|
||||
|
||||
#define DOE_REGION_MAGIC 0x44454F44ull /* 'DOED' */
|
||||
#define DOE_REGION_VERSION_0 0
|
||||
|
||||
#define DOE_REGION_PACK(ver) \
|
||||
(DOE_REGION_MAGIC | ((uint64_t)(ver) << 32))
|
||||
#define DOE_REGION_GET_MAGIC(m) ((uint32_t)((m) & 0xFFFFFFFFull))
|
||||
#define DOE_REGION_GET_VERSION(m) ((uint8_t)(((m) >> 32) & 0xFF))
|
||||
|
||||
typedef struct {
|
||||
uint64_t magic; /* DOE_REGION_PACK(...) */
|
||||
uint32_t devblocks; /* current slot-area size, in devblocks (excludes this header) */
|
||||
uint32_t head_slot; /* index of the oldest live record */
|
||||
uint32_t tail_slot; /* index where the NEXT record will be written */
|
||||
uint32_t record_count; /* live records, <= devblocks * DOE_SLOTS_PER_DEVBLOCK */
|
||||
uint64_t hdr_crc; /* covers every field above this one */
|
||||
uint8_t _pad[4096 - (8 + 4 + 4 + 4 + 4 + 8)];
|
||||
} doe_region_ctrl_t;
|
||||
|
||||
typedef char doe_region_ctrl_size_check[(sizeof(doe_region_ctrl_t) == 4096) ? 1 : -1];
|
||||
|
||||
/*===========================================================================
|
||||
* doe_trial_slot_t - one trial summary record, exactly DOE_SLOT_SIZE bytes.
|
||||
* Field order: uint64_t pair first (natural 8-byte alignment at offsets
|
||||
* 0/8), then uint32_t fields (4-byte aligned from offset 16 on), then the
|
||||
* ISA tag, then an EXPLICIT alignment field (_align_pad) so slot_crc lands
|
||||
* on an offset the compiler needs no implicit padding to reach -- same
|
||||
* "every field lands on its natural alignment with zero compiler-inserted
|
||||
* padding" discipline log_slot_t's own doc comment states. Leaving this to
|
||||
* an implicit trailing _pad array (as an earlier draft did) put slot_crc
|
||||
* right after an 8-byte char array at a non-8-aligned offset, so the
|
||||
* compiler silently inserted its own padding there instead and the
|
||||
* hand-computed trailing pad size no longer matched sizeof() -- caught by
|
||||
* this file's own static size-check failing to compile.
|
||||
*===========================================================================*/
|
||||
|
||||
#define DOE_SLOT_ISA_MAX 8u /* NUL-padded, e.g. "amd64", "aarch64", "riscv64" */
|
||||
|
||||
typedef struct {
|
||||
uint64_t fleet_k_q48; /* vm_physics_fleet_heat_sum() at trial end */
|
||||
uint64_t switch_count_cumulative; /* sk_vm_switch_signal_switch_count() at trial end */
|
||||
uint32_t run_id;
|
||||
uint32_t cfg;
|
||||
uint32_t rep;
|
||||
uint32_t nw;
|
||||
uint32_t mode;
|
||||
uint32_t fail_count;
|
||||
uint32_t fleet_conserved; /* vm_physics_conserved() at trial end, 0/1 */
|
||||
char isa[DOE_SLOT_ISA_MAX];
|
||||
uint32_t _align_pad; /* explicit -- brings the next field to offset 56 (8-aligned) */
|
||||
uint64_t slot_crc; /* covers every field above this one (§XXXV.12) --
|
||||
* a devblock holds DOE_SLOTS_PER_DEVBLOCK=64 slots via
|
||||
* one read-modify-write, so a crash mid-write to ANY
|
||||
* slot in a devblock can corrupt its siblings; this is
|
||||
* how a reader tells a genuine record from corruption. */
|
||||
} doe_trial_slot_t;
|
||||
|
||||
typedef char doe_trial_slot_size_check[(sizeof(doe_trial_slot_t) == DOE_SLOT_SIZE) ? 1 : -1];
|
||||
|
||||
/*===========================================================================
|
||||
* API
|
||||
*===========================================================================*/
|
||||
|
||||
/*
|
||||
* doe_region_append - Write one trial-summary record to the ring, growing
|
||||
* it (within DOE_REGION_MAX_DEVBLOCKS) or evicting the oldest record (ring
|
||||
* full and already at the growth ceiling) as needed. Initializes the ring
|
||||
* on first use (control header blank).
|
||||
*
|
||||
* @param run_id, cfg, rep, nw, mode, fail_count Same fields
|
||||
* MU-EMIT-TRIAL-MARKER already prints (multiuser-doe.4th Block 5051).
|
||||
* @param fleet_k_q48, fleet_conserved, switch_count_cumulative Trial-end
|
||||
* aggregate physics reads (§XXXV.4).
|
||||
* @param isa ISA tag, e.g. "amd64" (may be empty for the pre-per-isa-
|
||||
* doe.4th single-ISA campaign shape).
|
||||
* @param isa_len Length of isa (truncated to DOE_SLOT_ISA_MAX-1).
|
||||
* @return 0 on success, -1 on any read/write failure (ring left however
|
||||
* the failed operation left it -- blk_meta_zone_write() itself
|
||||
* never partially writes a devblock).
|
||||
*/
|
||||
int doe_region_append(uint32_t run_id, uint32_t cfg, uint32_t rep, uint32_t nw,
|
||||
uint32_t mode, uint32_t fail_count,
|
||||
uint64_t fleet_k_q48, uint32_t fleet_conserved,
|
||||
uint64_t switch_count_cumulative,
|
||||
const char *isa, uint32_t isa_len);
|
||||
|
||||
/*
|
||||
* doe_region_count_for_isa - Count CRC-valid, live records in the ring
|
||||
* whose isa tag matches the given string (§XXXV.12, campaign crash-resume).
|
||||
* A record that fails its slot_crc check does not count -- the caller
|
||||
* (DOE-RESUME-COUNT) treats a corrupted slot as "not actually done," so
|
||||
* that trial safely re-runs rather than being silently skipped. Returns 0
|
||||
* if the ring has never been initialized (matching doe_region_append()'s
|
||||
* own "blank == not yet initialized" discipline), never negative.
|
||||
*
|
||||
* @param isa ISA tag to match, e.g. "amd64".
|
||||
* @param isa_len Length of isa (truncated to DOE_SLOT_ISA_MAX-1).
|
||||
* @return Count of matching, CRC-valid records (0 or more).
|
||||
*/
|
||||
uint32_t doe_region_count_for_isa(const char *isa, uint32_t isa_len);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
#endif /* STARKERNEL_DOE_REGION_H */
|
||||
Reference in New Issue
Block a user