Build the DoE region reader + automated R/LaTeX report pipeline
Closes the "still open" reader-tool note from §XXXV.10: scripts/ extract_doe_region.py spools DOE-PERSIST-TRIAL's on-disk binary ring off disk/artemis.img into a CSV -- direct file read by default, or via a real loop device (--loop, losetup) per the literal request. Independent Python reimplementation of doe_region.c's own ring-walk, verified against a live 3-record write/read round trip. analyse_multiuser_doe.R (house style, ggplot2/svglite) and multiuser_doe_report.tex (same preamble convention as bare_metal_doe_report.tex) auto-discover whatever data exists and degrade gracefully to a placeholder when a campaign has 0 trials so far -- the whole point being this can be rebuilt at any point during a multi-day campaign, including mid-run. build_multiuser_doe_report.sh chains extraction -> R -> svg_to_pdf.py -> pdflatex into one command. Found and fixed a real bug while verifying: the R script's SCRIPT_DIR detection silently wrote charts/tables/report output to the repo root instead of the analysis directory when invoked from a different cwd -- switched to the robust commandArgs() "--file=" idiom. Verified live both branches (0 trials and a real 3-record write), reverted synthetic test data and incidental svg_to_pdf.py churn on ~85 unrelated committed figures before committing. FABRIC-3.md §XXXV.11 has the full account. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWpNjdwPtFLuVLaAq44L9K
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
aaf349946f
commit
c535252907
+75
@@ -5642,3 +5642,78 @@ scope per §XXXV.3 was the writer only, verified here by a one-off Python script
|
||||
bytes directly, not a maintained tool. Launching the first live multi-hour campaign is still a
|
||||
separate, later, explicit decision -- this entry closes the persistence-writer gap, nothing more.
|
||||
|
||||
### XXXV.11 -- The reader tool from §XXXV.10's own "still open" note: built, plus a full
|
||||
R/LaTeX report pipeline, all live-verified (2026-09-17)
|
||||
|
||||
Requested directly: "mount the virtual drive on a loop device... spool the csv's into a file
|
||||
for the proper deeply detailed R/LaTeX report with svg charts... build that workflow script for
|
||||
automated production of the PDF artifact while execution of the experiment is underway." Closes
|
||||
the one item §XXXV.10 itself flagged as still missing.
|
||||
|
||||
**Built:**
|
||||
- `scripts/extract_doe_region.py` -- standalone reader, independent reimplementation of
|
||||
`doe_region.c`'s own ring-walk logic in Python (same devblock_from_top fence math, same
|
||||
head/tail/record_count ring-order walk) reading `disk/artemis.img` directly. Two modes:
|
||||
`--img PATH` (default, no privilege needed -- opens the image as a plain file) and `--loop`
|
||||
(attaches via `losetup --read-only`, needs sudo, detaches on exit -- the literal loop-device
|
||||
step requested; noted in the script's own docstring that the ring is raw binary at a fixed
|
||||
offset, not a filesystem, so there is nothing to `mount` in the POSIX sense once attached, it
|
||||
reads exactly like a regular file). Read-only in both modes; a torn read mid-write from a
|
||||
live campaign is rejected the same way `doe_region_ctrl_read()` itself rejects a bad
|
||||
magic/version -- "not yet initialized," not a crash.
|
||||
- `experiments/bare_metal/analysis/analyse_multiuser_doe.R` -- new analysis script, house style
|
||||
(`ggplot2`/`svglite`, same `theme_light_report()` shape as `analyse_multirep.R`), but
|
||||
deliberately NOT that script's fixed-manifest pattern: auto-discovers whatever CSV or CSV
|
||||
directory it's given and **degrades gracefully to a placeholder table/chart when 0 trials
|
||||
exist yet**, rather than erroring -- the explicit "while execution of the experiment is
|
||||
underway" requirement. Also writes an auto-generated LaTeX fragment
|
||||
(`report/sections/multiuser_summary_autogen.tex`) with the actual current trial count, fail
|
||||
rate, and per-cell breakdown, computed fresh every run -- the report's own numbers stay
|
||||
truthful across re-runs instead of going stale.
|
||||
- `experiments/bare_metal/analysis/report/multiuser_doe_report.tex` -- new self-contained
|
||||
report (same preamble/colour-palette convention as `bare_metal_doe_report.tex`), every
|
||||
data-dependent figure wrapped in `\IfFileExists` so the report builds correctly whether 0
|
||||
trials or many have been extracted.
|
||||
- `scripts/build_multiuser_doe_report.sh` -- orchestrates all of the above: extract -> R
|
||||
analysis -> `svg_to_pdf.py` (existing, reused unmodified) -> `pdflatex` (twice, for
|
||||
cross-references) -> `multiuser_doe_report.pdf`. `--csv PATH` skips disk extraction entirely
|
||||
(smoke-testing against an arbitrary CSV); `--loop` passes through to the extractor.
|
||||
|
||||
**Real bug found and fixed while verifying, not left in:** the R script's original
|
||||
`SCRIPT_DIR` detection (`sys.frames()[[1]]$ofile`, the same pattern `analyse_multirep.R` already
|
||||
uses) silently fell back to `getwd()` when invoked from the orchestration script's own working
|
||||
directory -- wrote `charts/`, `tables/`, and `report/sections/` at the repo root instead of
|
||||
under `experiments/bare_metal/analysis/`, confirmed live (all three stray directories actually
|
||||
appeared there on the first orchestrated run). Fixed with the robust `commandArgs()`
|
||||
`"--file="` idiom, re-verified correct placement, stray directories removed. Not touched in
|
||||
`analyse_multirep.R` itself -- out of scope, not verified broken there, flag only.
|
||||
|
||||
**Verified live, full pipeline, both branches:**
|
||||
1. **No-data branch:** ran the full orchestration script against the current (post-§XXXV.10
|
||||
test-write-reverted) `disk/artemis.img` -- 0 trials, placeholder chart/table/summary
|
||||
generated, real 2-page PDF produced, correctly states "No trial data has been persisted to
|
||||
the DoE region yet."
|
||||
2. **Populated branch:** booted amd64, wrote 3 synthetic trial records via `DOE-PERSIST-TRIAL`
|
||||
at the console (one deliberately with `fail=1`), clean `BYE`. `extract_doe_region.py`
|
||||
round-tripped all 3 records correctly, in ring order, byte-exact. Full orchestration run
|
||||
(`--csv` override, to isolate the R/LaTeX stage from the disk-extraction stage) produced a
|
||||
real 4-page PDF with correct auto-generated summary text ("3 completed trial(s)... Overall
|
||||
fail rate... 33.3%...") and real embedded SVG-derived charts.
|
||||
3. Reverted the synthetic test write from `disk/artemis.img` afterward (same convention as
|
||||
§XXXV.10) and regenerated the final on-disk report/CSV against the real, empty state before
|
||||
committing -- what's in the repo now honestly reflects "no campaign has run yet," not
|
||||
leftover demo numbers.
|
||||
4. Reverted an unrelated side effect caught before committing: `svg_to_pdf.py` (existing,
|
||||
unmodified) re-converts every SVG in the shared `charts/` directory on each run, which
|
||||
touched ~85 pre-existing, unrelated committed figure PDFs with meaningless
|
||||
binary/metadata-only diffs (almost certainly `rsvg-convert` version drift, not content
|
||||
changes) -- reverted before staging, confirmed only the new `multiuser_*` files remain
|
||||
changed.
|
||||
|
||||
**Not done, explicitly out of scope for this pass:** the actual multi-day campaign launch is
|
||||
still a separate, deliberate decision -- this closes only the "how do we get a report out of
|
||||
it" side. `--loop` mode was implemented per the literal request but not live-tested (the
|
||||
default `--img` direct-file mode was what got exercised above; both modes share the same
|
||||
`read_ring()` code path once handed a device/file path, so this is a thin, low-risk gap, not
|
||||
an unverified core mechanism).
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Capsule Block Manifest — Auto-generated
|
||||
<!-- Generated by mkcapsule --manifest 2026-09-17T18:37:43Z -->
|
||||
<!-- Generated by mkcapsule --manifest 2026-09-17T20:52:35Z -->
|
||||
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
|
||||
<!-- Hand-written justifications and immutability notes live -->
|
||||
<!-- in MANIFEST.md alongside this auto-generated index. -->
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
#!/usr/bin/env Rscript
|
||||
# analyse_multiuser_doe.R
|
||||
# StarForth LithosAnanke — Multiuser/Multitasking DoE Analysis
|
||||
#
|
||||
# Analyses trial-summary records from the multiuser/multitasking campaign
|
||||
# (multiuser-doe.4th / per-isa-doe.4th, FABRIC-3.md §XXXIII/§XXXV) -- one
|
||||
# row per completed trial: concurrency level (nw: 2/4/8) x mode (0=uniform,
|
||||
# 1=heterogeneous), fail count, and trial-end fleet physics (fleet_k_q48,
|
||||
# fleet_conserved, switch_count_cumulative).
|
||||
#
|
||||
# Unlike analyse_multirep.R/analyse_bare_metal.R (which analyse dense
|
||||
# per-tick HB-ON streams from a fixed manifest of completed runs), this
|
||||
# script auto-discovers whatever CSV(s) are present and is designed to be
|
||||
# re-run repeatedly WHILE a campaign is still in progress -- it degrades
|
||||
# gracefully to a "no data yet" placeholder rather than erroring when a
|
||||
# campaign has produced 0 trials so far, and re-summarises from scratch
|
||||
# each run against however many trials currently exist. Input format is
|
||||
# extract_doe_region.py's own CSV schema, or (for smoke-testing this
|
||||
# pipeline before real campaign data exists) any CSV in this shape.
|
||||
|
||||
suppressPackageStartupMessages({
|
||||
library(ggplot2)
|
||||
library(svglite)
|
||||
library(dplyr)
|
||||
library(scales)
|
||||
})
|
||||
|
||||
# sys.frames()[[1]]$ofile is unreliable depending on how Rscript was
|
||||
# invoked (silently falls back to getwd(), which is wrong whenever this
|
||||
# is called from a different working directory -- e.g. an orchestration
|
||||
# script that cd's to the repo root first). commandArgs()'s "--file="
|
||||
# entry is the robust Rscript idiom and is what's used here.
|
||||
get_script_dir <- function() {
|
||||
raw_args <- commandArgs(trailingOnly = FALSE)
|
||||
file_arg <- grep("^--file=", raw_args, value = TRUE)
|
||||
if (length(file_arg) == 1) {
|
||||
return(dirname(normalizePath(sub("^--file=", "", file_arg))))
|
||||
}
|
||||
tryCatch(dirname(normalizePath(sys.frames()[[1]]$ofile)), error = function(e) getwd())
|
||||
}
|
||||
SCRIPT_DIR <- get_script_dir()
|
||||
BASE_DIR <- normalizePath(file.path(SCRIPT_DIR, ".."))
|
||||
OUT_CHARTS <- file.path(SCRIPT_DIR, "charts")
|
||||
OUT_TABLES <- file.path(SCRIPT_DIR, "tables")
|
||||
OUT_SECTION <- file.path(SCRIPT_DIR, "report", "sections", "multiuser_summary_autogen.tex")
|
||||
dir.create(OUT_CHARTS, showWarnings = FALSE, recursive = TRUE)
|
||||
dir.create(OUT_TABLES, showWarnings = FALSE, recursive = TRUE)
|
||||
dir.create(dirname(OUT_SECTION), showWarnings = FALSE, recursive = TRUE)
|
||||
|
||||
# LaTeX-escape a plain string for safe \newcommand/text use.
|
||||
tex_escape <- function(s) {
|
||||
s <- gsub("\\\\", "\\\\textbackslash{}", s)
|
||||
s <- gsub("([%&_#{}$])", "\\\\\\1", s)
|
||||
s
|
||||
}
|
||||
|
||||
write_tex_fragment <- function(lines) {
|
||||
writeLines(lines, OUT_SECTION)
|
||||
cat(sprintf(" Saved: %s\n", OUT_SECTION))
|
||||
}
|
||||
|
||||
# ── input: first CLI arg is a CSV path, or a directory to scan; default
|
||||
# is the extractor's own conventional output location ──────────────────
|
||||
args <- commandArgs(trailingOnly = TRUE)
|
||||
input_arg <- if (length(args) >= 1) args[1] else file.path(BASE_DIR, "runs", "multiuser-doe-trials.csv")
|
||||
|
||||
cat("══════════════════════════════════════════════════════════════════\n")
|
||||
cat(" StarForth LithosAnanke — Multiuser/Multitasking DoE Analysis\n")
|
||||
cat("══════════════════════════════════════════════════════════════════\n\n")
|
||||
|
||||
load_trials <- function(path) {
|
||||
if (dir.exists(path)) {
|
||||
files <- list.files(path, pattern = "\\.csv$", full.names = TRUE)
|
||||
if (length(files) == 0) return(NULL)
|
||||
dfs <- lapply(files, function(f) tryCatch(read.csv(f, stringsAsFactors = FALSE), error = function(e) NULL))
|
||||
dfs <- dfs[!sapply(dfs, is.null)]
|
||||
dfs <- dfs[sapply(dfs, function(d) all(c("cfg","nw","mode","fail_count") %in% names(d)))]
|
||||
if (length(dfs) == 0) return(NULL)
|
||||
bind_rows(dfs)
|
||||
} else if (file.exists(path)) {
|
||||
df <- read.csv(path, stringsAsFactors = FALSE)
|
||||
if (!all(c("cfg","nw","mode","fail_count") %in% names(df))) return(NULL)
|
||||
df
|
||||
} else {
|
||||
NULL
|
||||
}
|
||||
}
|
||||
|
||||
df <- load_trials(input_arg)
|
||||
n_trials <- if (is.null(df)) 0 else nrow(df)
|
||||
cat(sprintf(" Input: %s\n Trials found: %d\n\n", input_arg, n_trials))
|
||||
|
||||
Q48 <- 65536.0
|
||||
|
||||
write_placeholder <- function(reason) {
|
||||
cat(sprintf(" %s -- writing placeholder tables/charts so the report build\n", reason))
|
||||
cat(" still completes (this pipeline is meant to be re-run while the\n")
|
||||
cat(" campaign is still in progress).\n\n")
|
||||
|
||||
placeholder_tbl <- data.frame(status = reason, trials_so_far = n_trials)
|
||||
write.csv(placeholder_tbl, file.path(OUT_TABLES, "multiuser_status.csv"), row.names = FALSE)
|
||||
|
||||
p <- ggplot() +
|
||||
annotate("text", x = 0, y = 0, label = paste0(reason, "\n(", n_trials, " trials so far)"),
|
||||
size = 6, colour = "grey40") +
|
||||
theme_void()
|
||||
svglite(file.path(OUT_CHARTS, "multiuser_placeholder.svg"), width = 8, height = 4)
|
||||
print(p)
|
||||
dev.off()
|
||||
cat(" Saved: multiuser_placeholder.svg\n\n")
|
||||
}
|
||||
|
||||
if (n_trials == 0) {
|
||||
write_placeholder("No trials persisted yet")
|
||||
write_tex_fragment(c(
|
||||
"% Auto-generated by analyse_multiuser_doe.R -- do not edit by hand.",
|
||||
"No trial data has been persisted to the DoE region yet. This section",
|
||||
"will populate automatically the next time this report is built after",
|
||||
"the campaign has produced at least one completed trial."
|
||||
))
|
||||
quit(save = "no", status = 0)
|
||||
}
|
||||
|
||||
# ── normalise / label factors ────────────────────────────────────────────
|
||||
df$nw <- factor(df$nw, levels = c(2, 4, 8))
|
||||
df$mode <- factor(df$mode, levels = c(0, 1), labels = c("uniform", "heterogeneous"))
|
||||
df$cfg <- factor(df$cfg)
|
||||
if (!"isa" %in% names(df) || all(df$isa == "" | is.na(df$isa))) df$isa <- "amd64"
|
||||
df$isa <- factor(df$isa, levels = intersect(c("amd64","aarch64","riscv64"), unique(df$isa)))
|
||||
df$fleet_k <- df$fleet_k_q48 / Q48
|
||||
|
||||
# ── 1. Per-cfg summary ───────────────────────────────────────────────────
|
||||
cat("Computing per-cfg summary (concurrency x mode)...\n")
|
||||
cfg_tbl <- df %>%
|
||||
group_by(nw, mode) %>%
|
||||
summarise(
|
||||
n_trials = n(),
|
||||
n_fail = sum(fail_count > 0),
|
||||
fail_rate = mean(fail_count > 0),
|
||||
mean_fail_count = mean(fail_count),
|
||||
mean_fleet_k = mean(fleet_k, na.rm = TRUE),
|
||||
pct_conserved = mean(fleet_conserved == 1, na.rm = TRUE),
|
||||
mean_switch_cnt = mean(switch_count_cumulative, na.rm = TRUE),
|
||||
.groups = "drop"
|
||||
)
|
||||
write.csv(cfg_tbl, file.path(OUT_TABLES, "multiuser_cell_summary.csv"), row.names = FALSE)
|
||||
cat(" Saved: multiuser_cell_summary.csv\n")
|
||||
print(as.data.frame(cfg_tbl))
|
||||
cat("\n")
|
||||
|
||||
# ── 2. Per-ISA summary (only meaningful once >1 ISA present) ────────────
|
||||
if (nlevels(df$isa) > 1) {
|
||||
cat("Computing per-ISA summary...\n")
|
||||
isa_tbl <- df %>%
|
||||
group_by(isa) %>%
|
||||
summarise(
|
||||
n_trials = n(),
|
||||
fail_rate = mean(fail_count > 0),
|
||||
mean_fleet_k = mean(fleet_k, na.rm = TRUE),
|
||||
.groups = "drop"
|
||||
)
|
||||
write.csv(isa_tbl, file.path(OUT_TABLES, "multiuser_isa_summary.csv"), row.names = FALSE)
|
||||
cat(" Saved: multiuser_isa_summary.csv\n\n")
|
||||
}
|
||||
|
||||
# ── colour theme (house style, matches analyse_multirep.R) ──────────────
|
||||
theme_light_report <- function() {
|
||||
theme_minimal(base_size = 11) %+replace%
|
||||
theme(
|
||||
panel.grid.minor = element_blank(),
|
||||
panel.grid.major = element_line(colour = "grey90"),
|
||||
strip.text = element_text(face = "bold"),
|
||||
plot.title = element_text(face = "bold", size = 12),
|
||||
plot.subtitle = element_text(colour = "grey40", size = 9),
|
||||
legend.position = "bottom"
|
||||
)
|
||||
}
|
||||
|
||||
save_svg <- function(plot, name, w = 10, h = 6) {
|
||||
path <- file.path(OUT_CHARTS, paste0(name, ".svg"))
|
||||
svglite(path, width = w, height = h)
|
||||
print(plot)
|
||||
dev.off()
|
||||
cat(sprintf(" Saved: %s.svg\n", name))
|
||||
invisible(path)
|
||||
}
|
||||
|
||||
cat("Generating charts...\n\n")
|
||||
|
||||
# ── CHART MU-1: fail rate by concurrency x mode ──────────────────────────
|
||||
cat("[MU-1] Fail rate by concurrency x mode...\n")
|
||||
p_fail <- ggplot(cfg_tbl, aes(x = nw, y = fail_rate, fill = mode)) +
|
||||
geom_col(position = "dodge", alpha = 0.85) +
|
||||
geom_text(aes(label = sprintf("%d/%d", n_fail, n_trials)),
|
||||
position = position_dodge(width = 0.9), vjust = -0.4, size = 3) +
|
||||
scale_y_continuous(labels = percent_format(accuracy = 1), limits = c(0, NA)) +
|
||||
scale_fill_manual(values = c(uniform = "#4A90D9", heterogeneous = "#E07B39")) +
|
||||
labs(
|
||||
title = "Trial Fail Rate by Concurrency Level x Workload Mode",
|
||||
subtitle = paste0(n_trials, " trials"),
|
||||
x = "Concurrent workers",
|
||||
y = "Fraction of trials with fail_count > 0"
|
||||
) +
|
||||
theme_light_report()
|
||||
save_svg(p_fail, "multiuser_fail_rate", w = 8, h = 6)
|
||||
|
||||
# ── CHART MU-2: switch_count_cumulative distribution by concurrency ─────
|
||||
cat("[MU-2] Switch count by concurrency...\n")
|
||||
p_switch <- ggplot(df, aes(x = nw, y = switch_count_cumulative, fill = nw)) +
|
||||
geom_boxplot(alpha = 0.8, outlier.alpha = 0.4) +
|
||||
scale_fill_manual(values = c("2" = "#50C878", "4" = "#4A90D9", "8" = "#E07B39"), guide = "none") +
|
||||
labs(
|
||||
title = "Cumulative Preemptive Switch Count by Concurrency Level",
|
||||
subtitle = "Stage-3 switch signal count at trial end, by number of concurrent workers",
|
||||
x = "Concurrent workers",
|
||||
y = "switch_count_cumulative"
|
||||
) +
|
||||
theme_light_report()
|
||||
save_svg(p_switch, "multiuser_switch_count", w = 8, h = 6)
|
||||
|
||||
# ── CHART MU-3: fleet_k_q48 (conservation) across trials in run order ───
|
||||
cat("[MU-3] Fleet K conservation across trial sequence...\n")
|
||||
df_seq <- df %>% arrange(run_id)
|
||||
p_fleetk <- ggplot(df_seq, aes(x = run_id, y = fleet_k, colour = mode)) +
|
||||
geom_point(alpha = 0.6, size = 1.5) +
|
||||
geom_hline(yintercept = 1.0, colour = "grey50", linetype = "dashed") +
|
||||
scale_colour_manual(values = c(uniform = "#4A90D9", heterogeneous = "#E07B39")) +
|
||||
labs(
|
||||
title = "Fleet Heat Conservation (K) Across the Trial Sequence",
|
||||
subtitle = "Dashed line = Q48.16 conserved value (1.0); trial-end reads, shuffled run order",
|
||||
x = "Run id (shuffled trial index)",
|
||||
y = "fleet_k_q48 / 65536"
|
||||
) +
|
||||
theme_light_report()
|
||||
save_svg(p_fleetk, "multiuser_fleet_k_sequence", w = 10, h = 6)
|
||||
|
||||
# ── CHART MU-4: per-ISA panel (only when >1 ISA present) ─────────────────
|
||||
if (nlevels(df$isa) > 1) {
|
||||
cat("[MU-4] Per-ISA fail rate panel...\n")
|
||||
p_isa <- ggplot(df %>% group_by(isa, nw, mode) %>%
|
||||
summarise(fail_rate = mean(fail_count > 0), .groups = "drop"),
|
||||
aes(x = nw, y = fail_rate, fill = mode)) +
|
||||
geom_col(position = "dodge", alpha = 0.85) +
|
||||
scale_y_continuous(labels = percent_format(accuracy = 1), limits = c(0, NA)) +
|
||||
scale_fill_manual(values = c(uniform = "#4A90D9", heterogeneous = "#E07B39")) +
|
||||
facet_wrap(~ isa, ncol = 3) +
|
||||
labs(
|
||||
title = "Fail Rate by Concurrency x Mode, per ISA",
|
||||
x = "Concurrent workers",
|
||||
y = "Fraction of trials with fail_count > 0"
|
||||
) +
|
||||
theme_light_report()
|
||||
save_svg(p_isa, "multiuser_isa_panel", w = 12, h = 5)
|
||||
}
|
||||
|
||||
# ── summary ───────────────────────────────────────────────────────────────
|
||||
cat(sprintf("\n══════════════════════════════════════════════════════════════════\n"))
|
||||
cat(sprintf(" Multiuser DoE analysis complete. %d trials, %d cfg cells.\n",
|
||||
n_trials, nrow(cfg_tbl)))
|
||||
cat(sprintf("══════════════════════════════════════════════════════════════════\n\n"))
|
||||
|
||||
overall_fail_rate <- mean(df$fail_count > 0)
|
||||
overall_conserved <- mean(df$fleet_conserved == 1, na.rm = TRUE)
|
||||
cat(sprintf(" Overall fail rate: %.2f%%\n", overall_fail_rate * 100))
|
||||
cat(sprintf(" Fraction of trials conserved (fleet_conserved==1): %.2f%%\n", overall_conserved * 100))
|
||||
|
||||
# ── auto-generated LaTeX summary fragment (re-written every run) ────────
|
||||
isa_list <- paste(levels(df$isa), collapse = ", ")
|
||||
cell_lines <- apply(cfg_tbl, 1, function(r) {
|
||||
sprintf(" \\item %s workers, %s: %s trials, fail rate %.1f\\%%, mean fleet K = %.4f",
|
||||
r["nw"], tex_escape(as.character(r["mode"])), r["n_trials"],
|
||||
as.numeric(r["fail_rate"]) * 100, as.numeric(r["mean_fleet_k"]))
|
||||
})
|
||||
write_tex_fragment(c(
|
||||
"% Auto-generated by analyse_multiuser_doe.R -- do not edit by hand.",
|
||||
sprintf("This report reflects \\textbf{%d} completed trial(s) persisted via", n_trials),
|
||||
"\\texttt{DOE-PERSIST-TRIAL} as of the last extraction, spanning ISA(s)",
|
||||
sprintf("\\texttt{%s}. Overall fail rate across all trials so far is", tex_escape(isa_list)),
|
||||
sprintf("\\textbf{%.1f\\%%}; \\textbf{%.1f\\%%} of trials read", overall_fail_rate * 100, overall_conserved * 100),
|
||||
"\\texttt{fleet\\_conserved==1} at trial end.",
|
||||
"",
|
||||
"\\begin{itemize}",
|
||||
cell_lines,
|
||||
"\\end{itemize}"
|
||||
))
|
||||
@@ -0,0 +1,34 @@
|
||||
<?xml version='1.0' encoding='UTF-8' ?>
|
||||
<svg xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' width='576.00pt' height='288.00pt' viewBox='0 0 576.00 288.00'>
|
||||
<g class='svglite'>
|
||||
<defs>
|
||||
<style type='text/css'><![CDATA[
|
||||
.svglite line, .svglite polyline, .svglite polygon, .svglite path, .svglite rect, .svglite circle {
|
||||
fill: none;
|
||||
stroke: #000000;
|
||||
stroke-linecap: round;
|
||||
stroke-linejoin: round;
|
||||
stroke-miterlimit: 10.00;
|
||||
}
|
||||
.svglite text {
|
||||
white-space: pre;
|
||||
}
|
||||
.svglite g.glyphgroup path {
|
||||
fill: inherit;
|
||||
stroke: none;
|
||||
}
|
||||
]]></style>
|
||||
</defs>
|
||||
<rect width='100%' height='100%' style='stroke: none; fill: #FFFFFF;'/>
|
||||
<defs>
|
||||
<clipPath id='cpMC4wMHw1NzYuMDB8MC4wMHwyODguMDA='>
|
||||
<rect x='0.00' y='0.00' width='576.00' height='288.00' />
|
||||
</clipPath>
|
||||
</defs>
|
||||
<g clip-path='url(#cpMC4wMHw1NzYuMDB8MC4wMHwyODguMDA=)'>
|
||||
<rect x='0.00' y='0.00' width='576.00' height='288.00' style='stroke-width: 0.00; stroke: none;' />
|
||||
<text x='288.00' y='137.58' text-anchor='middle' style='font-size: 17.07px;fill: #666666; font-family: "Liberation Sans";' textLength='164.02px' lengthAdjust='spacingAndGlyphs'>No trials persisted yet</text>
|
||||
<text x='288.00' y='162.16' text-anchor='middle' style='font-size: 17.07px;fill: #666666; font-family: "Liberation Sans";' textLength='109.02px' lengthAdjust='spacingAndGlyphs'>(0 trials so far)</text>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.4 KiB |
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,148 @@
|
||||
\documentclass[10pt,a4paper]{article}
|
||||
|
||||
%% ── Packages ──────────────────────────────────────────────────────────────────
|
||||
\usepackage[T1]{fontenc}
|
||||
\usepackage[utf8]{inputenc}
|
||||
\usepackage[margin=2.5cm]{geometry}
|
||||
\usepackage{graphicx}
|
||||
\usepackage{subcaption}
|
||||
\usepackage{booktabs}
|
||||
\usepackage{amsmath}
|
||||
\usepackage{amssymb}
|
||||
\usepackage{xcolor}
|
||||
\usepackage{microtype}
|
||||
\usepackage{parskip}
|
||||
\usepackage{enumitem}
|
||||
\usepackage[hidelinks,colorlinks=true,linkcolor=black,citecolor=black,urlcolor=blue]{hyperref}
|
||||
\usepackage{lmodern}
|
||||
|
||||
%% ── Colour palette (matches bare_metal_doe_report.tex) ─────────────────────────
|
||||
\definecolor{sfblue}{RGB}{31,119,180}
|
||||
\definecolor{sforange}{RGB}{255,127,14}
|
||||
\definecolor{sfgreen}{RGB}{44,160,44}
|
||||
\definecolor{sfgray}{RGB}{80,80,80}
|
||||
|
||||
%% ── Figure path ───────────────────────────────────────────────────────────────
|
||||
\graphicspath{{figures/}}
|
||||
|
||||
%% ── Title metadata ────────────────────────────────────────────────────────────
|
||||
\title{%
|
||||
\textbf{LithosAnanke Multiuser/Multitasking Design of Experiments}\\[0.4em]
|
||||
\large Concurrent Worker VM Load Across Three Instruction-Set Architectures
|
||||
}
|
||||
\author{%
|
||||
R.\,A. James (Captain Bob)\\[0.2em]
|
||||
\small StarForth Project \quad\texttt{robert.allan.james@gmail.com}
|
||||
}
|
||||
\date{%
|
||||
Generated automatically by \texttt{scripts/build\_multiuser\_doe\_report.sh}\\[0.2em]
|
||||
\textnormal{(Patent Pending --- Provisional Filed December 2025)}
|
||||
}
|
||||
|
||||
%% ═════════════════════════════════════════════════════════════════════════════
|
||||
\begin{document}
|
||||
|
||||
\maketitle
|
||||
\thispagestyle{empty}
|
||||
|
||||
%% ── Abstract ──────────────────────────────────────────────────────────────────
|
||||
\begin{abstract}
|
||||
\noindent
|
||||
This report analyses the multiuser/multitasking Design of Experiments
|
||||
(FABRIC-3.md \S{}XXXIII/\S{}XXXV): a 6-cell factorial over concurrency level
|
||||
(2, 4, or 8 simultaneously live worker VMs, each birthed and touched twice
|
||||
by the Mama VM Hera) crossed with workload mode (uniform -- every worker
|
||||
runs the same workload; heterogeneous -- worker $n$ runs workload
|
||||
$n \bmod 10$), run per instruction-set architecture with a fixed seed.
|
||||
Trial outcomes -- concurrency, mode, fail count, and trial-end fleet
|
||||
physics (cumulative preemptive switch count, fleet heat conservation) --
|
||||
persist to a dedicated block-backed region on \texttt{disk/artemis.img}
|
||||
(\texttt{DOE-PERSIST-TRIAL}, \S{}XXXV.10) rather than depending on a live
|
||||
serial-log tail, so this report can be rebuilt at any point during a
|
||||
multi-day campaign, including while it is still running, from whatever
|
||||
trials have completed so far.
|
||||
\end{abstract}
|
||||
|
||||
\section{Current campaign status}
|
||||
|
||||
\input{sections/multiuser_summary_autogen.tex}
|
||||
|
||||
\IfFileExists{figures/multiuser_placeholder.pdf}{%
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=0.6\textwidth]{multiuser_placeholder.pdf}
|
||||
\end{figure}
|
||||
}{}
|
||||
|
||||
\IfFileExists{figures/multiuser_fail_rate.pdf}{%
|
||||
\section{Fail rate by concurrency and workload mode}
|
||||
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=0.75\textwidth]{multiuser_fail_rate.pdf}
|
||||
\caption{Fraction of trials with \texttt{fail\_count > 0}, by concurrency
|
||||
level (2/4/8 workers) and workload mode. Labels give (failed/total) trials
|
||||
per cell.}
|
||||
\end{figure}
|
||||
}{}
|
||||
|
||||
\IfFileExists{figures/multiuser_switch_count.pdf}{%
|
||||
\clearpage
|
||||
\section{Preemptive switch load}
|
||||
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=0.75\textwidth]{multiuser_switch_count.pdf}
|
||||
\caption{Distribution of cumulative Stage-3 preemptive switch count at
|
||||
trial end, by concurrency level. Higher concurrency is expected to drive
|
||||
more switch events per trial.}
|
||||
\end{figure}
|
||||
}{}
|
||||
|
||||
\IfFileExists{figures/multiuser_fleet_k_sequence.pdf}{%
|
||||
\section{Fleet heat conservation across the trial sequence}
|
||||
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{multiuser_fleet_k_sequence.pdf}
|
||||
\caption{Trial-end fleet heat (\texttt{fleet\_k\_q48}, decoded from
|
||||
Q48.16) across the shuffled trial run order. The dashed line marks the
|
||||
conserved value (1.0); persistent drift away from it across the
|
||||
sequence would indicate a real conservation violation rather than
|
||||
per-trial noise.}
|
||||
\end{figure}
|
||||
}{}
|
||||
|
||||
\IfFileExists{figures/multiuser_isa_panel.pdf}{%
|
||||
\clearpage
|
||||
\section{Per-ISA comparison}
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{multiuser_isa_panel.pdf}
|
||||
\caption{Fail rate by concurrency and mode, faceted by ISA -- populated
|
||||
once more than one architecture's trials have been persisted.}
|
||||
\end{figure}
|
||||
}{}
|
||||
|
||||
\section{Methodology notes}
|
||||
|
||||
\begin{itemize}
|
||||
\item \textbf{Persistence, not a serial-log dependency.} Earlier attempts
|
||||
at this campaign relied entirely on a live QEMU serial-log tail and
|
||||
were lost when the process was interrupted (FABRIC-3.md \S{}XXXV.10).
|
||||
Every number in this report comes from \texttt{disk/artemis.img}'s
|
||||
own on-disk DoE region, read directly by
|
||||
\texttt{scripts/extract\_doe\_region.py} with no live QEMU process
|
||||
required.
|
||||
\item \textbf{Rebuildable mid-campaign.} This report's build script
|
||||
(\texttt{scripts/build\_multiuser\_doe\_report.sh}) can be re-run at
|
||||
any point -- including while a multi-day campaign is still executing
|
||||
-- and will reflect however many trials have completed at that moment.
|
||||
\item \textbf{Trial-summary, not per-tick.} Unlike the per-tick
|
||||
\texttt{HB-ON} serial stream (which remains the authoritative live
|
||||
instrumentation during a run), the persisted region stores one summary
|
||||
record per completed trial (\S{}XXXV.4) -- sized to comfortably fit
|
||||
the device budget for the full campaign, not a raw tick mirror.
|
||||
\end{itemize}
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,4 @@
|
||||
% Auto-generated by analyse_multiuser_doe.R -- do not edit by hand.
|
||||
No trial data has been persisted to the DoE region yet. This section
|
||||
will populate automatically the next time this report is built after
|
||||
the campaign has produced at least one completed trial.
|
||||
@@ -0,0 +1,2 @@
|
||||
"status","trials_so_far"
|
||||
"No trials persisted yet",0
|
||||
|
@@ -0,0 +1 @@
|
||||
isa,run_id,cfg,rep,nw,mode,fail_count,fleet_k_q48,fleet_conserved,switch_count_cumulative
|
||||
|
File diff suppressed because it is too large
Load Diff
Executable
+96
@@ -0,0 +1,96 @@
|
||||
#!/usr/bin/env bash
|
||||
# build_multiuser_doe_report.sh — full pipeline: disk -> CSV -> R/SVG -> PDF
|
||||
#
|
||||
# Chains: extract_doe_region.py (spool DOE-PERSIST-TRIAL records off
|
||||
# disk/artemis.img) -> analyse_multiuser_doe.R (tables + SVG charts +
|
||||
# auto-generated summary fragment) -> svg_to_pdf.py (SVG -> PDF for
|
||||
# \includegraphics) -> pdflatex (twice, for cross-references) ->
|
||||
# multiuser_doe_report.pdf.
|
||||
#
|
||||
# Safe to re-run at any point, including while a multi-day campaign is
|
||||
# still executing on a separate QEMU instance: every step is read-only
|
||||
# with respect to disk/artemis.img, and analyse_multiuser_doe.R degrades
|
||||
# to a "no data yet" placeholder rather than failing when the DoE region
|
||||
# has 0 trials so far.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/build_multiuser_doe_report.sh [--loop] [--csv PATH]
|
||||
#
|
||||
# --loop Attach disk/artemis.img via losetup instead of reading
|
||||
# the file directly (needs sudo). See extract_doe_region.py.
|
||||
# --csv PATH Skip disk extraction entirely and analyse an existing
|
||||
# CSV instead (e.g. to smoke-test this pipeline against
|
||||
# an old per-tick run CSV, or a CSV saved from a previous
|
||||
# extraction).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
ANALYSIS_DIR="experiments/bare_metal/analysis"
|
||||
RUNS_DIR="experiments/bare_metal/runs"
|
||||
REPORT_DIR="$ANALYSIS_DIR/report"
|
||||
CSV_OUT="$RUNS_DIR/multiuser-doe-trials.csv"
|
||||
|
||||
USE_LOOP=0
|
||||
CSV_OVERRIDE=""
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--loop) USE_LOOP=1; shift ;;
|
||||
--csv) CSV_OVERRIDE="$2"; shift 2 ;;
|
||||
*) echo "Unknown argument: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo "=== [1/4] Extracting trial records ==="
|
||||
if [ -n "$CSV_OVERRIDE" ]; then
|
||||
echo " Using existing CSV: $CSV_OVERRIDE (skipping disk extraction)"
|
||||
CSV_OUT="$CSV_OVERRIDE"
|
||||
elif [ -f disk/artemis.img ]; then
|
||||
mkdir -p "$RUNS_DIR"
|
||||
if [ "$USE_LOOP" -eq 1 ]; then
|
||||
python3 scripts/extract_doe_region.py --img disk/artemis.img --loop -o "$CSV_OUT"
|
||||
else
|
||||
python3 scripts/extract_doe_region.py --img disk/artemis.img -o "$CSV_OUT"
|
||||
fi
|
||||
else
|
||||
echo " disk/artemis.img not found -- writing an empty CSV so the report" >&2
|
||||
echo " still builds with a 'no data yet' placeholder." >&2
|
||||
mkdir -p "$RUNS_DIR"
|
||||
echo "isa,run_id,cfg,rep,nw,mode,fail_count,fleet_k_q48,fleet_conserved,switch_count_cumulative" > "$CSV_OUT"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=== [2/4] Running R analysis (tables + charts + summary fragment) ==="
|
||||
Rscript "$ANALYSIS_DIR/analyse_multiuser_doe.R" "$CSV_OUT"
|
||||
|
||||
echo ""
|
||||
echo "=== [3/4] Converting SVG charts to PDF ==="
|
||||
python3 "$ANALYSIS_DIR/svg_to_pdf.py"
|
||||
|
||||
echo ""
|
||||
echo "=== [4/4] Building PDF report ==="
|
||||
mkdir -p "$REPORT_DIR/sections"
|
||||
if [ ! -f "$REPORT_DIR/sections/multiuser_summary_autogen.tex" ]; then
|
||||
echo "No trial data has been persisted to the DoE region yet." \
|
||||
> "$REPORT_DIR/sections/multiuser_summary_autogen.tex"
|
||||
fi
|
||||
|
||||
cd "$REPORT_DIR"
|
||||
pdflatex -interaction=nonstopmode -halt-on-error multiuser_doe_report.tex >/tmp/multiuser_doe_report_latex1.log 2>&1 || {
|
||||
echo "pdflatex (pass 1) failed -- see /tmp/multiuser_doe_report_latex1.log" >&2
|
||||
tail -40 /tmp/multiuser_doe_report_latex1.log >&2
|
||||
exit 1
|
||||
}
|
||||
pdflatex -interaction=nonstopmode -halt-on-error multiuser_doe_report.tex >/tmp/multiuser_doe_report_latex2.log 2>&1 || {
|
||||
echo "pdflatex (pass 2) failed -- see /tmp/multiuser_doe_report_latex2.log" >&2
|
||||
tail -40 /tmp/multiuser_doe_report_latex2.log >&2
|
||||
exit 1
|
||||
}
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
echo ""
|
||||
echo "=== Done ==="
|
||||
echo " PDF: $REPORT_DIR/multiuser_doe_report.pdf"
|
||||
Executable
+194
@@ -0,0 +1,194 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
extract_doe_region.py -- spool the DOE-PERSIST-TRIAL binary ring off
|
||||
disk/artemis.img into a CSV, for the multiuser/multitasking DoE campaign
|
||||
report (FABRIC-3.md §XXXV.10).
|
||||
|
||||
Reads the SAME devblock_from_top-addressed ring doe_region.c writes
|
||||
(include/starkernel/doe_region.h has the authoritative layout: control
|
||||
header at devblock_from_top=98, slot devblocks growable from 99, 64-byte
|
||||
doe_trial_slot_t records, 64 slots/devblock). Two access modes:
|
||||
|
||||
--img PATH Read the image file directly (default). No privilege
|
||||
needed, works on a plain regular file. This is what
|
||||
"reading the drive" reduces to when the image is
|
||||
just a file, which is the normal QEMU-dev case.
|
||||
|
||||
--loop PATH Attach PATH via losetup first (needs sudo), read the
|
||||
resulting /dev/loopN node, then detach it again on
|
||||
exit -- the literal "mount the virtual drive on a
|
||||
loop device" step for when this runs against a real
|
||||
block device or a copy pulled off real hardware. The
|
||||
ring is raw binary at a fixed byte offset, not a
|
||||
filesystem -- there is nothing to `mount` in the
|
||||
POSIX-filesystem sense, so this attaches the loop
|
||||
device and reads it exactly like a regular file.
|
||||
|
||||
Either mode is read-only and never modifies the image. Safe to run
|
||||
against a live campaign's artemis.img (the ring's control header CRC
|
||||
means a torn read mid-write is simply rejected as "ring not yet
|
||||
initialized" by the same discipline doe_region.c itself uses -- see
|
||||
doe_region_ctrl_read()); worst case is missing the very latest record
|
||||
until the next run of this script.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
DEVBLOCK_SIZE = 4096
|
||||
|
||||
# Mirrors include/starkernel/doe_region.h and include/starkernel/log_region.h
|
||||
ARTEMIS_SIG_DEVBLOCK_FROM_TOP = 64
|
||||
LOG_REGION_DEVBLOCK_FROM_TOP_BASE = ARTEMIS_SIG_DEVBLOCK_FROM_TOP + 1 # 65
|
||||
LOG_REGION_MAX_DEVBLOCKS = 32
|
||||
DOE_REGION_DEVBLOCK_FROM_TOP_BASE = (
|
||||
LOG_REGION_DEVBLOCK_FROM_TOP_BASE + 1 + LOG_REGION_MAX_DEVBLOCKS
|
||||
) # 98
|
||||
|
||||
DOE_SLOT_SIZE = 64
|
||||
DOE_SLOTS_PER_DEVBLOCK = DEVBLOCK_SIZE // DOE_SLOT_SIZE # 64
|
||||
DOE_REGION_MAGIC = 0x44454F44 # 'DOED'
|
||||
DOE_REGION_VERSION_0 = 0
|
||||
|
||||
# doe_region_ctrl_t: magic(u64) devblocks(u32) head_slot(u32) tail_slot(u32)
|
||||
# record_count(u32) hdr_crc(u64), little-endian, rest is padding to 4096.
|
||||
CTRL_FMT = "<QIIIIQ"
|
||||
CTRL_SIZE = struct.calcsize(CTRL_FMT)
|
||||
|
||||
# doe_trial_slot_t: fleet_k_q48(u64) switch_count_cumulative(u64)
|
||||
# run_id(u32) cfg(u32) rep(u32) nw(u32) mode(u32) fail_count(u32)
|
||||
# fleet_conserved(u32) isa(8s), rest is padding to 64.
|
||||
SLOT_FMT = "<QQIIIIIII8s"
|
||||
SLOT_SIZE = struct.calcsize(SLOT_FMT)
|
||||
assert SLOT_SIZE <= DOE_SLOT_SIZE
|
||||
|
||||
|
||||
def dft_to_offset(devblock_from_top: int, total_devblocks: int) -> int:
|
||||
idx = total_devblocks - 1 - devblock_from_top
|
||||
if idx < 0:
|
||||
raise ValueError(
|
||||
f"devblock_from_top={devblock_from_top} exceeds total_devblocks={total_devblocks}"
|
||||
)
|
||||
return idx * DEVBLOCK_SIZE
|
||||
|
||||
|
||||
@contextmanager
|
||||
def loop_attach(img_path: Path):
|
||||
"""Attach img_path via losetup (read-only), yield the /dev/loopN path,
|
||||
detach on exit. Requires sudo -- the literal 'mount on a loop device'
|
||||
step; nothing here is a filesystem mount."""
|
||||
result = subprocess.run(
|
||||
["sudo", "losetup", "--find", "--show", "--read-only", str(img_path)],
|
||||
capture_output=True, text=True, check=True,
|
||||
)
|
||||
loop_dev = result.stdout.strip()
|
||||
print(f" attached {img_path} -> {loop_dev}", file=sys.stderr)
|
||||
try:
|
||||
yield loop_dev
|
||||
finally:
|
||||
subprocess.run(["sudo", "losetup", "--detach", loop_dev], check=False)
|
||||
print(f" detached {loop_dev}", file=sys.stderr)
|
||||
|
||||
|
||||
def read_ring(dev_path: str):
|
||||
"""Returns a list of dicts, one per live trial record, in ring order
|
||||
(oldest first). Empty list if the ring has never been written."""
|
||||
size = Path(dev_path).stat().st_size if not dev_path.startswith("/dev/") else None
|
||||
if size is None:
|
||||
with open(dev_path, "rb") as f:
|
||||
f.seek(0, 2)
|
||||
size = f.tell()
|
||||
|
||||
total_devblocks = size // DEVBLOCK_SIZE
|
||||
if total_devblocks <= DOE_REGION_DEVBLOCK_FROM_TOP_BASE:
|
||||
raise ValueError(
|
||||
f"image too small ({size} bytes, {total_devblocks} devblocks) "
|
||||
f"for the DOE region fence (needs > {DOE_REGION_DEVBLOCK_FROM_TOP_BASE})"
|
||||
)
|
||||
|
||||
ctrl_off = dft_to_offset(DOE_REGION_DEVBLOCK_FROM_TOP_BASE, total_devblocks)
|
||||
with open(dev_path, "rb") as f:
|
||||
f.seek(ctrl_off)
|
||||
ctrl_raw = f.read(DEVBLOCK_SIZE)
|
||||
|
||||
magic, devblocks, head_slot, tail_slot, record_count, hdr_crc = struct.unpack_from(
|
||||
CTRL_FMT, ctrl_raw, 0
|
||||
)
|
||||
magic_lo = magic & 0xFFFFFFFF
|
||||
version = (magic >> 32) & 0xFF
|
||||
if magic_lo != DOE_REGION_MAGIC or version != DOE_REGION_VERSION_0:
|
||||
print(" DOE region not yet initialized (no trials persisted yet)", file=sys.stderr)
|
||||
return []
|
||||
|
||||
total_slots = devblocks * DOE_SLOTS_PER_DEVBLOCK
|
||||
records = []
|
||||
with open(dev_path, "rb") as f:
|
||||
idx = head_slot
|
||||
for _ in range(record_count):
|
||||
slot_devblock_dft = DOE_REGION_DEVBLOCK_FROM_TOP_BASE + 1 + (idx // DOE_SLOTS_PER_DEVBLOCK)
|
||||
slot_off_in_devblock = (idx % DOE_SLOTS_PER_DEVBLOCK) * DOE_SLOT_SIZE
|
||||
blk_off = dft_to_offset(slot_devblock_dft, total_devblocks)
|
||||
f.seek(blk_off + slot_off_in_devblock)
|
||||
raw = f.read(SLOT_SIZE)
|
||||
(fleet_k_q48, switch_count, run_id, cfg, rep, nw, mode,
|
||||
fail_count, fleet_conserved, isa_raw) = struct.unpack(SLOT_FMT, raw)
|
||||
isa = isa_raw.split(b"\x00", 1)[0].decode("ascii", errors="replace")
|
||||
records.append({
|
||||
"isa": isa,
|
||||
"run_id": run_id,
|
||||
"cfg": cfg,
|
||||
"rep": rep,
|
||||
"nw": nw,
|
||||
"mode": mode,
|
||||
"fail_count": fail_count,
|
||||
"fleet_k_q48": fleet_k_q48,
|
||||
"fleet_conserved": fleet_conserved,
|
||||
"switch_count_cumulative": switch_count,
|
||||
})
|
||||
idx = (idx + 1) % total_slots
|
||||
|
||||
print(f" ring: devblocks={devblocks} head={head_slot} tail={tail_slot} "
|
||||
f"record_count={record_count} -> {len(records)} records read", file=sys.stderr)
|
||||
return records
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--img", default="disk/artemis.img", help="Path to the disk image (default: disk/artemis.img)")
|
||||
ap.add_argument("--loop", action="store_true", help="Attach --img via losetup first (needs sudo)")
|
||||
ap.add_argument("-o", "--out", default=None, help="Output CSV path (default: stdout)")
|
||||
args = ap.parse_args()
|
||||
|
||||
img_path = Path(args.img)
|
||||
if not img_path.exists():
|
||||
print(f"extract_doe_region: image not found: {img_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.loop:
|
||||
with loop_attach(img_path) as loop_dev:
|
||||
records = read_ring(loop_dev)
|
||||
else:
|
||||
records = read_ring(str(img_path))
|
||||
|
||||
fieldnames = ["isa", "run_id", "cfg", "rep", "nw", "mode", "fail_count",
|
||||
"fleet_k_q48", "fleet_conserved", "switch_count_cumulative"]
|
||||
|
||||
out_f = open(args.out, "w", newline="") if args.out else sys.stdout
|
||||
try:
|
||||
writer = csv.DictWriter(out_f, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
for rec in records:
|
||||
writer.writerow(rec)
|
||||
finally:
|
||||
if args.out:
|
||||
out_f.close()
|
||||
print(f" wrote {len(records)} rows -> {args.out}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user