Files
LithosAnanke/experiments/std79-doe/report-20260912/combine.py
T
Claude 3bd81097e1 chore(license): an SPDX header on every source file that had none (LicenseRef-Starship-1.0) [skip ci]
The header scan (Captain Bob, 2026-11-10): 788 source files -- 351 with the
SL-1.0 header, 12 with the old name (docs/working/experiments, kept by
ruling), 31 under someone else's licence (tools/kconfig, Linux's, GPL-2.0:
kept), and 394 with none. Ruled: the SPDX form; capsules (.4th, .f18) and
proofs (.thy) left for a ruling of their own (mkcapsule's 64x16 block format,
content-addressed capsules).

253 files now begin
    SPDX-License-Identifier: LicenseRef-Starship-1.0
    Copyright (c) 2023-2026 Robert A. James
as a /* */ block (C, headers, assembly, linker scripts) or # lines (scripts,
Makefiles, Kconfig; below a shebang or a coding line); tools/mkcapsule.c,
which had the copyright line, gets the SPDX line only.

Not touched: docs/working/ (archive and old campaign material, as the
ruling kept the experiments); v4/system/post_cases.c (generated by
v4/tools/mkpost.py, "do not edit" -- the generator has the header).

Comments only: the kernel builds on amd64, aarch64, riscv64; make -C v4
test-32: 39 programs, 0 failures; every changed script parses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GE3SDvQn5EoxdGufk84zz2
2026-11-10 04:12:49 +00:00

81 lines
3.6 KiB
Python

#!/usr/bin/env python3
# SPDX-License-Identifier: LicenseRef-Starship-1.0
# Copyright (c) 2023-2026 Robert A. James
"""
combine.py -- build combined.csv from the three per-architecture
correlated CSVs (correlate_doe.py's output), decoding Q48.16 fixed-point
fields to plain floats and jitter_bits' IEEE754 bit pattern to a real
jitter_ns value, since R has no convenient native uint64/bit-punning
story worth using here.
v2 (2026-09-12): adds fleet_k (decoded fleet_k_q48 -- vm_physics_fleet_heat_sum()
over ALL live VMs, the genuine conservation invariant, requested explicitly
after the first analysis pass omitted it -- see doe_log.c's column 19/20
doc comment) and fleet_conserved (already 0/1, passed through as-is).
Usage:
python3 combine.py amd64-correlated.csv aarch64-correlated.csv riscv64-correlated.csv combined.csv
"""
import csv
import struct
import sys
Q48 = 65536.0
def main():
if len(sys.argv) < 3:
print("usage: combine.py <in1.csv> [in2.csv ...] <out.csv>", file=sys.stderr)
sys.exit(1)
*in_paths, out_path = sys.argv[1:]
fields = ["arch", "run_id", "id_idx", "id_label", "rep", "tick_number", "elapsed_ns",
"tick_interval_ns", "hot_word_count", "avg_word_heat", "window_width",
"actual_window_size", "predicted_label_hits", "jitter_ns", "apic_ticks",
"time_trust", "variance", "vm_call_depth_max", "hera_heat", "hermes_heat",
"artemis_heat", "fleet_k", "fleet_conserved"]
out_rows = []
for path in in_paths:
arch = path.split("-correlated.csv")[0].split("/")[-1]
with open(path) as f:
for row in csv.DictReader(f):
jitter_bits = int(row["jitter_bits"])
jitter_ns = struct.unpack('<d', struct.pack('<Q', jitter_bits))[0]
out = dict(row)
out["arch"] = arch
out["avg_word_heat"] = int(row["avg_word_heat_q48"]) / Q48
out["time_trust"] = int(row["time_trust_q48"]) / Q48
out["variance"] = int(row["variance_q48"]) / Q48
out["hera_heat"] = int(row["hera_heat_q48"]) / Q48
out["hermes_heat"] = int(row["hermes_heat_q48"]) / Q48
out["artemis_heat"] = int(row["artemis_heat_q48"]) / Q48
out["jitter_ns"] = jitter_ns
fleet_k_q48 = int(row["fleet_k_q48"])
out["fleet_k"] = fleet_k_q48 / Q48
# fleet_conserved is doe_log.c's very LAST printed field, with
# nothing after it in the row to bound the regex match against
# -- unlike every other field, a resumed trial digit with zero
# separator can merge into it (same failure mode
# correlate_doe.py's RUN_ID_MAP works around for id_idx,
# confirmed live 2026-09-12: raw values like
# "19223372036854775807" appeared where 0/1 was expected).
# Derive it instead from fleet_k_q48 directly, using the same
# epsilon and threshold vm_physics_conserved() uses
# (capsule_vm_physics.c: 3277/65536 = 5% of Q48_ONE) --
# correct by construction, immune to the splice.
Q48_ONE = 65536
EPSILON_Q48 = 3277
diff = abs(fleet_k_q48 - Q48_ONE)
out["fleet_conserved"] = 1 if diff < EPSILON_Q48 else 0
out_rows.append({k: out[k] for k in fields})
with open(out_path, "w", newline="") as f:
w = csv.DictWriter(f, fieldnames=fields)
w.writeheader()
w.writerows(out_rows)
print(f"rows written: {len(out_rows)}")
if __name__ == "__main__":
main()