docs: the 12 duplicates deleted; README rewritten to the system as it is

Captain Bob's rulings, 2026-10-14: "delete" -- the duplicates listed in
DELETE-CANDIDATES.md are removed (the list stays as the record; the
formal tier's provenance comments repointed at the maintained copies) --
and "rewrite" the README: v2.1.0, the Tripod as Hera/Artemis/Hestia,
kernel-Hermes, identities on thumbdrives, the corrected ACL overhead,
1052-test POST and the current parity line, keyboards and storage, board
images carrying both systems and how to switch, and docs/README.md as
the way in.  The book is left as it is, to be reworked later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
2026-10-14 16:45:39 -04:00
co-authored by Claude Opus 5.5
parent 776c89457e
commit f8fbc37698
23 changed files with 91 additions and 4932 deletions
File diff suppressed because it is too large Load Diff
@@ -1,179 +0,0 @@
================================================================================
PHYSICS ENGINE VALIDATION - QUICK REFERENCE GUIDE
================================================================================
EXPERIMENTS COMPLETED:
✓ 90-run comprehensive validation (3 configs × 30 runs each)
✓ CPU thermal monitoring (captured temperature & frequency)
✓ Data integrity validation (all 90 rows clean)
✓ Thermal hypothesis testing (REFUTED - no thermal noise)
✓ Variance source analysis (identified OS scheduling + memory latency)
CURRENT STATE:
Location: /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
Results: experiment_results.csv (90 runs, all data validated)
Analysis Files:
- analyze_thermal_hypothesis.py (thermal analysis)
- analyze_variance_sources.py (variance decomposition)
- VARIANCE_ANALYSIS_SUMMARY.md (comprehensive findings)
================================================================================
KEY FINDINGS AT A GLANCE
================================================================================
THERMAL HYPOTHESIS: ✗ REFUTED
- CPU remained at exactly 27°C throughout all 90 runs
- No thermal drift, no frequency throttling
- Conclusion: Thermal noise is NOT a factor in your measurements
VARIANCE SOURCE RANKING (Coefficient of Variation):
1. OS SCHEDULING NOISE (PRIMARY) - 60-70% CV ⚠️
A_BASELINE: 69.58% (range: 3.34-17.67ms, 305% span!)
C_FULL: 60.23% (range: 6.56-33.15ms, 299% span!)
B_CACHE: 32.35% (range: 6.27-19.26ms, 167% span)
→ This is the DOMINANT source of variance
→ Simple configs are WORSE (more sensitive to preemption)
→ Caused by: context switching, interrupts, page faults
2. MEMORY LATENCY VARIABILITY (SECONDARY) - 54-103% CV ⚠️
B_CACHE cache hit latency: 40.1 ns ± 41.4 ns (103.3% CV)
C_FULL cache hit latency: 37.9 ns ± 20.5 ns (54.0% CV)
→ Extreme variability in L1/L2 hit times
→ Indicates cache coherency misses, NUMA effects
→ 80ns swings in what should be 40ns hits
3. CACHE HIT RATE (EXCELLENT) - 0% CV ✓
B_CACHE: 17.39% ± 0.00%
C_FULL: 17.39% ± 0.00%
→ Perfect consistency! Cache behavior is deterministic
→ Variance NOT from unpredictable cache misses
→ Working set is stable across all runs
4. WARMUP EFFECTS (INTERESTING) - -25% to +6%
A_BASELINE: +6.4% slower (degradation over time)
B_CACHE: +0.5% stable (excellent warmup)
C_FULL: -25.4% FASTER (improves significantly!)
→ C_FULL benefits from warming caches
→ A_BASELINE shows slight system degradation
================================================================================
QUICK FIXES (DO THIS NOW)
================================================================================
STEP 1: CPU PINNING (Expected improvement: -30-40% variance)
Run your benchmark pinned to a single CPU core:
$ taskset -c 0 ./build/starforth ...
Or for the experiment:
$ taskset -c 0 scripts/run_comprehensive_physics_experiment.sh ...
STEP 2: SET PERFORMANCE MODE (Expected improvement: -10-15% variance)
Disable frequency scaling:
$ echo performance | sudo tee /sys/devices/system/cpu/*/cpufreq/scaling_governor
Verify:
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
STEP 3: USE ROBUST STATISTICS (Immediate improvement)
Don't use mean ± stdev when you have 69% CV!
Instead, use trimmed mean:
- Sort runtimes
- Remove top/bottom 10%
- Average the middle 80%
Or use median + IQR (interquartile range)
Python example:
runtimes = [4.5, 4.7, 5.2, 17.6] # 17.6 is outlier
sorted_rt = sorted(runtimes)
trimmed = sorted_rt[1:-1] # Remove min and max
mean = sum(trimmed) / len(trimmed)
COMBINED EXPECTED IMPROVEMENT:
Current: A_BASELINE ~70% CV
After Step 1+2: ~30-40% CV
After Step 3: Statistics properly account for outliers
Result: Much clearer performance comparisons!
================================================================================
DETAILED MITIGATION PRIORITY
================================================================================
IMMEDIATE (This week):
[1] CPU pinning with taskset
[2] Performance governor setting
[3] Rerun experiment with CPU pinning
[4] Switch to trimmed mean analysis
SHORT TERM (Next 1-2 weeks):
[1] Disable SMT (hyper-threading)
[2] Increase warmup phase (5 runs instead of current)
[3] Use perf to measure cache misses
$ perf stat -e cache-references,cache-misses \
taskset -c 0 ./build/starforth ...
MEDIUM TERM (If variance still >30%):
[1] Try SCHED_FIFO real-time scheduling (requires root)
[2] NUMA binding with numactl
[3] Isolate CPU cores (kernel parameter isolcpus)
[4] Profile with perf-record to find hotspots
LONG TERM (For production benchmarks):
[1] Use real-time kernel (RT_PREEMPT)
[2] Dedicated measurement system
[3] Disable all background services
================================================================================
FILES TO READ
================================================================================
DETAILED ANALYSIS:
→ /home/rajames/CLionProjects/StarForth/physics_experiment/VARIANCE_ANALYSIS_SUMMARY.md
(Full findings, root cause analysis, mitigation strategies)
DATA FILES:
→ /home/rajames/CLionProjects/StarForth/physics_experiment/full_90_run_comprehensive/
experiment_results.csv (90 rows of clean data, all metrics)
ANALYSIS SCRIPTS:
→ analyze_thermal_hypothesis.py (thermal analysis)
→ analyze_variance_sources.py (variance decomposition - rerun anytime)
================================================================================
QUICK TEST: VALIDATE IMPROVEMENTS
================================================================================
After applying mitigations, rerun and compare:
# Before mitigations (baseline)
$ ./scripts/run_comprehensive_physics_experiment.sh before_fix
$ python3 analyze_variance_sources.py \
before_fix/experiment_results.csv
# After mitigations
$ taskset -c 0 ./scripts/run_comprehensive_physics_experiment.sh after_fix
$ python3 analyze_variance_sources.py \
after_fix/experiment_results.csv
Expected: Coefficient of Variation drops from 60-70% to <40%
================================================================================
BOTTOM LINE
================================================================================
Your data collection script: ✓ PERFECT (no issues found)
Your thermal hypothesis: ✗ NOT CONFIRMED (system is cool)
Your actual problem: → OS scheduling noise (expected, solvable)
Next action: Pin to CPU + set performance mode, then rerun.
Expected result: Variance cut by 40% or more.
For questions, see VARIANCE_ANALYSIS_SUMMARY.md for full technical details.
================================================================================
@@ -1,16 +0,0 @@
Experiment Start: 2025-11-07T06:40:43
Output Directory: physics_experiment/full_90_run_comprehensive
Runs per Configuration: 30
Total Runs: 90
Benchmark Iterations: 100000
Build Profile: fastest
Execution: Randomized order (eliminates temporal bias)
Experiment End: 2025-11-07T06:49:08
Total Runtime: 8 minutes (505 seconds)
Results CSV: physics_experiment/full_90_run_comprehensive/experiment_results.csv
Run Logs: physics_experiment/full_90_run_comprehensive/run_logs/
Test Matrix: physics_experiment/full_90_run_comprehensive/run_logs/test_matrix.txt
@@ -1,90 +0,0 @@
A_BASELINE,0,0,8
B_CACHE,1,0,21
A_BASELINE,0,0,17
B_CACHE,1,0,25
C_FULL,1,1,23
B_CACHE,1,0,7
B_CACHE,1,0,16
B_CACHE,1,0,13
B_CACHE,1,0,30
C_FULL,1,1,26
A_BASELINE,0,0,28
C_FULL,1,1,16
B_CACHE,1,0,22
A_BASELINE,0,0,24
A_BASELINE,0,0,18
B_CACHE,1,0,8
C_FULL,1,1,12
A_BASELINE,0,0,19
C_FULL,1,1,4
A_BASELINE,0,0,3
C_FULL,1,1,14
B_CACHE,1,0,9
A_BASELINE,0,0,11
A_BASELINE,0,0,10
C_FULL,1,1,22
B_CACHE,1,0,1
B_CACHE,1,0,28
C_FULL,1,1,15
C_FULL,1,1,1
C_FULL,1,1,30
A_BASELINE,0,0,13
C_FULL,1,1,29
B_CACHE,1,0,11
B_CACHE,1,0,3
A_BASELINE,0,0,29
C_FULL,1,1,6
A_BASELINE,0,0,20
A_BASELINE,0,0,15
B_CACHE,1,0,14
B_CACHE,1,0,5
C_FULL,1,1,20
A_BASELINE,0,0,16
C_FULL,1,1,3
A_BASELINE,0,0,23
B_CACHE,1,0,6
A_BASELINE,0,0,30
A_BASELINE,0,0,7
C_FULL,1,1,19
C_FULL,1,1,5
B_CACHE,1,0,15
B_CACHE,1,0,29
C_FULL,1,1,13
C_FULL,1,1,9
C_FULL,1,1,25
A_BASELINE,0,0,4
B_CACHE,1,0,17
C_FULL,1,1,21
C_FULL,1,1,17
B_CACHE,1,0,20
B_CACHE,1,0,26
C_FULL,1,1,11
A_BASELINE,0,0,27
B_CACHE,1,0,18
B_CACHE,1,0,12
C_FULL,1,1,2
A_BASELINE,0,0,25
A_BASELINE,0,0,14
C_FULL,1,1,24
C_FULL,1,1,8
B_CACHE,1,0,27
B_CACHE,1,0,4
C_FULL,1,1,7
A_BASELINE,0,0,9
A_BASELINE,0,0,2
C_FULL,1,1,18
B_CACHE,1,0,19
A_BASELINE,0,0,21
A_BASELINE,0,0,22
C_FULL,1,1,28
A_BASELINE,0,0,12
A_BASELINE,0,0,26
B_CACHE,1,0,24
B_CACHE,1,0,10
B_CACHE,1,0,2
B_CACHE,1,0,23
A_BASELINE,0,0,6
A_BASELINE,0,0,5
C_FULL,1,1,10
C_FULL,1,1,27
A_BASELINE,0,0,1
File diff suppressed because it is too large Load Diff