tools(v4): mkpost --all -- v3's whole POST as a table, measured on v4: 849 of 1020 (#189)
v4/tools/mkpost.py --all --out FILE: every case of v3's whole POST (its 22 POST modules in test_runner.c's order, Q48, stress, integration), v3 the reference, without touching system/post_cases.c or its document. Run on Hera in the kernel's fabric after Phase 7 Task 4: 1020 cases, 849 pass, 171 fail -- by word and kind in docs/v4.0.0/POST-PARITY.md section 5. Not pushed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
@@ -71,3 +71,25 @@ The 317 test words v4 already has; they are left out only because `mkpost.py` ke
|
||||
About 1,050 cases: 888 in section 1, 146 FORTH-line cases in section 2's modules, and the C modules' 12 tests -- 1,046. v4 has 539 today.
|
||||
The loops' own words (physics freeze, inference, entropy, top words) are 22 cases; the ACL and Mama
|
||||
modules wait on section 3.
|
||||
|
||||
## 5. Measured (2026-11-14, Task 8)
|
||||
|
||||
`v4/tools/mkpost.py --all --out FILE` writes every case of v3's whole POST -- v3's 22 POST modules in
|
||||
`test_runner.c`'s order, Q48, the stress and integration tests -- as a table, v3 the reference, leaving
|
||||
`system/post_cases.c` as it is. Run on Hera in the kernel's fabric (the mesh boot, with the kernel's
|
||||
capsules) on the tree after Task 4: **1020 cases, 849 pass, 171 fail.**
|
||||
|
||||
Failures by word: `C,` 8, `TOP-WORDS` 7, `CASE` 5, `M/MOD` 5, `U/MOD` 5, `(-` 4, `BASE_OUTPUT` 4, `COMPARE` 4, `LATEST` 4, `M+` 4, `SEARCH` 4, `WORD` 4, `-!` 3, `2!` 3, `COMPLETE_PROGRAMS` 3, `FENCE` 3, `MEMORY_STRESS` 3, `ORDER` 3, `SCAN` 3, `SKIP` 3, `SP@` 3, `STACK_DEPTH` 3, `WORD-ENTROPY` 3, `2,` 2, `2>R` 2, `2R>` 2, `2R@` 2, `>BODY` 2, `AGAIN` 2, `BEGIN` 2, `CMOVE>` 2, `DEEP_NESTING` 2, `EXIT` 2, `FROZEN?` 2, `MIXED_COMPLEX` 2, `NUMERIC_BOUNDARY` 2, `RETURN_STACK` 2, `S"` 2, `STACK_BOUNDARY` 2, `STACK_MGMT` 2, `STRING_EDGE` 2, `THRU` 2, `WHILE` 2, `+LOOP` 1, `."` 1, `?` 1, `ALGORITHMS` 1, `BL` 1, `COMPILATION` 1, `COMPILE` 1, `CONTROL_EDGE` 1, `CONVERT` 1, `DABS` 1, `DECAY-RATE@` 1, `DMIN` 1, `ENTROPY@` 1, `ERROR_RECOVERY` 1, `EXECUTE` 1, `FLUSH` 1, `FREEZE-WORD` 1, `HEAT!` 1, `HEAT@` 1, `HIDDEN` 1, `INFER-DECAY-SLOPE` 1, `INFER-EARLY-EXIT@` 1, `INFER-VARIANCE@` 1, `IS` 1, `LITERAL` 1, `Q./` 1, `Q.0` 1, `Q.1` 1, `Q.SCALE` 1, `Q.VARIANCE` 1, `REPEAT` 1, `RESET-ENTROPY` 1, `SMUDGE` 1, `SP!` 1, `SPAN` 1, `STATE` 1, `UNFREEZE-WORD` 1, `UNTIL` 1, `[']` 1, `[COMPILE]` 1.
|
||||
|
||||
What they are:
|
||||
1. **Address units.** v3's cases do byte-address arithmetic in 8-byte cells (`C,`, `2!`, `SEARCH`,
|
||||
`COMPARE`, `CMOVE>`, `-!` ...); v4's address unit is a cell (D-1). The case wants v4's units, as
|
||||
`post79_rules.ADDRESS` already gives 20.
|
||||
2. **What v4 lacks or does otherwise.** `SP@` `SP!`; `CASE` (its cases define `TC` with it); `U/MOD` and
|
||||
`M/MOD` in v3's shape; `SKIP` `SCAN` `SEARCH` `COMPARE`; `ORDER` `THRU` `FLUSH`; the vocabulary and
|
||||
compile-time edges. To be built, or the difference shown to be the standard's.
|
||||
3. **v3's output naming v3's own words.** `TOP-WORDS` expects v3's hottest -- `EXIT`, `DROP`, `LIT`, `."` --
|
||||
and `WORD-ENTROPY` v3's word count: v4's system is not v3's.
|
||||
4. **v3 not doing what its own table says** (29 cases; e.g. `2!.basic`): the expectation recorded is v3's
|
||||
error, and a v4 that does what the table says fails it.
|
||||
5. The rest -- formatting, the stress and integration groups -- case by case.
|
||||
|
||||
+22
-5
@@ -55,6 +55,14 @@ MODULES = [
|
||||
"system_words_test.c", "defining_words_tests.c", "control_words_test.c",
|
||||
]
|
||||
|
||||
# v3's whole POST (Captain Bob, 2026-11-13: Phase 7 is accepted on "v3's whole POST"): test_modules[] in
|
||||
# v3/src/test_runner/test_runner.c, in its order, then the modules it runs besides.
|
||||
MODULES_ALL = MODULES + [
|
||||
"starforth_words_test.c", "acl_words_test.c", "mama_forth_words_test.c",
|
||||
"inference_words_test.c", "physics_freeze_words_test.c",
|
||||
"q48_words_test.c", "stress_tests.c", "integration_tests.c",
|
||||
]
|
||||
|
||||
# The FORTH-79 Standard's Required Word Set, by the standard's own groups.
|
||||
# Written from the standard's word list; check it against the document
|
||||
# before relying on it for a claim of compliance.
|
||||
@@ -307,12 +315,19 @@ def main():
|
||||
ap.add_argument("--v3", default=os.path.join(ROOT, "lfs", "amd64", "starforth"))
|
||||
ap.add_argument("--report", default=None)
|
||||
ap.add_argument("--dump", default=None, help="write what v3 did with every case")
|
||||
ap.add_argument("--all", action="store_true",
|
||||
help="every case of v3's whole POST, every module and word, not the Required set alone "
|
||||
"(#189 Phase 7, Task 8: the measure of parity); with --out, and no document written")
|
||||
ap.add_argument("--out", default=None, help="write the table here, not system/post_cases.c")
|
||||
args = ap.parse_args()
|
||||
global OUTPUT
|
||||
if args.out:
|
||||
OUTPUT = os.path.abspath(args.out)
|
||||
|
||||
required, left_out, cases, seen = set(REQUIRED), [], [], {}
|
||||
for module in MODULES:
|
||||
for module in (MODULES_ALL if args.all else MODULES):
|
||||
for word, name, text, should_error in read_module(os.path.join(MODULE_DIR, module)):
|
||||
if word not in required:
|
||||
if word not in required and not args.all:
|
||||
continue
|
||||
label = ("%s.%s" % (word, name))[:31]
|
||||
seen[label] = seen.get(label, 0) + 1
|
||||
@@ -396,7 +411,7 @@ def main():
|
||||
emit(rule["label"], lines, expectation(rule.get("error", False), rule.get("stack", []), rule.get("out", "")))
|
||||
applied["extra"] += 1
|
||||
unknown = [k for k in list(rules.STANDARD) + list(rules.ADDRESS) if k not in {c["label"] for c in cases}]
|
||||
assert not unknown, "rules for cases that do not exist: %s" % unknown
|
||||
assert args.all or not unknown, "rules for cases that do not exist: %s" % unknown
|
||||
if args.dump:
|
||||
open(args.dump, "w").write("\n".join(dump))
|
||||
write_table(table)
|
||||
@@ -405,7 +420,8 @@ def main():
|
||||
untested = [w for w in REQUIRED if w not in set(tested)]
|
||||
dropped = sum(1 for v in rules.STANDARD.values() if v is None)
|
||||
total = len(cases) - dropped + len(rules.EXTRA)
|
||||
assert total == len(table), (total, len(table))
|
||||
assert args.all or total == len(table), (total, len(table))
|
||||
total = len(table)
|
||||
report = ["POST cases: %d, in %s" % (total, os.path.relpath(OUTPUT, ROOT)),
|
||||
" from v3 unchanged: %d" % (len(cases) - applied["address"] - applied["standard"]),
|
||||
" from v3, address-dependent (post79_rules.ADDRESS): %d" % applied["address"],
|
||||
@@ -418,7 +434,8 @@ def main():
|
||||
report += ["", "v3 did not do what its own table expects (%d):" % len(disagreements)]
|
||||
report += [" %-32s %s" % item for item in disagreements]
|
||||
text = "\n".join(report) + "\n"
|
||||
write_doc(cases, results, left_out, disagreements, untested, total, applied, dropped)
|
||||
if not args.all:
|
||||
write_doc(cases, results, left_out, disagreements, untested, total, applied, dropped)
|
||||
if args.report:
|
||||
open(args.report, "w").write(text)
|
||||
sys.stdout.write(text)
|
||||
|
||||
Reference in New Issue
Block a user