The fabric keeps its occupied places as a list in ascending order, and the step's nine passes walk that list in the same order as before. A place removed mid-walk keeps its link, so the walk goes on to the next node. Test first: test_fabric_walk.c, 13 checks. Every existing fabric and mesh test passes with the same checks, and so do test-32 and ASan/UBSan. The host mesh boot runs POST 990/990. Over five alternating runs its median falls from 11.695 s to 7.999 s. Logs are in logs/20261117-mechanism-a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
123 lines
6.3 KiB
Markdown
123 lines
6.3 KiB
Markdown
# The fabric step's cost -- the regression at 046ce0be fixed -- design, for ruling
|
|
|
|
**Corrected 2026-11-17, Captain Bob:** this is not a regression. "It was architecturally incorrect. That's
|
|
why we went back and did this wiring; to eliminate the defective architecture and locate slowdowns" -- with
|
|
ASan, UBSan, perf et cetera. The direct loop before 046ce0be was the defective architecture. The fabric
|
|
wiring removed it and exposed the slowdowns, and the instruments locate them. The measurements below stand;
|
|
where this document says "regression", read "the slowdown the correct architecture exposed".
|
|
|
|
Written 2026-11-17. Captain Bob, asked how to handle aarch64 running out of its 570 s in POST:
|
|
**"Fix the regression"**. This is treated as a correctness regression, not as tuning. The background
|
|
is in `2026-11-17-the-regression-at-046ce0be.md`.
|
|
|
|
**Nothing here is built.** The design and its first step wait for his ruling.
|
|
|
|
## 1. What stays exactly as it is
|
|
|
|
The fabric's semantics are ruled, and they do not change:
|
|
- "Every node that is not blocked runs; on one processor that is each in rotation, an instruction
|
|
word at a time ... not a policy" (ENGINE.md §3d, ruled "A").
|
|
- "There is no choice of whose turn it is" (MESH.md ruling 7 and §4.3; `fabric.h`).
|
|
|
|
**Unchanged:**
|
|
- the order of a step's phases;
|
|
- which node executes;
|
|
- what is handed over, and when;
|
|
- what every node sees;
|
|
- POST's results, `dict_hash` on every ISA, and every existing test's checks.
|
|
|
|
**What changes:** only what a step costs, when nodes are absent, blocked or have nothing new.
|
|
|
|
## 2. What it costs now (code, `v4/src/fabric.c:423-490`)
|
|
|
|
One step is one instruction word for each awake node. Every step walks all `f->capacity` places, 16 in
|
|
the kernel, in ten passes:
|
|
1. the have-mask, an 8-port loop for each awake node;
|
|
2. `look_about`, an 8-port loop, for each node that is not blocked;
|
|
3. one instruction word for each awake node;
|
|
4. the have-fetched clear;
|
|
5. and 6. two operation passes;
|
|
7. the handing over of writes;
|
|
8. the handing over of reads, an 8-port device loop;
|
|
9. the gone-error pass.
|
|
|
|
The mesh then polls the console's queue (`v4_mesh_step`, `v4_mesh_tell`).
|
|
|
|
**In the boot, POST included, only Hera exists:** 15 of the 16 places are empty (`v4/system/mesh.c`;
|
|
births come later). f10d2bc8 did, for each instruction word, one have-mask and one
|
|
`v4_exec_step_word`. 046ce0be does the whole walk for each.
|
|
|
|
## 3. Step 0 -- measure before choosing (no change to the code)
|
|
|
|
**Intent:** to know which parts of the step the time goes to before deciding what to change.
|
|
|
|
**How:**
|
|
- Profile the host mesh boot (`make -C v4 mesh-boot`, POST 990 on Hera, in the fabric), with gprof
|
|
or perf on the same build.
|
|
- Count, per instruction word executed, the work in each of the ten passes and in `takes()`.
|
|
|
|
**The observation that proves it** (named before the run): the share of the time in
|
|
`v4_fabric_step`, and in each pass, adds up to the gap between f10d2bc8's direct loop and the fabric.
|
|
|
|
**Evidence:** the profile and the counts, committed under `logs/<TS>/fabric-cost/`.
|
|
|
|
## 4. The candidates (chosen with Step 0's data, and ruled before any is built)
|
|
|
|
| | Mechanism | From the record | Risk to the semantics |
|
|
|---|---|---|---|
|
|
| A | The fabric keeps a list of the places that hold a node, and each pass walks that list, not every place. | FABRIC-3.5 §XXII and §XXXV.2: no per-tick walk over every place ("hundreds of VMs") | None: an empty place takes part in no pass. The order of the walk is the same. |
|
|
| B | A node that is blocked, asleep or stopped is skipped in the passes that cannot touch it, by one flag test, not by the 8-port loops. | FABRIC-3 §XXVIII: "a flag set, not a walk" | Low: each pass's condition is kept, read once. |
|
|
| C | The have-mask and `look_about` are recomputed only for a node whose wires or neighbours changed since the last step. | FABRIC-0 GAP-A1; 3.5 §XLIII: the target takes its own on its own turn | Higher: what changed must be tracked exactly, or a node misses a message. Only if A and B are not enough. |
|
|
|
|
## 5. How each built change is verified (named now)
|
|
|
|
1. **Semantics unchanged.** These tests pass with every check as before:
|
|
- `test_fabric.c` (334 checks);
|
|
- `test_host_mesh.c`, including the 26,814 disturbances;
|
|
- `test_host_unit.c`, `test_host_depth.c`, `test_manage.c`;
|
|
- `test_mesh_boot.c`;
|
|
- `make -C v4 test test-32 sanitize hosted-check`.
|
|
2. **The same machine.** POST 990 of 990, and `dict_hash` equal on all three ISAs and to today's value
|
|
(`0xa6bae4f1038dafd9`).
|
|
3. **The cost, measured as HEAT-OP's was** (`logs/20261113-005047/heat-op/timing.txt`):
|
|
- five runs each, alternating OLD and NEW, of the host mesh boot;
|
|
- medians and instruction counts;
|
|
- then the kernel boots.
|
|
4. **The gate.** The protocol on a clean tree: aarch64 inside its 570 s, with no timeout raised and no
|
|
case cut.
|
|
|
|
## 6. For ruling
|
|
|
|
Run Step 0 (measure only), then bring back the data and the choice among A, B and C?
|
|
|
|
## 7. As built: mechanism A (2026-11-17, Captain Bob: "Go")
|
|
|
|
**What changed:**
|
|
- The fabric keeps the places that hold a node as a list in ascending order (`v4_place.next`,
|
|
`v4_fabric.first`; `v4_fabric_first` and `v4_fabric_next` in `fabric.h`). `v4_fabric_add` puts a node
|
|
into the list and `v4_fabric_remove` takes it out.
|
|
- Each of the step's nine passes walks this list, not every place, in the same order as before.
|
|
- A place removed while a pass is at it keeps its link, so the walk goes on to the next node, as the loop
|
|
over every place did. A node added in a step after the place being walked is walked in that step, as
|
|
before.
|
|
|
|
**Test first:** `v4/tests/test_fabric_walk.c`, 13 checks. It failed to build before the change (no walk)
|
|
and passes after.
|
|
|
|
**Verified** (`logs/20261117-mechanism-a/`):
|
|
- Semantics unchanged, every check as before: `test_fabric` 334, `test_host_mesh` 244 (with its
|
|
disturbances), `test_host_unit` 126, `test_host_depth`, `test_manage` 78, and all 52 tests at 64 bits.
|
|
- The 32-bit build (`test-32`) passes.
|
|
- ASan and UBSan are clean on the fabric tests and the mesh boot.
|
|
- The host mesh boot runs POST 990 of 990, with 103 checks.
|
|
- **The cost** (`timing.txt`, five runs each, alternating OLD = d34fa415 and NEW):
|
|
|
|
| | Median | Change |
|
|
|---|---|---|
|
|
| OLD | 11.695 s | |
|
|
| NEW | 7.999 s | 32% less |
|
|
|
|
POST was 990 of 990 in every run. Instruction counts are not given: there is no `perf` in this
|
|
container.
|
|
- **The gate:** the protocol, next.
|