feat(v4): Phase 7 Task 2b -- pipelining and the hot-words cache, built, off as v3 (#189)

Rulings, Captain Bob 2026-11-13: "Build a hot-words cache"; then, v3 being found to have both off
(ENABLE_PIPELINING, ENABLE_HOTWORDS_CACHE: n in Kconfig.physics and kernel/Makefile), "Built, off as v3".

- The kernel, at every call (mesh.c): v3's hit check, the transition, the likeliest next word (kept as each
  count rises; v3 scans 1024 a call -- the same word but in a tie v3's Q48.16 rounding makes), speculation
  at v3's thresholds into the node's cache; a word past heat 50 promoted at its call (v3 promotes at a
  lookup; v4's heat is the kernel's). Counters from the owner's heap, given back when a word goes.
- The node (dict.f18): (LOOKUP) tries (HOT)'s xts first, in F18, calling only C@; the kernel keeps it true
  (entries that go, or whose name is defined again, taken out). Empty unless the build has the cache.
- Tests: test_mesh_boot.c as built (57 checks) and with both on (mesh-boot-on, 65). v4 test: all pass.
- Reported, v3's own: when full, its promote takes the next slot without looking for the word there (a few
  hot words churn all 32 slots); its prefetch misses are never counted. Both reproduced.
- amd64 boots, RESULT: PASS, but 174 s and 181 s against 2a's 146 s, while on the host the mesh boot is no
  slower: being measured. Not pushed (held to the Phase 7 boundary).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
Claude
2026-11-13 21:52:01 +00:00
parent b880a6eaed
commit 3a70e824a2
19 changed files with 473 additions and 145 deletions
+4
View File
@@ -140,3 +140,7 @@ him -> the tag, on his word.
milestone."). What it holds now is all it gets: #189 (the blocker), #191, #192, #193 and what was already in it.
If he proposes another item for alpha.5, remind him of this ruling and offer the next milestone; only his
explicit word that he is overriding it reopens it.
- **Push point: the Phase 7 boundary** (Captain Bob, 2026-11-13: "Yup, phase 7 boundary is our push point";
"I'm not wanting to trigger a new build"). Each push to StarForth-v4.0.0 starts build.yml and moves PR 164's
head. During #189 Phase 7, commit locally and hold; push once, behind a passing protocol, at the Phase 7
boundary. Before then, only on his word.
+24
View File
@@ -201,11 +201,35 @@ set HDR-NFA here ", (TELL-FORGOTTEN) #id, #, 0 #, HDR-NFA #, HDR-LINK set
\ name fetched, the two names compared byte by byte -- so that the only call
\ it makes is C@: it is called from the interpreter for every word of every
\ line, with the user's return stack beneath it.
\ THE HOT-WORDS CACHE first (v3's hotwords_cache_lookup; #189 Phase 7, Captain
\ Bob 2026-11-13: "Build a hot-words cache", "Built, off as v3"): (HOT) holds
\ its count, the kernel's next slot, this word's index, then up to 32 xts the
\ kernel has put there. A visible one with the name is the answer; then the
\ dictionary, as before. The kernel keeps it true: an entry that goes, or
\ one whose name is defined again, it takes out. Empty unless the kernel's
\ build has ENABLE_HOTWORDS_CACHE.
: (LOOKUP)
dup (D)+1 a! ! \ cut the string to 31 characters
C@ -32 + -if LONG drop jump CUT
LONG: drop 31 (D)+1 a! @ C!
CUT:
0 (HOT)+2 a! !
H: (HOT)+2 a! @ (HOT) a! @ xor if HDONE drop
(HOT)+2 a! @ dup 1 + ! \ i, and i + 1 kept
(HOT)+3 + a! @ if HNONE
(D)+2 a! !
(D)+2 a! @ -3 + a! @ 2 and if HVIS drop jump H
HVIS: drop
(D)+2 a! @ -2 + a! @ 4* (D) a! ! \ its name
(D)+1 a! @ C@ 1 + (D)+3 a! !
HC: (D)+3 a! @ if HSAME -1 + dup !
dup (D) a! @ + C@
SWAP (D)+1 a! @ + C@
xor if HEQ drop jump H
HEQ: drop jump HC
HSAME: drop (D)+2 a! @ ;
HNONE: drop jump H
HDONE: drop
0 (D)+4 a! ! \ not yet on to FORTH
CONTEXT a! @ a! @
L: if END
+69 -68
View File
@@ -22,81 +22,82 @@ constant CG-BSLOTS 3
constant STATE 16371
constant (C) 16322
constant (CFP) 16370
constant (Q) 14708
constant (Q) 14673
constant (F) 16279
constant (S) 15812
constant (W) 16285
constant (X) 15792
constant (BOOT) 15750
constant (NEXT-ID) 15746
constant (BOOT-ID) 15747
constant (LOG-LEVEL) 14707
constant (ACL-HOOK) 14706
constant SCR 15742
constant BLK 15743
constant (SRC) 15744
constant (SRC-HOOK) 15745
constant (LINE-STATUS) 14703
constant (PORT) 14572
constant (ME) 14699
constant (CONSOLE) 14696
constant (ROUTE#) 14695
constant (ROUTE-DEFAULT) 14694
constant (LOST) 14693
constant (PRINT-TO) 14692
constant (PRINT-PORT) 14679
constant (DONE-PORT) 14678
constant (ROUTES) 14616
constant (HOT) 15757
constant (BOOT) 15715
constant (NEXT-ID) 15711
constant (BOOT-ID) 15712
constant (LOG-LEVEL) 14672
constant (ACL-HOOK) 14671
constant SCR 15707
constant BLK 15708
constant (SRC) 15709
constant (SRC-HOOK) 15710
constant (LINE-STATUS) 14668
constant (PORT) 14537
constant (ME) 14664
constant (CONSOLE) 14661
constant (ROUTE#) 14660
constant (ROUTE-DEFAULT) 14659
constant (LOST) 14658
constant (PRINT-TO) 14657
constant (PRINT-PORT) 14644
constant (DONE-PORT) 14643
constant (ROUTES) 14581
constant -PORTS -8
constant (NEAR) 14648
constant (AWAIT-FROM) 14673
constant (WIRE) 14584
constant (WIRE-A) 14585
constant (WIRE-B) 14586
constant (WIRE-DO) 14587
constant (WIRE-HOW) 14588
constant (WIRE-HAVE) 14589
constant (S-HAVE) 14677
constant (S-MASK) 14676
constant (S-WIRE) 14675
constant (S-ON) 14674
constant (KEEP) 14672
constant (DEFER) 14671
constant (D-MASK) 14704
constant (IN) 14656
constant (ONE) 14663
constant (OUT-HDR) 14600
constant (W-NEED) 14607
constant (W-PORT) 14608
constant (P-WAY) 14609
constant (P-NEED) 14612
constant (H-PORT) 14613
constant (H-NEED) 14614
constant (H-MASK) 14615
constant (A-SELF) 14610
constant (SEND^) 14611
constant (OUT-TEXT) 14248
constant (SEND-TEXT) 13912
constant (QUIET) 14702
constant (REFUSED) 14705
constant (OUT^) 14698
constant (REPLY) 14697
constant (MSG) 14664
constant (OUT) 14312
constant (WORD-DEFINED) 14701
constant (WORD-FORGOTTEN) 14700
constant (NEAR) 14613
constant (AWAIT-FROM) 14638
constant (WIRE) 14549
constant (WIRE-A) 14550
constant (WIRE-B) 14551
constant (WIRE-DO) 14552
constant (WIRE-HOW) 14553
constant (WIRE-HAVE) 14554
constant (S-HAVE) 14642
constant (S-MASK) 14641
constant (S-WIRE) 14640
constant (S-ON) 14639
constant (KEEP) 14637
constant (DEFER) 14636
constant (D-MASK) 14669
constant (IN) 14621
constant (ONE) 14628
constant (OUT-HDR) 14565
constant (W-NEED) 14572
constant (W-PORT) 14573
constant (P-WAY) 14574
constant (P-NEED) 14577
constant (H-PORT) 14578
constant (H-NEED) 14579
constant (H-MASK) 14580
constant (A-SELF) 14575
constant (SEND^) 14576
constant (OUT-TEXT) 14213
constant (SEND-TEXT) 13877
constant (QUIET) 14667
constant (REFUSED) 14670
constant (OUT^) 14663
constant (REPLY) 14662
constant (MSG) 14629
constant (OUT) 14277
constant (WORD-DEFINED) 14666
constant (WORD-FORGOTTEN) 14665
constant BLOCK# -1
constant BUFFER# -2
constant UPDATE# -3
constant SAVE-BUFFERS# -4
constant EMPTY-BUFFERS# -5
constant (Q/) 15752
constant (QE) 15757
constant (QR) 15765
constant (QL) 15770
constant (QT) 15777
constant (QP) 15783
constant (DP) 15787
constant (Q/) 15717
constant (QE) 15722
constant (QR) 15730
constant (QL) 15735
constant (QT) 15742
constant (QP) 15748
constant (DP) 15752
constant ADDR-DIGITS 8
constant MSB -2147483648
constant MAXHI 2147483647
@@ -115,8 +116,8 @@ constant -HFLOOR -63186
constant DSTACK-DEPTH 16369
constant RSTACK-DEPTH 16368
constant HEAT-OP 13824
constant (EXEC) 14690
constant (RUNX) 14597
constant (EXEC) 14655
constant (RUNX) 14562
constant ANTICLOCK 13856
constant CFBASE 16288
constant CFEND 16320
@@ -127,4 +128,4 @@ constant ROM-DNEGATE 16387
constant ROM-Q.* 16388
constant ROM-Q./ 16389
constant DICT-W 8192
constant (IMG) 14593
constant (IMG) 14558
+69 -68
View File
@@ -22,81 +22,82 @@ constant CG-BSLOTS 3
constant STATE 16371
constant (C) 16322
constant (CFP) 16370
constant (Q) 14708
constant (Q) 14673
constant (F) 16279
constant (S) 15812
constant (W) 16285
constant (X) 15792
constant (BOOT) 15750
constant (NEXT-ID) 15746
constant (BOOT-ID) 15747
constant (LOG-LEVEL) 14707
constant (ACL-HOOK) 14706
constant SCR 15742
constant BLK 15743
constant (SRC) 15744
constant (SRC-HOOK) 15745
constant (LINE-STATUS) 14703
constant (PORT) 14572
constant (ME) 14699
constant (CONSOLE) 14696
constant (ROUTE#) 14695
constant (ROUTE-DEFAULT) 14694
constant (LOST) 14693
constant (PRINT-TO) 14692
constant (PRINT-PORT) 14679
constant (DONE-PORT) 14678
constant (ROUTES) 14616
constant (HOT) 15757
constant (BOOT) 15715
constant (NEXT-ID) 15711
constant (BOOT-ID) 15712
constant (LOG-LEVEL) 14672
constant (ACL-HOOK) 14671
constant SCR 15707
constant BLK 15708
constant (SRC) 15709
constant (SRC-HOOK) 15710
constant (LINE-STATUS) 14668
constant (PORT) 14537
constant (ME) 14664
constant (CONSOLE) 14661
constant (ROUTE#) 14660
constant (ROUTE-DEFAULT) 14659
constant (LOST) 14658
constant (PRINT-TO) 14657
constant (PRINT-PORT) 14644
constant (DONE-PORT) 14643
constant (ROUTES) 14581
constant -PORTS -8
constant (NEAR) 14648
constant (AWAIT-FROM) 14673
constant (WIRE) 14584
constant (WIRE-A) 14585
constant (WIRE-B) 14586
constant (WIRE-DO) 14587
constant (WIRE-HOW) 14588
constant (WIRE-HAVE) 14589
constant (S-HAVE) 14677
constant (S-MASK) 14676
constant (S-WIRE) 14675
constant (S-ON) 14674
constant (KEEP) 14672
constant (DEFER) 14671
constant (D-MASK) 14704
constant (IN) 14656
constant (ONE) 14663
constant (OUT-HDR) 14600
constant (W-NEED) 14607
constant (W-PORT) 14608
constant (P-WAY) 14609
constant (P-NEED) 14612
constant (H-PORT) 14613
constant (H-NEED) 14614
constant (H-MASK) 14615
constant (A-SELF) 14610
constant (SEND^) 14611
constant (OUT-TEXT) 14248
constant (SEND-TEXT) 13912
constant (QUIET) 14702
constant (REFUSED) 14705
constant (OUT^) 14698
constant (REPLY) 14697
constant (MSG) 14664
constant (OUT) 14312
constant (WORD-DEFINED) 14701
constant (WORD-FORGOTTEN) 14700
constant (NEAR) 14613
constant (AWAIT-FROM) 14638
constant (WIRE) 14549
constant (WIRE-A) 14550
constant (WIRE-B) 14551
constant (WIRE-DO) 14552
constant (WIRE-HOW) 14553
constant (WIRE-HAVE) 14554
constant (S-HAVE) 14642
constant (S-MASK) 14641
constant (S-WIRE) 14640
constant (S-ON) 14639
constant (KEEP) 14637
constant (DEFER) 14636
constant (D-MASK) 14669
constant (IN) 14621
constant (ONE) 14628
constant (OUT-HDR) 14565
constant (W-NEED) 14572
constant (W-PORT) 14573
constant (P-WAY) 14574
constant (P-NEED) 14577
constant (H-PORT) 14578
constant (H-NEED) 14579
constant (H-MASK) 14580
constant (A-SELF) 14575
constant (SEND^) 14576
constant (OUT-TEXT) 14213
constant (SEND-TEXT) 13877
constant (QUIET) 14667
constant (REFUSED) 14670
constant (OUT^) 14663
constant (REPLY) 14662
constant (MSG) 14629
constant (OUT) 14277
constant (WORD-DEFINED) 14666
constant (WORD-FORGOTTEN) 14665
constant BLOCK# -1
constant BUFFER# -2
constant UPDATE# -3
constant SAVE-BUFFERS# -4
constant EMPTY-BUFFERS# -5
constant (Q/) 15752
constant (QE) 15757
constant (QR) 15765
constant (QL) 15770
constant (QT) 15777
constant (QP) 15783
constant (DP) 15787
constant (Q/) 15717
constant (QE) 15722
constant (QR) 15730
constant (QL) 15735
constant (QT) 15742
constant (QP) 15748
constant (DP) 15752
constant ADDR-DIGITS 16
constant MSB -9223372036854775808
constant MAXHI 9223372036854775807
@@ -115,8 +116,8 @@ constant -HFLOOR -63186
constant DSTACK-DEPTH 16369
constant RSTACK-DEPTH 16368
constant HEAT-OP 13824
constant (EXEC) 14690
constant (RUNX) 14597
constant (EXEC) 14655
constant (RUNX) 14562
constant ANTICLOCK 13856
constant CFBASE 16288
constant CFEND 16320
@@ -127,4 +128,4 @@ constant ROM-DNEGATE 16387
constant ROM-Q.* 16388
constant ROM-Q./ 16389
constant DICT-W 8192
constant (IMG) 14593
constant (IMG) 14558
@@ -98,6 +98,24 @@ Phase 4).
Task 3 brings L8, Loop #2 and #3 are on (v3's boot sets both off and lets its card decide).
- [x] Measured: amd64 boot 146 s, against Phase 2's 142 s (about +3%).
### Task 2b: pipelining (transitions, the likely next word, speculation) and the hot-words cache in front of FIND, behind `ENABLE_PIPELINING` and `ENABLE_HOTWORDS_CACHE`, off as v3
- [x] Test first: `test_mesh_boot.c` built twice -- as the build is (off: no transitions, an empty
cache; 57 checks) and with both on (`make -C v4 mesh-boot-on`, 65 checks): PA's 12 transitions to
PB, its likeliest next; speculation after 10 puts PB in the node's cache; hits; FIND through the
cache; PB defined again is taken out; a word past heat 50 goes in; FORGET takes the rest out.
- [x] The kernel (`mesh.c`), at every call: v3's hit check, the transition, the likeliest next (kept as
each count rises, not a scan of 1024 a call: the same word but in a tie v3's Q48.16 rounding
makes, recorded in the code), speculation at v3's thresholds (1/2, 10 samples). Counters from
the owner's heap on a word's first transition; given back when it goes.
- [x] The node (`dict.f18`, `(LOOKUP)`): `(HOT)`'s xts tried first, in F18, calling only `C@` as before;
the kernel fills it as v3's promote, and takes out entries that go or whose name is defined again.
Empty unless the kernel's build has `ENABLE_HOTWORDS_CACHE`.
- [x] One departure, ruled by default (Captain Bob did not choose; recorded): v3 promotes a hot word
when a lookup finds it in the dictionary; v4's heat is the kernel's, so it is promoted at the call
where it is hot.
- **Reported, not changed (v3's own):** with the cache full, v3's promote takes the next slot in turn
without looking for the word already there, so a few hot words churn through all 32 slots
(`hotwords_cache_promote`, `v3/src/physics_hotwords_cache.c`). v4 does the same. And v3 never calls
`transition_metrics_record_prefetch_miss`: its misses are always 0, as v4's.
### Task 3: the per-tick loops (tuner, slope, background decay, L8, inference, snapshot)
### Task 4: the words that read and set them (`FREEZE-WORD`, the inference words, ...), and their POST
### Task 5: the StarForth words (Phase 3's: `init.4th`, `starforth.4th`), and their POST
+11
View File
@@ -79,6 +79,16 @@ static uint64_t sk_v4_charge_calls[V4_MESH_WORD_IDS];
static uint32_t sk_v4_charge_n;
static v4_mesh_physics sk_v4_physics[SK_V4_PLACES]; /* each node's physics: its tick, decay slope, L8's gates (#189, Phase 7) */
static uint64_t sk_v4_heartbeat_ticks(void) { return heartbeat_ticks(); } /* what decay is measured by, as v3's VM tick */
/* a word's transition counters, from the kernel's heap, zeroed (pipelining, when the build has it) */
static void *sk_v4_alloc(void *self, unsigned long bytes)
{
unsigned char *p = (unsigned char *)kmalloc((size_t)bytes);
unsigned long i;
(void)self;
if (p) for (i = 0; i < bytes; i++) p[i] = 0;
return p;
}
static void sk_v4_release(void *self, void *p) { (void)self; kfree(p); }
static void sk_v4_charge_one(void *ctx, v4_cell id, v4_uheat_t calls)
{
(void)ctx;
@@ -313,6 +323,7 @@ void sk_v4_boot(void (*chain)(void), const BootDisk *boot_disk)
sk_v4_mesh.forgot = sk_v4_forgot;
sk_v4_mesh.physics = sk_v4_physics;
sk_v4_mesh.ticks = sk_v4_heartbeat_ticks;
sk_v4_mesh.alloc = sk_v4_alloc; sk_v4_mesh.release = sk_v4_release;
sk_v4_mesh.out = sk_v4_out; sk_v4_mesh.key = sk_v4_key;
sk_v4_mesh.words = sk_v4_words; sk_v4_mesh.word_count = sizeof sk_v4_words / sizeof sk_v4_words[0];
sk_v4_mesh.serve = sk_v4_serve;
+1
View File
@@ -0,0 +1 @@
Development boot, not acceptance evidence (2026-11-13): amd64 with #189 Phase 7 Task 2b (pipelining and the hot-words cache, built off as v3). RESULT: PASS; POST 539/539; dict_hash 0x2e4445996bed4f6c ((LOOKUP) changed). Kept for the timing question: this boot was slower than 2a's (174 s and 181 s against 146 s); see the next commits.
Binary file not shown.
+1
View File
@@ -0,0 +1 @@
Development boot, not acceptance evidence (2026-11-13): amd64 with #189 Phase 7 Task 2b (pipelining and the hot-words cache, built off as v3). RESULT: PASS; POST 539/539; dict_hash 0x2e4445996bed4f6c ((LOOKUP) changed). Kept for the timing question: this boot was slower than 2a's (174 s and 181 s against 146 s); see the next commits.
Binary file not shown.
+12 -1
View File
@@ -367,6 +367,17 @@ mesh-boot: $(BINDIR)/test_mesh_boot
@echo " [mesh boot, V4_CELL_BITS=64]"
@$(BINDIR)/test_mesh_boot
# The same with pipelining and the hot-words cache on (#189 Phase 7; built, off as v3 by default).
MESH_ON := -DENABLE_PIPELINING=1 -DENABLE_HOTWORDS_CACHE=1
$(BINDIR)/test_mesh_boot_on: $(BINDIR)/test_mesh_boot
$(CC) $(CFLAGS) $(MESH_ON) -D_POSIX_C_SOURCE=200809L -I$(HERE)/include $(V3_INC) -DV4_CELL_BITS=64 $(HOST_DEFS) -c $(HERE)/tests/test_mesh_boot.c -o $(BINDIR)/test_mesh_boot_on.o
$(CC) $(SYSTEM_CFLAGS) $(MESH_ON) $(BINDIR)/test_mesh_boot_on.o $(HERE)/system/mesh.c $(HERE)/src/manage.c $(BINDIR)/mesh-rom-64.c $(MACHINE_64) $(CAPSULE_DIR_C) $(ENGINE_SRCS) $(SYSTEM_SRCS) $(BINDIR)/v3o-mesh/*.o -o $@
.PHONY: mesh-boot-on
mesh-boot-on: $(BINDIR)/test_mesh_boot_on
@echo " [mesh boot, pipelining and the hot-words cache on]"
@$(BINDIR)/test_mesh_boot_on
# The same, under ASan+UBSan (Captain Bob, 2026-11-13: "Yes, add it to v4-checks"): run by `sanitize`.
MESH_SAN := -O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined -fno-sanitize-recover=all
$(BINDIR)/san-test_mesh_boot: $(BINDIR)/test_mesh_boot
@@ -380,7 +391,7 @@ san-mesh-boot: $(BINDIR)/san-test_mesh_boot
@echo " [ASan+UBSan mesh boot, V4_CELL_BITS=64]"
@$(BINDIR)/san-test_mesh_boot
test: $(addprefix test-,$(WIDTHS)) mesh-boot
test: $(addprefix test-,$(WIDTHS)) mesh-boot mesh-boot-on
@echo "all v4 tests passed"
clean:
+6
View File
@@ -71,6 +71,11 @@ typedef struct {
* and when entries go, or 0 (dict.f18); a kernel that keeps records puts its own there */
v4_cell word_defined, word_forgotten;
/* (HOT): the hot-words cache (#189, Phase 7; Captain Bob, 2026-11-13): its count, the slot the kernel
* takes next when it is full, the lookup's index, then V4_IMAGE_HOT_SIZE xts, which (LOOKUP) tries before
* the dictionary. The kernel fills it, when its build has ENABLE_HOTWORDS_CACHE; otherwise it is empty. */
v4_cell hot;
/* the block window: the word address of the slots the kernel copies
* blocks into (v4/blocks.h) */
v4_cell block_window;
@@ -90,6 +95,7 @@ typedef struct {
/* The memory map's description of the system (the generated file defines
* it): what is known of it before it is built. */
extern const v4_image v4_map_image;
#define V4_IMAGE_HOT_SIZE 32 /* v3's HOTWORDS_CACHE_SIZE */
/* What only the assembled system knows, read from the image record of the
* node `n` that holds it into `im`, which the map's description is in.
+34
View File
@@ -61,6 +61,13 @@ typedef struct {
uint64_t last_decay_tick; /* the node's tick when decay was last applied: the only clock decay reads */
uint16_t temperature_q8; /* the heat, smoothed (an EMA), in Q8 */
uint8_t state_flags; /* the kernel's own: V4_MESH_STATE_PINNED, _FROZEN */
/* pipelining (Loop #4; v3's WordTransitionMetrics), when ENABLE_PIPELINING: how often each word ID came
* next -- V4_MESH_WORD_IDS counters from the owner's alloc, on the first transition -- and the likeliest */
uint64_t *transitions;
uint64_t total_transitions;
uint64_t most_likely_count;
uint32_t most_likely_next;
uint64_t prefetch_attempts, prefetch_hits;
} v4_mesh_account;
/* A word's state flags, as v3's (PHYSICS_STATE_*): immediate and hidden are the node's (its header's
@@ -73,6 +80,23 @@ typedef struct {
/* Each node's physics (v3's VM fields): its heartbeat's tick, the decay slope (Q48.16 heat a tick;
* v3's INITIAL_DECAY_SLOPE_Q48, which the inference engine tunes), and L8's gate on Loop #3. */
#define V4_MESH_INITIAL_DECAY_SLOPE_Q48 21845u
/* Pipelining and the hot-words cache: built, and off unless the build turns them on, as v3's
* (Kconfig.physics; Captain Bob, 2026-11-13: "Built, off as v3"). v3's thresholds. */
#ifndef ENABLE_PIPELINING
#define ENABLE_PIPELINING 0
#endif
#ifndef ENABLE_HOTWORDS_CACHE
#define ENABLE_HOTWORDS_CACHE 0
#endif
#ifndef SPECULATION_THRESHOLD_Q48
#define SPECULATION_THRESHOLD_Q48 0x8000 /* a probability of 1/2, Q48.16 */
#endif
#ifndef MIN_SAMPLES_FOR_SPECULATION
#define MIN_SAMPLES_FOR_SPECULATION 10
#endif
#define V4_MESH_HOT_HEAT 50u /* v3's HOTWORDS_EXECUTION_HEAT_THRESHOLD */
#define V4_MESH_NO_WORD 0xFFFFFFFFu
#define V4_MESH_WINDOW 4096u /* v3's ROLLING_WINDOW_SIZE */
#define V4_MESH_WINDOW_WARM 1024u /* executions before the window is warm, as v3 */
#define V4_MESH_ADAPTIVE_CHECK 512u /* v3's ADAPTIVE_CHECK_FREQUENCY: executions between diversity checks */
@@ -89,6 +113,13 @@ typedef struct {
uint32_t adaptive_check_accumulator;
uint8_t is_warm;
uint8_t adaptive_pending; /* the heartbeat should check the window's diversity */
/* pipelining (Loop #4): the word called before, L8's gate, and the node's counts */
uint32_t prev_id; /* V4_MESH_NO_WORD: none yet */
uint8_t l4_pipelining;
uint64_t prefetch_attempts, prefetch_hits;
/* the hot-words cache (the node's (HOT), v4/image.h): L8's gate (v3's L1_heat_tracking), and its counts */
uint8_t l1_cache;
uint64_t hot_promotions, hot_evictions;
} v4_mesh_physics;
typedef struct v4_mesh v4_mesh;
@@ -126,6 +157,9 @@ struct v4_mesh {
unsigned char *is_word;
v4_mesh_physics *physics; /* `capacity` of them: each node's (#189, Phase 7) */
uint64_t (*ticks)(void); /* the heartbeat's tick count, read at each call (0: v4_mesh_set_tick's) */
/* memory for a word's transition counters, zeroed, and given back (pipelining; may be 0: none recorded) */
void *(*alloc)(void *self, unsigned long bytes);
void (*release)(void *self, void *p);
/* a word has gone, its ID free for the next: the owner forgets what it keeps by it (may be 0) */
void (*forgot)(void *self, unsigned place, v4_cell id);
void *self;
+140 -4
View File
@@ -148,17 +148,32 @@ static int is_word(unsigned place, v4_cell a)
return a > 4 && a < (v4_cell)V4_NODE_WORDS && (bits_of(place)[a >> 3] >> (a & 7)) & 1u;
}
/* a record emptied, its transition counters given back */
static void record_clear(v4_mesh_account *a)
{
if (a->transitions && the->release) the->release(the->self, a->transitions);
a->transitions = 0;
a->calls = 0; a->since = 0; a->xt = 0;
a->heat = 0; a->last_decay_tick = 0; a->temperature_q8 = 0; a->state_flags = 0;
a->total_transitions = 0; a->most_likely_count = 0; a->most_likely_next = 0;
a->prefetch_attempts = 0; a->prefetch_hits = 0;
}
static void hot_evict(unsigned place, v4_cell w, v4_cell xt);
/* an entry has been made, its code at xt: it has a record from now */
static void word_made(unsigned place, const v4_node *n, v4_cell xt)
{
v4_cell id;
if (!the->accounts || xt <= 4 || xt >= (v4_cell)V4_NODE_WORDS) return;
bits_of(place)[xt >> 3] |= (unsigned char)(1u << (xt & 7));
hot_evict(place, 0, xt); /* a word of its name cached before: out, as v3's evict_name */
id = n->mem[xt - 4];
if (id >= 0 && id < (v4_cell)V4_MESH_WORD_IDS) {
v4_mesh_account *a = &accounts_of(place)[id];
a->calls = 0; a->since = 0; a->xt = xt;
a->heat = 0; a->last_decay_tick = 0; a->temperature_q8 = 0; a->state_flags = 0; /* as v3's physics_metadata_init */
record_clear(a);
a->xt = xt; /* as v3's physics_metadata_init: all else 0 */
}
}
@@ -168,6 +183,7 @@ static void words_gone(unsigned place, const v4_node *n, v4_cell w)
v4_cell a;
if (!the->accounts) return;
if (w < 5) w = 5;
hot_evict(place, w, 0); /* and out of the cache */
for (a = w; a < (v4_cell)V4_NODE_WORDS; a++) {
v4_cell id;
if (!is_word(place, a)) continue;
@@ -175,8 +191,7 @@ static void words_gone(unsigned place, const v4_node *n, v4_cell w)
id = n->mem[a - 4];
if (id >= 0 && id < (v4_cell)V4_MESH_WORD_IDS && accounts_of(place)[id].xt == a) {
v4_mesh_account *r = &accounts_of(place)[id];
r->calls = 0; r->since = 0; r->xt = 0;
r->heat = 0; r->last_decay_tick = 0; r->temperature_q8 = 0; r->state_flags = 0;
record_clear(r);
if (the->forgot) the->forgot(the->self, place, id);
}
}
@@ -217,6 +232,114 @@ static void window_record(v4_mesh_physics *p, uint32_t id)
}
}
/* ---- the hot-words cache: the node's (HOT), which (LOOKUP) tries first (dict.f18) ---- */
/* do the entries at xt1 and xt2 have the same name? (dict.f18: the name's word address at xt-2) */
static int same_name(const v4_node *n, v4_cell xt1, v4_cell xt2)
{
v4_cell a = n->mem[xt1 - 2], b = n->mem[xt2 - 2];
unsigned len = (unsigned)((v4_ucell)n->mem[a] & 31u), cells = (len + 4u) / 4u, i;
if (((v4_ucell)n->mem[b] & 31u) != len) return 0;
for (i = 0; i < cells; i++) if (n->mem[a + (v4_cell)i] != n->mem[b + (v4_cell)i]) return 0;
return 1;
}
/* take out slot i, the rest moved down (v3's hotwords_cache_remove_at) */
static void hot_remove(v4_node *n, v4_cell hot, v4_cell i)
{
v4_cell count = n->mem[hot], j;
for (j = i; j + 1 < count; j++) n->mem[hot + 3 + j] = n->mem[hot + 3 + j + 1];
count--;
n->mem[hot] = count;
n->mem[hot + 3 + count] = 0;
if (n->mem[hot + 1] >= count) n->mem[hot + 1] = 0;
}
/* entries gone (from word address w up), or one whose name is now another word's (xt's): out of the cache,
* so that what (LOOKUP) finds there is always what the dictionary would give */
static void hot_evict(unsigned place, v4_cell w, v4_cell xt)
{
v4_fabric_node *x = v4_fabric_node_at(&the->f, place);
v4_cell hot = the->im->hot, i;
if (!x || hot <= 0) return;
for (i = x->n.mem[hot]; i-- > 0; ) {
v4_cell e = x->n.mem[hot + 3 + i];
if (e == 0 || (w > 0 && e >= w) || (xt > 0 && e != xt && same_name(&x->n, e, xt))) hot_remove(&x->n, hot, i);
}
}
#if ENABLE_HOTWORDS_CACHE
/* v3's hotwords_cache_promote: full, the next slot in turn is taken; else, if not there, it goes at the end */
static void promote(unsigned place, v4_cell xt)
{
v4_fabric_node *x = v4_fabric_node_at(&the->f, place);
v4_mesh_physics *p = &the->physics[place];
v4_cell hot = the->im->hot, count, i;
if (!x || hot <= 0) return;
count = x->n.mem[hot];
if (count >= V4_IMAGE_HOT_SIZE) {
x->n.mem[hot + 3 + x->n.mem[hot + 1]] = xt;
x->n.mem[hot + 1] = (x->n.mem[hot + 1] + 1) % V4_IMAGE_HOT_SIZE;
p->hot_evictions++;
return;
}
for (i = 0; i < count; i++) if (x->n.mem[hot + 3 + i] == xt) return;
x->n.mem[hot + 3 + count] = xt;
x->n.mem[hot] = count + 1;
p->hot_promotions++;
}
static int hot_has(unsigned place, v4_cell xt)
{
v4_fabric_node *x = v4_fabric_node_at(&the->f, place);
v4_cell hot = the->im->hot, i;
if (!x || hot <= 0) return 0;
for (i = 0; i < x->n.mem[hot]; i++) if (x->n.mem[hot + 3 + i] == xt) return 1;
return 0;
}
#endif
#if ENABLE_PIPELINING
/* Loop #4 (v3's physics_pre_execute, the pipelining half): was this word the one the word before it
* predicted? -- then the transition, the likeliest next word, and speculation on it */
static void pipeline(unsigned place, v4_mesh_physics *p, uint32_t id)
{
v4_mesh_account *prev, *spec;
uint32_t next;
if (p->prev_id >= V4_MESH_WORD_IDS) return;
prev = &accounts_of(place)[p->prev_id];
if (!prev->xt) return;
if (prev->most_likely_next == id && prev->prefetch_attempts > 0) { prev->prefetch_hits++; p->prefetch_hits++; }
if (!prev->transitions) { /* on its first transition, as v3 */
if (!the->alloc) return;
prev->transitions = (uint64_t *)the->alloc(the->self, (unsigned long)V4_MESH_WORD_IDS * sizeof(uint64_t));
if (!prev->transitions) return;
}
prev->transitions[id]++;
prev->total_transitions++;
/* the likeliest next: v3 scans every ID for the highest count, lowest ID on a tie; the same, kept as
* each count rises (v3 compares Q48.16 probabilities, which can round two counts equal: then it picks
* the lower ID where this keeps the higher count) */
if (prev->transitions[id] > prev->most_likely_count
|| (prev->transitions[id] == prev->most_likely_count && id < prev->most_likely_next)) {
prev->most_likely_count = prev->transitions[id];
prev->most_likely_next = id;
}
next = prev->most_likely_next;
if (prev->total_transitions < MIN_SAMPLES_FOR_SPECULATION) return;
if ((int64_t)((prev->transitions[next] << 16) / prev->total_transitions) < (int64_t)SPECULATION_THRESHOLD_Q48) return;
spec = &accounts_of(place)[next];
if (!spec->xt) return;
#if ENABLE_HOTWORDS_CACHE
if (p->l1_cache) {
spec->heat = V4_MESH_HOT_HEAT + 1u; /* as v3 */
promote(place, spec->xt);
prev->prefetch_attempts++;
p->prefetch_attempts++;
}
#endif
}
#endif
/* at `call` (exec.h), on a word -- any other target, nothing -- in v3's order (physics_pre_execute,
* V3-PARITY.md 2.1; Captain Bob, 2026-11-13: "Every call, as v3"): decay by the ticks since, heat +1,
* the count the Stadium is charged by, the temperature, the rolling window */
@@ -237,6 +360,16 @@ static void on_call(void *ctx, v4_node *n, v4_cell target)
a->heat++;
a->temperature_q8 = temperature(a->heat, a->temperature_q8);
if (p->l2_window) window_record(p, (uint32_t)id);
#if ENABLE_PIPELINING
if (p->l4_pipelining) pipeline(place, p, (uint32_t)id);
#endif
#if ENABLE_HOTWORDS_CACHE
/* hot enough, and not cached: in (v3 promotes when a lookup finds such a word in the dictionary; on
* v4 the heat is the kernel's, so it is done at the call where the word is hot -- the same words,
* at their call rather than their lookup) */
if (p->l1_cache && a->heat > V4_MESH_HOT_HEAT && !hot_has(place, target)) promote(place, target);
#endif
p->prev_id = (uint32_t)id;
}
a->calls++;
a->since++;
@@ -274,7 +407,7 @@ static void words_attach(unsigned place, v4_fabric_node *x)
unsigned k;
if (!the->accounts || !the->is_word) return;
for (k = 0; k < WORD_BITS_BYTES; k++) bits_of(place)[k] = 0;
for (k = 0; k < V4_MESH_WORD_IDS; k++) { accounts_of(place)[k].calls = 0; accounts_of(place)[k].since = 0; accounts_of(place)[k].xt = 0; }
for (k = 0; k < V4_MESH_WORD_IDS; k++) record_clear(&accounts_of(place)[k]);
for (xt = x->n.mem[im->latest], k = 0; xt > 4 && xt < (v4_cell)V4_NODE_WORDS && k < V4_NODE_WORDS; xt = x->n.mem[xt - 1], k++) {
word_made(place, &x->n, xt);
if (!told && named(&x->n, xt, "(TELL-DEFINED)")) told = xt;
@@ -295,6 +428,9 @@ static void words_attach(unsigned place, v4_fabric_node *x)
p->l3_decay = 1;
p->l2_window = 1; /* v3: "No config yet: recorded" */
p->effective_window_size = V4_MESH_WINDOW;
p->prev_id = V4_MESH_NO_WORD;
p->l4_pipelining = ENABLE_PIPELINING ? 1 : 0; /* the build's ceiling; L8 turns it within it */
p->l1_cache = ENABLE_HOTWORDS_CACHE ? 1 : 0;
}
x->es.on_call = on_call;
x->es.on_call_ctx = &the->kernel[place];
+4 -1
View File
@@ -63,7 +63,9 @@
#define HBUF_W (SVARS - 16) /* the hold buffer: 16 cells = 64 bytes */
#define HEND ((HBUF_W + 16) * 4)
#define XVARS (HBUF_W - 4) /* (X): words.f18, 4 cells */
#define QBASE (XVARS - 40) /* the Q words' and DUMP's scratch cells, 39 in all */
#define HOTVARS (XVARS - 35) /* (HOT): the hot-words cache (#189 Phase 7): its count, the kernel's next slot to take when it is full, the lookup's index, then 32 xts */
#define HOT_SIZE 32
#define QBASE (HOTVARS - 40) /* the Q words' and DUMP's scratch cells, 39 in all */
#define BVARS (QBASE - 16) /* blocks.f18: 6 cells not used now, then SCR BLK (SRC) (SRC-HOOK) */
#define SCR (BVARS + 6)
#define BLK (BVARS + 7)
@@ -286,6 +288,7 @@ static unsigned host_branch_slots(void)
X("(S)", SVARS) \
X("(W)", WVARS) \
X("(X)", XVARS) \
X("(HOT)", HOTVARS) \
X("(BOOT)", BOOT_CELLS) \
X("(NEXT-ID)", NEXT_ID) \
X("(BOOT-ID)", BOOT_ID) \
+72 -1
View File
@@ -41,6 +41,26 @@ static v4_cell forgotten_id = -1;
static void forgot(void *self, unsigned place, v4_cell id) { (void)self; (void)place; forgotten++; forgotten_id = id; }
static v4_uheat_t charged[V4_MESH_WORD_IDS];
static void charge(void *ctx, v4_cell id, v4_uheat_t calls) { (void)ctx; charged[id] += calls; }
#define TABLES 1024u /* every word with a successor has one, as on the kernel's heap */
static uint64_t tables[TABLES][V4_MESH_WORD_IDS]; /* transition counters, given out in turn */
static int table_used[TABLES];
static void *alloc(void *self, unsigned long bytes)
{
unsigned k; (void)self;
if (bytes > sizeof tables[0]) return 0;
for (k = 0; k < TABLES; k++) if (!table_used[k]) { table_used[k] = 1; memset(tables[k], 0, sizeof tables[k]); return tables[k]; }
return 0;
}
static void release(void *self, void *p) { (void)self; table_used[(uint64_t (*)[V4_MESH_WORD_IDS])p - tables] = 0; }
#if ENABLE_HOTWORDS_CACHE
static int hot_holds(v4_cell xt)
{
v4_fabric_node *h = v4_mesh_node(&ms, V4_MESH_HERA);
v4_cell i;
for (i = 0; i < h->n.mem[im.hot]; i++) if (h->n.mem[im.hot + 3 + i] == xt) return 1;
return 0;
}
#endif
static int no_room; /* the owner has no room to give */
static v4_fabric_node *room(void *self) { unsigned k; (void)self; if (no_room) return 0; for (k = 0; k < PLACES; k++) if (!used[k]) { used[k] = 1; roomed++; return &pool[k]; } return 0; }
@@ -90,7 +110,7 @@ int main(void)
ms.out = out; ms.key = key; ms.words = words; ms.word_count = 1; ms.serve = serve;
ms.capsules = capsules; ms.capsule_count = 2; ms.im = &im; ms.self = 0;
ms.accounts = accounts; ms.is_word = is_word; ms.forgot = forgot;
ms.physics = physics;
ms.physics = physics; ms.alloc = alloc; ms.release = release;
/* ---- the boot ---- */
CHECK(v4_mesh_boot(&ms) == 1, "the boot: %s", said);
@@ -285,6 +305,57 @@ int main(void)
}
}
#if ENABLE_PIPELINING && ENABLE_HOTWORDS_CACHE
/* ---- pipelining and the hot-words cache, on (built off as v3; Captain Bob, 2026-11-13) ---- */
{
v4_cell pa, pb, pb2, hw;
const v4_mesh_account *aa;
v4_mesh_physics *p = &ms.physics[0];
r = line(V4_MESH_HERA, ": PA 1 ; : PB 2 ; ' PA .", &how);
pa = (v4_cell)strtol(r, 0, 10);
r = line(V4_MESH_HERA, "' PB .", &how); pb = (v4_cell)strtol(r, 0, 10);
r = line(V4_MESH_HERA, ": PAT PA PB PA PB PA PB PA PB PA PB PA PB PA PB PA PB PA PB PA PB PA PB PA PB ; PAT", &how);
aa = v4_mesh_account_at(&ms, 0, pa);
CHECK(how == V4_TEXT_COMPLETED && aa && aa->total_transitions == 12 && aa->transitions
&& aa->most_likely_next == (uint32_t)v4_mesh_node(&ms, V4_MESH_HERA)->n.mem[pb - 4],
"PA's transitions: 12, every one to PB, its likeliest next (%lu)", aa ? (unsigned long)aa->total_transitions : 0ul);
CHECK(aa && aa->prefetch_attempts > 0 && hot_holds(pb) && p->hot_promotions > 0,
"after 10, speculation on PB: it is put in the node's cache (%lu attempts)", aa ? (unsigned long)aa->prefetch_attempts : 0ul);
CHECK(aa && aa->prefetch_hits > 0 && p->prefetch_hits > 0, "and PB coming next counts as hits (%lu)", aa ? (unsigned long)aa->prefetch_hits : 0ul);
r = line(V4_MESH_HERA, "PB .", &how);
CHECK(strcmp(r, "2 ") == 0, "a cached word is found by its name: \"%s\"", r);
r = line(V4_MESH_HERA, ": PB 3 ; PB .", &how);
CHECK(strcmp(r, "3 ") == 0 && !hot_holds(pb), "PB defined again: the old one is out of the cache, the new one found: \"%s\"", r);
r = line(V4_MESH_HERA, "' PB .", &how); pb2 = (v4_cell)strtol(r, 0, 10);
/* v3's promote, when the cache is full, takes the next slot in turn without looking for the word
* already there: a few hot words then churn through all 32 slots (reproduced, as v3; reported). So
* the heat promotion is seen on a cache emptied first. */
CHECK(v4_mesh_node(&ms, V4_MESH_HERA)->n.mem[im.hot] == V4_IMAGE_HOT_SIZE && p->hot_evictions > 0,
"by now the cache is full, and taking slots in turn, as v3's");
v4_mesh_node(&ms, V4_MESH_HERA)->n.mem[im.hot] = 0;
v4_mesh_node(&ms, V4_MESH_HERA)->n.mem[im.hot + 1] = 0;
r = line(V4_MESH_HERA, ": HW ; : LH 60 0 DO HW LOOP ; LH ' HW .", &how);
hw = (v4_cell)strtol(r, 0, 10);
CHECK(hot_holds(hw), "a word called past heat 50 goes into the cache");
r = line(V4_MESH_HERA, "FORGET PA", &how);
CHECK(!hot_holds(hw) && !hot_holds(pb2) && v4_mesh_account_at(&ms, 0, pa) == 0,
"FORGET takes everything from PA on out of the cache, and PA's counters back");
r = line(V4_MESH_HERA, "3 4 + .", &how);
CHECK(strcmp(r, "7 ") == 0, "and lookups go on as before: \"%s\"", r);
}
#else
{
const v4_mesh_account *aa;
v4_cell pa;
r = line(V4_MESH_HERA, ": PA 1 ; : PB 2 ; : PAT PA PB PA PB PA PB PA PB PA PB PA PB PA PB ; PAT ' PA .", &how);
pa = (v4_cell)strtol(r, 0, 10);
aa = v4_mesh_account_at(&ms, 0, pa);
CHECK(aa && aa->total_transitions == 0 && !aa->transitions && v4_mesh_node(&ms, V4_MESH_HERA)->n.mem[im.hot] == 0,
"built off, as v3: no transitions recorded, nothing in the cache");
r = line(V4_MESH_HERA, "FORGET PA", &how);
}
#endif
printf(" %d checks, %d failures\n", checks, failures);
return failures ? 1 : 0;
}
+2 -2
View File
@@ -53,10 +53,10 @@ static int image_description(void)
printf("#include \"v4/image.h\"\n\nconst v4_image v4_map_image = {\n");
printf(" %uu, %uu, %uu, %uu,\n", (unsigned)V4_CELL_BITS, (unsigned)V4_NODE_WORDS, (unsigned)V4_DATA_RING, (unsigned)V4_RET_RING);
printf(" -1,\n -1, %ld,\n -1, -1,\n", (long)PORT);
printf(" %ld, %ld, %ld,\n %ld, %ld,\n %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld,\n",
printf(" %ld, %ld, %ld,\n %ld, %ld,\n %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld, %ld,\n %ld,\n %ld,\n",
(long)CONSOLE_TX, (long)CONSOLE_RX, (long)CONSOLE_ST, (long)DSTACK_REG, (long)RSTACK_REG,
(long)NODE_ERROR, (long)HEAT_REG, (long)ME, (long)DP, (long)LATEST, (long)BOOT_CELLS, (long)FENCE,
(long)NEXT_ID, (long)BOOT_ID, (long)WORD_DEFINED, (long)WORD_FORGOTTEN, (long)BUF0_W);
(long)NEXT_ID, (long)BOOT_ID, (long)WORD_DEFINED, (long)WORD_FORGOTTEN, (long)HOTVARS, (long)BUF0_W);
printf(" %ld, %ld, %ld\n};\n", (long)ROM_IMG, (long)ROM_FAILED, (long)ROM_ERRLINE);
return 0;
}