Stage S-1c: WORD's buffer is per VM; input buffers allotted at VM init, below all compiled code (FABRIC-3.9.0 §8.1)

- The static word_scratch_addr shared by every VM is gone: WORD builds in
  vm->word_buf (257 bytes: count + 255 + NUL). A token over 255 chars is
  refused whole (delimiter restored, >IN unconsumed, vm->error), logged at
  LOG_ERROR and counted -- no more silent cut to 62 chars.
- Pre-existing corruption found by the probe: TIB/>IN/SPAN (and WORD's
  scratch) were allotted lazily at HERE; POST's restore_dict_state(),
  FORGET and COLD rolled HERE below them and compiled the dictionary over
  them (Hera's TIB lay under ACL-RECHECK's body; QUERY/WORD wrote into
  code). Per Captain Bob, all four are now allotted first in both
  vm_init()s (offsets 0-1305); COLD's clamp never goes below them.

Probe (reverted), identical on amd64/aarch64/riscv64: 0 colon bodies inside
the buffers; per-VM isolation; 62/100/255-char tokens full length; 300
refused and counted; interpreter sane after full-width TIB writes.
3-ISA acceptance passes; M7.1a hash and dict_hash set unchanged (here
0xd8 -> 0x5f8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-24 16:21:25 -04:00
co-authored by Claude Opus 5.5
parent a2e3356b2e
commit 316be9fc4b
25 changed files with 73545 additions and 19 deletions
+76 -1
View File
@@ -1,6 +1,6 @@
# FABRIC-3.9.8.md — Observers and participants: execution plan and log
**Status: IN PROGRESS.** Done: O-A, the stack-convention fix, S-1a and S-1b (§2). The design and
**Status: IN PROGRESS.** Done: O-A, the stack-convention fix, S-1a, S-1b and S-1c (§2). The design and
its rulings (O-1 to O-4, the discard, and §7/§8's M- and S- rulings) are in `FABRIC-3.9.0.md`.
Each stage:
@@ -338,3 +338,78 @@ Evidence is named **before** each stage runs.
callers.
- No word-id field narrower than 32 bits.
- Nothing else found.
- [x] **Stage S-1c — `WORD`'s buffer is per VM, and the input buffers sit below all compiled
code (`FABRIC-3.9.0.md` §8.1, class C).** 2026-09-24. Captain Bob: *"go ahead with S-1c"*,
then, on the defect below, *"Allot at VM init"*.
**The defect as scoped.** `WORD` built its counted string in one `static vaddr_t
word_scratch_addr` shared by every VM:
- It was `vm_allot`'d once, in whichever VM called first, at wherever that VM's `HERE` stood.
- A second VM got an address meant for someone else's space.
- Tokens were silently cut to 62 characters (a 64-byte buffer).
**The deeper defect, found by the first probe.** The input buffers were allotted lazily by
`vm_input_ensure()` at the current `HERE`: the TIB (`INPUT_BUFFER_SIZE`), `>IN`, `SPAN`, and
`WORD`'s scratch.
- Three things later roll `HERE` back below them and compile the dictionary over them: POST's
test harness (`restore_dict_state()`), `FORGET`, and `COLD` (which clamps `HERE` to 1024).
- On Hera, POST allotted the TIB at offset 120 and then restored `HERE`. **`ACL-RECHECK`'s
colon body lived at offset 824, inside the TIB.**
- The first S-1c probe wrote the TIB, as `QUERY` does, which zeroed that body. Hera hit
`execute_colon_word: NULL cell in 'ACL-RECHECK'` and never reached her prompt
(`logs/20260924-151033/amd64/`; layout diagnostic `-152047/amd64/`).
- So before this stage, `QUERY` wrote into Hera's compiled code, and so did `WORD` through
the old static scratch. Nothing in a normal boot calls either afterwards, which is why
acceptance never saw it. The first draft of the per-VM buffer, allotted the same lazy way,
inherited the defect.
**The change:**
- **`vm.h`:**
- `VM.word_buf`, one per VM.
- `WORD_BUFFER_SIZE 257`: a count byte, 255 characters (the most a counted string can
hold, a format limit, not a chosen bound), and a NUL.
- **`vm_api.c`:** `vm_input_ensure()` also allots `word_buf`, after `SPAN`, so TIB/`>IN`/
`SPAN` keep their relative layout.
- **Both `vm_init()`s** (`src/vm_bootstrap.c`, `src/starkernel/vm/vm_bootstrap.c`) call
`vm_input_ensure()` first, before POST, capsules or any definition.
- The buffers take offsets 0–1305 in every VM.
- Every later `HERE` rollback (test restore, `FORGET`) lands above them.
- If the allotment fails, `vm_init` fails with an error.
- **`COLD`** (`system_words.c`) still clamps `HERE` to 1024, but never below the end of the
input buffers.
- **`string_words.c`:** the static is gone. `WORD` builds into `vm->word_buf`.
- A token over 255 characters is **refused whole**: the delimiter goes back on the stack,
`>IN` is not consumed, and `vm->error` is set.
- The refusal is logged at `LOG_ERROR` ("WORD: token over 255 chars refused") and counted
(`string_word_word_refusals()`).
**Evidence** (probe, reverted; byte-identical on all three ISAs, 37 lines;
`logs/20260924-152935/amd64/`, `-153409/aarch64/`, `-153842/riscv64/`):
| Check | Result |
|---|---|
| Layout, Hera and Hestia | TIB offset 0, `>IN` 1032, `SPAN` 1040, `word_buf` 1048, end 1305. Colon bodies inside the buffers: **0** (Hera 26 checked, Hestia 179). |
| Isolation | Hera's `WORD` returns her own `word_buf`, in her own memory. Hestia's returns hers, in hers, and the two are distinct. Hera's `HERAWORD` is intact after Hestia parses `HESTIAWORD`. |
| Length | 62, 100 and 255 characters come back at full length with the correct last character. It used to be 62 at most. |
| 300-char token | Refused, `vm->error=1`, refusals 1, the ERROR line logged, the delimiter restored, `>IN` 0. |
| After the probe | The probe wrote the TIB full-width, and `1 2 + .` still prints `3`. Hera reaches `[zuse@Hera] ok>`. |
**Acceptance** on the clean tree (`logs/20260924-154256/amd64/`, `-155542/aarch64/`,
`-160845/riscv64/`):
- Boot M7.1a `word_count=531`, hash `0x572daac83b797332`, unchanged. Only its `here` moves,
from `0xd8` to `0x5f8`, the 1,305 buffer bytes now allotted first.
- The `dict_hash` set is unchanged (Hera `0xda9742ce9c3376a2`, Hestia `0x413e7d086f342d8b`,
Artemis `0xdfd68316d5f5ba85`). The canonical hash walks word ids and body cells, and no
colon body carries an absolute address that the shift would change.
- The O-A checks pass: G interleave 40/40/48, `HHH` precedence, and a typed `1 2 + .` gives
`3` mid-stream.
- Mint → WIREBIND → `USE` → `3`. Idle log growth 0. No faults. QEMU exits cleanly.
- Clean kernel builds on all three ISAs, plus the hosted build.
**Found, reported, not fixed:**
1. **`COLD` rewinds `HERE` to 1024 without rewinding `latest`.** Every word defined past 1024
stays linked while new definitions are compiled over its body. S-1c only keeps the clamp
off the input buffers.
2. **`PAD` overlaps the block I/O window.** It returns `VM_MEMORY_SIZE - 512`, which is inside
the top block slot (`BLK_VM_WINDOW_BASE` = `VM_MEMORY_SIZE - 4096`), so `PAD` and the
4th `BLOCK` buffer share bytes.
+8
View File
@@ -155,6 +155,9 @@ static inline cell_t CELL(vaddr_t a) { return (cell_t)(int64_t)a; }
#define WORD_ID_INVALID UINT32_MAX
#define VM_MEMORY_SIZE (5 * 1024 * 1024) /* 5 MB total VM memory */
#define INPUT_BUFFER_SIZE 1025
/* WORD's buffer: a count byte, up to 255 chars (the most a counted string can
* hold -- a format limit, not a chosen bound), and a trailing NUL. */
#define WORD_BUFFER_SIZE 257
#define WORD_NAME_MAX 31
#define COMPILE_BUFFER_SIZE 1024
@@ -562,6 +565,11 @@ typedef struct VM
size_t tib_cap; /* capacity in bytes */
cell_t* in_var; /* >IN (legacy; will migrate to VM addr) */
cell_t* span_var; /* SPAN (legacy; will migrate to VM addr) */
/* FABRIC-3.9.0.md §8.1 (Stage S-1c): WORD's counted-string buffer, one per
* VM, allotted with the TIB (vm_input_ensure()). Was a single static
* shared by every VM, allotted in the first caller's space wherever HERE
* happened to be, 64 bytes, silently truncating tokens to 62 chars. */
unsigned char* word_buf;
/* Block system (VM-backed variables / addresses) */
vaddr_t scr_addr;
Binary file not shown.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+15
View File
@@ -52,6 +52,7 @@
#include "../include/rolling_window_of_truth.h"
#include "../include/dictionary_heat_optimization.h"
#include "../include/ssm_jacquard.h"
#include "../include/vm_api.h" /* vm_input_ensure() -- S-1c */
#include "word_source/include/vocabulary_words.h"
#include "vm_internal.h"
#ifdef __STARKERNEL__
@@ -209,6 +210,20 @@ void vm_init_with_host(VM* vm, const VMHostServices *host)
log_message(LOG_DEBUG, "vm_init: memory=%p here=%zu", (void*)vm->memory, vm->here);
/* FABRIC-3.9.8 Stage S-1c (Captain Bob, 2026-09-24: "allot at VM init"):
* the input buffers -- TIB, >IN, SPAN and WORD's buffer -- are allotted
* first, at HERE = 0, before POST, capsules or any definition. They used
* to be allotted lazily wherever HERE stood; POST's restore_dict_state(),
* FORGET and COLD then rolled HERE back below them and compiled the
* dictionary over them (Hera's TIB sat under ACL-RECHECK's body). No
* HERE rollback can reach below them now. */
if (vm_input_ensure(vm) != 0)
{
log_message(LOG_ERROR, "vm_init: input buffers could not be allotted");
vm->error = 1;
return;
}
vm_bootstrap_scr(vm);
/* STATE (0=interpret, -1=compile) */
+12
View File
@@ -142,6 +142,18 @@ int vm_input_ensure(VM *vm) {
}
*vm->span_var = 0;
}
if (!vm->word_buf) {
/* S-1c: WORD's buffer, per VM, in this VM's own space, placed with
* the other input buffers -- never mid-dictionary, never inside a
* definition being compiled. After SPAN so TIB/>IN/SPAN keep their
* addresses. */
vm->word_buf = (unsigned char *) vm_allot(vm, WORD_BUFFER_SIZE);
if (!vm->word_buf) {
vm->error = 1;
return 1;
}
memset(vm->word_buf, 0, WORD_BUFFER_SIZE);
}
return 0;
}
+12
View File
@@ -62,6 +62,7 @@
#include "../include/dictionary_heat_optimization.h"
#include "../include/ssm_jacquard.h"
#include "../include/platform_alloc.h"
#include "../include/vm_api.h" /* vm_input_ensure() -- S-1c */
#include "word_source/include/vocabulary_words.h"
#ifdef __STARKERNEL__
@@ -245,6 +246,17 @@ void vm_init(VM* vm)
memset(p, 0, 64);
}
/* FABRIC-3.9.8 Stage S-1c: the input buffers (TIB, >IN, SPAN, WORD's
* buffer) are allotted here, before any definition, so no HERE rollback
* (test harness restore, FORGET, COLD) can reach below them. Same as the
* kernel's vm_init(). */
if (vm_input_ensure(vm) != 0)
{
vm->error = 1;
log_message(LOG_ERROR, "vm_init: input buffers could not be allotted");
return;
}
vm_bootstrap_scr(vm);
vm->mode = MODE_INTERPRET;
+3
View File
@@ -149,4 +149,7 @@
*/
void register_string_words(VM * vm);
/* S-1c: WORD tokens refused for exceeding a counted string's 255 chars. */
uint32_t string_word_word_refusals(void);
#endif /* STRING_WORDS_H */
+18 -17
View File
@@ -75,9 +75,12 @@ static inline int ensure_input(VM *vm) {
return err;
}
/* Static VM-backed scratch buffer for WORD (allocated on first use) */
#define WORD_SCRATCH_CAP 64
static vaddr_t word_scratch_addr = 0;
/* S-1c: WORD's buffer is per VM now (vm->word_buf, vm_input_ensure()). A
* token too long for a counted string (> 255 chars) is refused, logged and
* counted here -- never silently truncated. */
static uint32_t g_word_too_long;
uint32_t string_word_word_refusals(void) { return g_word_too_long; }
/* Helper function to convert string to number (base-10 only here) */
static int convert_string_to_number(const char *str, size_t len, cell_t *result) {
@@ -248,33 +251,31 @@ void string_word_word(VM *vm) {
while ((size_t) ip < tib_len && tib[ip] != (unsigned char) delimiter) ip++;
size_t end = (size_t) ip;
size_t word_len = end - start;
if (word_len > WORD_SCRATCH_CAP - 2) word_len = WORD_SCRATCH_CAP - 2;
/* Allocate scratch once */
if (word_scratch_addr == 0) {
void *p = vm_allot(vm, WORD_SCRATCH_CAP);
if (!p) {
vm->error = 1;
return;
}
word_scratch_addr = vaddr_from_ptr(vm, p);
if (word_len > WORD_BUFFER_SIZE - 2) {
/* Refused whole: the delimiter goes back on the stack and >IN is
* left where it was, so nothing is consumed. */
g_word_too_long++;
log_message(LOG_ERROR, "WORD: token over 255 chars refused");
vm_push(vm, c);
vm->error = 1;
return;
}
/* Build counted string at scratch */
uint8_t *wb = vm_ptr(vm, word_scratch_addr);
/* Build the counted string in this VM's own WORD buffer */
uint8_t *wb = vm->word_buf;
if (!wb) {
vm->error = 1;
return;
}
wb[0] = (uint8_t) word_len;
if (word_len) memcpy(&wb[1], &tib[start], word_len);
if (word_len + 1 < WORD_SCRATCH_CAP) wb[word_len + 1] = '\0'; /* safety */
wb[word_len + 1] = '\0'; /* safety */
/* Update >IN to char after the delimiter (if any) */
while ((size_t) ip < tib_len && tib[ip] == (unsigned char) delimiter) ip++;
*inp = ip;
vm_push(vm, CELL(word_scratch_addr));
vm_push(vm, CELL(vaddr_from_ptr(vm, wb)));
}
/* >IN ( -- addr ) Input stream pointer cell (VM address) */
+10 -1
View File
@@ -90,7 +90,16 @@ static void reset_vm_state(VM *vm, int cold_start) {
if (cold_start) {
/* In a complete system, this would restore a boot image.
For now we keep a minimal fence by not crushing the base dictionary. */
if (vm->here > 1024) vm->here = 1024;
if (vm->here > 1024) {
/* S-1c: never below the input buffers allotted at vm_init()
* (TIB/>IN/SPAN/WORD buffer end past 1024). */
size_t floor = 1024;
if (vm->word_buf) {
size_t buf_end = (size_t)(vm->word_buf - vm->memory) + WORD_BUFFER_SIZE;
if (buf_end > floor) floor = buf_end;
}
if (vm->here > floor) vm->here = floor;
}
}
system_running = 1;