Captain Bob: close the messaging gap first; the arena must be dynamic;
audit every fixed-size structure ("no arrays ... it should all be fairly
dynamic"); failures must be handled properly (never silent, never
partial, logged + counted + refused, parity marked INCOMPLETE). All
rulings as proposed; S-2 (Stadium cell pool) left open -- it was a
question, not a proposal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
478 lines
30 KiB
Markdown
478 lines
30 KiB
Markdown
# FABRIC-3.9.0.md — Observers and participants: the Tripod steps outside the compudynamics
|
||
|
||
**Status: DESIGN, for Captain Bob's review. Nothing here authorizes an edit.** The execution
|
||
plan and log are in `FABRIC-3.9.8.md`.
|
||
|
||
**Provenance.** Opened 2026-09-24. This design follows `FABRIC-3.8.md`, whose Stages A–I are
|
||
all done. The per-ISA DoE relaunch on the rebuilt driver (`FABRIC-3.8.md` §6) was stopped once a
|
||
progress check showed:
|
||
- Hera's `MU-WAIT-ALL` busy loop took an equal share of the CPU as workers finished;
|
||
- it wrote 43% of all heartbeat rows so far (Hera 75,117 against `mu0`'s 75,114).
|
||
|
||
Two fixes were tried and set aside:
|
||
- **A capsule-level `100 WAIT`,** reverted. Captain Bob: *"not liking this much … redistribute
|
||
work when hera is the major consumer."* The application must not steer its own scheduling.
|
||
- **A kernel-side absolute-heat-share charge.** Built and tested (a cfg-0 trial took about 150 s
|
||
against about 210 s), but never committed. It is superseded by this design (§5).
|
||
|
||
Captain Bob, 2026-09-24:
|
||
- *"Hera, Hestia, and Artemis no longer participate in the calculations. However, they observe
|
||
them and act on them."*
|
||
- *"let's do it."*
|
||
|
||
---
|
||
|
||
## 1. The idea
|
||
|
||
The fleet splits into two roles.
|
||
|
||
- **Participants** are every VM that does the work being done and measured: DoE workers,
|
||
WIREBIND user and console VMs, and anything `BIRTH`-ed. For these, FABRIC-3.8's compudynamics
|
||
stays exactly as built:
|
||
- per-beat charge;
|
||
- work heat, with floor on contact;
|
||
- coolest VM with work goes next;
|
||
- ties to the incumbent, then to the one waiting longest.
|
||
- **Observers** are Hera, Artemis and Hestia, the Tripod. They are not compared, not charged work
|
||
heat, and not counted in any floor. They **observe** the participants' physics and **act on
|
||
events**: mail, console input, device events.
|
||
|
||
**In thermodynamic terms,** the observers are the environment (the heat bath) and the
|
||
participants are the system under study.
|
||
- Hera already behaves like a reservoir: she starts holding all of `Q48_ONE`, as the root that
|
||
`vm_physics_retire()` returns heat to.
|
||
- Every problem since Stage G came from making the reservoir compete as if it were one more
|
||
particle:
|
||
- her endowment made her "hottest";
|
||
- her busy-wait took a fair share of the CPU and polluted the data;
|
||
- two fixes had to push her back out.
|
||
|
||
## 2. What changes for observers
|
||
|
||
**2.1 Who is an observer.** It's a structural fact, set by the kernel, not a name test in the
|
||
switcher.
|
||
- `capsule_birth_baby()` already computes `is_fleet_foundation` (Hera/Artemis/Hestia) for session
|
||
pinning. The observer role reuses that.
|
||
- Hera, born through `capsule_birth_mama()`, is marked where `kernel_main.c` registers her.
|
||
- Nothing a VM executes can set or clear the role.
|
||
|
||
**2.2 Not in the calculation.**
|
||
- `sk_vm_switch_decide()` never offers an observer as a heat candidate.
|
||
- `vm_physics_charge()` doesn't accrue observers' work heat.
|
||
- `sk_vm_switch_signal_note_work_gained()` leaves observers out of the floor.
|
||
- Proposed (ruling O-2): observers aren't charged **absolute** heat either. The reservoir doesn't
|
||
heat itself by running. Participants still pull absolute heat off it through their own charges,
|
||
and `retire` returns it, so fleet K stays conserved.
|
||
|
||
**2.3 Observers act on events.** An observer is runnable only when it has an event:
|
||
- **Mail:** its kernel-Hermes queue is non-empty (`MU-DONE`, `BLK-ATTACH`, `ELEVATE-REQUEST`, a
|
||
relayed console line).
|
||
- **Console input**, for Hera: `sk_console_key_available()`, as today.
|
||
- **A device event**, for Hera: xHCI and USB attach or detach. See §4, ruling O-4.
|
||
|
||
**2.4 When an observer runs (ruling O-1: the central question).** Proposed: **at the next clean
|
||
boundary, an observer with an event goes ahead of the participants.**
|
||
- The boundaries are the same owned checkpoints, trampoline points and idle point FABRIC-3.8
|
||
built.
|
||
- Among several observers with events, the one waiting longest (the existing unique wait stamps)
|
||
goes first, with no heat.
|
||
- Once it has no events left it parks, and the participants' physics resumes.
|
||
|
||
Why this isn't a scheduler:
|
||
- There is no run queue, no priority number, and no ordering computed or kept.
|
||
- It is a **role** (environment versus system), not a rank among peers.
|
||
- Observers never compete with participants on the same scale; they respond to what happens.
|
||
- The honest risk: an observer with a long event (a heavy command typed at Hera's own prompt)
|
||
holds the CPU until it finishes, because nothing displaces an observer. §4 lists this. Work
|
||
meant to share the CPU belongs in a participant. A user's commands already go to their own
|
||
WIREBIND user VM, which is a participant.
|
||
|
||
The alternative, observers run only when no participant has work, is pure but brings back
|
||
C-4 (α)'s starvation: under load, the console, USB and storage wait until every worker is idle.
|
||
|
||
**2.5 Hera's own mail is delivered at the idle point, not on the timer.**
|
||
- Today her queue is drained by `sk_repl_idle()`'s pump at the `SK_IDLE_BEAT_INTERVAL` cadence
|
||
(timer-paced; the last such delivery left, `FABRIC-3.8.md` Stage I).
|
||
- As an observer that acts on events, her mail must be acted on promptly, so her drain moves to
|
||
`sk_vm_idle_point()`.
|
||
- It keeps the readline `n > 0` mid-edit gate, so a drained payload never prints into a line
|
||
being typed.
|
||
|
||
## 3. What this lets the DoE become
|
||
|
||
**The driver becomes event-driven. Hera never waits in a loop.**
|
||
- `DOE` sets up the campaign and starts trial 1 (birth all, send all), then returns to the prompt.
|
||
- Each worker's final `SEND` delivers `MU-DONE` to Hera as mail.
|
||
- She acts on it at her idle point: increments the count, and on the last one ends the trial
|
||
(`KILL` all, marker, persist) and starts the next.
|
||
- After the last trial she prints `MU-CAMPAIGN: complete`.
|
||
- `MU-WAIT-ALL` is deleted. This is the §3.9 design's own "ordinary words" rule, applied: Hera
|
||
reacts to messages, which is what messages are for.
|
||
|
||
**The heartbeat rows then measure the system, not the environment.**
|
||
- An observer produces rows only while acting on an event.
|
||
- Proposed (ruling O-3): its rows stay in the CSV, tagged by `vm_name` as today, and the analysis
|
||
filters by role. This keeps the CSV layout unchanged. The alternative is to suppress them.
|
||
|
||
## 4. Known limits and open questions
|
||
|
||
- **O-4: device servicing.** xHCI event polling and USB attach/detach run in `sk_repl_idle()` at
|
||
the timer-paced `SK_IDLE_BEAT_INTERVAL`, and only when Hera reaches her idle point.
|
||
- Under participant load, Hera reaches it only when she has another event, so USB goes
|
||
unserviced. A USB keyboard is a device event that can only be *seen* by polling.
|
||
- Options: (a) treat "the device-poll interval elapsed" as a Hera event; that's I/O
|
||
housekeeping triggering a switch, not a scheduling decision, but it is timer-paced. (b) Move
|
||
xHCI polling to an interrupt path. (c) Accept it for now; the DoE doesn't need USB mid-run.
|
||
- Proposed: (c) for the relaunch, (b) as later work.
|
||
- **A long event on an observer holds the CPU** (§2.4). Accepted by the role design: heavy work
|
||
belongs in participants.
|
||
- **Artemis and Hestia become observers too.**
|
||
- Artemis's storage work (attach, block sync) and Hestia's framebuffer are responses to events,
|
||
which fits.
|
||
- If a future Tripod duty is compute-heavy, it would have to move to a participant.
|
||
- **Stage G's evidence** (two participants interleave) and C-b's walk-order property are
|
||
unaffected: observers aren't candidates.
|
||
- **The WIREBIND relay** stays participant-delivered: the user VM gets the console line as mail
|
||
and is switched to by heat among participants. Only Hera's own forwarding is observer work.
|
||
|
||
## 5. What this supersedes
|
||
|
||
- **Absolute-heat-share charge** (uncommitted: `capsule_vm_physics.c/.h` working-tree changes on
|
||
2026-09-24). No longer needed once Hera isn't compared. Proposed: discard it.
|
||
- **`MU-WAIT-ALL`**, and every busy-wait in the DoE driver.
|
||
- **The C-4 (α) consequence** recorded at C-b, where the console got no turn under worker load.
|
||
An observer with console input goes ahead of the participants at the next boundary.
|
||
|
||
## 6. Rulings needed from Captain Bob
|
||
|
||
1. **O-1:** observers with an event go ahead of participants at the next clean boundary
|
||
(proposed), or observers run only when no participant has work.
|
||
2. **O-2:** observers aren't charged absolute heat either (proposed), or they keep the C-a
|
||
absolute charge and are only left out of work heat.
|
||
3. **O-3:** observer heartbeat rows kept and tagged, filtered in analysis (proposed), or
|
||
suppressed.
|
||
4. **O-4:** device servicing under load: (c) accept for the relaunch and (b) interrupt-driven
|
||
later (proposed), or (a) treat the device-poll interval as a Hera event.
|
||
5. **Discard the uncommitted absolute-heat-share charge** (proposed), or keep it as a safety net.
|
||
|
||
**RULED 2026-09-24, Captain Bob: "all as proposed."** O-1 through O-4 and the discard (5) all go
|
||
as proposed above.
|
||
|
||
## 7. Raised 2026-09-24: messaging should flow on its own (OPEN, not yet designed)
|
||
|
||
**Captain Bob, verbatim:** *"Another potential defect is perhaps in messaging itself. We
|
||
shouldn't have to actually specifically decide to send a message … that should be autonomous by
|
||
the operating system itself … The messages should just flow. … However, I would still have a
|
||
manual method of sending and receiving messages … I should be able to create channels manually,
|
||
I should be able to destroy channels manually, I should be able to inject messages into the
|
||
queue, all of that should be manual, but also able to operate autonomously."*
|
||
|
||
**What exists today** (traced, not proposed):
|
||
- **The kernel side is complete in C:** `sk_hermes_channel_create/destroy/subscribe/unsubscribe/
|
||
is_member/member_count`, `sk_hermes_publish`, `sk_hermes_send_one`, and the negotiation calls
|
||
(`channel_request/respond/close/open_policy`), in `kernel_hermes.c`.
|
||
- **Only `SEND` reaches FORTH.** There are also the fixed-purpose `KH-BLK-ATTACH-SEND` and
|
||
`KH-ELEVATE-SEND`. There is no FORTH word to create or destroy a channel, subscribe, publish,
|
||
or inspect a queue.
|
||
- **Every message today is sent because some code decided to send it:** the console relay,
|
||
BLK-ATTACH, `SEND` in a job. Nothing is routed by the OS on its own.
|
||
|
||
**Two parts, which separate cleanly:**
|
||
- **Manual.** FORTH words over the existing C API: create/destroy a channel, subscribe and
|
||
unsubscribe, publish, inject a message into a queue, and inspect queues. Mechanical: mostly
|
||
exposure, not new mechanism. The ACL decides who may use them (`ACL.4th`, never C).
|
||
- **Autonomous.** What should flow without anyone deciding, and on what trigger, is a real design
|
||
question. Candidates to discuss:
|
||
- events the OS already knows about (a VM born, killed, parked idle; a device attached);
|
||
- results a VM leaves on its stack or in a known place, routed to whoever subscribed;
|
||
- heat/physics events routed to observers, which fits "observers act on events".
|
||
|
||
Not designed. It needs Captain Bob's framing first.
|
||
|
||
**Relation to the O stages:** independent of O-B to O-D. The event-driven DoE driver (O-C) is one
|
||
concrete case: a worker's completion notice is today a `SEND` in the job text, and could become an
|
||
OS event ("worker parked idle after its job") if autonomous flow lands first.
|
||
|
||
**Captain Bob, 2026-09-24:** *"Let's go and close the gap first."* The messaging gap goes ahead of
|
||
O-B to O-D. The design follows, **for review; nothing here authorizes an edit.**
|
||
|
||
### 7.0 First: kernel-Hermes gets no fixed small-scale bounds (defect, M-0)
|
||
|
||
**The rule already exists.**
|
||
- `FABRIC-3.5.md` §XLV.2 (B2, 2026-09-21) ruled the switch table dynamic, *"not to be fixed and
|
||
static"*. §XLVI 3.0(c) sized it from RAM at boot, like Stadium's `stadium_max_vm_count_val`.
|
||
`capsule_vm_switch_signal.c` cites the same principle as `feedback_no_hardcoded_small_scale_
|
||
bounds`.
|
||
- **Kernel-Hermes was never brought under it.** Four fixed bounds remain (traced):
|
||
|
||
| Bound | Where | Today |
|
||
|---|---|---|
|
||
| Message arena | `sk_hermes_msgs[SK_HERMES_MSG_MAX]`, a static array | 32 messages, system-wide |
|
||
| Per-VM pending queue | `SkHermesPendingQueue.slots[SK_HERMES_PENDING_MAX]` | 32 per VM |
|
||
| Channel membership | `SkHermesMembership.members[SK_HERMES_MEMBER_MAX]` | 64 per channel |
|
||
| Per-message heat | `SK_HERMES_Q_SLOT = Q48_ONE / SK_HERMES_MSG_MAX` | 1/32 of a reservoir. **Tied to the arena size.** `FABRIC-3.6.md` task 2.2 already noted: *"worth remembering if `SK_HERMES_MSG_MAX` is ever resized independently of `Q_SLOT`'s divisor."* |
|
||
|
||
The reassembly scratch array (`present[SK_HERMES_MSG_MAX]`, `sk_hermes_reassemble()`) sizes
|
||
from the same constant.
|
||
|
||
**Proposed:**
|
||
- **The arena, the queues and the member lists grow on demand** from the kernel heap (kmalloc,
|
||
grow by doubling, never shrink). This is the same allocate-copy-free pattern `vm_frame.c`
|
||
already uses, so there's no boot-time cap to choose, and the only limit is RAM, reported when
|
||
hit. *(Alternative: size them once at boot from RAM, like the switch table. Simpler, but still
|
||
a ceiling.)*
|
||
- **Message identity moves off the array index.** Growing the arena moves messages, so a queue
|
||
holds message ids resolved through the arena, never pointers. Today `idx = msg -
|
||
sk_hermes_msgs` already makes it an index; it stays an index into a table that can grow.
|
||
- **Per-message heat is decoupled from arena size** (ruling M-0b). How many messages a VM can
|
||
have in flight is a **heat** question: its reservoir, `Q_SLOT` per message. Proposed: `Q_SLOT`
|
||
becomes its own named constant with its current value (1/32 of a reservoir), **no longer
|
||
derived from the arena**. The heat budget stays a physics limit per VM, while capacity stops
|
||
being a memory limit. The alternative is to derive `Q_SLOT` from something else, which is Bob's
|
||
call.
|
||
|
||
`M-0` lands **before** the manual and autonomous work (M-A/M-B), since both add traffic.
|
||
|
||
### 7.1 Constraints the design has to live with (traced)
|
||
|
||
- **One arena of 32 messages system-wide** (`SK_HERMES_MSG_MAX`). Each queue is capped at the same
|
||
32 (`SK_HERMES_PENDING_MAX`), and each channel at 64 members (`SK_HERMES_MEMBER_MAX`). Autonomous
|
||
events share this arena with every `SEND`. **Captain Bob, 2026-09-24: "The arena should be
|
||
dynamic. We talked about that earlier and it wasn't done."** This is a defect against a standing
|
||
rule, not a design option; see §7.0.
|
||
- **Every message is funded from its sender's Stadium reservoir.** `sk_hermes_alloc()` pulls
|
||
`SK_HERMES_Q_SLOT` = `Q48_ONE`/32 and refuses if the sender can't pay. So an OS-originated
|
||
message still needs an identity to be paid from, and conservation must still hold.
|
||
- **A payload is FORTH text the recipient interprets** on its own stack. An event arrives as text,
|
||
so the recipient must have defined whatever words that text calls. Otherwise it gets
|
||
`UNKNOWN WORD`.
|
||
- **Channel 0 (common) is permanent.** Every born VM joins it (task 3.2). Channel slots number
|
||
`stadium_max_vm_count()`.
|
||
|
||
### 7.2 Manual control: FORTH words over the existing C API (proposed)
|
||
|
||
These are registered for every VM. No C-side restriction: who may use which word is `ACL.4th`'s
|
||
job. Names are placeholders for ruling M-1.
|
||
|
||
| Word | Stack | Does |
|
||
|---|---|---|
|
||
| `CHANNEL` | `( -- ch \| -1 )` | `sk_hermes_channel_create()` |
|
||
| `CHANNEL-CLOSE` | `( ch -- ok? )` | `sk_hermes_channel_destroy()`; refuses channel 0 |
|
||
| `JOIN` / `LEAVE` | `( ch -- ok? )` | subscribe or unsubscribe the **calling** VM |
|
||
| `JOIN-VM` / `LEAVE-VM` | `( ch name-c name-u -- ok? )` | subscribe or unsubscribe a **named** VM (administration) |
|
||
| `PUBLISH` | `( paddr plen ch -- n )` | send a copy to every member; returns how many were delivered |
|
||
| `SEND` | `( paddr plen name-c name-u -- ok? )` | exists (Stage G) |
|
||
| `INJECT` | `( paddr plen name-c name-u -- ok? )` | put a message in a VM's queue **as if from the kernel**: sender recorded as the root (Hera) and funded from her reservoir. For administration and testing. |
|
||
| `MAIL?` | `( name-c name-u -- n )` | a VM's pending count |
|
||
| `.MAIL` | `( name-c name-u -- )` | list a VM's pending messages: type, sender, length, first bytes |
|
||
| `.CHANNELS` | `( -- )` | list channels in use, with their members |
|
||
|
||
`from` is always derived in C, never taken from the stack, except that `INJECT` deliberately
|
||
records the root. That is exactly why it's ACL-gated.
|
||
|
||
### 7.3 Autonomous flow: the OS publishes what it already knows (proposed)
|
||
|
||
The kernel already sees the moments worth telling someone about. The proposal is that it **publishes
|
||
them itself, on well-known event channels, to whoever has joined.** Nothing is sent to a VM that
|
||
didn't join, so a VM never receives text it has no words for.
|
||
|
||
| Event channel | Fired by (existing kernel point) | Payload text |
|
||
|---|---|---|
|
||
| `EV-DONE` | a **participant** goes from having work to having none: its trampoline finds its queue empty after draining, and it parks idle | `S" <name>" ON-DONE` |
|
||
| `EV-BORN` | `capsule_birth_baby()` reaches LIVE | `S" <name>" ON-BORN` |
|
||
| `EV-DIED` | `capsule_vm_kill()` / `capsule_vm_force_reap()` succeed | `S" <name>" ON-DIED` |
|
||
| *(later)* `EV-DEVICE` | WIREBIND/USB attach or detach | *(ruling M-3)* |
|
||
|
||
The handler convention is: **joining an event channel is the promise to define its `ON-…` word.**
|
||
The event channels are created at boot, beside channel 0. Their ids are fixed and published as FORTH
|
||
constants, so both `JOIN` and the ACL can name them.
|
||
|
||
- **Who pays** (ruling M-5). The VM the event is about is the natural sender: it "emits" its own
|
||
completion or birth, just without its code deciding to. A dying VM can't pay after its reservoir
|
||
is gone, so `EV-DIED` comes from the root.
|
||
- **Which VMs** (ruling M-4). Events are fired for **participants only**. Observers already act on
|
||
events; they don't need to announce their own idleness, and that also rules out any loop of
|
||
observers reacting to each other's events.
|
||
- **When the arena is full** (ruling M-6). An event that can't be admitted is **dropped, counted
|
||
and logged, never blocking**, since a kernel event point must not wait on a queue. Dropped events
|
||
are visible in the ledger and in a counter the DoE can check.
|
||
|
||
### 7.4 What this does for the DoE (O-C, redone)
|
||
|
||
- Hera `JOIN`s `EV-DONE` once and defines `ON-DONE ( c-addr u -- )`, which counts.
|
||
- Each worker's job becomes just `WL-RUN`: no `WL-DONE-ARGS`, and no `SEND` in the job text.
|
||
- A worker's completion reaches Hera autonomously, as the OS's `EV-DONE`.
|
||
- `WL-DONE-ARGS` comes out of the workload capsules again.
|
||
- The event-driven driver of §3 stays: the last `ON-DONE` ends the trial and starts the next.
|
||
|
||
### 7.5 Rulings needed from Captain Bob
|
||
|
||
1. **M-1:** the word names in §7.2 (placeholders).
|
||
2. **M-2:** `INJECT`, sent as the root and ACL-gated (proposed), or sent as the caller (which would
|
||
make it just `SEND`).
|
||
3. **M-3:** the first event set: `EV-DONE`, `EV-BORN`, `EV-DIED` now, with device events later
|
||
(proposed).
|
||
4. **M-4:** events for participants only (proposed), or for every VM.
|
||
5. **M-5:** funding: the VM the event is about, with `EV-DIED` from the root (proposed).
|
||
6. **M-6:** when a message can't be admitted, drop, count and log, never block (proposed). After
|
||
M-0 that happens only when a sender's **reservoir** can't fund it, or RAM runs out; the arena
|
||
itself is no longer a limit.
|
||
7. **M-0 (defect, ahead of everything):** the arena, queues and member lists grow on demand
|
||
(proposed), or are sized at boot from RAM.
|
||
8. **M-0b:** `Q_SLOT` decoupled from arena size and kept at 1/32 of a reservoir as its own
|
||
constant (proposed), or derived some other way.
|
||
|
||
## 8. Audit: every fixed-size structure in the kernel (2026-09-24)
|
||
|
||
**Captain Bob, verbatim:** *"completely audit everything and find out where we're using static data
|
||
structures … no absolutely dedicated sized memory slots … if you're slotting, we can only do X
|
||
number of VMs or Y number of messages or whatever … None of that should exist. No arrays, in other
|
||
words. It should all be fairly dynamic."*
|
||
|
||
**Method** (read-only, 2026-09-24).
|
||
- Searched `src/`, `include/` and `src/starkernel/` (test runner excluded) for:
|
||
- every `#define` whose name carries `MAX`, `CAP`, `SLOTS`, `COUNT`, `DEPTH`, `LIMIT`,
|
||
`ENTRIES`, `SIZE` or `LEN`;
|
||
- every static or global array with a constant size;
|
||
- every constant-sized array inside a struct;
|
||
- every table sized once at boot.
|
||
- For each capacity limit, checked **what happens at the limit** by reading its use sites.
|
||
- Grouped by what the size actually limits. Classes C and D are the cases Captain Bob named;
|
||
classes E–H are listed so that nothing is hidden, with a recommendation.
|
||
|
||
### 8.1 Class C — silent truncation (worse than a limit; fix first)
|
||
|
||
| Where | Today | At the limit |
|
||
|---|---|---|
|
||
| `parity.c:498/646`, `MAX_DICT_ENTRIES` 2048 | parity walks at most 2048 dictionary entries | **Hashing stops.** Beyond 2048 words, `dict_hash` silently ignores the rest, so the determinism evidence would be computed over part of the dictionary. |
|
||
| `vm.h` `DICTIONARY_SIZE` 1024: `word_id_map[1024]`, `recycled_word_ids[1024]`; `dictionary_management.c:87` | word ids 0–1023 | **Word 1025 onward gets `WORD_ID_INVALID`,** and the physics hooks skip it (`physics_execution_hooks.c:95`). Execution heat, pipelining and transitions silently stop for later words. Hera is at 531. |
|
||
| `string_words.c` `WORD` scratch, `WORD_SCRATCH_CAP` 64 | one **static** `word_scratch_addr`, allotted in the first caller's VM space | Not a count limit, but a static shared across VMs: a second VM calling `WORD` gets an address in someone else's space. Found at Stage G. |
|
||
|
||
**Captain Bob, 2026-09-24:** *"both of those failure modes that you've noted need to be handled
|
||
properly as well."* Removing the limit is not enough. **Every place something can fail to grow
|
||
must fail properly:**
|
||
|
||
1. **Never silently.** Every failure is logged (`log_message` at `LOG_ERROR`, which the log
|
||
region persists), and counted in a counter that a FORTH word or DoE row can read.
|
||
2. **Never partially.** The operation is refused whole, with a return value the caller checks.
|
||
Nothing is done halfway.
|
||
3. **With the right consequence for what it guards:**
|
||
- **Parity (`dict_hash`)** must cover the whole dictionary. If the walk can't be completed, for
|
||
example the entry list can't be allocated, the parity record is marked **incomplete**
|
||
(`PARITY:… INCOMPLETE`) rather than printing a hash that looks valid. It becomes a hard,
|
||
visible failure of the determinism evidence, not a quiet wrong value.
|
||
- **Word ids:** the id map grows with the dictionary. If growth fails, defining the word
|
||
**fails with an error** (`vm->error`, same as any other dictionary-full condition), rather
|
||
than defining a word with `WORD_ID_INVALID` that silently runs outside the physics.
|
||
|
||
This applies to every item in §8.2–§8.4 as it becomes dynamic: each grow path gets a checked
|
||
failure, a log line, a counter and a clean refusal. The evidence for each stage includes a probe
|
||
that **forces** the failure (an injected allocation failure, reverted) and shows the refusal, the
|
||
log line and the counter.
|
||
|
||
### 8.2 Class D — fixed counts of things (must become dynamic)
|
||
|
||
| Where | Limits | At the limit |
|
||
|---|---|---|
|
||
| `kernel_hermes.c` `sk_hermes_msgs[32]` (`SK_HERMES_MSG_MAX`) | messages in flight, **system-wide** | send refused (§7.0, M-0) |
|
||
| `kernel_hermes.c` queue `slots[32]` (`SK_HERMES_PENDING_MAX`) | pending messages per VM | send refused |
|
||
| `kernel_hermes.h` `members[64]` (`SK_HERMES_MEMBER_MAX`) | VMs per channel | subscribe refused |
|
||
| `kernel_hermes.c:698` `present[32]`; `kernel_main.c:886` `msgs[32]` | chunks per reassembly; a self-test | follows `SK_HERMES_MSG_MAX` |
|
||
| `capsule_wirebind.c:90` `g_wirebind_live[13]` (`WIREBIND_MAX_LIVE`) | **identities attached at once: 13** | attach/detach tracking degraded (message printed) |
|
||
| `block_subsystem.c:199` `reloc[500]` (`BLK_RELOC_MAX_ENTRIES`) | block relocations | `BLK_ENOMEM` |
|
||
| `hal.c:173` `hal_exec_regions[8]` | executable regions registered | refused |
|
||
| `capsule_run.c:57` `run_log[1024]` | run records kept | wraps; oldest records lost |
|
||
| `kernel_main.c`/`capsule_loader.c` `KRD_MAX_BLOCKS` 1024, `KRD_BUF_SIZE` 1 MiB | the kernel RAM disk | fixed size |
|
||
| `vm.h` `CF_STACK_MAX` 64: `cf_stack`, `leave_addrs`, `endof_addrs`, … | control-flow nesting while compiling | compile error past 64 |
|
||
| `include/starkernel/capsule.h` `CAPSULE_MAX_COUNT` 256 | *(nothing: defined, never used)* | dead; remove |
|
||
|
||
### 8.3 Class D′ — tables sized once at boot (a ceiling, just a RAM-derived one)
|
||
|
||
The B2 precedent (`FABRIC-3.5.md` §XLV.2) made these "dynamic" by sizing them from RAM at boot. By
|
||
the rule stated today that is still a slot count: *"we can only do X number of VMs."*
|
||
|
||
| Where | Sized from | Limits |
|
||
|---|---|---|
|
||
| `stadium.c` `stadium_max_vm_count_val` | `STADIUM_VM_MEMORY_PERCENT` of free heap ÷ `VM_MEMORY_SIZE` | **the VM population**: `capsule_birth.c:731` refuses births past it |
|
||
| `stadium.c` cells, bitmap, owner, resident lists | `STADIUM_MEMORY_PERCENT` of free heap ÷ cell size | Stadium cells (patrons, messages) |
|
||
| `stadium.c` `owner` is `uint16_t` | type width | ≤ 65,535 quota slots, whatever the RAM |
|
||
| `capsule_vm_switch_signal.c` slots; `kernel_hermes.c` channel table and queue table; `session.c` sessions; `stadium_blocks.c` counters; Stadium quota table | `stadium_max_vm_count()` | participants, channels, queues, sessions |
|
||
|
||
**Proposed:** all of these grow on demand (kmalloc, doubling, as `vm_frame.c` does), and the VM
|
||
population is bounded only by whether a birth can actually allocate. Stadium's cell pool is the
|
||
one to discuss: it is the physics' own substrate, and whether it grows or stays a fixed "stadium"
|
||
is a physics question (ruling S-2).
|
||
|
||
### 8.4 Class E — sized by hardware or firmware (size from what the device or table reports)
|
||
|
||
| Where | Today | Proposed |
|
||
|---|---|---|
|
||
| `pmm.c:61` `pmm_bitmap[2 MiB]` (static) | tracks at most ~64 GiB of RAM | allocate from the UEFI memory map's actual extent |
|
||
| `ioapic.c:123` `g_redir_low/valid[24]` | 24 redirection entries | the IOAPIC reports its own count (up to 240); size from it |
|
||
| `ioapic.c:116` `g_overrides[16]` | MADT interrupt-source overrides | size from the MADT |
|
||
| `xhci_driver.h:197` `port_slot_id[32]`; `XHCI_MAX_TRACKED_PORTS` | 32 root ports | the controller reports `MaxPorts`; size from it |
|
||
| `xhci.h` `XHCI_RING_TRB_COUNT` 256; virtio `VQUEUE_SIZE` 4/1, `EVENTQ_SIZE` 8 | ring and queue depths | devices negotiate their queue size; could size from it. Behaves as a throughput limit, not a count of things. |
|
||
| `rpi5_native_boot.c` memmap/RAM/reserved-range arrays | firmware regions | size from the device tree |
|
||
| `interrupts.c` `idt[256]` | **architecture-defined** (x86 has exactly 256 vectors) | keep |
|
||
| `uefi_loader.c` `KERNEL_MAX_SIZE` 8 MiB | kernel image size the loader accepts | read the image size from the file instead |
|
||
|
||
### 8.5 Class F — the per-VM model (a design decision, not a slot count)
|
||
|
||
| Where | Today |
|
||
|---|---|
|
||
| `vm.h` `VM_MEMORY_SIZE` / `arena.c` `SK_VM_MEMORY_SIZE` | 5 MiB address space per VM (dictionary plus blocks) |
|
||
| `vm.h` `STACK_SIZE` | 1024 cells, data and return stacks |
|
||
| `arena.c` `SK_VM_STACK_SIZE` | 2 MiB native stack per VM |
|
||
| `vm.h` `INPUT_BUFFER_SIZE` 1025 | one block per line, **already ruled** (`.claude/CLAUDE.md`, §XLV.3) |
|
||
| `vm.h` `WORD_NAME_MAX` 31; `capsule_run.h` `VM_NAME_MAX` 64; `USER_IDENTITY_*_MAX` | name lengths |
|
||
| `kernel_args.h` default heap 2 GiB | kernel heap |
|
||
|
||
These bound how big one VM or one name can be, not how many of anything there are. Growable VM
|
||
memory and stacks are possible but reach deep into the vendored VM (`VM_ADDR`/`vaddr_t` addressing).
|
||
Proposed: leave for a separate decision (ruling S-4).
|
||
|
||
### 8.6 Class G — on-disk formats and persistence rings
|
||
|
||
- `log_region.h`: `LOG_SLOT_SIZE` 1024, and `LOG_REGION_MAX_DEVBLOCKS` 32.
|
||
- `doe_region.h`: `DOE_SLOT_SIZE` 64, `DOE_REGION_MAX_DEVBLOCKS` 4, and `DOE_SLOT_ISA_MAX` 8.
|
||
- `capsule_wirebind.c`: `WIREBIND_CERT_MAX_DEVBLOCKS` 4.
|
||
|
||
Slot sizes are the persistence **format**. Region sizes are ring capacities on the disk image
|
||
(older entries are overwritten). Changing either is a format change with a migration question.
|
||
Proposed: keep the formats, and make region sizes a disk-layout decision (ruling S-5).
|
||
|
||
### 8.7 Class H — not capacity: caches, statistical windows, tables of constants
|
||
|
||
- **Caches:** `HOTWORDS_CACHE_SIZE`, the TTF caches, the vt100 scrollback (1000/4096 lines),
|
||
`DISK_CACHE_SLOTS` 8, `BLK_VM_SLOTS` 4, and FORTH-79 `BUFFER_COUNT` 4. More would be faster; none
|
||
refuses anything.
|
||
- **Statistical windows:** `ROLLING_WINDOW_SIZE`, `TRANSITION_WINDOW_SIZE`,
|
||
`VM_FLEET_WINDOW_DEPTH`, `TIME_WINDOW_SIZE`, `CD_MAX_CLASSIFY_DEPTH`, and `HEARTBEAT_TICK_BUFFER_
|
||
SIZE`. These are measurement parameters, Kconfig-tunable, and part of the DoE's definition.
|
||
- **Scratch and staging buffers** sized to one payload or line (`g_send_buf`, `g_kh_*_buf`,
|
||
`DOE_BUF_SIZE`, `LOG_MSG_LINE_MAX`), and device rings (`i8042` 32, virtio-input diagnostics 16).
|
||
- **Constant tables:** crypto constants, font bitmaps, keymaps, and the TTF parser's per-glyph
|
||
bounds (a font-format limit, refused cleanly).
|
||
- **Pools that refill:** `vm_uuid_pool[16]`, which is a batch, not a limit.
|
||
|
||
Proposed: leave class H as it is.
|
||
|
||
### 8.8 Rulings needed
|
||
|
||
1. **S-1:** fix class C first, each on its own: parity hashing, dictionary word ids, `WORD` scratch
|
||
(proposed). Each gets the failure handling ruled in §8.1: visible, counted, refused whole, and
|
||
parity marked incomplete rather than quietly wrong.
|
||
2. **S-2:** class D′, Stadium's cell pool. Grow on demand like everything else, or stay a fixed
|
||
"stadium" sized at boot, since it is the physics' substrate?
|
||
3. **S-3:** classes D and D′ (except S-2) grow on demand, in this order: kernel-Hermes (M-0);
|
||
WIREBIND identities; the VM population and its per-VM tables (switch slots, sessions, queues,
|
||
channels, quotas); relocations, exec regions, run log and control-flow nesting (proposed).
|
||
4. **S-4:** class F left for a separate decision (proposed).
|
||
5. **S-5:** class G formats kept; region sizes are a disk-layout decision (proposed).
|
||
6. **S-6:** class E sized from what the hardware or firmware reports (proposed), IDT excepted.
|
||
|
||
**RULED 2026-09-24, Captain Bob: "all as proposed, commit and start S-1."** S-1, S-3, S-4, S-5
|
||
and S-6 go as proposed. So do §7's M-0, M-0b and M-1 to M-6, including the §7.2 word names as
|
||
written. **S-2 stays open:** it was posed as a question with no proposal (does Stadium's cell pool
|
||
grow, or stay a fixed stadium?). It needs its own answer before S-3 reaches Stadium.
|