Payload bound and chunking -- FABRIC-3.6.md task 3.5

sk_hermes_publish() now enforces SK_HERMES_CHUNK_MAX_PAYLOAD (1024) on
every message's payload_len uniformly, chunked or not -- closing the
gap task 3.3 explicitly parked. A chunk carrier is
[SkHermesChunkHeader][content slice], slice capped at
1024 - sizeof(header) rather than 1024 itself, so every message on the
wire satisfies the same one-block bound vm_interpret()'s own drain
limit already requires -- a future chunk-aware drain never has to
special-case a carrier that can't be handed to vm_interpret() as-is.

Deliberately no chunking-sender API: building one would need
kernel-Hermes to own chunk-buffer memory with a real lifetime it has no
way to track (kept alive until every subscriber drains it). Sending is
a loop pattern a caller writes with sk_hermes_chunk_count() +
sk_hermes_publish(), demonstrated by this task's own self-test.
sk_hermes_reassemble() is pure and memory-agnostic: validates msg_id
agreement, exact seq coverage, and per-chunk slice sizes before a
single memcpy, with the total length computed once and checked against
the caller's buffer once -- never order-dependent on which chunk
happens to overflow.

Verified live on all three architectures: a 1024-byte payload as one
message, a 1025-byte send refused outright with the ledger untouched,
and a 3000-byte payload split into 3 chunks, drained, and reassembled
byte-exact against the original. dict_hash unmoved and identical
across architectures.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-22 06:51:48 -04:00
co-authored by Claude Sonnet 5
parent 2b1ba031a5
commit 8a2ee0fdad
7 changed files with 28126 additions and 6 deletions
+33 -1
View File
@@ -902,10 +902,42 @@ ruling]** cannot be written precisely until Captain Bob settles the named sub-it
real disk-backed block storage, which a throwaway diagnostic has no business perturbing.
`dict_hash` for Hermes/Hestia unmoved and identical across all three architectures (this
task touches no capsule).
- [ ] **3.5** — **Payload bound and chunking** (B4) **[needs ruling 3.0d]**. Max 1024 bytes per
- [x] **3.5** — **Payload bound and chunking** (B4) **[needs ruling 3.0d]**. Max 1024 bytes per
message; larger payloads sent as ordered chunks, reassembled before drain. *Check:*
1024-byte payload single message; 3000-byte payload chunked and reassembled byte-exact;
1025-byte single-message send refused.
2026-09-22 · `logs/20260922-063122/amd64/`, `logs/20260922-063629/aarch64/`,
`logs/20260922-064147/riscv64/` — all three reach `[zuse@Hera] ok>`, zero `UNKNOWN WORD`,
`mkcapsule --lint capsules/` clean (38 files, 0 violations, unchanged -- no capsule
touched). Sizing decision (checked with `advisor()`, not itself a separate ruling -- see
the header's own note): `SK_HERMES_CHUNK_MAX_PAYLOAD` (1024) bounds every message's
`payload_len` **uniformly**, chunked or not -- a chunk carrier is
`[SkHermesChunkHeader][content slice]` where the slice is capped at
`1024 - sizeof(header)`, not 1024 itself. Rejected the literal alternative (slice up to
1024, carrier up to `1024+sizeof(header)`): under that reading a chunk carrier could never
be handed to `vm_interpret()` as-is, making a future chunk-aware drain a special case
instead of the same one-block invariant every other message already satisfies.
`sk_hermes_publish()` (task 3.3) now enforces this bound directly, closing the gap that
task's own doc comment explicitly parked ("3.3 does not enforce a payload size limit").
**Deliberately no chunking-SENDER API** (`sk_hermes_publish_chunked()` or similar) --
`advisor()` flagged that building one raises a real memory-lifetime question kernel-Hermes
has never answered (chunk buffers would need to stay alive until every subscriber drains
them, and nothing here can know when that is); sending is a loop pattern a caller writes
with `sk_hermes_chunk_count()` + `sk_hermes_publish()`, demonstrated by this task's own
self-test rather than hidden behind a new allocator. `sk_hermes_reassemble()` is pure and
memory-agnostic (caller-owned input chunks, caller-owned output buffer) -- validates
`msg_id` agreement, exact `seq` coverage `{0..n_chunks-1}` with `is_last` on exactly the
last, and every non-final slice at exactly `SK_HERMES_CHUNK_MAX_SLICE` bytes, before a
single `memcpy`; the reassembled total length is computed once from validated seq
completeness and checked against `out_buf_cap` once, so a too-small output buffer is
refused cleanly with zero partial copy rather than order-dependently on whichever chunk
happens to overflow. Self-test (kernel_main.c) proves exactly the task's three checks: a
1024-byte payload as one message; a 1025-byte single-message send refused outright, no
allocation, ledger untouched; a 3000-byte payload split into 3 chunks, drained off a
synthetic subscriber's own pending queue (task 3.3 primitives), reassembled, and confirmed
byte-exact against the original. Ledger audit and `stadium_conserved()` hold before and
after. `dict_hash` for Hermes/Hestia unmoved and identical across all three architectures
(this task touches no capsule).
- [ ] **3.6** — **ACK/NACK and private-channel negotiation** (B1) **[needs ruling 3.0a]**.
`request` → `grant`/`deny` on the common channel; a grant creates a private topic;
close tears it down. ACK/NACK are message types on the existing allocator.
+104 -5
View File
@@ -416,12 +416,15 @@ int sk_hermes_queues_boot_init(void);
* (-1) if invalid/not in use.
* @param type Message type code, passed through unchanged.
* @param payload_addr Out-of-line payload address, passed through
* unchanged (bound/chunking is task 3.5's scope, not
* this one's -- 3.3 does not enforce a payload size
* limit).
* @param payload_len Payload length in bytes, passed through unchanged.
* unchanged.
* @param payload_len Payload length in bytes. Refused (-1, no
* allocation attempted) if it exceeds
* SK_HERMES_CHUNK_MAX_PAYLOAD (task 3.5, added here
* since task 3.3 deliberately parked this check --
* see that section's own doc comment below).
* @return Count of subscribers successfully enqueued to (0..member count),
* or -1 if channel_id itself was invalid.
* or -1 if channel_id itself was invalid or payload_len exceeded
* the bound.
*/
int sk_hermes_publish(VMUuid from, int channel_id, uint32_t type,
void *payload_addr, uint32_t payload_len);
@@ -510,6 +513,102 @@ typedef struct VM VM;
*/
int sk_hermes_drain_checkpoint(VM *vm);
/*
* Payload bound and chunking (FABRIC-3.6.md task 3.5, B4 / FABRIC-3.5.md
* SXLV.3; chunk framing ruled 2026-09-21: each chunk carries (msg_id,
* seq, is_last) ahead of the real content).
*
* Sizing decision (not itself ruled -- see SXLV.4's own note that the
* concrete framing was a task-writing prerequisite, not a ruling):
* SK_HERMES_CHUNK_MAX_PAYLOAD (1024, one block, SXLIV.1) bounds a
* message's payload_len UNIFORMLY -- chunked or not. A chunk carrier's
* own payload is [SkHermesChunkHeader][content slice], so the slice
* itself is capped at 1024 - sizeof(SkHermesChunkHeader), not 1024,
* keeping every message on the wire under the same one-block bound
* SXLIII.6.1 already requires for vm_interpret()'s own drain limit.
* (The alternative -- slice up to 1024, carrier up to
* 1024+sizeof(header) -- was considered and rejected: it would mean a
* chunk carrier can never be handed to vm_interpret() as-is, making a
* later chunk-aware drain a special case instead of the same drain path
* every other message already uses.)
*
* sk_hermes_publish() (task 3.3, above) enforces this bound directly --
* refuses (-1) any single payload_len over SK_HERMES_CHUNK_MAX_PAYLOAD,
* whether or not it's a chunk carrier, since both cases hold the same
* invariant now.
*
* SENDING a payload over SK_HERMES_CHUNK_MAX_SLICE bytes is deliberately
* NOT a kernel-Hermes API here -- it is a loop a caller writes with the
* primitives below plus sk_hermes_publish(), the same way this task's
* own self-test proves reassembly (kernel_main.c: build N static chunk
* buffers, sk_hermes_publish() each). Building a chunking sender would
* need kernel-Hermes to own chunk-buffer memory with a real lifetime
* (kept alive until every subscriber has drained it, freed only once
* kernel-Hermes has no way to know that) -- a genuine new question this
* task's inert scope does not call for. A real multi-chunk sender is a
* later task's problem, when a real message type actually needs one
* (3.8+, the same boundary chunk-aware drain is already deferred past).
*/
/* One block = 1024 bytes (64x16, SXLIV.1) -- the uniform bound every
* message's payload_len must satisfy, chunked or not (see this
* section's own sizing note above). */
#define SK_HERMES_CHUNK_MAX_PAYLOAD 1024
/* SkHermesChunkHeader - prepended to a chunk carrier's own payload,
* ahead of up to SK_HERMES_CHUNK_MAX_SLICE bytes of real content.
* msg_id is caller-chosen and must be unique per logical multi-chunk
* send from a given sender -- reassembly groups chunks by it. seq is
* 0-based, ascending, no gaps, {0..n_chunks-1}. is_last is set on
* exactly the chunk whose seq is n_chunks-1, no other. */
typedef struct {
uint32_t msg_id;
uint32_t seq;
int is_last;
} SkHermesChunkHeader;
/* Content bytes per chunk -- the one-block bound minus header overhead. */
#define SK_HERMES_CHUNK_MAX_SLICE (SK_HERMES_CHUNK_MAX_PAYLOAD - (uint32_t)sizeof(SkHermesChunkHeader))
/* sk_hermes_chunk_count - number of SK_HERMES_CHUNK_MAX_SLICE-sized
* chunks `len` bytes needs (ceiling division). 0 bytes needs 0 chunks;
* 1..SK_HERMES_CHUNK_MAX_SLICE needs 1; and so on. Pure arithmetic, no
* side effects -- a caller sizing its own chunk-send loop calls this
* first. */
uint32_t sk_hermes_chunk_count(uint32_t len);
/* sk_hermes_reassemble - concatenates n_chunks chunk carrier messages
* (already drained off a subscriber's own pending queue via
* sk_hermes_pending_peek()/sk_hermes_pending_pop(), task 3.3) sharing
* one msg_id back into one byte-exact buffer at out_buf, in ascending
* seq order regardless of the order chunks[] itself is passed in.
*
* Refuses (-1, *out_len untouched, no partial copy ever lands in
* out_buf) if: n_chunks is 0 or exceeds SK_HERMES_MSG_MAX (the system-
* wide message arena size -- chunking can never need more chunk-carrier
* messages than the whole arena holds); any chunk is NULL, has no
* payload, or its payload is shorter than sizeof(SkHermesChunkHeader);
* the chunks' msg_ids disagree; seq values are not exactly
* {0..n_chunks-1} (a duplicate or a gap); is_last is not set on exactly
* the seq==n_chunks-1 chunk and no other; a non-final chunk's content
* slice is not exactly SK_HERMES_CHUNK_MAX_SLICE bytes, or the final
* chunk's is 0 or over that bound; or the total reassembled length
* would exceed out_buf_cap (checked once, from validated seq
* completeness, before any memcpy -- never order-dependent on which
* chunk happens to overflow first).
*
* @param chunks Array of n_chunks chunk carrier message pointers.
* @param n_chunks Chunk count (from sk_hermes_chunk_count() at send
* time, or simply len(chunks[])).
* @param out_buf Caller-owned destination buffer.
* @param out_buf_cap Capacity of out_buf in bytes.
* @param out_len Set to the reassembled length on success only.
* @return 0 on success, -1 on any refusal above.
*/
int sk_hermes_reassemble(SkHermesMessage **chunks, int n_chunks,
uint8_t *out_buf, uint32_t out_buf_cap,
uint32_t *out_len);
#endif /* __STARKERNEL__ */
#endif /* STARKERNEL_VM_KERNEL_HERMES_H */
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+116
View File
@@ -1504,6 +1504,122 @@ static void kernel_main_deep(BootInfo *boot_info) {
}
}
/* FABRIC-3.6.md task 3.5 self-test: payload bound and chunking.
* Three checks, exactly the task's own: a 1024-byte payload as one
* message; a 3000-byte payload chunked (sk_hermes_chunk_count()) and
* reassembled byte-exact (sk_hermes_reassemble()); a 1025-byte
* single-message send refused by sk_hermes_publish() itself. No
* chunking-SENDER API exists (deliberately, see kernel_hermes.h's
* own doc comment on this section) -- this self-test builds its own
* chunk buffers directly, the pattern a real caller would follow.
* Large working buffers are function-static, not stack locals, to
* stay clear of any kernel-stack-size assumption. */
{
static uint8_t chunk_src[3000];
static uint8_t chunk_buf[3][SK_HERMES_CHUNK_MAX_PAYLOAD];
static uint8_t chunk_out[3072];
static uint8_t oversize_payload[SK_HERMES_CHUNK_MAX_PAYLOAD + 1];
VMUuid pub_id4, sub_id4;
int ch;
int chunk_ok = 1;
int grant_rc;
uint32_t n_chunks, i;
uint64_t held_before, pulled_before, returned_before, consumed_before;
uint64_t held_after, pulled_after, returned_after, consumed_after;
pub_id4.hi = 0; pub_id4.lo = 13;
sub_id4.hi = 0; sub_id4.lo = 14;
console_puts("Kernel-Hermes chunk self-test: ");
grant_rc = stadium_grant_quota(pub_id4, vm_uuid_hera());
if (grant_rc != 0) {
console_println("SKIPPED (quota grant failed)");
} else {
ch = sk_hermes_channel_create();
if (ch < 0) chunk_ok = 0;
if (chunk_ok && sk_hermes_channel_subscribe(ch, sub_id4) != 0) chunk_ok = 0;
sk_hermes_ledger(&held_before, &pulled_before, &returned_before, &consumed_before);
/* Check 1: exactly SK_HERMES_CHUNK_MAX_PAYLOAD bytes -- one
* message, no chunk header, must be accepted. */
if (chunk_ok) {
static uint8_t one_block[SK_HERMES_CHUNK_MAX_PAYLOAD];
for (i = 0; i < SK_HERMES_CHUNK_MAX_PAYLOAD; i++) one_block[i] = (uint8_t)i;
if (sk_hermes_publish(pub_id4, ch, 0, one_block, SK_HERMES_CHUNK_MAX_PAYLOAD) != 1)
chunk_ok = 0;
if (chunk_ok && sk_hermes_pending_count(sub_id4) != 1) chunk_ok = 0;
if (chunk_ok) {
SkHermesMessage *msg = sk_hermes_pending_peek(sub_id4);
if (!msg || msg->payload_len != SK_HERMES_CHUNK_MAX_PAYLOAD) chunk_ok = 0;
if (chunk_ok && sk_hermes_release(msg) != 0) chunk_ok = 0;
if (chunk_ok && sk_hermes_pending_pop(sub_id4) != 0) chunk_ok = 0;
}
}
/* Check 3: one byte over the bound -- must be refused
* outright, no allocation, ledger untouched. */
if (chunk_ok) {
if (sk_hermes_publish(pub_id4, ch, 0, oversize_payload,
SK_HERMES_CHUNK_MAX_PAYLOAD + 1) != -1) chunk_ok = 0;
if (chunk_ok && sk_hermes_pending_count(sub_id4) != 0) chunk_ok = 0;
}
/* Check 2: 3000 bytes, chunked and reassembled byte-exact. */
if (chunk_ok) {
for (i = 0; i < sizeof(chunk_src); i++) chunk_src[i] = (uint8_t)((i * 7 + 3) & 0xFF);
n_chunks = sk_hermes_chunk_count(sizeof(chunk_src));
if (n_chunks == 0 || n_chunks > 3) chunk_ok = 0;
}
if (chunk_ok) {
uint32_t sent = 0;
for (i = 0; i < n_chunks; i++) {
SkHermesChunkHeader *h = (SkHermesChunkHeader *)&chunk_buf[i][0];
uint32_t remaining = (uint32_t)sizeof(chunk_src) - sent;
uint32_t slice = (remaining > SK_HERMES_CHUNK_MAX_SLICE) ?
SK_HERMES_CHUNK_MAX_SLICE : remaining;
h->msg_id = 0xC5;
h->seq = i;
h->is_last = (i == n_chunks - 1) ? 1 : 0;
memcpy(&chunk_buf[i][0] + sizeof(SkHermesChunkHeader), &chunk_src[sent], slice);
if (sk_hermes_publish(pub_id4, ch, 0, &chunk_buf[i][0],
(uint32_t)sizeof(SkHermesChunkHeader) + slice) != 1) chunk_ok = 0;
sent += slice;
}
if (chunk_ok && sent != sizeof(chunk_src)) chunk_ok = 0;
if (chunk_ok && sk_hermes_pending_count(sub_id4) != (int)n_chunks) chunk_ok = 0;
}
if (chunk_ok) {
SkHermesMessage *msgs[3];
uint32_t out_len = 0;
for (i = 0; i < n_chunks; i++) {
msgs[i] = sk_hermes_pending_peek(sub_id4);
if (!msgs[i]) { chunk_ok = 0; break; }
if (sk_hermes_pending_pop(sub_id4) != 0) { chunk_ok = 0; break; }
}
if (chunk_ok &&
sk_hermes_reassemble(msgs, (int)n_chunks, chunk_out, sizeof(chunk_out), &out_len) != 0)
chunk_ok = 0;
if (chunk_ok && out_len != sizeof(chunk_src)) chunk_ok = 0;
if (chunk_ok && memcmp(chunk_out, chunk_src, sizeof(chunk_src)) != 0) chunk_ok = 0;
for (i = 0; i < n_chunks; i++) {
if (sk_hermes_release(msgs[i]) != 0) chunk_ok = 0;
}
}
sk_hermes_ledger(&held_after, &pulled_after, &returned_after, &consumed_after);
if (held_after != held_before) chunk_ok = 0; /* every allocation released, back to baseline */
if (!sk_hermes_audit_values(held_after, pulled_after, returned_after, consumed_after)) chunk_ok = 0;
if (!stadium_conserved(pub_id4)) chunk_ok = 0;
if (chunk_ok && sk_hermes_channel_destroy(ch) != 0) chunk_ok = 0;
console_println(chunk_ok ? "PASS" : "FAIL");
}
}
/* Decided 2026-09-05: no console for the running system unless a
* thumbdrive is present -- headless by default (EMERGENCY_CONSOLE_
* ENABLED off), reusing that flag's own existing "does this build
+72
View File
@@ -488,6 +488,11 @@ int sk_hermes_publish(VMUuid from, int channel_id, uint32_t type,
int delivered = 0;
if (!channel_valid(channel_id)) return -1;
/* Task 3.5: the one-block bound, enforced uniformly here whether or
* not payload_addr happens to be a chunk carrier -- see
* kernel_hermes.h's own sizing note on why SK_HERMES_CHUNK_MAX_SLICE
* (not SK_HERMES_CHUNK_MAX_PAYLOAD) is a chunk's own content size. */
if (payload_len > SK_HERMES_CHUNK_MAX_PAYLOAD) return -1;
m = &sk_hermes_channels[channel_id].membership;
for (i = 0; i < m->count; i++) {
@@ -587,4 +592,71 @@ int sk_hermes_drain_checkpoint(VM *vm) {
return drain_error ? -1 : 1;
}
uint32_t sk_hermes_chunk_count(uint32_t len) {
if (len == 0) return 0;
return (len + SK_HERMES_CHUNK_MAX_SLICE - 1) / SK_HERMES_CHUNK_MAX_SLICE;
}
int sk_hermes_reassemble(SkHermesMessage **chunks, int n_chunks,
uint8_t *out_buf, uint32_t out_buf_cap,
uint32_t *out_len) {
int present[SK_HERMES_MSG_MAX];
int i;
uint32_t msg_id;
uint32_t total;
uint32_t last_slice_len = 0;
if (!chunks || n_chunks <= 0 || !out_buf || !out_len) return -1;
if (n_chunks > SK_HERMES_MSG_MAX) return -1; /* guard before indexing present[] */
for (i = 0; i < n_chunks; i++) present[i] = 0;
if (!chunks[0] || !chunks[0]->payload_addr ||
chunks[0]->payload_len < (uint32_t)sizeof(SkHermesChunkHeader)) return -1;
msg_id = ((SkHermesChunkHeader *)chunks[0]->payload_addr)->msg_id;
for (i = 0; i < n_chunks; i++) {
SkHermesMessage *m = chunks[i];
SkHermesChunkHeader *h;
uint32_t slice_len;
if (!m || !m->payload_addr || m->payload_len < (uint32_t)sizeof(SkHermesChunkHeader)) return -1;
h = (SkHermesChunkHeader *)m->payload_addr;
slice_len = m->payload_len - (uint32_t)sizeof(SkHermesChunkHeader);
if (h->msg_id != msg_id) return -1;
if (h->seq >= (uint32_t)n_chunks) return -1;
if (present[h->seq]) return -1; /* duplicate seq */
present[h->seq] = 1;
if (h->seq == (uint32_t)(n_chunks - 1)) {
if (!h->is_last) return -1;
if (slice_len == 0 || slice_len > SK_HERMES_CHUNK_MAX_SLICE) return -1;
last_slice_len = slice_len;
} else {
if (h->is_last) return -1;
if (slice_len != SK_HERMES_CHUNK_MAX_SLICE) return -1;
}
}
for (i = 0; i < n_chunks; i++) if (!present[i]) return -1; /* gap */
/* Total computed once from validated seq completeness, checked once
* against out_buf_cap, before any memcpy -- never order-dependent on
* which chunk happens to overflow first. */
total = (uint32_t)(n_chunks - 1) * SK_HERMES_CHUNK_MAX_SLICE + last_slice_len;
if (total > out_buf_cap) return -1;
for (i = 0; i < n_chunks; i++) {
SkHermesMessage *m = chunks[i];
SkHermesChunkHeader *h = (SkHermesChunkHeader *)m->payload_addr;
uint32_t slice_len = m->payload_len - (uint32_t)sizeof(SkHermesChunkHeader);
uint32_t offset = h->seq * SK_HERMES_CHUNK_MAX_SLICE;
memcpy(out_buf + offset, (uint8_t *)m->payload_addr + sizeof(SkHermesChunkHeader), slice_len);
}
*out_len = total;
return 0;
}
#endif /* __STARKERNEL__ */