Byte order: v4 is little-endian by rule; the kernel and v3 are unchecked, and nothing tests it #194

New Issue

2026-11-13T16:23:27Z

rajames commented

2026-11-13 16:23:27 +00:00

Captain Bob, 2026-11-13: "Are we using little Indian?", then "File it for the next milestone" (alpha.5 is closed to new work).

What is known (checked 2026-11-13)

v4 is little-endian by rule, not because the host is. The byte order is written into the code with shifts on cell values, never by reading memory as bytes:

Bytes in cells. A node is word-addressed, and a cell holds four bytes, little-endian, at both cell widths, so compiled code is the same at 32 and 64 bits. Ruled in docs/v4.0.0/DECOMPOSITION.md (D-1, D-9); C@ and C! do it with shifts.
Messages pack their characters into cells by shifting: v4/src/message.c:25.
Blocks convert to and from the kernel's block bytes by shifting: v4/system/blocks.c:47, :117.
dict_hash hashes each cell's bytes low byte first, extracted by shifting (v4/src/image.c:61), so the hash does not depend on the host's byte order.

What is not known

Nothing tests it. All three acceptance targets (amd64, aarch64 as run, riscv64) are little-endian, so a big-endian assumption anywhere would never show up.
The kernel and v3 were not checked. Capsule images (tools/mkcapsule.c, capsule_generated.c), the Stadium's patron cells, the identity and Artemis disk formats, GPT and the block devices, and the v3 VM may rely on the host's byte order. That is harmless on today's targets, but it isn't proven.

Proposed

An audit of the kernel and v3 for host-byte-order dependence: struct images written to disk or into the kernel image, byte-pointer casts over multi-byte values, and memcpy of integers into byte buffers. Record each finding: harmless on little-endian, or a real portability hole.
A test that fails if v4 starts to depend on the host's byte order. One way: run v4's host tests on a big-endian target under qemu-user (for example s390x or big-endian ppc64; make init already installs qemu-user for v4-checks) and require the same dict_hash and POST results. Whether that becomes part of v4-checks is Captain Bob's call.
Whether big-endian hosts are a goal at all is Captain Bob's call. If they are not, write "little-endian only" down as a stated limit in boards/BRINGUP.md.

Captain Bob, 2026-11-13: "Are we using little Indian?", then "File it for the next milestone" (alpha.5 is closed to new work). ## What is known (checked 2026-11-13) **v4 is little-endian by rule, not because the host is.** The byte order is written into the code with shifts on cell values, never by reading memory as bytes: - **Bytes in cells.** A node is word-addressed, and a cell holds four bytes, little-endian, at both cell widths, so compiled code is the same at 32 and 64 bits. Ruled in `docs/v4.0.0/DECOMPOSITION.md` (D-1, D-9); `C@` and `C!` do it with shifts. - **Messages** pack their characters into cells by shifting: `v4/src/message.c:25`. - **Blocks** convert to and from the kernel's block bytes by shifting: `v4/system/blocks.c:47`, `:117`. - **`dict_hash`** hashes each cell's bytes low byte first, extracted by shifting (`v4/src/image.c:61`), so the hash does not depend on the host's byte order. ## What is not known 1. **Nothing tests it.** All three acceptance targets (amd64, aarch64 as run, riscv64) are little-endian, so a big-endian assumption anywhere would never show up. 2. **The kernel and v3 were not checked.** Capsule images (`tools/mkcapsule.c`, `capsule_generated.c`), the Stadium's patron cells, the identity and Artemis disk formats, GPT and the block devices, and the v3 VM may rely on the host's byte order. That is harmless on today's targets, but it isn't proven. ## Proposed - An audit of the kernel and v3 for host-byte-order dependence: struct images written to disk or into the kernel image, byte-pointer casts over multi-byte values, and `memcpy` of integers into byte buffers. Record each finding: harmless on little-endian, or a real portability hole. - A test that fails if v4 starts to depend on the host's byte order. One way: run v4's host tests on a big-endian target under qemu-user (for example s390x or big-endian ppc64; `make init` already installs qemu-user for v4-checks) and require the same `dict_hash` and POST results. Whether that becomes part of `v4-checks` is Captain Bob's call. - Whether big-endian hosts are a goal at all is Captain Bob's call. If they are not, write "little-endian only" down as a stated limit in `boards/BRINGUP.md`.

rajames added this to the v4.0.0-beta.0 milestone 2026-11-13 16:23:27 +00:00

rajames added the

kind

chore

area/v3 area/kernel area/v4

priority

nice

labels 2026-11-13 16:23:29 +00:00

rajames commented

2026-11-13 16:29:56 +00:00

Ruled (Captain Bob, 2026-11-13): "Little Indian only. Yeah, we need a build gate to check that."

So this issue's scope is now:

Little-endian only is the stated limit. Write it down in boards/BRINGUP.md and in v4's design (DECOMPOSITION.md D-1).
A build gate that refuses to build for a big-endian target. Proposed: one header included by the kernel, v4 and the host tools, with a preprocessor check (__BYTE_ORDER__ / __ORDER_LITTLE_ENDIAN__, which GCC and Clang predefine; preprocessor only, no language extension) and an #error naming this issue. A compiler that doesn't define the macros is refused too, not let through. Also a boot-time check of a known integer's first byte, which prints and stops on a mismatch, so a build that slips past the preprocessor still can't run silently wrong.
A test that the gate refuses: compile the header with __BYTE_ORDER__ set to big-endian, and require the build to fail.
The audit of the kernel and v3 shrinks to a check that the gate covers every build: the kernel at all three architectures, v4, tools/, and the hosted v3.
The big-endian qemu-user run proposed above is dropped, since big-endian targets are now refused rather than supported.

Milestone: beta.0. alpha.5 is closed to new work (2026-11-13) unless Captain Bob overrides that.

**Ruled (Captain Bob, 2026-11-13):** "Little Indian only. Yeah, we need a build gate to check that." So this issue's scope is now: - **Little-endian only is the stated limit.** Write it down in `boards/BRINGUP.md` and in v4's design (DECOMPOSITION.md D-1). - **A build gate** that refuses to build for a big-endian target. Proposed: one header included by the kernel, v4 and the host tools, with a preprocessor check (`__BYTE_ORDER__` / `__ORDER_LITTLE_ENDIAN__`, which GCC and Clang predefine; preprocessor only, no language extension) and an `#error` naming this issue. A compiler that doesn't define the macros is refused too, not let through. Also a boot-time check of a known integer's first byte, which prints and stops on a mismatch, so a build that slips past the preprocessor still can't run silently wrong. - A test that the gate refuses: compile the header with `__BYTE_ORDER__` set to big-endian, and require the build to fail. - The audit of the kernel and v3 shrinks to a check that the gate covers every build: the kernel at all three architectures, v4, `tools/`, and the hosted v3. - The big-endian qemu-user run proposed above is dropped, since big-endian targets are now refused rather than supported. Milestone: beta.0. alpha.5 is closed to new work (2026-11-13) unless Captain Bob overrides that.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: rajames/LithosAnanke#194