Reorganize source tree: kernel/, v3/, v4/ split and board infrastructure

Source tree reorganization: - Move StarForth v3 engine to v3/ (src/, include/, Makefile) - Move kernel to kernel/ (src/, include/, linker/, Makefile) - Create v4/ skeleton for F18-ISA golden model (DECOMPOSITION.md, JUSTIFICATION.md) - Move FABRIC-0..4.md to docs/fabric/ - Move ONTOLOGY.md and ROADMAP.md to docs/ Board infrastructure: - Add boards/ser5/, boards/raspi/, boards/milkv/, boards/zynq7020/ - Each board has board.mk (ISA, CPU flags, boot recipe) and README.md - Root Makefile becomes thin dispatcher: boot_image, all, clean, docs take TARGET - make boot_image TARGET=SER5|RASPI|MILKV builds one GPT/MBR image per board - ZYNQ7020 target exists but stops with clear error (ARMv7 port not built yet) - scripts/mkdiskimage.sh builds disk images for all boards Docs pipeline: - docs/book/ with LaTeX master (main.tex) and Makefile - pandoc converts Markdown to LaTeX at build time - Two Lua filters: table-widths.lua (wide tables wrap), code-breaks.lua (inline code breaks) - make docs builds single PDF (754 pages, 0 missing characters) - make docs TARGET=<board> adds board appendix - build/docs/<book|board>/meta.tex stamps git commit into PDF Bug fixes: - 42 include paths that only worked by accident now use correct relative paths - clang-18 hardcode replaced with configurable CC variable (fixed aarch64 build) - Pi 5: kernel_2712.img linked at 0x80000, .bss zeroed, memory reserved - Doxyfile, .clang-tidy, README.md, Kconfig paths updated Verified: - Hosted v3 build passes 1012 tests, 0 failures - SER5 image boots in QEMU (OVMF), POST passes, K exact (65536 = Q48_ONE) - Milk-V image boots in QEMU (OpenSBI + U-Boot + bootefi), POST passes - make clean TARGET=<board> removes only that board and its ISA objects - make all builds all boards, hosted v3, and docs in one run Co-authored-by: Junie <junie@jetbrains.com>
2026-10-01 15:40:09 -04:00
parent 3c709c115b
commit a8b70e88d3
630 changed files with 2646 additions and 2036 deletions
@@ -0,0 +1,233 @@
+/*
+  StarForth — Steady-State Virtual Machine Runtime
+
+  Copyright (c) 2023–2025 Robert A. James
+  All rights reserved.
+
+  This file is part of the StarForth project.
+
+  Licensed under the StarForth License, Version 1.0 (the "License");
+  you may not use this file except in compliance with the License.
+
+  You may obtain a copy of the License at:
+      https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
+
+  This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
+  express or implied, including but not limited to the warranties of
+  merchantability, fitness for a particular purpose, and noninfringement.
+
+  See the License for the specific language governing permissions and
+  limitations under the License.
+ */
+
+/*
+ * platform/alloc_kernel.c - Kernel (bare-metal) memory allocator
+ *
+ * FABRIC-3.md §X.4, 2026-09-07: this used to be its own isolated static
+ * 4MB arena (first-fit free list, no splitting/coalescing) -- sized small
+ * enough, and fragile enough under concurrent VM churn, that a live
+ * identity-heap-capacity test found the arena topping out around 6
+ * concurrent WIREBIND-born VMs, failing *before* true capacity exhaustion
+ * (fragmentation from concurrent background allocation, no coalescing to
+ * recover from it). Meanwhile `kmalloc.c` -- the kernel's general heap,
+ * already initialised at boot (M6, well before any VM is ever born) --
+ * sits right next to it: reserved from real PMM-tracked physical memory
+ * (not a fixed compile-time array), defaults to a 2 GiB floor explicitly
+ * sized "for 256+ baby VMs" per its own comment, overridable via the
+ * `--heap=` boot flag, and its free list *does* coalesce neighboring
+ * blocks on every free.
+ *
+ * Every VM's word dictionary (`vm_create_word()`, the shared/vendored VM
+ * core) allocates through this file's `sf_malloc()`/`sf_free()` --
+ * `platform_alloc.h`'s portable allocator abstraction, kernel-side. This
+ * file now simply delegates to `kmalloc()`/`kfree()` instead of managing
+ * its own separate, much smaller arena: same API contract callers already
+ * depend on, real headroom (whatever the boot-time heap ends up being,
+ * not a hardcoded 4MB), and real coalescing. No new allocator was
+ * invented -- `kmalloc.c` already existed, was already boot-tested on all
+ * three arches, and was simply never wired up as the backing store for
+ * VM dictionaries specifically.
+ *
+ * `sf_alloc_init()` is now a no-op (see its own doc comment below): with
+ * a *shared* heap serving every kernel subsystem, not an isolated arena
+ * used only for VM dictionaries, "reset" is no longer a sane operation --
+ * nothing external ever called it anyway (confirmed: no callers besides
+ * `sf_malloc()`'s own lazy-init guard, which this file's `sf_malloc()`
+ * still keeps, now as a readiness check rather than an initializer).
+ *
+ * This file's own alignment guarantee (`SF_ALIGN`, 8 bytes) is preserved
+ * via `kmalloc_aligned()` -- `kmalloc()`'s own default alignment
+ * (`KMALLOC_MIN_ALIGN`, 16 bytes) already satisfies it, but asking
+ * explicitly keeps this file's contract self-documenting rather than
+ * relying on kmalloc.c's own default not changing under it.
+ */
+
+#include "platform_alloc.h"
+#include "starkernel/kmalloc.h"
+#include <stdint.h>
+#include <stddef.h>
+
+/* Alignment for all allocations (8 bytes for 64-bit safety) -- unchanged
+ * from this file's previous arena-based implementation. */
+#define SF_ALIGN 8
+
+/* alloc_count/free_count have no kmalloc.c equivalent (it tracks bytes,
+ * not call counts) -- kept here as simple diagnostic counters layered on
+ * top of kmalloc's own byte-accurate stats, which sf_alloc_get_stats()
+ * below reads fresh on every call rather than shadowing them locally. */
+static size_t g_alloc_count = 0;
+static size_t g_free_count = 0;
+
+/**
+ * @brief No-op: kept only for API compatibility with `platform_alloc.h`.
+ *
+ * The backing store is now the shared kernel heap (`kmalloc.c`), already
+ * initialised at boot (M6) well before any VM allocation can occur --
+ * there is nothing left for this function to set up, and "resetting" a
+ * heap shared by every other kernel subsystem would be actively wrong.
+ * Kept callable (matches its documented "safe to call at any point"
+ * contract) so no caller needs to change.
+ *
+ * @return 0 always (cannot fail)
+ */
+int sf_alloc_init(void)
+{
+    return 0;
+}
+
+/*
+ * @brief Allocate memory from the shared kernel heap.
+ *
+ * Delegates to `kmalloc_aligned()` (`SF_ALIGN`-byte aligned, matching
+ * this file's previous guarantee). Returns @c NULL for zero-size
+ * requests and whenever the shared heap itself returns NULL (not yet
+ * initialised, or genuinely out of memory) -- both already part of this
+ * function's documented contract.
+ *
+ * @param size Number of bytes to allocate.
+ * @return Pointer to the allocated block on success, @c NULL on failure.
+ */
+void* sf_malloc(size_t size)
+{
+    if (size == 0) return (void*)0;
+
+    void *ptr = kmalloc_aligned(size, SF_ALIGN);
+    if (ptr) g_alloc_count++;
+    return ptr;
+}
+
+/*
+ * @brief Allocate and zero-initialise a contiguous array from the kernel heap.
+ *
+ * Unchanged from this file's previous implementation: computes
+ * @p count × @p size, checks for integer overflow via
+ * @c total/size != count, then delegates to @c sf_malloc(). The returned
+ * block is zeroed with a manual byte loop rather than @c memset so that
+ * the kernel build remains freestanding with no libc dependency.
+ * Returns @c NULL when either argument is zero, on overflow, or on heap
+ * exhaustion.
+ *
+ * @param count Number of elements to allocate.
+ * @param size  Size of each element in bytes.
+ * @return Pointer to the zeroed block on success, @c NULL on failure.
+ */
+void* sf_calloc(size_t count, size_t size)
+{
+    if (count == 0 || size == 0) return (void*)0;
+
+    size_t total = count * size;
+
+    /* Check for overflow */
+    if (size != 0 && total / size != count)
+    {
+        return (void*)0;
+    }
+
+    void* ptr = sf_malloc(total);
+    if (ptr)
+    {
+        /* Zero-initialize */
+        uint8_t* p = (uint8_t*)ptr;
+        for (size_t i = 0; i < total; i++)
+        {
+            p[i] = 0;
+        }
+    }
+    return ptr;
+}
+
+/*
+ * @brief Resize an allocation in the kernel heap.
+ *
+ * Unchanged from this file's previous implementation: `kmalloc.c` has no
+ * realloc-equivalent, so a fresh block of @p new_size bytes is allocated
+ * and returned; the original block at @p ptr is left for the caller to
+ * free explicitly (this function does not free it, matching this file's
+ * prior documented behavior exactly -- not a regression introduced by
+ * the kmalloc.c switch).
+ *
+ * Special cases match the C standard:
+ * - @p ptr == @c NULL → equivalent to @c sf_malloc(@p new_size).
+ * - @p new_size == 0 → returns @c NULL (caller treats old block as freed).
+ *
+ * The caller is responsible for copying content from the old block
+ * before discarding the old pointer; this function does not perform the
+ * copy.
+ *
+ * @param ptr      Pointer to the existing allocation (may be @c NULL).
+ * @param new_size Desired size of the new block in bytes.
+ * @return Pointer to the new block on success, @c NULL on failure or when
+ *         @p new_size is zero.
+ */
+void* sf_realloc(void* ptr, size_t new_size)
+{
+    if (!ptr) return sf_malloc(new_size);
+    if (new_size == 0) return (void*)0;
+
+    return sf_malloc(new_size);
+}
+
+/*
+ * @brief Release a kernel heap allocation, making it available for reuse.
+ *
+ * Delegates to @c kfree(), which coalesces the freed block with any free
+ * neighbors -- real reclamation, unlike this file's previous first-fit-
+ * no-splitting free list (FABRIC-3.md §X.4: that lack of coalescing was
+ * the proximate cause of a live-observed allocation failure under
+ * concurrent VM churn even with more than enough free bytes overall).
+ *
+ * @param ptr Pointer previously returned by @c sf_malloc() / @c sf_calloc()
+ *            (may be @c NULL; silently ignored, matching @c kfree()'s own
+ *            contract). Must not be used again by the caller after this
+ *            call, and must not be freed twice.
+ */
+void sf_free(void* ptr)
+{
+    if (!ptr) return;
+
+    kfree(ptr);
+    g_free_count++;
+}
+
+/*
+ * @brief Retrieve a snapshot of kernel allocator statistics.
+ *
+ * Reads fresh from `kmalloc_get_stats()` on every call -- the shared heap
+ * is the real source of truth, not a value shadowed locally. alloc_count/
+ * free_count (which kmalloc.c does not track) come from this file's own
+ * counters instead.
+ *
+ * @param stats Output buffer to receive the statistics; silently returns
+ *              without writing if @p stats is @c NULL.
+ */
+void sf_alloc_get_stats(sf_alloc_stats_t* stats)
+{
+    if (!stats) return;
+
+    kmalloc_stats_t k = kmalloc_get_stats();
+    stats->total_bytes = (size_t)k.total_bytes;
+    stats->used_bytes  = (size_t)k.used_bytes;
+    stats->peak_bytes  = (size_t)k.peak_bytes;
+    stats->alloc_count = g_alloc_count;
+    stats->free_count  = g_free_count;
+}