Reorganize source tree: kernel/, v3/, v4/ split and board infrastructure

Source tree reorganization:
- Move StarForth v3 engine to v3/ (src/, include/, Makefile)
- Move kernel to kernel/ (src/, include/, linker/, Makefile)
- Create v4/ skeleton for F18-ISA golden model (DECOMPOSITION.md, JUSTIFICATION.md)
- Move FABRIC-0..4.md to docs/fabric/
- Move ONTOLOGY.md and ROADMAP.md to docs/

Board infrastructure:
- Add boards/ser5/, boards/raspi/, boards/milkv/, boards/zynq7020/
- Each board has board.mk (ISA, CPU flags, boot recipe) and README.md
- Root Makefile becomes thin dispatcher: boot_image, all, clean, docs take TARGET
- make boot_image TARGET=SER5|RASPI|MILKV builds one GPT/MBR image per board
- ZYNQ7020 target exists but stops with clear error (ARMv7 port not built yet)
- scripts/mkdiskimage.sh builds disk images for all boards

Docs pipeline:
- docs/book/ with LaTeX master (main.tex) and Makefile
- pandoc converts Markdown to LaTeX at build time
- Two Lua filters: table-widths.lua (wide tables wrap), code-breaks.lua (inline code breaks)
- make docs builds single PDF (754 pages, 0 missing characters)
- make docs TARGET=<board> adds board appendix
- build/docs/<book|board>/meta.tex stamps git commit into PDF

Bug fixes:
- 42 include paths that only worked by accident now use correct relative paths
- clang-18 hardcode replaced with configurable CC variable (fixed aarch64 build)
- Pi 5: kernel_2712.img linked at 0x80000, .bss zeroed, memory reserved
- Doxyfile, .clang-tidy, README.md, Kconfig paths updated

Verified:
- Hosted v3 build passes 1012 tests, 0 failures
- SER5 image boots in QEMU (OVMF), POST passes, K exact (65536 = Q48_ONE)
- Milk-V image boots in QEMU (OpenSBI + U-Boot + bootefi), POST passes
- make clean TARGET=<board> removes only that board and its ISA objects
- make all builds all boards, hosted v3, and docs in one run

Co-authored-by: Junie <junie@jetbrains.com>
This commit is contained in:
rajames
2026-10-01 15:40:09 -04:00
co-authored by Junie
parent 3c709c115b
commit a8b70e88d3
630 changed files with 2646 additions and 2036 deletions
+347
View File
@@ -0,0 +1,347 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
https://github.com/star.4th@proton.me/StarForth/LICENSE.txt
This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND,
express or implied, including but not limited to the warranties of
merchantability, fitness for a particular purpose, and noninfringement.
See the License for the specific language governing permissions and
limitations under the License.
*/
/**
* capsule_birth.h - VM Birth Protocol (M7.1)
*
* Functions for birthing VMs from capsules:
* - Mama init: Execute core/init.4th to establish Mama's PERSONALITY
* - Baby birth: Create new VM from (p) capsule
* - Experiment run: Execute (e) capsule on Mama
*/
#ifndef STARKERNEL_CAPSULE_BIRTH_H
#define STARKERNEL_CAPSULE_BIRTH_H
#include <stdint.h>
#include "starkernel/capsule.h"
#include "starkernel/capsule_run.h"
#ifdef __cplusplus
extern "C" {
#endif
/*===========================================================================
* VM Execution Hook
*
* The birth protocol needs to execute FORTH code on a VM.
* This hook is provided by the VM layer.
*===========================================================================*/
/**
* VM execution function type
*
* @param vm_ctx Opaque pointer to VM context
* @param code FORTH source code to execute
* @param code_len Length of code in bytes
* @return 0 on success, non-zero on error
*/
typedef int (*CapsuleExecFn)(void *vm_ctx, const char *code, uint64_t code_len);
/**
* Dictionary hash function type
*
* @param vm_ctx Opaque pointer to VM context
* @return 64-bit hash of dictionary state
*/
typedef uint64_t (*CapsuleDictHashFn)(void *vm_ctx);
/**
* VM allocation function type (for baby birth)
*
* @return Opaque pointer to new VM context, or NULL on failure
*/
typedef void *(*CapsuleVMAllocFn)(void);
/**
* capsule_birth_set_hooks - Configure VM execution hooks
*
* Must be called before any birth operations.
*
* @param exec_fn Function to execute FORTH code on VM
* @param dict_hash_fn Function to compute dictionary hash
* @param vm_alloc_fn Function to allocate new VM (for babies)
*/
void capsule_birth_set_hooks(
CapsuleExecFn exec_fn,
CapsuleDictHashFn dict_hash_fn,
CapsuleVMAllocFn vm_alloc_fn
);
/*===========================================================================
* Mama Init
*===========================================================================*/
/**
* capsule_birth_mama - Execute Mama's init capsule
*
* Finds the MAMA_INIT capsule by flag, validates it, executes it on Mama's VM.
*
* @param mama_vm Mama's VM context
* @param dir Capsule directory header
* @param descs Capsule descriptor array
* @param names Capsule name entry array (parallel to descs)
* @param arena Capsule payload arena
* @return CAPSULE_RUN_OK on success, error code otherwise
*/
CapsuleRunResult capsule_birth_mama(
void *mama_vm,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena
);
/*===========================================================================
* Baby Birth
*===========================================================================*/
/**
* capsule_birth_baby - Birth a new VM from a named (p) capsule
*
* Finds the capsule by colon-separated name, validates it, allocates a new
* VM, executes the capsule payload as IDENTITY, then (if present) executes
* unit.4th from the baby's block space as PERSONALITY.
*
* @param capsule_name Colon-separated capsule name, e.g. "production:myvm.4th"
* @param dir Capsule directory header
* @param descs Capsule descriptor array
* @param names Capsule name entry array (parallel to descs)
* @param arena Capsule payload arena
* @param parent Who is birthing this VM (FABRIC-2.md §H.12 step 7) --
* the caller's own VMUuid (e.g. vm->stadium_vm_id for
* a FORTH word handler), recorded on the new VM's
* Session.parent. Every current call site has one in
* scope, directly or one level up; traced live rather
* than assumed (checked all 6 call sites across
* mama_forth_words.c/capsule_console.c/
* capsule_runcap.c/capsule_wirebind.c).
* @param skip_pki_sig 0 for every build-time capsule (the normal case --
* checked against the compile-time-baked signature
* array via capsule_get_signatures()). Non-zero only
* for RUNCAP (FABRIC-2.md §F.6/F.18): a heap-built,
* single-entry directory sourced from a user's own
* thumbdrive has no entry in that array at all --
* index 0 would silently compare against whatever
* real capsule happens to occupy slot 0, which is
* not a security check, just a guaranteed-wrong one.
* Trust for that content comes from CERTVERIFY (a
* separate root, the user's own Zuse-signed cert)
* already having run before RUNCAP is ever called,
* not from this flag -- this only skips a check that
* was never meaningful for that content in the first
* place. Deliberately a plain flag, not a new entry
* point, so the policy is one call-site decision,
* trivially reversible.
* @param out_vm_id Output: assigned VM ID
* @param out_vm_ctx Output: new VM context
* @return CAPSULE_RUN_OK on success, error code otherwise
*/
CapsuleRunResult capsule_birth_baby(
const char *capsule_name,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena,
VMUuid parent,
int skip_pki_sig,
VMUuid *out_vm_id,
void **out_vm_ctx
);
/*===========================================================================
* Experiment Execution
*===========================================================================*/
/**
* capsule_run_experiment - Execute a named (e) capsule on Mama
*
* Finds the experiment capsule by colon-separated name, validates it,
* executes it on Mama's VM without creating a new VM.
*
* @param mama_vm Mama's VM context
* @param capsule_name Colon-separated capsule name, e.g. "experiments:doe-l8:init-l8-stable.4th"
* @param dir Capsule directory header
* @param descs Capsule descriptor array
* @param names Capsule name entry array (parallel to descs)
* @param arena Capsule payload arena
* @param out_run_id Output: assigned run ID
* @return CAPSULE_RUN_OK on success, error code otherwise
*/
CapsuleRunResult capsule_run_experiment(
void *mama_vm,
const char *capsule_name,
const CapsuleDirHeader *dir,
const CapsuleDesc *descs,
const CapsuleNameEntry *names,
const uint8_t *arena,
uint64_t *out_run_id
);
/*===========================================================================
* VM Hook Registration
*===========================================================================*/
/**
* capsule_vm_hooks_register - Wire concrete VM hooks into the capsule subsystem
*
* Registers capsule_exec_hook, capsule_dict_hash_hook, and capsule_vm_alloc_hook.
* Must be called after vm_init() on Mama's VM and before any birth operations.
*/
void capsule_vm_hooks_register(void);
/*===========================================================================
* VM Registry
*===========================================================================*/
/**
* capsule_vm_registry_init - Initialize VM registry
*
* Allocates Mama's registry node (VM 0, name "Hera") via kmalloc and
* stores mama_vm_ptr so KILL can guard against destroying Mama.
* Must be called after kmalloc_init().
*
* @param mama_vm_ptr Pointer to Mama's VM object (e.g. &sk_mama_vm)
*/
void capsule_vm_registry_init(void *mama_vm_ptr);
/**
* capsule_vm_kill - Destroy a named VM and release all its resources
*
* Looks up the VM by name (case-insensitive). Hera (VM 0) cannot be
* killed. If the VM is already DEAD the call is a no-op.
* On success: vm_cleanup + sf_free, state → VM_STATE_DEAD, name cleared.
*
* @param name Symbolic name of the VM to kill (case-insensitive)
* @return 0 on success (or already dead), -1 if not found or refused
*/
int capsule_vm_kill(const char *name);
/**
* capsule_vm_registry_get - Get VM registry entry by ID
*
* @param vm_id VM ID to look up
* @param out Output: registry entry copy
* @return 0 on success, -1 if not found
*/
int capsule_vm_registry_get(VMUuid vm_id, VMRegistryEntry *out);
/**
* capsule_vm_registry_count - Get number of registered VMs
*/
uint32_t capsule_vm_registry_count(void);
/**
* capsule_vm_registry_get_by_index - Get registry entry by list position
* (birth order, stable within a boot session -- the registry is
* append-only). For enumeration (e.g. the idle-loop messaging pump,
* FABRIC-2.md Phase C, 2026-08-28), where no vm_id is known up front.
* Index range is [0, capsule_vm_registry_count()).
*
* @param index Zero-based position in birth order
* @param out Output: registry entry copy
* @return 0 on success, -1 if index is out of range
*/
int capsule_vm_registry_get_by_index(uint32_t index, VMRegistryEntry *out);
/**
* capsule_vm_find_by_name - Find VM registry entry by symbolic name
*
* Case-sensitive. Returns the first match.
*
* @param name Symbolic VM name, e.g. "Hermes"
* @param out Output: registry entry copy
* @return 0 if found, -1 if not found
*/
int capsule_vm_find_by_name(const char *name, VMRegistryEntry *out);
/**
* capsule_vm_find_by_name_nocase - Find VM registry entry by name (case-insensitive)
*
* Used by BIRTH for idempotency: prevents birthing a second VM with the
* same name regardless of case differences.
*
* @param name Symbolic VM name (compared case-insensitively)
* @param out Output: registry entry copy
* @return 0 if found, -1 if not found
*/
int capsule_vm_find_by_name_nocase(const char *name, VMRegistryEntry *out);
/**
* capsule_vm_set_state - Update a VM's state in the registry
*
* @param vm_id VM ID to update
* @param state New VMState value
*/
void capsule_vm_set_state(VMUuid vm_id, uint32_t state);
/**
* capsule_vm_set_pending_reap - FABRIC-3.md §XXVIII Stage 4 (2026-09-14):
* mark vm_id for deferred teardown once it is no longer worth resuming --
* see VMRegistryEntry.pending_reap's own doc comment for the full
* rationale. No-op if vm_id isn't registered.
*
* @param vm_id VM ID to mark.
* @param pending 1 to mark, 0 to clear (e.g. a re-attach of the same
* identity before the switcher ever reaped it).
*/
void capsule_vm_set_pending_reap(VMUuid vm_id, int pending);
/**
* capsule_vm_force_reap - FABRIC-3.md §XXVIII Stage 4 (2026-09-14):
* unconditionally tear down vm_id regardless of VM_STATE_SWITCHED_OUT --
* the one caller allowed to bypass capsule_vm_kill()'s own refusal there,
* because this is called *by* the Stage 3 switcher itself (vm_core.c's
* checkpoint, on noticing pending_reap set), at the one point that
* genuinely knows the parked native-stack context will never be resumed.
* Also releases the VM's own switch-signal slot
* (sk_vm_switch_signal_unregister()) -- generic cleanup, independent of
* whatever subsystem (WIREBIND today) set pending_reap in the first
* place. No-op if vm_id isn't registered or is already DEAD.
*
* @param vm_id VM ID to reap.
*/
void capsule_vm_force_reap(VMUuid vm_id);
/**
* capsule_vm_registry_set_name - Assign a symbolic name to a registered VM
*
* Truncates to VM_NAME_MAX-1 characters. No-op if vm_id not found.
*
* @param vm_id VM ID to name
* @param name Symbolic name string
*/
void capsule_vm_registry_set_name(VMUuid vm_id, const char *name);
/**
* capsule_vm_kill_all_nonmama - Kill every non-Mama VM in the registry.
*
* Sets halted, calls vm_cleanup + sf_free, marks state DEAD. Used by
* Hera's BYE immediately before arch_cold_reset() to reap all children.
*/
void capsule_vm_kill_all_nonmama(void);
#ifdef __cplusplus
}
#endif
#endif /* STARKERNEL_CAPSULE_BIRTH_H */