Reorganize source tree: kernel/, v3/, v4/ split and board infrastructure
Source tree reorganization: - Move StarForth v3 engine to v3/ (src/, include/, Makefile) - Move kernel to kernel/ (src/, include/, linker/, Makefile) - Create v4/ skeleton for F18-ISA golden model (DECOMPOSITION.md, JUSTIFICATION.md) - Move FABRIC-0..4.md to docs/fabric/ - Move ONTOLOGY.md and ROADMAP.md to docs/ Board infrastructure: - Add boards/ser5/, boards/raspi/, boards/milkv/, boards/zynq7020/ - Each board has board.mk (ISA, CPU flags, boot recipe) and README.md - Root Makefile becomes thin dispatcher: boot_image, all, clean, docs take TARGET - make boot_image TARGET=SER5|RASPI|MILKV builds one GPT/MBR image per board - ZYNQ7020 target exists but stops with clear error (ARMv7 port not built yet) - scripts/mkdiskimage.sh builds disk images for all boards Docs pipeline: - docs/book/ with LaTeX master (main.tex) and Makefile - pandoc converts Markdown to LaTeX at build time - Two Lua filters: table-widths.lua (wide tables wrap), code-breaks.lua (inline code breaks) - make docs builds single PDF (754 pages, 0 missing characters) - make docs TARGET=<board> adds board appendix - build/docs/<book|board>/meta.tex stamps git commit into PDF Bug fixes: - 42 include paths that only worked by accident now use correct relative paths - clang-18 hardcode replaced with configurable CC variable (fixed aarch64 build) - Pi 5: kernel_2712.img linked at 0x80000, .bss zeroed, memory reserved - Doxyfile, .clang-tidy, README.md, Kconfig paths updated Verified: - Hosted v3 build passes 1012 tests, 0 failures - SER5 image boots in QEMU (OVMF), POST passes, K exact (65536 = Q48_ONE) - Milk-V image boots in QEMU (OpenSBI + U-Boot + bootefi), POST passes - make clean TARGET=<board> removes only that board and its ISA objects - make all builds all boards, hosted v3, and docs in one run Co-authored-by: Junie <junie@jetbrains.com>
This commit is contained in:
rajames
@@ -0,0 +1,700 @@
|
||||
# StarForth Ontology, Taxonomy, and Lexicon
|
||||
|
||||
**Version**: 1.1
|
||||
**Date**: 2026-08-13
|
||||
**Purpose**: Formal conceptual framework for precise academic discourse
|
||||
|
||||
---
|
||||
|
||||
## I. ONTOLOGY (Conceptual Framework)
|
||||
|
||||
### 1.1 Core Concepts
|
||||
|
||||
```
|
||||
StarForth Adaptive Runtime System
|
||||
├── Execution Metrics
|
||||
│ ├── Execution Frequency (primary measurable)
|
||||
│ ├── Temporal Decay (derived quantity)
|
||||
│ └── Transition Probability (derived quantity)
|
||||
├── Adaptive Mechanisms
|
||||
│ ├── Frequency-Based Caching
|
||||
│ ├── Window-Based Inference
|
||||
│ └── Decay-Based Pruning
|
||||
├── Convergence Properties
|
||||
│ ├── Deterministic Behavior
|
||||
│ ├── Steady-State Equilibrium
|
||||
│ └── Variance Reduction
|
||||
└── Analysis Frameworks
|
||||
├── Dynamical Systems View
|
||||
├── Statistical Inference View
|
||||
└── Control Theory View
|
||||
```
|
||||
|
||||
### 1.2 Conceptual Relationships
|
||||
|
||||
```
|
||||
METAPHORICAL MAPPING (Thermodynamics → Execution):
|
||||
Thermal Energy ≈ Execution Frequency (measurable count)
|
||||
Heat Dissipation ≈ Exponential Decay (time-based reduction)
|
||||
Thermal Equilibrium ≈ Steady-State Convergence (stable metrics)
|
||||
Temperature ≈ Normalized Frequency Rank
|
||||
Cooling Rate ≈ Decay Coefficient (λ)
|
||||
|
||||
LITERAL IMPLEMENTATIONS (No Metaphor):
|
||||
Frequency Counter → Integer increment on execution
|
||||
Decay Function → f(t) = f₀ * e^(-λt)
|
||||
Hot-Words Cache → Top-K frequency-sorted entries
|
||||
Rolling Window → Circular buffer of execution records
|
||||
ANOVA Test → Statistical variance analysis (Levene's test)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## II. TAXONOMY (Hierarchical Classification)
|
||||
|
||||
### 2.1 Adaptive Runtime Components
|
||||
|
||||
```
|
||||
1. Measurement Layer
|
||||
1.1 Execution Frequency Tracking
|
||||
└── Dictionary Entry Counter (per-word increment)
|
||||
1.2 Temporal Recording
|
||||
└── Rolling Window of Truth (circular buffer)
|
||||
1.3 Transition Tracking
|
||||
└── Word-to-Word Transition Matrix
|
||||
|
||||
2. Transformation Layer
|
||||
2.1 Decay Models
|
||||
2.1.1 Linear Decay (Loop #3)
|
||||
└── Δf = -k * Δt
|
||||
2.1.2 Exponential Decay (Loop #6 Inference)
|
||||
└── f(t) = f₀ * e^(-λt)
|
||||
2.2 Normalization
|
||||
2.2.1 Frequency Ranking
|
||||
2.2.2 Probability Calculation
|
||||
|
||||
3. Inference Layer
|
||||
3.1 Window Width Inference (Loop #5)
|
||||
3.1.1 Variance Analysis (Levene's Test)
|
||||
3.1.2 Binary Search (Variance Inflection Point)
|
||||
3.2 Decay Slope Inference (Loop #6)
|
||||
3.2.1 Exponential Regression
|
||||
3.2.2 Least Squares Fitting
|
||||
|
||||
4. Actuation Layer
|
||||
4.1 Hot-Words Cache (Loop #1)
|
||||
4.1.1 Frequency-Based Sorting
|
||||
4.1.2 Top-K Selection
|
||||
4.1.3 Fast-Path Dictionary Lookup
|
||||
4.2 Speculative Execution (Loop #4)
|
||||
4.2.1 Transition Probability Calculation
|
||||
4.2.2 Prefetch Decision
|
||||
|
||||
5. Coordination Layer
|
||||
5.1 Heartbeat System (Loop #7)
|
||||
5.1.1 Time-Driven Tick Generation
|
||||
5.1.2 Loop Orchestration
|
||||
5.1.3 Adaptive Tick Rate
|
||||
```
|
||||
|
||||
### 2.2 Feedback Loop Taxonomy
|
||||
|
||||
```
|
||||
Feedback Loops (7 Total)
|
||||
├── Positive Loops (Amplifying)
|
||||
│ ├── Loop #1: Execution Heat Tracking
|
||||
│ │ └── More executions → Higher rank → More cache hits → More executions
|
||||
│ └── Loop #4: Pipelining Metrics
|
||||
│ └── More transitions → Better prediction → More prefetch hits
|
||||
├── Negative Loops (Stabilizing)
|
||||
│ ├── Loop #3: Linear Decay
|
||||
│ │ └── High frequency → Faster decay → Lower frequency → Slower decay
|
||||
│ ├── Loop #5: Window Width Inference
|
||||
│ │ └── High variance → Smaller window → Lower variance
|
||||
│ └── Loop #6: Decay Slope Inference
|
||||
│ └── Unstable metrics → Steeper decay → Faster stabilization
|
||||
├── Neutral Loops (Monitoring)
|
||||
│ └── Loop #2: Rolling Window History
|
||||
│ └── Execution occurs → Record in window → Historical data available
|
||||
└── Meta-Loop (Adaptive Coordination)
|
||||
└── Loop #7: Adaptive Heartbeat
|
||||
└── Stable system → Slower ticks → Less overhead
|
||||
```
|
||||
|
||||
### 2.3 Convergence Taxonomy
|
||||
|
||||
```
|
||||
Convergence Properties
|
||||
├── Determinism
|
||||
│ ├── Algorithmic Determinism (same inputs → same outputs)
|
||||
│ ├── Temporal Determinism (time-invariant steady state)
|
||||
│ └── Statistical Determinism (0% variance)
|
||||
├── Steady-State Characteristics
|
||||
│ ├── Fixed Point (attractors in phase space)
|
||||
│ ├── Periodic Behavior (limit cycles)
|
||||
│ └── Chaotic Behavior (sensitive dependence - avoided)
|
||||
└── Variance Metrics
|
||||
├── Inter-Run Variance (across multiple executions)
|
||||
├── Intra-Run Variance (within single execution)
|
||||
└── Temporal Variance (across time windows)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## III. LEXICON (Precise Definitions)
|
||||
|
||||
### 3.1 Core Terms (Alphabetical)
|
||||
|
||||
**Adaptive Heartbeat**
|
||||
*Definition*: Time-driven coordination mechanism that orchestrates feedback loop execution at dynamically-adjusted intervals.
|
||||
*Formal*: Thread T executing `vm_tick()` at frequency f_tick, where f_tick ∈ [f_min, f_max] adapts based on system stability.
|
||||
*Measurement*: Tick period in nanoseconds (configurable: HEARTBEAT_TICK_NS).
|
||||
*Category*: Coordination mechanism.
|
||||
|
||||
**Attractor**
|
||||
*Definition*: Stable equilibrium point or region in phase space toward which execution trajectories converge.
|
||||
*Formal*: Fixed point x* where F(x*) = x* for dynamical system x_{t+1} = F(x_t).
|
||||
*Measurement*: Coordinates in (window_size, decay_slope, variance) space.
|
||||
*Category*: Dynamical systems concept.
|
||||
|
||||
**Decay Coefficient (λ)**
|
||||
*Definition*: Rate parameter controlling exponential reduction in execution frequency over time.
|
||||
*Formal*: λ in f(t) = f₀ * e^(-λt), units of [1/time].
|
||||
*Measurement*: Derived via exponential regression; stored as Q48.16 fixed-point.
|
||||
*Category*: Transformation parameter.
|
||||
|
||||
**Deterministic Convergence**
|
||||
*Definition*: Property whereby repeated executions of identical workloads produce statistically indistinguishable steady-state metrics.
|
||||
*Formal*: ∀ executions i,j: |metric_i - metric_j| / σ < ε, where ε → 0 as t → ∞.
|
||||
*Measurement*: Coefficient of variation CV = σ/μ → 0%.
|
||||
*Category*: Convergence property.
|
||||
|
||||
**Execution Frequency**
|
||||
*Definition*: Count of times a dictionary entry has been executed since VM initialization, optionally adjusted by decay.
|
||||
*Formal*: f = Σ executions - ∫ decay(t) dt.
|
||||
*Measurement*: Unsigned 64-bit integer (`uint64_t execution_heat`).
|
||||
*Category*: Primary measurable quantity.
|
||||
*Note*: "Heat" is metaphorical naming; actual quantity is frequency.
|
||||
|
||||
**Exponential Decay**
|
||||
*Definition*: Mathematical function modeling reduction in execution frequency proportional to current value.
|
||||
*Formal*: f(t) = f₀ * e^(-λt), where f₀ is initial frequency.
|
||||
*Measurement*: Applied periodically by heartbeat system.
|
||||
*Category*: Transformation function.
|
||||
*Metaphor*: Analogous to radioactive decay or thermal dissipation.
|
||||
|
||||
**Hot-Words Cache**
|
||||
*Definition*: Fixed-size array storing pointers to the K most frequently executed dictionary entries for O(1) lookup acceleration.
|
||||
*Formal*: Cache C = {e_1, e_2, ..., e_K} where f(e_i) ≥ f(e_{i+1}) ∀ i.
|
||||
*Measurement*: Cache size K (configurable: HOTWORDS_CACHE_SIZE).
|
||||
*Category*: Frequency-based optimization.
|
||||
|
||||
**Levene's Test**
|
||||
*Definition*: Non-parametric statistical test for homogeneity of variance across groups.
|
||||
*Formal*: H₀: σ₁² = σ₂² = ... = σ_k² (variances equal).
|
||||
*Measurement*: F-statistic with p-value threshold (typically 0.05).
|
||||
*Category*: Statistical inference method.
|
||||
*Application*: Used to detect variance changes when adjusting window size.
|
||||
|
||||
**Phase Space**
|
||||
*Definition*: Multi-dimensional coordinate system where each axis represents a system state variable.
|
||||
*Formal*: Space S = {(w, λ, σ²) | w ∈ ℕ, λ ∈ ℝ⁺, σ² ∈ ℝ⁺}.
|
||||
*Measurement*: Coordinates: (window_size, decay_slope_Q48, variance_Q48).
|
||||
*Category*: Dynamical systems representation.
|
||||
|
||||
**Rolling Window of Truth**
|
||||
*Definition*: Circular buffer recording recent execution history for deterministic metric seeding.
|
||||
*Formal*: Buffer B[i] = word_id at execution event i mod |B|.
|
||||
*Measurement*: Buffer size (configurable: ROLLING_WINDOW_SIZE = 4096).
|
||||
*Category*: Temporal recording mechanism.
|
||||
*Purpose*: Ensures identical initial conditions for reproducibility.
|
||||
|
||||
**Steady-State Equilibrium**
|
||||
*Definition*: Condition where adaptive system metrics stabilize within bounded oscillation.
|
||||
*Formal*: ∃ t_0: ∀ t > t_0, |x(t) - x*| < δ for small δ.
|
||||
*Measurement*: Variance CV < 0.1% over 1000-tick window.
|
||||
*Category*: Convergence property.
|
||||
*Metaphor*: Analogous to thermodynamic equilibrium.
|
||||
|
||||
**Thermodynamic Metaphor**
|
||||
*Definition*: Conceptual mapping between thermodynamic quantities and execution metrics.
|
||||
*Mapping*:
|
||||
- Heat ↔ Execution Frequency
|
||||
- Temperature ↔ Normalized Rank
|
||||
- Cooling ↔ Exponential Decay
|
||||
- Equilibrium ↔ Steady State
|
||||
*Category*: Conceptual framework (not literal physics).
|
||||
*Warning*: Must qualify as metaphor in academic writing.
|
||||
|
||||
**Transition Probability**
|
||||
*Definition*: Conditional probability that word B is executed immediately after word A.
|
||||
*Formal*: P(B|A) = count(A→B) / count(A).
|
||||
*Measurement*: Stored as Q48.16 fixed-point in transition matrix.
|
||||
*Category*: Derived metric for speculative execution.
|
||||
|
||||
**Variance Inflection Point**
|
||||
*Definition*: Window size w* where variance begins to increase when window shrinks below w*.
|
||||
*Formal*: w* = arg min_w { Var(w) | w < w_current }.
|
||||
*Measurement*: Found via binary search with Levene's test.
|
||||
*Category*: Inference target.
|
||||
*Purpose*: Optimal window size for stable metrics.
|
||||
|
||||
**Word Transition**
|
||||
*Definition*: Sequential execution of word B immediately following word A in threaded code.
|
||||
*Formal*: Event (A, B, t) where A executes at time t and B at time t+ε.
|
||||
*Measurement*: Recorded in transition_metrics.history[] circular buffer.
|
||||
*Category*: Execution event.
|
||||
|
||||
---
|
||||
|
||||
### 3.2 Avoid/Deprecated Terms
|
||||
|
||||
| ❌ Avoid | ✅ Use Instead | Reason |
|
||||
|---------|---------------|--------|
|
||||
| "Physics-based" | "Thermodynamically-inspired metaphor" | Not literal physics |
|
||||
| "Execution heat" (in formal writing) | "Execution frequency with decay" | "Heat" is metaphorical |
|
||||
| "Temperature" | "Normalized frequency rank" | No actual thermal quantity |
|
||||
| "Quantum-inspired" | N/A | No quantum mechanics involved |
|
||||
| "AI-driven" | "Statistically-inferred" | No neural networks or ML |
|
||||
| "Learning" | "Adaptive inference" | Not machine learning |
|
||||
| "Training" | "Convergence to steady state" | Not supervised learning |
|
||||
|
||||
---
|
||||
|
||||
## IV. MATHEMATICAL FORMALISM
|
||||
|
||||
### 4.1 Execution Frequency Evolution
|
||||
|
||||
**Discrete-time update**:
|
||||
```
|
||||
f[t+1] = f[t] + Δexec[t] - Δdecay[t]
|
||||
|
||||
where:
|
||||
Δexec[t] = 1 if word executed at tick t, else 0
|
||||
Δdecay[t] = λ * f[t] * Δt (exponential decay approximation)
|
||||
```
|
||||
|
||||
**Continuous-time model**:
|
||||
```
|
||||
df/dt = r(t) - λf(t)
|
||||
|
||||
where:
|
||||
r(t) = execution rate [executions/second]
|
||||
λ = decay coefficient [1/second]
|
||||
|
||||
Solution:
|
||||
f(t) = e^(-λt) * [f₀ + ∫₀ᵗ r(τ)e^(λτ) dτ]
|
||||
```
|
||||
|
||||
### 4.2 Hot-Words Cache Selection
|
||||
|
||||
**Cache membership criterion**:
|
||||
```
|
||||
e ∈ Cache ⟺ rank(e) ≤ K
|
||||
|
||||
where:
|
||||
rank(e) = |{e' ∈ Dictionary : f(e') > f(e)}| + 1
|
||||
K = cache size (constant)
|
||||
```
|
||||
|
||||
### 4.3 Window Width Inference
|
||||
|
||||
**Objective function**:
|
||||
```
|
||||
w* = arg min { Var(w) : w ∈ [w_min, w_current] }
|
||||
|
||||
Subject to:
|
||||
Levene(w, w_current) → F-statistic
|
||||
p-value(F) < α (typically α = 0.05)
|
||||
```
|
||||
|
||||
### 4.4 Decay Slope Inference
|
||||
|
||||
**Exponential regression**:
|
||||
```
|
||||
Given: {(t_i, f_i)}_{i=1}^N from rolling window
|
||||
|
||||
Model: f(t) = f₀ * e^(-λt)
|
||||
|
||||
Log-transform: ln(f) = ln(f₀) - λt
|
||||
|
||||
Least squares:
|
||||
λ* = arg min Σ [ln(f_i) - (ln(f₀) - λt_i)]²
|
||||
```
|
||||
|
||||
### 4.5 Transition Probability
|
||||
|
||||
**Maximum likelihood estimate**:
|
||||
```
|
||||
P(B|A) = count(A→B) / count(A)
|
||||
|
||||
where:
|
||||
count(A→B) = # times B executed immediately after A
|
||||
count(A) = # times A executed
|
||||
```
|
||||
|
||||
### 4.6 Convergence Metric
|
||||
|
||||
**Coefficient of Variation**:
|
||||
```
|
||||
CV = σ / μ
|
||||
|
||||
where:
|
||||
σ = √(Var[metric]) = standard deviation
|
||||
μ = E[metric] = mean
|
||||
|
||||
Convergence achieved when: CV → 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## V. RELATIONSHIP DIAGRAM
|
||||
|
||||
### 5.1 Component Dependencies
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Execution Event │
|
||||
│ (word executed) │
|
||||
└──────────────────────┬──────────────────────────────────────┘
|
||||
│
|
||||
┌───────────┴───────────┐
|
||||
│ │
|
||||
▼ ▼
|
||||
┌────────────────┐ ┌──────────────────┐
|
||||
│ Frequency │ │ Rolling Window │
|
||||
│ Increment │ │ Recording │
|
||||
│ (Loop #1) │ │ (Loop #2) │
|
||||
└────────┬───────┘ └─────────┬────────┘
|
||||
│ │
|
||||
│ │
|
||||
▼ ▼
|
||||
┌────────────────┐ ┌──────────────────┐
|
||||
│ Decay │ │ Inference │
|
||||
│ Application │ │ Engine │
|
||||
│ (Loop #3,#6) │ │ (Loop #5,#6) │
|
||||
└────────┬───────┘ └─────────┬────────┘
|
||||
│ │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ Hot-Words Cache │
|
||||
│ Reorganization │
|
||||
└───────────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ Optimized Lookup │
|
||||
│ (O(1) cache hit) │
|
||||
└───────────────────────┘
|
||||
```
|
||||
|
||||
### 5.2 Feedback Loop Interactions
|
||||
|
||||
```
|
||||
Execution → Frequency ↑ → Cache Rank ↑ → Lookup Speed ↑ → More Execution
|
||||
▲ │
|
||||
│ │
|
||||
└─────────────── (Positive Feedback) ────────────┘
|
||||
|
||||
Frequency ↑ → Decay ↑ → Frequency ↓ → Decay ↓ → Frequency Stabilizes
|
||||
▲ │
|
||||
│ │
|
||||
└──── (Negative Feedback) ────┘
|
||||
|
||||
Variance ↑ → Window Shrink → Variance ↓ → Window Stable
|
||||
▲ │
|
||||
│ │
|
||||
└──── (Negative Feedback) ──┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VI. ONTOLOGICAL COMMITMENTS
|
||||
|
||||
### 6.1 Foundational Assumptions
|
||||
|
||||
1. **Frequency as Proxy for Importance**
|
||||
- Assumption: Frequently executed code is more important to optimize
|
||||
- Justification: Empirical validation (Zipf's law in execution patterns)
|
||||
|
||||
2. **Decay Models Temporal Relevance**
|
||||
- Assumption: Recent executions are more relevant than distant past
|
||||
- Justification: Locality of reference (temporal locality principle)
|
||||
|
||||
3. **Determinism Through Convergence**
|
||||
- Assumption: Adaptive systems can converge to deterministic steady states
|
||||
- Justification: Fixed-point theorems for contractive mappings
|
||||
|
||||
4. **Statistical Inference Validity**
|
||||
- Assumption: Execution patterns are statistically analyzable
|
||||
- Justification: Central Limit Theorem for large sample sizes
|
||||
|
||||
### 6.2 Scope Limitations
|
||||
|
||||
**What This Ontology DOES Cover**:
|
||||
- ✓ Execution frequency measurement and decay
|
||||
- ✓ Adaptive caching and inference mechanisms
|
||||
- ✓ Dynamical systems characterization
|
||||
- ✓ Statistical convergence properties
|
||||
|
||||
**What This Ontology DOES NOT Cover**:
|
||||
- ✗ Actual thermodynamic processes (metaphor only)
|
||||
- ✗ Machine learning or neural networks (no training)
|
||||
- ✗ Quantum computing (no quantum effects)
|
||||
- ✗ Biological neural systems (no biomimicry)
|
||||
|
||||
---
|
||||
|
||||
## VII. USAGE GUIDELINES
|
||||
|
||||
### 7.1 Academic Writing
|
||||
|
||||
**In Abstracts/Titles**: Use precise, non-metaphorical terms
|
||||
```
|
||||
✅ "Thermodynamically-Inspired Adaptive Runtime"
|
||||
❌ "Physics-Based Virtual Machine"
|
||||
```
|
||||
|
||||
**In Technical Sections**: Qualify metaphors explicitly
|
||||
```
|
||||
✅ "We employ a thermodynamic metaphor where execution frequency
|
||||
is treated as 'heat' that dissipates over time..."
|
||||
❌ "The physics model applies heat decay..."
|
||||
```
|
||||
|
||||
**In Formalism**: Use mathematical definitions, not analogies
|
||||
```
|
||||
✅ "Frequency evolves as f(t) = f₀ * e^(-λt)"
|
||||
❌ "Heat cools exponentially like in Newton's law"
|
||||
```
|
||||
|
||||
### 7.2 Code Comments
|
||||
|
||||
**In Source Code**: Use concrete terms from lexicon
|
||||
```c
|
||||
// ✅ GOOD: Increment execution frequency counter
|
||||
entry->execution_heat++;
|
||||
|
||||
// ❌ BAD: Increase temperature of word
|
||||
entry->execution_heat++; // heat up!
|
||||
```
|
||||
|
||||
### 7.3 Presentation/Talks
|
||||
|
||||
**Slides**: Use metaphor for intuition, then formalize
|
||||
```
|
||||
Slide 1: "Think of execution frequency like heat..."
|
||||
Slide 2: "Formally: f(t) = f₀ * e^(-λt)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VIII. REFERENCES
|
||||
|
||||
### 8.1 Foundational Concepts
|
||||
|
||||
- **Dynamical Systems**: Strogatz, S. (2015). *Nonlinear Dynamics and Chaos*
|
||||
- **Control Theory**: Åström, K. & Murray, R. (2008). *Feedback Systems*
|
||||
- **Statistical Inference**: Casella, G. & Berger, R. (2002). *Statistical Inference*
|
||||
- **Exponential Decay**: Standard mathematical function (e^(-λt))
|
||||
|
||||
### 8.2 Related Work
|
||||
|
||||
- **Trace-based JIT**: Bolz et al. (2009). "Tracing the Meta-Level: PyPy's Tracing JIT Compiler"
|
||||
- **Adaptive Systems**: Garlan et al. (2004). "Rainbow: Architecture-Based Self-Adaptation"
|
||||
- **Forth Optimization**: Ertl, M.A. (1996). "Stack Caching for Interpreters"
|
||||
|
||||
---
|
||||
|
||||
## IX. STADIUM / FABRIC VOCABULARY (v1.1 extension)
|
||||
|
||||
**Added 2026-08-13**, closing a real, acknowledged documentation-debt item: the physics
|
||||
vocabulary built on top of the original heat/decay/inference model above (patron, mass,
|
||||
density, Stadium, and the Kconfig knob names built on them) accumulated across
|
||||
`FABRIC-0.md`/`FABRIC-1.md` without an explicit statement, in one place, that these are named
|
||||
analogies — the same discipline Section III already applies to "heat," extended to the
|
||||
newer terms. Definitions below are drawn directly from `FABRIC-0.md`'s own DECIDED sections
|
||||
(cited inline), not invented here.
|
||||
|
||||
### 9.1 The metaphor, restated for this layer
|
||||
|
||||
```
|
||||
METAPHORICAL MAPPING (Stadium → Runtime state):
|
||||
Stadium ≈ Bounded live-state region (a venue, not a building)
|
||||
Patron ≈ A live thing occupying floor space in that region
|
||||
Mass ≈ How much floor space a patron occupies
|
||||
Density ≈ How much heat a patron holds per unit of floor space
|
||||
Warehouse ≈ What is not currently live (Artemis, the dictionary at rest)
|
||||
Utility ≈ Infrastructure beside the Stadium, never inside it (framebuffer)
|
||||
|
||||
LITERAL IMPLEMENTATIONS (No Metaphor):
|
||||
Stadium → A fixed-size array of cells, index-linked, allocated at boot
|
||||
Patron → A cell (or linked run of cells) with a nine-field header
|
||||
Mass → Integer count of cells a patron occupies (≥ 1)
|
||||
Density → heat ÷ mass, a derived ratio, not stored
|
||||
K → Σ execution_heat_q48 across live entries, conserved at 1.0
|
||||
```
|
||||
|
||||
The Stadium metaphor is a **venue**, not a building: §2 (`FABRIC-0.md`) is explicit that a
|
||||
patron *arrives and departs* — the Stadium holds what is live, not everything that exists.
|
||||
That is the load-bearing distinction between "Stadium" and "warehouse" below, and it is why
|
||||
the dictionary-at-rest is warehouse, not Stadium, even though every dictionary entry is
|
||||
*eligible* to become a patron.
|
||||
|
||||
### 9.2 Lexicon additions (alphabetical, same format as Section III)
|
||||
|
||||
**Cell**
|
||||
*Definition*: The Stadium's fixed-size unit of storage — a patron header, or a continuation
|
||||
cell owned by exactly one patron.
|
||||
*Formal*: A closed, two-constructor sum type: `Cell = Header | Continuation(owner)`, fixed
|
||||
at build time (`FABRIC-0.md` §3).
|
||||
*Measurement*: Proposed size 64 bytes (`FABRIC-0.md` §23.3, **not yet built** — see
|
||||
`FABRIC-1.md`'s §23.4 #2 closure: the one real patron kind actually shipped, Hermes
|
||||
messages, did not adopt this cell shape).
|
||||
*Category*: Storage unit.
|
||||
*Warning*: Not a pointer target — links between cells are Stadium-array indices, never
|
||||
addresses (`FABRIC-0.md` §3, §13).
|
||||
|
||||
**Code field**
|
||||
*Definition*: A closed-enumeration behaviour tag stored in a patron header, dispatched by
|
||||
the engine rather than branched on by kind.
|
||||
*Formal*: `behaviour ∈ {MIGRATE, DELIVER, EXPIRE, COOL, ...}` — a fixed datatype, not an
|
||||
open extension point (`FABRIC-0.md` §3, §13, §18.3).
|
||||
*Category*: Dispatch mechanism.
|
||||
*Note*: This is deliberately **the entire type system** — the engine never asks what kind
|
||||
of patron it holds, only what its code field says to do (`FABRIC-0.md` §3).
|
||||
|
||||
**Density**
|
||||
*Definition*: A patron's heat share divided by its mass — heat per unit of occupied
|
||||
Stadium floor space.
|
||||
*Formal*: `density(p) = heat(p) / mass(p)`, derived, never stored (`FABRIC-0.md` §19.2).
|
||||
*Measurement*: Used for ranking and admission (denser evicts sparser), not computed by a
|
||||
separate scheduler — it is read, not decided (`FABRIC-0.md` §19.3).
|
||||
*Category*: Derived quantity.
|
||||
*Warning*: This is the one place the thermodynamic metaphor is literal in form (quantity
|
||||
per unit volume) even though heat itself remains metaphorical.
|
||||
|
||||
**K (conservation constant)**
|
||||
*Definition*: The sum of every live entry's normalised heat share.
|
||||
*Formal*: `K = Σ execution_heat_q48`, held at `Q48_ONE` (1.0) by construction — verified by
|
||||
`vm_physics_conserved()` (`capsule_vm_physics.c`).
|
||||
*Measurement*: A correctness check, not a diagnostic — it is 1.0 by construction and can
|
||||
never itself signal system state (`FABRIC-0.md` §14, §19.1). Entropy over the heat
|
||||
*distribution* is the actual instrument; K is not.
|
||||
*Category*: Invariant.
|
||||
*Warning*: Not an occupancy ratio (`Σmass / capacity`) — that definition was considered and
|
||||
explicitly rejected because it contradicts the already-implemented, tested mechanism
|
||||
(`FABRIC-0.md` §19.1).
|
||||
|
||||
**Mass**
|
||||
*Definition*: The number of Stadium cells a patron occupies — its footprint.
|
||||
*Formal*: Integer ≥ 1, an entry field independent of heat; never enters K
|
||||
(`FABRIC-0.md` §3, §19.2).
|
||||
*Category*: Structural quantity, not conserved.
|
||||
*Note*: A large patron is not "referenced" from the Stadium — it simply occupies more
|
||||
cells. By-reference storage is reserved for things outside the Stadium entirely
|
||||
(`FABRIC-0.md` §19.3, §23.1).
|
||||
|
||||
**Mass conservation vs. heat conservation**
|
||||
*Definition*: Two independent invariants that must not be confused. Heat (K) is conserved
|
||||
globally and moved by traffic. Mass is conserved *per patron* during a residency —
|
||||
identity and mass never change as a side effect of use — but is not a fleet-wide summed
|
||||
invariant the way heat is (`FABRIC-0.md` §24.2).
|
||||
|
||||
**Patron**
|
||||
*Definition*: A live thing occupying a Stadium cell or linked run of cells — the unit the
|
||||
Stadium actually holds.
|
||||
*Formal*: One structure, no variants, no type field — identified by its code field's
|
||||
behaviour, not by a kind tag (`FABRIC-0.md` §3, §13).
|
||||
*Measurement*: Nine header wires: identity, heat, TTL, pin, link, code field, mass,
|
||||
payload, contains (`FABRIC-0.md` §3, with amendments cited in place).
|
||||
*Category*: The Stadium's sole occupant type.
|
||||
*Known kinds (as of this writing)*: words (dictionary — fully migrated and live via
|
||||
`stadium_word_dispatch()`, `FABRIC-0.md` §16.3/§17.7), blocks (Artemis), messages (Hermes — shipped with a
|
||||
**different** shape than this header, see `FABRIC-1.md`'s §23.4 #2 closure), VMs (Tripod,
|
||||
`FABRIC-0.md` §20), dirty events (console, `FABRIC-0.md` §17.5). Five kinds, not four — an
|
||||
earlier draft undercounted (`FABRIC-0.md` §1).
|
||||
|
||||
**Reap**
|
||||
*Definition*: The event that ends a patron's residency — not destruction, a return of its
|
||||
cells and remaining heat to the pool.
|
||||
*Formal*: Consumed on delivery (messages), on redraw (dirty events), on cooling below floor
|
||||
(VMs), or on eviction by a denser newcomer (`FABRIC-0.md` §17.2, §17.5, §19.3).
|
||||
*Category*: Lifecycle event.
|
||||
|
||||
**Stadium**
|
||||
*Definition*: A single, bounded region of memory outside any VM, holding everything
|
||||
currently live.
|
||||
*Formal*: A fixed-size array of cells, index-linked (not pointer-linked), allocated at
|
||||
boot before any VM exists, not part of the heap (`FABRIC-0.md` §2, §17.6).
|
||||
*Measurement*: Per-VM quota over a single global cell pool, elastic under pressure,
|
||||
arbitrated by Hera (`FABRIC-0.md` §22) — not per-VM physical regions.
|
||||
*Category*: The bounded live-state substrate this whole vocabulary describes.
|
||||
*Warning*: "Live" is the operative word — the Stadium holds what is live, not everything
|
||||
that exists. A dictionary word sitting cold and unexecuted is warehouse, not Stadium, even
|
||||
once the dictionary itself migrates into this representation (`FABRIC-0.md` §2, §16.3).
|
||||
|
||||
**Warehouse**
|
||||
*Definition*: What lies beneath the Stadium — storage for things not currently live.
|
||||
*Formal*: Artemis (block storage) and the resting dictionary are warehouse; the Stadium is
|
||||
the floor above it (`FABRIC-0.md` §17.5's three-category taxonomy: warehouse / Stadium /
|
||||
utility).
|
||||
*Category*: Storage layer, outside the engine's live-state model.
|
||||
|
||||
**Utility**
|
||||
*Definition*: External infrastructure the Stadium consumes but never contains — beside the
|
||||
building, not inside or beneath it.
|
||||
*Formal*: The framebuffer and devices generally; the engine never learns about a utility's
|
||||
existence, which is what makes it a boundary rather than an exception (`FABRIC-0.md` §17.5).
|
||||
*Category*: Third taxonomy category, coordinate with warehouse and Stadium, not a
|
||||
sub-case of either.
|
||||
|
||||
### 9.3 Kconfig knob names built on this vocabulary
|
||||
|
||||
Per Bob's own flag: code, Kconfig, and documentation terminology must not drift apart.
|
||||
The table below is the current, verified wiring status of every `STADIUM_*` Kconfig
|
||||
symbol — confirmed by direct source grep during the `FABRIC-1.md` closure work this
|
||||
vocabulary accompanies, not assumed from the symbol's name or Kconfig help text alone.
|
||||
|
||||
| Kconfig symbol | Vocabulary term | Wiring status (2026-08-13) |
|
||||
|---|---|---|
|
||||
| `STADIUM_MAX_VM_COUNT` | Patron (VM kind) population bound | **Live** — read in `stadium.c`, `stadium_words.c`, `capsule_birth.c` |
|
||||
| `STADIUM_MEMORY_PERCENT` | Stadium capacity sizing | **Live** — read in `stadium.c` |
|
||||
| `STADIUM_WORD_HEAT_QUANTUM` | Heat (word-patron admission) | **Live** — read in `stadium_words.c` |
|
||||
| `STADIUM_WORD_COOL_RATE_Q48` | Heat decay rate (word patrons) | **Live** — read in `stadium_words.c` |
|
||||
| `STADIUM_CONTAINS_DEPTH_MAX` | `contains` wire depth cap (§3, §8) | **Declared, not yet enforced** — reap-gating enforcement is item 3.5's scope, not implemented (`FABRIC-1.md` §A item 5.3 area; not a bug, a known sequencing gap) |
|
||||
| `STADIUM_CAPACITY_TICK` | Capacity-loop cadence, distinct from heat's (§22.4) | **Dead** — declared with the correct intended default (1000, matching §22.4's precedent) but never read by any code path; only appearance beyond its own default is a compile-time `> 0` static assertion in `stadium.h`. Found and reported during this session's §12 Q5 closure (`FABRIC-1.md`) — `vm_physics_heartbeat_tick()` still literally shares `HEARTBEAT_INFERENCE_FREQUENCY` instead |
|
||||
|
||||
This table is a snapshot, not a promise — it will drift the moment any of these symbols is
|
||||
wired or unwired. Re-verify against source before relying on it for anything past a casual
|
||||
read; do not assume "Live" here still means live without checking.
|
||||
|
||||
---
|
||||
|
||||
## X. VERSION HISTORY
|
||||
|
||||
**v1.1** (2026-08-13):
|
||||
- Added Section IX: Stadium/Fabric vocabulary (patron, mass, density, Stadium, warehouse,
|
||||
utility, K, cell, code field, reap) as named analogies, matching Section III's existing
|
||||
discipline for the original heat/decay vocabulary
|
||||
- Added a Kconfig knob-name-to-concept mapping table with verified (not assumed) wiring
|
||||
status, per Captain Bob's 2026-08-04 flag that code/Kconfig/documentation terminology
|
||||
must not drift apart
|
||||
|
||||
**v1.0** (2025-12-13):
|
||||
- Initial ontology, taxonomy, and lexicon
|
||||
- Formal mathematical definitions
|
||||
- Relationship diagrams
|
||||
- Usage guidelines
|
||||
|
||||
---
|
||||
|
||||
## XI. ACKNOWLEDGMENTS
|
||||
|
||||
This ontology provides the formal conceptual framework for the StarForth project.
|
||||
It aims to eliminate ambiguity and enable precise academic discourse while
|
||||
acknowledging the metaphorical nature of certain conceptual mappings.
|
||||
|
||||
**License**: See ./LICENSE
|
||||
+663
@@ -0,0 +1,663 @@
|
||||
# StarForth Roadmap: VM → Kernel → OS → FPGA
|
||||
|
||||
> **OBSOLETE (flagged 2026-08-26, Captain Bob's call — this is more than "stale," the
|
||||
> architecture, branch topology, and terminology it describes no longer exist).** This is a
|
||||
> StarForth-era plan dated 2025-12-14, written before the repo split, before
|
||||
> Tripod/Stadium/word-level ACL existed, and before "Phase 1 Starting (HAL)" had any of the
|
||||
> meaning it has now. Kept for historical record only. For current status, see
|
||||
> `docs/lithosananke/ROADMAP.md` (repo-specific, LithosAnanke roadmap) and
|
||||
> `FABRIC-0.md`/`FABRIC-1.md`/`FABRIC-2.md` (design history and current work) — not this
|
||||
> document.
|
||||
|
||||
**Version**: 1.0
|
||||
**Date**: 2025-12-14
|
||||
**Timeline**: 2025-2028 (3.5 years)
|
||||
**Status**: Phase 0 Complete (VM), Phase 1 Starting (HAL)
|
||||
|
||||
---
|
||||
|
||||
## VISION
|
||||
|
||||
**From**: Adaptive FORTH VM running on hosted OS (Linux)
|
||||
**To**: Self-hosting operating system running on custom FPGA hardware
|
||||
|
||||
```
|
||||
2025 (NOW) 2026 2027 2028
|
||||
│ │ │ │
|
||||
V V V V
|
||||
┌─────────┐ ┌──────────┐ ┌────────────┐ ┌──────────┐
|
||||
│StarForth│ → │StarKernel│ → │ StarshipOS │ → │ FPGA │
|
||||
│ VM │ │(Bare Metal) │(Self-hosting) │ Hardware │
|
||||
└─────────┘ └──────────┘ └────────────┘ └──────────┘
|
||||
COMPLETE 6-8 months 12-18 months 12-18 months
|
||||
```
|
||||
|
||||
**Key Principle**: Each phase builds on the previous. No skipping steps.
|
||||
|
||||
---
|
||||
|
||||
## CURRENT STATE (Phase 0: COMPLETE ✅)
|
||||
|
||||
### What We Have
|
||||
|
||||
**StarForth VM** - Production-ready adaptive FORTH interpreter:
|
||||
- ✅ FORTH-79 compliant (780+ tests passing)
|
||||
- ✅ Adaptive runtime (7 feedback loops, 0% algorithmic variance)
|
||||
- ✅ Platforms: Linux (HISTORICAL: also L4Re/Fiasco.OC through mid-2026; removed as an active target)
|
||||
- ✅ Performance: 100M iterations/sec, profile-guided optimization
|
||||
- ✅ Quality: 9.2/10 codebase rating, strict ANSI C99
|
||||
- ✅ Validation: Peer-reviewed, 90-run statistical proof
|
||||
- ✅ Patent pending: Adaptive runtime mechanisms
|
||||
|
||||
**Documentation**:
|
||||
- ✅ 121-page academic paper
|
||||
- ✅ Scientific defense documents (9 files)
|
||||
- ✅ Ontology & formal definitions
|
||||
- ✅ HAL architecture (documented, not implemented)
|
||||
|
||||
**Infrastructure**:
|
||||
- ✅ Make-based build system (40+ targets)
|
||||
- ✅ DoE (Design of Experiments) framework
|
||||
- ✅ Reproducibility guarantees
|
||||
- ✅ CI-ready test suite
|
||||
|
||||
### What's Missing
|
||||
|
||||
**See**: [docs/GAP_ANALYSIS.md](docs/GAP_ANALYSIS.md) for complete list (37 gaps identified)
|
||||
|
||||
**Critical Blockers**:
|
||||
1. HAL abstraction layer (8 components) - **Blocks everything**
|
||||
2. StarKernel implementation (13 components)
|
||||
3. StarshipOS subsystems (12 components)
|
||||
4. FPGA feasibility & design (19 components)
|
||||
|
||||
---
|
||||
|
||||
## PHASE 1: HAL IMPLEMENTATION (Months 1-2)
|
||||
|
||||
**Goal**: Make StarForth platform-agnostic via Hardware Abstraction Layer
|
||||
|
||||
**Why**: HAL is the foundation for all future work. Without HAL, cannot build kernel or port to FPGA.
|
||||
|
||||
### Deliverables
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 1.1 | Define HAL interfaces (6 headers) | 500 | 1 week | 📋 Planned |
|
||||
| 1.2 | Implement Linux HAL | 1500 | 2 weeks | 📋 Planned |
|
||||
| 1.3 | Migrate VM core to HAL | 800 | 2 weeks | 📋 Planned |
|
||||
| 1.4 | Migrate physics subsystems | 300 | 1 week | 📋 Planned |
|
||||
| 1.5 | Validate determinism (DoE) | Testing | 2 days | 📋 Planned |
|
||||
|
||||
**Total**: ~3,100 LOC, 6-7 weeks
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- ✅ All HAL interface headers exist (`hal_time.h`, `hal_interrupt.h`, etc.)
|
||||
- ✅ Linux HAL implementation complete
|
||||
- ✅ VM core has zero direct platform dependencies
|
||||
- ✅ All 780+ tests pass on HAL
|
||||
- ✅ 0% algorithmic variance maintained
|
||||
- ✅ No measurable performance regression
|
||||
|
||||
### Milestones
|
||||
|
||||
**Week 2**: HAL headers defined, Linux implementation started
|
||||
**Week 4**: Linux HAL complete, VM migration started
|
||||
**Week 6**: All tests passing, determinism validated
|
||||
|
||||
**Blocker Risk**: Low (well-understood refactoring)
|
||||
|
||||
---
|
||||
|
||||
## PHASE 2: STARKERNEL (Months 3-5)
|
||||
|
||||
**Goal**: Boot StarForth on bare metal (UEFI → kernel → "ok" prompt)
|
||||
|
||||
**Why**: Kernel enables true OS development and demonstrates platform independence.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ UEFI Firmware │
|
||||
│ • Memory map, ACPI, framebuffer │
|
||||
└────────────────┬────────────────────────────┘
|
||||
│ ExitBootServices()
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ StarKernel Boot (boot/uefi_loader.c) │
|
||||
│ • Collect BootInfo │
|
||||
│ • Jump to kernel_main() │
|
||||
└────────────────┬────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ StarKernel HAL (platform/kernel/) │
|
||||
│ • CPU init (GDT, IDT, interrupts) │
|
||||
│ • Memory (PMM, VMM, heap) │
|
||||
│ • Time (TSC, HPET, APIC timer) │
|
||||
│ • Console (UART + framebuffer) │
|
||||
└────────────────┬────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ StarForth VM │
|
||||
│ • vm_create() │
|
||||
│ • Physics subsystems │
|
||||
│ • REPL → "ok" prompt │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 2.1 | UEFI boot loader | 800 | 2 weeks | 📋 Planned |
|
||||
| 2.2 | Physical memory manager (PMM) | 1200 | 2 weeks | 📋 Planned |
|
||||
| 2.3 | Virtual memory manager (VMM) | 1500 | 2 weeks | 📋 Planned |
|
||||
| 2.4 | Kernel heap (kmalloc) | 800 | 1 week | 📋 Planned |
|
||||
| 2.5 | CPU initialization (GDT/IDT) | 1000 | 1 week | 📋 Planned |
|
||||
| 2.6 | UART driver (serial console) | 600 | 1 week | 📋 Planned |
|
||||
| 2.7 | Time subsystem (TSC/HPET) | 1000 | 2 weeks | 📋 Planned |
|
||||
| 2.8 | APIC interrupt controller | 1200 | 2 weeks | 📋 Planned |
|
||||
| 2.9 | Framebuffer driver | 800 | 1 week | 📋 Planned |
|
||||
| 2.10 | ACPI parsing | 600 | 1 week | 📋 Planned |
|
||||
| 2.11 | HAL implementation (kernel) | 1000 | 2 weeks | 📋 Planned |
|
||||
| 2.12 | Build system integration | 500 | 1 week | 📋 Planned |
|
||||
|
||||
**Total**: ~11,000 LOC, 12-14 weeks
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- ✅ UEFI boots on QEMU + OVMF
|
||||
- ✅ Serial console prints "StarKernel booting..."
|
||||
- ✅ PMM/VMM allocate memory correctly
|
||||
- ✅ Kernel heap works (kmalloc/kfree)
|
||||
- ✅ Interrupts and timers functional
|
||||
- ✅ VM starts and prints "ok" prompt
|
||||
- ✅ Can execute simple FORTH code (`1 2 + .` → `3`)
|
||||
- ✅ Physics subsystems operational on bare metal
|
||||
|
||||
### Milestones
|
||||
|
||||
**Month 3**: UEFI boot + serial output working
|
||||
**Month 4**: Memory managers complete, VM starts
|
||||
**Month 5**: Full HAL working, "ok" prompt on QEMU
|
||||
|
||||
**Blocker Risk**: Medium (complex low-level code, but well-documented)
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
**Development**: QEMU with OVMF (UEFI firmware)
|
||||
**Validation**: Physical hardware (Intel NUC or similar)
|
||||
**Automation**: QEMU headless testing in CI
|
||||
|
||||
---
|
||||
|
||||
## PHASE 3: STARSHIPOS (Months 6-18)
|
||||
|
||||
**Goal**: Self-hosting operating system (compiles itself, runs applications)
|
||||
|
||||
**Why**: Demonstrates complete independence from Linux/POSIX, enables FORTH-native ecosystem.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ StarshipOS Layers │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ Userland │
|
||||
│ • Shell, editors, compilers, tools │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ System Services │
|
||||
│ • Filesystem (FAT32 or custom) │
|
||||
│ • Networking (TCP/IP stack) │
|
||||
│ • Process manager, scheduler │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ Drivers │
|
||||
│ • Storage (AHCI, NVMe) │
|
||||
│ • Network (VirtIO, E1000) │
|
||||
│ • Block/char device abstraction │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ StarKernel (from Phase 2) │
|
||||
│ • CPU, memory, time, interrupts │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Deliverables (Staged)
|
||||
|
||||
#### Stage 3A: Storage (Months 6-9)
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 3.1 | AHCI driver | 2500 | 1 month | 📋 Planned |
|
||||
| 3.2 | NVMe driver | 2000 | 1 month | 📋 Planned |
|
||||
| 3.3 | Block device abstraction | 1500 | 2 weeks | 📋 Planned |
|
||||
| 3.4 | FAT32 filesystem | 3000 | 1 month | 📋 Planned |
|
||||
| 3.5 | VFS layer | 2000 | 2 weeks | 📋 Planned |
|
||||
|
||||
**Subtotal**: ~11,000 LOC, 3-4 months
|
||||
|
||||
**Success Criteria**:
|
||||
- ✅ Read/write files to disk
|
||||
- ✅ Boot from hard drive (not just RAM)
|
||||
- ✅ Persistent FORTH definitions
|
||||
|
||||
---
|
||||
|
||||
#### Stage 3B: Networking (Months 10-12)
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 3.6 | VirtIO-net driver | 2000 | 1 month | 📋 Planned |
|
||||
| 3.7 | E1000 driver | 1500 | 2 weeks | 📋 Planned |
|
||||
| 3.8 | lwIP TCP/IP stack (port) | 8000 | 2 months | 📋 Planned |
|
||||
| 3.9 | Socket API (FORTH-native) | 1500 | 2 weeks | 📋 Planned |
|
||||
|
||||
**Subtotal**: ~13,000 LOC, 3-4 months
|
||||
|
||||
**Success Criteria**:
|
||||
- ✅ Send/receive packets
|
||||
- ✅ TCP connections work
|
||||
- ✅ Simple HTTP server in FORTH
|
||||
|
||||
---
|
||||
|
||||
#### Stage 3C: Process Model (Months 13-15)
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 3.10 | FORTH task abstraction | 2000 | 1 month | 📋 Planned |
|
||||
| 3.11 | Scheduler (round-robin) | 1500 | 2 weeks | 📋 Planned |
|
||||
| 3.12 | IPC mechanism | 2500 | 1 month | 📋 Planned |
|
||||
| 3.13 | System call interface | 1000 | 2 weeks | 📋 Planned |
|
||||
|
||||
**Subtotal**: ~7,000 LOC, 2-3 months
|
||||
|
||||
**Success Criteria**:
|
||||
- ✅ Multiple concurrent tasks
|
||||
- ✅ Task switching works
|
||||
- ✅ Tasks can communicate (send/receive)
|
||||
|
||||
---
|
||||
|
||||
#### Stage 3D: Self-Hosting (Months 16-18)
|
||||
|
||||
| # | Component | LOC | Duration | Status |
|
||||
|---|-----------|-----|----------|--------|
|
||||
| 3.14 | FORTH compiler (meta-compilation) | 3000 | 1 month | 📋 Planned |
|
||||
| 3.15 | Assembler (FORTH-hosted) | 2000 | 2 weeks | 📋 Planned |
|
||||
| 3.16 | Linker/loader | 1500 | 2 weeks | 📋 Planned |
|
||||
| 3.17 | Shell & core utilities | 2500 | 1 month | 📋 Planned |
|
||||
| 3.18 | Build system (FORTH-based) | 1000 | 2 weeks | 📋 Planned |
|
||||
|
||||
**Subtotal**: ~10,000 LOC, 2-3 months
|
||||
|
||||
**Success Criteria**:
|
||||
- ✅ StarshipOS compiles itself
|
||||
- ✅ No dependency on Linux toolchain
|
||||
- ✅ Can build applications entirely in FORTH
|
||||
|
||||
---
|
||||
|
||||
**Phase 3 Total**: ~41,000 LOC, 12-15 months
|
||||
|
||||
### Milestones
|
||||
|
||||
**Month 9**: Persistent storage working, boot from disk
|
||||
**Month 12**: Network stack operational, HTTP demo
|
||||
**Month 15**: Multitasking working, IPC functional
|
||||
**Month 18**: Self-hosting achieved, OS compiles itself
|
||||
|
||||
**Blocker Risk**: High (large scope, complex subsystems)
|
||||
|
||||
**De-risking Strategy**: Port existing code where possible (lwIP for TCP/IP, FAT32 reference)
|
||||
|
||||
---
|
||||
|
||||
## PHASE 4: FPGA FEASIBILITY (Months 6-7)
|
||||
|
||||
**Goal**: Determine if FPGA implementation is worth pursuing
|
||||
|
||||
**Why**: FPGA is a massive investment (12-18 months). Need go/no-go decision before committing.
|
||||
|
||||
### Feasibility Study
|
||||
|
||||
| # | Task | Duration | Deliverable |
|
||||
|---|------|----------|-------------|
|
||||
| 4.1 | Survey existing FORTH FPGA cores | 1 week | Comparison table |
|
||||
| 4.2 | Estimate resource usage (LUTs, BRAMs) | 3 days | Resource budget |
|
||||
| 4.3 | Select target FPGA | 3 days | FPGA selection rationale |
|
||||
| 4.4 | Prototype stack machine (Verilog) | 2 weeks | Minimal stack machine |
|
||||
| 4.5 | Synthesize & test on FPGA board | 1 week | Synthesis report |
|
||||
| 4.6 | Performance analysis | 3 days | Speed vs. software |
|
||||
| 4.7 | Cost/benefit analysis | 2 days | Go/no-go recommendation |
|
||||
|
||||
**Total**: 5-6 weeks
|
||||
|
||||
### Decision Criteria
|
||||
|
||||
**GO if**:
|
||||
- ✅ Fits in affordable FPGA (~$200-500 board)
|
||||
- ✅ Performance ≥ 50 MHz clock (faster than software?)
|
||||
- ✅ Adaptive runtime feasible in hardware (critical!)
|
||||
- ✅ Development time justified by benefits
|
||||
|
||||
**NO-GO if**:
|
||||
- ❌ Too large for affordable FPGAs
|
||||
- ❌ Slower than software implementation
|
||||
- ❌ Adaptive runtime not synthesizable
|
||||
- ❌ >18 months to implement (opportunity cost too high)
|
||||
|
||||
### Prototype Spec
|
||||
|
||||
**Minimal Stack Machine** (proof of concept):
|
||||
- Data stack (16 entries × 64-bit)
|
||||
- Return stack (16 entries × 64-bit)
|
||||
- Basic operations: +, -, *, /, DUP, DROP, SWAP
|
||||
- Dictionary ROM (256 entries)
|
||||
- Simple UART I/O
|
||||
|
||||
**Target**: Prove concept works in hardware, get resource usage numbers
|
||||
|
||||
---
|
||||
|
||||
## PHASE 5: FPGA IMPLEMENTATION (Months 8-24)
|
||||
|
||||
**Status**: CONDITIONAL (only if Phase 4 says GO)
|
||||
|
||||
**Goal**: Hardware FORTH processor on FPGA
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────┐
|
||||
│ FPGA Top Module │
|
||||
├────────────────────────────────────────────────┤
|
||||
│ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Stack Machine│ │ Memory │ │
|
||||
│ │ • Data stack│ │ Controller │ │
|
||||
│ │ • Return stk│ │ • Dict ROM │ │
|
||||
│ │ • ALU │ │ • RAM │ │
|
||||
│ └──────────────┘ └──────────────┘ │
|
||||
├────────────────────────────────────────────────┤
|
||||
│ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ UART │ │ Timers │ │
|
||||
│ │ IP Core │ │ Counters │ │
|
||||
│ └──────────────┘ └──────────────┘ │
|
||||
├────────────────────────────────────────────────┤
|
||||
│ ┌──────────────────────────────────────────┐ │
|
||||
│ │ Adaptive Runtime (if feasible) │ │
|
||||
│ │ • Execution counters │ │
|
||||
│ │ • Hot-word cache │ │
|
||||
│ │ • Hardware decay (?) │ │
|
||||
│ └──────────────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
#### Stage 5A: Core Implementation (Months 8-13)
|
||||
|
||||
| # | Component | LOC (HDL) | Duration | Status |
|
||||
|---|-----------|-----------|----------|--------|
|
||||
| 5.1 | Data stack (16 deep) | 500 | 2 weeks | 📋 Conditional |
|
||||
| 5.2 | Return stack (16 deep) | 500 | 2 weeks | 📋 Conditional |
|
||||
| 5.3 | ALU (arithmetic/logic ops) | 2000 | 1 month | 📋 Conditional |
|
||||
| 5.4 | Dictionary memory (ROM) | 1000 | 2 weeks | 📋 Conditional |
|
||||
| 5.5 | RAM controller | 1500 | 3 weeks | 📋 Conditional |
|
||||
| 5.6 | Instruction decoder | 2000 | 1 month | 📋 Conditional |
|
||||
| 5.7 | Control FSM | 1500 | 3 weeks | 📋 Conditional |
|
||||
|
||||
**Subtotal**: ~9,000 lines Verilog, 4-5 months
|
||||
|
||||
---
|
||||
|
||||
#### Stage 5B: I/O & Peripherals (Months 14-16)
|
||||
|
||||
| # | Component | LOC (HDL) | Duration | Status |
|
||||
|---|-----------|-----------|----------|--------|
|
||||
| 5.8 | UART transmitter/receiver | 800 | 2 weeks | 📋 Conditional |
|
||||
| 5.9 | Timer/counter IP cores | 600 | 1 week | 📋 Conditional |
|
||||
| 5.10 | Interrupt controller | 1000 | 2 weeks | 📋 Conditional |
|
||||
| 5.11 | Memory-mapped I/O bus | 1200 | 3 weeks | 📋 Conditional |
|
||||
|
||||
**Subtotal**: ~3,600 lines Verilog, 2-3 months
|
||||
|
||||
---
|
||||
|
||||
#### Stage 5C: Adaptive Runtime (Months 17-20)
|
||||
|
||||
**CRITICAL UNCERTAINTY**: Can we synthesize this?
|
||||
|
||||
| # | Component | LOC (HDL) | Duration | Status |
|
||||
|---|-----------|-----------|----------|--------|
|
||||
| 5.12 | Execution frequency counters | 1500 | 1 month | 📋 Conditional |
|
||||
| 5.13 | Hot-word cache (top-K) | 2000 | 1 month | 📋 Conditional |
|
||||
| 5.14 | Hardware decay logic | 1000 | 2 weeks | 📋 Conditional |
|
||||
| 5.15 | Transition tracking | 1500 | 3 weeks | 📋 Conditional |
|
||||
|
||||
**Subtotal**: ~6,000 lines Verilog, 3-4 months
|
||||
|
||||
**Fallback**: If adaptive runtime doesn't synthesize well, implement in software on soft-core CPU (MicroBlaze/NIOS II) alongside FORTH core.
|
||||
|
||||
---
|
||||
|
||||
#### Stage 5D: Validation & Optimization (Months 21-24)
|
||||
|
||||
| # | Task | Duration | Deliverable |
|
||||
|---|------|----------|-------------|
|
||||
| 5.16 | Testbench (all modules) | 2 months | 100% coverage |
|
||||
| 5.17 | Formal verification (critical paths) | 1 month | Proofs |
|
||||
| 5.18 | Synthesis optimization | 1 month | Timing closure |
|
||||
| 5.19 | Hardware-in-loop testing | 2 weeks | Physical validation |
|
||||
| 5.20 | Performance tuning | 2 weeks | Final optimizations |
|
||||
|
||||
**Subtotal**: 4-5 months
|
||||
|
||||
---
|
||||
|
||||
**Phase 5 Total**: ~18,600 lines Verilog + testbenches, 15-18 months
|
||||
|
||||
### Target FPGAs (Options)
|
||||
|
||||
| FPGA | Cost | Resources | Speed | Notes |
|
||||
|------|------|-----------|-------|-------|
|
||||
| **Xilinx Zynq-7000** | $300 | 85K LUTs, ARM core | 200 MHz | **Recommended**: Hybrid ARM+FPGA |
|
||||
| Intel Cyclone V | $250 | 77K LEs | 150 MHz | Good alternative |
|
||||
| Lattice iCE40 UP5K | $50 | 5K LUTs | 50 MHz | Budget option, slower |
|
||||
| Xilinx Artix-7 | $200 | 33K LUTs | 200 MHz | Pure FPGA, no ARM |
|
||||
|
||||
**Recommendation**: Zynq-7000 (ARM + FPGA fabric)
|
||||
- Run StarshipOS on ARM
|
||||
- Offload FORTH core to FPGA fabric
|
||||
- Best of both worlds
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- ✅ FPGA boots, prints "ok" via UART
|
||||
- ✅ Can execute FORTH code
|
||||
- ✅ Performance ≥ software on same workload
|
||||
- ✅ Adaptive runtime functional (or gracefully degraded)
|
||||
- ✅ 780+ tests pass on hardware
|
||||
- ✅ 0% algorithmic variance maintained (if adaptive works)
|
||||
|
||||
### Milestones
|
||||
|
||||
**Month 13**: Core stack machine working in simulation
|
||||
**Month 16**: I/O functional, boots on real FPGA
|
||||
**Month 20**: Adaptive runtime implemented (or fallback decided)
|
||||
**Month 24**: Full validation, production-ready FPGA core
|
||||
|
||||
**Blocker Risk**: VERY HIGH
|
||||
- HDL expertise required
|
||||
- Timing closure can be unpredictable
|
||||
- Adaptive runtime may not be synthesizable
|
||||
- Resource constraints on affordable FPGAs
|
||||
|
||||
---
|
||||
|
||||
## INTEGRATION TIMELINE
|
||||
|
||||
### Parallel Paths
|
||||
|
||||
**Months 1-5**: Sequential (HAL → Kernel)
|
||||
**Months 6-18**: **Parallel** (OS + FPGA feasibility)
|
||||
**Months 8-24**: **Conditional** (FPGA impl if feasible)
|
||||
|
||||
```
|
||||
2025 2026 2027 2028
|
||||
│ │ │ │
|
||||
V V V V
|
||||
Phase 1 Phase 2 Phase 3 Phase 5
|
||||
HAL Kernel OS FPGA
|
||||
(2m) (3m) ┌──────────────────(12m)────────────┐ (16m)
|
||||
│ │ │ │
|
||||
│ Phase 4: Feasibility (2m) │ │
|
||||
│ └─────┬─────────────────────────────┘ │
|
||||
│ │ │
|
||||
└────────────────┴──GO/NO-GO Decision───────────────┘
|
||||
│
|
||||
If NO-GO: Focus on OS only
|
||||
If GO: Parallel OS + FPGA work
|
||||
```
|
||||
|
||||
### Resource Allocation (Solo Development)
|
||||
|
||||
**Months 1-5** (HAL + Kernel): 100% focus
|
||||
**Months 6-7** (Feasibility): 80% OS, 20% FPGA study
|
||||
**Months 8-18** (Parallel): 70% OS, 30% FPGA (if GO)
|
||||
**Months 19-24** (FPGA finish): 50% OS, 50% FPGA
|
||||
|
||||
**Community Contributions**: Welcomed at all stages (see CONTRIBUTING.md)
|
||||
|
||||
---
|
||||
|
||||
## RISK MITIGATION
|
||||
|
||||
### High-Risk Items
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|-------------|--------|------------|
|
||||
| **HAL breaks determinism** | Low | Critical | Extensive testing, DoE validation |
|
||||
| **Kernel never boots** | Medium | Critical | Incremental dev, QEMU testing |
|
||||
| **FPGA too slow** | Medium | High | Feasibility study gates investment |
|
||||
| **Adaptive runtime unsynthesizable** | High | High | Software fallback on ARM core |
|
||||
| **Project too large (solo)** | High | Medium | Community outreach, phased delivery |
|
||||
| **FPGA cost too high** | Low | Medium | Target affordable boards ($200-500) |
|
||||
|
||||
### Fallback Plans
|
||||
|
||||
**If HAL migration fails**: Continue hosted-only, defer kernel
|
||||
**If OS scope too large**: Deliver minimal components (storage + network only)
|
||||
**If FPGA infeasible**: Software-only path is still valuable
|
||||
|
||||
---
|
||||
|
||||
## DELIVERABLES SUMMARY
|
||||
|
||||
### By End of 2025 (Months 1-2)
|
||||
- ✅ HAL implementation complete
|
||||
- ✅ StarForth platform-agnostic
|
||||
- ✅ All tests passing on HAL
|
||||
|
||||
### By Mid-2026 (Months 3-5)
|
||||
- ✅ StarKernel boots to "ok" on QEMU
|
||||
- ✅ Physics subsystems work on bare metal
|
||||
|
||||
### By End of 2026 (Months 6-12)
|
||||
- ✅ Storage subsystem operational
|
||||
- ✅ Network stack functional
|
||||
- ✅ FPGA feasibility determined (GO/NO-GO)
|
||||
|
||||
### By Mid-2027 (Months 13-18)
|
||||
- ✅ Process model working
|
||||
- ✅ StarshipOS self-hosting
|
||||
- ✅ (If FPGA GO): Prototype FPGA core functional
|
||||
|
||||
### By End of 2027 (Months 19-24)
|
||||
- ✅ StarshipOS production-ready
|
||||
- ✅ (If FPGA GO): FPGA core validated
|
||||
|
||||
### By 2028 (Future)
|
||||
- 🎯 Ecosystem growth (applications, libraries)
|
||||
- 🎯 FPGA optimizations & variants
|
||||
- 🎯 Community-driven features
|
||||
|
||||
---
|
||||
|
||||
## SUCCESS METRICS
|
||||
|
||||
### Technical Metrics
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
|--------|--------|-------------|
|
||||
| **HAL overhead** | <5% | Benchmark comparison |
|
||||
| **Kernel boot time** | <1 second | QEMU timing |
|
||||
| **OS compile speed** | <10 min | Self-hosting build |
|
||||
| **FPGA clock speed** | ≥50 MHz | Synthesis report |
|
||||
| **Test pass rate** | 100% (780+) | All platforms |
|
||||
| **Determinism** | 0% CV | DoE validation |
|
||||
|
||||
### Project Health Metrics
|
||||
|
||||
| Metric | Target | Current |
|
||||
|--------|--------|---------|
|
||||
| **Code quality** | ≥9.0/10 | 9.2/10 ✅ |
|
||||
| **Test coverage** | ≥80% | ~90% ✅ |
|
||||
| **Documentation** | Complete | Strong ✅ |
|
||||
| **Community** | ≥10 contributors | 1 (solo) |
|
||||
| **GitHub stars** | ≥500 | TBD |
|
||||
|
||||
---
|
||||
|
||||
## COMMUNITY & ECOSYSTEM
|
||||
|
||||
### How to Contribute
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
||||
|
||||
**High-value contributions**:
|
||||
1. HAL implementations (new platforms)
|
||||
2. Kernel drivers (AHCI, NVMe, network)
|
||||
3. FPGA expertise (Verilog, synthesis)
|
||||
4. Testing & validation
|
||||
5. Documentation improvements
|
||||
|
||||
### Seeking Collaborators
|
||||
|
||||
**Needed expertise**:
|
||||
- Kernel development (UEFI, memory management)
|
||||
- FPGA design (Verilog/VHDL)
|
||||
- OS design (filesystems, networking)
|
||||
- Formal verification (HDL proofs)
|
||||
|
||||
**Contact**: rajames440@gmail.com (Robert A. James)
|
||||
|
||||
---
|
||||
|
||||
## CONCLUSION
|
||||
|
||||
**This roadmap is ambitious but achievable.**
|
||||
|
||||
**Key Success Factors**:
|
||||
1. ✅ Strong foundation (Phase 0 complete)
|
||||
2. ✅ Clear milestones (phased delivery)
|
||||
3. ✅ Risk mitigation (fallback plans)
|
||||
4. ⚠️ Community growth (critical for timeline)
|
||||
5. ⚠️ FPGA decision (feasibility gates investment)
|
||||
|
||||
**Realistic Timeline**:
|
||||
- Solo: 3.5 years (42 months)
|
||||
- With 3-5 contributors: 2 years (24 months)
|
||||
- With 10+ contributors: 1.5 years (18 months)
|
||||
|
||||
**The vision is clear. The path is defined. Now we build.**
|
||||
|
||||
---
|
||||
|
||||
**Next Steps**:
|
||||
1. Read [docs/GAP_ANALYSIS.md](docs/GAP_ANALYSIS.md) for detailed gaps
|
||||
2. Start Phase 1: HAL implementation
|
||||
3. Join the project (see CONTRIBUTING.md)
|
||||
|
||||
**License**: See ./LICENSE
|
||||
@@ -0,0 +1,112 @@
|
||||
# docs/book/Makefile -- the single LithosAnanke book, built from LaTeX.
|
||||
#
|
||||
# Run from the repo root (the root Makefile does this):
|
||||
# make docs -> build/docs/LithosAnanke.pdf
|
||||
# make docs TARGET=<board> -> build/docs/LithosAnanke-<board>.pdf
|
||||
# (same book + that board's appendix)
|
||||
#
|
||||
# docs/book/main.tex is the master. Markdown sources are converted by pandoc
|
||||
# into LaTeX fragments under build/docs/gen/ and \input from main.tex; the
|
||||
# Markdown stays the source of truth until a chapter is rewritten in LaTeX.
|
||||
# Figures and tables that present data read the CSV directly at LaTeX time
|
||||
# (pgfplots / pgfplotstable) and cite it with \datasource{path}; see
|
||||
# docs/book/README.md.
|
||||
|
||||
BOOK_DIR := docs/book
|
||||
OUT := build/docs
|
||||
GEN := $(OUT)/gen
|
||||
|
||||
BOARD ?=
|
||||
BOARD_NAME ?=
|
||||
VARIANT := $(if $(BOARD),$(BOARD),book)
|
||||
VAR_DIR := $(OUT)/$(VARIANT)
|
||||
PDF := $(OUT)/LithosAnanke$(if $(BOARD),-$(BOARD)).pdf
|
||||
|
||||
PANDOC ?= pandoc
|
||||
LATEXMK ?= latexmk
|
||||
PANDOC_FILTERS := $(BOOK_DIR)/pandoc/table-widths.lua $(BOOK_DIR)/pandoc/code-breaks.lua
|
||||
PANDOC_FLAGS := -f gfm -t latex --top-level-division=chapter --wrap=preserve \
|
||||
$(foreach f,$(PANDOC_FILTERS),--lua-filter=$(f))
|
||||
|
||||
# fragment name : Markdown source. The order of \input lines is main.tex's.
|
||||
CHAPTERS := \
|
||||
ontology:docs/ONTOLOGY.md \
|
||||
roadmap:docs/ROADMAP.md \
|
||||
boards:boards/README.md \
|
||||
justification:docs/v4.0.0/JUSTIFICATION.md \
|
||||
decomposition:docs/v4.0.0/DECOMPOSITION.md \
|
||||
fabric-0:docs/fabric/FABRIC-0.md \
|
||||
fabric-1:docs/fabric/FABRIC-1.md \
|
||||
fabric-2:docs/fabric/FABRIC-2.md \
|
||||
fabric-3:docs/fabric/FABRIC-3.md \
|
||||
fabric-3-5:docs/fabric/FABRIC-3.5.md \
|
||||
fabric-3-6:docs/fabric/FABRIC-3.6.md \
|
||||
fabric-3-7:docs/fabric/FABRIC-3.7.md \
|
||||
fabric-4:docs/fabric/FABRIC-4.md
|
||||
|
||||
ifneq ($(BOARD),)
|
||||
ifeq ($(wildcard boards/$(BOARD)/README.md),)
|
||||
$(error boards/$(BOARD)/README.md is missing: it is the board's appendix)
|
||||
endif
|
||||
CHAPTERS += board-$(BOARD):boards/$(BOARD)/README.md
|
||||
endif
|
||||
|
||||
chapter_name = $(word 1,$(subst :, ,$(1)))
|
||||
chapter_src = $(word 2,$(subst :, ,$(1)))
|
||||
FRAGMENTS := $(foreach c,$(CHAPTERS),$(GEN)/$(call chapter_name,$(c)).tex)
|
||||
|
||||
.PHONY: all tools FORCE
|
||||
all: $(PDF)
|
||||
|
||||
tools:
|
||||
@missing=""; \
|
||||
for t in $(PANDOC) $(LATEXMK) xelatex; do command -v $$t >/dev/null 2>&1 || missing="$$missing $$t"; done; \
|
||||
if [ -n "$$missing" ]; then \
|
||||
echo "Error: make docs needs:$$missing"; \
|
||||
echo " sudo apt-get install -y pandoc latexmk texlive-xetex texlive-latex-extra texlive-pictures texlive-fonts-recommended fonts-dejavu fonts-dejavu-extra"; \
|
||||
echo " (or a user-local TinyTeX + pandoc in ~/.local; see docs/book/README.md)"; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
# --id-prefix keeps heading labels unique across documents that reuse
|
||||
# section names ("Summary", "Open questions", ...).
|
||||
define chapter_rule
|
||||
$(GEN)/$(call chapter_name,$(1)).tex: $(call chapter_src,$(1)) $(PANDOC_FILTERS) | tools
|
||||
@mkdir -p $(GEN)
|
||||
@echo " PANDOC $$< -> $$@"
|
||||
@$(PANDOC) $(PANDOC_FLAGS) --id-prefix=$(call chapter_name,$(1))- $$< -o $$@
|
||||
endef
|
||||
$(foreach c,$(CHAPTERS),$(eval $(call chapter_rule,$(c))))
|
||||
|
||||
# Pandoc's syntax-highlighting macros, taken from the installed pandoc so the
|
||||
# fragments and their macros always come from the same version.
|
||||
$(GEN)/pandoc-highlighting.tex: $(BOOK_DIR)/pandoc/highlighting.latex | tools
|
||||
@mkdir -p $(GEN)
|
||||
@printf '```c\nx\n```\n' | $(PANDOC) -f gfm -t latex -s --template=$< -o $@
|
||||
|
||||
# Per-build facts the book prints: the commit every \datasource refers to,
|
||||
# and which board appendix (if any) is included.
|
||||
$(VAR_DIR)/meta.tex: FORCE
|
||||
@mkdir -p $(VAR_DIR)
|
||||
@{ \
|
||||
c=$$(git rev-parse --short=12 HEAD 2>/dev/null || echo unknown); \
|
||||
git diff --quiet HEAD -- 2>/dev/null || c="$$c (+ uncommitted changes)"; \
|
||||
printf '\\newcommand{\\bookcommit}{%s}\n' "$$c"; \
|
||||
printf '\\newcommand{\\bookdate}{%s}\n' "$$(date -u +%Y-%m-%d)"; \
|
||||
printf '\\def\\bookboard{%s}\n' "$(BOARD_NAME)"; \
|
||||
$(if $(BOARD),printf '\\newcommand{\\bookboardappendix}{board-%s}\n' "$(BOARD)";) \
|
||||
} > $@.tmp
|
||||
@cmp -s $@.tmp $@ && rm -f $@.tmp || mv $@.tmp $@
|
||||
|
||||
FORCE:
|
||||
|
||||
TEXINPUTS_BOOK := $(abspath $(VAR_DIR)):$(abspath $(GEN)):$(abspath $(BOOK_DIR)):
|
||||
|
||||
$(PDF): $(BOOK_DIR)/main.tex $(FRAGMENTS) $(GEN)/pandoc-highlighting.tex $(VAR_DIR)/meta.tex | tools
|
||||
@echo " LATEXMK $(BOOK_DIR)/main.tex -> $@"
|
||||
@TEXINPUTS=$(TEXINPUTS_BOOK) $(LATEXMK) -xelatex -interaction=nonstopmode -halt-on-error \
|
||||
-file-line-error -outdir=$(VAR_DIR) $(BOOK_DIR)/main.tex > $(VAR_DIR)/latexmk.log 2>&1 || { \
|
||||
grep -A4 -E '^(.*:[0-9]+:|!)' $(VAR_DIR)/main.log | head -40; \
|
||||
echo "Error: LaTeX failed; full log: $(VAR_DIR)/main.log"; exit 1; }
|
||||
@cp $(VAR_DIR)/main.pdf $@
|
||||
@echo " PDF $@"
|
||||
@@ -0,0 +1,66 @@
|
||||
# docs/book/
|
||||
|
||||
`make docs` builds one PDF, `build/docs/LithosAnanke.pdf`, from `main.tex` with xelatex.
|
||||
`make docs TARGET=<board>` builds `build/docs/LithosAnanke-<board>.pdf`: the same book with
|
||||
`boards/<board>/README.md` added as an appendix.
|
||||
|
||||
Tools: `sudo apt-get install -y pandoc latexmk texlive-xetex texlive-latex-extra texlive-pictures texlive-fonts-recommended fonts-dejavu fonts-dejavu-extra`
|
||||
|
||||
Without root, a user-local TeX Live works too (this is how the book was first built):
|
||||
TinyTeX (`curl -sL https://yihui.org/tinytex/install-bin-unix.sh | sh`) plus
|
||||
`tlmgr install latexmk xetex fontspec pgf pgfplots booktabs multirow fancyvrb fvextra lineno upquote ulem enumitem newunicodechar bookmark hyperref geometry xcolor graphics tools etoolbox fancyhdr truncate amsfonts`,
|
||||
and the pandoc release tarball unpacked into `~/.local`.
|
||||
|
||||
The installed DejaVu decides italics: `fonts-dejavu-core` alone has no serif italic, so
|
||||
`main.tex` falls back to a slanted upright face; `fonts-dejavu-extra` gives real italics.
|
||||
|
||||
## How it fits together
|
||||
|
||||
- `main.tex` is the master file. It sets the parts and the chapter order.
|
||||
- Chapters that are still Markdown are listed in `Makefile` (`CHAPTERS`, as `name:source.md`).
|
||||
At build time pandoc converts each one into `build/docs/gen/<name>.tex`, and `main.tex`
|
||||
includes it with `\input{<name>}`. The Markdown stays the source until the chapter is
|
||||
rewritten in LaTeX. At that point the `.tex` moves into this directory and the `CHAPTERS`
|
||||
entry is removed.
|
||||
- `pandoc/highlighting.latex` extracts the code-highlighting macros from the installed pandoc,
|
||||
so the macros and the fragments always come from the same pandoc version.
|
||||
- Two Lua filters run on every chapter. `pandoc/table-widths.lua` gives wide tables
|
||||
proportional wrapping columns, because pandoc's gfm reader leaves column widths unset.
|
||||
`pandoc/code-breaks.lua` lets long inline identifiers and paths break after `_ / . - :`.
|
||||
Code blocks wrap through fvextra (`breaklines`).
|
||||
- `build/docs/<book|board>/meta.tex` is written on every build. It records the commit (and
|
||||
whether there were uncommitted changes), the date, and the board.
|
||||
|
||||
## Rule for data
|
||||
|
||||
Any figure or table that shows measured numbers is generated from the CSV when the book is
|
||||
built. Numbers are never typed in by hand:
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}
|
||||
\begin{axis}[xlabel=run, ylabel=K]
|
||||
\addplot table[col sep=comma, x=run, y=K]{experiments/<campaign>/results.csv};
|
||||
\end{axis}
|
||||
\end{tikzpicture}
|
||||
\datasource{experiments/<campaign>/results.csv}
|
||||
```
|
||||
|
||||
Use `\pgfplotstabletypeset[col sep=comma]{...}` for tables. `\datasource` prints the CSV path
|
||||
and the commit, so a reader can find the exact file the figure was drawn from. CSV paths are
|
||||
repo-relative, because xelatex runs from the repo root.
|
||||
|
||||
## Older pipelines to fold in or retire
|
||||
|
||||
These still exist and still have their own targets. Each one needs a decision during the
|
||||
curation pass: move it into this book, or retire it.
|
||||
|
||||
| Source | Current target | Produces |
|
||||
|---|---|---|
|
||||
| `docs/formal/vol1-vm-physics`, `vol2-kernel`, `vol3-research` | `make -C docs/formal vols` | three volume PDFs |
|
||||
| `docs/formal/dev-guide`, `user-guide`, `cookbook` | `make -C docs/formal books` | three practitioner PDFs |
|
||||
| `docs/formal/experiments`, `proofs`, `ssrn`, `patent` | `make -C docs/formal standalone` | four standalone PDFs |
|
||||
| Doxygen (`Doxyfile`) | `make -C docs/formal doxygen` | API reference PDF |
|
||||
| `scripts/generate-doxygen-appendix.sh` | `make -f v3/Makefile api-docs` | AsciiDoc API appendix |
|
||||
| `docs/src/internal/formal/*.thy` | `make -f v3/Makefile docs-isabelle` | Isabelle report |
|
||||
| `scripts/asciidoc-to-latex.sh` | `make -f v3/Makefile docs-latex` | `docs/latex/` |
|
||||
| `docs/SSRN_companion/Math_Companion_SSRN.tex` | `make -f v3/Makefile math-companion` | SSRN math companion |
|
||||
@@ -0,0 +1,161 @@
|
||||
% docs/book/main.tex -- master file of the single LithosAnanke book.
|
||||
%
|
||||
% Built by `make docs [TARGET=<board>]` (docs/book/Makefile) with xelatex.
|
||||
% Fragments named below (ontology, fabric-0, ...) are generated by pandoc
|
||||
% from the Markdown listed in docs/book/Makefile's CHAPTERS into
|
||||
% build/docs/gen/; meta.tex is generated per build. Neither is committed.
|
||||
\documentclass[11pt,openany]{book}
|
||||
|
||||
\usepackage{amsmath,amssymb}
|
||||
\usepackage{fontspec}
|
||||
% fonts-dejavu-core has no serif/sans italics (fonts-dejavu-extra does); when a
|
||||
% shape is missing, slant the upright one rather than silently drop emphasis.
|
||||
\setmainfont{DejaVu Serif}[AutoFakeSlant=0.2]
|
||||
\setsansfont{DejaVu Sans}[AutoFakeSlant=0.2]
|
||||
\setmonofont{DejaVu Sans Mono}[Scale=0.85]
|
||||
\newfontfamily\symbolfont{DejaVu Sans}
|
||||
|
||||
\usepackage[letterpaper,margin=1in]{geometry}
|
||||
\usepackage{xcolor}
|
||||
\usepackage{graphicx}
|
||||
\usepackage{longtable,booktabs,array,calc,multirow}
|
||||
\usepackage{fancyvrb}
|
||||
\usepackage[normalem]{ulem}
|
||||
\usepackage{enumitem}
|
||||
\usepackage{newunicodechar}
|
||||
\usepackage{pgfplots}
|
||||
\usepackage{pgfplotstable}
|
||||
\pgfplotsset{compat=1.18}
|
||||
\usepackage[hidelinks]{hyperref}
|
||||
\usepackage{bookmark}
|
||||
|
||||
% --- what pandoc's LaTeX fragments expect (normally from its own template)
|
||||
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
|
||||
\providecommand{\pandocbounded}[1]{#1}
|
||||
\providecommand{\st}[1]{\sout{#1}}
|
||||
\newcounter{none} % pandoc: {\def\LTcaptype{none} ...} marks unnumbered tables
|
||||
\usepackage{etoolbox}
|
||||
\makeatletter
|
||||
\def\fnum@table{\tablename~\thetable}
|
||||
\patchcmd\longtable{\par}{\if@noskipsec\mbox{}\fi\par}{}{}
|
||||
\makeatother
|
||||
\usepackage{fvextra}
|
||||
\input{pandoc-highlighting}
|
||||
% Code and diagrams are wider than the page in places: wrap, never overflow.
|
||||
\RecustomVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\},breaklines,breakanywhere,fontsize=\small}
|
||||
\RecustomVerbatimEnvironment{verbatim}{Verbatim}{breaklines,breakanywhere,fontsize=\small}
|
||||
\setlistdepth{9}
|
||||
\renewlist{itemize}{itemize}{9}
|
||||
\setlist[itemize]{label=\textbullet}
|
||||
\renewlist{enumerate}{enumerate}{9}
|
||||
\setlist[enumerate]{label=\arabic*.}
|
||||
% Running heads: chapter on the left page, section on the right, both cut to
|
||||
% the page width (several design-record titles are a full sentence long).
|
||||
\usepackage{fancyhdr}
|
||||
\usepackage[fit]{truncate}
|
||||
\pagestyle{fancy}
|
||||
\fancyhf{}
|
||||
\renewcommand{\chaptermark}[1]{\markboth{#1}{}}
|
||||
\renewcommand{\sectionmark}[1]{\markright{#1}}
|
||||
\fancyhead[LE]{\small\truncate{\dimexpr\headwidth-3em}{\leftmark}}
|
||||
\fancyhead[RO]{\small\truncate{\dimexpr\headwidth-3em}{\rightmark}}
|
||||
\fancyhead[RE,LO]{\small\thepage}
|
||||
\renewcommand{\headrulewidth}{0.4pt}
|
||||
\setlength{\headheight}{14pt}
|
||||
\fancypagestyle{plain}{\fancyhf{}\fancyfoot[C]{\small\thepage}\renewcommand{\headrulewidth}{0pt}}
|
||||
\setlength{\emergencystretch}{3em}
|
||||
\setlength{\parindent}{0pt}
|
||||
\setlength{\parskip}{0.5em}
|
||||
|
||||
% --- glyphs DejaVu Serif lacks: the status marks used across the docs
|
||||
\newunicodechar{✅}{{\symbolfont ✔}}
|
||||
\newunicodechar{✓}{{\symbolfont ✓}}
|
||||
\newunicodechar{❌}{{\symbolfont ✘}}
|
||||
\newunicodechar{✗}{{\symbolfont ✗}}
|
||||
\newunicodechar{⬜}{{\symbolfont ☐}}
|
||||
\newunicodechar{⭐}{{\symbolfont ★}}
|
||||
\newunicodechar{🔶}{{\symbolfont ◆}}
|
||||
\newunicodechar{⚠}{{\symbolfont ⚠}}
|
||||
\newunicodechar{📋}{}
|
||||
\newunicodechar{🐛}{[bug]}
|
||||
\newunicodechar{🎯}{[goal]}
|
||||
\newunicodechar{🟡}{{\symbolfont ●}}
|
||||
\newunicodechar{📍}{{\symbolfont ▸}}
|
||||
\newunicodechar{🔓}{[unlocked]}
|
||||
\newunicodechar{❓}{?}
|
||||
\newunicodechar{⟺}{\ensuremath{\Longleftrightarrow}}
|
||||
\newunicodechar{⋯}{\ensuremath{\cdots}}
|
||||
\newunicodechar{^^^^fe0f}{}
|
||||
|
||||
% --- the documents number their own sections (§III.1, D-3, ...)
|
||||
\setcounter{secnumdepth}{0}
|
||||
\setcounter{tocdepth}{1}
|
||||
|
||||
\input{meta}
|
||||
|
||||
% \datasource{path/to/data.csv}: cite the data behind a figure or table.
|
||||
% Every figure or table of measured numbers reads its CSV at build time
|
||||
% (\addplot table / \pgfplotstabletypeset) and is followed by this line, so
|
||||
% the reader can find the exact file at the exact commit.
|
||||
\newcommand{\datasource}[1]{%
|
||||
\par{\small Data: \texttt{\detokenize{#1}} at commit \texttt{\bookcommit}.}\par}
|
||||
|
||||
\title{LithosAnanke and StarForth}
|
||||
\author{}
|
||||
\date{Built \bookdate{} from commit \texttt{\bookcommit}%
|
||||
\ifx\bookboard\empty\else\\Board appendix: \bookboard\fi}
|
||||
|
||||
\begin{document}
|
||||
\frontmatter
|
||||
\maketitle
|
||||
\tableofcontents
|
||||
|
||||
\chapter{About this book}
|
||||
This book is generated from the repository by \texttt{make docs}. Each
|
||||
chapter below is still maintained as the Markdown file named in the table and
|
||||
converted at build time; a chapter moves into LaTeX source when it is
|
||||
rewritten. Part~III is the design record: those documents are archival and
|
||||
are reproduced as written, including their struck-through corrections.
|
||||
|
||||
\begin{longtable}{@{}ll@{}}
|
||||
\toprule
|
||||
Chapter & Source \\
|
||||
\midrule
|
||||
\endhead
|
||||
Ontology & \texttt{docs/ONTOLOGY.md} \\
|
||||
Roadmap & \texttt{docs/ROADMAP.md} \\
|
||||
Boot targets & \texttt{boards/README.md} \\
|
||||
v4 justification & \texttt{docs/v4.0.0/JUSTIFICATION.md} \\
|
||||
v4 decomposition & \texttt{docs/v4.0.0/DECOMPOSITION.md} \\
|
||||
FABRIC-0 \dots{} FABRIC-4 & \texttt{docs/fabric/FABRIC-*.md} \\
|
||||
Board appendix (\texttt{TARGET=}) & \texttt{boards/<board>/README.md} \\
|
||||
\bottomrule
|
||||
\end{longtable}
|
||||
|
||||
\mainmatter
|
||||
|
||||
\part{The system}
|
||||
\input{ontology}
|
||||
\input{roadmap}
|
||||
\input{boards}
|
||||
|
||||
\part{StarForth v4}
|
||||
\input{justification}
|
||||
\input{decomposition}
|
||||
|
||||
\part{Design record (FABRIC series)}
|
||||
\input{fabric-0}
|
||||
\input{fabric-1}
|
||||
\input{fabric-2}
|
||||
\input{fabric-3}
|
||||
\input{fabric-3-5}
|
||||
\input{fabric-3-6}
|
||||
\input{fabric-3-7}
|
||||
\input{fabric-4}
|
||||
|
||||
\ifdefined\bookboardappendix
|
||||
\appendix
|
||||
\input{\bookboardappendix}
|
||||
\fi
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,29 @@
|
||||
-- docs/book/pandoc/code-breaks.lua
|
||||
--
|
||||
-- Inline code in the design record is full of long identifiers and paths
|
||||
-- (g_wirebind_attached_username, kernel/src/arch/riscv64/...), which LaTeX
|
||||
-- cannot break, so they run into the margin. Emit inline code as \texttt
|
||||
-- ourselves, escaped the same way pandoc does, with a break opportunity
|
||||
-- after each separator character.
|
||||
|
||||
local BREAK_AFTER = { ["_"] = true, ["/"] = true, ["."] = true, ["-"] = true,
|
||||
[":"] = true, [","] = true, ["("] = true, ["="] = true }
|
||||
|
||||
local ESCAPE = {
|
||||
["\\"] = "\\textbackslash{}", ["{"] = "\\{", ["}"] = "\\}",
|
||||
["$"] = "\\$", ["&"] = "\\&", ["#"] = "\\#", ["%"] = "\\%", ["_"] = "\\_",
|
||||
["^"] = "\\textasciicircum{}", ["~"] = "\\textasciitilde{}",
|
||||
["'"] = "\\textquotesingle{}", ["`"] = "\\textasciigrave{}",
|
||||
[" "] = "\\ ",
|
||||
}
|
||||
|
||||
function Code(el)
|
||||
if not FORMAT:match("latex") then return nil end
|
||||
local out = {}
|
||||
for _, cp in utf8.codes(el.text) do
|
||||
local ch = utf8.char(cp)
|
||||
out[#out + 1] = ESCAPE[ch] or ch
|
||||
if BREAK_AFTER[ch] then out[#out + 1] = "\\allowbreak{}" end
|
||||
end
|
||||
return pandoc.RawInline("latex", "\\texttt{" .. table.concat(out) .. "}")
|
||||
end
|
||||
@@ -0,0 +1,3 @@
|
||||
$if(highlighting-macros)$
|
||||
$highlighting-macros$
|
||||
$endif$
|
||||
@@ -0,0 +1,55 @@
|
||||
-- docs/book/pandoc/table-widths.lua
|
||||
--
|
||||
-- pandoc's gfm reader leaves every column width unset, so the LaTeX writer
|
||||
-- emits `l` columns and a table with long cells runs off the page. For any
|
||||
-- table wider than WIDE characters, give each column a width proportional
|
||||
-- to its longest cell (clamped to [MIN_COL, MAX_COL] characters); the
|
||||
-- writer then emits wrapping p{} columns that together fill \linewidth.
|
||||
|
||||
local WIDE = 72
|
||||
local MIN_COL = 6
|
||||
local MAX_COL = 60
|
||||
|
||||
local stringify = pandoc.utils.stringify
|
||||
|
||||
local function measure(rows, widths)
|
||||
for _, row in ipairs(rows) do
|
||||
local col = 1
|
||||
for _, cell in ipairs(row.cells) do
|
||||
local span = cell.col_span or 1
|
||||
local len = utf8.len(stringify(cell.contents)) or #stringify(cell.contents)
|
||||
local per = len / span
|
||||
for c = col, col + span - 1 do
|
||||
if per > (widths[c] or 0) then widths[c] = per end
|
||||
end
|
||||
col = col + span
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
function Table(tbl)
|
||||
local n = #tbl.colspecs
|
||||
local widths = {}
|
||||
for i = 1, n do widths[i] = 0 end
|
||||
|
||||
measure(tbl.head.rows, widths)
|
||||
for _, body in ipairs(tbl.bodies) do
|
||||
measure(body.head, widths)
|
||||
measure(body.body, widths)
|
||||
end
|
||||
measure(tbl.foot.rows, widths)
|
||||
|
||||
local total = 0
|
||||
for i = 1, n do total = total + widths[i] end
|
||||
if total <= WIDE then return nil end
|
||||
|
||||
local sum = 0
|
||||
for i = 1, n do
|
||||
widths[i] = math.max(MIN_COL, math.min(MAX_COL, widths[i]))
|
||||
sum = sum + widths[i]
|
||||
end
|
||||
for i = 1, n do
|
||||
tbl.colspecs[i] = { tbl.colspecs[i][1], widths[i] / sum }
|
||||
end
|
||||
return tbl
|
||||
end
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,205 @@
|
||||
# FABRIC-3.7.md — Phase 8 PKI: the elevation entrypoint
|
||||
|
||||
**Status: OPEN — design only, no code written or authorized.**
|
||||
|
||||
**CORRECTION (2026-09-22, before any code was written against this document): §2's central
|
||||
claim — that the old `SEND-ELEVATE-REQUEST` passed a raw cross-VM address into `ELEVATE-GRANT`'s
|
||||
`waddr` — is wrong.** Found while starting Part A's implementation: reading the actual deleted
|
||||
source (`git show 3e201c8^:capsules/common/messaging.4th`, block 5040) shows
|
||||
`SEND-ELEVATE-REQUEST` copied the target word's **name as literal character bytes** (via
|
||||
`ELEVATE-REQ-APPEND`'s `CMOVE`) into a scratch buffer, building the text `S" <name-text>" <pk0>
|
||||
<pk1> <pk2> <pk3> ELEVATE-GRANT`, and sent *that whole string* to Hera. When Hera's own
|
||||
interpreter runs `S" <name-text>"`, it allocates a fresh string **in Hera's own memory** and
|
||||
pushes Hera's own valid address — no numeric cross-VM address ever appears anywhere in this
|
||||
flow. §2 was written from `ELEVATE-GRANT`'s signature alone, assuming the caller forwarded a raw
|
||||
address, without first reading how the caller actually built its message. It didn't.
|
||||
|
||||
**What is still real, much narrower than originally claimed:** if a future caller ever spliced
|
||||
*attacker-influenced* text into the name field without checking for an embedded `"` character,
|
||||
that could break out of the `S" ... "` literal early and inject arbitrary FORTH source, executed
|
||||
with Hera's privilege. That's an input-validation discipline question for whoever writes the new
|
||||
caller (validate: no embedded `"`, or just always use a compile-time-fixed literal name, never a
|
||||
runtime-supplied one) — not an architectural cross-VM-memory defect requiring the buffer/message
|
||||
redesign §3 originally called for. **§3's proposed mechanism (Hera-side fixed receive buffer,
|
||||
kernel-constructed integer-literal-only command) is not needed** — the original text-copy
|
||||
design was already safe against the bug as actually diagnosed. Kept below, struck through, for
|
||||
traceability, per this series' own rule against silently rewriting a prior state.
|
||||
|
||||
Successor to
|
||||
`FABRIC-3.5.md`/`FABRIC-3.6.md` (both CLOSED/ARCHIVAL at `v2.1.0`) for exactly one topic: the
|
||||
Ed25519-challenge-response elevation entrypoint that `.claude/CLAUDE.md`'s ACL section names as
|
||||
Phase 8, the last open item in the word-level ACL system. This is a **new document**, not a
|
||||
reopening of `FABRIC-3.6.md` — that document's own close header says future fleet work gets its
|
||||
own document, and this is that.
|
||||
|
||||
**Provenance.** Written 2026-09-22, immediately after `FABRIC-3.6.md`'s close, from a design
|
||||
conversation with Captain Bob about a security concern he raised directly: how to rebuild the
|
||||
elevation entrypoint that Phase 4's Category B strip left dangling, without reopening a hole.
|
||||
The design below was proposed, and Captain Bob asked for it in writing here rather than left
|
||||
only in session memory.
|
||||
|
||||
---
|
||||
|
||||
## 1. What's dangling, and why
|
||||
|
||||
`FABRIC-3.6.md` Phase 4 (Category B strip, 2026-09-22) deleted `capsules/common/messaging.4th`
|
||||
after every FORTH-owned message type had been cut over to kernel-Hermes. One casualty was
|
||||
collateral, not intended: `SEND-ELEVATE-REQUEST` (`messaging.4th` block 5040) was the only
|
||||
caller of both `KH-ELEVATE-SEND` (`src/starkernel/repl.c`) and, transitively, `ELEVATE-GRANT`
|
||||
(`capsules/zuse-eligibility.4th`, blocks 4021–4022). All three still exist in the tree.
|
||||
`ELEVATE-GRANT` is still loaded at boot (`capsules/init.4th:18`). Nothing can call it any more.
|
||||
|
||||
Captain Bob's decision at the time (`FABRIC-3.6.md`'s own Phase 4 entry): leave it unreachable,
|
||||
don't patch a caller back in as part of that strip. Phase 8 builds its own entrypoint instead of
|
||||
resuming this one. **This document is that entrypoint's design.**
|
||||
|
||||
<details>
|
||||
<summary>Original §2/§3 (WRONG — see the correction at the top of this document; kept for
|
||||
traceability, not current design)</summary>
|
||||
|
||||
### 2. The security hole in the old mechanism — found before any code was written
|
||||
|
||||
`ELEVATE-GRANT`'s signature, unchanged since it was written:
|
||||
|
||||
```
|
||||
ELEVATE-GRANT ( waddr wu pk0 pk1 pk2 pk3 -- )
|
||||
```
|
||||
|
||||
`waddr`/`wu` are an address/length pair meant to point at the string naming the word to elevate.
|
||||
`pk0`–`pk3` are the caller's Ed25519 pubkey, packed 8 bytes per cell (`ELEVATE-PUBKEY-UNPACK`,
|
||||
`mama_forth_words.c`).
|
||||
|
||||
**The old `SEND-ELEVATE-REQUEST` computed `waddr` in the *sending* VM's own address space, but
|
||||
`ELEVATE-GRANT` always executes on Hera** (`ELEVATE-GRANT always runs on Hera` — `repl.c`'s own
|
||||
comment on `KH-ELEVATE-SEND`, still there). A word's name string lives in the sending VM's
|
||||
memory. `ELEVATE-GRANT` dereferences `waddr` in Hera's memory. Those are not the same address
|
||||
space by construction — `vaddr_t` is per-VM.
|
||||
|
||||
**Consequence:** whoever controls `waddr` controls what bytes `NAME>XT` reads and resolves as a
|
||||
word name, in Hera's dictionary, not the caller's. This is not "the string might be malformed" —
|
||||
it is a primitive for making Hera's own `ELEVATE-GRANT` grant `ACL-ALLOW!`/`ACL-TTL!` on
|
||||
*whatever dictionary entry the attacker's chosen `waddr` happens to land on*, regardless of what
|
||||
word name the caller claims to be requesting elevation for. A caller who can influence `waddr`
|
||||
at all — not forge a signature, not defeat `zuse_eligibility_is_member()`, just choose a number
|
||||
— has a privilege-escalation primitive against the fleet governor.
|
||||
|
||||
This was never exploited (the entrypoint has had zero live callers since the file that called it
|
||||
was deleted), and is reported here as a design defect found by inspection, not a live incident.
|
||||
|
||||
### 3. The fix: never cross an address, only ever cross bytes
|
||||
|
||||
This project already solved the general version of this problem once, this same session
|
||||
(`FABRIC-3.6.md` tasks 3.8/3.9, the payload-aliasing fix): a kernel-Hermes message's payload
|
||||
must be **copied into the message's own storage**, never a pointer into the sender's memory that
|
||||
might be reused or freed before the receiver drains it. `SkHermesMessage.payload_buf`
|
||||
(`include/starkernel/vm/kernel_hermes.h:160`, `SK_HERMES_CHUNK_MAX_PAYLOAD` = 1024 bytes) is
|
||||
exactly that fix, already built, already proven on all three architectures.
|
||||
|
||||
**The elevation entrypoint's hole is the same defect one level up: an address crossing a
|
||||
boundary it isn't valid on the other side of.** The fix generalizes directly:
|
||||
|
||||
1. **Never send `waddr`/`wu` across the kernel-Hermes boundary.** Send the pubkey (32 bytes,
|
||||
already the right shape for `payload_buf`) and the target word's **name, as literal bytes**,
|
||||
copied inline into the message payload — not an address, the actual characters. This is
|
||||
already how `CONSOLE-CMD-EVENT`'s payload works (a command string's bytes, not a pointer to
|
||||
one), so this isn't a new pattern, it's applying the existing one to the one caller that
|
||||
still passed a raw address.
|
||||
|
||||
2. **On receipt, kernel-Hermes's C drain-checkpoint copies those name bytes into a small,
|
||||
fixed, kernel-owned buffer that already lives in Hera's own VM memory** — a receive-side
|
||||
mirror of the existing send-side pattern (`g_kh_elevate_buf`, `repl.c:445`, is the
|
||||
already-built precedent for "a static buffer this mechanism owns"; this needs its Hera-side
|
||||
counterpart). The buffer's address is a compile-time constant, known to the kernel, never
|
||||
computed from anything the caller supplied.
|
||||
|
||||
3. **The FORTH command handed to `vm_interpret()` on Hera references only that fixed buffer's
|
||||
address and length as plain integer literals.** Both are always kernel-controlled. Neither is
|
||||
ever derived from caller input. `ELEVATE-GRANT` itself does not change — same signature, same
|
||||
`zuse_eligibility_is_member()` check, same `ACL-ALLOW!`/`ACL-TTL!` grant. Policy logic stays
|
||||
in FORTH, per `ACL.4th`'s own rule (no new C primitive for policy) — this fix is entirely
|
||||
about how bytes get from one VM to another, not about who is allowed to grant what.
|
||||
|
||||
**Why this closes the hole structurally, not by validation:** there is no string to sanitize and
|
||||
no address to bounds-check, because the interpreted command never contains anything an attacker
|
||||
touched except opaque data bytes that get copied, never dereferenced as a pointer, by the
|
||||
receiving side. The class of bug (cross-address-space pointer confusion) becomes impossible by
|
||||
construction, the same way `payload_buf` made use-after-free impossible by construction rather
|
||||
than by careful lifetime tracking.
|
||||
|
||||
</details>
|
||||
|
||||
## 2 (corrected). What the old mechanism actually did, and the one real gap in it
|
||||
|
||||
Re-read from the actual deleted source (`git show 3e201c8^:capsules/common/messaging.4th`,
|
||||
blocks 5039–5040): `SEND-ELEVATE-REQUEST ( pk3 pk2 pk1 pk0 waddr wu -- )` used `waddr`/`wu` only
|
||||
to `CMOVE` the target word's **name bytes**, as text, into a scratch buffer
|
||||
(`ELEVATE-REQ-BUF`/`ELEVATE-REQ-APPEND`) it owned — building the literal string `S"
|
||||
<name-text>" <pk0> <pk1> <pk2> <pk3> ELEVATE-GRANT` entirely in the *sending* VM's own memory.
|
||||
Only that finished string — not `waddr` itself — went to `KH-ELEVATE-SEND` and across to Hera.
|
||||
When Hera's interpreter runs `S" <name-text>"`, Hera's own `S"` allocates a fresh string **in
|
||||
Hera's own memory** and pushes Hera's own valid address. `waddr`/`wu` never cross the VM
|
||||
boundary as numbers at any point — only as copied character content. There is no cross-VM
|
||||
pointer dereference anywhere in this flow.
|
||||
|
||||
**The one real, much narrower gap:** the name text is spliced into `S" ... "` with no check for
|
||||
an embedded `"` character. If a future caller ever passed attacker-influenced text as the name
|
||||
(none ever did — the word had zero live callers), a `"` in the name would close the string
|
||||
literal early and let the rest of the name execute as raw FORTH source, with Hera's privilege.
|
||||
This is a caller-discipline / input-validation question, not an architectural defect: either
|
||||
always use a compile-time-fixed name literal at the call site (no runtime input, no risk at
|
||||
all), or validate for an embedded `"` before building the command if a name ever does need to
|
||||
come from something less trusted than the call site's own source code.
|
||||
|
||||
**Net effect on Phase 8 v1's scope:** Part A, as originally conceived in §3 above, is not
|
||||
needed. If a `SEND-ELEVATE-REQUEST` replacement is ever built, it can follow the original
|
||||
text-copy design as-is, with the one-line `"`-check added if and only if the name is ever
|
||||
runtime-supplied rather than a fixed literal. No kernel-Hermes/`repl.c` changes required. Part B
|
||||
(`capsules/zuse.4th`, gating `ZUSE-ELIGIBILITY-ADD`) stands on its own, independently verified,
|
||||
unaffected by this correction.
|
||||
|
||||
## 4. What Phase 8 actually needs to build
|
||||
|
||||
Corrected per §2's re-read above. Concretely, when Phase 8 next picks this up:
|
||||
|
||||
- If a caller into `ELEVATE-GRANT` is ever needed again, rebuild it close to the original
|
||||
`SEND-ELEVATE-REQUEST` shape (`ELEVATE-REQ-BUF`/`ELEVATE-REQ-APPEND`/text-copy into `S" ...
|
||||
"`) — it was already safe. Add the one-line embedded-`"` check only if the name is ever
|
||||
runtime-supplied rather than a call-site literal. No `kernel_hermes.c`/`kernel_hermes.h`/
|
||||
`repl.c` changes needed for this.
|
||||
- `ELEVATE-GRANT` unchanged either way.
|
||||
- **Part B is done** (`capsules/zuse.4th`, committed and three-arch verified this session,
|
||||
2026-09-22) — `ZUSE-ELIGIBILITY-ADD` denied by default, granted only inside `ACL-ZUSE-BOOT`'s
|
||||
authenticated branch. This closes the actual "grant yourself eligibility with no real drive at
|
||||
all" path — a real, independently-confirmed gap, unaffected by this correction.
|
||||
- **Defending against a cloned drive — settled, 2026-09-23: not going to happen, by design.**
|
||||
Today, WIREBIND/MINT trust whatever identity is stored on an attached thumbdrive with no
|
||||
challenge at all (confirmed by grep: no `ed25519_sign`/`ed25519_verify` call anywhere in
|
||||
`capsule_wirebind.c` or `capsule_mint.c`), and the private key seed itself is stored in
|
||||
plaintext on the drive, read in the same devblock as the pubkey/cert. **This is accepted, not
|
||||
a gap.** Captain Bob, directly: *"nothing like a pin or a password or secret code or any
|
||||
bullshit... Everybody has secrets. There's only the drive."* Physical possession of the drive
|
||||
is the entire, deliberate credential model — a byte-for-byte clone being equivalent to the
|
||||
real drive is the accepted design, not a defect to close. A PIN/passphrase second factor was
|
||||
built, live-tested on all three architectures, and fully reverted before commit
|
||||
(`/home/rajames/.claude/plans/jiggly-cuddling-stallman.md`, now marked rejected; memory
|
||||
`feedback_no_knowledge_factor_identity`) — **do not revisit a knowledge-factor approach here.**
|
||||
Any future work in this space needs a fundamentally different mechanism (not something typed
|
||||
and known) or stays an accepted limitation.
|
||||
- **Phase 8 v3, 2026-09-23 — done, a distinct and narrower concern from the item above.** The
|
||||
"accepted limitation" above is about a drive image copied *outside* StarshipOS entirely (e.g.
|
||||
imaged on an external computer) — that's still accepted, unchanged by this item. Captain Bob
|
||||
separately asked to close a narrower, different threat: **cloning a device's block content
|
||||
from *within* StarshipOS's own console**, using its own stock, unpinned words
|
||||
(`<src> BLOCK <dst> BUFFER 1024 MOVE`/`RELOCATE-BLOCK`). That's now closed — `MOVE`/`CMOVE`/
|
||||
`CMOVE>`/`RELOCATE-BLOCK` all refuse a same-VM, cross-device copy, verified live on all three
|
||||
architectures. See `/home/rajames/.claude/plans/jiggly-cuddling-stallman.md`'s "Phase 8 v3"
|
||||
section for the full design and verification record. The identity record itself
|
||||
(seed/pubkey/cert) was already unreachable from FORTH before this — this closes the one real
|
||||
gap the research found: ordinary block content, not the identity record.
|
||||
|
||||
## 5. What this document is not
|
||||
|
||||
Not a reopening of `FABRIC-3.6.md`, not a change to anything currently built, not an
|
||||
authorization to write code. Per this series' own convention: design here, execution gets its
|
||||
own document when the work actually starts, the same relationship `FABRIC-3.5.md` had to
|
||||
`FABRIC-3.6.md`.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,106 @@
|
||||
# FABRIC-4.md — forward-looking design notes
|
||||
|
||||
**Status:** Not a successor to `FABRIC-3.md` and not a punch-list document. `FABRIC-3.md` is
|
||||
the current living document (bare metal boot, opened 2026-09-04, successor to the now-closed
|
||||
`FABRIC-2.md`) — nothing here supersedes it, and this file doesn't follow the
|
||||
close-and-carry-forward discipline `FABRIC-0.md` → `FABRIC-1.md` → `FABRIC-2.md` → `FABRIC-3.md`
|
||||
used (that chain triggers on *closing* a document; `FABRIC-3.md` isn't closed). This is a
|
||||
separate, lower-discipline scratchpad for design ideas caught early — theory-stage,
|
||||
pre-punch-list, often missing a stated "why" on purpose (captured as raised, not fully
|
||||
justified yet). An idea graduates out of here into `FABRIC-3.md` (or wherever the living
|
||||
document is by then) once it has a real scope, not before.
|
||||
|
||||
---
|
||||
|
||||
## 1. Stadium-level "wheel" — a second clock alongside the heartbeat (2026-09-03)
|
||||
|
||||
Raised in conversation with Captain Bob, theory stage — no code, no scope, deliberately not
|
||||
punch-listed yet.
|
||||
|
||||
**The mechanism.** A second clock, independent of but derived from the true heartbeat
|
||||
(`HeartbeatState`, Loop #7), advanced by a fixed-point (Q48.16, this project's standard)
|
||||
phase accumulator: each master heartbeat tick adds a `rate` scalar to the wheel's own
|
||||
`phase`; whenever `phase` crosses `1.0` the wheel fires (advances its own state) and the
|
||||
crossing remainder carries forward. This is a continuous scalar, not an integer tick-divisor
|
||||
— `rate` can sit anywhere, including above `1.0`. A wheel with `rate > 1` fires more than
|
||||
once per master tick (multiple threshold-crossings per tick, remainder still carried); one
|
||||
with `rate < 1` accumulates across several master ticks before firing once. Nothing about the
|
||||
mechanism privileges either side of `1.0` — "runs ahead of baseline" and "runs stretched
|
||||
behind baseline" are the same math with a different scalar.
|
||||
|
||||
**What feeds it.** Not a new signal source — this wheel listens to two already-real, already-
|
||||
built mechanisms: `rolling_window_of_truth.c` (Loop #2's circular-buffer execution history —
|
||||
`rolling_window_measure_diversity()`, `rolling_window_find_hottest_word()`,
|
||||
`rolling_window_count_transition()`, `rolling_window_pattern_capture_rate()`,
|
||||
`rolling_window_is_warm()`) and `inference_engine.c` (Loops #5/#6 — `InferenceOutputs`:
|
||||
decay-slope, variance-inflection point, fit quality). Both exist today at the **per-VM**
|
||||
level (each patron/word-owning VM has its own rolling window and runs its own inference
|
||||
pass) — the wheel itself is meant to live at the **Stadium** level, one layer up. This is a
|
||||
genuinely new axis: nothing today aggregates RWOT/inference-engine output across the whole
|
||||
Stadium floor into one fleet-wide signal. **Open, not decided:** what the aggregation actually
|
||||
is (sum across patrons? mean? the hottest patron's own values only? something else) — flagged
|
||||
in conversation, not resolved.
|
||||
|
||||
**What it's for.** Not a sieve itself. The wheel is a *scheduler/gate*: it decides *when* to
|
||||
slip an appropriate sieve into a stack of sieves that doesn't exist yet (see §2 below) —
|
||||
running its own clock, potentially ahead of the true heartbeat, is what buys it lead time to
|
||||
notice a transition worth reacting to before the baseline heartbeat would get there on its
|
||||
own. It is not itself the thing doing the sieving.
|
||||
|
||||
**Build sequencing, decided in conversation.** `rate` starts as a fixed build parameter (same
|
||||
shape as `ACL-BASE-TTL`/`Q.SLOT` — a plain constant or Kconfig symbol, not adaptive from day
|
||||
one). Reasoning, Captain Bob's own: a fixed rate is the only version that can go through this
|
||||
project's existing DoE apparatus and produce a clean, attributable measurement (rate held
|
||||
constant, everything else varies) — if the rate self-tunes from day one there's no fixed
|
||||
baseline to compare "does the wheel actually help" against. Only *after* a real, measured
|
||||
baseline exists does `rate` itself graduate to adaptive/self-tuning — reusing the same
|
||||
inference-engine machinery the wheel already listens to, not new plumbing. This mirrors the
|
||||
L8 Jacquard selector's own real history: hardcoded compudynamics constant first, later
|
||||
replaced by the VM Fleet Attractor's dynamically-inferred rate
|
||||
(`docs/working/architecture/VM-FLEET-ATTRACTOR-DESIGN-20260705.md`). Same graduation, one
|
||||
level up the stack.
|
||||
|
||||
**Explicitly not decided yet, per Captain Bob ("we'll talk about why later"):**
|
||||
- The actual *purpose* — why this wheel, what real problem it solves. Deferred on purpose.
|
||||
- The sieve stack itself — what a "sieve" is concretely, what gets sieved, how many. A larger,
|
||||
separate vision (128–256 wheels, each fed by an "appropriate" — possibly learned/neural,
|
||||
vocabulary not yet resolved against anything this codebase currently has — source, forming
|
||||
a coincidence-pattern sieve) was raised earlier in the same conversation as a *distinct*,
|
||||
further-out idea; this single-wheel scheduler is deliberately scoped smaller and doesn't
|
||||
presuppose that larger picture is right.
|
||||
- Whether "trivial to add a wheel" (an earlier framing) still matters once there's only one —
|
||||
a generic `{rate, phase, state}` table is a premature abstraction for a single instance;
|
||||
build the one wheel concretely first, generalize only if/when a second one is actually
|
||||
needed (this project's own standing convention — three similar lines beat a premature
|
||||
abstraction).
|
||||
|
||||
## 2. A pinned GPIO VM per real-hardware board (2026-09-04)
|
||||
|
||||
Raised in conversation with Captain Bob during `FABRIC-3.md` §IV/§V's bare-metal-boot
|
||||
planning — theory stage, no code, no scope, deliberately not punch-listed yet.
|
||||
|
||||
**The idea, as stated:** the Raspberry Pi 5 will need a special pinned VM dedicated to GPIO —
|
||||
presumably following the same Tripod precedent Hermes/Artemis already established (a
|
||||
permanently-pinned, born-at-boot VM with a fixed role, per `FABRIC-2.md` §H.1's "pinned
|
||||
sessions never leave" model). Whether the Milk-V Mars needs the same treatment is explicitly
|
||||
**not yet decided** — Captain Bob's own framing: "same for Milk-V (? not sure here)." Worth
|
||||
noting the Mars does have its own 40-pin GPIO header (§VI), so the question isn't "does it
|
||||
have GPIO" but whether it needs its own *dedicated pinned VM* for it the same way the Pi 5
|
||||
does, or whether GPIO access there fits some other shape.
|
||||
|
||||
**Why a dedicated VM, not just a C driver, presumably (not yet confirmed as the actual
|
||||
reasoning — inferred from the Tripod precedent, not stated outright):** GPIO is shared,
|
||||
stateful hardware that multiple capsules/patrons might want to touch — the same category of
|
||||
problem Hermes (messaging) and Artemis (block storage) already solve by being a dedicated,
|
||||
addressable VM other patrons talk to, rather than a bare C primitive anyone can call directly.
|
||||
|
||||
**Explicitly not decided yet:**
|
||||
- Whether Milk-V Mars needs the same pinned-VM treatment, or something else.
|
||||
- What this VM's own vocabulary/interface would look like (a `GPIO-READ`/`GPIO-WRITE`-shaped
|
||||
word set? something richer, given the Pi 5's GPIO header also muxes UART/SPI per §IV.1?).
|
||||
- Whether this is one pinned VM per board doing double duty (GPIO + whatever else that board's
|
||||
bring-up needs) or GPIO specifically warrants its own separate VM distinct from, say, a
|
||||
future UART-handling VM.
|
||||
- How this interacts with §IV.1's own still-open native-boot-flow work — a GPIO VM needs GPIO
|
||||
register addresses from the DTB the same way the rest of that bring-up does, so this idea
|
||||
isn't independent of that work, just downstream of it.
|
||||
Reference in New Issue
Block a user