admin/LithosAnanake
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FABRIC-3.10.8 MC-1a: the core table, and amd64's second core online and parked

Browse Source 
hal/smp.c: dense logical core ids, a heap-sized core table, one-at-a-time
bring-up under a 1 s bound, park-wake reporting at the idle point.
amd64: MADT enumeration, a relocation-free real->long-mode trampoline
copied below 1 MiB, INIT-SIPI-SIPI via the LAPIC ICR; QEMU -smp 2.
Core 1 reports online with GS on its own block, parks with 0 wakes;
aarch64/riscv64 report 1 core until MC-1b/c. 3-ISA acceptance clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-26 12:40:51 -04:00
co-authored by Claude Opus 5.5
parent 8c068e00e3
commit 24bde47481
19 changed files with 37381 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
FABRIC-3.10.8.md
+37
View File
@@ -837,3 +837,40 @@ The ledger reads pulled 319488 / returned 316703 on all three. That is the K-3 b
- **Parked, not spinning:** no park-wake line in the whole run. On the host, core 1's QEMU vCPU thread sits near 0% CPU (`ps -L`), if TCG gives it its own thread.
- **aarch64 and riscv64:** they report 1 core and are otherwise unchanged.
- **Standard sequence on all three:** the K-10 dict_hash set, G A=40 B=40 byte-identical (still one worker core), all ok `3`, 0 B idle growth.
### 5.1 MC-1a — the core table, and amd64's second core online and parked (2026-09-26)
**Change.**
- **`include/starkernel/smp.h`, `hal/smp.c`: the core table.** It holds dense logical ids, is heap-sized from enumeration, and gives each extra core a heap block and a 16 KiB stack.
- `sk_smp_init()` brings the cores up one at a time, each waited for under a 1 s bound on the raw counter.
- `sk_smp_secondary_main()` records the per-CPU register readback, marks the core online with a release store, and parks.
- `sk_smp_report()` runs at the idle point, after the lock report, and logs park wakes only when they change.
- aarch64 and riscv64 report their boot core only, through stubs in `hal/smp.c` that MC-1b/c replace. Their "hw id 0x0" is that stub, not a real MPIDR or hartid.
- **`sk_percpu_t`** gains `hw_id`, `reg_readback`, `online` and `park_wakes`.
- **`arch/amd64/smp.c`:**
- MADT enumeration: enabled type-0 entries, then type-9 entries with ids up to 255, de-duplicated.
- The trampoline page: the lowest conventional page in [4 KiB, 1 MiB) from the UEFI memory map.
- The control-state copy: CR0/CR3/CR4, CR4 without PCIDE for the 32-bit step, EFER without LMA, PAT, and XCR0 when OSXSAVE is set. The kernel GDTR and IDTR are copied too.
- INIT, 10 ms, STARTUP, 200 µs, then a second STARTUP only if the core isn't online yet.
- **`arch/amd64/ap_trampoline.S`:** the relocation-free blob (§5 path). **`apic_send_ipi()`** in `apic.c` writes ICR high then low, and polls delivery status.
- **Wiring:** `kernel_main_deep()` calls `sk_smp_init()` after the heartbeat init. The Makefile builds the two amd64 files into the loader, and `-smp 2` is on the `make qemu` amd64 line only.
**Two failed debug boots, found and fixed; their logs are deleted as flawed:**
1. **Core 1 triple-faulted and reset the machine.** QEMU runs with `-no-reboot`, so it exited just after the "2 core(s) found" line.
2. **A rerun with `QEMU_EXTRA="-d int,cpu_reset"`** showed core 1 had reached 64-bit mode and loaded the kernel GDT. It then page-faulted on `push 0x08`, the far return's first push, with RSP still 0 (CR2 `0xfffffffffffffff8`).
- The fix loads the stack before the far return.
- EFER.LMA is also masked out of the copied value, since only the CPU may set it.
3. **The traced rerun after the fix** (`logs/20260926-120808`, kept): 0 exceptions on either core over the boot.
**Evidence** (clean builds, no trace; the tracked diff is byte-identical to the saved stage diff):
| ISA | log | SMP lines | standard sequence |
|---|---|---|---|
| amd64 (`-smp 2`) | `logs/20260926-121008` | `SMP: 2 core(s) found; core 0 is the boot core, hw id 0x0`; `SMP: core 1 online: hw id 0x1, block 0x0000000001780130, per-CPU register 0x0000000001780130 (its own block), 59 us, parked`; `SMP: 2 core(s) online` | K-10 hashes, G A=40 B=40 byte-identical, HHH, all ok `3`, ledger unchanged, 0 B idle, 0 / 0; **0 park-wake lines, 0 `LOCK:` lines** |
| aarch64 | `logs/20260926-122030` | 1 core found, 1 online | same |
| riscv64 | `logs/20260926-123103` | 1 core found, 1 online | same |
- The 59 µs is measured from the end of `arch_smp_start()`, which already includes the 10 ms INIT wait and the 200 µs STARTUP wait. The core arrived during the second STARTUP's delivery, not 59 µs after INIT.
- **Not as named in advance:** the host-side check (core 1's vCPU thread near 0% in `ps -L`) could not be made. QEMU names its threads per vCPU only with `-name debug-threads=on`, which is not on the command line. The park-wake counter (0 wakes) is the evidence of record. Adding `-name debug-threads=on` is a candidate for MC-1b's run line.
**MC-1a is complete.** amd64 now boots two cores, and the second is parked with interrupts masked. Next: MC-1b (aarch64, PSCI `CPU_ON`).
Makefile.starkernel
+4 -1
View File
@@ -480,6 +480,7 @@ LOADER_ASM := \
ifeq ($(ARCH),amd64)
LOADER_ASM += $(KERNEL_SRC)/arch/$(ARCH)/kernel_entry.S
LOADER_ASM += $(KERNEL_SRC)/arch/$(ARCH)/ap_trampoline.S
endif
ifeq ($(ARCH),riscv64)
LOADER_ASM += $(KERNEL_SRC)/arch/$(ARCH)/kernel_entry.S
@@ -494,7 +495,8 @@ LOADER_ARCH_SRCS := \
ifeq ($(ARCH),amd64)
LOADER_ARCH_SRCS += \
$(KERNEL_SRC)/arch/$(ARCH)/ioapic.c \
$(KERNEL_SRC)/arch/$(ARCH)/i8042.c
$(KERNEL_SRC)/arch/$(ARCH)/i8042.c \
$(KERNEL_SRC)/arch/$(ARCH)/smp.c
endif
ifeq ($(ARCH),riscv64)
LOADER_ARCH_SRCS += \
@@ -865,6 +867,7 @@ ifeq ($(ARCH),amd64)
qemu-system-x86_64 \
-machine q35,accel=tcg \
-cpu qemu64 \
-smp 2 \
-m 1024 \
-vga std \
-drive if=pflash,format=raw,readonly=on,file=$(OVMF_CODE) \
experiments/bare_metal/runs/doe-aarch64-20260926-122030.csv
View File
experiments/bare_metal/runs/doe-amd64-20260926-120808.csv
View File
experiments/bare_metal/runs/doe-amd64-20260926-121008.csv
View File
experiments/bare_metal/runs/doe-riscv64-20260926-123103.csv
View File
include/starkernel/apic.h
+5
View File
@@ -67,6 +67,11 @@ int apic_init(BootInfo *boot_info);
* destination field when programming I/O APIC redirection entries. */
uint8_t apic_id(void);
/* amd64 (FABRIC-3.10.8 MC-1a): send one IPI -- ICR high = destination,
* then ICR low = `icr_low` -- and wait for its delivery status to clear.
* Returns 0 once delivered, -1 if still pending after the bound. */
int apic_send_ipi(uint8_t dest_apic_id, uint32_t icr_low);
/* Send End-of-Interrupt signal */
void apic_eoi(void);
include/starkernel/percpu.h
+6
View File
@@ -61,6 +61,12 @@ typedef struct sk_percpu {
/* K-11 (spinlock.c): locks this core holds right now, per rank -- the
* lock-order check compares a new lock's rank with the highest held. */
uint32_t held_rank[SK_LOCK_RANK_COUNT];
/* MC-1 (hal/smp.c): the core's hardware id (APIC id / MPIDR / hartid;
* never the lock owner word), and what it reports on coming online. */
uint64_t hw_id;
uint64_t reg_readback; /* its per-CPU register, read on arrival */
volatile uint32_t online; /* 1 once parked (release store) */
volatile uint64_t park_wakes; /* times woken while parked */
} sk_percpu_t;
/* Installs the boot core's block in this core's per-CPU register. Call
include/starkernel/smp.h
+63
View File
@@ -0,0 +1,63 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* smp.h - The core table and secondary-core bring-up (FABRIC-3.10.8 MC-1).
*
* Core ids are dense and logical: core 0 is the boot core, whatever its
* hardware id; the others are numbered in enumeration order. The hardware
* id lives in the core's block (sk_percpu_t.hw_id) and never in a lock's
* owner word.
*
* MC-1: a secondary core comes up, reports, and parks -- interrupts masked,
* halted, every wake counted. It does no console, log or heap work.
*/
#ifndef STARKERNEL_SMP_H
#define STARKERNEL_SMP_H
#include <stdint.h>
#include "starkernel/percpu.h"
#include "uefi.h" /* BootInfo */
/* Enumerate the cores, bring each secondary up in turn and wait (bounded)
* for it to park. Boot core only, interrupts off; after the heartbeat
* init. Prints one line per core. */
void sk_smp_init(BootInfo *boot_info);
uint32_t sk_smp_core_count(void);
sk_percpu_t *sk_smp_core(uint32_t core_id);
/* The idle point: logs a parked core's wake count when it has changed.
* Call with no lock held. */
void sk_smp_report(void);
/* Where every secondary core's entry path lands, with its per-CPU register
* already set to `pc`. Never returns. */
void sk_smp_secondary_main(sk_percpu_t *pc) __attribute__((noreturn));
/* ---- per ISA (arch/<isa>/smp.c) ---------------------------------------- */
/* The cores this machine has, boot core included: fills hw_ids[0..cap) in
* enumeration order and returns the total (call with cap 0 to count). */
uint32_t arch_smp_enumerate(BootInfo *boot_info, uint64_t *hw_ids, uint32_t cap);
/* The boot core's own hardware id. */
uint64_t arch_smp_boot_hw_id(void);
/* Start the core `pc` describes on `stack_top`; returns 0 once the start
* sequence was issued (arrival is observed through pc->online). */
int arch_smp_start(BootInfo *boot_info, sk_percpu_t *pc, void *stack_top);
/* One park step: interrupts stay masked; returns when the core wakes. */
void arch_smp_park_once(void);
#endif /* STARKERNEL_SMP_H */
logs/20260926-120808/amd64/qemu-amd64-20260926-120808.log
+9125
View File
File diff suppressed because it is too large Load Diff
logs/20260926-121008/amd64/qemu-amd64-20260926-121008.log
+9158
View File
File diff suppressed because it is too large Load Diff
logs/20260926-122030/aarch64/qemu-aarch64-20260926-122030.log
+9129
View File
File diff suppressed because it is too large Load Diff
logs/20260926-123103/riscv64/qemu-riscv64-20260926-123103.log
+9200
View File
File diff suppressed because it is too large Load Diff
src/starkernel/arch/amd64/ap_trampoline.S
+164
View File
@@ -0,0 +1,164 @@
/*
* ap_trampoline.S -- secondary-core entry (amd64), FABRIC-3.10.8 MC-1a.
*
* A position-independent blob, copied by arch/amd64/smp.c into a page T
* below 1 MiB; the STARTUP IPI's vector is T >> 12, so the core starts in
* real mode at CS = T >> 4, IP = 0. The blob holds no relocations: every
* absolute address it needs is patched into it by smp.c at copy time (the
* two far-jump offsets, its own GDT's base, and the data block below).
*
* Path: real mode -> protected mode (own GDT, flat 32-bit segments) ->
* the boot core's CR4 (without PCIDE, which only long mode accepts), CR3,
* EFER, PAT, then CR0 with PG, which activates long mode -> 64-bit code ->
* the kernel's GDT and IDT, segment registers reloaded, and only then the
* GS base (a GS reload clears it) -> the full CR4 and CR3, XCR0 if the
* boot core uses it, FPU/SSE reset -> sk_smp_secondary_main(block) on the
* core's own stack, SysV ABI. Interrupts stay masked throughout.
*
* %ebx holds T from the first instructions on.
*/
.intel_syntax noprefix
.section .text
.balign 16
.global sk_ap_tramp_start
.global sk_ap_tramp_end
.global sk_ap_tr_pm_jmp
.global sk_ap_tr_lm_jmp
.global sk_ap_tr_pm32
.global sk_ap_tr_lm64
.global sk_ap_tr_gdt
.global sk_ap_tr_gdtr
.global sk_ap_tr_data
/* Offsets into the data block (arch/amd64/smp.c: ap_tramp_data_t). */
.equ D_CR0, 0
.equ D_CR3, 8
.equ D_CR4_BOOT, 16
.equ D_CR4, 24
.equ D_EFER, 32
.equ D_PAT, 40
.equ D_XCR0, 48
.equ D_XCR0_VALID, 56
.equ D_PERCPU, 64
.equ D_STACK, 72
.equ D_ENTRY, 80
.equ D_KGDTR, 88
.equ D_KIDTR, 98
.code16
sk_ap_tramp_start:
cli
cld
mov ax, cs
mov ds, ax
xor ebx, ebx
mov bx, ax
shl ebx, 4 /* ebx = T */
lgdt [sk_ap_tr_gdtr - sk_ap_tramp_start]
mov eax, cr0
or eax, 1
mov cr0, eax
/* jmp 0x08:(T + pm32) -- ptr16:32, offset patched */
.byte 0x66, 0xEA
sk_ap_tr_pm_jmp:
.long 0
.word 0x08
.code32
sk_ap_tr_pm32:
mov ax, 0x10
mov ds, ax
mov es, ax
mov ss, ax
mov fs, ax
mov gs, ax
lea esi, [ebx + (sk_ap_tr_data - sk_ap_tramp_start)]
mov eax, [esi + D_CR4_BOOT]
mov cr4, eax
mov eax, [esi + D_CR3]
mov cr3, eax
mov ecx, 0xC0000080 /* IA32_EFER */
mov eax, [esi + D_EFER]
mov edx, [esi + D_EFER + 4]
wrmsr
mov ecx, 0x277 /* IA32_PAT */
mov eax, [esi + D_PAT]
mov edx, [esi + D_PAT + 4]
wrmsr
mov eax, [esi + D_CR0]
mov cr0, eax /* PG: long mode, compatibility */
/* jmp 0x18:(T + lm64) -- offset patched */
.byte 0xEA
sk_ap_tr_lm_jmp:
.long 0
.word 0x18
.code64
sk_ap_tr_lm64:
mov ebx, ebx /* upper halves are undefined after the switch */
lea rsi, [rbx + (sk_ap_tr_data - sk_ap_tramp_start)]
mov rsp, [rsi + D_STACK] /* the far return below pushes */
and rsp, -16
lgdt [rsi + D_KGDTR]
lea rax, [rbx + (.Llm_cs - sk_ap_tramp_start)]
push 0x08
push rax
retfq
.Llm_cs:
mov ax, 0x10
mov ds, ax
mov es, ax
mov ss, ax
xor eax, eax
mov fs, ax
mov gs, ax
lidt [rsi + D_KIDTR]
mov rax, [rsi + D_CR4]
mov cr4, rax
mov rax, [rsi + D_CR3]
mov cr3, rax
mov rax, [rsi + D_XCR0_VALID]
test rax, rax
jz 2f
xor ecx, ecx
mov eax, [rsi + D_XCR0]
mov edx, [rsi + D_XCR0 + 4]
xsetbv
2:
mov rdi, [rsi + D_PERCPU]
mov ecx, 0xC0000101 /* IA32_GS_BASE */
mov eax, edi
mov rdx, rdi
shr rdx, 32
wrmsr
mov rsp, [rsi + D_STACK] /* again: an aligned, empty stack for the call */
and rsp, -16
fninit
sub rsp, 16
mov dword ptr [rsp], 0x1F80 /* MXCSR default */
ldmxcsr [rsp]
add rsp, 16
mov rax, [rsi + D_ENTRY]
call rax /* sk_smp_secondary_main(rdi), never returns */
3:
cli
hlt
jmp 3b
.balign 16
sk_ap_tr_gdt:
.quad 0
.quad 0x00CF9A000000FFFF /* 0x08: 32-bit code, flat */
.quad 0x00CF92000000FFFF /* 0x10: data, flat */
.quad 0x00AF9A000000FFFF /* 0x18: 64-bit code */
sk_ap_tr_gdt_end:
sk_ap_tr_gdtr:
.word sk_ap_tr_gdt_end - sk_ap_tr_gdt - 1
.long 0 /* patched: T + (gdt - start) */
.balign 8
sk_ap_tr_data:
.space 108 /* ap_tramp_data_t, filled by smp.c */
sk_ap_tramp_end:
src/starkernel/arch/amd64/apic.c
+18
View File
@@ -348,6 +348,24 @@ uint8_t apic_id(void) {
return (uint8_t)(lapic_read(APIC_REG_ID) >> 24);
}
/* FABRIC-3.10.8 MC-1a: the Interrupt Command Register (xAPIC MMIO, as the
* timer uses), for INIT and STARTUP IPIs to a secondary core. */
#define APIC_REG_ICR_LOW 0x300
#define APIC_REG_ICR_HIGH 0x310
#define ICR_DELIVERY_PENDING (1u << 12)
int apic_send_ipi(uint8_t dest_apic_id, uint32_t icr_low)
{
uint32_t spin;
lapic_write(APIC_REG_ICR_HIGH, (uint32_t)dest_apic_id << 24);
lapic_write(APIC_REG_ICR_LOW, icr_low);
for (spin = 0; spin < 10000000u; spin++) {
if (!(lapic_read(APIC_REG_ICR_LOW) & ICR_DELIVERY_PENDING)) return 0;
__asm__ volatile ("pause");
}
return -1; /* still pending: reported by the caller */
}
/* ============================================================================
* APIC Timer
* ============================================================================ */
src/starkernel/arch/amd64/smp.c
+263
View File
@@ -0,0 +1,263 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* smp.c (amd64) - secondary-core bring-up, FABRIC-3.10.8 MC-1a.
*
* Cores come from the MADT's enabled Local APIC entries. Each is started
* with INIT, 10 ms, STARTUP, 200 us, STARTUP into ap_trampoline.S, copied
* into a conventional-memory page below 1 MiB taken from the UEFI memory
* map. The trampoline enters long mode on the boot core's own control
* state and calls sk_smp_secondary_main() with its GS base already set.
*
* Limit (recorded for SER5): the trampoline loads CR3 in 32-bit mode, so
* the page tables must lie below 4 GiB; a start is refused otherwise.
*/
#include "starkernel/smp.h"
#include "starkernel/apic.h"
#include "starkernel/timer.h"
#include "starkernel/console.h"
#include "uefi.h"
extern char sk_ap_tramp_start[], sk_ap_tramp_end[];
extern char sk_ap_tr_pm_jmp[], sk_ap_tr_lm_jmp[], sk_ap_tr_pm32[], sk_ap_tr_lm64[];
extern char sk_ap_tr_gdt[], sk_ap_tr_gdtr[], sk_ap_tr_data[];
/* The trampoline's data block (ap_trampoline.S, D_* offsets). */
typedef struct __attribute__((packed)) {
uint64_t cr0, cr3, cr4_boot, cr4, efer, pat, xcr0, xcr0_valid;
uint64_t percpu, stack, entry;
uint16_t kgdt_limit; uint64_t kgdt_base;
uint16_t kidt_limit; uint64_t kidt_base;
} ap_tramp_data_t;
#define CR4_PCIDE (1ull << 17)
#define CR4_OSXSAVE (1ull << 18)
#define ICR_INIT 0x00004500u /* INIT, level assert */
#define ICR_STARTUP 0x00004600u /* STARTUP; low byte = vector (page number) */
/* ---- MADT ---------------------------------------------------------------- */
typedef struct __attribute__((packed)) {
uint8_t signature[8];
uint8_t checksum;
uint8_t oem_id[6];
uint8_t revision;
uint32_t rsdt_address;
uint32_t length;
uint64_t xsdt_address;
} smp_rsdp_t;
typedef struct __attribute__((packed)) {
uint8_t signature[4];
uint32_t length;
uint8_t rest[28];
} smp_sdt_t;
static int sig_eq(const uint8_t *p, const char *s, int n)
{
int i;
for (i = 0; i < n; i++) if (p[i] != (uint8_t)s[i]) return 0;
return 1;
}
static const uint8_t *find_madt(void *rsdp_ptr)
{
const smp_rsdp_t *r = (const smp_rsdp_t *)rsdp_ptr;
const smp_sdt_t *xsdt;
const uint64_t *ent;
uint32_t n, i;
if (!r || !sig_eq(r->signature, "RSD PTR ", 8) || r->revision < 2 || !r->xsdt_address) return 0;
xsdt = (const smp_sdt_t *)(uintptr_t)r->xsdt_address;
if (!sig_eq(xsdt->signature, "XSDT", 4) || xsdt->length <= 36u) return 0;
n = (xsdt->length - 36u) / 8u;
ent = (const uint64_t *)((const uint8_t *)xsdt + 36u);
for (i = 0; i < n; i++) {
const smp_sdt_t *sdt = (const smp_sdt_t *)(uintptr_t)ent[i];
if (sdt && sig_eq(sdt->signature, "APIC", 4)) return (const uint8_t *)sdt;
}
return 0;
}
static int id_seen(const uint64_t *ids, uint32_t n, uint64_t id)
{
uint32_t i;
for (i = 0; i < n; i++) if (ids[i] == id) return 1;
return 0;
}
/* Enabled type-0 (Local APIC) entries, then type-9 (x2APIC) entries whose
* id fits xAPIC (<= 255) and was not already listed. */
uint32_t arch_smp_enumerate(BootInfo *boot_info, uint64_t *hw_ids, uint32_t cap)
{
const uint8_t *madt = find_madt(boot_info ? boot_info->acpi_table : (void *)0);
const uint8_t *p, *end;
uint32_t n = 0, pass;
if (!madt) {
if (cap) hw_ids[0] = apic_id();
return 1;
}
end = madt + ((const smp_sdt_t *)madt)->length;
for (pass = 0; pass < 2; pass++) {
for (p = madt + 44; p + 2 <= end; p += p[1]) {
uint64_t id;
uint32_t flags;
if (p[1] == 0 || p + p[1] > end) break;
if (pass == 0 && p[0] == 0 && p[1] >= 8) {
id = p[3];
flags = (uint32_t)p[4] | ((uint32_t)p[5] << 8) | ((uint32_t)p[6] << 16) | ((uint32_t)p[7] << 24);
} else if (pass == 1 && p[0] == 9 && p[1] >= 16) {
id = (uint64_t)p[4] | ((uint64_t)p[5] << 8) | ((uint64_t)p[6] << 16) | ((uint64_t)p[7] << 24);
flags = (uint32_t)p[8] | ((uint32_t)p[9] << 8) | ((uint32_t)p[10] << 16) | ((uint32_t)p[11] << 24);
if (id > 255u) continue;
} else {
continue;
}
if (!(flags & 1u)) continue; /* not enabled */
if (cap) {
if (id_seen(hw_ids, n < cap ? n : cap, id)) continue;
if (n < cap) hw_ids[n] = id;
}
n++;
}
}
if (n == 0) {
if (cap) hw_ids[0] = apic_id();
return 1;
}
return n;
}
uint64_t arch_smp_boot_hw_id(void) { return apic_id(); }
void arch_smp_park_once(void)
{
__asm__ volatile ("cli; hlt" ::: "memory");
}
/* ---- the trampoline page ------------------------------------------------- */
static uint64_t g_tramp_page; /* 0 = not chosen yet */
/* The lowest conventional page in [0x1000, 1 MiB) whose whole 4 KiB is
* free in the UEFI memory map. */
static uint64_t pick_tramp_page(BootInfo *bi)
{
uint64_t best = 0;
UINTN n, i;
if (!bi || !bi->memory_map || !bi->memory_map_descriptor_size) return 0;
n = bi->memory_map_size / bi->memory_map_descriptor_size;
for (i = 0; i < n; i++) {
const EFI_MEMORY_DESCRIPTOR *d = (const EFI_MEMORY_DESCRIPTOR *)
((const uint8_t *)bi->memory_map + i * bi->memory_map_descriptor_size);
uint64_t start, end;
if (d->Type != EfiConventionalMemory) continue;
start = d->PhysicalStart;
end = start + d->NumberOfPages * 4096ull;
if (start < 0x1000ull) start = 0x1000ull;
start = (start + 0xFFFull) & ~0xFFFull;
if (start + 4096ull > end || start + 4096ull > 0x100000ull) continue;
if (!best || start < best) best = start;
}
return best;
}
static uint64_t rdmsr64(uint32_t msr)
{
uint32_t lo, hi;
__asm__ volatile ("rdmsr" : "=a"(lo), "=d"(hi) : "c"(msr));
return ((uint64_t)hi << 32) | lo;
}
static void wait_us(uint64_t us)
{
uint64_t hz = timer_tsc_hz(), t0 = heartbeat_read_counter();
uint64_t budget = hz ? (hz / 1000000u) * us : 0;
uint64_t spin = us * 1000u; /* only if the counter's rate is unknown */
while (hz ? (heartbeat_read_counter() - t0 < budget) : (spin-- > 0)) {
__asm__ volatile ("pause");
}
}
int arch_smp_start(BootInfo *boot_info, sk_percpu_t *pc, void *stack_top)
{
uint8_t *page;
size_t len = (size_t)(sk_ap_tramp_end - sk_ap_tramp_start), k;
ap_tramp_data_t *d;
uint64_t cr0, cr3, cr4;
struct __attribute__((packed)) { uint16_t limit; uint64_t base; } gdtr, idtr;
uint32_t T;
if (pc->hw_id > 255u || len > 4096u) return -1;
if (!g_tramp_page) g_tramp_page = pick_tramp_page(boot_info);
if (!g_tramp_page) {
console_println("SMP: no conventional page below 1 MiB for the trampoline");
return -1;
}
__asm__ volatile ("mov %%cr0, %0" : "=r"(cr0));
__asm__ volatile ("mov %%cr3, %0" : "=r"(cr3));
__asm__ volatile ("mov %%cr4, %0" : "=r"(cr4));
if (cr3 >> 32) {
console_println("SMP: page tables above 4 GiB -- the trampoline cannot load CR3");
return -1;
}
__asm__ volatile ("sgdt %0" : "=m"(gdtr));
__asm__ volatile ("sidt %0" : "=m"(idtr));
T = (uint32_t)g_tramp_page;
page = (uint8_t *)(uintptr_t)g_tramp_page;
for (k = 0; k < len; k++) page[k] = (uint8_t)sk_ap_tramp_start[k];
*(uint32_t *)(void *)(page + (sk_ap_tr_pm_jmp - sk_ap_tramp_start)) =
T + (uint32_t)(sk_ap_tr_pm32 - sk_ap_tramp_start);
*(uint32_t *)(void *)(page + (sk_ap_tr_lm_jmp - sk_ap_tramp_start)) =
T + (uint32_t)(sk_ap_tr_lm64 - sk_ap_tramp_start);
*(uint32_t *)(void *)(page + (sk_ap_tr_gdtr - sk_ap_tramp_start) + 2) =
T + (uint32_t)(sk_ap_tr_gdt - sk_ap_tramp_start);
d = (ap_tramp_data_t *)(void *)(page + (sk_ap_tr_data - sk_ap_tramp_start));
d->cr0 = cr0;
d->cr3 = cr3;
d->cr4 = cr4;
d->cr4_boot = cr4 & ~CR4_PCIDE;
d->efer = rdmsr64(0xC0000080u) & ~(1ull << 10); /* LMA is the CPU's to set */
d->pat = rdmsr64(0x277u);
d->xcr0_valid = (cr4 & CR4_OSXSAVE) ? 1u : 0u;
d->xcr0 = 0;
if (d->xcr0_valid) {
uint32_t lo, hi;
__asm__ volatile ("xgetbv" : "=a"(lo), "=d"(hi) : "c"(0));
d->xcr0 = ((uint64_t)hi << 32) | lo;
}
d->percpu = (uint64_t)(uintptr_t)pc;
d->stack = (uint64_t)(uintptr_t)stack_top;
d->entry = (uint64_t)(uintptr_t)&sk_smp_secondary_main;
d->kgdt_limit = gdtr.limit; d->kgdt_base = gdtr.base;
d->kidt_limit = idtr.limit; d->kidt_base = idtr.base;
__asm__ volatile ("mfence" ::: "memory");
if (apic_send_ipi((uint8_t)pc->hw_id, ICR_INIT) != 0) {
console_println("SMP: INIT IPI not delivered");
return -1;
}
wait_us(10000u);
(void)apic_send_ipi((uint8_t)pc->hw_id, ICR_STARTUP | (T >> 12));
wait_us(200u);
if (!__atomic_load_n(&pc->online, __ATOMIC_ACQUIRE)) {
(void)apic_send_ipi((uint8_t)pc->hw_id, ICR_STARTUP | (T >> 12));
}
return 0;
}
src/starkernel/hal/smp.c
+202
View File
@@ -0,0 +1,202 @@
/*
StarForth — Steady-State Virtual Machine Runtime
Copyright (c) 2023–2025 Robert A. James
All rights reserved.
This file is part of the StarForth project.
Licensed under the StarForth License, Version 1.0 (the "License");
you may not use this file except in compliance with the License.
*/
/**
* smp.c - The core table and secondary-core bring-up (FABRIC-3.10.8 MC-1).
* See starkernel/smp.h. MC-1a implements amd64; aarch64 and riscv64 report
* their boot core only until MC-1b and MC-1c.
*/
#include "starkernel/smp.h"
#include "starkernel/kmalloc.h"
#include "starkernel/console.h"
#include "starkernel/timer.h" /* heartbeat_read_counter(), timer_tsc_hz() */
#include "freestanding/stdio.h" /* snprintf() */
#include "log.h"
#define SMP_STACK_BYTES (16u * 1024u) /* a parked core's stack (MC-1) */
#define SMP_ONLINE_MS 1000u /* bound on one core's arrival */
/* Heap-sized to the enumeration: g_cores[0] is the boot core's block. */
static sk_percpu_t **g_cores;
static uint32_t g_core_count = 1;
static uint64_t *g_wakes_reported;
uint32_t sk_smp_core_count(void) { return g_core_count; }
sk_percpu_t *sk_smp_core(uint32_t core_id)
{
if (core_id == 0 && !g_cores) return sk_percpu();
return (g_cores && core_id < g_core_count) ? g_cores[core_id] : (sk_percpu_t *)0;
}
/* Freestanding snprintf has no %llx: hex by hand. */
static void hex64(char out[19], uint64_t v)
{
int i;
out[0] = '0'; out[1] = 'x';
for (i = 17; i >= 2; i--) {
uint32_t n = (uint32_t)(v & 0xFu);
out[i] = (char)(n < 10u ? '0' + n : 'a' + (n - 10u));
v >>= 4;
}
out[18] = '\0';
}
void sk_smp_secondary_main(sk_percpu_t *pc)
{
/* The entry path set this core's per-CPU register before calling here;
* read it back so the boot core can check it names this block. */
pc->reg_readback = (uint64_t)(uintptr_t)sk_percpu();
__atomic_store_n(&pc->online, 1u, __ATOMIC_RELEASE);
for (;;) {
arch_smp_park_once();
__atomic_add_fetch(&pc->park_wakes, 1u, __ATOMIC_RELAXED);
}
}
#if defined(__x86_64__)
#define SMP_ISA_BRINGS_UP 1
#else
/* MC-1b (aarch64) and MC-1c (riscv64): not yet -- the boot core only. */
#define SMP_ISA_BRINGS_UP 0
uint32_t arch_smp_enumerate(BootInfo *b, uint64_t *ids, uint32_t cap)
{
(void)b;
if (cap) ids[0] = 0;
return 1;
}
uint64_t arch_smp_boot_hw_id(void) { return 0; }
int arch_smp_start(BootInfo *b, sk_percpu_t *pc, void *st) { (void)b; (void)pc; (void)st; return -1; }
void arch_smp_park_once(void) { }
#endif
void sk_smp_init(BootInfo *boot_info)
{
char line[160], a[19], b[19];
uint32_t n, i, next = 1;
uint64_t boot_id, *ids;
sk_percpu_t *boot = sk_percpu();
boot_id = arch_smp_boot_hw_id();
boot->hw_id = boot_id;
boot->online = 1;
n = SMP_ISA_BRINGS_UP ? arch_smp_enumerate(boot_info, (uint64_t *)0, 0) : 1;
if (n < 1) n = 1;
ids = (uint64_t *)kmalloc((size_t)n * sizeof(uint64_t));
g_cores = (sk_percpu_t **)kmalloc((size_t)n * sizeof(sk_percpu_t *));
g_wakes_reported = (uint64_t *)kmalloc((size_t)n * sizeof(uint64_t));
if (!ids || !g_cores || !g_wakes_reported) {
console_println("SMP: no memory for the core table -- boot core only");
if (ids) kfree(ids);
if (g_cores) kfree(g_cores);
if (g_wakes_reported) kfree(g_wakes_reported);
g_cores = (sk_percpu_t **)0; g_wakes_reported = (uint64_t *)0;
return;
}
if (SMP_ISA_BRINGS_UP) {
/* The filling pass drops duplicates the counting pass can't see. */
uint32_t filled = arch_smp_enumerate(boot_info, ids, n);
if (filled < n) n = filled;
} else {
ids[0] = boot_id;
}
for (i = 0; i < n; i++) { g_cores[i] = (sk_percpu_t *)0; g_wakes_reported[i] = 0; }
g_cores[0] = boot;
hex64(a, boot_id);
snprintf(line, sizeof line, "SMP: %u core(s) found; core 0 is the boot core, hw id %s",
(unsigned)n, a);
console_println(line);
/* One at a time: bring up, wait (bounded), report. */
for (i = 0; i < n; i++) {
sk_percpu_t *pc;
uint8_t *stack;
uint64_t hz, t0, budget, waited;
uint32_t k;
if (ids[i] == boot_id) continue;
pc = (sk_percpu_t *)kmalloc(sizeof(sk_percpu_t));
stack = (uint8_t *)kmalloc(SMP_STACK_BYTES);
if (!pc || !stack) {
if (pc) kfree(pc);
if (stack) kfree(stack);
console_println("SMP: no memory for a core's block -- stopping bring-up");
break;
}
{
uint8_t *z = (uint8_t *)pc;
for (k = 0; k < sizeof(sk_percpu_t); k++) z[k] = 0;
}
pc->self = pc;
pc->core_id = next;
pc->hw_id = ids[i];
hex64(a, ids[i]);
if (arch_smp_start(boot_info, pc, stack + SMP_STACK_BYTES) != 0) {
snprintf(line, sizeof line, "SMP: core hw id %s: start refused", a);
console_println(line);
kfree(pc); kfree(stack);
continue;
}
hz = timer_tsc_hz();
t0 = heartbeat_read_counter();
budget = hz ? (hz / 1000u) * SMP_ONLINE_MS : 0;
{
uint64_t spin = 0x40000000u; /* only if the counter's rate is unknown */
while (!__atomic_load_n(&pc->online, __ATOMIC_ACQUIRE)) {
if (hz ? (heartbeat_read_counter() - t0 >= budget) : (--spin == 0)) break;
}
}
waited = heartbeat_read_counter() - t0;
if (!__atomic_load_n(&pc->online, __ATOMIC_ACQUIRE)) {
/* Its block and stack stay allocated: the core may still arrive. */
snprintf(line, sizeof line, "SMP: core hw id %s did not come online within %u ms",
a, (unsigned)SMP_ONLINE_MS);
console_println(line);
continue;
}
g_cores[next++] = pc;
hex64(b, pc->reg_readback);
{
char blk[19];
hex64(blk, (uint64_t)(uintptr_t)pc);
snprintf(line, sizeof line,
"SMP: core %u online: hw id %s, block %s, per-CPU register %s (%s), %llu us, parked",
(unsigned)pc->core_id, a, blk, b,
pc->reg_readback == (uint64_t)(uintptr_t)pc &&
pc->reg_readback != (uint64_t)(uintptr_t)boot ? "its own block" : "WRONG",
(unsigned long long)(hz ? waited / (hz / 1000000u ? hz / 1000000u : 1u) : 0));
}
console_println(line);
}
g_core_count = next;
kfree(ids);
snprintf(line, sizeof line, "SMP: %u core(s) online", (unsigned)g_core_count);
console_println(line);
}
void sk_smp_report(void)
{
uint32_t i;
if (!g_cores || !g_wakes_reported) return;
for (i = 1; i < g_core_count; i++) {
uint64_t w = __atomic_load_n(&g_cores[i]->park_wakes, __ATOMIC_RELAXED);
if (w != g_wakes_reported[i]) {
g_wakes_reported[i] = w;
log_message(LOG_WARN, "SMP: core %u woke from park x%llu",
(unsigned)i, (unsigned long long)w);
}
}
}
src/starkernel/kernel_main.c
+5
View File
@@ -67,6 +67,7 @@ EFI_RUNTIME_SERVICES *g_sk_runtime_services = NULL;
#include "starkernel/capsule_vm_switch_signal.h" /* FABRIC-3.md §XXVIII Stage 3 */
#include "starkernel/vm/switch.h" /* sk_vm_switch_set_current() -- Stage 3 follow-on */
#include "starkernel/percpu.h" /* FABRIC-3.10.8 MC-0a -- sk_percpu_boot_init() */
#include "starkernel/smp.h" /* FABRIC-3.10.8 MC-1 -- sk_smp_init() */
#include "starkernel/artemis_sig.h" /* artemis_sig_check/genesis_stamp */
#include "starkernel/kmalloc.h"
#include "starkernel/repl.h"
@@ -516,6 +517,10 @@ static void kernel_main_deep(BootInfo *boot_info) {
heartbeat_init(tsc_hz, 100); /* 100 Hz tick rate */
console_println("Heartbeat: init done");
/* FABRIC-3.10.8 MC-1: find the other cores, bring each up and park it
* (interrupts are still off here; the raw counter bounds each wait). */
sk_smp_init(boot_info);
console_println("Kernel initialization complete.");
console_println("Boot successful!\n");
src/starkernel/vm/switch.c
+2
View File
@@ -36,6 +36,7 @@
#include "starkernel/vm/bootstrap/sk_vm_bootstrap.h" /* sk_get_mama_vm() -- the idle root */
#include "starkernel/percpu.h" /* FABRIC-3.10.8 MC-0a -- this core's current VM */
#include "starkernel/spinlock.h" /* FABRIC-3.10.8 MC-0b -- no lock crosses a switch */
#include "starkernel/smp.h" /* FABRIC-3.10.8 MC-1 -- sk_smp_report() */
#include <stdint.h>
/* Arch-specific (switch.S in each arch dir). sk_vm_switch_prime() builds
@@ -231,6 +232,7 @@ void sk_vm_idle_point(int may_drain_self) {
* which makes this the place to log any lock failure counted since. */
sk_lock_check_none_held("idle point");
sk_lock_report();
sk_smp_report(); /* FABRIC-3.10.8 MC-1: a parked core that woke */
if (!self || self != (VM *)sk_get_mama_vm()) return;
if (self->frames.count != 0) return; /* inside a word (KEY/EXPECT): not idle */