fix(raspi): the Pi 5's native path turns the MMU and caches on before any C; bounded mailbox; framebuffer address from dma-ranges
boards/BRINGUP.md R2/R3, FABRIC-3 SXXXIX. The entry stub cleans the cache over the image, builds an identity map (the image's GiB Normal, the rest Device) and enables MMU and caches at EL2 (or EL1); rpi5_mmu.c makes the devicetree's RAM Normal and the framebuffer uncached. Mailbox waits give up after one second; the buffer is cleaned around the exchange; the framebuffer address goes through dma-ranges, not a Pi 4 mask. New rpi5-rehearsal target runs the native image on QEMU virt at EL2. It found three more that would each have stopped the board: the PL011 shift (mine, c9519173; the host test asserted it), the GIC address not carried through /soc's ranges (new fdt_reg_address(), checked against the real Pi 5 DTB), and no timer IRQ at EL2 (HCR_EL2.IMO/FMO/AMO). Rehearsal: POST 1052 passed, same parity hash as UEFI. Standard acceptance on three ISAs: POST 1052, hash 0x82f4ae7cb2e2d424. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rajames
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
4da370f492
commit
0e2c60dac3
+9
-5
@@ -49,8 +49,8 @@ the prompt was told a key was waiting on every poll and read `0xFF` for ever.
|
||||
| # | What | Where | Effect |
|
||||
|---|---|---|---|
|
||||
| R1 | **Fixed 2026-10-13.** Every character the console printed was also written to a PL011 at `0x09000000`, QEMU's address, which on a Pi 5 is ordinary RAM. The console now finds its serial line from the devicetree's `/chosen/stdout-path`: on the Pi 5 that is `serial10`, the debug connector, a PL011 at `0x107d001000`. | `kernel/src/hal/console.c`, `kernel/src/hal/fdt_uart.c`; `tools/fdt_uart_test.c` reads the board's real devicetree | Checked off the board only. The first run is the board's. |
|
||||
| R2 | Entry is at EL2 with the MMU off. Nothing on this path has ever run. | `boards/raspi/README.md`; `FABRIC-3.md` IV.3 | Unknown. |
|
||||
| R3 | Two details of the framebuffer request are unverified: the size field of the allocate tag, and whether the address returned needs its top bits masked. The mode is fixed at 1920x1080x32. | `FABRIC-3.md` IV.3 item 3 | No picture, or a wrong one. With R1 unfixed there is then no channel at all. |
|
||||
| R2 | **Fixed 2026-10-14.** Entry is at EL2 with the MMU off, where every access is Device memory: unaligned accesses and atomics fault, nothing is cached. The entry stub now cleans the cache over the image, turns the MMU and caches on before any C (identity map: the image's GiB Normal in 2 MiB blocks, the rest Device), and routes interrupts to EL2; C then makes the rest of the devicetree's RAM Normal. Rehearsing it found and fixed three more: the PL011 register shift, the GIC address (not carried through `/soc`'s `ranges`), and the EL2 timer interrupt never being taken. | `native_rpi5_entry.S`, `rpi5_mmu.c`, `fdt.c` (`fdt_reg_address`), `apic.c`; `FABRIC-3.md` XXXIX | Rehearsed on QEMU virt at EL2 (section 3.2): POST passes, same parity hash. Not yet on the board. |
|
||||
| R3 | **Partly fixed 2026-10-14.** The address returned is now carried through the devicetree's `dma-ranges` (on the Pi 5 it is already physical; the old `& 0x3FFFFFFF` was the Pi 4's rule), every mailbox wait gives up after one second, the request buffer is cleaned to RAM and back around the exchange, and the framebuffer is mapped uncached. Still unverified: the size field of the allocate tag. The mode is fixed at 1920x1080x32. | `rpi5_mailbox.c`, `rpi5_mmu.c`; `FABRIC-3.md` IV.3 item 3, XXXIX | QEMU virt has no mailbox: none of this runs before the board. No picture shows as no framebuffer, not a hang. |
|
||||
| R4 | USB, Ethernet and GPIO are behind the RP1 chip on PCIe, which needs a Broadcom-specific driver that is not written. | `FABRIC-3.md` IV.3 item 5 | No USB on the Pi 5 whatever is done about A1. |
|
||||
| R5 | No hardware entropy source is known for the BCM2712. | `FABRIC-3.md` IV.2 | A5 stands. |
|
||||
|
||||
@@ -150,8 +150,12 @@ One QEMU at a time. Use a copy of the image: the kernel writes to its disk.
|
||||
- [ ] **SER5:** q35 with OVMF, the image as the only disk, no `-cdrom`, no virtio disk, no
|
||||
thumbdrive. This is the machine the README's "booted in QEMU" refers to.
|
||||
- [ ] **MILKV:** the command in `boards/milkv/README.md` (OpenSBI, U-Boot, `bootefi`).
|
||||
- [ ] **RASPI:** there is no QEMU model of the Pi 5. Nothing can be rehearsed; the first run
|
||||
is on the board.
|
||||
- [ ] **RASPI:** there is no QEMU model of the Pi 5, but its native path -- raw image, entry
|
||||
at EL2 with the MMU off, devicetree in x0 -- is rehearsed on QEMU virt:
|
||||
`make -f kernel/Makefile ARCH=aarch64 RPI5_NATIVE_BASE=0x40080000 rpi5-rehearsal`
|
||||
(log under `logs/<TS>/aarch64-rpi5/`). Not rehearsed: the mailbox and framebuffer, the
|
||||
Pi 5's own UART and GIC addresses (checked against its real devicetree by
|
||||
`tools/fdt_uart_test.c` instead), and PCI (virt's needs ACPI, the Pi 5's is R4).
|
||||
|
||||
### 3.3 What to settle in the rehearsal
|
||||
|
||||
@@ -167,7 +171,7 @@ One QEMU at a time. Use a copy of the image: the kernel writes to its disk.
|
||||
## 4. On the bench
|
||||
|
||||
Do one board at a time, in this order: SER5, then the Mars, then the Pi 5. The SER5's path is
|
||||
the one QEMU exercises most closely, and the Pi 5's cannot be rehearsed at all.
|
||||
the one QEMU exercises most closely, and the Pi 5's can be rehearsed only in part (3.2).
|
||||
|
||||
### 4.1 Writing the media, every board
|
||||
|
||||
|
||||
@@ -5676,3 +5676,63 @@ ERROR probe, sent by itself at once, under source `00` (`-114847`); the lost-rec
|
||||
(`-120526`) and riscv64 (`-120746`). Standard acceptance, all three ISAs: `zuse@Hera`, POST 1052
|
||||
passed, no unknown word, **new baseline `word_count=526 ... hash=0x82f4ae7cb2e2d424`**
|
||||
(deliberate: Hera's new words) (`logs/20261014-120027`, `-120148`, `-120356`).
|
||||
|
||||
## XXXIX. The Pi 5's native path: MMU and caches on before any C, bounded mailbox, framebuffer address from the devicetree -- and a rehearsal on QEMU that found three more (2026-10-14)
|
||||
|
||||
Captain Bob, 2026-10-14: "fix the pi 5 problems" -- the three from the Friday review (`boards/BRINGUP.md`
|
||||
R2, R3).
|
||||
|
||||
**1. MMU and caches off (R2).** The firmware enters at EL2 with the MMU off: every access is
|
||||
Device memory, where an unaligned access or an exclusive faults and nothing is cached. The
|
||||
aarch64 kernel had never needed to turn the MMU on -- under UEFI it runs on the firmware's map
|
||||
(`arch_mmu_init()` is a stub; `vmm.c` builds x86-shaped tables it never loads on aarch64).
|
||||
`native_rpi5_entry.S` now: cleans and invalidates the data cache over the whole image first
|
||||
(so nothing the firmware left there lands on top of what follows); builds an identity map, the
|
||||
image's GiB Normal write-back in 2 MiB blocks, every other GiB Device-nGnRE and never executable;
|
||||
sets MAIR, TCR and TTBR0 and turns on M, C and I at whichever EL it was entered at (EL2 on the
|
||||
Pi 5, EL1 handled too), with WXN and alignment checking off. In C, `rpi5_mmu.c` then makes every
|
||||
further GiB the devicetree's `/memory` covers entirely Normal, and makes the framebuffer Normal
|
||||
non-cacheable (the display reads it straight from RAM), each change break-before-make.
|
||||
|
||||
**2. Unbounded mailbox waits (R3).** Every wait now gives up after one second by the
|
||||
architectural counter (`-5`); before any console exists a silent VideoCore must cost a missing
|
||||
framebuffer, not a hang. With the caches on, the request buffer is cleaned to RAM before it is
|
||||
sent and invalidated before the answer is read: the VideoCore does not see the ARM's caches.
|
||||
|
||||
**3. The framebuffer address (R3).** `& 0x3FFFFFFF` was the Pi 4's rule. The address the firmware
|
||||
returns is now carried through the devicetree's `dma-ranges` from the firmware node up: on the
|
||||
Pi 5 (the real `bcm2712-rpi-5-b.dtb`) the firmware node's is empty and `/soc` has none, so it is
|
||||
already physical; on a Pi 4 `/soc` maps bus `0xC0000000` onto 0, and that is handled too. An
|
||||
address no entry holds is refused (`-6`). Still unverified: the allocate tag's size field.
|
||||
|
||||
**The rehearsal.** There is no QEMU model of the Pi 5, but QEMU's virt machine with
|
||||
`virtualization=on` enters a raw image exactly as the Pi 5's firmware does: at EL2, MMU off,
|
||||
devicetree in x0. New `make -f kernel/Makefile ARCH=aarch64 RPI5_NATIVE_BASE=0x40080000
|
||||
rpi5-rehearsal` links the same native image at virt's address (the linker script's base is now
|
||||
`RPI5_NATIVE_BASE`, 0x80000 by default) and boots it, Cortex-A76, one QEMU, foreground. It found
|
||||
three more, each of which would have stopped the Pi 5:
|
||||
- **The PL011's register shift**, mine (`c9519173`): `fdt_uart.c` gave a devicetree PL011 a shift
|
||||
of 0, but `console.c` numbers its registers by word -- the flag register was read at +6, not
|
||||
+0x18: an alignment fault on Device memory at the first character. The UEFI path never met it
|
||||
(it uses a built-in default with the right shift), and the host test asserted the wrong value.
|
||||
Shift 2 now; the test expects it.
|
||||
- **The GIC's address**: `apic.c` read the GIC's `reg` as two 64-bit cells with no translation --
|
||||
right only where the GIC is a child of the root. On the Pi 5 it is on `/soc` (one cell,
|
||||
`0x7fff9000`), `0x107fff9000` to the CPU. New `fdt_reg_address()` (`fdt.c`) reads a node's `reg`
|
||||
entry in its parent's cells and carries it up through every bus's `ranges`; the GIC lookup uses
|
||||
it, and accepts virt's `arm,cortex-a15-gic` too. Checked against the real Pi 5 devicetree:
|
||||
`tools/fdt_uart_test.c <dtb> pl011 0x107d001000 2 4 arm,gic-400 0x107fff9000 0x107fffa000`.
|
||||
- **No timer interrupt at EL2**: with `HCR_EL2.IMO/FMO/AMO` clear an IRQ targets EL1 and is never
|
||||
taken while the CPU runs at EL2 -- "Heartbeat: 0 ticks". The entry stub sets them at EL2. The
|
||||
UEFI path runs at EL1 and never met it.
|
||||
|
||||
**Verified.** Rehearsal, final: POST 1052 passed, `PARITY:M7.1a word_count=526 ...
|
||||
hash=0x82f4ae7cb2e2d424` -- the same as the UEFI build -- the Tripod born, the heartbeat ticking,
|
||||
no exception (`logs/20261014-124530/aarch64-rpi5/`). Earlier rehearsal and the hand-run QEMU
|
||||
sessions that found the three are not logged (`-d int` to a scratch file); `logs/20261014-123129`
|
||||
is the first, silent, run. Not rehearsed: the mailbox and framebuffer (virt has neither), the Pi
|
||||
5's own UART and GIC addresses (the host test checks them against its devicetree), PCI (virt's
|
||||
needs ACPI; the Pi 5's is R4). The board image (`make boot_image TARGET=RASPI`) links at 0x80000
|
||||
and ends at 0x834000. Standard acceptance, all three ISAs: POST 1052 passed, same hash
|
||||
(`logs/20261014-125305`, `-125427`, `-125635`).
|
||||
|
||||
|
||||
@@ -1202,15 +1202,65 @@ RPI5_NATIVE_LINKER_SCRIPT := kernel/linker/starkernel-native-rpi5.ld
|
||||
RPI5_NATIVE_ELF := $(BUILD_DIR)/starkernel_rpi5.elf
|
||||
RPI5_NATIVE_IMG := $(BUILD_DIR)/kernel_2712.img
|
||||
|
||||
# RPI5_NATIVE_BASE: the image's load address. 0x80000 is the Pi 5's
|
||||
# (boards/raspi/config.txt kernel_address). Change it only to rehearse the
|
||||
# native path on QEMU's virt machine at EL2:
|
||||
# make -f kernel/Makefile ARCH=aarch64 RPI5_NATIVE_BASE=0x40080000 rpi5-rehearsal
|
||||
RPI5_NATIVE_BASE ?= 0x80000
|
||||
|
||||
$(RPI5_NATIVE_ELF): $(KERNEL_OBJS) $(RPI5_NATIVE_LINKER_SCRIPT) | $(BUILD_DIR)
|
||||
@echo "LD (rpi5 native) $@"
|
||||
@$(LD) -T $(RPI5_NATIVE_LINKER_SCRIPT) -nostdlib --build-id=none \
|
||||
--defsym=RPI5_NATIVE_BASE=$(RPI5_NATIVE_BASE) \
|
||||
-e rpi5_native_start $(KERNEL_OBJS) -o $@
|
||||
|
||||
$(RPI5_NATIVE_IMG): $(RPI5_NATIVE_ELF)
|
||||
@echo "OBJCOPY (raw) $@"
|
||||
@$(OBJCOPY) -O binary $< $@
|
||||
|
||||
# rpi5-rehearsal: the Pi 5's native path -- no UEFI, raw image, entered at
|
||||
# EL2 with the MMU off and x0 = the devicetree -- rehearsed on QEMU's virt
|
||||
# machine, which has no Pi 5 model but enters a raw image exactly that way
|
||||
# with virtualization=on. Linked at virt's 0x40080000 instead of 0x80000;
|
||||
# everything else is the Pi 5 image. One QEMU, foreground, serial to
|
||||
# logs/<TS>/aarch64-rpi5/ (2026-10-14, FABRIC-3.md SXXXIX).
|
||||
.PHONY: rpi5-rehearsal
|
||||
rpi5-rehearsal:
|
||||
@if [ "$(ARCH)" != "aarch64" ] || [ "$(RPI5_NATIVE_BASE)" != "0x40080000" ]; then \
|
||||
echo "rpi5-rehearsal: ARCH=aarch64 RPI5_NATIVE_BASE=0x40080000"; exit 1; fi
|
||||
@rm -f $(RPI5_NATIVE_ELF) $(RPI5_NATIVE_IMG)
|
||||
@$(MAKE) -f kernel/Makefile ARCH=aarch64 RPI5_NATIVE_BASE=$(RPI5_NATIVE_BASE) $(RPI5_NATIVE_IMG)
|
||||
@mkdir -p $(QEMU_LOG_DIR)
|
||||
@TS=$$(date +%Y%m%d-%H%M%S); \
|
||||
mkdir -p $(QEMU_LOG_DIR)/$$TS/aarch64-rpi5; \
|
||||
LOG=$(QEMU_LOG_DIR)/$$TS/aarch64-rpi5/qemu-aarch64-rpi5-$$TS.log; \
|
||||
SERIAL_SOCK=$(BUILD_DIR)/qemu-serial-aarch64-$$TS.sock; \
|
||||
QMP_SOCK=$(BUILD_DIR)/qemu-qmp-aarch64-$$TS.sock; \
|
||||
echo "=== Rehearsing the Pi 5 native path on QEMU virt at EL2 ==="; \
|
||||
echo " Serial log: $$LOG"; \
|
||||
touch $$LOG; \
|
||||
qemu-system-aarch64 \
|
||||
-machine virt,virtualization=on \
|
||||
-cpu cortex-a76 \
|
||||
-kernel $(RPI5_NATIVE_IMG) \
|
||||
-drive id=artdisk,file=$(ARTDISK),format=raw,if=none \
|
||||
-device virtio-blk-pci,drive=artdisk \
|
||||
-device virtio-keyboard-pci,addr=0x3 \
|
||||
-device qemu-xhci,id=xhci0,addr=0x4,p2=$(XHCI_PORTS),p3=$(XHCI_PORTS) \
|
||||
$(ZUSEDISK_QEMU_ARGS) \
|
||||
-object rng-random,id=rng0,filename=/dev/urandom \
|
||||
-device virtio-rng-pci,rng=rng0,addr=0x5 \
|
||||
-chardev socket,id=cserial,path=$$SERIAL_SOCK,server=on,wait=off,logfile=$$LOG \
|
||||
-serial chardev:cserial \
|
||||
-qmp unix:$$QMP_SOCK,server=on,wait=off \
|
||||
$(QEMU_EXTRA) \
|
||||
-display none \
|
||||
-m 4096 \
|
||||
-no-reboot \
|
||||
-d guest_errors; \
|
||||
rm -f $$SERIAL_SOCK $$QMP_SOCK; \
|
||||
echo "=== Serial log: $$LOG ==="
|
||||
|
||||
.PHONY: boot_image
|
||||
ifeq ($(strip $(BOARD)),)
|
||||
boot_image:
|
||||
|
||||
@@ -200,4 +200,18 @@ const void* fdt_root_node(const void* fdt);
|
||||
const void* fdt_find_child(const void* fdt, const void* parent,
|
||||
const char* name, uint32_t name_len);
|
||||
|
||||
/**
|
||||
* @brief The CPU's address for entry @p index of the `reg` of the first
|
||||
* enabled node compatible with @p compatible (2026-10-14).
|
||||
*
|
||||
* The entry is read in its parent's #address-cells/#size-cells and carried
|
||||
* up through every bus's `ranges` to the root -- an empty `ranges` keeps
|
||||
* the address, a bus with no `ranges` has no way up. The Pi 5's GIC, for
|
||||
* one, sits at 0x7fff9000 on /soc, which is 0x107fff9000 to the CPU.
|
||||
*
|
||||
* @return 1 and the address in @p out, or 0 if there is no such node,
|
||||
* entry or way up.
|
||||
*/
|
||||
int fdt_reg_address(const void* fdt, const char* compatible, uint32_t index, uint64_t* out);
|
||||
|
||||
#endif /* STARKERNEL_FDT_H */
|
||||
|
||||
@@ -58,9 +58,11 @@ typedef struct {
|
||||
* driver has been designed against; others are not
|
||||
* refused outright but are unverified).
|
||||
* @param out Populated on success; untouched on failure.
|
||||
* @return 0 on success, negative on failure (mailbox node not found
|
||||
* in the DTB, VC firmware rejected the request, or a response
|
||||
* tag came back with an unexpected size).
|
||||
* @return 0 on success, negative on failure: -2 no mailbox node in the
|
||||
* DTB, -3 the VC firmware rejected the request, -4 a response
|
||||
* came back empty, -5 no answer within one second (every wait is
|
||||
* bounded), -6 the devicetree's dma-ranges cannot place the
|
||||
* framebuffer's bus address.
|
||||
*/
|
||||
int rpi5_mailbox_get_framebuffer(const void* dtb, uint32_t width, uint32_t height,
|
||||
uint32_t bpp, Rpi5FramebufferInfo* out);
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
/*
|
||||
StarForth — Steady-State Virtual Machine Runtime
|
||||
Copyright (c) 2023–2026 Robert A. James. All rights reserved.
|
||||
Licensed under the StarForth License, Version 1.0.
|
||||
*/
|
||||
|
||||
/**
|
||||
* rpi5_mmu.h - the Pi 5 native path's identity map, once C runs
|
||||
* (rpi5_mmu.c; native_rpi5_entry.S turns the MMU on).
|
||||
*/
|
||||
|
||||
#ifndef STARKERNEL_RPI5_MMU_H
|
||||
#define STARKERNEL_RPI5_MMU_H
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
/* Makes every further GiB the devicetree's /memory covers entirely Normal
|
||||
* write-back. Returns how many GiB entries changed, or -1 if /memory
|
||||
* cannot be read. */
|
||||
int rpi5_mmu_map_ram(const void *dtb);
|
||||
|
||||
/* Makes [addr, addr+size) Normal non-cacheable (2 MiB blocks in the first
|
||||
* GiB, a whole GiB above it). Returns 0, or -1 if it cannot. */
|
||||
int rpi5_mmu_uncached(uint64_t addr, uint64_t size);
|
||||
|
||||
#endif /* STARKERNEL_RPI5_MMU_H */
|
||||
@@ -24,7 +24,10 @@ ENTRY(rpi5_native_start)
|
||||
|
||||
SECTIONS
|
||||
{
|
||||
. = 0x80000;
|
||||
/* RPI5_NATIVE_BASE comes from kernel/Makefile (--defsym): 0x80000 on
|
||||
* the Pi 5; 0x40080000 for the QEMU virt rehearsal only, whose RAM
|
||||
* starts at 0x40000000 and which loads a raw image 0x80000 into it. */
|
||||
. = RPI5_NATIVE_BASE;
|
||||
|
||||
__kernel_start = .;
|
||||
|
||||
|
||||
@@ -126,42 +126,7 @@ static uint32_t s_timer_ppi = TIMER_PPI_EL1;
|
||||
static uint64_t s_timer_period_tsc = 0;
|
||||
static uint64_t s_counter_hz_apic = 0; /* CNTFRQ_EL0 (item 0.8/§26) */
|
||||
|
||||
/**
|
||||
* @brief Read one big-endian 32-bit cell from a devicetree property blob.
|
||||
*
|
||||
* `fdt.c` keeps its own `be32()` helper file-local (freestanding, no shared
|
||||
* byte-swap utility to reuse) -- duplicated here rather than exposing it,
|
||||
* same "a few lines is simpler than a new shared dependency" precedent
|
||||
* `rpi5_dtb.c`'s own `reg_first_cell_be32()` already set for this exact
|
||||
* situation.
|
||||
*/
|
||||
static uint32_t be32_cell(const unsigned char *b)
|
||||
{
|
||||
return ((uint32_t) b[0] << 24) | ((uint32_t) b[1] << 16) |
|
||||
((uint32_t) b[2] << 8) | (uint32_t) b[3];
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Read one two-cell (64-bit) big-endian address from a `reg` entry.
|
||||
*
|
||||
* BCM2712's `gicv2` node (see file header) uses two address cells and two
|
||||
* size cells per `reg` entry -- confirmed directly against `bcm2712.dtsi`'s
|
||||
* `axi`/`gicv2` nodes (`#address-cells = <2>; #size-cells = <2>;`), unlike
|
||||
* `rpi5_dtb.c`'s UART/mailbox lookups which read a single 32-bit cell
|
||||
* because their `soc` parent node declares `#address-cells = <1>`. Reads
|
||||
* only the two address cells; the caller advances past the two size cells
|
||||
* itself via @c GIC_REG_ENTRY_BYTES to reach the next entry.
|
||||
*/
|
||||
static uint64_t be64_addr_cell_pair(const unsigned char *b)
|
||||
{
|
||||
return ((uint64_t) be32_cell(b) << 32) | (uint64_t) be32_cell(b + 4);
|
||||
}
|
||||
|
||||
/* One gicv2 `reg` entry is 4 cells (2 address + 2 size) * 4 bytes = 16
|
||||
* bytes. Entry 0 = GICD (distributor), entry 1 = GICC (CPU interface 0) --
|
||||
* confirmed against bcm2712.dtsi's own `reg` list order (GICD, GICC, GICH,
|
||||
* GICV, the standard arm,gic-400 binding order), not assumed. */
|
||||
#define GIC_REG_ENTRY_BYTES 16u
|
||||
|
||||
/**
|
||||
* @brief Try to discover the real GICD/GICC base addresses from the DTB.
|
||||
@@ -181,21 +146,28 @@ static uint64_t be64_addr_cell_pair(const unsigned char *b)
|
||||
*/
|
||||
static int gic_bases_from_dtb(const void *dtb, uintptr_t *gicd_out, uintptr_t *gicc_out)
|
||||
{
|
||||
const void *node;
|
||||
const unsigned char *reg;
|
||||
uint32_t len;
|
||||
/* Entry 0 of the GIC's reg is the distributor, entry 1 the CPU
|
||||
* interface, each carried through its buses' ranges to the CPU's
|
||||
* address (fdt_reg_address(), 2026-10-14). Until then reg was read as
|
||||
* two 64-bit cells with no translation: right only where the GIC is a
|
||||
* direct child of the root, as on QEMU's virt machine. On the Pi 5 it
|
||||
* is on /soc, one cell, 0x7fff9000 -- which is 0x107fff9000 to the
|
||||
* CPU. "arm,cortex-a15-gic" is QEMU virt's GICv2, so the rehearsal of
|
||||
* the Pi 5's native path finds its GIC the same way (FABRIC-3.md
|
||||
* SXXXIX). */
|
||||
static const char *const compatible[] = { "arm,gic-400", "arm,cortex-a15-gic" };
|
||||
uint64_t d, c;
|
||||
unsigned k;
|
||||
|
||||
if (!fdt_valid(dtb)) return 0;
|
||||
|
||||
node = fdt_find_node_by_compatible(dtb, "arm,gic-400");
|
||||
if (!node) return 0;
|
||||
|
||||
reg = (const unsigned char *) fdt_find_prop_in_node(dtb, node, "reg", &len);
|
||||
if (!reg || len < 2u * GIC_REG_ENTRY_BYTES) return 0;
|
||||
|
||||
*gicd_out = (uintptr_t) be64_addr_cell_pair(reg);
|
||||
*gicc_out = (uintptr_t) be64_addr_cell_pair(reg + GIC_REG_ENTRY_BYTES);
|
||||
return 1;
|
||||
for (k = 0; k < sizeof compatible / sizeof compatible[0]; k++) {
|
||||
if (fdt_reg_address(dtb, compatible[k], 0, &d) && fdt_reg_address(dtb, compatible[k], 1, &c)) {
|
||||
*gicd_out = (uintptr_t) d;
|
||||
*gicc_out = (uintptr_t) c;
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
* native_rpi5_entry.S (aarch64) - Raspberry Pi 5 native (non-UEFI) entry
|
||||
* stub (FABRIC-3.md §IV.3 item 1, 2026-09-04).
|
||||
*
|
||||
* **Never run on real hardware** -- the Pi 5 isn't in hand until
|
||||
* 2026-09-17 (FABRIC-3.md §II). Compile-only-verified.
|
||||
* Never run on the board yet. Rehearsed on QEMU's virt machine at EL2
|
||||
* since 2026-10-14 (`make ... rpi5-rehearsal`, FABRIC-3.md §XXXIX): POST
|
||||
* passes there, same parity hash as the UEFI build.
|
||||
*
|
||||
* Entered directly by Pi 5 firmware at the classic bare-metal load address
|
||||
* 0x80000 (item 6, `config.txt`'s `os_check=0`, is still open -- that's
|
||||
@@ -47,6 +48,18 @@
|
||||
g_rpi5_dtb_ptr:
|
||||
.space 8 /* uint64_t -- masked DTB pointer */
|
||||
|
||||
/* The early identity map (2026-10-14): one level-1 table of 512 1 GiB
|
||||
* entries (39-bit VA, 4 KiB granule), and one level-2 table that splits
|
||||
* the GiB the image is in into 2 MiB blocks, so that a framebuffer
|
||||
* inside it can be made uncached alone (rpi5_mmu.c). */
|
||||
.align 12
|
||||
.global g_rpi5_l1
|
||||
g_rpi5_l1:
|
||||
.space 4096
|
||||
.global g_rpi5_l2
|
||||
g_rpi5_l2:
|
||||
.space 4096
|
||||
|
||||
.align 4
|
||||
g_rpi5_native_stack:
|
||||
.space 0x200000 /* 2 MiB, same convention as the
|
||||
@@ -63,6 +76,30 @@ rpi5_native_start:
|
||||
mov x1, #0xffffffff
|
||||
and x0, x0, x1
|
||||
|
||||
/* Clean and invalidate the data cache over the whole image, .bss and
|
||||
* stack included, FIRST: whatever the firmware may have left in the
|
||||
* cache for these addresses goes to RAM now, before anything below
|
||||
* writes them with the cache off -- a dirty line cleaned later would
|
||||
* land on top of that work. Line size from CTR_EL0.DminLine. */
|
||||
mrs x6, ctr_el0
|
||||
ubfx x6, x6, #16, #4
|
||||
mov x7, #4
|
||||
lsl x7, x7, x6 /* x7 = line size in bytes */
|
||||
sub x8, x7, #1
|
||||
adrp x4, __kernel_start
|
||||
add x4, x4, :lo12:__kernel_start
|
||||
bic x4, x4, x8
|
||||
adrp x5, __kernel_end
|
||||
add x5, x5, :lo12:__kernel_end
|
||||
.Lcache_clean:
|
||||
cmp x4, x5
|
||||
b.hs .Lcache_done
|
||||
dc civac, x4
|
||||
add x4, x4, x7
|
||||
b .Lcache_clean
|
||||
.Lcache_done:
|
||||
dsb sy
|
||||
|
||||
/* Zero .bss: [__bss_start, __bss_end), both 16-byte aligned by the
|
||||
* native linker script. */
|
||||
adrp x4, __bss_start
|
||||
@@ -85,6 +122,114 @@ rpi5_native_start:
|
||||
add x0, x0, :lo12:g_rpi5_native_stack_top
|
||||
mov sp, x0
|
||||
|
||||
/* MMU and caches on, before any C (2026-10-14). The firmware enters
|
||||
* with the MMU off, where every access is Device memory: an unaligned
|
||||
* access or an exclusive faults, and nothing is cached. Identity map:
|
||||
* the first GiB Normal write-back (every Pi 5 has at least that much
|
||||
* RAM; rpi5_mmu.c makes the rest of the devicetree's RAM Normal once C
|
||||
* runs), everything else Device-nGnRE, never executable. At
|
||||
* whichever EL the firmware chose -- EL2 on the Pi 5, EL1 handled too.
|
||||
* "The first GiB" is precisely the GiB the image is in: 0 on the Pi 5
|
||||
* (0x80000), 1 on QEMU's virt machine, where it is rehearsed.
|
||||
* MAIR: attr0 Normal WB RA/WA (0xFF), attr1 Device-nGnRE (0x04),
|
||||
* attr2 Normal non-cacheable (0x44).
|
||||
* block: AF (bit 10), SH inner (bits 9:8 = 3) for Normal, AttrIndx
|
||||
* (bits 4:2), bit 54 XN/UXN for Device. */
|
||||
adrp x1, g_rpi5_l1
|
||||
add x1, x1, :lo12:g_rpi5_l1
|
||||
adrp x2, g_rpi5_l2
|
||||
add x2, x2, :lo12:g_rpi5_l2
|
||||
adrp x10, __kernel_start
|
||||
add x10, x10, :lo12:__kernel_start
|
||||
lsr x10, x10, #30 /* x10 = the GiB the image is in */
|
||||
|
||||
lsl x11, x10, #30 /* level 2: that GiB, 512 x 2 MiB Normal */
|
||||
mov x3, #0
|
||||
mov x4, #0x701
|
||||
.Ll2_fill:
|
||||
lsl x5, x3, #21
|
||||
add x5, x5, x11
|
||||
orr x5, x5, x4
|
||||
str x5, [x2, x3, lsl #3]
|
||||
add x3, x3, #1
|
||||
cmp x3, #512
|
||||
b.lo .Ll2_fill
|
||||
|
||||
mov x3, #0 /* level 1: 1 GiB Device, but the image's GiB */
|
||||
mov x4, #0x405
|
||||
orr x4, x4, #(1 << 54)
|
||||
.Ll1_fill:
|
||||
lsl x5, x3, #30
|
||||
orr x5, x5, x4
|
||||
cmp x3, x10
|
||||
b.ne .Ll1_store
|
||||
orr x5, x2, #3 /* -> the level 2 table */
|
||||
.Ll1_store:
|
||||
str x5, [x1, x3, lsl #3]
|
||||
add x3, x3, #1
|
||||
cmp x3, #512
|
||||
b.lo .Ll1_fill
|
||||
dsb sy
|
||||
|
||||
movz x6, #0x04FF
|
||||
movk x6, #0x0044, lsl #16 /* MAIR = 0x4404FF */
|
||||
mov x7, #(1 << 0) | (1 << 2) | (1 << 12) /* SCTLR: M, C, I */
|
||||
|
||||
mrs x8, CurrentEL
|
||||
ubfx x8, x8, #2, #2
|
||||
cmp x8, #2
|
||||
b.ne .Lmmu_el1
|
||||
|
||||
msr mair_el2, x6
|
||||
movz x9, #0x3519 /* T0SZ 25, IRGN0/ORGN0 WB, SH0 inner, TG0 4K */
|
||||
movk x9, #0x8082, lsl #16 /* PS 40-bit, RES1 bits 23 and 31 */
|
||||
msr tcr_el2, x9
|
||||
msr ttbr0_el2, x1
|
||||
isb
|
||||
tlbi alle2
|
||||
dsb ish
|
||||
isb
|
||||
ic iallu
|
||||
dsb ish
|
||||
mrs x9, sctlr_el2
|
||||
orr x9, x9, x7
|
||||
bic x9, x9, #(1 << 1) /* A: no alignment checking */
|
||||
bic x9, x9, #(1 << 19) /* WXN: the image is writable Normal memory */
|
||||
msr sctlr_el2, x9
|
||||
isb
|
||||
/* Interrupts to EL2, where the kernel runs: with HCR_EL2.IMO/FMO/AMO
|
||||
* clear (as a stub that hands Linux EL1 leaves them, and QEMU resets
|
||||
* them) an IRQ targets EL1 and is never taken at EL2 -- the timer
|
||||
* never ticked (FABRIC-3.md SXXXIX). */
|
||||
mrs x9, hcr_el2
|
||||
orr x9, x9, #(1 << 3) /* FMO */
|
||||
orr x9, x9, #(1 << 4) /* IMO */
|
||||
orr x9, x9, #(1 << 5) /* AMO */
|
||||
msr hcr_el2, x9
|
||||
isb
|
||||
b .Lmmu_done
|
||||
|
||||
.Lmmu_el1:
|
||||
msr mair_el1, x6
|
||||
movz x9, #0x3519 /* T0SZ 25, IRGN0/ORGN0 WB, SH0 inner, TG0 4K */
|
||||
movk x9, #0x0080, lsl #16 /* EPD1: no TTBR1 walks */
|
||||
movk x9, #0x0002, lsl #32 /* IPS 40-bit */
|
||||
msr tcr_el1, x9
|
||||
msr ttbr0_el1, x1
|
||||
isb
|
||||
tlbi vmalle1
|
||||
dsb ish
|
||||
isb
|
||||
ic iallu
|
||||
dsb ish
|
||||
mrs x9, sctlr_el1
|
||||
orr x9, x9, x7
|
||||
bic x9, x9, #(1 << 1)
|
||||
bic x9, x9, #(1 << 19)
|
||||
msr sctlr_el1, x9
|
||||
isb
|
||||
.Lmmu_done:
|
||||
|
||||
/* x0 was clobbered above for the stack-top address -- reload the
|
||||
* masked DTB pointer from g_rpi5_dtb_ptr before calling in. */
|
||||
adrp x0, g_rpi5_dtb_ptr
|
||||
|
||||
@@ -57,6 +57,7 @@
|
||||
|
||||
#include "starkernel/rpi5_mailbox.h"
|
||||
#include "starkernel/rpi5_dtb.h"
|
||||
#include "starkernel/fdt.h"
|
||||
|
||||
#define MBOX_OFF_READ 0x00u
|
||||
#define MBOX_OFF_STATUS0 0x18u
|
||||
@@ -108,6 +109,128 @@ static inline uint32_t mmio_read32(uint64_t addr)
|
||||
return *(volatile uint32_t*) addr;
|
||||
}
|
||||
|
||||
/* Every wait on the mailbox is bounded (2026-10-14): this runs before any
|
||||
* console exists, so a VideoCore that never answers must cost a missing
|
||||
* framebuffer, not a silent hang. One second, by the architectural
|
||||
* counter -- there at any EL with no setup. */
|
||||
static uint64_t counter_now(void)
|
||||
{
|
||||
uint64_t v;
|
||||
__asm__ volatile("isb\n\tmrs %0, cntpct_el0" : "=r"(v));
|
||||
return v;
|
||||
}
|
||||
|
||||
static uint64_t counter_freq(void)
|
||||
{
|
||||
uint64_t v;
|
||||
__asm__ volatile("mrs %0, cntfrq_el0" : "=r"(v));
|
||||
return v ? v : 54000000u; /* the Pi 5's 54 MHz, if the firmware left it unset */
|
||||
}
|
||||
|
||||
/* The VideoCore reads and writes RAM directly, not through the ARM's
|
||||
* caches (the MMU and caches are on by now -- native_rpi5_entry.S): the
|
||||
* request goes out to RAM before it is sent, and the cached copy is
|
||||
* dropped before the answer is read. */
|
||||
static void buffer_to_ram(void)
|
||||
{
|
||||
uint64_t a;
|
||||
for (a = (uint64_t)(uintptr_t)g_mbox_buf; a < (uint64_t)(uintptr_t)(g_mbox_buf + 40); a += 16)
|
||||
__asm__ volatile("dc civac, %0" :: "r"(a) : "memory");
|
||||
__asm__ volatile("dsb sy" ::: "memory");
|
||||
}
|
||||
|
||||
/* fdt helpers for the bus-address translation below. */
|
||||
static uint32_t mb_be32(const unsigned char *p)
|
||||
{
|
||||
return ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) | ((uint32_t)p[2] << 8) | p[3];
|
||||
}
|
||||
|
||||
static uint64_t mb_cells(const unsigned char *p, uint32_t n)
|
||||
{
|
||||
return n == 2 ? ((uint64_t)mb_be32(p) << 32) | mb_be32(p + 4) : mb_be32(p);
|
||||
}
|
||||
|
||||
static uint32_t mb_cell_prop(const void *dtb, const void *node, const char *name, uint32_t otherwise)
|
||||
{
|
||||
uint32_t len;
|
||||
const void *v = fdt_find_prop_in_node(dtb, node, name, &len);
|
||||
return (v && len == 4) ? mb_be32((const unsigned char *)v) : otherwise;
|
||||
}
|
||||
|
||||
static int mb_compatible(const void *dtb, const void *node, const char *want)
|
||||
{
|
||||
uint32_t len, i = 0, wl = 0;
|
||||
const char *c = (const char *)fdt_find_prop_in_node(dtb, node, "compatible", &len);
|
||||
while (want[wl]) wl++;
|
||||
while (c && i < len) {
|
||||
uint32_t l = 0, k;
|
||||
while (i + l < len && c[i + l]) l++;
|
||||
if (l == wl) {
|
||||
for (k = 0; k < l && c[i + k] == want[k]; k++) { }
|
||||
if (k == l) return 1;
|
||||
}
|
||||
i += l + 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* The path from the root to the first node compatible with `want`:
|
||||
* path[0] is the root, path[depth-1] the node. Depth-first. */
|
||||
static int mb_find_path(const void *dtb, const void *node, const char *want,
|
||||
const void **path, int depth, int max)
|
||||
{
|
||||
const void *child = (const void *)0;
|
||||
if (depth >= max) return 0;
|
||||
path[depth] = node;
|
||||
if (mb_compatible(dtb, node, want)) return depth + 1;
|
||||
while ((child = fdt_next_child_node(dtb, node, child)) != (const void *)0) {
|
||||
int d = mb_find_path(dtb, child, want, path, depth + 1, max);
|
||||
if (d) return d;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* The firmware's framebuffer address is an address on the firmware's own
|
||||
* bus. It becomes an ARM physical address through the devicetree's
|
||||
* dma-ranges, from the firmware node up to the root (2026-10-14): a node
|
||||
* with none, or an empty one, maps one to one; otherwise the entry that
|
||||
* holds the address moves it into the parent's space. On the Pi 5 the
|
||||
* firmware node's is empty and /soc has none -- the address is already
|
||||
* physical. (The 0x3FFFFFFF mask this replaced was the Pi 4's habit:
|
||||
* there /soc's dma-ranges maps bus 0xC0000000 onto physical 0, which
|
||||
* this handles as well.) Returns 0, or -1 if no entry holds it. */
|
||||
static int bus_to_phys(const void *dtb, uint64_t bus, uint64_t *phys)
|
||||
{
|
||||
const void *path[12];
|
||||
int depth, i;
|
||||
const void *root = fdt_root_node(dtb);
|
||||
|
||||
if (!root) return -1;
|
||||
depth = mb_find_path(dtb, root, "raspberrypi,bcm2835-firmware", path, 0, 12);
|
||||
if (depth < 2) return -1;
|
||||
|
||||
for (i = depth - 1; i >= 1; i--) {
|
||||
uint32_t len, ca, pa, sz, entry, k;
|
||||
const unsigned char *r =
|
||||
(const unsigned char *)fdt_find_prop_in_node(dtb, path[i], "dma-ranges", &len);
|
||||
if (!r || len == 0) continue; /* one to one */
|
||||
ca = mb_cell_prop(dtb, path[i], "#address-cells", 2);
|
||||
sz = mb_cell_prop(dtb, path[i], "#size-cells", 1);
|
||||
pa = mb_cell_prop(dtb, path[i - 1], "#address-cells", 2);
|
||||
if (ca < 1 || ca > 2 || pa < 1 || pa > 2 || sz < 1 || sz > 2) return -1;
|
||||
entry = (ca + pa + sz) * 4u;
|
||||
for (k = 0; k + entry <= len; k += entry) {
|
||||
uint64_t cb = mb_cells(r + k, ca);
|
||||
uint64_t pb = mb_cells(r + k + ca * 4u, pa);
|
||||
uint64_t n = mb_cells(r + k + (ca + pa) * 4u, sz);
|
||||
if (bus >= cb && bus - cb < n) { bus = pb + (bus - cb); break; }
|
||||
}
|
||||
if (k + entry > len) return -1; /* no entry holds it */
|
||||
}
|
||||
*phys = bus;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Appends one tag (id, value-buffer size in bytes, request size in
|
||||
* bytes, then `nvalues` u32 value words) to g_mbox_buf starting at
|
||||
* word index *idx, advancing *idx past it (including alignment
|
||||
@@ -176,21 +299,31 @@ int rpi5_mailbox_get_framebuffer(const void* dtb, uint32_t width, uint32_t heigh
|
||||
|
||||
send_val = ((uint32_t) (uintptr_t) g_mbox_buf & ~0xFu) | MBOX_CH_PROPERTY;
|
||||
|
||||
while ((mmio_read32(base + MBOX_OFF_STATUS1) & MBOX_FULL) != 0)
|
||||
{ /* spin until MBOX1 (send) has room */
|
||||
}
|
||||
mmio_write32(base + MBOX_OFF_WRITE, send_val);
|
||||
|
||||
for (;;)
|
||||
buffer_to_ram();
|
||||
{
|
||||
while ((mmio_read32(base + MBOX_OFF_STATUS0) & MBOX_EMPTY) != 0)
|
||||
{ /* spin until MBOX0 (receive) has a message */
|
||||
uint64_t limit = counter_freq(); /* one second, for the whole exchange */
|
||||
uint64_t t0 = counter_now();
|
||||
|
||||
while ((mmio_read32(base + MBOX_OFF_STATUS1) & MBOX_FULL) != 0)
|
||||
{ /* MBOX1 (send) has no room yet */
|
||||
if (counter_now() - t0 > limit) return -5;
|
||||
}
|
||||
mmio_write32(base + MBOX_OFF_WRITE, send_val);
|
||||
|
||||
for (;;)
|
||||
{
|
||||
while ((mmio_read32(base + MBOX_OFF_STATUS0) & MBOX_EMPTY) != 0)
|
||||
{ /* MBOX0 (receive) has nothing yet */
|
||||
if (counter_now() - t0 > limit) return -5;
|
||||
}
|
||||
if (mmio_read32(base + MBOX_OFF_READ) == send_val) break;
|
||||
/* A message arrived, but not the one we sent (another channel's
|
||||
* traffic) -- discard and keep waiting, per the wiki's own
|
||||
* documented channel-matching convention. */
|
||||
if (counter_now() - t0 > limit) return -5;
|
||||
}
|
||||
if (mmio_read32(base + MBOX_OFF_READ) == send_val) break;
|
||||
/* A message arrived, but not the one we sent (another channel's
|
||||
* traffic) -- discard and keep waiting, per the wiki's own
|
||||
* documented channel-matching convention. */
|
||||
}
|
||||
buffer_to_ram(); /* drops the cached copy: what follows reads the answer */
|
||||
|
||||
if (g_mbox_buf[1] != MBOX_CODE_RESPONSE_OK) return -3; /* VC rejected the request */
|
||||
|
||||
@@ -215,16 +348,12 @@ int rpi5_mailbox_get_framebuffer(const void* dtb, uint32_t width, uint32_t heigh
|
||||
|
||||
if (fb_addr == 0 || fb_size == 0 || pitch_bytes == 0) return -4;
|
||||
|
||||
/* Allocate-buffer's own response address is a VC bus address,
|
||||
* not necessarily the plain ARM-physical one -- masking the
|
||||
* top bits to recover the ARM-physical alias is the documented
|
||||
* convention on earlier Pi chips; kept here defensively even
|
||||
* though the Pi-5-specific reference this file cites for the
|
||||
* *send* side found no bus-alias bit in play there. Genuinely
|
||||
* unverified which of these is correct for Pi 5's own
|
||||
* allocate-buffer response specifically -- flagged, not
|
||||
* guessed past this comment. */
|
||||
out->base = (void*) (uintptr_t) (fb_addr & 0x3FFFFFFFu);
|
||||
/* Allocate-buffer answers with an address on the firmware's bus:
|
||||
* translated by the devicetree (bus_to_phys(), above), not masked
|
||||
* by a rule from another chip. */
|
||||
uint64_t fb_phys;
|
||||
if (bus_to_phys(dtb, fb_addr, &fb_phys) != 0) return -6;
|
||||
out->base = (void*) (uintptr_t) fb_phys;
|
||||
out->size = fb_size;
|
||||
out->width = width;
|
||||
out->height = height;
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
/*
|
||||
StarForth — Steady-State Virtual Machine Runtime
|
||||
Copyright (c) 2023–2026 Robert A. James. All rights reserved.
|
||||
Licensed under the StarForth License, Version 1.0.
|
||||
*/
|
||||
|
||||
/**
|
||||
* rpi5_mmu.c - the Pi 5 native path's identity map, once C runs
|
||||
* (2026-10-14; boards/BRINGUP.md R2).
|
||||
*
|
||||
* native_rpi5_entry.S turns the MMU and caches on before any C: the GiB
|
||||
* the image is in Normal write-back in 2 MiB blocks (g_rpi5_l2) -- GiB 0
|
||||
* on the Pi 5 -- everything else Device, 1 GiB blocks (g_rpi5_l1). From
|
||||
* here:
|
||||
* - rpi5_mmu_map_ram() makes every further GiB the devicetree's /memory
|
||||
* covers entirely Normal write-back, so RAM past the first GiB is
|
||||
* cached and can hold unaligned data -- a GiB only partly RAM stays
|
||||
* Device rather than let the CPU speculate into what is not RAM;
|
||||
* - rpi5_mmu_uncached() makes a range Normal non-cacheable, for the
|
||||
* framebuffer, which the display reads straight from RAM.
|
||||
* Each change breaks before it makes (invalid entry, TLB flush, new
|
||||
* entry), as the architecture requires for a change of memory type.
|
||||
*
|
||||
* Never run on real hardware yet. Rehearsed on QEMU's virt machine at EL2
|
||||
* (FABRIC-3.md §XXXIX).
|
||||
*/
|
||||
|
||||
#include "starkernel/rpi5_mmu.h"
|
||||
#include "starkernel/fdt.h"
|
||||
|
||||
extern uint64_t g_rpi5_l1[512];
|
||||
extern uint64_t g_rpi5_l2[512];
|
||||
extern char __kernel_start[], __kernel_end[];
|
||||
|
||||
#define GIB (1ull << 30)
|
||||
#define MIB2 (1ull << 21)
|
||||
#define BLOCK_NORMAL_WB 0x701ull /* AF, SH inner, AttrIndx 0 */
|
||||
#define BLOCK_NORMAL_NC 0x709ull /* AF, SH inner, AttrIndx 2 */
|
||||
|
||||
/* The GiB that level 2 splits: the one the image is in. */
|
||||
static uint64_t l2_gib(void)
|
||||
{
|
||||
return (uint64_t)(uintptr_t)__kernel_start >> 30;
|
||||
}
|
||||
|
||||
static uint32_t current_el(void)
|
||||
{
|
||||
uint64_t el;
|
||||
__asm__ volatile("mrs %0, CurrentEL" : "=r"(el));
|
||||
return (uint32_t)((el >> 2) & 3u);
|
||||
}
|
||||
|
||||
static void tlb_flush_all(void)
|
||||
{
|
||||
__asm__ volatile("dsb ishst" ::: "memory");
|
||||
if (current_el() == 2)
|
||||
__asm__ volatile("tlbi alle2" ::: "memory");
|
||||
else
|
||||
__asm__ volatile("tlbi vmalle1" ::: "memory");
|
||||
__asm__ volatile("dsb ish\n\tisb" ::: "memory");
|
||||
}
|
||||
|
||||
/* Break before make: the old entry is made invalid and the TLB flushed
|
||||
* before the new one is written. */
|
||||
static void replace_entry(volatile uint64_t *entry, uint64_t value)
|
||||
{
|
||||
*entry = 0;
|
||||
tlb_flush_all();
|
||||
*entry = value;
|
||||
tlb_flush_all();
|
||||
}
|
||||
|
||||
static uint32_t be32_at(const unsigned char *p)
|
||||
{
|
||||
return ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) |
|
||||
((uint32_t)p[2] << 8) | (uint32_t)p[3];
|
||||
}
|
||||
|
||||
static uint64_t cells_at(const unsigned char *p, uint32_t n)
|
||||
{
|
||||
return n == 2 ? ((uint64_t)be32_at(p) << 32) | be32_at(p + 4) : be32_at(p);
|
||||
}
|
||||
|
||||
int rpi5_mmu_map_ram(const void *dtb)
|
||||
{
|
||||
const void *mem, *reg;
|
||||
uint32_t acells, scells, len, entry, i, made = 0;
|
||||
const unsigned char *p;
|
||||
|
||||
if (!fdt_prop_u32(dtb, "#address-cells", &acells) ||
|
||||
!fdt_prop_u32(dtb, "#size-cells", &scells) ||
|
||||
(acells != 1 && acells != 2) || (scells != 1 && scells != 2))
|
||||
return -1;
|
||||
mem = fdt_find_node_by_device_type(dtb, "memory");
|
||||
if (!mem) return -1;
|
||||
reg = fdt_find_prop_in_node(dtb, mem, "reg", &len);
|
||||
if (!reg) return -1;
|
||||
|
||||
entry = (acells + scells) * 4u;
|
||||
p = (const unsigned char *)reg;
|
||||
for (; len >= entry; len -= entry, p += entry) {
|
||||
uint64_t start = cells_at(p, acells);
|
||||
uint64_t end = start + cells_at(p + acells * 4u, scells);
|
||||
uint64_t first = (start + GIB - 1) / GIB; /* whole GiBs only */
|
||||
uint64_t last = end / GIB; /* exclusive */
|
||||
for (i = (uint32_t)first; i < last && i < 512u; i++) {
|
||||
uint64_t want = ((uint64_t)i << 30) | BLOCK_NORMAL_WB;
|
||||
if (i == l2_gib()) continue; /* level 2's, already Normal */
|
||||
if (g_rpi5_l1[i] != want) {
|
||||
replace_entry(&g_rpi5_l1[i], want);
|
||||
made++;
|
||||
}
|
||||
}
|
||||
}
|
||||
return (int)made;
|
||||
}
|
||||
|
||||
int rpi5_mmu_uncached(uint64_t addr, uint64_t size)
|
||||
{
|
||||
uint64_t a, end = addr + size;
|
||||
|
||||
if (size == 0) return -1;
|
||||
/* Never a block that holds the image: its entry goes invalid for a
|
||||
* moment below, and the code doing it would lose itself. */
|
||||
{
|
||||
uint64_t ks = (uint64_t)(uintptr_t)__kernel_start & ~(MIB2 - 1);
|
||||
uint64_t ke = ((uint64_t)(uintptr_t)__kernel_end + MIB2 - 1) & ~(MIB2 - 1);
|
||||
if (addr < ke && end > ks) return -1;
|
||||
}
|
||||
/* Clean and invalidate the range first, so no cached copy of it
|
||||
* outlives the change -- line by line, 64 bytes at a time at most. */
|
||||
for (a = addr & ~63ull; a < end; a += 64)
|
||||
__asm__ volatile("dc civac, %0" :: "r"(a) : "memory");
|
||||
__asm__ volatile("dsb sy" ::: "memory");
|
||||
|
||||
if ((addr >> 30) != ((end - 1) >> 30))
|
||||
return -1; /* straddles a GiB boundary: not mapped uncached */
|
||||
if ((addr >> 30) == l2_gib()) {
|
||||
for (a = addr & ~(MIB2 - 1); a < end; a += MIB2)
|
||||
replace_entry(&g_rpi5_l2[(a >> 21) & 511u], (a & ~(MIB2 - 1)) | BLOCK_NORMAL_NC);
|
||||
return 0;
|
||||
}
|
||||
{
|
||||
/* In another GiB: the whole GiB goes uncached. The image is in
|
||||
* level 2's GiB, so this never touches it. */
|
||||
replace_entry(&g_rpi5_l1[addr >> 30], ((addr >> 30) << 30) | BLOCK_NORMAL_NC);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -56,6 +56,7 @@
|
||||
#include "starkernel/rpi5_native_boot.h"
|
||||
#include "starkernel/fdt.h"
|
||||
#include "starkernel/rpi5_mailbox.h"
|
||||
#include "starkernel/rpi5_mmu.h"
|
||||
#include "starkernel/cmdline.h"
|
||||
#include "starkernel/uefi.h"
|
||||
|
||||
@@ -355,6 +356,11 @@ void rpi5_native_boot(uint64_t dtb)
|
||||
rpi5_native_halt();
|
||||
}
|
||||
|
||||
/* The entry stub turned the MMU on with only the first GiB as RAM
|
||||
* (Normal); the rest of the devicetree's RAM becomes Normal here,
|
||||
* before anything allocates from it. */
|
||||
(void)rpi5_mmu_map_ram(dtb_ptr);
|
||||
|
||||
memmap_count = build_memory_map(dtb_ptr);
|
||||
if (memmap_count == 0)
|
||||
{
|
||||
@@ -398,9 +404,18 @@ void rpi5_native_boot(uint64_t dtb)
|
||||
g_rpi5_boot_info.framebuffer.pixels_per_scanline = 0;
|
||||
g_rpi5_boot_info.framebuffer.pixel_format = 0;
|
||||
|
||||
fb.base = (void *)0; /* untouched by a mailbox call that fails */
|
||||
if (rpi5_mailbox_get_framebuffer(dtb_ptr, RPI5_FB_DEFAULT_WIDTH,
|
||||
RPI5_FB_DEFAULT_HEIGHT,
|
||||
RPI5_FB_DEFAULT_BPP, &fb) == 0)
|
||||
{
|
||||
/* The display reads the framebuffer straight from RAM: uncached,
|
||||
* or what the console draws sits in the cache (rpi5_mmu.c). A
|
||||
* framebuffer that cannot be made uncached is not used. */
|
||||
if (rpi5_mmu_uncached((uint64_t)(uintptr_t)fb.base, fb.size) != 0)
|
||||
fb.base = (void *)0;
|
||||
}
|
||||
if (fb.base)
|
||||
{
|
||||
g_rpi5_boot_info.framebuffer.base = fb.base;
|
||||
g_rpi5_boot_info.framebuffer.size = fb.size;
|
||||
|
||||
@@ -654,3 +654,97 @@ const void* fdt_find_prop_in_node(const void* fdt, const void* node,
|
||||
|
||||
return (void*)0;
|
||||
}
|
||||
|
||||
/* ---- a node's address, carried through its buses (2026-10-14) ---- */
|
||||
|
||||
#define FDT_PATH_MAX 12u
|
||||
|
||||
static uint32_t node_cells(const void* fdt, const void* node, const char* name, uint32_t otherwise)
|
||||
{
|
||||
uint32_t len;
|
||||
const void* v = fdt_find_prop_in_node(fdt, node, name, &len);
|
||||
return (v && len == 4) ? be32(v) : otherwise;
|
||||
}
|
||||
|
||||
static uint64_t read_cells_at(const unsigned char* p, uint32_t n)
|
||||
{
|
||||
return n == 2 ? ((uint64_t)be32(p) << 32) | be32(p + 4) : (uint64_t)be32(p);
|
||||
}
|
||||
|
||||
static int node_enabled(const void* fdt, const void* node)
|
||||
{
|
||||
uint32_t len;
|
||||
const char* v = (const char*)fdt_find_prop_in_node(fdt, node, "status", &len);
|
||||
return !v || compat_list_contains(v, len, "okay") || compat_list_contains(v, len, "ok");
|
||||
}
|
||||
|
||||
/* Depth first: the path from the root to the first enabled node
|
||||
* compatible with `want` -- path[0] the root, path[depth-1] the node. */
|
||||
static uint32_t path_to(const void* fdt, const void* node, const char* want,
|
||||
const void** path, uint32_t depth)
|
||||
{
|
||||
const void* child = (const void*)0;
|
||||
uint32_t len;
|
||||
const char* c;
|
||||
|
||||
if (depth >= FDT_PATH_MAX) return 0;
|
||||
path[depth] = node;
|
||||
c = (const char*)fdt_find_prop_in_node(fdt, node, "compatible", &len);
|
||||
if (c && compat_list_contains(c, len, want) && node_enabled(fdt, node)) return depth + 1;
|
||||
while ((child = fdt_next_child_node(fdt, node, child)) != (const void*)0)
|
||||
{
|
||||
uint32_t d = path_to(fdt, child, want, path, depth + 1);
|
||||
if (d) return d;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int fdt_reg_address(const void* fdt, const char* compatible, uint32_t index, uint64_t* out)
|
||||
{
|
||||
const void* path[FDT_PATH_MAX];
|
||||
const unsigned char* reg;
|
||||
const void* root;
|
||||
uint32_t depth, len, ac, sc, i;
|
||||
uint64_t addr;
|
||||
|
||||
if (!out || !compatible || !fdt_complete(fdt)) return 0;
|
||||
root = fdt_root_node(fdt);
|
||||
if (!root) return 0;
|
||||
depth = path_to(fdt, root, compatible, path, 0);
|
||||
if (depth < 2) return 0;
|
||||
|
||||
/* entry `index` of its reg, in its parent's cells */
|
||||
ac = node_cells(fdt, path[depth - 2], "#address-cells", 2);
|
||||
sc = node_cells(fdt, path[depth - 2], "#size-cells", 1);
|
||||
if (ac < 1 || ac > 2 || sc > 2) return 0;
|
||||
reg = (const unsigned char*)fdt_find_prop_in_node(fdt, path[depth - 1], "reg", &len);
|
||||
if (!reg || len < 4u * (ac + sc) * (index + 1u)) return 0;
|
||||
addr = read_cells_at(reg + 4u * (ac + sc) * index, ac);
|
||||
|
||||
/* up through each bus to the CPU's addresses: an empty ranges is the
|
||||
* same addresses above, no ranges at all is no way up */
|
||||
for (i = depth - 2; i >= 1; i--)
|
||||
{
|
||||
uint32_t cac = node_cells(fdt, path[i], "#address-cells", 2);
|
||||
uint32_t csc = node_cells(fdt, path[i], "#size-cells", 1);
|
||||
uint32_t pac = node_cells(fdt, path[i - 1], "#address-cells", 2);
|
||||
const unsigned char* r = (const unsigned char*)fdt_find_prop_in_node(fdt, path[i], "ranges", &len);
|
||||
uint32_t entry, at;
|
||||
int carried = 0;
|
||||
|
||||
if (!r) return 0;
|
||||
if (len == 0) continue;
|
||||
if (cac < 1 || cac > 2 || pac < 1 || pac > 2 || csc < 1 || csc > 2) return 0;
|
||||
entry = 4u * (cac + pac + csc);
|
||||
for (at = 0; at + entry <= len; at += entry)
|
||||
{
|
||||
uint64_t from = read_cells_at(r + at, cac);
|
||||
uint64_t to = read_cells_at(r + at + 4u * cac, pac);
|
||||
uint64_t size = read_cells_at(r + at + 4u * (cac + pac), csc);
|
||||
if (addr >= from && addr - from < size) { addr = to + (addr - from); carried = 1; break; }
|
||||
}
|
||||
if (!carried) return 0;
|
||||
}
|
||||
*out = addr;
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -146,7 +146,11 @@ int fdt_console_uart(const void* fdt, fdt_uart_t* out)
|
||||
if (!v) return 0;
|
||||
if (compatible_has(v, len, "arm,pl011") || compatible_has(v, len, "arm,pl011-axi"))
|
||||
{
|
||||
out->reg_shift = 0;
|
||||
/* A PL011's registers are 32-bit words: register n is at n << 2.
|
||||
* (0 until 2026-10-14, which put the flag register at +6 instead
|
||||
* of +0x18 -- an alignment fault on Device memory, found
|
||||
* rehearsing the Pi 5's native path; FABRIC-3.md SXXXIX.) */
|
||||
out->reg_shift = 2;
|
||||
out->io_width = 4;
|
||||
out->base = addr;
|
||||
out->kind = FDT_UART_PL011;
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+10
-3
@@ -174,7 +174,7 @@ int main(int argc, char **argv)
|
||||
fdt_uart_t u;
|
||||
const void *b;
|
||||
|
||||
if (argc == 6) { /* a real blob, and what it should give */
|
||||
if (argc == 6 || argc == 9) { /* a real blob, and what it should give */
|
||||
static unsigned char real[2 * 1024 * 1024];
|
||||
FILE *f = fopen(argv[1], "rb");
|
||||
size_t n;
|
||||
@@ -186,6 +186,13 @@ int main(int argc, char **argv)
|
||||
CHECK(n > 40 && fdt_console_uart(real, &u) && is(&u, kind, strtoull(argv[3], 0, 0), (uint32_t)strtoul(argv[4], 0, 0), (uint32_t)strtoul(argv[5], 0, 0)),
|
||||
"%s: %s at 0x%llx, registers %u apart (shift), %u wide", argv[1], SHOW(u));
|
||||
printf(" %s: %s at 0x%llx, shift %u, width %u\n", argv[1], SHOW(u));
|
||||
if (argc == 9) { /* and a node's reg, carried through its buses: argv[6] compatible, [7] [8] entries 0 and 1 */
|
||||
uint64_t a0 = 0, a1 = 0;
|
||||
CHECK(fdt_reg_address(real, argv[6], 0, &a0) && a0 == strtoull(argv[7], 0, 0) &&
|
||||
fdt_reg_address(real, argv[6], 1, &a1) && a1 == strtoull(argv[8], 0, 0),
|
||||
"%s: reg 0 at 0x%llx, reg 1 at 0x%llx", argv[6], (unsigned long long)a0, (unsigned long long)a1);
|
||||
printf(" %s: reg 0 at 0x%llx, reg 1 at 0x%llx\n", argv[6], (unsigned long long)a0, (unsigned long long)a1);
|
||||
}
|
||||
printf(" %d checks, %d failures\n", checks, failures);
|
||||
return failures != 0;
|
||||
}
|
||||
@@ -200,9 +207,9 @@ int main(int argc, char **argv)
|
||||
CHECK(fdt_console_uart(mars("serial0:115200n8", "okay", 1), &u) && u.kind == FDT_UART_16550, "and with status okay");
|
||||
|
||||
memset(&u, 0xAA, sizeof u);
|
||||
CHECK(fdt_console_uart(pi5("arm,pl011-axi\0arm,pl011\0arm,primecell", 37), &u) && is(&u, FDT_UART_PL011, 0x107d001000ull, 0, 4),
|
||||
CHECK(fdt_console_uart(pi5("arm,pl011-axi\0arm,pl011\0arm,primecell", 37), &u) && is(&u, FDT_UART_PL011, 0x107d001000ull, 2, 4),
|
||||
"the Pi 5: serial10, a PL011, its address carried through the bus's ranges to 0x107d001000: %s 0x%llx %u %u", SHOW(u));
|
||||
CHECK(fdt_console_uart(pi5("arm,pl011-axi", 14), &u) && is(&u, FDT_UART_PL011, 0x107d001000ull, 0, 4), "a node that says only arm,pl011-axi is a PL011 too");
|
||||
CHECK(fdt_console_uart(pi5("arm,pl011-axi", 14), &u) && is(&u, FDT_UART_PL011, 0x107d001000ull, 2, 4), "a node that says only arm,pl011-axi is a PL011 too");
|
||||
|
||||
memset(&u, 0xAA, sizeof u);
|
||||
CHECK(fdt_console_uart(virt(), &u) && is(&u, FDT_UART_16550, 0x10000000ull, 0, 1),
|
||||
|
||||
Reference in New Issue
Block a user