Stage MC-0b2: block subsystem behind g_lock_block (FABRIC-3.10.8 §3.3)

- block_subsystem.c (vendored): every public function's body becomes a
  static *_locked function; bodies call each other only through the
  _locked forms, breaking the internal cycles (relocate, meta relocate,
  migration scan, firsttouch, the meta accessors). Thin public wrappers
  take g_lock_block under __STARKERNEL__; hosted build has no lock and
  still builds clean.
- Lock order: hermes, switch, registry, doe, log, block, console, heap,
  vmm, pmm (Stadium, MC-0d, above block).
- Documented, not coverable by this lock: blk_get_buffer() pointers are
  used after release -- block-buffer ownership across cores is open for
  MC-3.
- Probe (removed): block lock balances on all three ISAs (2,057-2,379
  acquisitions), 0 recursions, 0 held at switch; vblk_io timeouts
  (0/0/4 probe, 0/0/2 acceptance) within the pre-lock riscv64 baseline
  of 0-6.
- Standard sequence on amd64/aarch64/riscv64: hashes and G byte-identical
  to MC-0b, MINT/WIREBIND/USE ok, conservation true, 0 idle growth.
- FABRIC-3.10.8 §3.2 wording fix; lock order noted as checked by
  inspection only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-25 18:57:38 -04:00
co-authored by Claude Opus 5.5
parent 5bc7b4dee7
commit 33f4090d6d
16 changed files with 55305 additions and 82 deletions
+63 -2
View File
@@ -143,7 +143,7 @@ Everything that would break if two cores ran kernel or VM code at the same momen
That crashed run's log was deleted as flawed. That crashed run's log was deleted as flawed.
- `sf_mutex_*` (the vendored VM's `dict_lock` and `tuning_lock`, `shim.c`) now use the same lock word. They are no longer no-ops. - `sf_mutex_*` (the vendored VM's `dict_lock` and `tuning_lock`, `shim.c`) now use the same lock word. They are no longer no-ops.
**The kernel locks, in acquisition order** (spinlock.h; MC-3's deadlock freedom rests on this order): **The kernel locks, in acquisition order** (spinlock.h; MC-3's deadlock freedom rests on this order; checked by reading the code, not yet by a counted check; MC-0b2 inserts `block` between `log` and `console`, §3.3):
1. `hermes` 1. `hermes`
2. `switch` 2. `switch`
3. `registry` 3. `registry`
@@ -186,7 +186,7 @@ It ran at Hera's prompt (after O-A's G, PREC and LOAD), and again on rajames aft
- Every lock balances individually on every ISA, at both readings. - Every lock balances individually on every ISA, at both readings.
- The amd64 and aarch64 probe runs predate the `sk_percpu_ready` guard. That guard only changes behaviour before `sk_percpu_boot_init()`, and the acceptance runs below are on the final code. - The amd64 and aarch64 probe runs predate the `sk_percpu_ready` guard. That guard only changes behaviour before `sk_percpu_boot_init()`, and the acceptance runs below are on the final code.
- The riscv64 `vblk_io timed out` count, which the advisor flagged (defect 3 sits in the region now masked), is 2 in the probe run, the same as MC-0a's acceptance run. It is 0 in this stage's acceptance run. - The riscv64 `vblk_io timed out` count, watched because known defect 3 sits in code that now runs with interrupts masked, is 2 in the probe run, the same as MC-0a's acceptance run. It is 0 in this stage's acceptance run.
**Finding, for MC-3.** Hera's input-poll idle loop runs the switch decision without pause, and every pass takes the registry, switch and Hermes locks once per slot. That is the hundreds of millions of acquisitions above (amd64: registry 379 M, switch 858 M, Hermes 266 M). It is harmless on one core. With several cores, core 0 spinning through these same locks would contend with every worker's claim. MC-3 should settle how that loop waits before cores share these locks. **Finding, for MC-3.** Hera's input-poll idle loop runs the switch decision without pause, and every pass takes the registry, switch and Hermes locks once per slot. That is the hundreds of millions of acquisitions above (amd64: registry 379 M, switch 858 M, Hermes 266 M). It is harmless on one core. With several cores, core 0 spinning through these same locks would contend with every worker's claim. MC-3 should settle how that loop waits before cores share these locks.
@@ -201,3 +201,64 @@ It ran at Hera's prompt (after O-A's G, PREC and LOAD), and again on rajames aft
- The G string is byte-identical to MC-0a on all three ISAs, so masking interrupts under the locks didn't move the interleave. - The G string is byte-identical to MC-0a on all three ISAs, so masking interrupts under the locks didn't move the interleave.
- The `dict_hash` set and the ledger line are unchanged from MC-0a. - The `dict_hash` set and the ledger line are unchanged from MC-0a.
- No stop condition was hit. **MC-0b done** (the block subsystem follows as MC-0b2). - No stop condition was hit. **MC-0b done** (the block subsystem follows as MC-0b2).
### 3.3 MC-0b2 — the block subsystem's lock (2026-09-25)
**Change.** `src/block_subsystem.c` (vendored) is split into bodies and wrappers:
- Every public function's body is now a static `*_locked` function.
- Bodies call each other only through the `_locked` forms. That breaks the internal cycles:
- relocate → get buffer / update / mark free;
- meta relocate → relocate;
- migration scan → meta relocate;
- firsttouch → get meta / set meta / mark allocated;
- the six meta accessors.
- A block of thin public wrappers at the end of the file takes the lock around each body.
- Under `__STARKERNEL__` that lock is `g_lock_block`.
- The hosted build has no lock, and the vendored hosted `make` still builds clean.
- The lock covers the device chain, every slot's cache and BAM, the relocation table, and the device I/O issued on their behalf.
- Its place in the order is **below `log` and `doe`** (both regions write through the block subsystem) and **above `console`** (block logs), `heap` (`calloc`/`free`), `vmm` and `pmm`:
1. `hermes`
2. `switch`
3. `registry`
4. `doe`
5. `log`
6. `block`
7. `console`
8. `heap`
9. `vmm`
10. `pmm`
Stadium (MC-0d) sits above `block`, because `STADIUM_BEHAVIOUR_MIGRATE` calls `blk_flush()`.
**Not covered, and not coverable by this lock:** `blk_get_buffer()` returns a pointer into RAM, a ramdrive, or a device cache slot. The caller (`BLOCK`/`BUFFER`, Artemis) reads and writes through that pointer after the lock is released. On one core nothing can evict the slot in between; with worker cores (MC-3) another core's block access can. Block-buffer ownership across cores is an open item for MC-3.
**The masking risk, measured.** `virtio_blk.c` records (2026-09-09) that masking interrupts around `vblk_io()` once made missed completions worse. All block I/O now runs with interrupts masked (the MC-0b primitive), so the probe runs counted `vblk_io timed out` on every ISA:
| Where | amd64 | aarch64 | riscv64 |
|---|---|---|---|
| This stage's probe runs | 0 | 0 | 4 |
| This stage's acceptance runs | 0 | 0 | 2 |
The riscv64 counts sit inside the pre-lock baseline: the 24 most recent riscv64 logs, from 2026-09-24 (no locks at all) through this stage, range from 0 to 6, with 4, 5 and 6 all seen before MC-0b. No regression is visible at this sample size. The masking stays as MC-0b ruled.
**Probe** (temporary, removed before commit). This is MC-0b's `MC0B-PROBE` word, extended with the block lock. Same sequence: O-A, then Hera's prompt, then MINT, WIREBIND, `USE` and `1 2 + .`, then rajames. The table shows the final (rajames) reading:
| ISA | log | block acquired = released | total acquired = released | depth | switch/idle checks | held at switch | recursions / bad releases / destroyed held |
|---|---|---|---|---|---|---|---|
| amd64 | `logs/20260925-173901` | 2,379 | 1,387,861,111 | 0 | 46,795,001 | 0 | 0 / 0 / 0 |
| aarch64 | `logs/20260925-175215` | 2,057 | 247,761,816 | 0 | 8,014,413 | 0 | 0 / 0 / 0 |
| riscv64 | `logs/20260925-180546` | 2,300 | 555,650,314 | 0 | 18,447,619 | 0 | 0 / 0 / 0 |
Every lock balances individually on every ISA, at both readings. 0 recursions means no public-to-public call inside `block_subsystem.c` was missed.
**Standard sequence** (clean build, with the probe reverted; the tree is byte-identical to the saved stage diff).
| ISA | log | O-A (G, PREC, LOAD) | M7.1a hash | MINT / WIREBIND / USE / `1 2 + .` | idle growth | UNKNOWN / faults |
|---|---|---|---|---|---|---|
| amd64 | `logs/20260925-181908` | A=40 B=40, HHH, ok | `0x99b01134c3d3049c` | all ok, `3` | 0 B / 20 s | 0 / 0 |
| aarch64 | `logs/20260925-183151` | A=40 B=40, HHH, ok | `0x99b01134c3d3049c` | all ok, `3` | 0 B / 20 s | 0 / 0 |
| riscv64 | `logs/20260925-184454` | A=40 B=40, HHH, ok | `0x99b01134c3d3049c` | all ok, `3` | 0 B / 20 s | 0 / 0 |
- The G string is byte-identical to MC-0b and MC-0a on all three ISAs.
- The `dict_hash` set and the ledger line are unchanged.
- No stop condition was hit. **MC-0b2 done.**
+4
View File
@@ -52,6 +52,9 @@
* g_lock_registry VM registry list and entries (capsule_birth.c) * g_lock_registry VM registry list and entries (capsule_birth.c)
* g_lock_doe DoE region (doe_region.c) * g_lock_doe DoE region (doe_region.c)
* g_lock_log log region (log_region.c) * g_lock_log log region (log_region.c)
* g_lock_block block subsystem: device chain, caches, BAM,
* relocation table, and its device I/O
* (block_subsystem.c, MC-0b2)
* g_lock_console console output and line state (console.c) * g_lock_console console output and line state (console.c)
* g_lock_heap kmalloc heap * g_lock_heap kmalloc heap
* g_lock_vmm page tables (allocates tables from the PMM) * g_lock_vmm page tables (allocates tables from the PMM)
@@ -103,6 +106,7 @@ extern sk_spinlock_t g_lock_switch;
extern sk_spinlock_t g_lock_hermes; extern sk_spinlock_t g_lock_hermes;
extern sk_spinlock_t g_lock_doe; extern sk_spinlock_t g_lock_doe;
extern sk_spinlock_t g_lock_log; extern sk_spinlock_t g_lock_log;
extern sk_spinlock_t g_lock_block;
extern sk_spinlock_t g_lock_console; extern sk_spinlock_t g_lock_console;
extern sk_spinlock_t g_lock_heap; extern sk_spinlock_t g_lock_heap;
extern sk_spinlock_t g_lock_pmm; extern sk_spinlock_t g_lock_pmm;
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+244 -80
View File
@@ -58,6 +58,54 @@
#include <stdlib.h> #include <stdlib.h>
#include <stdint.h> #include <stdint.h>
/* ===== locking (FABRIC-3.10.8 Stage MC-0b2) =====
*
* The kernel build serializes this whole subsystem -- the device chain,
* every slot's cache and BAM, the relocation table, and the device I/O
* issued on their behalf -- behind one spinlock, g_lock_block
* (starkernel/spinlock.h; below the log and DoE regions, above the console
* and heap in the lock order). The hosted build has no second thread and
* no lock.
*
* Every public function is a thin wrapper at the bottom of this file that
* takes the lock around its *_locked body. Bodies call each other only
* through the *_locked forms (this file's public functions call one
* another in cycles: set/get meta, the migration scan, relocation), so the
* lock is never taken twice.
*
* Not covered, and cannot be by a lock here: blk_get_buffer() returns a
* pointer into a RAM block, a ramdrive, or a device cache slot, which the
* caller reads and writes after the lock is released. On one core nothing
* can evict that slot in between; with worker cores (MC-3) it can. */
#ifdef __STARKERNEL__
#include "starkernel/spinlock.h"
#define BLK_LOCK() sk_spin_lock(&g_lock_block)
#define BLK_UNLOCK() sk_spin_unlock(&g_lock_block)
#else
#define BLK_LOCK() ((void) 0)
#define BLK_UNLOCK() ((void) 0)
#endif
struct blkio_dev;
static int blk_subsys_init_locked(VM *vm, uint8_t *ram_base, size_t ram_size);
static int blk_subsys_add_raw_device_locked(uint8_t *buf, uint32_t nblocks);
static int blk_subsys_attach_device_locked(struct blkio_dev *dev);
static int blk_subsys_detach_device_locked(struct blkio_dev *dev);
static int blk_subsys_relocate_block_locked(uint32_t home_lbn, uint32_t target_lbn);
static int blk_subsys_shutdown_locked(void);
static int blk_subsys_confirm_format_locked(uint32_t lbn);
static uint8_t *blk_get_buffer_locked(uint32_t block_num, int writable);
static int blk_update_locked(uint32_t block_num);
static int blk_flush_locked(uint32_t block_num);
static int blk_is_allocated_locked(uint32_t block_num);
static int blk_mark_allocated_locked(uint32_t block_num);
static int blk_mark_free_locked(uint32_t block_num);
static int blk_allocate_locked(uint32_t *block_num);
static int blk_is_valid_locked(uint32_t block_num);
static int blk_get_meta_locked(uint32_t block_num, blk_meta_t *meta);
static int blk_set_meta_locked(uint32_t block_num, const blk_meta_t *meta);
static int blk_meta_relocate_devblock_locked(uint32_t home_devblock, uint32_t target_devblock);
/* ===== compile-time config ===== */ /* ===== compile-time config ===== */
#ifndef BLK_FORTH_SYS_RESERVED #ifndef BLK_FORTH_SYS_RESERVED
# define BLK_FORTH_SYS_RESERVED 32u # define BLK_FORTH_SYS_RESERVED 32u
@@ -644,7 +692,7 @@ static inline uint64_t blk_get_timestamp(void) {
/* ===== public API ===== */ /* ===== public API ===== */
int blk_subsys_init(VM *vm, uint8_t *ram_base, size_t ram_size) { static int blk_subsys_init_locked(VM *vm, uint8_t *ram_base, size_t ram_size) {
if (!vm || !ram_base || ram_size < ((size_t) BLK_RAM_BLOCKS * BLK_FORTH_SIZE)) if (!vm || !ram_base || ram_size < ((size_t) BLK_RAM_BLOCKS * BLK_FORTH_SIZE))
return BLK_EINVAL; return BLK_EINVAL;
memset(&g, 0, sizeof(g)); memset(&g, 0, sizeof(g));
@@ -659,7 +707,7 @@ int blk_subsys_init(VM *vm, uint8_t *ram_base, size_t ram_size) {
return BLK_OK; return BLK_OK;
} }
int blk_subsys_add_raw_device(uint8_t *buf, uint32_t nblocks) { static int blk_subsys_add_raw_device_locked(uint8_t *buf, uint32_t nblocks) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
if (!buf || nblocks == 0) return BLK_EINVAL; if (!buf || nblocks == 0) return BLK_EINVAL;
@@ -685,7 +733,7 @@ int blk_subsys_add_raw_device(uint8_t *buf, uint32_t nblocks) {
static blk_dev_slot_t *first_disk_slot(void); /* defined below; used here to identify the static blk_dev_slot_t *first_disk_slot(void); /* defined below; used here to identify the
* relocation-table owner right after attach */ * relocation-table owner right after attach */
int blk_subsys_attach_device(struct blkio_dev *dev) { static int blk_subsys_attach_device_locked(struct blkio_dev *dev) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
if (!dev) return BLK_EINVAL; if (!dev) return BLK_EINVAL;
@@ -752,7 +800,7 @@ int blk_subsys_attach_device(struct blkio_dev *dev) {
* opposed to a surprise removal) wants a best-effort flush first; today * opposed to a surprise removal) wants a best-effort flush first; today
* every removal this driver can observe is a surprise removal. * every removal this driver can observe is a surprise removal.
*/ */
int blk_subsys_detach_device(struct blkio_dev *dev) { static int blk_subsys_detach_device_locked(struct blkio_dev *dev) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
if (!dev) return BLK_EINVAL; if (!dev) return BLK_EINVAL;
@@ -788,7 +836,7 @@ int blk_subsys_detach_device(struct blkio_dev *dev) {
* job) or validate that target_lbn is genuinely owned by whoever is * job) or validate that target_lbn is genuinely owned by whoever is
* asking (also ACL's job) -- it just executes one, correctly, once told * asking (also ACL's job) -- it just executes one, correctly, once told
* to. */ * to. */
int blk_subsys_relocate_block(uint32_t home_lbn, uint32_t target_lbn) { static int blk_subsys_relocate_block_locked(uint32_t home_lbn, uint32_t target_lbn) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
if (home_lbn == target_lbn) return BLK_EINVAL; if (home_lbn == target_lbn) return BLK_EINVAL;
@@ -803,7 +851,7 @@ int blk_subsys_relocate_block(uint32_t home_lbn, uint32_t target_lbn) {
} }
if (g.reloc_count >= BLK_RELOC_MAX_ENTRIES) return BLK_ENOMEM; if (g.reloc_count >= BLK_RELOC_MAX_ENTRIES) return BLK_ENOMEM;
if (!blk_is_valid(home_lbn) || !blk_is_valid(target_lbn)) return BLK_ERANGE; if (!blk_is_valid_locked(home_lbn) || !blk_is_valid_locked(target_lbn)) return BLK_ERANGE;
/* Same-device only (Phase 8 v3, 2026-09-23): this function's own purpose /* Same-device only (Phase 8 v3, 2026-09-23): this function's own purpose
* is relocating a block to a new LBN on the device it's being relocated * is relocating a block to a new LBN on the device it's being relocated
@@ -824,21 +872,21 @@ int blk_subsys_relocate_block(uint32_t home_lbn, uint32_t target_lbn) {
* device's cache array, if home_lbn and target_lbn happen to share a * device's cache array, if home_lbn and target_lbn happen to share a
* device. Not a hypothetical: this is exactly the class of stale * device. Not a hypothetical: this is exactly the class of stale
* pointer this file's own blk_vm_evict() comment already warns about. */ * pointer this file's own blk_vm_evict() comment already warns about. */
uint8_t *src = blk_get_buffer(home_lbn, 0); uint8_t *src = blk_get_buffer_locked(home_lbn, 0);
if (!src) return BLK_EIO; if (!src) return BLK_EIO;
uint8_t staged[BLK_FORTH_SIZE]; uint8_t staged[BLK_FORTH_SIZE];
memcpy(staged, src, BLK_FORTH_SIZE); memcpy(staged, src, BLK_FORTH_SIZE);
uint8_t *dst = blk_get_buffer(target_lbn, 1); uint8_t *dst = blk_get_buffer_locked(target_lbn, 1);
if (!dst) return BLK_EIO; if (!dst) return BLK_EIO;
memcpy(dst, staged, BLK_FORTH_SIZE); memcpy(dst, staged, BLK_FORTH_SIZE);
if (blk_update(target_lbn) != BLK_OK) return BLK_EIO; if (blk_update_locked(target_lbn) != BLK_OK) return BLK_EIO;
/* Free home_lbn's original backing block -- must happen before the /* Free home_lbn's original backing block -- must happen before the
* redirect is inserted below, while resolve_lbn(home_lbn) still * redirect is inserted below, while resolve_lbn(home_lbn) still
* resolves to itself; inserting the redirect first would make this * resolves to itself; inserting the redirect first would make this
* call free target_lbn instead. */ * call free target_lbn instead. */
(void) blk_mark_free(home_lbn); (void) blk_mark_free_locked(home_lbn);
g.reloc[g.reloc_count].home_lbn = home_lbn; g.reloc[g.reloc_count].home_lbn = home_lbn;
g.reloc[g.reloc_count].actual_lbn = target_lbn; g.reloc[g.reloc_count].actual_lbn = target_lbn;
@@ -851,14 +899,10 @@ int blk_subsys_relocate_block(uint32_t home_lbn, uint32_t target_lbn) {
return BLK_OK; return BLK_OK;
} }
uint64_t blk_subsys_epoch(void) { static int blk_subsys_shutdown_locked(void) {
return g.epoch;
}
int blk_subsys_shutdown(void) {
if (!g.initialized) return BLK_OK; if (!g.initialized) return BLK_OK;
blk_flush(0); blk_flush_locked(0);
blk_dev_slot_t *s = g.head; blk_dev_slot_t *s = g.head;
while (s) { while (s) {
@@ -889,7 +933,7 @@ int blk_subsys_shutdown(void) {
* decided it is safe to touch (e.g. Artemis's ART-BOOT-DETECT resolving * decided it is safe to touch (e.g. Artemis's ART-BOOT-DETECT resolving
* to BLANK or a recognized marker). Must NOT be called on the path that * to BLANK or a recognized marker). Must NOT be called on the path that
* halts for unrecognized content — that is what keeps the disk untouched. */ * halts for unrecognized content — that is what keeps the disk untouched. */
int blk_subsys_confirm_format(uint32_t lbn) { static int blk_subsys_confirm_format_locked(uint32_t lbn) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
blk_dev_slot_t *slot = lbn_to_slot(lbn); blk_dev_slot_t *slot = lbn_to_slot(lbn);
if (!slot || !slot->dev) return BLK_ERANGE; if (!slot || !slot->dev) return BLK_ERANGE;
@@ -898,7 +942,7 @@ int blk_subsys_confirm_format(uint32_t lbn) {
/* ===== block buffer access ===== */ /* ===== block buffer access ===== */
uint8_t *blk_get_buffer(uint32_t block_num, int writable) { static uint8_t *blk_get_buffer_locked(uint32_t block_num, int writable) {
if (!g.initialized) return NULL; if (!g.initialized) return NULL;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
@@ -935,13 +979,13 @@ uint8_t *blk_get_buffer(uint32_t block_num, int writable) {
return c->data + pack * BLK_FORTH_SIZE; return c->data + pack * BLK_FORTH_SIZE;
} }
uint8_t *blk_get_empty_buffer(uint32_t block_num) { static uint8_t *blk_get_empty_buffer_locked(uint32_t block_num) {
uint8_t *p = blk_get_buffer(block_num, 1); uint8_t *p = blk_get_buffer_locked(block_num, 1);
if (p) memset(p, 0, BLK_FORTH_SIZE); if (p) memset(p, 0, BLK_FORTH_SIZE);
return p; return p;
} }
int blk_update(uint32_t block_num) { static int blk_update_locked(uint32_t block_num) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
@@ -997,7 +1041,7 @@ int blk_update(uint32_t block_num) {
return BLK_OK; return BLK_OK;
} }
int blk_flush(uint32_t block_num) { static int blk_flush_locked(uint32_t block_num) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
if (block_num > 0) block_num = resolve_lbn(block_num); /* 0 is the "flush all" sentinel */ if (block_num > 0) block_num = resolve_lbn(block_num); /* 0 is the "flush all" sentinel */
@@ -1052,7 +1096,7 @@ int blk_flush(uint32_t block_num) {
/* ===== BAM wrappers ===== */ /* ===== BAM wrappers ===== */
int blk_is_allocated(uint32_t block_num) { static int blk_is_allocated_locked(uint32_t block_num) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
if (block_num < g.ram_user) return BLK_EINVAL; if (block_num < g.ram_user) return BLK_EINVAL;
@@ -1061,7 +1105,7 @@ int blk_is_allocated(uint32_t block_num) {
return slot->bam[block_num - slot->start_lbn].allocated ? 1 : 0; return slot->bam[block_num - slot->start_lbn].allocated ? 1 : 0;
} }
int blk_mark_allocated(uint32_t block_num) { static int blk_mark_allocated_locked(uint32_t block_num) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
if (block_num < g.ram_user) return BLK_EINVAL; if (block_num < g.ram_user) return BLK_EINVAL;
@@ -1080,7 +1124,7 @@ int blk_mark_allocated(uint32_t block_num) {
return BLK_OK; return BLK_OK;
} }
int blk_mark_free(uint32_t block_num) { static int blk_mark_free_locked(uint32_t block_num) {
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
if (block_num < g.ram_user) return BLK_EINVAL; if (block_num < g.ram_user) return BLK_EINVAL;
@@ -1125,11 +1169,11 @@ static int devblock_claimed_by_lbn(blk_dev_slot_t *slot, uint32_t lbn) {
uint32_t offset = lbn - slot->start_lbn; uint32_t offset = lbn - slot->start_lbn;
if (offset < pack) return 0; if (offset < pack) return 0;
blk_meta_t meta; blk_meta_t meta;
if (blk_get_meta(lbn - pack, &meta) != BLK_OK) return 0; if (blk_get_meta_locked(lbn - pack, &meta) != BLK_OK) return 0;
return (meta.flags & BLK_FLAG_CLAIMED) != 0; return (meta.flags & BLK_FLAG_CLAIMED) != 0;
} }
int blk_allocate(uint32_t *block_num) { static int blk_allocate_locked(uint32_t *block_num) {
if (!block_num) return BLK_EINVAL; if (!block_num) return BLK_EINVAL;
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
@@ -1170,7 +1214,7 @@ static blk_dev_slot_t *first_disk_slot(void) {
return NULL; return NULL;
} }
int blk_get_volume_meta(blk_volume_meta_t *meta) { static int blk_get_volume_meta_locked(blk_volume_meta_t *meta) {
if (!meta) return BLK_EINVAL; if (!meta) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
if (!slot) return BLK_ENODEV; if (!slot) return BLK_ENODEV;
@@ -1178,7 +1222,7 @@ int blk_get_volume_meta(blk_volume_meta_t *meta) {
return BLK_OK; return BLK_OK;
} }
int blk_set_volume_meta(const blk_volume_meta_t *meta) { static int blk_set_volume_meta_locked(const blk_volume_meta_t *meta) {
if (!meta) return BLK_EINVAL; if (!meta) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
if (!slot) return BLK_ENODEV; if (!slot) return BLK_ENODEV;
@@ -1197,7 +1241,7 @@ static blk_dev_slot_t *slot_by_dev(struct blkio_dev *dev) {
return NULL; return NULL;
} }
int blk_get_device_range(struct blkio_dev *dev, uint32_t *out_start_lbn, uint32_t *out_count) { static int blk_get_device_range_locked(struct blkio_dev *dev, uint32_t *out_start_lbn, uint32_t *out_count) {
if (!dev || !out_start_lbn || !out_count) return BLK_EINVAL; if (!dev || !out_start_lbn || !out_count) return BLK_EINVAL;
blk_dev_slot_t *slot = slot_by_dev(dev); blk_dev_slot_t *slot = slot_by_dev(dev);
if (!slot) return BLK_ENODEV; if (!slot) return BLK_ENODEV;
@@ -1206,11 +1250,11 @@ int blk_get_device_range(struct blkio_dev *dev, uint32_t *out_start_lbn, uint32_
return BLK_OK; return BLK_OK;
} }
const void *blk_lbn_device_handle(uint32_t lbn) { static const void *blk_lbn_device_handle_locked(uint32_t lbn) {
return (const void *) lbn_to_slot(resolve_lbn(lbn)); return (const void *) lbn_to_slot(resolve_lbn(lbn));
} }
int blk_get_device_free_blocks(struct blkio_dev *dev, uint64_t *out_free, uint64_t *out_total) { static int blk_get_device_free_blocks_locked(struct blkio_dev *dev, uint64_t *out_free, uint64_t *out_total) {
if (!dev || !out_free || !out_total) return BLK_EINVAL; if (!dev || !out_free || !out_total) return BLK_EINVAL;
blk_dev_slot_t *slot = slot_by_dev(dev); blk_dev_slot_t *slot = slot_by_dev(dev);
if (!slot) return BLK_ENODEV; if (!slot) return BLK_ENODEV;
@@ -1220,7 +1264,7 @@ int blk_get_device_free_blocks(struct blkio_dev *dev, uint64_t *out_free, uint64
return BLK_OK; return BLK_OK;
} }
int blk_get_first_disk_range(uint32_t *out_start_lbn, uint32_t *out_count) { static int blk_get_first_disk_range_locked(uint32_t *out_start_lbn, uint32_t *out_count) {
if (!out_start_lbn || !out_count) return BLK_EINVAL; if (!out_start_lbn || !out_count) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
if (!slot) return BLK_ENODEV; if (!slot) return BLK_ENODEV;
@@ -1241,7 +1285,7 @@ int blk_get_first_disk_range(uint32_t *out_start_lbn, uint32_t *out_count) {
* No FORTH word wraps this -- C-only, same discipline as * No FORTH word wraps this -- C-only, same discipline as
* vm_zuse_cert_install() itself; Zuse's cert is this zone's first tenant. * vm_zuse_cert_install() itself; Zuse's cert is this zone's first tenant.
*/ */
int blk_meta_zone_read(uint32_t devblock_from_top, uint8_t buf[4096]) { static int blk_meta_zone_read_locked(uint32_t devblock_from_top, uint8_t buf[4096]) {
if (!buf) return BLK_EINVAL; if (!buf) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
if (!slot || !slot->dev) return BLK_ENODEV; if (!slot || !slot->dev) return BLK_ENODEV;
@@ -1261,7 +1305,7 @@ int blk_meta_zone_read(uint32_t devblock_from_top, uint8_t buf[4096]) {
return BLK_OK; return BLK_OK;
} }
int blk_meta_zone_write(uint32_t devblock_from_top, const uint8_t buf[4096]) { static int blk_meta_zone_write_locked(uint32_t devblock_from_top, const uint8_t buf[4096]) {
if (!buf) return BLK_EINVAL; if (!buf) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
if (!slot || !slot->dev) return BLK_ENODEV; if (!slot || !slot->dev) return BLK_ENODEV;
@@ -1280,19 +1324,14 @@ int blk_meta_zone_write(uint32_t devblock_from_top, const uint8_t buf[4096]) {
return BLK_OK; return BLK_OK;
} }
int blk_is_valid(uint32_t block_num) { static int blk_is_valid_locked(uint32_t block_num) {
if (!g.initialized) return 0; if (!g.initialized) return 0;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
if (block_num < g.ram_user) return 1; if (block_num < g.ram_user) return 1;
return lbn_to_slot(block_num) ? 1 : 0; return lbn_to_slot(block_num) ? 1 : 0;
} }
uint32_t blk_get_total_blocks(void) { static int blk_get_meta_locked(uint32_t block_num, blk_meta_t *meta) {
uint64_t n = g.total_user_lbn;
return (n > 0xFFFFFFFFu) ? 0xFFFFFFFFu : (uint32_t) n;
}
int blk_get_meta(uint32_t block_num, blk_meta_t *meta) {
if (!meta) return BLK_EINVAL; if (!meta) return BLK_EINVAL;
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
@@ -1315,7 +1354,7 @@ int blk_get_meta(uint32_t block_num, blk_meta_t *meta) {
return BLK_OK; return BLK_OK;
} }
int blk_set_meta(uint32_t block_num, const blk_meta_t *meta) { static int blk_set_meta_locked(uint32_t block_num, const blk_meta_t *meta) {
if (!meta) return BLK_EINVAL; if (!meta) return BLK_EINVAL;
if (!g.initialized) return BLK_ENODEV; if (!g.initialized) return BLK_ENODEV;
block_num = resolve_lbn(block_num); block_num = resolve_lbn(block_num);
@@ -1340,78 +1379,78 @@ int blk_set_meta(uint32_t block_num, const blk_meta_t *meta) {
* which already own caching/dirty-tracking -- these add no state of * which already own caching/dirty-tracking -- these add no state of
* their own. */ * their own. */
int blk_owner_fp_get(uint32_t block_num, uint8_t out_fp[8]) { static int blk_owner_fp_get_locked(uint32_t block_num, uint8_t out_fp[8]) {
blk_meta_t meta; blk_meta_t meta;
int rc; int rc;
if (!out_fp) return BLK_EINVAL; if (!out_fp) return BLK_EINVAL;
rc = blk_get_meta(block_num, &meta); rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
memcpy(out_fp, meta.owner_fp, sizeof(meta.owner_fp)); memcpy(out_fp, meta.owner_fp, sizeof(meta.owner_fp));
return BLK_OK; return BLK_OK;
} }
int blk_owner_fp_set(uint32_t block_num, const uint8_t fp[8]) { static int blk_owner_fp_set_locked(uint32_t block_num, const uint8_t fp[8]) {
blk_meta_t meta; blk_meta_t meta;
int rc; int rc;
if (!fp) return BLK_EINVAL; if (!fp) return BLK_EINVAL;
rc = blk_get_meta(block_num, &meta); rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
memcpy(meta.owner_fp, fp, sizeof(meta.owner_fp)); memcpy(meta.owner_fp, fp, sizeof(meta.owner_fp));
return blk_set_meta(block_num, &meta); return blk_set_meta_locked(block_num, &meta);
} }
int blk_acl_allow_get(uint32_t block_num, uint8_t *out_allow) { static int blk_acl_allow_get_locked(uint32_t block_num, uint8_t *out_allow) {
blk_meta_t meta; blk_meta_t meta;
int rc; int rc;
if (!out_allow) return BLK_EINVAL; if (!out_allow) return BLK_EINVAL;
rc = blk_get_meta(block_num, &meta); rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
*out_allow = meta.acl_allow; *out_allow = meta.acl_allow;
return BLK_OK; return BLK_OK;
} }
int blk_acl_allow_set(uint32_t block_num, uint8_t allow) { static int blk_acl_allow_set_locked(uint32_t block_num, uint8_t allow) {
blk_meta_t meta; blk_meta_t meta;
int rc = blk_get_meta(block_num, &meta); int rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
meta.acl_allow = allow; meta.acl_allow = allow;
return blk_set_meta(block_num, &meta); return blk_set_meta_locked(block_num, &meta);
} }
int blk_acl_ttl_get(uint32_t block_num, uint32_t *out_ttl) { static int blk_acl_ttl_get_locked(uint32_t block_num, uint32_t *out_ttl) {
blk_meta_t meta; blk_meta_t meta;
int rc; int rc;
if (!out_ttl) return BLK_EINVAL; if (!out_ttl) return BLK_EINVAL;
rc = blk_get_meta(block_num, &meta); rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
*out_ttl = meta.acl_ttl; *out_ttl = meta.acl_ttl;
return BLK_OK; return BLK_OK;
} }
int blk_acl_ttl_set(uint32_t block_num, uint32_t ttl) { static int blk_acl_ttl_set_locked(uint32_t block_num, uint32_t ttl) {
blk_meta_t meta; blk_meta_t meta;
int rc = blk_get_meta(block_num, &meta); int rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
meta.acl_ttl = ttl; meta.acl_ttl = ttl;
return blk_set_meta(block_num, &meta); return blk_set_meta_locked(block_num, &meta);
} }
int blk_flags_get(uint32_t block_num, uint64_t *out_flags) { static int blk_flags_get_locked(uint32_t block_num, uint64_t *out_flags) {
blk_meta_t meta; blk_meta_t meta;
int rc; int rc;
if (!out_flags) return BLK_EINVAL; if (!out_flags) return BLK_EINVAL;
rc = blk_get_meta(block_num, &meta); rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
*out_flags = meta.flags; *out_flags = meta.flags;
return BLK_OK; return BLK_OK;
} }
int blk_flags_set(uint32_t block_num, uint64_t flags) { static int blk_flags_set_locked(uint32_t block_num, uint64_t flags) {
blk_meta_t meta; blk_meta_t meta;
int rc = blk_get_meta(block_num, &meta); int rc = blk_get_meta_locked(block_num, &meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
meta.flags = flags; meta.flags = flags;
return blk_set_meta(block_num, &meta); return blk_set_meta_locked(block_num, &meta);
} }
/* FABRIC-2.md §F.11/§I.2, built 2026-09-04. is_lbn_zero() -- BLK_FLAG_ /* FABRIC-2.md §F.11/§I.2, built 2026-09-04. is_lbn_zero() -- BLK_FLAG_
@@ -1454,7 +1493,7 @@ static int devblock_is_free(blk_dev_slot_t *slot, uint32_t rep_lbn, const blk_me
* devblocks in one claim. */ * devblocks in one claim. */
#define BLK_FIRSTTOUCH_MAX_CLAIM 256u #define BLK_FIRSTTOUCH_MAX_CLAIM 256u
int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *out_chain_head) { static int blk_firsttouch_claim_locked(const uint8_t owner_fp[8], uint32_t count, uint32_t *out_chain_head) {
if (!owner_fp || !out_chain_head || count == 0) return BLK_EINVAL; if (!owner_fp || !out_chain_head || count == 0) return BLK_EINVAL;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
@@ -1477,7 +1516,7 @@ int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *ou
if (slot_pbn_pack_offset(rel_pbn) != 0) continue; /* not this devblock's representative LBN */ if (slot_pbn_pack_offset(rel_pbn) != 0) continue; /* not this devblock's representative LBN */
blk_meta_t meta; blk_meta_t meta;
if (blk_get_meta(lbn, &meta) != BLK_OK) continue; if (blk_get_meta_locked(lbn, &meta) != BLK_OK) continue;
if (devblock_is_free(slot, lbn, &meta)) found[nfound++] = lbn; if (devblock_is_free(slot, lbn, &meta)) found[nfound++] = lbn;
} }
@@ -1488,13 +1527,13 @@ int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *ou
* locally from any member without walking the chain. */ * locally from any member without walking the chain. */
for (uint32_t i = 0; i < nfound; i++) { for (uint32_t i = 0; i < nfound; i++) {
blk_meta_t meta; blk_meta_t meta;
if (blk_get_meta(found[i], &meta) != BLK_OK) return BLK_EIO; if (blk_get_meta_locked(found[i], &meta) != BLK_OK) return BLK_EIO;
memcpy(meta.owner_fp, owner_fp, 8); memcpy(meta.owner_fp, owner_fp, 8);
meta.flags |= BLK_FLAG_CLAIMED; meta.flags |= BLK_FLAG_CLAIMED;
meta.prev_block = (i == 0) ? 0 : found[i - 1]; meta.prev_block = (i == 0) ? 0 : found[i - 1];
meta.next_block = (i + 1 == nfound) ? 0 : found[i + 1]; meta.next_block = (i + 1 == nfound) ? 0 : found[i + 1];
meta.chain_length = nfound; meta.chain_length = nfound;
if (blk_set_meta(found[i], &meta) != BLK_OK) return BLK_EIO; if (blk_set_meta_locked(found[i], &meta) != BLK_OK) return BLK_EIO;
/* blk_meta_t/BAM reconciliation, 2026-09-05: blk_set_meta() above /* blk_meta_t/BAM reconciliation, 2026-09-05: blk_set_meta() above
* only touches blk_meta_t -- mark the BAM allocated too, for all * only touches blk_meta_t -- mark the BAM allocated too, for all
@@ -1504,7 +1543,7 @@ int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *ou
* (blk_mark_allocated()'s own bookkeeping) correctly reflects the * (blk_mark_allocated()'s own bookkeeping) correctly reflects the
* claim instead of silently overstating free space. */ * claim instead of silently overstating free space. */
for (uint32_t j = 0; j < BLK_PACK_RATIO; j++) { for (uint32_t j = 0; j < BLK_PACK_RATIO; j++) {
(void) blk_mark_allocated(found[i] + j); (void) blk_mark_allocated_locked(found[i] + j);
} }
} }
@@ -1515,7 +1554,7 @@ int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *ou
/* blk_owner_has_claim - FABRIC-2.md §I.2, 2026-09-05. Disk-scan answer to /* blk_owner_has_claim - FABRIC-2.md §I.2, 2026-09-05. Disk-scan answer to
* "does owner_fp already own a claim," per the decision recorded in * "does owner_fp already own a claim," per the decision recorded in
* block_subsystem.h: matching BLK_FLAG_CLAIMED devblock, not a RAM flag. */ * block_subsystem.h: matching BLK_FLAG_CLAIMED devblock, not a RAM flag. */
int blk_owner_has_claim(const uint8_t owner_fp[8]) { static int blk_owner_has_claim_locked(const uint8_t owner_fp[8]) {
if (!owner_fp) return 0; if (!owner_fp) return 0;
blk_dev_slot_t *slot = first_disk_slot(); blk_dev_slot_t *slot = first_disk_slot();
@@ -1528,7 +1567,7 @@ int blk_owner_has_claim(const uint8_t owner_fp[8]) {
if (slot_pbn_pack_offset(rel_pbn) != 0) continue; if (slot_pbn_pack_offset(rel_pbn) != 0) continue;
blk_meta_t meta; blk_meta_t meta;
if (blk_get_meta(lbn, &meta) != BLK_OK) continue; if (blk_get_meta_locked(lbn, &meta) != BLK_OK) continue;
if ((meta.flags & BLK_FLAG_CLAIMED) && if ((meta.flags & BLK_FLAG_CLAIMED) &&
memcmp(meta.owner_fp, owner_fp, 8) == 0) { memcmp(meta.owner_fp, owner_fp, 8) == 0) {
return 1; return 1;
@@ -1537,15 +1576,15 @@ int blk_owner_has_claim(const uint8_t owner_fp[8]) {
return 0; return 0;
} }
int blk_meta_relocate_devblock(uint32_t home_devblock, uint32_t target_devblock) { static int blk_meta_relocate_devblock_locked(uint32_t home_devblock, uint32_t target_devblock) {
if (home_devblock == target_devblock) return BLK_EINVAL; if (home_devblock == target_devblock) return BLK_EINVAL;
blk_meta_t home_meta; blk_meta_t home_meta;
int rc = blk_get_meta(home_devblock, &home_meta); int rc = blk_get_meta_locked(home_devblock, &home_meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
home_meta.flags |= BLK_FLAG_MIGRATING; home_meta.flags |= BLK_FLAG_MIGRATING;
if (blk_set_meta(home_devblock, &home_meta) != BLK_OK) return BLK_EIO; if (blk_set_meta_locked(home_devblock, &home_meta) != BLK_OK) return BLK_EIO;
/* Devblock granularity is BLK_PACK_RATIO FORTH blocks -- move each /* Devblock granularity is BLK_PACK_RATIO FORTH blocks -- move each
* one via the existing, only, FORTH-block-granularity relocation * one via the existing, only, FORTH-block-granularity relocation
@@ -1555,24 +1594,24 @@ int blk_meta_relocate_devblock(uint32_t home_devblock, uint32_t target_devblock)
* transferred -- a real, documented limitation, not silently * transferred -- a real, documented limitation, not silently
* dropped). */ * dropped). */
for (uint32_t i = 0; i < BLK_PACK_RATIO; i++) { for (uint32_t i = 0; i < BLK_PACK_RATIO; i++) {
rc = blk_subsys_relocate_block(home_devblock + i, target_devblock + i); rc = blk_subsys_relocate_block_locked(home_devblock + i, target_devblock + i);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
} }
blk_meta_t target_meta; blk_meta_t target_meta;
rc = blk_get_meta(target_devblock, &target_meta); rc = blk_get_meta_locked(target_devblock, &target_meta);
if (rc != BLK_OK) return rc; if (rc != BLK_OK) return rc;
memcpy(target_meta.owner_fp, home_meta.owner_fp, sizeof(target_meta.owner_fp)); memcpy(target_meta.owner_fp, home_meta.owner_fp, sizeof(target_meta.owner_fp));
target_meta.acl_allow = home_meta.acl_allow; target_meta.acl_allow = home_meta.acl_allow;
target_meta.acl_ttl = home_meta.acl_ttl; target_meta.acl_ttl = home_meta.acl_ttl;
target_meta.flags |= BLK_FLAG_CLAIMED; target_meta.flags |= BLK_FLAG_CLAIMED;
if (blk_set_meta(target_devblock, &target_meta) != BLK_OK) return BLK_EIO; if (blk_set_meta_locked(target_devblock, &target_meta) != BLK_OK) return BLK_EIO;
memset(home_meta.owner_fp, 0, sizeof(home_meta.owner_fp)); memset(home_meta.owner_fp, 0, sizeof(home_meta.owner_fp));
home_meta.acl_allow = 0; home_meta.acl_allow = 0;
home_meta.acl_ttl = 0; home_meta.acl_ttl = 0;
home_meta.flags &= ~(BLK_FLAG_CLAIMED | BLK_FLAG_MIGRATING); home_meta.flags &= ~(BLK_FLAG_CLAIMED | BLK_FLAG_MIGRATING);
return blk_set_meta(home_devblock, &home_meta); return blk_set_meta_locked(home_devblock, &home_meta);
} }
/* FABRIC-2.md §I.2, 2026-09-04: heat/wear-leveling migration trigger. /* FABRIC-2.md §I.2, 2026-09-04: heat/wear-leveling migration trigger.
@@ -1647,7 +1686,7 @@ static void blk_migration_idle_check_slot(blk_dev_slot_t *slot) {
uint32_t rel_pbn = lbn_to_slot_pbn(slot, lbn); uint32_t rel_pbn = lbn_to_slot_pbn(slot, lbn);
if (slot_pbn_pack_offset(rel_pbn) == 0) { if (slot_pbn_pack_offset(rel_pbn) == 0) {
blk_meta_t meta; blk_meta_t meta;
if (blk_get_meta(lbn, &meta) == BLK_OK) { if (blk_get_meta_locked(lbn, &meta) == BLK_OK) {
if (!have_hot && (meta.flags & BLK_FLAG_CLAIMED) && if (!have_hot && (meta.flags & BLK_FLAG_CLAIMED) &&
!(meta.flags & BLK_FLAG_MIGRATING) && !(meta.flags & BLK_FLAG_MIGRATING) &&
meta.write_count >= MIGRATION_WEAR_THRESHOLD) { meta.write_count >= MIGRATION_WEAR_THRESHOLD) {
@@ -1667,7 +1706,7 @@ static void blk_migration_idle_check_slot(blk_dev_slot_t *slot) {
slot->migration_scan_lbn = lbn; slot->migration_scan_lbn = lbn;
if (have_hot && have_free && hot_lbn != free_lbn) { if (have_hot && have_free && hot_lbn != free_lbn) {
(void) blk_meta_relocate_devblock(hot_lbn, free_lbn); (void) blk_meta_relocate_devblock_locked(hot_lbn, free_lbn);
} }
} }
@@ -1681,7 +1720,7 @@ static void blk_migration_idle_check_slot(blk_dev_slot_t *slot) {
* Artemis-disk special case bolted onto an otherwise device-agnostic * Artemis-disk special case bolted onto an otherwise device-agnostic
* file. Fixed to walk the full slot chain -- every attached device now * file. Fixed to walk the full slot chain -- every attached device now
* gets the same wear-leveling scan, none hardcoded, none skipped. */ * gets the same wear-leveling scan, none hardcoded, none skipped. */
void blk_migration_idle_check(void) { static void blk_migration_idle_check_locked(void) {
blk_dev_slot_t *slot; blk_dev_slot_t *slot;
for (slot = g.head; slot; slot = slot->next) { for (slot = g.head; slot; slot = slot->next) {
if (!slot->dev) continue; if (!slot->dev) continue;
@@ -1689,6 +1728,131 @@ void blk_migration_idle_check(void) {
} }
} }
/* ===== public API: each takes g_lock_block around its *_locked body =====
* (MC-0b2 -- see "locking" at the top of this file). */
int blk_subsys_init(VM *vm, uint8_t *ram_base, size_t ram_size) {
int rc; BLK_LOCK(); rc = blk_subsys_init_locked(vm, ram_base, ram_size); BLK_UNLOCK(); return rc;
}
int blk_subsys_add_raw_device(uint8_t *buf, uint32_t nblocks) {
int rc; BLK_LOCK(); rc = blk_subsys_add_raw_device_locked(buf, nblocks); BLK_UNLOCK(); return rc;
}
int blk_subsys_attach_device(struct blkio_dev *dev) {
int rc; BLK_LOCK(); rc = blk_subsys_attach_device_locked(dev); BLK_UNLOCK(); return rc;
}
int blk_subsys_detach_device(struct blkio_dev *dev) {
int rc; BLK_LOCK(); rc = blk_subsys_detach_device_locked(dev); BLK_UNLOCK(); return rc;
}
int blk_subsys_relocate_block(uint32_t home_lbn, uint32_t target_lbn) {
int rc; BLK_LOCK(); rc = blk_subsys_relocate_block_locked(home_lbn, target_lbn); BLK_UNLOCK(); return rc;
}
uint64_t blk_subsys_epoch(void) {
uint64_t e; BLK_LOCK(); e = g.epoch; BLK_UNLOCK(); return e;
}
int blk_subsys_shutdown(void) {
int rc; BLK_LOCK(); rc = blk_subsys_shutdown_locked(); BLK_UNLOCK(); return rc;
}
int blk_subsys_confirm_format(uint32_t lbn) {
int rc; BLK_LOCK(); rc = blk_subsys_confirm_format_locked(lbn); BLK_UNLOCK(); return rc;
}
uint8_t *blk_get_buffer(uint32_t block_num, int writable) {
uint8_t *p; BLK_LOCK(); p = blk_get_buffer_locked(block_num, writable); BLK_UNLOCK(); return p;
}
uint8_t *blk_get_empty_buffer(uint32_t block_num) {
uint8_t *p; BLK_LOCK(); p = blk_get_empty_buffer_locked(block_num); BLK_UNLOCK(); return p;
}
int blk_update(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_update_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_flush(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_flush_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_is_allocated(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_is_allocated_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_mark_allocated(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_mark_allocated_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_mark_free(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_mark_free_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_allocate(uint32_t *block_num) {
int rc; BLK_LOCK(); rc = blk_allocate_locked(block_num); BLK_UNLOCK(); return rc;
}
int blk_get_volume_meta(blk_volume_meta_t *meta) {
int rc; BLK_LOCK(); rc = blk_get_volume_meta_locked(meta); BLK_UNLOCK(); return rc;
}
int blk_set_volume_meta(const blk_volume_meta_t *meta) {
int rc; BLK_LOCK(); rc = blk_set_volume_meta_locked(meta); BLK_UNLOCK(); return rc;
}
int blk_get_device_range(struct blkio_dev *dev, uint32_t *out_start_lbn, uint32_t *out_count) {
int rc; BLK_LOCK(); rc = blk_get_device_range_locked(dev, out_start_lbn, out_count); BLK_UNLOCK(); return rc;
}
const void *blk_lbn_device_handle(uint32_t lbn) {
const void *h; BLK_LOCK(); h = blk_lbn_device_handle_locked(lbn); BLK_UNLOCK(); return h;
}
int blk_get_device_free_blocks(struct blkio_dev *dev, uint64_t *out_free, uint64_t *out_total) {
int rc; BLK_LOCK(); rc = blk_get_device_free_blocks_locked(dev, out_free, out_total); BLK_UNLOCK(); return rc;
}
int blk_get_first_disk_range(uint32_t *out_start_lbn, uint32_t *out_count) {
int rc; BLK_LOCK(); rc = blk_get_first_disk_range_locked(out_start_lbn, out_count); BLK_UNLOCK(); return rc;
}
int blk_meta_zone_read(uint32_t devblock_from_top, uint8_t buf[4096]) {
int rc; BLK_LOCK(); rc = blk_meta_zone_read_locked(devblock_from_top, buf); BLK_UNLOCK(); return rc;
}
int blk_meta_zone_write(uint32_t devblock_from_top, const uint8_t buf[4096]) {
int rc; BLK_LOCK(); rc = blk_meta_zone_write_locked(devblock_from_top, buf); BLK_UNLOCK(); return rc;
}
int blk_is_valid(uint32_t block_num) {
int rc; BLK_LOCK(); rc = blk_is_valid_locked(block_num); BLK_UNLOCK(); return rc;
}
uint32_t blk_get_total_blocks(void) {
uint64_t n; BLK_LOCK(); n = g.total_user_lbn; BLK_UNLOCK();
return (n > 0xFFFFFFFFu) ? 0xFFFFFFFFu : (uint32_t) n;
}
int blk_get_meta(uint32_t block_num, blk_meta_t *meta) {
int rc; BLK_LOCK(); rc = blk_get_meta_locked(block_num, meta); BLK_UNLOCK(); return rc;
}
int blk_set_meta(uint32_t block_num, const blk_meta_t *meta) {
int rc; BLK_LOCK(); rc = blk_set_meta_locked(block_num, meta); BLK_UNLOCK(); return rc;
}
int blk_owner_fp_get(uint32_t block_num, uint8_t out_fp[8]) {
int rc; BLK_LOCK(); rc = blk_owner_fp_get_locked(block_num, out_fp); BLK_UNLOCK(); return rc;
}
int blk_owner_fp_set(uint32_t block_num, const uint8_t fp[8]) {
int rc; BLK_LOCK(); rc = blk_owner_fp_set_locked(block_num, fp); BLK_UNLOCK(); return rc;
}
int blk_acl_allow_get(uint32_t block_num, uint8_t *out_allow) {
int rc; BLK_LOCK(); rc = blk_acl_allow_get_locked(block_num, out_allow); BLK_UNLOCK(); return rc;
}
int blk_acl_allow_set(uint32_t block_num, uint8_t allow) {
int rc; BLK_LOCK(); rc = blk_acl_allow_set_locked(block_num, allow); BLK_UNLOCK(); return rc;
}
int blk_acl_ttl_get(uint32_t block_num, uint32_t *out_ttl) {
int rc; BLK_LOCK(); rc = blk_acl_ttl_get_locked(block_num, out_ttl); BLK_UNLOCK(); return rc;
}
int blk_acl_ttl_set(uint32_t block_num, uint32_t ttl) {
int rc; BLK_LOCK(); rc = blk_acl_ttl_set_locked(block_num, ttl); BLK_UNLOCK(); return rc;
}
int blk_flags_get(uint32_t block_num, uint64_t *out_flags) {
int rc; BLK_LOCK(); rc = blk_flags_get_locked(block_num, out_flags); BLK_UNLOCK(); return rc;
}
int blk_flags_set(uint32_t block_num, uint64_t flags) {
int rc; BLK_LOCK(); rc = blk_flags_set_locked(block_num, flags); BLK_UNLOCK(); return rc;
}
int blk_firsttouch_claim(const uint8_t owner_fp[8], uint32_t count, uint32_t *out_chain_head) {
int rc; BLK_LOCK(); rc = blk_firsttouch_claim_locked(owner_fp, count, out_chain_head); BLK_UNLOCK(); return rc;
}
int blk_owner_has_claim(const uint8_t owner_fp[8]) {
int rc; BLK_LOCK(); rc = blk_owner_has_claim_locked(owner_fp); BLK_UNLOCK(); return rc;
}
int blk_meta_relocate_devblock(uint32_t home_devblock, uint32_t target_devblock) {
int rc; BLK_LOCK(); rc = blk_meta_relocate_devblock_locked(home_devblock, target_devblock); BLK_UNLOCK(); return rc;
}
void blk_migration_idle_check(void) {
BLK_LOCK(); blk_migration_idle_check_locked(); BLK_UNLOCK();
}
/* ===== weak hook (for main.c) ===== */ /* ===== weak hook (for main.c) ===== */
#if defined(__GNUC__) || defined(__clang__) #if defined(__GNUC__) || defined(__clang__)
__attribute__((weak)) __attribute__((weak))
+2
View File
@@ -35,6 +35,7 @@ sk_spinlock_t g_lock_switch = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_hermes = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_hermes = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_doe = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_doe = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_log = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_log = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_block = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_console = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_console = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_heap = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_heap = SK_SPINLOCK_INIT;
sk_spinlock_t g_lock_pmm = SK_SPINLOCK_INIT; sk_spinlock_t g_lock_pmm = SK_SPINLOCK_INIT;
@@ -183,6 +184,7 @@ static const char *lock_name(const volatile uint32_t *w)
if (w == &g_lock_hermes.owner) return "hermes"; if (w == &g_lock_hermes.owner) return "hermes";
if (w == &g_lock_doe.owner) return "doe"; if (w == &g_lock_doe.owner) return "doe";
if (w == &g_lock_log.owner) return "log"; if (w == &g_lock_log.owner) return "log";
if (w == &g_lock_block.owner) return "block";
if (w == &g_lock_console.owner) return "console"; if (w == &g_lock_console.owner) return "console";
if (w == &g_lock_heap.owner) return "heap"; if (w == &g_lock_heap.owner) return "heap";
if (w == &g_lock_pmm.owner) return "pmm"; if (w == &g_lock_pmm.owner) return "pmm";