Fix xHCI heap corruption: allocate the Output Device Context full size at Address Device, never swap it (found at O-C)

xhci_cmd_address_device() sized the Output Device Context for Slot + EP0
(64 bytes); xhci_cmd_configure_endpoint() then allocated a bigger copy and
repointed DCBAA[slot_id]. The controller keeps the address it was given at
Address Device (QEMU caches it), so every bulk Endpoint Context was written
past the 64-byte block into the next kmalloc header -- on every USB
mass-storage attach, every ISA. After O-C's smoke campaign reshuffled the
heap, a kfree at thumbdrive unplug walked into it: #GP in coalesce_neighbors().

Found with temporary instrumentation (load base, code bytes at RIP, event
crumbs, a heap-list checker): the corrupt word was an EP Context (Bulk IN,
Running, MPS 1024) exactly DCI 3 x 32 bytes into the 64-byte context.

Now: XHCI_DEVICE_CTX_BYTES (32 contexts x 32 bytes) allocated once per
slot at Address Device, zeroed whole, never reallocated; Configure
Endpoint just fills it. Also ends the per-attach leak of the old context.

Verified: heap checker clean from boot; campaign -> O-A -> MINT -> unplug
-> re-attach passes with 0 exceptions; 3-ISA acceptance passes, parity and
dict_hash set unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Allan James
2026-09-25 10:57:43 -04:00
co-authored by Claude Opus 5.5
parent ee4c401d1c
commit df150e8626
28 changed files with 110993 additions and 24 deletions
+61
View File
@@ -738,3 +738,64 @@ Evidence is named **before** each stage runs.
- The relay and `BLK-ATTACH` flows are unchanged: mint → WIREBIND → `USE` → `3`, with the
ledger at the standard `held=2048 pulled=258048 returned=255263 consumed=737`.
- Idle log growth is 0, there are no faults, and QEMU exits cleanly.
- [x] **Fix, found at O-C: the xHCI driver corrupted the kernel heap on every USB mass-storage
attach** (2026-09-25). Captain Bob: *"1, investigate the crash first"*, then *"go ahead with
the xHCI fix"*.
**The symptom.** After O-C's 1-rep smoke campaign, the standard sequence halted with a `#GP`
(vector 13) right after `MINT`, when Bob's thumbdrive is unplugged:
- `logs/20260925-013706/amd64/`;
- reproduced as `-043257`, `-055533` and `-071858`.
The kernel was unchanged since O-B; O-C is capsule-only. A fresh boot passed
(`-025933`), and so did 28 births and kills with trivial jobs (`-041213`). Only a real
campaign followed by the unplug crashed.
**How it was found** (temporary instrumentation, all reverted):
- **Mapping the fault.** Printing the load base showed that the amd64 kernel is relocated by
a whole number of pages plus 0x440, and differently on every boot. RIP-to-symbol mapping
against the wrong base was the earlier dead end.
- **Code bytes at RIP.** Dumping them and matching them in the ELF located the fault exactly:
`kfree` → `coalesce_neighbors()`, at `mov %rdx,0x20(%rax)`, writing a free-list neighbour's
`prev` through a garbage pointer.
- **Event breadcrumbs** put it inside `blk_subsys_detach_device()`, between `blkio_close()`
and the end of the two `free()`s.
- **A heap-list checker, run at every event,** found the corruption already at boot, during
the first USB attach, Zuse's thumbdrive (`-084150`, `-084728`). One allocated block's
header `prev` field held `0x0400003600000001`.
**The defect.** That value is the first 8 bytes of an xHCI **Endpoint Context**: EP State
Running (1); EP Type 6 (Bulk IN), CErr 3, Max Packet Size 1024.
- The controller writes it at DCI 3 × 32 = 0x60 bytes into the device context.
- 0x3253060 − 0x60 = 0x3253000 is the start of the **64-byte Output Device Context** allocated
at Address Device, whose heap block ends at the damaged header, 0x3253040.
- `xhci_cmd_address_device()` sized that context for Slot + EP0 only.
- `xhci_cmd_configure_endpoint()` then allocated a bigger copy and repointed `DCBAA[slot_id]`.
- But the controller takes the Output Device Context address at Address Device and keeps it.
QEMU caches it and never re-reads DCBAA. So every bulk EP Context was written past the end
of the 64-byte block, into the next heap header, on every mass-storage attach, on every ISA.
- It is usually harmless in practice: the damaged header is never walked. The campaign's
churn moves the heap so that a later `kfree` at unplug walks straight into it.
- The old context also leaked on every attach.
**The fix** (`xhci.c`, `xhci.h`, `xhci_driver.h`):
- The Output Device Context is allocated **full size** at Address Device, once per slot:
`XHCI_DEVICE_CTX_BYTES` = 32 contexts × 32 bytes (CSZ = 0, the only mode this driver
implements).
- It is zeroed across its whole length and never swapped.
- Configure Endpoint no longer grows, copies or repoints it. It refuses a DCI past the context.
- That also ends the per-attach leak.
**Verification:**
- Instrumented, fixed kernel (`-090302`): **0** heap-checker reports at boot attach and
throughout. Then the full reproduction (smoke campaign, 6/6 trials `fail=0`, O-A, `MINT` →
unplug → re-attach → `WIREBIND` → `3`) passes with **0** exceptions.
- Clean tree, instrumentation reverted, 3-ISA acceptance (`logs/20260925-101854/amd64/`,
`-103145/aarch64/`, `-104448/riscv64/`): boot parity and the `dict_hash` set unchanged (546
words, `0x99b01134c3d3049c`), the O-A checks pass, mint → WIREBIND → `USE` → `3` with the
standard ledger, idle growth 0, no faults.
- These boots also carry O-C's capsules and serve as its standard sequence (next entry).
**Process note.** During the investigation, a `make clean` for amd64 was run while that
ISA's QEMU was up. It deleted the running QEMU's control sockets; that QEMU was stopped with
SIGTERM, since no QMP socket was left, and its log was discarded as flawed.