qemu_session.py: a failed check prints where the CPUs are (cpu_snapshot)
CI's Acceptance fails on aarch64 at BIRTH on every head since 58d0e887.
The birth takes more than 300 s there and does not reproduce here: about
11 s, unsigned. The heartbeat spiral was ruled out by an instrumented
scratch kernel.
Captain Bob ruled "Diagnose on CI". On a failed check, the session script
asks QEMU's monitor for the registers three times, a second apart, and
prints the program counters, so CI's next failure shows where the kernel
is and whether it moves. A monitor that does not answer does not stop
the run.
Test first: scripts/test_qemu_session_snapshot.sh, 5 checks; 3 failed
before. The register text was checked on real, paused QEMU for aarch64,
riscv64 and x86.
The evidence and the measurements are in logs/20261119-ci-snapshot. The
local aarch64 timing run is in logs/20261118-235613.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
+21
-1
@@ -349,13 +349,33 @@ def excerpt(near=None, after=20, tail=40):
|
||||
print(" ---", flush=True)
|
||||
|
||||
|
||||
def cpu_snapshot(times=3, apart=1.0):
|
||||
"""Where the CPUs are (Captain Bob, 2026-11-19: "Diagnose on CI"): QEMU's monitor asked for
|
||||
the registers `times` times, `apart` seconds apart, and the lines that name the program
|
||||
counter printed -- so a failed check's log shows where the kernel is, and whether it moves.
|
||||
A monitor that does not answer is said, and the run goes on."""
|
||||
for k in range(times):
|
||||
try:
|
||||
r = qcmd("human-monitor-command", **{"command-line": "info registers"})
|
||||
said = r.get("return", "") if isinstance(r, dict) else str(r)
|
||||
except Exception as e: # the monitor gone: the run is not stopped by it
|
||||
print(f" --- the CPUs, {k + 1} of {times}: the monitor did not answer ({e})", flush=True)
|
||||
return
|
||||
pcs = [l.strip() for l in said.splitlines() if re.search(r"\b(PC|RIP|EIP|pc)\s*[=:]?\s*[0-9a-fA-F]", l)]
|
||||
print(f" --- the CPUs, {k + 1} of {times}: " + (" | ".join(pcs) if pcs else said.strip()[:200]), flush=True)
|
||||
if k + 1 < times:
|
||||
time.sleep(apart)
|
||||
|
||||
|
||||
def check(name, cond, near=None):
|
||||
"""PASS or FAIL; a FAIL also prints the serial log where the check looked: after the first
|
||||
line matching `near` (a regex), when it is given, else the log's end."""
|
||||
line matching `near` (a regex), when it is given, else the log's end -- and where the CPUs
|
||||
are (cpu_snapshot)."""
|
||||
global ok
|
||||
print(f"{'PASS' if cond else 'FAIL'}: {name}", flush=True)
|
||||
if not cond:
|
||||
excerpt(near)
|
||||
cpu_snapshot()
|
||||
ok = ok and cond
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user