Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr review? That way the items are forced into a manual review beforehand?" -- "Yes, add it to the plan". Each FMEA row names its paths; the PR carries a review record with a decision for every row its diff touches; a check_fmea_review.sh beside check_protocol.sh refuses the PR without one, or with one for another diff. A sixth point to rule. Nothing built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
7.4 KiB
A CAPA & FMEA gate in the build pipelines -- plan, for ruling
Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines" -- then "Yes" to writing it up. Nothing here is built. Every choice below marked To rule waits for Captain Bob; the rest are my calls, open to a ruling. Decide on paper first, then build in small increments, each tested first and seen to fail.
1. What it is for
The gate (scripts/nightly.sh stages, the same by hand, in a PR and in the nightly) proves the
system works. It does not prove that a fault which got past it once cannot get past it again,
nor that every known way the system can fail has something that would see it. Silent failure is
this project's dominant failure mode; this gate is aimed at it.
- CAPA (Corrective and Preventive Action): a record for each escaped defect -- a fault the gate that should have caught it let through. The correction fixes this one; the preventive control is a check, wired into a stage, that would have caught it.
- FMEA (Failure Modes and Effects Analysis): a table of the ways each stage or subsystem can fail -- effect, cause, Severity, Occurrence, Detection (1-10 each), their product the RPN -- and the control that detects each. Detection is the project's rule "name, before running, the observation that proves it worked", made a number.
2. The gate
A new stage, scripts/nightly.sh quality, run in acceptance.yml (so in the PR into master), the
nightly and the release. It fails when:
- an issue labelled
kind/escapedhas no CAPA record, or its record's preventive control does not exist; - a CAPA's control is not run by any stage -- including one that stopped running (a stage renamed or removed);
- an FMEA row with RPN at or over the threshold has no control in a gate stage;
- a record is malformed (missing fields, scores out of range, a control naming nothing).
It checks the records against the tree; it does not judge them. Whether a root cause is the real one is Captain Bob's ruling, in the PR.
3. The records
To rule: format. My call: YAML under docs/quality/ -- capa/CAPA-NNNN.yml, one per escaped
defect, and fmea.yml -- because the gate must read them; a Markdown view generated from them for
the Book of Fabric. A control is named as <stage>:<check>, e.g. accept:put-backs-tracked, and
each check is a named function in the stage's script, so the gate can find it.
# CAPA-0001.yml
id: CAPA-0001
issue: 168 # the kind/escaped issue
found: 2026-11-11
escaped: acceptance # the gate it got past
defect: nightly.sh images stops: git checkout names the untracked version.h
root_cause: untracking a file without searching for every use; acceptance does not run images
correction: 9415d7ed
prevention:
control: accept:put-backs-tracked
added_in: <commit>
seen_to_fail: <log> # the control run against the defect, failing
4. Seeded from 2026-11-11 (to rule: which become CAPAs)
| CAPA | Escaped | Root cause | Correction | Preventive control (proposed) |
|---|---|---|---|---|
| 0001 | acceptance | version.h untracked while six put-backs still named it; acceptance does not run images |
9415d7ed |
every path in a put-back git checkout is tracked (or images in the gate) |
| 0002 | every gate, 3 months | LFS pointers committed without their objects (a5ed8c3d); nothing checks the server |
aec7ddb5 |
git lfs fsck and every pointer's object present on the server |
| 0003 | the rules | acceptance run in the background; a rule nothing enforces | stopped, rerun | qemu_session.py refuses to run without a terminal (TTY) |
| 0004 | CI | a shared concurrency group cancels the waiting run (#140) | 88d5f777 |
no run in the last N "cancelled after 0s" |
| 0005 | acceptance | 146 logs committed as plain text, the LFS filter off | 06f1d41a |
every path .gitattributes gives to LFS is a pointer |
The code-review findings (.claude/memory/code-review.md: MINT-ZUSE's false "nothing changed",
manage.c's missing depth checks, the descriptor loop, ...) are FMEA rows, not CAPAs: none escaped a
gate into a fault seen in use.
5. The scales (to rule)
My call, 1-10 each: Severity 10 = data lost or a stick erased silently, 7 = a release or the nightly fails, 4 = a wrong message, 1 = cosmetic. Occurrence from the tracker's history, not guessed. Detection 10 = nothing would see it, 1 = a gate stage fails on it every time. Threshold: RPN >= 100, or Severity >= 9 at any RPN.
6. Increments (once ruled)
- The record formats and
nightly.sh qualitychecking them (malformed refused) -- test first. - CAPA-0001..0005 and their controls, each control seen to fail against its defect.
fmea.ymlseeded: the acceptance stages, the stick, LFS, CI; rows from the review findings.qualityinacceptance.yml, the nightly and the release; the Markdown view in the Book.- The
kind/escapedlabel; the issue template asks for it.
7. To rule
- Scope: CAPA for escaped defects only (my call), or every defect?
- Format: YAML the gate checks (my call), or Markdown read by a person?
- Where: a stage in
acceptance.yml(my call), or a separate required check? - The scales and the threshold (section 5).
- Which of section 4's five become CAPAs.
- The FMEA in the PR review (section 8), or not.
Don't over-engineer: start with the escaped defects and an FMEA grown as defects arrive, not a large table written up front.
8. The FMEA in the PR review: a manual review forced before the merge
Captain Bob, 2026-11-12: "Is it possible to include the FMEA in the pr review? That way the items are forced into a manual review beforehand?" -- then "Yes, add it to the plan". As with the protocol's evidence (#135), the review is a record committed in the PR, and the gate refuses the PR without it.
- Each FMEA row names the paths it concerns (
paths:infmea.yml), narrowly: the MINT-ZUSE rowkernel/src/v4/sk_v4_mint.c, the put-back rowscripts/nightly.shand the scripts it lists. - The PR carries
docs/quality/reviews/<TS>.yml: for every row whose paths the PR's diff againstmastertouches -- the row's id; the decision (scores unchanged / raised / lowered, a control added); one line of why; and any failure mode the change brings, as a new row infmea.yml. - The gate:
scripts/check_fmea_review.shinacceptance.yml, besidecheck_protocol.sh, refuses the PR when a touched row has no entry, an entry names a row that does not exist, or the review was made for another diff (it names the base and head it reviewed, as PROTOCOL.md names its commit). It prints exactly which rows the PR triggers, so the review is of those and no more. - The PR template gets an "FMEA review" section pointing at the file: the review is done before the PR is opened, not after.
- What it forces, and what it cannot: that every affected row was looked at and a decision written down. Whether the thinking was honest stays a person's: Captain Bob's while there is one developer; with a second, branch protection's required approval as well, the record telling the reviewer what to check.
- The risk: rows mapped too coarsely make every PR touch dozens, and the review becomes ticking boxes. Rows are mapped narrowly, and a row that triggers on most PRs is split.
To rule: a sixth point -- the FMEA review in the PR, as above, or not.