plan: a CAPA & FMEA gate in the build pipelines -- for ruling, nothing built
Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines", then yes to writing it up. A nightly.sh quality stage run in the acceptance gate: every kind/escaped issue has a CAPA record whose preventive control exists and runs in a stage; every FMEA row over the threshold has a control in the gate. Seeded with five CAPAs from 2026-11-11. Five points to rule (section 7). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RzZgrf7V7Zk2iczBXDRbm
This commit is contained in:
@@ -0,0 +1,96 @@
|
||||
# A CAPA & FMEA gate in the build pipelines -- plan, for ruling
|
||||
|
||||
Captain Bob, 2026-11-11: "I'm thinking about adding a CAPA & FMEA GATE to our build pipelines" --
|
||||
then "Yes" to writing it up. **Nothing here is built.** Every choice below marked *To rule* waits for
|
||||
Captain Bob; the rest are my calls, open to a ruling. Decide on paper first, then build in small
|
||||
increments, each tested first and seen to fail.
|
||||
|
||||
## 1. What it is for
|
||||
|
||||
The gate (`scripts/nightly.sh` stages, the same by hand, in a PR and in the nightly) proves the
|
||||
system works. It does not prove that **a fault which got past it once cannot get past it again**,
|
||||
nor that **every known way the system can fail has something that would see it**. Silent failure is
|
||||
this project's dominant failure mode; this gate is aimed at it.
|
||||
|
||||
- **CAPA** (Corrective and Preventive Action): a record for each *escaped* defect -- a fault the gate
|
||||
that should have caught it let through. The correction fixes this one; the **preventive control**
|
||||
is a check, wired into a stage, that would have caught it.
|
||||
- **FMEA** (Failure Modes and Effects Analysis): a table of the ways each stage or subsystem can
|
||||
fail -- effect, cause, Severity, Occurrence, Detection (1-10 each), their product the RPN -- and
|
||||
the **control** that detects each. Detection is the project's rule "name, before running, the
|
||||
observation that proves it worked", made a number.
|
||||
|
||||
## 2. The gate
|
||||
|
||||
A new stage, `scripts/nightly.sh quality`, run in `acceptance.yml` (so in the PR into `master`), the
|
||||
nightly and the release. It fails when:
|
||||
|
||||
1. an issue labelled **`kind/escaped`** has no CAPA record, or its record's preventive control does
|
||||
not exist;
|
||||
2. a CAPA's control is not run by any stage -- including one that stopped running (a stage renamed
|
||||
or removed);
|
||||
3. an FMEA row with RPN at or over the threshold has no control in a gate stage;
|
||||
4. a record is malformed (missing fields, scores out of range, a control naming nothing).
|
||||
|
||||
It checks the records against the tree; it does not judge them. Whether a root cause is the real one
|
||||
is Captain Bob's ruling, in the PR.
|
||||
|
||||
## 3. The records
|
||||
|
||||
*To rule:* format. My call: **YAML** under `docs/quality/` -- `capa/CAPA-NNNN.yml`, one per escaped
|
||||
defect, and `fmea.yml` -- because the gate must read them; a Markdown view generated from them for
|
||||
the Book of Fabric. A control is named as `<stage>:<check>`, e.g. `accept:put-backs-tracked`, and
|
||||
each check is a named function in the stage's script, so the gate can find it.
|
||||
|
||||
# CAPA-0001.yml
|
||||
id: CAPA-0001
|
||||
issue: 168 # the kind/escaped issue
|
||||
found: 2026-11-11
|
||||
escaped: acceptance # the gate it got past
|
||||
defect: nightly.sh images stops: git checkout names the untracked version.h
|
||||
root_cause: untracking a file without searching for every use; acceptance does not run images
|
||||
correction: 9415d7ed
|
||||
prevention:
|
||||
control: accept:put-backs-tracked
|
||||
added_in: <commit>
|
||||
seen_to_fail: <log> # the control run against the defect, failing
|
||||
|
||||
## 4. Seeded from 2026-11-11 (to rule: which become CAPAs)
|
||||
|
||||
| CAPA | Escaped | Root cause | Correction | Preventive control (proposed) |
|
||||
|---|---|---|---|---|
|
||||
| 0001 | acceptance | `version.h` untracked while six put-backs still named it; acceptance does not run `images` | `9415d7ed` | every path in a put-back `git checkout` is tracked (or `images` in the gate) |
|
||||
| 0002 | every gate, 3 months | LFS pointers committed without their objects (`a5ed8c3d`); nothing checks the server | `aec7ddb5` | `git lfs fsck` and every pointer's object present on the server |
|
||||
| 0003 | the rules | acceptance run in the background; a rule nothing enforces | stopped, rerun | `qemu_session.py` refuses to run without a terminal (TTY) |
|
||||
| 0004 | CI | a shared concurrency group cancels the waiting run (#140) | `88d5f777` | no run in the last N "cancelled after 0s" |
|
||||
| 0005 | acceptance | 146 logs committed as plain text, the LFS filter off | `06f1d41a` | every path `.gitattributes` gives to LFS is a pointer |
|
||||
|
||||
The code-review findings (`.claude/memory/code-review.md`: MINT-ZUSE's false "nothing changed",
|
||||
`manage.c`'s missing depth checks, the descriptor loop, ...) are FMEA rows, not CAPAs: none escaped a
|
||||
gate into a fault seen in use.
|
||||
|
||||
## 5. The scales (to rule)
|
||||
|
||||
My call, 1-10 each: **Severity** 10 = data lost or a stick erased silently, 7 = a release or the
|
||||
nightly fails, 4 = a wrong message, 1 = cosmetic. **Occurrence** from the tracker's history, not
|
||||
guessed. **Detection** 10 = nothing would see it, 1 = a gate stage fails on it every time.
|
||||
Threshold: RPN >= 100, or Severity >= 9 at any RPN.
|
||||
|
||||
## 6. Increments (once ruled)
|
||||
|
||||
1. The record formats and `nightly.sh quality` checking them (malformed refused) -- test first.
|
||||
2. CAPA-0001..0005 and their controls, each control seen to fail against its defect.
|
||||
3. `fmea.yml` seeded: the acceptance stages, the stick, LFS, CI; rows from the review findings.
|
||||
4. `quality` in `acceptance.yml`, the nightly and the release; the Markdown view in the Book.
|
||||
5. The `kind/escaped` label; the issue template asks for it.
|
||||
|
||||
## 7. To rule
|
||||
|
||||
1. Scope: CAPA for escaped defects only (my call), or every defect?
|
||||
2. Format: YAML the gate checks (my call), or Markdown read by a person?
|
||||
3. Where: a stage in `acceptance.yml` (my call), or a separate required check?
|
||||
4. The scales and the threshold (section 5).
|
||||
5. Which of section 4's five become CAPAs.
|
||||
|
||||
Don't over-engineer: start with the escaped defects and an FMEA grown as defects arrive, not a
|
||||
large table written up front.
|
||||
Reference in New Issue
Block a user