Fast Confidence Loop — Measurements Record¶

Home: docs/developer/fast-confidence-loop-measurements.md (plan Task 15). Every row carries its box; rows are never compared across boxes silently.

Measurement box reality (precedes every number): the spec's §1 baselines are labeled "GB300 dev box: 72 cores, 494 GB RAM", but ALL rows below were measured on the CURRENT box: 16 cores, MemTotal 65,744,332 kB = 62.69 GiB (/proc/meminfo, nproc, read 2026-09-14, corroborated by t3-draft/measurement-plan.md §"Box reality"). Targets written for the fatter box are stretch goals here and labeled as such per arm. §6.2's "< 10 min" is a fat-box target; on this box the parallel arms are OWNER-DEFERRED to a 96 GiB machine (ledger §9 release-event deferral row) — the serial control arm is this box's honest envelope.

M1-pre-plan datapoints (spec §9 / Task 8 scaffold rows)¶

Ledger-measured rows¶

Row B1 — backend integration tier wall-time (M3 T4c program)¶

Row B2 — combined backend full gate wall (-n auto; spec §6.1 target <30 min fat box)¶

Row B3 — combined coverage (gate is 80; bar NEVER moved — spec §7)¶

Row F1 — frontend full vitest suite (spec §6.2; <10 min = fat-box target)¶

Row F2 — fast-tier playbook (spec §6.3; 5 changes ≤10 min each)¶

scripts/fast-validation-playbook.sh (committed; the plan's LISTFILE GOTCHA corrected + dirty-tree setup guard added before the revert trap — see commit message). BASE = git merge-base HEAD origin/main (no local main in this repo).

Run 1 (pre-fix, BASE=merge-base): all five printed walls 7/8/73/73/68 s but the runs behind three of them were NOT real (vitest v5.0.0 at wrong root, Python-invalid probe, tee RC-mask — see runners' RECONCILIATION headers; evidence preserved /home/agent/gate19-prep/fcl-playbook-run1-logs/). Run 2 (post-fix, BASE=HEAD probe-only diffs) is the quotable one:

Summary: 5/5 within the spec §6.3 10-minute wall; playbook rc=0 (/tmp/fcl-playbook2.log; per-case logs trap-wiped at exit by design — the outer log carries the table). Fastest real case 7 s vs the ~11 min full backend gate and ~50 min full frontend suite — the §6.3 aim (≤10 min/change) met with ≥30× headroom on the backend cases.

Footer contract check (plan T13 Step 3, exercised by --fast after rows land): SELECTED: n files total + loud FAST TIER — no coverage proof; full gate still required before merge. + NO VALIDATION SUCCESSFUL string on the --fast path.

Row F3 — WP2.5 playbook: 7 transitive cases + measured bound (2026-09-16)¶

scripts/fast-validation-playbook.sh gains seven TRANSITIVE cases (plan asked ≥3) and the wall retargets 600 s → 900 s = the WP2.4 measured budget (p50 428 s / p95 513 s, n=20, ledger WP2.4 row; owner ruling tiered). Wall time alone cannot tell a closure selection from an accidental full run, so the new cases additionally assert WHICH files the selector chose (REQUIRED / FORBIDDEN / exact TOTAL) against the raw --list-out — before the tier filter. The backend RUN list is the same WP2.4 f4 tier filter the pre-push hook applies, so the playbook certifies the tier the hook ships; the two integration-tier proofs (x2/x4) assert on the raw selection and are excluded from the run by contract, not by accident.

Every REQUIRED was probe-verified against the shipped selector at 515a4810 (one behavior-neutral comment appended, --base HEAD, why-lines read). Two reconciliation calls, both from evidence: the plan-era draft probe for x1 (alert_service.py → test_alerts.py) reaches its target only through the routes-package __init__ hub, so it became its own case (x1b) labelled for what it actually proves, and event_broadcaster.py — whose hop to test_alerts runs through an UNTOUCHED intermediate — carries the x1 claim; a "narrower" x4 candidate (schemas/jobs → test_jobs.py) was rejected because test_jobs.py references schemas.jobs DIRECTLY (depth 0 masquerading as a chain proof).

Run history (the quotable run is the one from the SHIPPED script, whole):

• Run 1 (pre-defect-fixes, killed at 7/12): the record the three defect commits cite — route(alerts) 78 s rc=1 MANIFEST CANNOT-RUN naming the schemathesis stub (→ delete commit), the census it triggered (→ test_utils rename), and the deletion-planning blocker (→ deleted-test phantom fix). x1-intm green at 467 s/633 files — first proof the hub shape fits 900 s. Its verdict for x2 read "OK" while rc=1-era bugs hid a non-run: the rc-blind-fold lesson that put rc into the verdict line.
• Run 2 (defect fixes landed; 10/12 green): every executed case green with assertions folded — x1/x1b/x3/x5/x6 OK, incl. x6's FORBIDDEN prefixes and x3's TOTAL=1. x2/x4 died rc=2 BEFORE running: their names contain / and the log path $LOGDIR/$NAME.log resolved into a nonexistent directory (dash: cannot create ... Directory nonexistent) — zero signal, zero run. (This retroactively explains Run 1's same "0 s rc=2" row, misdiagnosed at the time as a pkill artifact.) Fix: log filenames go through tr / _ (the human table keeps the readable name); the in-file comment carries the failure quote as the guard's proof.
• Run 3 (post-fix, BASE=HEAD): the quotable table below.

Counts: raw = fast_select's own --list-out (assertions grade this list, pre-tier-filter; frozen at the 515a4810 probe), in-tier = the runner's SELECTED line after the f4 filter. The two −1 cells vs probe/Run-1 are the deleted schemathesis stub: it entered the selection ONLY when the diff touches backend/api/** (route in-tier 145→144 across runs; x4 285→284 vs probe) and every non-api case is bit-identical (x1-intm 633, x1b 145, x2 635, service 145 both runs) — the arithmetic corroboration that the directory-policy deletion removed exactly the defect and nothing else.

Summary: 12/12 cases green — every wall ≤ 900 s (max 484 s, the 674-file conftest-tree case; hub-class x1/x2 at 465 s), every assertion folded into its OK verdict, playbook rc=0 (/tmp/wp25/pb-run3.log; per-case logs trap-wiped at exit). Total battery wall 2092 s across 12 full tier runs — the WP2.4 bound certified against the mechanism it protects, not just its clock.

Row H1 — honesty acceptance (spec §6.4)¶

Row S1 — full-gate strictness byte-identity (spec §6.5; quote in PR)¶

• Quote for the PR, measured per-file at the M2 tip (e53c2b0e; docs commits after it touch none of these paths): git diff 1be83995..HEAD -- scripts/validate.sh pyproject.toml frontend/vite.config.ts frontend/package.json scripts/AGENTS.md → validate.sh +97/−1 · vite.config.ts +11/−3 · package.json 2/2 · AGENTS.md +9/−0 — pyproject.toml: ZERO delta (verified at T13 apply and again at this tip). 1be83995 = M1-final gate-19 tip; base anchor is the §6 record.
• Sanctioned delta 1 (T3): package.json NODE_OPTIONS wrapper — quote-style change only (2/2); effective NODE_OPTIONS identical with envs unset (control arm re-proved: wrapper fires --max-old-space-size=${VITEST_HEAP_MB:-8192}).
• Sanctioned delta 2 (T13): with --fast absent RUN_FAST stays false → zero new lines execute; --bogus still exits 1 with identical two lines (probe: /tmp/t13-bogus.log); --help shows the flag.
• First validate.sh --fast exercise (T13 Step 2, /tmp/t13-fast-probe.sh, VALIDATE_BASE=HEAD docs-only diff): rc=0; footer SELECTED: 0 files total (backend: 0 …, frontend: 0 …) + loud FAST TIER — no coverage proof; full gate still required before merge.; grep -c "VALIDATION SUCCESSFUL" = 0 on the fast-path log; tree clean after. A prior probe variant also captured the honest REAL-change path: run 2's hook case left useAlertsQuery.ts dirty (a race, recorded honestly) and fast tier fed exactly src/hooks/ useAlertsQuery.ts to related → 20 files / 672 tests, rc=0.

Row S2 — flake allowlist hygiene (spec §6.6)¶

M1 CI-blind-spot findings (Task 15 Step 2 — carried from M1 ledger)¶

1. profiler disabled in CI masked the pyroscope aarch64 crash class.
2. the frontend exit-0-on-first-summary hack hid the R-T7-VITEST class for months (hack still LIVE in ci.yml push job — T4-class fix; the nightly workflow deliberately does NOT copy it, header states so).
3. zero-byte tracked test files lived green on main (collection-sanity class; script + job landed c5de7d57; trio allowlisted honestly a4c061fd).

Nightly trend rows (T14 home)¶

GitHub registers workflows ONLY from the default branch (main); the nightly file lives on feat/context-map-2026-09-12 and returns 404 to dispatch until the branch lands on main (observed 2026-09-14, gh api). First row fills after merge: first 04:17 UTC schedule run, or manual dispatch post-merge.