Mikhail ChusavitinandClaude Sonnet 5
b11018ac5e
NVIDIA GPUs deliberately downclock PCIe to Gen1 at idle for power saving, and applyPCIeLinkSpeedWarning fired on every idle collector pass regardless - since component-status DB records never downgrade (Record() only ever raises severity), one boot-time idle sample permanently pinned pcie:gpu:nvidia to Warning for the rest of the session even after every load-bearing GPU SAT test passed clean. Two prior fixes (nvidia-smi-sourced link speed, pcie_aspm=off boot flag) didn't hold up against this hardware/driver combination - see bible-local/decisions/2026-08-24-pcie-gpu-gen1-idle-warning.md for the full history. Rather than add a downgrade path, stop writing an unverified status in the first place: parseLspciDevice no longer calls applyPCIeLinkSpeedWarning on the idle path. LinkSpeed/MaxLinkSpeed stay populated as plain descriptive fields; only a verified-under-load caller may now turn them into a status verdict. Two new SAT targets provide that verified signal: - pcie-link (platform/pcie_link_check.go): forces every enabled PCIe device - not just GPUs - to retrain via the PCIe spec's Link Control "Retrain Link" bit, then compares the negotiated speed against the device's max. Covers NICs/HBAs/switches that have no bee-gpu-burn equivalent load tool. Classifies by PCI class code + vendor ID, not name substrings. Routes gpu_nvidia/gpu_amd/other sub-verdicts into their own component-status keys so a degraded NIC never reads as a GPU fault. - nvidia-pcie-bandwidth (platform/nvidia_pcie_bandwidth.go): drives real host<->device traffic via dcgmi diag -r nvbandwidth and resamples link speed immediately after, independent of nvbandwidth's own pass/fail. Both wired into the task queue/webui the same way as nvidia-config (routes, dispatch, priority, Validate page cards, Run All Check SAT). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bee — Project Bible
Project-specific architecture, decisions, and runtime contracts.
Generic engineering rules live in bible/rules/patterns/.
Files
| File | Contents |
|---|---|
architecture/system-overview.md |
What bee does, scope, tech stack |
architecture/runtime-flows.md |
Boot sequence, audit flow, service order |
docs/customer-gpu-test-methodology.md |
Customer-facing GPU PCIe Validate / Validate -> Stress test list |
docs/hardware-ingest-contract.md |
Current Reanimator hardware ingest JSON contract |
docs/validate-vs-burn.md |
Validate and Validate -> Stress hardware test policy |
decisions/ |
Architectural decision log, including read-only submodule policy |
proposals/ |
RFCs and contract change proposals for Reanimator Core |
Validate Test Matrix
Validate
- CPU check
lscpusensorsstress-ng
- Memory check
freetimeout <timeout_sec> memtesterfree
- NVMe storage check
nvme id-ctrlnvme smart-lognvme device-self-test
- SATA/SAS storage check
smartctl -H -Asmartctl -t short
- Basic NVIDIA GPU check
nvidia-smi -pm 1nvidia-smi -qdmidecode -t baseboarddmidecode -t systemdcgmi diag -r 2
- Inter-GPU communication check
all_reduce_perf
- GPU bandwidth check
dcgmi diag -r nvbandwidth(per CPU socket, then all selected GPUs, on multi-socket systems -- seedecisions/2026-07-27-nvbandwidth-per-socket-split.md)
Validate -> Stress
- Extended NVIDIA GPU check
nvidia-smi -pm 1nvidia-smi -qdmidecode -t baseboarddmidecode -t systemdcgmi diag -r 3
- NVIDIA targeted stress
nvidia-smi -pm 1nvidia-smi -qdcgmi diag -r targeted_stress
- NVIDIA targeted power
nvidia-smi -pm 1nvidia-smi -qdcgmi diag -r targeted_power
- NVIDIA pulse test
nvidia-smi -pm 1nvidia-smi -qdcgmi diag -r pulse_test
- Inter-GPU communication check
all_reduce_perf
- GPU bandwidth check
dcgmi diag -r nvbandwidth(per CPU socket, then all selected GPUs, on multi-socket systems -- seedecisions/2026-07-27-nvbandwidth-per-socket-split.md)