9 Commits
Author SHA1 Message Date
Mikhail ChusavitinandClaude Sonnet 5 8cb250f3f4 fix(sat): fan check uses hottest GPU load + IPMI-hang-proof polling
- GPU load switches from bee-gpu-burn (compute burn, ~88% TDP) to
  dcgmproftester -t 1004 / targeted_power via
  resolveBenchmarkPowerLoadCommand — the same engine Power/Thermal Fit
  uses and the hottest sustained NVIDIA load we have, so fans are
  actually pushed toward their ceiling.
- Sample loop is now IPMI-hang-proof: every ipmitool read is time-boxed
  in an abandonable goroutine, and the poll interval backs off
  geometrically (1s→30s) when reads are slow, tightening again on
  recovery. A plateau is only trusted while telemetry is healthy;
  degraded runs ride out to MaxLoadSec. Summary gains fan_samples /
  telemetry_degraded. Drops the per-second nvidia-smi+power+cpu-temp
  sampling from the hot loop.
- Dead code removed: FanStressRow, GPUStressMetric, sampleFanStressRow,
  sampleGPUStressMetrics, WriteFanStressCSV/WriteFanSensorsCSV,
  analyzeMaxTemp, sampleSystemPowerResolved.

Topology fan tiles:
- size encodes the fan's ceiling RPM (its class), not current speed;
  coloured fill rising from the bottom encodes live duty cycle
  (current / ceiling), shown only when the ceiling was measured.
- glyph spin rate now maps absolute RPM into a human-perceptible band
  (fanSpinPeriodSec: 2.2s/turn at <=1000 RPM, 0.35s at >=13000).
- the "N fans · N OK · tile size ∝ …" caption line is gone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VHG21rgTUiR1G3qFHTVmN
2026-09-04 11:02:31 +03:00
Mikhail ChusavitinandClaude Sonnet 5 bb2a501a28 feat(sat): fan ceiling check + topology fan tiles
Repurpose the previously-unwired RunFanStressTest into RunFanCheck, a
Load-tier SAT test that drives stressapptest (CPU+memory) and, when a GPU
is present, a GPU burn to 100% simultaneously, then watches every fan
until none has climbed for ~60s. The observed peak RPM per fan is the
"ceiling"; it is persisted through the existing fan-observation store.

MSI G4201 / AMI MegaRAC exposes no host-side fan force (every OEM IPMI
command returns 0xc1; Redfish Thermal is GET-only), so load-driven ramp
is the closest safe equivalent. See
bible-local/decisions/2026-09-04-fan-ceiling-check.md.

- platform.ResolveFanMaxRPM: per-fan max with fallback (persisted peak ->
  peer peak -> current RPM), resolved in platform, not the view.
- platform.ErrTestNotApplicable: no load source or no fan sensors ->
  task lands as cancelled ("not applicable"), never failed, so an
  engineer never sees a false red. executeTaskWithOptions maps the
  sentinel; finalizeTaskForResult honours a pre-set TaskCancelled.
- Verdict FAIL only for a fan at 0 RPM / IPMI cr-nr under load.
- /topo: one small spinning square per fan, sized by RPM / resolved max,
  clickable through to a new "fan" component-detail type; per-fan status
  recorded to the component-status DB from the fan SAT summary.
- Wiring: /api/sat/fan/run route, "fan" task target, Load-page card,
  stress-mode Run All.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VHG21rgTUiR1G3qFHTVmN
2026-09-04 10:35:34 +03:00
Mikhail Chusavitin af95216c6f feat(tpm): add read-only TPM validation 2026-08-26 17:13:50 +03:00
Mikhail ChusavitinandClaude Sonnet 5 e036a9c7b8 boot: drop pci=realloc from default/toram GRUB entries, fold into troubleshooting entry
pci=realloc was forcing a full PCI resource reallocation on every default
and toram boot, added as a GPU BAR workaround for BIOS without Above 4G
Decoding. On a dense NVMe-backplane server it exhausted MMIO32 space and
silently dropped 4 of 6 NVMe controllers from the OS even though the BIOS
and lspci both saw them fine. It now only applies to the no-GUI/gsp-off
troubleshooting entry, alongside a bible-local ADR for the two prior
rounds of SAT/blackbox durability work.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-28 09:34:25 +03:00
Mikhail Chusavitin 518082c2e2 proposals: RFC for sfp_modules[] contract extension (v2.10 → v2.11) 2026-06-19 18:14:46 +03:00
Mikhail ChusavitinandClaude Sonnet 4.6 0b8a2ff83f Add validate test matrix and GPU test methodology docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 10:47:08 +03:00
Mikhail Chusavitin 11d00b9442 Document read-only submodules policy 2026-04-29 09:54:23 +03:00
Mikhail Chusavitin ab5a4be7ac Align hardware export with ingest contract 2026-03-15 21:04:53 +03:00
mchusandClaude Sonnet 4.6 871c766194 docs: add bible-local with architecture and decisions, fix PLAN.md versions
- bible-local/architecture/system-overview.md: scope, tech stack, key paths
- bible-local/architecture/runtime-flows.md: boot sequence, ISO build, collector flow
- bible-local/decisions/2026-03-05-nvidia-proprietary-driver.md
- PLAN.md: update KERNEL_VERSION 6.6→6.12, NVIDIA 550.54.15→590.48.01

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 18:15:07 +03:00