Mikhail ChusavitinandClaude Sonnet 5
bb2a501a28
Repurpose the previously-unwired RunFanStressTest into RunFanCheck, a
Load-tier SAT test that drives stressapptest (CPU+memory) and, when a GPU
is present, a GPU burn to 100% simultaneously, then watches every fan
until none has climbed for ~60s. The observed peak RPM per fan is the
"ceiling"; it is persisted through the existing fan-observation store.
MSI G4201 / AMI MegaRAC exposes no host-side fan force (every OEM IPMI
command returns 0xc1; Redfish Thermal is GET-only), so load-driven ramp
is the closest safe equivalent. See
bible-local/decisions/2026-09-04-fan-ceiling-check.md.
- platform.ResolveFanMaxRPM: per-fan max with fallback (persisted peak ->
peer peak -> current RPM), resolved in platform, not the view.
- platform.ErrTestNotApplicable: no load source or no fan sensors ->
task lands as cancelled ("not applicable"), never failed, so an
engineer never sees a false red. executeTaskWithOptions maps the
sentinel; finalizeTaskForResult honours a pre-set TaskCancelled.
- Verdict FAIL only for a fan at 0 RPM / IPMI cr-nr under load.
- /topo: one small spinning square per fan, sized by RPM / resolved max,
clickable through to a new "fan" component-detail type; per-fan status
recorded to the component-status DB from the fan SAT summary.
- Wiring: /api/sat/fan/run route, "fan" task target, Load-page card,
stress-mode Run All.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VHG21rgTUiR1G3qFHTVmN
69 lines
3.2 KiB
Markdown
69 lines
3.2 KiB
Markdown
# Decision: Fan check discovers the RPM ceiling by CPU+GPU load, not by forcing fans over the BMC
|
||
|
||
**Date:** 2026-09-04
|
||
**Status:** active
|
||
|
||
## Context
|
||
|
||
We want a SAT test that pushes every system fan to its top speed, records that
|
||
speed as the fan's ceiling, and flags a fan that will not spin. The recorded
|
||
ceiling is what the Topology view sizes each fan tile against
|
||
(`platform.ResolveFanMaxRPM`).
|
||
|
||
The obvious approach — force the fans to 100% PWM over IPMI/Redfish and read the
|
||
resulting RPM — does not work on our current platform:
|
||
|
||
- Stand: MSI G4201 / MS-S3831, AMI MegaRAC BMC (fw 1.08).
|
||
- Every documented host-side fan-control OEM command returns
|
||
`rsp=0xc1 Invalid command`: Supermicro `0x30 0x45`, ASRock/AMI reference
|
||
`0x3a 0x01 / 0xd0 0x12 / 0xd0 0x0f / 0xd6 / 0xd7 / 0xda`, `0x30 0x30..0x32`.
|
||
BIOS `KCS Access Control Policy = Allow All`, so this is not KCS filtering —
|
||
MSI simply does not implement them.
|
||
- The MSI G4201 Redfish API guide documents Thermal as **GET only**; no
|
||
fan-mode / fan-PWM PATCH endpoint exists.
|
||
- The only fan knob in BIOS is `Fan PWM Offset` (0–100, additive to the auto
|
||
curve, reboot-gated) — not a runtime 100% force.
|
||
|
||
## Decision
|
||
|
||
The `fan` SAT test (`platform.RunFanCheck`, formerly the unwired
|
||
`RunFanStressTest`) drives load, not the BMC:
|
||
|
||
1. Start `stressapptest` (CPU + memory) and, when a GPU is present, a GPU burn
|
||
(`bee-gpu-burn` / `rvs gst`) **simultaneously**, each in its own goroutine.
|
||
A missing GPU is not an error — CPU/memory load alone exercises the cooling
|
||
loop. Load sources reach full power at different times, so the plateau clock
|
||
only starts once every launched source reports its process running (plus a
|
||
fixed GPU ramp grace).
|
||
2. Sample every fan once a second. Per fan, track the peak RPM and the last time
|
||
it climbed by more than `PlateauDeltaRPM` (default 50).
|
||
3. When no fan has climbed for `PlateauHoldSec` (default 60 s) and at least one
|
||
fan rose meaningfully above baseline, declare the ceiling found and stop —
|
||
**success**. `MaxLoadSec` (default 900 s) is a hard cap; hitting it is also
|
||
success (the highest RPM seen is still recorded).
|
||
4. Persist each peak through the existing `updateFanObservation` path
|
||
(`/var/log/bee-sat/fan-observation.json`), which is what `ResolveFanMaxRPM`
|
||
reads.
|
||
|
||
## Verdict mapping
|
||
|
||
- A fan reading **0 RPM**, or IPMI status **cr/nr**, while under full load →
|
||
`FAILED` (dead / stuck rotor).
|
||
- No load source available (no `stressapptest`/`stress-ng` and no GPU burn
|
||
tool), or no fan sensors at all → `platform.ErrTestNotApplicable`, which the
|
||
task layer lands as **cancelled ("not applicable")**, never failed, with a
|
||
detailed log. An engineer must not see a red for "this platform can't run the
|
||
test".
|
||
|
||
## Tier
|
||
|
||
Load only (`/load`, "3. Load"). It is a sustained full-load test, so it does
|
||
not belong on the read-only Check page. Added to the stress-mode `Run All`.
|
||
|
||
## Consequences
|
||
|
||
- The ceiling is *observed*, not a spec value — only as high as the load drove
|
||
the fans. Good enough for tile sizing and stuck-fan detection.
|
||
- If a future platform does expose a safe host-side fan force, revisit: a real
|
||
100% force is a stronger test than load-driven ramp.
|