Files

T

mchusandClaude Opus 4.8 a9924b08b0 feat(scenario): add nvbandwidth-acs-ab, an ACS off/on A/B for cross-socket P2P

Replaces the old nvbandwidth-all-gpu-power-watch scenario (dropped: the
power/IPMI-sampler framing was a dead-end — the blackbox shows every other
load test, incl. NCCL all-reduce across all 6 GPUs, passes with power fully
connected; only nvbandwidth on cross-socket GPU P2P reboots the box).

nvbandwidth-acs-ab is a two-phase reproduction for the MSI CG480-S6053 case.
Phase 1 disables PCIe ACS redirect at runtime via setpci (saving each
bridge's original ACS control word to /run/bee-acs-orig) then runs
nvbandwidth across all 6 GPUs — expected to survive. Phase 2 restores the
saved ACS values and repeats — expected to reproduce the reboot. Safe phase
first so its full blackbox reaches the USB stick before the risky phase can
reset the box (sync-bracket before the final job flushes phase-1 evidence).

ACS mutation runs via `bash -c` since scenario cmd is exec'd directly with no
shell. Both shipped copies (repo scenarios/ and image overlay) identical;
verified both parse via ParseScenarioJSON (4 jobs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-29 13:48:05 +03:00

nvbandwidth-acs-ab.json

feat(scenario): add nvbandwidth-acs-ab, an ACS off/on A/B for cross-socket P2P

2026-07-29 13:48:05 +03:00

README.md

platform/app/webui: ship the power-watch scenario baked into the image

2026-07-28 19:10:35 +03:00

README.md

bee test scenarios

A scenario is a plain JSON file describing an ad-hoc diagnostic run: which commands to execute (sequentially or in parallel) and what to sample in the background while they run — without hardcoding a new test into bee's own code. See audit/internal/platform/scenario.go (ParseScenarioJSON, ScenarioSpec) for the full field reference.

Running one

bee run <path-to-file.json>

or, for a bare name (no /, no .json):

bee run <name>

which resolves scenarios/<name>.json by checking, in order: (1) the scenarios shipped with the image itself — always available, no media needed — then (2) any mounted removable drive (e.g. the same USB stick already plugged in for blackbox), for scenarios authored on another machine and dropped there without a network path onto the host. Same resolution powers the "6. Scenario" page in the web UI.

bee scenario run <arg> is the same command under a longer name.

Files checked in here

nvbandwidth-all-gpu-power-watch.json — the scenario that reproduced the CG480-S6053 reboot: full nvbandwidth across all GPUs at once (the per-GPU-socket passes alone never reproduced it), with ipmitool sensor and nvidia-smi power/temp sampled every 2s in the background so a crash mid-run still leaves telemetry to check for a power-delivery correlation. gpu_indices is host-specific — update it to match the GPU indices nvidia-smi -L actually reports on the box under test before running.

This directory is the source of truth; iso/builder/build.sh's "preparing staged overlay" step rsyncs the whole overlay tree — including a checked-in copy under iso/overlay/usr/share/bee/scenarios/ — into the built image, so anything meant to ship baked-in needs to exist in both places (this repo doesn't auto-copy one into the other). platform.LocalScenariosDir (/usr/share/bee/scenarios on the built host) is what bee run <name>/the web UI actually reads at runtime.

Adding more

Not every scenario needs to ship in the image. For a one-off test on a specific host (especially air-gapped), it's simpler to write the JSON file directly onto the blackbox USB stick under scenarios/<name>.json and run bee run <name> — no code change, no rebuild. Check a scenario into iso/overlay/usr/share/bee/scenarios/ only when it's worth keeping around as a reusable/named test that should be available on every host without a USB stick.