Mikhail ChusavitinandClaude Sonnet 5
6c8be629d2
The NVSwitch-detection fix (983f41a) was necessary but not sufficient on
HGX B200 (Kaytus KR9288-X3): ExecCondition now correctly runs fabricmanager,
but nvidia-fabricmanager-start.sh then aborts with "Kernel module ib_umad
has not been loaded" on this NVL5+ board, leaving the fabric stuck in
"In Progress" and cascading into failed dcgmi diag/NVBandwidth/NCCL/Validate
GPU SAT tasks (confirmed via support bundle 20260729-165331).
Neither nvidia-fabricmanager nor DCGM declare the NVLink5 support packages
as apt dependencies (checked against the cuda-repo Packages index directly):
ib_umad must be modprobed before FM starts, nvlsm (InfiniBand Subnet Manager)
is launched internally by FM's own start script once present, and DCGM
dlopens libnvidia-nscq at runtime for NVSwitch health monitoring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ISO Build
bee ISO is built inside a Debian 12 builder container via iso/builder/build-in-container.sh.
Requirements
- Docker Desktop or another Docker-compatible container runtime
- Privileged containers enabled
- Enough free disk space for builder cache, Debian live-build artifacts, NVIDIA driver cache, and CUDA userspace packages
Build On macOS
From the repository root:
sh iso/builder/build-in-container.sh
The script defaults to linux/amd64 builder containers, so it works on:
- Intel Mac
- Apple Silicon (
M1/M2/M3/M4) via Docker Desktop's Linux VM
You do not need to pass --platform manually for normal ISO builds.
Useful Options
Build with explicit SSH keys baked into the ISO:
sh iso/builder/build-in-container.sh --authorized-keys ~/.ssh/id_ed25519.pub
Force a clean rebuild of the builder image and build caches:
sh iso/builder/build-in-container.sh --clean-build
Use a custom cache directory:
sh iso/builder/build-in-container.sh --cache-dir /path/to/cache
Notes
- The builder image is automatically rebuilt if the local tag exists for the wrong architecture.
- The live ISO boots with Debian
live-boottoram, so the read-only medium is copied into RAM during boot and the runtime no longer depends on the original USB/BMC virtual media staying present. - Target systems need enough RAM for the full compressed live medium plus normal runtime overhead, or boot may fail before reaching the TUI.
- The NVIDIA variant installs DCGM 4 packages matched to the CUDA user-mode driver major version. For driver branch
580/ CUDA13.x, the package family isdatacenter-gpu-manager-4-cuda13rather than legacydatacenter-gpu-manager. - Override the container platform only if you know why:
BEE_BUILDER_PLATFORM=linux/amd64 sh iso/builder/build-in-container.sh
- The shipped ISO is still
amd64. - Output ISO artifacts are written under
dist/.