Mikhail ChusavitinandClaude Sonnet 5
6c8be629d2
The NVSwitch-detection fix (983f41a) was necessary but not sufficient on
HGX B200 (Kaytus KR9288-X3): ExecCondition now correctly runs fabricmanager,
but nvidia-fabricmanager-start.sh then aborts with "Kernel module ib_umad
has not been loaded" on this NVL5+ board, leaving the fabric stuck in
"In Progress" and cascading into failed dcgmi diag/NVBandwidth/NCCL/Validate
GPU SAT tasks (confirmed via support bundle 20260729-165331).
Neither nvidia-fabricmanager nor DCGM declare the NVLink5 support packages
as apt dependencies (checked against the cuda-repo Packages index directly):
ib_umad must be modprobed before FM starts, nvlsm (InfiniBand Subnet Manager)
is launched internally by FM's own start script once present, and DCGM
dlopens libnvidia-nscq at runtime for NVSwitch health monitoring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
37 lines
1.9 KiB
Plaintext
37 lines
1.9 KiB
Plaintext
# NVIDIA DCGM (Data Center GPU Manager).
|
|
# Validate uses dcgmi diagnostics; Burn uses dcgmproftester as the official
|
|
# NVIDIA max-compute recipe. The smoketest/runtime contract treats
|
|
# dcgmproftester as required in the LiveCD.
|
|
# DCGM 4 is packaged per CUDA major. The image ships NVIDIA driver 580 with
|
|
# CUDA 13 userspace, so install the CUDA 13 build plus proprietary components
|
|
# explicitly. datacenter-gpu-manager-4-core must be pinned too, not just the
|
|
# cuda13/proprietary variants that depend on it: without an explicit version,
|
|
# apt's solver picks the newest core in the repo and then reports the
|
|
# versioned deps as unmet instead of resolving to the pinned version.
|
|
nvtop
|
|
nvidia-fabricmanager=%%NVIDIA_FABRICMANAGER_VERSION%%
|
|
datacenter-gpu-manager-4-core=1:%%DCGM_VERSION%%
|
|
datacenter-gpu-manager-4-cuda13=1:%%DCGM_VERSION%%
|
|
datacenter-gpu-manager-4-proprietary=1:%%DCGM_VERSION%%
|
|
datacenter-gpu-manager-4-proprietary-cuda13=1:%%DCGM_VERSION%%
|
|
ocl-icd-libopencl1
|
|
clinfo
|
|
|
|
# 4th-gen NVSwitch (NVLink5) support for HGX/DGX B100/B200/B300. Neither
|
|
# nvidia-fabricmanager nor datacenter-gpu-manager declare these as apt
|
|
# dependencies (confirmed against the cuda-repo Packages index), so the
|
|
# fabricmanager start script and DCGM's NVSwitch backend silently fail
|
|
# without them:
|
|
# - libibumad3: userspace lib nvidia-fabricmanager-start.sh needs to poll
|
|
# the CX7 management NIC (mlx5_*) before training the NVSwitch fabric.
|
|
# Comes from Debian's rdma-core, not the cuda-repo.
|
|
# - nvlsm: InfiniBand-compliant Subnet Manager. FM's own start script
|
|
# launches it internally on NVL5+ systems once present — no separate
|
|
# systemd unit to enable. Versioned independently of the driver branch.
|
|
# - libnvidia-nscq: NVSwitch query library DCGM dlopens at runtime for
|
|
# `dcgmi diag`/health monitoring on NVSwitch systems; must match the
|
|
# driver branch like nvidia-fabricmanager does.
|
|
libibumad3
|
|
nvlsm
|
|
libnvidia-nscq=%%NVIDIA_FABRICMANAGER_VERSION%%
|