sat: collect NVLink status/errors, fix nv-hostengine restart via systemd

Add NVLink port status (nvidia-smi nvlink -s), error counters
(nvidia-smi nvlink -e), and dcgmi nvlink status to the support bundle,
and enrich HardwarePCIeDevice entries with per-link telemetry.

Replace the raw nv-hostengine pkill/restart dance in bee-nvidia-load
with systemctl restart/start of nvidia-dcgm.service, and order
bee-nvidia.service Before= nvidia-dcgm.service and
nvidia-fabricmanager.service so modules/device nodes exist before
those units start.
This commit is contained in:
Mikhail Chusavitin
2026-07-07 10:21:56 +03:00
parent a98721b038
commit 33d6eee9cf
6 changed files with 339 additions and 81 deletions
+21
View File
@@ -176,6 +176,27 @@ if command -v nvidia-smi >/dev/null 2>&1; then
else
echo "nvidia-smi not found"
fi
`}},
{name: "system/nvidia-smi-nvlink-status.txt", cmd: []string{"sh", "-c", `
if command -v nvidia-smi >/dev/null 2>&1; then
nvidia-smi nvlink -s 2>&1 || true
else
echo "nvidia-smi not found"
fi
`}},
{name: "system/nvidia-smi-nvlink-errors.txt", cmd: []string{"sh", "-c", `
if command -v nvidia-smi >/dev/null 2>&1; then
nvidia-smi nvlink -e 2>&1 || true
else
echo "nvidia-smi not found"
fi
`}},
{name: "system/dcgmi-nvlink-status.txt", cmd: []string{"sh", "-c", `
if command -v dcgmi >/dev/null 2>&1; then
dcgmi nvlink --link-status 2>&1 || true
else
echo "dcgmi not found"
fi
`}},
{name: "system/systemctl-nvidia-units.txt", cmd: []string{"sh", "-c", `
if ! command -v systemctl >/dev/null 2>&1; then