refactor: harden diagnostics and consolidate runtime code
This commit is contained in:
@@ -263,6 +263,11 @@ bee-blackbox.service (separate process from bee-web/bee-audit)
|
||||
a time-derived staging path: two builds in the same wall-clock second (two
|
||||
operators, or an on-demand build racing the blackbox worker) then share one
|
||||
tree and one's deferred `os.RemoveAll` truncates the other's archive.
|
||||
- The resulting archive is also unique (`os.CreateTemp` with the readable
|
||||
support-bundle name as its prefix) and remains a hidden `.partial` from
|
||||
retention cleanup until an atomic rename publishes the closed `.tar.gz`.
|
||||
A timestamp-only output path lets same-second writers share one inode and
|
||||
can corrupt both gzip streams.
|
||||
|
||||
## NVIDIA SAT Web UI flow
|
||||
|
||||
@@ -306,3 +311,15 @@ Web UI: "Run All" button -> POST /api/sat/run-all
|
||||
- `RunNvidiaBandwidthPack` runs one all-GPU `nvbandwidth` pass in Validate; the
|
||||
per-NUMA-node matrix is Stress-tier only (`fullMatrix` arg). See
|
||||
`decisions/2026-08-31-nvbandwidth-validate-single-deep-matrix.md`.
|
||||
- `nvidia-config` validates the active-link count independently at both ends of
|
||||
every `NVx` pair reported by `nvidia-smi topo -m`; aggregate counts must not
|
||||
be compared with a per-endpoint `NVx` value. It also persists matching
|
||||
NVIDIA degraded-mode kernel lines in `06-dmesg-nvlink-degraded.log`.
|
||||
- Generic SAT jobs may attach an output validator. A validator runs only after
|
||||
exit status 0 and returns `OK`, `FAILED`, or `UNSUPPORTED`; its detail is
|
||||
persisted in both the job log and `<job>_detail` in `summary.txt`.
|
||||
- `nccl-tests` explicitly validates the pinned `all_reduce_perf` correctness
|
||||
summary and both `#wrong` columns, and cross-checks selected NVLink endpoints
|
||||
against the captured topology/status. `Avg bus bandwidth` is retained as a
|
||||
diagnostic value only: no model-independent bandwidth threshold is valid.
|
||||
NCCL `P2P` alone is not proof of NVLink because P2P can also use PCIe.
|
||||
|
||||
Reference in New Issue
Block a user