platform: split nvbandwidth SAT into per-socket + all-GPU passes

On multi-socket systems, run the NVIDIA bandwidth diagnostic once per
CPU socket before the all-GPU pass, so a crash confined to the all-GPU
run (with clean per-socket passes preceding it) isolates a cross-socket
peer-to-peer fault instead of leaving it conflated with a general
GPU/PCIe issue. Single-socket systems keep the original one-pass shape.

Also expand the support-bundle README with reference notes distilled
from a real analysis pass (BMC clock drift, "0/empty" tool output
meaning absent hardware rather than a fault, timestamp-matching before
assigning causality, and a normal-power-cycle SEL signature), plus a
step-by-step recipe for diagnosing an unexpected reboot/crash during a
specific test.

audit/internal/app/assets/README.md

@@ -40,14 +40,18 @@ diagnostic command output, and logging its own systemd services.
   drive in `JBOD`/`UBad` state cannot join a new virtual disk without first
   being converted (`set good force`).
 - **"GPU topology / NVLink health?"** →
-  `export/techdump/nvidia-smi-topo.txt` (which GPUs are NVLink-bonded to
-  which, and how many links), `nvidia-smi-nvlink-status.txt` (per-link
-  active/inactive — only present in bundles built after this capture was
-  added; older bundles only have the topo -m aggregate), `nvidia-smi-nvlink-errors.txt`
-  (replay/recovery/CRC error counters, should be zero). All lanes of a
-  bonded pair are expected to show active; even one `<inactive>` lane next
-  to otherwise-active ones is a real fault signature, not benign — "no
-  NVLink present" instead shows *all* lanes inactive.
+  `export/techdump/nvidia-smi-topo.txt` (`NV#` = bonded NVLink pair, `PIX`/
+  `NODE`/`SYS` = PCIe-only, no NVLink), `nvidia-smi-nvlink-status.txt`
+  (per-link active/inactive), `nvidia-smi-nvlink-errors.txt` (replay/
+  recovery/CRC counters, should be zero), and `bee-sat/nvidia-config-*/summary.txt`'s
+  `nvlink_pairs_checked` (how many pairs it found to check, not how many
+  passed). **Not every GPU config has NVLink bridges — check whether this
+  SKU/order is supposed to have them before calling their absence a fault**
+  (all-`PIX` topology, empty nvlink-status/errors files, and
+  `nvlink_pairs_checked=0` together mean "none detected," which is only a
+  problem if the config calls for NVLink). If NVLink is expected, one
+  `<inactive>` lane next to active ones on an otherwise-bonded pair is the
+  real fault signature — a degraded link, not an absent one.
 - **"What tasks were run from the web UI, in what order, with what
   result?"** → `export/tasks-state.json` is the index (id, target, status,
   timestamps, paths). Each task also has its own directory
@@ -60,6 +64,43 @@ diagnostic command output, and logging its own systemd services.
   release tags if you're checking whether a specific fix shipped in this
   build.
 
+## Signatures
+
+- `ipmitool-sel-time.txt` disagrees with `manifest.txt`'s `generated_at_utc`
+  by more than minutes → BMC RTC drifted → all `ipmitool-sel.txt` timestamps
+  unreliable; use record-ID order (hex counter, column 1), not printed date.
+- `storcli64` → `"Status": "Failure", "Description": "No Controller found"`
+  → no legacy MegaRAID controller present, not a drive/RAID fault. Cross-ref
+  `storcli2-show-all.json`'s `"Number of Controllers"` and `lspci`/
+  `nvme-list.json` for actual storage hardware. Same pattern as
+  `nvlink_pairs_checked=0` above.
+- `systemctl restart ... timed out` in a `bee-*.log` → implicates a SAT run
+  only if its timestamp is inside that run's `run_at_utc`/`started_at`–
+  `done_at` window in `tasks-state.json`; outside that window (e.g. boot
+  bring-up) it's unrelated.
+- SEL sequence `Power Supply Failure detected`/`AC lost` (all PSUs) →
+  `ACPI Legacy OFF` → `Chassis intrusion` → `Power Button pressed` →
+  `Legacy ON` → full power-cycle, not a failing PSU. Isolated failure
+  assertion with no OFF/ON bracket, or one PSU failing while siblings stay
+  healthy → real PSU fault.
+- Every conclusion needs a file/line/key citation. `Failure`/`Critical`/
+  `timeout`/`Error` matched without reading the surrounding context is not
+  a citation.
+
+## Diagnosing "it rebooted/crashed during test X"
+
+1. `tasks-state.json` → task stuck `pending`/`running`, or last
+   `bee-sat/<target>-*/` with no `summary.txt` = check running at crash time.
+2. That directory's `verbose.log` → last subprocess with no matching
+   `finish`/exit-code line = the trigger.
+3. `system/dmesg.txt` starting at uptime 0, ending after a few hundred
+   seconds = fresh-boot log captured after an unclean reset (confirms
+   unclean reboot; does not contain the crash itself).
+4. Diff against a step that passed cleanly → isolates what's specific to
+   the failing workload (e.g. GPU-to-GPU/NVLink traffic vs. per-GPU
+   compute-only) instead of a generic cause (thermal, power) that would
+   also hit the passing step.
+
 ## Top-level layout
 
 ```