# PCIe link verdict comes from real traffic, not forced retraining

**Date:** 2026-09-03
**Status:** active

## Evidence

On `bee@172.16.41.97`, build `14.02-1-g642e686`, three independent runs of
`PCIe Link Check (forced retrain)` reported all eight NVIDIA GPUs and their
eight upstream bridges at Gen1 x16 after retraining, against Gen5 x16
capability. In the same application history, every `dcgmi diag -r
nvbandwidth` run brought every GPU to Gen5 x16 and passed. There were no AER
or NVIDIA Xid events supporting a physical link fault.

This disproves the premise that setting the PCIe Retrain Link bit is a
generic substitute for device traffic. On this hardware and driver, a link
can complete a forced retrain and remain in its idle Gen1 state. Counting both
the endpoint and its upstream bridge also reports one physical link twice.

## Decision

- Remove the standalone `pcie-link` forced-retrain SAT from code, API, UI,
  and Run All planning.
- Remove the separate `nvidia-pcie-bandwidth` SAT. It duplicated the same
  approximately 45-minute `nvbandwidth` workload already run by
  `nvidia-bandwidth`.
- Extend `nvidia-bandwidth` to record a sysfs baseline before the test and
  sample negotiated speed and width synchronously when the final all-GPU
  `dcgmi diag -r nvbandwidth` command exits. The sample runs before later
  health probes and artifact processing can let the link return to idle.
- Append `pcie_link_under_load_status`, per-GPU PCIe status, and the degraded
  count to the existing summary. A supported GPU that is still below its own
  maximum speed or width after real traffic fails the overall SAT. Missing
  sysfs attributes produce `UNSUPPORTED`, not a false failure.
- Keep non-GPU PCIe inspection read-only and diagnostic. Without a
  device-specific traffic generator, current speed is inventory rather than
  a health verdict.
- When a new `nvidia-bandwidth` result is recorded, recompute the NVIDIA GPU
  component status while ignoring the retired `sat:pcie-link` and
  `sat:nvidia-pcie-bandwidth` sources. Their history remains auditable. Other
  failures, such as `nvidia-config` or Xid-derived warnings, remain sticky.

## Consequences

Run All performs one fewer short but misleading test and one fewer duplicate
long GPU bandwidth test. A Gen1 idle baseline no longer alarms when the same
link reaches Gen5 x16 under traffic. A genuine under-load speed or width
downgrade still fails the bandwidth SAT and reaches `pcie:gpu:nvidia`.

Old task directories and component history remain readable. The removed API
targets are no longer accepted for new tasks.