fix(pcie): verify GPU links under real bandwidth load

audit/internal/app/app.go

@@ -156,8 +156,6 @@ type satRunner interface {
 	RunTPMValidationPack(ctx context.Context, baseDir string, logFunc func(string)) (string, error)
 	TPMPresent() bool
 	RunNvidiaConfigCheckPack(ctx context.Context, baseDir string, logFunc func(string)) (string, error)
-	RunPCIeLinkCheckPack(ctx context.Context, baseDir string, logFunc func(string)) (string, error)
-	RunNvidiaPCIeBandwidthPack(ctx context.Context, baseDir string, gpuIndices []int, logFunc func(string)) (string, error)
 	RunCPUAcceptancePack(ctx context.Context, baseDir string, durationSec int, logFunc func(string)) (string, error)
 	ListNvidiaGPUs() ([]platform.NvidiaGPU, error)
 	ListNvidiaGPUSettings() ([]platform.NvidiaGPUSetting, error)

audit/internal/app/app_packs.go

@@ -255,20 +255,6 @@ func (a *App) RunNvidiaConfigCheckPackCtx(ctx context.Context, baseDir string, l
 	return a.sat.RunNvidiaConfigCheckPack(ctx, baseDir, logFunc)
 }
 
-func (a *App) RunPCIeLinkCheckPackCtx(ctx context.Context, baseDir string, logFunc func(string)) (string, error) {
-	if strings.TrimSpace(baseDir) == "" {
-		baseDir = DefaultSATBaseDir
-	}
-	return a.sat.RunPCIeLinkCheckPack(ctx, baseDir, logFunc)
-}
-
-func (a *App) RunNvidiaPCIeBandwidthPackCtx(ctx context.Context, baseDir string, gpuIndices []int, logFunc func(string)) (string, error) {
-	if strings.TrimSpace(baseDir) == "" {
-		baseDir = DefaultSATBaseDir
-	}
-	return a.sat.RunNvidiaPCIeBandwidthPack(ctx, baseDir, gpuIndices, logFunc)
-}
-
 func (a *App) RunAMDAcceptancePackCtx(ctx context.Context, baseDir string, logFunc func(string)) (string, error) {
 	if strings.TrimSpace(baseDir) == "" {
 		baseDir = DefaultSATBaseDir

audit/internal/app/app_test.go

@@ -295,14 +295,6 @@ func (f fakeSAT) RunNvidiaConfigCheckPack(_ context.Context, baseDir string, _ f
 	return "", nil
 }
 
-func (f fakeSAT) RunPCIeLinkCheckPack(_ context.Context, baseDir string, _ func(string)) (string, error) {
-	return "", nil
-}
-
-func (f fakeSAT) RunNvidiaPCIeBandwidthPack(_ context.Context, baseDir string, _ []int, _ func(string)) (string, error) {
-	return "", nil
-}
-
 func (f fakeSAT) RunCPUAcceptancePack(_ context.Context, baseDir string, durationSec int, _ func(string)) (string, error) {
 	if f.runCPUFn != nil {
 		return f.runCPUFn(baseDir, durationSec)

audit/internal/app/assets/README.md

@@ -45,8 +45,7 @@ which bee component happened to produce it:
   `UNSUPPORTED`).
 - **"What's the *current* rolled-up health per component?"** →
   `status/component-status.json`. One record per component key (e.g.
-  `pcie:gpu:nvidia`, `pcie:link:other`, `cpu:all`, `psu:all`,
-  `sat:pcie-link`), each with a `status` and a
+  `pcie:gpu:nvidia`, `cpu:all`, `psu:all`), each with a `status` and a
   `history` array. This is what feeds `reanimator.json`'s `status` fields —
   it's the source, not a duplicate. **The history is a transition log, not a
   per-check journal** — a component whose status never changed only has one
@@ -82,16 +81,15 @@ which bee component happened to produce it:
   for NVLink). If NVLink is expected, one `<inactive>` lane next to active
   ones on an otherwise-bonded pair is the real fault signature — a degraded
   link, not an absent one.
-- **"PCIe link speeds / retrain check?"** → `tasks/bee-sat/pcie-link-*/`:
-  `summary.txt` (`overall_status`, `gpu_<vendor>_status`, `other_status`,
-  `devices_tested`, `other_devices_degraded`, `warnings`) and
-  `pcie-link-report.txt` (one line per BDF — `OK` / `DEGRADED` /
-  `FELL OFF BUS` / `skipped: <reason>`, with `before`/`after`/`max` speed
-  and width). Both endpoints and the bridges above them are listed; a
-  bridge's `max` is its own port capability capped to what the device
-  downstream of it supports, so `before == after == max` — even with a
-  `(port capability GenX, limited by downstream device)` note — means the
-  link runs at the fastest rate that pair supports, not degraded.
+- **"NVIDIA PCIe link speed under real traffic?"** ->
+  `tasks/bee-sat/gpu-nvidia-bandwidth-*/summary.txt`: inspect
+  `pcie_link_under_load_status`, `pcie_gpuN_status`, and `pcie_degraded`.
+  `nvidia-pcie-link-under-load-report.txt` records each GPU's idle baseline,
+  post-`nvbandwidth` speed, maximum speed, and negotiated/max width. A Gen1
+  baseline is informational; only a link still below maximum immediately
+  after real traffic fails. For NICs/HBAs and other non-GPU PCIe devices,
+  use the read-only inventory and AER logs; there is no generic link-speed
+  health verdict without a device-specific traffic generator.
 - **"TPM state / validation?"** → `tasks/bee-sat/tpm-*/summary.txt`
   (`overall_status` plus per-step `N-<step>_status`); raw dumps in
   `export/platform/tpm-*.txt`. Read-only — it never writes the TPM.
@@ -122,11 +120,10 @@ which bee component happened to produce it:
   `storcli2-show-all.json`'s `"Number of Controllers"` and `lspci`/
   `nvme-list.json` for actual storage hardware. Same pattern as
   `nvlink_pairs_checked=0` above.
-- `pcie-link-report.txt` bridge line `DEGRADED` with `before == after ==
-  max` and `(port capability GenX, limited by downstream device)`, while
-  the endpoint below it on the same link reads `OK` → link is at the
-  downstream device's native maximum, not a fault. A real bridge fault has
-  `after` below the (already capped) `max`, or `FELL OFF BUS`.
+- `nvidia-pcie-link-under-load-report.txt` shows `before=Gen1` and
+  `after=max` at full width -> normal GPU idle power management, not a PCIe
+  downgrade. Treat `after < max` or `width < max_width` as a fault only when
+  the paired `nvbandwidth` command actually ran.
 - `systemctl restart ... timed out` in a `tasks/_services/bee-*.log` →
   implicates a SAT run only if its timestamp is inside that run's
   `run_at_utc`/`started_at`–`done_at` window in
@@ -209,7 +206,7 @@ current-state answer.
 
 | Path | What it is |
 |---|---|
-| `bee-sat/<target>-<timestamp>/` | One directory per acceptance-test run — `target` is what was tested (`gpu-nvidia`, `cpu`, `memory`, `storage`, `nccl-tests`, `gpu-nvidia-bandwidth`, `nvidia-config` covering GPU config/NVLink/Confidential-Computing readiness, `pcie-link` (whole-bus link-retrain check — see "Fastest answers"), `tpm` (read-only TPM validation), etc). Each contains `summary.txt` (key=value, always has `overall_status`), a full human-readable report (e.g. `storage`'s per-drive `disk-NN-<dev>-report.txt` — a narrative verdict, not raw tool output; the raw data behind it lives in `export/storage/`), numbered per-job logs, and `verbose.log` (every subprocess invocation + exit code — the place to look when a summary doesn't explain *why* something failed). |
+| `bee-sat/<target>-<timestamp>/` | One directory per acceptance-test run — `target` is what was tested (`gpu-nvidia`, `cpu`, `memory`, `storage`, `nccl-tests`, `gpu-nvidia-bandwidth` including the NVIDIA PCIe link-under-load verdict, `nvidia-config` covering GPU config/NVLink/Confidential-Computing readiness, `tpm` (read-only TPM validation), etc). Each contains `summary.txt` (key=value, always has `overall_status`), a full human-readable report (e.g. `storage`'s per-drive `disk-NN-<dev>-report.txt` — a narrative verdict, not raw tool output; the raw data behind it lives in `export/storage/`), numbered per-job logs, and `verbose.log` (every subprocess invocation + exit code — the place to look when a summary doesn't explain *why* something failed). |
 | `bee-bench/` | Benchmark run artifacts, if any have been run. |
 | `<NNN>_<slug>_<done\|failed>/` | One directory per task launched from the web UI's task queue — see "Fastest answers" above. |
 | `_state/bee-audit.json` | The full hardware-audit document as bee last generated it — same content as `export/reanimator.json`, kept here as bee's own internal record. |

audit/internal/app/component_status_db.go

@@ -125,6 +125,55 @@ func lastEntryFromSource(history []ComponentStatusEntry, source string) *Compone
 	return nil
 }
 
+// recomputeIgnoringSources retires obsolete producers from the current
+// component verdict without deleting their audit history. It selects the
+// highest-severity latest observation from every remaining source.
+func (db *ComponentStatusDB) recomputeIgnoringSources(key string, sources ...string) {
+	if db == nil {
+		return
+	}
+	ignored := make(map[string]struct{}, len(sources))
+	for _, source := range sources {
+		ignored[source] = struct{}{}
+	}
+
+	db.mu.Lock()
+	defer db.mu.Unlock()
+	db.reloadLocked()
+	rec := db.records[key]
+	if rec == nil {
+		return
+	}
+
+	latest := make(map[string]ComponentStatusEntry)
+	for i := len(rec.History) - 1; i >= 0; i-- {
+		entry := rec.History[i]
+		if _, skip := ignored[entry.Source]; skip {
+			continue
+		}
+		if _, seen := latest[entry.Source]; !seen {
+			latest[entry.Source] = entry
+		}
+	}
+	var winner *ComponentStatusEntry
+	for _, entry := range latest {
+		candidate := entry
+		if winner == nil || componentSeverity(candidate.Status) > componentSeverity(winner.Status) ||
+			(componentSeverity(candidate.Status) == componentSeverity(winner.Status) && candidate.At.After(winner.At)) {
+			winner = &candidate
+		}
+	}
+	if winner == nil {
+		return
+	}
+	if rec.Status != winner.Status || rec.ErrorSummary != winner.Detail {
+		rec.LastChangedAt = time.Now().UTC()
+	}
+	rec.Status = winner.Status
+	rec.ErrorSummary = winner.Detail
+	_ = db.saveLocked()
+}
+
 // Get returns the current record for a component key.
 func (db *ComponentStatusDB) Get(key string) (ComponentStatusRecord, bool) {
 	if db == nil {
@@ -258,32 +307,13 @@ func ApplySATResultToDB(db *ComponentStatusDB, target, archivePath string) {
 	// otherwise fails to match any real BDF.
 	switch target {
 	case "nvidia", "nvidia-targeted-stress", "nvidia-compute", "nvidia-targeted-power", "nvidia-pulse",
-		"nvidia-interconnect", "nvidia-bandwidth", "nvidia-stress", "nvidia-config", "nvidia-pcie-bandwidth":
+		"nvidia-interconnect", "nvidia-bandwidth", "nvidia-stress", "nvidia-config":
 		db.Record("pcie:gpu:nvidia", source, dbStatus, detail)
+		if target == "nvidia-bandwidth" {
+			db.recomputeIgnoringSources("pcie:gpu:nvidia", "sat:pcie-link", "sat:nvidia-pcie-bandwidth")
+		}
 	case "amd", "amd-stress", "amd-mem", "amd-bandwidth":
 		db.Record("pcie:gpu:amd", source, dbStatus, detail)
-	case "pcie-link":
-		// Forced-retrain PCIe link check (audit/internal/platform/pcie_link_check.go):
-		// the only verified (non-idle-sampled) source for PCIe link-speed
-		// status. summary.txt carries up to three independent sub-verdicts
-		// — record each into its own component key rather than collapsing
-		// them into one, since a degraded NIC/HBA shouldn't be reported as
-		// a GPU fault or vice versa.
-		recordPCIeLinkSubStatus := func(key, kvKey string) {
-			v, ok := kv[kvKey]
-			if !ok {
-				return
-			}
-			st := strings.ToUpper(strings.TrimSpace(v))
-			d := "pcie-link SAT: " + st
-			if st != "OK" && kv["warnings"] != "" {
-				d += " — " + kv["warnings"]
-			}
-			db.Record(key, source, satStatusToDBStatus(st), d)
-		}
-		recordPCIeLinkSubStatus("pcie:gpu:nvidia", "gpu_nvidia_status")
-		recordPCIeLinkSubStatus("pcie:gpu:amd", "gpu_amd_status")
-		recordPCIeLinkSubStatus("pcie:link:other", "other_status")
 	case "memory", "memory-stress", "sat-stress":
 		db.Record("memory:all", source, dbStatus, detail)
 	case "cpu", "platform-stress":

audit/internal/app/component_status_db_test.go

@@ -75,6 +75,46 @@ func TestApplySATResultToDBNormalizesGPUKeyByVendor(t *testing.T) {
 	}
 }
 
+func TestNvidiaBandwidthRetiresForcedRetrainWarning(t *testing.T) {
+	db, err := OpenComponentStatusDB(filepath.Join(t.TempDir(), "component-status.json"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	db.Record("pcie:gpu:nvidia", "sat:pcie-link", "Warning", "idle forced retrain stayed at Gen1")
+
+	ApplySATResultToDB(db, "nvidia-bandwidth", writeSATSummary(t, "OK"))
+
+	rec, ok := db.Get("pcie:gpu:nvidia")
+	if !ok {
+		t.Fatal("expected pcie:gpu:nvidia record")
+	}
+	if rec.Status != "OK" {
+		t.Fatalf("status=%q want OK after real-traffic check supersedes forced retrain", rec.Status)
+	}
+	if lastEntryFromSource(rec.History, "sat:pcie-link") == nil {
+		t.Fatal("retired forced-retrain observation must remain in audit history")
+	}
+}
+
+func TestNvidiaBandwidthDoesNotClearIndependentGPUWarning(t *testing.T) {
+	db, err := OpenComponentStatusDB(filepath.Join(t.TempDir(), "component-status.json"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	db.Record("pcie:gpu:nvidia", "sat:pcie-link", "Warning", "idle forced retrain stayed at Gen1")
+	db.Record("pcie:gpu:nvidia", "sat:nvidia-config", "Warning", "NVLink error counters are non-zero")
+
+	ApplySATResultToDB(db, "nvidia-bandwidth", writeSATSummary(t, "OK"))
+
+	rec, ok := db.Get("pcie:gpu:nvidia")
+	if !ok {
+		t.Fatal("expected pcie:gpu:nvidia record")
+	}
+	if rec.Status != "Warning" || !strings.Contains(rec.ErrorSummary, "NVLink") {
+		t.Fatalf("record=%+v want independent nvidia-config Warning to survive", rec)
+	}
+}
+
 func TestApplyComponentStatusDBMatchesGPUByVendor(t *testing.T) {
 	db, err := OpenComponentStatusDB(filepath.Join(t.TempDir(), "component-status.json"))
 	if err != nil {

audit/internal/collector/pcie.go

@@ -353,7 +353,7 @@ func normalizePCILinkSpeed(raw string) string {
 
 // NormalizePCILinkSpeed maps a raw sysfs current_link_speed/max_link_speed
 // string (e.g. "16.0 GT/s PCIe") to a "GenN" label. Exported for reuse by
-// the platform package's PCIe link-retrain SAT, which needs the same
+// the platform package's NVIDIA link-under-load check, which needs the same
 // mapping outside the idle collector path.
 func NormalizePCILinkSpeed(raw string) string {
 	raw = strings.TrimSpace(strings.ToLower(raw))

audit/internal/platform/nvidia_bandwidth_socket_test.go

@@ -194,7 +194,7 @@ func TestRunNvidiaBandwidthPackSplitsPerSocketThenAll(t *testing.T) {
 		"03-dcgmi-nvbandwidth-socket0.log",
 		"04-dcgmi-nvbandwidth-socket1.log",
 		"05-dcgmi-nvbandwidth-all.log",
-		"06-nvidia-smi-after.log",
+		"nvidia-pcie-link-under-load-report.txt",
 	}
 	for _, name := range wantFiles {
 		if _, err := os.Stat(filepath.Join(runDir, name)); err != nil {

audit/internal/platform/nvidia_pcie_bandwidth.go

@@ -1,14 +1,14 @@
 package platform
 
 import (
-	"context"
 	"fmt"
 	"os"
 	"path/filepath"
 	"sort"
 	"strconv"
 	"strings"
-	"time"
+
+	"bee/audit/internal/collector"
 )
 
 // nvidiaPCIeBandwidthFinding is one GPU's post-load PCIe link-speed result.
@@ -21,38 +21,18 @@ type nvidiaPCIeBandwidthFinding struct {
 	Width       int
 	MaxWidth    int
 	Degraded    bool
+	Supported   bool
+	Sampled     bool
 }
 
-// RunNvidiaPCIeBandwidthPack drives real host<->device traffic across the
-// GPUs' PCIe links (via `dcgmi diag -r nvbandwidth`, the same tool the
-// existing nvidia-bandwidth SAT uses for GPU-to-GPU throughput) and then
-// resamples each GPU's PCIe link speed from sysfs immediately afterward.
-//
-// This is deliberately a separate, narrower check from "nvidia-bandwidth":
-// that SAT's overall_status reflects nvbandwidth's own pass/fail (did the
-// measured GB/s clear its internal threshold), never PCIe link speed
-// itself. This pack exists purely to answer "did sustained real traffic
-// make the link train up to its negotiated maximum" — the load-bearing
-// counterpart to the idle-time collector reading that used to (falsely)
-// drive pcie:gpu:nvidia's status. See
-// bible-local/decisions/2026-08-24-pcie-gpu-gen1-idle-warning-unresolved.md.
-func (s *System) RunNvidiaPCIeBandwidthPack(ctx context.Context, baseDir string, gpuIndices []int, logFunc func(string)) (string, error) {
-	if ctx == nil {
-		ctx = context.Background()
-	}
-	if baseDir == "" {
-		baseDir = "/var/log/bee-sat"
-	}
-	ts := time.Now().UTC().Format("20060102-150405")
-	runDir := filepath.Join(baseDir, "nvidia-pcie-bandwidth-"+ts)
-	if err := os.MkdirAll(runDir, 0755); err != nil {
-		return "", err
-	}
-	verboseLog := filepath.Join(runDir, "verbose.log")
-
+// captureNvidiaPCIeLinkBaseline records link capabilities immediately before
+// the existing nvbandwidth SAT. The post-load half is completed by
+// finishNvidiaPCIeLinkCheck as soon as that same SAT returns, so one real PCIe
+// traffic pass provides both the DCGM verdict and the negotiated-link verdict.
+func captureNvidiaPCIeLinkBaseline(gpuIndices []int) ([]nvidiaPCIeBandwidthFinding, error) {
 	bdfByIndex, err := gpuIndexToBDF(gpuIndices)
 	if err != nil {
-		return "", fmt.Errorf("resolve GPU BDFs: %w", err)
+		return nil, fmt.Errorf("resolve GPU BDFs: %w", err)
 	}
 	indices := make([]int, 0, len(bdfByIndex))
 	for idx := range bdfByIndex {
@@ -63,39 +43,37 @@ func (s *System) RunNvidiaPCIeBandwidthPack(ctx context.Context, baseDir string,
 	findings := make([]nvidiaPCIeBandwidthFinding, 0, len(indices))
 	for _, idx := range indices {
 		bdf := bdfByIndex[idx]
-		before, _ := readPCIeSysfsString(bdf, "current_link_speed")
-		max, _ := readPCIeSysfsString(bdf, "max_link_speed")
-		maxWidth, _ := readPCIeSysfsInt(bdf, "max_link_width")
+		before, beforeOK := readPCIeSysfsString(bdf, "current_link_speed")
+		max, maxOK := readPCIeSysfsString(bdf, "max_link_speed")
+		maxWidth, widthOK := readPCIeSysfsInt(bdf, "max_link_width")
 		findings = append(findings, nvidiaPCIeBandwidthFinding{
 			Index: idx, BDF: bdf, BeforeSpeed: before, MaxSpeed: max, MaxWidth: maxWidth,
+			Supported: beforeOK && maxOK && widthOK,
 		})
 	}
+	return findings, nil
+}
 
-	cmd := []string{"dcgmi", "diag", "-r", "nvbandwidth"}
-	if len(indices) > 0 {
-		cmd = append(cmd, "-i", joinIndexList(indices))
+func finishNvidiaPCIeLinkCheck(runDir string, findings []nvidiaPCIeBandwidthFinding) error {
+	report := renderNvidiaPCIeBandwidthReport(findings)
+	if err := os.WriteFile(filepath.Join(runDir, "nvidia-pcie-link-under-load-report.txt"), []byte(report), 0644); err != nil {
+		return err
 	}
-	out, runErr := runSATCommandCtx(ctx, verboseLog, "dcgmi-nvbandwidth", cmd, nil, logFunc)
-	_ = os.WriteFile(filepath.Join(runDir, "01-dcgmi-nvbandwidth.log"), out, 0644)
+	return appendNvidiaPCIeLinkSummary(filepath.Join(runDir, "summary.txt"), findings)
+}
 
+func sampleNvidiaPCIeLinkAfterLoad(findings []nvidiaPCIeBandwidthFinding) {
 	for i := range findings {
 		bdf := findings[i].BDF
-		after, _ := readPCIeSysfsString(bdf, "current_link_speed")
-		width, _ := readPCIeSysfsInt(bdf, "current_link_width")
+		after, afterOK := readPCIeSysfsString(bdf, "current_link_speed")
+		width, widthOK := readPCIeSysfsInt(bdf, "current_link_width")
 		findings[i].AfterSpeed = after
 		findings[i].Width = width
-		findings[i].Degraded = after != findings[i].MaxSpeed
+		findings[i].Sampled = true
+		findings[i].Supported = findings[i].Supported && afterOK && widthOK
+		findings[i].Degraded = findings[i].Supported &&
+			(after != findings[i].MaxSpeed || width != findings[i].MaxWidth)
 	}
-
-	summary := renderNvidiaPCIeBandwidthSummary(findings, runErr)
-	if err := os.WriteFile(filepath.Join(runDir, "summary.txt"), []byte(summary), 0644); err != nil {
-		return "", err
-	}
-	report := renderNvidiaPCIeBandwidthReport(findings, runErr)
-	if err := os.WriteFile(filepath.Join(runDir, "nvidia-pcie-bandwidth-report.txt"), []byte(report), 0644); err != nil {
-		return "", err
-	}
-	return runDir, nil
 }
 
 // gpuIndexToBDF resolves each of gpuIndices to its PCI BDF via nvidia-smi.
@@ -131,35 +109,79 @@ func gpuIndexToBDF(gpuIndices []int) (map[int]string, error) {
 	return result, nil
 }
 
-func renderNvidiaPCIeBandwidthSummary(findings []nvidiaPCIeBandwidthFinding, runErr error) string {
-	var b strings.Builder
-	fmt.Fprintf(&b, "run_at_utc=%s\n", time.Now().UTC().Format(time.RFC3339))
-	fmt.Fprintf(&b, "gpu_count=%d\n", len(findings))
+func readPCIeSysfsString(bdf, attr string) (string, bool) {
+	raw, err := satReadFile(filepath.Join("/sys/bus/pci/devices", bdf, attr))
+	if err != nil {
+		return "", false
+	}
+	v := strings.TrimSpace(string(raw))
+	if v == "" {
+		return "", false
+	}
+	return collector.NormalizePCILinkSpeed(v), true
+}
+
+func readPCIeSysfsInt(bdf, attr string) (int, bool) {
+	raw, err := satReadFile(filepath.Join("/sys/bus/pci/devices", bdf, attr))
+	if err != nil {
+		return 0, false
+	}
+	v, err := strconv.Atoi(strings.TrimSpace(string(raw)))
+	if err != nil || v < 0 {
+		return 0, false
+	}
+	return v, true
+}
+
+func appendNvidiaPCIeLinkSummary(path string, findings []nvidiaPCIeBandwidthFinding) error {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return err
+	}
+	var extra strings.Builder
+	fmt.Fprintf(&extra, "pcie_gpu_count=%d\n", len(findings))
 	degraded := 0
+	supported := 0
+	var reasons []string
 	for _, f := range findings {
-		fmt.Fprintf(&b, "gpu%d_status=%s\n", f.Index, statusLabel(!f.Degraded))
+		status := "UNSUPPORTED"
+		if f.Supported && f.Sampled {
+			supported++
+			status = statusLabel(!f.Degraded)
+		}
+		fmt.Fprintf(&extra, "pcie_gpu%d_status=%s\n", f.Index, status)
 		if f.Degraded {
 			degraded++
+			reasons = append(reasons, fmt.Sprintf("GPU%d (%s): under load at %s x%d, capable of %s x%d",
+				f.Index, f.BDF, f.AfterSpeed, f.Width, f.MaxSpeed, f.MaxWidth))
 		}
 	}
-	fmt.Fprintf(&b, "degraded=%d\n", degraded)
-	if runErr != nil {
-		fmt.Fprintf(&b, "dcgmi_nvbandwidth_error=%s\n", runErr.Error())
-	}
+	fmt.Fprintf(&extra, "pcie_degraded=%d\n", degraded)
+	linkStatus := "UNSUPPORTED"
 	if degraded > 0 {
-		fmt.Fprintln(&b, "overall_status=FAILED")
-		var reasons []string
-		for _, f := range findings {
-			if f.Degraded {
-				reasons = append(reasons, fmt.Sprintf("GPU%d (%s): still at %s under load, capable of %s",
-					f.Index, f.BDF, f.AfterSpeed, f.MaxSpeed))
+		linkStatus = "FAILED"
+	} else if supported == len(findings) && supported > 0 {
+		linkStatus = "OK"
+	}
+	fmt.Fprintf(&extra, "pcie_link_under_load_status=%s\n", linkStatus)
+	if len(reasons) > 0 {
+		fmt.Fprintf(&extra, "pcie_link_under_load_detail=%s\n", strings.Join(reasons, "; "))
+	}
+	if linkStatus == "FAILED" {
+		lines := strings.Split(string(raw), "\n")
+		for i := range lines {
+			if strings.HasPrefix(lines[i], "overall_status=") {
+				lines[i] = "overall_status=FAILED"
+				break
 			}
 		}
-		fmt.Fprintf(&b, "warnings=%s\n", strings.Join(reasons, "; "))
-	} else {
-		fmt.Fprintln(&b, "overall_status=OK")
+		raw = []byte(strings.Join(lines, "\n"))
 	}
-	return b.String()
+	if len(raw) > 0 && raw[len(raw)-1] != '\n' {
+		raw = append(raw, '\n')
+	}
+	raw = append(raw, extra.String()...)
+	return os.WriteFile(path, raw, 0644)
 }
 
 func statusLabel(ok bool) string {
@@ -169,18 +191,17 @@ func statusLabel(ok bool) string {
 	return "FAILED"
 }
 
-func renderNvidiaPCIeBandwidthReport(findings []nvidiaPCIeBandwidthFinding, runErr error) string {
+func renderNvidiaPCIeBandwidthReport(findings []nvidiaPCIeBandwidthFinding) string {
 	var b strings.Builder
 	line := strings.Repeat("=", 80)
 	b.WriteString(line + "\n")
-	b.WriteString("NVIDIA GPU PCIe Bandwidth / Link-Under-Load Check\n")
+	b.WriteString("NVIDIA GPU PCIe Link-Under-Load Check\n")
 	b.WriteString(line + "\n\n")
-	if runErr != nil {
-		fmt.Fprintf(&b, "dcgmi diag -r nvbandwidth: %s\n\n", runErr.Error())
-	}
 	for _, f := range findings {
-		verdict := "OK"
-		if f.Degraded {
+		verdict := "UNSUPPORTED"
+		if f.Supported && f.Sampled && !f.Degraded {
+			verdict = "OK"
+		} else if f.Degraded {
 			verdict = "DEGRADED"
 		}
 		fmt.Fprintf(&b, "GPU%d (%s): %s\n", f.Index, f.BDF, verdict)

audit/internal/platform/nvidia_pcie_bandwidth_test.go

@@ -0,0 +1,77 @@
+package platform
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestPCIeLinkSummaryAcceptsIdleGen1WhenLinkReachesMaximumUnderLoad(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "summary.txt")
+	if err := os.WriteFile(path, []byte("overall_status=OK\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	findings := []nvidiaPCIeBandwidthFinding{{
+		Index: 0, BDF: "0000:05:00.0", BeforeSpeed: "Gen1", AfterSpeed: "Gen5",
+		MaxSpeed: "Gen5", Width: 16, MaxWidth: 16, Supported: true, Sampled: true,
+	}}
+
+	if err := appendNvidiaPCIeLinkSummary(path, findings); err != nil {
+		t.Fatal(err)
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	got := string(raw)
+	if !strings.Contains(got, "overall_status=OK\n") || !strings.Contains(got, "pcie_link_under_load_status=OK\n") {
+		t.Fatalf("summary did not accept a link that rose from idle Gen1 to maximum under load:\n%s", got)
+	}
+}
+
+func TestPCIeLinkSummaryFailsLinkStillBelowMaximumAfterLoad(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "summary.txt")
+	if err := os.WriteFile(path, []byte("overall_status=UNSUPPORTED\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	findings := []nvidiaPCIeBandwidthFinding{{
+		Index: 0, BDF: "0000:05:00.0", BeforeSpeed: "Gen1", AfterSpeed: "Gen1",
+		MaxSpeed: "Gen5", Width: 16, MaxWidth: 16, Supported: true, Sampled: true, Degraded: true,
+	}}
+
+	if err := appendNvidiaPCIeLinkSummary(path, findings); err != nil {
+		t.Fatal(err)
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	got := string(raw)
+	if !strings.Contains(got, "overall_status=FAILED\n") || !strings.Contains(got, "pcie_link_under_load_status=FAILED\n") {
+		t.Fatalf("summary did not fail a link that stayed degraded under load:\n%s", got)
+	}
+}
+
+func TestPCIeLinkSummaryIsUnsupportedWhenLoadNeverRan(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "summary.txt")
+	if err := os.WriteFile(path, []byte("overall_status=FAILED\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	findings := []nvidiaPCIeBandwidthFinding{{
+		Index: 0, BDF: "0000:05:00.0", BeforeSpeed: "Gen1", MaxSpeed: "Gen5",
+		MaxWidth: 16, Supported: true,
+	}}
+
+	if err := appendNvidiaPCIeLinkSummary(path, findings); err != nil {
+		t.Fatal(err)
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	got := string(raw)
+	if !strings.Contains(got, "pcie_link_under_load_status=UNSUPPORTED\n") {
+		t.Fatalf("summary claimed a link verdict although the load callback never ran:\n%s", got)
+	}
+}

audit/internal/platform/pcie_link_check.go

@@ -1,506 +0,0 @@
-package platform
-
-import (
-	"context"
-	"fmt"
-	"os"
-	"path/filepath"
-	"sort"
-	"strconv"
-	"strings"
-	"time"
-
-	"bee/audit/internal/collector"
-)
-
-// pcieLinkRetrainTimeout bounds how long we wait for a device to finish
-// retraining (clear the Link Status "Link Training" bit) before giving up
-// and reading whatever speed it settled on anyway. The PCIe spec allows up
-// to 100ms for Gen1-3 and longer for higher generations with equalization;
-// this is generous headroom above that.
-const pcieLinkRetrainTimeout = 2 * time.Second
-
-// pcieLinkFinding is one device's before/after link-speed retrain result.
-type pcieLinkFinding struct {
-	BDF          string
-	Description  string
-	VendorID     string
-	ClassCode    string
-	IsGPU        bool
-	GPUVendor    string // "nvidia" or "amd", only set when IsGPU
-	Skipped      string // non-empty reason this device wasn't retrained
-	BeforeSpeed  string
-	AfterSpeed   string
-	MaxSpeed     string
-	PortMaxSpeed string // bridge's own capability when MaxSpeed is limited by its downstream peer
-	Width        int
-	MaxWidth     int
-	PortMaxWidth int // bridge's own capability when MaxWidth is limited by its downstream peer
-	Degraded     bool
-	NotPresent   bool // true when the slot trained to zero lanes: nothing is plugged in (or it fell off the bus), not a speed regression
-}
-
-// RunPCIeLinkCheckPack forces every enabled PCIe device to retrain its link
-// (via the PCIe Link Control register's spec-defined Retrain Link bit — see
-// PCIe base spec, Link Control Register, bit 5) and compares the
-// post-retrain negotiated speed against the device's own reported maximum.
-//
-// This exists because a plain idle-time sysfs read of current_link_speed is
-// not a reliable fault signal: NVIDIA GPUs (and other devices with runtime
-// power management) deliberately downclock their PCIe link to save power
-// while idle, which is indistinguishable from a real degraded slot/riser/
-// cable without either sustained traffic or a forced retrain. Forcing a
-// retrain sidesteps needing a device-specific load generator (bee-gpu-burn
-// exists for GPUs; nothing plays that role for NICs, HBAs, or PCIe
-// switches) — retraining is a PCIe-spec mechanism every endpoint supports,
-// so this one check covers every PCIe device in the machine, not just
-// GPUs. See bible-local/decisions/2026-08-24-pcie-gpu-gen1-idle-warning-unresolved.md
-// for the history of narrower attempts that didn't generalize.
-//
-// Disabled devices (sysfs enable==0 — e.g. PCIe fabric-management endpoints
-// the kernel never activates, per the 2026-06-12 decision) are left alone:
-// they carry no data traffic, so there is nothing to verify and no reason
-// to poke them.
-func (s *System) RunPCIeLinkCheckPack(ctx context.Context, baseDir string, logFunc func(string)) (string, error) {
-	if ctx == nil {
-		ctx = context.Background()
-	}
-	if baseDir == "" {
-		baseDir = "/var/log/bee-sat"
-	}
-	ts := time.Now().UTC().Format("20060102-150405")
-	runDir := filepath.Join(baseDir, "pcie-link-"+ts)
-	if err := os.MkdirAll(runDir, 0755); err != nil {
-		return "", err
-	}
-	verboseLog := filepath.Join(runDir, "verbose.log")
-
-	bdfs, err := listPCIDeviceBDFs()
-	if err != nil {
-		return "", fmt.Errorf("list PCI devices: %w", err)
-	}
-
-	var findings []pcieLinkFinding
-	for _, bdf := range bdfs {
-		if logFunc != nil {
-			logFunc(fmt.Sprintf("=== %s ===", bdf))
-		}
-		f := retrainAndSamplePCIeDevice(ctx, verboseLog, bdf, logFunc)
-		findings = append(findings, f)
-	}
-
-	summary := renderPCIeLinkCheckSummary(findings)
-	if err := os.WriteFile(filepath.Join(runDir, "summary.txt"), []byte(summary), 0644); err != nil {
-		return "", err
-	}
-	report := renderPCIeLinkCheckReport(findings)
-	if err := os.WriteFile(filepath.Join(runDir, "pcie-link-report.txt"), []byte(report), 0644); err != nil {
-		return "", err
-	}
-	return runDir, nil
-}
-
-// listPCIDeviceBDFs returns every BDF under /sys/bus/pci/devices, sorted for
-// deterministic report ordering.
-func listPCIDeviceBDFs() ([]string, error) {
-	entries, err := os.ReadDir("/sys/bus/pci/devices")
-	if err != nil {
-		return nil, err
-	}
-	bdfs := make([]string, 0, len(entries))
-	for _, e := range entries {
-		bdfs = append(bdfs, e.Name())
-	}
-	sort.Strings(bdfs)
-	return bdfs, nil
-}
-
-func retrainAndSamplePCIeDevice(ctx context.Context, verboseLog, bdf string, logFunc func(string)) pcieLinkFinding {
-	f := pcieLinkFinding{BDF: bdf}
-
-	vendor, _ := readPCIeSysfsHex(bdf, "vendor")
-	class, _ := readPCIeSysfsHex(bdf, "class")
-	f.VendorID = vendor
-	f.ClassCode = class
-	f.IsGPU, f.GPUVendor = classifyGPUFromVendorClass(vendor, class)
-	f.Description = pcieDeviceDescription(ctx, verboseLog, bdf, logFunc)
-
-	if enabled, ok := readPCIeSysfsInt(bdf, "enable"); ok && enabled == 0 {
-		f.Skipped = "device disabled (no data traffic; link state has no operational impact)"
-		return f
-	}
-
-	before, beforeOK := readPCIeSysfsString(bdf, "current_link_speed")
-	maxSpeed, maxOK := readPCIeSysfsString(bdf, "max_link_speed")
-	width, _ := readPCIeSysfsInt(bdf, "current_link_width")
-	maxWidth, _ := readPCIeSysfsInt(bdf, "max_link_width")
-	f.BeforeSpeed = before
-	f.MaxSpeed = maxSpeed
-	f.MaxWidth = maxWidth
-	if !beforeOK || !maxOK {
-		f.Skipped = "no PCIe link-speed attributes in sysfs (not a link-trained endpoint)"
-		return f
-	}
-
-	if width == 0 {
-		// A downstream switch/root port with nothing seated reads zero
-		// trained lanes even before we touch it. Plenty of legitimate
-		// configs leave slots like this unpopulated (not every server ships
-		// every NIC/riser slot filled), so this is not by itself evidence
-		// of anything wrong — retraining an empty slot can't produce a
-		// meaningful speed reading, and there's no baseline here to say
-		// "this used to have a card." Skip it exactly like a disabled
-		// device: nothing to verify, no reason to fail the run over it.
-		f.NotPresent = true
-		f.Skipped = "no device present downstream (empty slot/riser — nothing to retrain)"
-		return f
-	}
-
-	// A bridge/root port reports its own maximum capability in sysfs, not
-	// the maximum mutually supported by the device at the other end of the
-	// link.  Comparing a Gen4 x16 root port directly with a Gen3 x8 or Gen2
-	// x4 endpoint therefore produces a false degradation even though the
-	// link is running at the fastest rate the endpoint supports.  The child
-	// device is tested separately, so use its advertised capability to
-	// calculate the real target for this bridge-side view of the same link.
-	if isPCIeBridgeClass(class) {
-		if peerSpeed, peerWidth, ok := downstreamPCIeLinkCapability(bdf); ok {
-			f.PortMaxSpeed = f.MaxSpeed
-			f.PortMaxWidth = f.MaxWidth
-			f.MaxSpeed = minPCIeLinkSpeed(f.MaxSpeed, peerSpeed)
-			f.MaxWidth = minPositiveInt(f.MaxWidth, peerWidth)
-			// The degradation verdict below compares against maxSpeed, so
-			// it has to track the peer-capped target too — otherwise a
-			// bridge whose port out-specs its downstream device (e.g. a
-			// Gen5 root port feeding a Gen4 HBA) is flagged DEGRADED even
-			// though the link is at the fastest rate the pair supports.
-			maxSpeed = f.MaxSpeed
-		}
-	}
-
-	if err := retrainPCIeLink(ctx, verboseLog, bdf, logFunc); err != nil {
-		f.Skipped = "retrain failed: " + err.Error()
-		f.AfterSpeed = before
-		f.Width = width
-		// before/maxSpeed are already normalized "GenN" labels (see
-		// readPCIeSysfsString) — compare directly, don't re-normalize.
-		f.Degraded = before != maxSpeed
-		return f
-	}
-
-	after, _ := readPCIeSysfsString(bdf, "current_link_speed")
-	widthAfter, _ := readPCIeSysfsInt(bdf, "current_link_width")
-	if widthAfter == 0 {
-		// The device answered before the retrain but is gone immediately
-		// after it (fell off the bus mid-check) — unlike the pre-retrain
-		// case above, this had a live link a moment ago, so it's worth
-		// surfacing rather than silently skipping.
-		f.AfterSpeed = after
-		f.Width = widthAfter
-		f.NotPresent = true
-		f.Degraded = true
-		return f
-	}
-	f.AfterSpeed = after
-	f.Width = widthAfter
-	f.Degraded = after != maxSpeed
-	return f
-}
-
-// retrainPCIeLink sets the Retrain Link bit (bit 5) of the PCI Express
-// Capability's Link Control register via setpci, then polls the Link
-// Status register's Link Training bit (bit 11) until it clears or
-// pcieLinkRetrainTimeout elapses.
-func retrainPCIeLink(ctx context.Context, verboseLog, bdf string, logFunc func(string)) error {
-	linkCtrlOut, err := runSATCommandCtx(ctx, verboseLog, "setpci-read-"+bdf,
-		[]string{"setpci", "-s", bdf, "CAP_EXP+0x10.w"}, nil, logFunc)
-	if err != nil {
-		return fmt.Errorf("read Link Control: %w", err)
-	}
-	cur, err := strconv.ParseUint(strings.TrimSpace(string(linkCtrlOut)), 16, 16)
-	if err != nil {
-		return fmt.Errorf("parse Link Control %q: %w", linkCtrlOut, err)
-	}
-	const retrainLinkBit = 0x0020
-	newVal := uint16(cur) | retrainLinkBit
-
-	if _, err := runSATCommandCtx(ctx, verboseLog, "setpci-retrain-"+bdf,
-		[]string{"setpci", "-s", bdf, fmt.Sprintf("CAP_EXP+0x10.w=%04x", newVal)}, nil, logFunc); err != nil {
-		return fmt.Errorf("write Retrain Link bit: %w", err)
-	}
-
-	const linkTrainingBit = 0x0800
-	deadline := time.Now().Add(pcieLinkRetrainTimeout)
-	for time.Now().Before(deadline) {
-		statusOut, err := runSATCommandCtx(ctx, verboseLog, "setpci-status-"+bdf,
-			[]string{"setpci", "-s", bdf, "CAP_EXP+0x12.w"}, nil, nil)
-		if err == nil {
-			if status, perr := strconv.ParseUint(strings.TrimSpace(string(statusOut)), 16, 16); perr == nil {
-				if status&linkTrainingBit == 0 {
-					return nil
-				}
-			}
-		}
-		time.Sleep(50 * time.Millisecond)
-	}
-	// Timed out waiting for training to clear; the caller still samples
-	// whatever speed sysfs reports, which is the honest answer either way.
-	return nil
-}
-
-func pcieDeviceDescription(ctx context.Context, verboseLog, bdf string, logFunc func(string)) string {
-	out, err := runSATCommandCtx(ctx, verboseLog, "lspci-"+bdf, []string{"lspci", "-s", bdf}, nil, logFunc)
-	if err != nil {
-		return ""
-	}
-	line := strings.TrimSpace(string(out))
-	if idx := strings.Index(line, "\n"); idx >= 0 {
-		line = line[:idx]
-	}
-	if idx := strings.Index(line, " "); idx >= 0 {
-		return strings.TrimSpace(line[idx+1:])
-	}
-	return line
-}
-
-func readPCIeSysfsString(bdf, attr string) (string, bool) {
-	raw, err := os.ReadFile(filepath.Join("/sys/bus/pci/devices", bdf, attr))
-	if err != nil {
-		return "", false
-	}
-	v := strings.TrimSpace(string(raw))
-	if v == "" {
-		return "", false
-	}
-	return collector.NormalizePCILinkSpeed(v), true
-}
-
-func readPCIeSysfsInt(bdf, attr string) (int, bool) {
-	raw, err := os.ReadFile(filepath.Join("/sys/bus/pci/devices", bdf, attr))
-	if err != nil {
-		return 0, false
-	}
-	v, err := strconv.Atoi(strings.TrimSpace(string(raw)))
-	if err != nil || v < 0 {
-		return 0, false
-	}
-	return v, true
-}
-
-func readPCIeSysfsHex(bdf, attr string) (string, bool) {
-	raw, err := os.ReadFile(filepath.Join("/sys/bus/pci/devices", bdf, attr))
-	if err != nil {
-		return "", false
-	}
-	return strings.TrimSpace(string(raw)), true
-}
-
-// isPCIeBridgeClass matches PCI class 0x0604xx (PCI-to-PCI bridge).  These
-// functions describe the upstream side of a downstream link, so their own
-// max_link_* values must be capped by the peer's capability.
-func isPCIeBridgeClass(classHex string) bool {
-	c := strings.TrimPrefix(strings.ToLower(strings.TrimSpace(classHex)), "0x")
-	return len(c) >= 4 && c[:4] == "0604"
-}
-
-// downstreamPCIeLinkCapability returns the strongest capability advertised
-// by a bridge's immediate child functions.  In sysfs those functions are
-// direct entries below the bridge device directory.  Multifunction devices
-// expose several children for one physical link; taking the strongest values
-// avoids understating the link because one auxiliary function omitted data.
-func downstreamPCIeLinkCapability(bdf string) (speed string, width int, ok bool) {
-	bridgeDir := filepath.Join("/sys/bus/pci/devices", bdf)
-	return downstreamPCIeLinkCapabilityAt(bridgeDir)
-}
-
-func downstreamPCIeLinkCapabilityAt(bridgeDir string) (speed string, width int, ok bool) {
-	entries, err := os.ReadDir(bridgeDir)
-	if err != nil {
-		return "", 0, false
-	}
-	for _, entry := range entries {
-		if !isFullPCIBDF(entry.Name()) {
-			continue
-		}
-		childDir := filepath.Join(bridgeDir, entry.Name())
-		rawSpeed, speedErr := os.ReadFile(filepath.Join(childDir, "max_link_speed"))
-		if speedErr != nil {
-			continue
-		}
-		childSpeed := collector.NormalizePCILinkSpeed(strings.TrimSpace(string(rawSpeed)))
-		if pcieGeneration(childSpeed) > pcieGeneration(speed) {
-			speed = childSpeed
-		}
-		if rawWidth, widthErr := os.ReadFile(filepath.Join(childDir, "max_link_width")); widthErr == nil {
-			if childWidth, parseErr := strconv.Atoi(strings.TrimSpace(string(rawWidth))); parseErr == nil && childWidth > width {
-				width = childWidth
-			}
-		}
-		ok = true
-	}
-	return speed, width, ok && speed != ""
-}
-
-func isFullPCIBDF(s string) bool {
-	if len(s) != len("0000:00:00.0") || s[4] != ':' || s[7] != ':' || s[10] != '.' {
-		return false
-	}
-	for i, r := range s {
-		if i == 4 || i == 7 || i == 10 {
-			continue
-		}
-		if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f') || (r >= 'A' && r <= 'F')) {
-			return false
-		}
-	}
-	return true
-}
-
-func minPCIeLinkSpeed(a, b string) string {
-	ga, gb := pcieGeneration(a), pcieGeneration(b)
-	switch {
-	case ga == 0:
-		return b
-	case gb == 0 || ga <= gb:
-		return a
-	default:
-		return b
-	}
-}
-
-func pcieGeneration(speed string) int {
-	v := strings.TrimPrefix(strings.TrimSpace(speed), "Gen")
-	gen, _ := strconv.Atoi(v)
-	return gen
-}
-
-func minPositiveInt(a, b int) int {
-	switch {
-	case a <= 0:
-		return b
-	case b <= 0 || a <= b:
-		return a
-	default:
-		return b
-	}
-}
-
-// classifyGPUFromVendorClass reports whether a device is a GPU die itself
-// (PCI base class 0x03 — Display Controller — under NVIDIA/AMD's vendor
-// ID), as opposed to a same-vendor companion device (NIC, storage
-// controller, NVLink bridge) that shares the GPU's PCI vendor ID. Class-code
-// based, not name-substring based, per the same reasoning as
-// collector.IsGPUClass.
-func classifyGPUFromVendorClass(vendorHex, classHex string) (isGPU bool, vendor string) {
-	v := strings.TrimPrefix(strings.ToLower(strings.TrimSpace(vendorHex)), "0x")
-	c := strings.TrimPrefix(strings.ToLower(strings.TrimSpace(classHex)), "0x")
-	if len(c) < 2 || c[:2] != "03" {
-		return false, ""
-	}
-	switch v {
-	case "10de":
-		return true, "nvidia"
-	case "1002":
-		return true, "amd"
-	default:
-		return false, ""
-	}
-}
-
-func renderPCIeLinkCheckSummary(findings []pcieLinkFinding) string {
-	var b strings.Builder
-	fmt.Fprintf(&b, "run_at_utc=%s\n", time.Now().UTC().Format(time.RFC3339))
-	fmt.Fprintf(&b, "devices_tested=%d\n", len(findings))
-
-	gpuStatus := map[string]string{} // vendor -> OK/FAILED
-	otherDegraded := 0
-	otherTested := 0
-	anyDegraded := false
-	for _, f := range findings {
-		if f.Skipped != "" && f.AfterSpeed == "" {
-			continue
-		}
-		if f.IsGPU {
-			if _, ok := gpuStatus[f.GPUVendor]; !ok {
-				gpuStatus[f.GPUVendor] = "OK"
-			}
-			if f.Degraded {
-				gpuStatus[f.GPUVendor] = "FAILED"
-				anyDegraded = true
-			}
-			continue
-		}
-		otherTested++
-		if f.Degraded {
-			otherDegraded++
-			anyDegraded = true
-		}
-	}
-	for _, vendor := range []string{"nvidia", "amd"} {
-		if status, ok := gpuStatus[vendor]; ok {
-			fmt.Fprintf(&b, "gpu_%s_status=%s\n", vendor, status)
-		}
-	}
-	fmt.Fprintf(&b, "other_devices_tested=%d\n", otherTested)
-	fmt.Fprintf(&b, "other_devices_degraded=%d\n", otherDegraded)
-	if otherTested > 0 {
-		if otherDegraded > 0 {
-			fmt.Fprintln(&b, "other_status=FAILED")
-		} else {
-			fmt.Fprintln(&b, "other_status=OK")
-		}
-	}
-
-	if anyDegraded {
-		fmt.Fprintln(&b, "overall_status=FAILED")
-		var reasons []string
-		for _, f := range findings {
-			if !f.Degraded {
-				continue
-			}
-			if f.NotPresent {
-				reasons = append(reasons, fmt.Sprintf("%s (%s): no device detected downstream (link down / empty slot or riser, capable of %s)",
-					f.BDF, nonEmptyOr(f.Description, "unknown device"), f.MaxSpeed))
-				continue
-			}
-			reasons = append(reasons, fmt.Sprintf("%s (%s): retrained to %s, capable of %s",
-				f.BDF, nonEmptyOr(f.Description, "unknown device"), f.AfterSpeed, f.MaxSpeed))
-		}
-		fmt.Fprintf(&b, "warnings=%s\n", strings.Join(reasons, "; "))
-	} else {
-		fmt.Fprintln(&b, "overall_status=OK")
-	}
-	return b.String()
-}
-
-func renderPCIeLinkCheckReport(findings []pcieLinkFinding) string {
-	var b strings.Builder
-	line := strings.Repeat("=", 80)
-	b.WriteString(line + "\n")
-	b.WriteString("PCIe Link Retrain Check\n")
-	b.WriteString(line + "\n\n")
-	for _, f := range findings {
-		fmt.Fprintf(&b, "%s  %s\n", f.BDF, nonEmptyOr(f.Description, "(unknown device)"))
-		if f.Skipped != "" && f.AfterSpeed == "" {
-			fmt.Fprintf(&b, "    skipped: %s\n", f.Skipped)
-			continue
-		}
-		verdict := "OK"
-		switch {
-		case f.NotPresent:
-			verdict = "FELL OFF BUS"
-		case f.Degraded:
-			verdict = "DEGRADED"
-		}
-		fmt.Fprintf(&b, "    %s: before=%s after=%s max=%s width=%d/%d",
-			verdict, f.BeforeSpeed, f.AfterSpeed, f.MaxSpeed, f.Width, f.MaxWidth)
-		if f.PortMaxSpeed != "" && (f.PortMaxSpeed != f.MaxSpeed || f.PortMaxWidth != f.MaxWidth) {
-			fmt.Fprintf(&b, " (port capability %s x%d, limited by downstream device)", f.PortMaxSpeed, f.PortMaxWidth)
-		}
-		fmt.Fprintln(&b)
-		if f.Skipped != "" {
-			fmt.Fprintf(&b, "    note: %s\n", f.Skipped)
-		}
-	}
-	return b.String()
-}

audit/internal/platform/pcie_link_check_test.go

@@ -1,192 +0,0 @@
-package platform
-
-import (
-	"os"
-	"path/filepath"
-	"strings"
-	"testing"
-)
-
-func TestClassifyGPUFromVendorClass(t *testing.T) {
-	cases := []struct {
-		name       string
-		vendor     string
-		class      string
-		wantIsGPU  bool
-		wantVendor string
-	}{
-		{"nvidia GPU", "0x10de", "0x030200", true, "nvidia"},
-		{"amd GPU", "0x1002", "0x030000", true, "amd"},
-		{"nvidia NIC (same vendor, not display class)", "0x10de", "0x020000", false, ""},
-		{"intel NIC", "0x8086", "0x020000", false, ""},
-		{"unrelated display-class vendor", "0x1234", "0x030000", false, ""},
-	}
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			gotIsGPU, gotVendor := classifyGPUFromVendorClass(tc.vendor, tc.class)
-			if gotIsGPU != tc.wantIsGPU || gotVendor != tc.wantVendor {
-				t.Fatalf("classifyGPUFromVendorClass(%q,%q) = (%v,%q), want (%v,%q)",
-					tc.vendor, tc.class, gotIsGPU, gotVendor, tc.wantIsGPU, tc.wantVendor)
-			}
-		})
-	}
-}
-
-func TestPCIeBridgeUsesDownstreamCapability(t *testing.T) {
-	cases := []struct {
-		name          string
-		portSpeed     string
-		portWidth     int
-		endpointSpeed string
-		endpointWidth int
-		wantSpeed     string
-		wantWidth     int
-	}{
-		{"ConnectX-5 behind Gen4 root port", "Gen4", 16, "Gen3", 8, "Gen3", 8},
-		{"Adaptec SAS behind Gen4 root port", "Gen4", 16, "Gen3", 8, "Gen3", 8},
-		{"I350 behind Gen4 root port", "Gen4", 16, "Gen2", 4, "Gen2", 4},
-		{"faster endpoint remains port-limited", "Gen3", 8, "Gen4", 16, "Gen3", 8},
-	}
-
-	for _, tc := range cases {
-		t.Run(tc.name, func(t *testing.T) {
-			gotSpeed := minPCIeLinkSpeed(tc.portSpeed, tc.endpointSpeed)
-			gotWidth := minPositiveInt(tc.portWidth, tc.endpointWidth)
-			if gotSpeed != tc.wantSpeed || gotWidth != tc.wantWidth {
-				t.Fatalf("effective capability = %s x%d, want %s x%d", gotSpeed, gotWidth, tc.wantSpeed, tc.wantWidth)
-			}
-		})
-	}
-}
-
-func TestPCIeBridgeAtEndpointMaximumPasses(t *testing.T) {
-	maxSpeed := minPCIeLinkSpeed("Gen4", "Gen3")
-	maxWidth := minPositiveInt(16, 8)
-	finding := pcieLinkFinding{
-		BDF:          "0000:4a:02.0",
-		Description:  "Intel root port to ConnectX-5",
-		BeforeSpeed:  "Gen3",
-		AfterSpeed:   "Gen3",
-		MaxSpeed:     maxSpeed,
-		PortMaxSpeed: "Gen4",
-		Width:        8,
-		MaxWidth:     maxWidth,
-		PortMaxWidth: 16,
-		Degraded:     "Gen3" != maxSpeed,
-	}
-
-	summary := renderPCIeLinkCheckSummary([]pcieLinkFinding{finding})
-	if !strings.Contains(summary, "overall_status=OK") {
-		t.Fatalf("endpoint-limited bridge should pass, got:\n%s", summary)
-	}
-	report := renderPCIeLinkCheckReport([]pcieLinkFinding{finding})
-	if !strings.Contains(report, "limited by downstream device") {
-		t.Fatalf("report should explain the effective maximum, got:\n%s", report)
-	}
-}
-
-func TestDownstreamPCIeLinkCapabilityFromSysfsTopology(t *testing.T) {
-	bridgeDir := t.TempDir()
-	childDir := filepath.Join(bridgeDir, "0000:4b:00.0")
-	if err := os.Mkdir(childDir, 0755); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(filepath.Join(childDir, "max_link_speed"), []byte("8.0 GT/s PCIe\n"), 0644); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(filepath.Join(childDir, "max_link_width"), []byte("8\n"), 0644); err != nil {
-		t.Fatal(err)
-	}
-	// A non-BDF sysfs entry must not influence peer discovery.
-	if err := os.WriteFile(filepath.Join(bridgeDir, "max_link_speed"), []byte("16.0 GT/s PCIe\n"), 0644); err != nil {
-		t.Fatal(err)
-	}
-
-	speed, width, ok := downstreamPCIeLinkCapabilityAt(bridgeDir)
-	if !ok || speed != "Gen3" || width != 8 {
-		t.Fatalf("downstream capability = (%q, %d, %v), want (Gen3, 8, true)", speed, width, ok)
-	}
-}
-
-func TestRenderPCIeLinkCheckSummaryDegradedGPU(t *testing.T) {
-	findings := []pcieLinkFinding{
-		{BDF: "0000:0d:00.0", Description: "NVIDIA GPU", IsGPU: true, GPUVendor: "nvidia",
-			BeforeSpeed: "Gen1", AfterSpeed: "Gen1", MaxSpeed: "Gen5", Degraded: true},
-		{BDF: "0000:37:00.0", Description: "NVIDIA GPU", IsGPU: true, GPUVendor: "nvidia",
-			BeforeSpeed: "Gen1", AfterSpeed: "Gen5", MaxSpeed: "Gen5", Degraded: false},
-		{BDF: "0000:01:00.0", Description: "Mellanox NIC", IsGPU: false,
-			BeforeSpeed: "Gen3", AfterSpeed: "Gen4", MaxSpeed: "Gen4", Degraded: false},
-	}
-	summary := renderPCIeLinkCheckSummary(findings)
-	if !strings.Contains(summary, "gpu_nvidia_status=FAILED") {
-		t.Fatalf("expected gpu_nvidia_status=FAILED (one degraded GPU should fail the vendor group), got:\n%s", summary)
-	}
-	if !strings.Contains(summary, "other_status=OK") {
-		t.Fatalf("expected other_status=OK, got:\n%s", summary)
-	}
-	if !strings.Contains(summary, "overall_status=FAILED") {
-		t.Fatalf("expected overall_status=FAILED, got:\n%s", summary)
-	}
-}
-
-func TestRenderPCIeLinkCheckSummaryAllClean(t *testing.T) {
-	findings := []pcieLinkFinding{
-		{BDF: "0000:0d:00.0", IsGPU: true, GPUVendor: "nvidia", BeforeSpeed: "Gen1", AfterSpeed: "Gen5", MaxSpeed: "Gen5"},
-		{BDF: "0000:2c:00.0", Skipped: "device disabled (no data traffic; link state has no operational impact)"},
-	}
-	summary := renderPCIeLinkCheckSummary(findings)
-	if !strings.Contains(summary, "overall_status=OK") {
-		t.Fatalf("expected overall_status=OK, got:\n%s", summary)
-	}
-	if strings.Contains(summary, "other_status=") {
-		t.Fatalf("disabled/skipped-only device should not produce an other_status line, got:\n%s", summary)
-	}
-}
-
-func TestRenderPCIeLinkCheckSummaryEmptySlotDoesNotFail(t *testing.T) {
-	// An unpopulated switch downstream port (no card ever seated) is a
-	// normal, common configuration — it must not fail the SAT the way a
-	// genuinely degraded link does.
-	findings := []pcieLinkFinding{
-		{BDF: "0000:0d:00.0", IsGPU: true, GPUVendor: "nvidia", BeforeSpeed: "Gen5", AfterSpeed: "Gen5", MaxSpeed: "Gen5"},
-		{BDF: "0000:2a:00.0", Description: "PEX890xx PCIe Gen 5 Switch",
-			NotPresent: true, Skipped: "no device present downstream (empty slot/riser — nothing to retrain)"},
-	}
-	summary := renderPCIeLinkCheckSummary(findings)
-	if !strings.Contains(summary, "overall_status=OK") {
-		t.Fatalf("expected overall_status=OK for an empty slot, got:\n%s", summary)
-	}
-	if strings.Contains(summary, "other_status=") {
-		t.Fatalf("empty-slot-only device should not produce an other_status line, got:\n%s", summary)
-	}
-
-	report := renderPCIeLinkCheckReport(findings)
-	if !strings.Contains(report, "skipped: no device present downstream") {
-		t.Fatalf("expected empty slot to be reported as skipped, not degraded, got:\n%s", report)
-	}
-	if strings.Contains(report, "retrained to") {
-		t.Fatalf("empty slot must not be reported with a fabricated retrain speed, got:\n%s", report)
-	}
-}
-
-func TestRenderPCIeLinkCheckSummaryDeviceFellOffBusStillFails(t *testing.T) {
-	// A device that had a live link before the retrain and is gone right
-	// after it is a real regression, distinct from a slot that was never
-	// populated — this must still fail the check.
-	findings := []pcieLinkFinding{
-		{BDF: "0000:ab:00.0", Description: "PEX890xx PCIe Gen 5 Switch",
-			BeforeSpeed: "Gen5", AfterSpeed: "Gen5", MaxSpeed: "Gen5", Width: 0, NotPresent: true, Degraded: true},
-	}
-	summary := renderPCIeLinkCheckSummary(findings)
-	if !strings.Contains(summary, "overall_status=FAILED") {
-		t.Fatalf("expected overall_status=FAILED when a device disappears after retrain, got:\n%s", summary)
-	}
-	if !strings.Contains(summary, "no device detected downstream") {
-		t.Fatalf("expected an honest 'no device detected' reason, not a fabricated speed, got:\n%s", summary)
-	}
-
-	report := renderPCIeLinkCheckReport(findings)
-	if !strings.Contains(report, "FELL OFF BUS") {
-		t.Fatalf("expected FELL OFF BUS verdict in report, got:\n%s", report)
-	}
-}

audit/internal/platform/sat.go

@@ -197,6 +197,10 @@ type satJob struct {
 	name string
 	cmd  []string
 	env  []string // extra env vars (appended to os.Environ)
+	// afterRun executes immediately after the command exits, before health
+	// probes or artifact processing can let a briefly boosted link return to
+	// its idle power state.
+	afterRun func()
 	// validate checks successful command output against the tool's documented
 	// result format. It may inspect artifacts from earlier jobs in runDir.
 	// FAILED means the tool proved a test failure; UNSUPPORTED means the pinned
@@ -373,6 +377,9 @@ func runAcceptancePackCtx(ctx context.Context, baseDir, prefix string, jobs []sa
 				}
 				time.Sleep(2 * time.Second)
 			}
+			if job.afterRun != nil {
+				job.afterRun()
+			}
 		}
 
 		if nvidiaPack && nvidiaJobNeedsHealthCheck(job) {

audit/internal/platform/sat_gpu_packs.go

@@ -451,6 +451,10 @@ func (s *System) RunNvidiaBandwidthPack(ctx context.Context, baseDir string, gpu
 		return "", err
 	}
 	killStaleNvidiaTestWorkers(logFunc)
+	linkFindings, err := captureNvidiaPCIeLinkBaseline(selected)
+	if err != nil {
+		return "", err
+	}
 	jobs := []satJob{
 		{name: "01-nvidia-smi-q.log", cmd: []string{"nvidia-smi", "-q"}},
 		{name: "02-dcgmi-discovery.log", cmd: []string{"dcgmi", "discovery", "-l"}, informational: true, retries: 2},
@@ -476,6 +480,7 @@ func (s *System) RunNvidiaBandwidthPack(ctx context.Context, baseDir string, gpu
 			collectGPU:  true,
 			gpuIndices:  selected,
 			syncBracket: true,
+			afterRun:    func() { sampleNvidiaPCIeLinkAfterLoad(linkFindings) },
 		})
 		step++
 	} else {
@@ -495,15 +500,17 @@ func (s *System) RunNvidiaBandwidthPack(ctx context.Context, baseDir string, gpu
 			collectGPU:  true,
 			gpuIndices:  selected,
 			syncBracket: true,
+			afterRun:    func() { sampleNvidiaPCIeLinkAfterLoad(linkFindings) },
 		})
-		step++
 	}
-	jobs = append(jobs, satJob{
-		name: fmt.Sprintf("%02d-nvidia-smi-after.log", step),
-		cmd:  []string{"nvidia-smi", "--query-gpu=index,name,temperature.gpu,power.draw,utilization.gpu,memory.used,memory.total", "--format=csv,noheader,nounits"},
-	})
-
-	return runAcceptancePackCtx(ctx, baseDir, "gpu-nvidia-bandwidth", withNvidiaPersistenceMode(jobs...), logFunc)
+	runDir, err := runAcceptancePackCtx(ctx, baseDir, "gpu-nvidia-bandwidth", withNvidiaPersistenceMode(jobs...), logFunc)
+	if err != nil {
+		return "", err
+	}
+	if err := finishNvidiaPCIeLinkCheck(runDir, linkFindings); err != nil {
+		return "", err
+	}
+	return runDir, nil
 }
 
 func (s *System) RunNvidiaAcceptancePack(baseDir string, logFunc func(string)) (string, error) {

audit/internal/webui/api.go

@@ -132,7 +132,7 @@ func defaultTaskPriority(target string, params taskParams) int {
 		return taskPriorityBurn
 	case "nvidia", "nvidia-targeted-stress", "nvidia-targeted-power", "nvidia-pulse",
 		"nvidia-interconnect", "nvidia-bandwidth", "memory", "storage", "cpu",
-		"amd", "amd-mem", "amd-bandwidth", "nvidia-config", "pcie-link", "nvidia-pcie-bandwidth":
+		"amd", "amd-mem", "amd-bandwidth", "nvidia-config":
 		if params.StressMode {
 			return taskPriorityValidateStress
 		}

audit/internal/webui/api_sat_runall.go

@@ -101,7 +101,6 @@ func (h *handler) planSATRunAll(ctx context.Context, req satRunAllRequest) ([]sa
 		satRunAllSpec{target: "cpu", params: taskParams{Duration: cpuDur, StressMode: req.StressMode}},
 		satRunAllSpec{target: "memory", params: taskParams{StressMode: req.StressMode}},
 		satRunAllSpec{target: "storage", params: taskParams{StressMode: req.StressMode}},
-		satRunAllSpec{target: "pcie-link", params: taskParams{}},
 	)
 
 	if h.opts.App.TPMPresent() {
@@ -139,7 +138,7 @@ func (h *handler) planSATRunAll(ctx context.Context, req satRunAllRequest) ([]sa
 			if !health.CUDAReady {
 				notes = append(notes, "NVIDIA: CUDA runtime not confirmed ready; GPU tests queued anyway")
 			}
-			gpuTargets := []string{"nvidia", "nvidia-interconnect", "nvidia-bandwidth", "nvidia-pcie-bandwidth"}
+			gpuTargets := []string{"nvidia", "nvidia-interconnect", "nvidia-bandwidth"}
 			if req.StressMode {
 				// Stress tier adds the targeted dcgmi diag load tests.
 				gpuTargets = append(gpuTargets, "nvidia-targeted-stress", "nvidia-targeted-power", "nvidia-pulse")

audit/internal/webui/api_sat_runall_test.go

@@ -80,7 +80,7 @@ func TestPlanSATRunAllNoAcceleratorNoTPM(t *testing.T) {
 	for _, s := range specs {
 		targets = append(targets, s.target)
 	}
-	want := []string{"cpu", "memory", "storage", "pcie-link"}
+	want := []string{"cpu", "memory", "storage"}
 	if !reflect.DeepEqual(targets, want) {
 		t.Fatalf("targets=%v want %v", targets, want)
 	}

audit/internal/webui/page_validate.go

@@ -189,11 +189,11 @@ func renderValidateMode(opts HandlerOptions, stressDefault bool) string {
 			`<code>all_reduce_perf</code> (NCCL tests)`,
 			validateFmtDur(platform.SATEstimatedNvidiaInterconnectSec)+` (all GPUs simultaneously, requires ≥2).`,
 		)) +
-		renderSATCard("nvidia-bandwidth", "NVIDIA Bandwidth (NVBandwidth)", "runNvidiaFabricValidate('nvidia-bandwidth')", "", renderValidateCardBody(
+		renderSATCard("nvidia-bandwidth", "NVIDIA Bandwidth + PCIe Link", "runNvidiaFabricValidate('nvidia-bandwidth')", "", renderValidateCardBody(
 			inv.NVIDIA,
-			`Validates GPU memory copy and peer-to-peer bandwidth paths using NVBandwidth.`,
-			`<code>nvbandwidth</code>`,
-			validateFmtDur(platform.SATEstimatedNvidiaBandwidthSec)+` (all GPUs simultaneously; nvbandwidth runs all built-in tests without a time limit — duration set by the tool).`,
+			`Validates GPU memory copy and peer-to-peer bandwidth paths, then samples each GPU's negotiated PCIe speed and width immediately after real traffic. Idle or forced-retrain Gen1 readings do not fail the test.`,
+			`<code>dcgmi diag -r nvbandwidth</code>, sysfs PCIe link attributes`,
+			validateFmtDur(platform.SATEstimatedNvidiaBandwidthSec)+` (all GPUs simultaneously; nvbandwidth runs all built-in tests without a time limit - duration set by the tool).`,
 		)) +
 		`</div>
 <div class="grid3" style="margin-top:16px">
@@ -669,18 +669,6 @@ func renderCheck(opts HandlerOptions) string {
 			`<code>nvidia-smi --query-gpu=...</code>, <code>nvidia-smi topo -m</code>, <code>nvidia-smi nvlink -s/-e</code>, <code>nvidia-smi conf-compute -q</code>, <code>dmesg</code>`,
 			`Seconds — read-only query only.`,
 		)) +
-		renderSATCard("pcie-link", "PCIe Link Check", "runSAT('pcie-link')", "", renderValidateCardBody(
-			`Every enabled PCIe device in the machine (GPUs, NICs, RAID/HBA controllers, PCIe switches).`,
-			`Forces every enabled PCIe device to retrain its link (PCIe spec Link Control "Retrain Link" bit) and compares the negotiated speed against the device's own maximum. An idle sysfs reading alone can't tell a real degraded slot/riser from a device that's simply power-managed down at idle (GPUs do this routinely); a forced retrain settles that without needing a device-specific load generator, so this one check covers non-GPU PCIe hardware too, not just GPUs.`,
-			`<code>setpci</code> (Link Control/Link Status registers), <code>lspci</code>`,
-			`Seconds per device — brief link retrain, no sustained traffic.`,
-		)) +
-		renderSATCard("nvidia-pcie-bandwidth", "GPU PCIe Bandwidth", "runSAT('nvidia-pcie-bandwidth')", "", renderValidateCardBody(
-			inv.NVIDIA,
-			`Drives real host&lt;-&gt;device traffic across each GPU's PCIe link and resamples link speed immediately after, to confirm the link actually trains to its negotiated maximum under real load — the load-bearing counterpart to PCIe Link Check for GPUs specifically.`,
-			`<code>dcgmi diag -r nvbandwidth</code>, sysfs link-speed resample`,
-			`Depends on nvbandwidth's built-in test duration.`,
-		)) +
 		`</div>
 <div style="height:1px;background:var(--border);margin:16px 0"></div>
 <div class="card" style="margin-bottom:16px">
@@ -711,10 +699,10 @@ func renderCheck(opts HandlerOptions) string {
 			`<code>all_reduce_perf</code> (NCCL tests)`,
 			validateFmtDur(platform.SATEstimatedNvidiaInterconnectSec)+` (all GPUs simultaneously, requires ≥2).`,
 		)) +
-		renderSATCard("nvidia-bandwidth", "NVIDIA Bandwidth (NVBandwidth)", "runNvidiaFabricValidate('nvidia-bandwidth')", "", renderValidateCardBody(
+		renderSATCard("nvidia-bandwidth", "NVIDIA Bandwidth + PCIe Link", "runNvidiaFabricValidate('nvidia-bandwidth')", "", renderValidateCardBody(
 			inv.NVIDIA,
-			`Validates GPU memory copy and peer-to-peer bandwidth paths using NVBandwidth.`,
-			`<code>nvbandwidth</code>`,
+			`Validates GPU memory copy and peer-to-peer bandwidth paths, then samples each GPU's negotiated PCIe speed and width immediately after real traffic.`,
+			`<code>dcgmi diag -r nvbandwidth</code>, sysfs PCIe link attributes`,
 			validateFmtDur(platform.SATEstimatedNvidiaBandwidthSec)+` (all GPUs simultaneously).`,
 		)) +
 		`</div>
@@ -742,7 +730,7 @@ func renderCheck(opts HandlerOptions) string {
 <script>
 let satES = null;
 function satLabels() {
-  return {nvidia:'Check GPU (DCGM L2)', 'nvidia-interconnect':'NVIDIA Interconnect (NCCL)', 'nvidia-bandwidth':'NVIDIA Bandwidth (NVBandwidth)', memory:'Check Memory', storage:'Check Storage', tpm:'Check TPM (read-only)', cpu:'Check CPU', amd:'Check AMD GPU', 'amd-mem':'AMD GPU MEM Integrity', 'amd-bandwidth':'AMD GPU MEM Bandwidth', 'nvidia-config':'Check GPU Config & NVLink', 'pcie-link':'PCIe Link Check', 'nvidia-pcie-bandwidth':'GPU PCIe Bandwidth Check'};
+  return {nvidia:'Check GPU (DCGM L2)', 'nvidia-interconnect':'NVIDIA Interconnect (NCCL)', 'nvidia-bandwidth':'NVIDIA Bandwidth + PCIe Link Under Load', memory:'Check Memory', storage:'Check Storage', tpm:'Check TPM (read-only)', cpu:'Check CPU', amd:'Check AMD GPU', 'amd-mem':'AMD GPU MEM Integrity', 'amd-bandwidth':'AMD GPU MEM Bandwidth', 'nvidia-config':'Check GPU Config & NVLink'};
 }
 let satNvidiaGPUsPromise = null;
 function loadSatNvidiaGPUs() {

audit/internal/webui/server.go

@@ -264,8 +264,6 @@ func NewHandler(opts HandlerOptions) http.Handler {
 	mux.HandleFunc("POST /api/sat/storage/run", h.handleAPISATRun("storage"))
 	mux.HandleFunc("POST /api/sat/tpm/run", h.handleAPISATRun("tpm"))
 	mux.HandleFunc("POST /api/sat/nvidia-config/run", h.handleAPISATRun("nvidia-config"))
-	mux.HandleFunc("POST /api/sat/pcie-link/run", h.handleAPISATRun("pcie-link"))
-	mux.HandleFunc("POST /api/sat/nvidia-pcie-bandwidth/run", h.handleAPISATRun("nvidia-pcie-bandwidth"))
 	mux.HandleFunc("POST /api/sat/cpu/run", h.handleAPISATRun("cpu"))
 	mux.HandleFunc("POST /api/sat/amd/run", h.handleAPISATRun("amd"))
 	mux.HandleFunc("POST /api/sat/amd-mem/run", h.handleAPISATRun("amd-mem"))

audit/internal/webui/server_test.go

@@ -815,7 +815,7 @@ func TestCheckPageRendersGPUSelectionAndNvidiaCards(t *testing.T) {
 		`Select All`,
 		`id="sat-btn-nvidia"`,
 		`NVIDIA Interconnect (NCCL)`,
-		`NVIDIA Bandwidth (NVBandwidth)`,
+		`NVIDIA Bandwidth + PCIe Link`,
 		`Non-destructive`,
 	} {
 		if !strings.Contains(body, needle) {
@@ -834,7 +834,7 @@ func TestCheckPageRendersNvidiaFabricCards(t *testing.T) {
 	body := rec.Body.String()
 	for _, needle := range []string{
 		`NVIDIA Interconnect (NCCL)`,
-		`NVIDIA Bandwidth (NVBandwidth)`,
+		`NVIDIA Bandwidth + PCIe Link`,
 		`nvbandwidth`,
 		`all_reduce_perf`,
 	} {

audit/internal/webui/task_runner.go

@@ -303,18 +303,6 @@ func executeTaskWithOptions(opts *HandlerOptions, t *Task, j *jobState, ctx cont
 			break
 		}
 		archive, err = runNvidiaConfigCheckPackCtx(a, ctx, "", j.append)
-	case "pcie-link":
-		if a == nil {
-			err = fmt.Errorf("app not configured")
-			break
-		}
-		archive, err = runPCIeLinkCheckPackCtx(a, ctx, "", j.append)
-	case "nvidia-pcie-bandwidth":
-		if a == nil {
-			err = fmt.Errorf("app not configured")
-			break
-		}
-		archive, err = runNvidiaPCIeBandwidthPackCtx(a, ctx, "", t.params.GPUIndices, j.append)
 	case "cpu":
 		if a == nil {
 			err = fmt.Errorf("app not configured")

audit/internal/webui/tasks.go

@@ -45,8 +45,6 @@ var taskNames = map[string]string{
 	"storage":                "Storage SAT",
 	"tpm":                    "TPM Read-only Check",
 	"nvidia-config":          "GPU Config & NVLink Check",
-	"pcie-link":              "PCIe Link Check (forced retrain)",
-	"nvidia-pcie-bandwidth":  "NVIDIA GPU PCIe Bandwidth Check",
 	"cpu":                    "CPU SAT",
 	"amd":                    "AMD GPU SAT",
 	"amd-mem":                "AMD GPU MEM Integrity",
@@ -321,12 +319,6 @@ var (
 	runNvidiaConfigCheckPackCtx = func(a *app.App, ctx context.Context, baseDir string, logFunc func(string)) (string, error) {
 		return a.RunNvidiaConfigCheckPackCtx(ctx, baseDir, logFunc)
 	}
-	runPCIeLinkCheckPackCtx = func(a *app.App, ctx context.Context, baseDir string, logFunc func(string)) (string, error) {
-		return a.RunPCIeLinkCheckPackCtx(ctx, baseDir, logFunc)
-	}
-	runNvidiaPCIeBandwidthPackCtx = func(a *app.App, ctx context.Context, baseDir string, gpuIndices []int, logFunc func(string)) (string, error) {
-		return a.RunNvidiaPCIeBandwidthPackCtx(ctx, baseDir, gpuIndices, logFunc)
-	}
 	runCPUAcceptancePackCtx = func(a *app.App, ctx context.Context, baseDir string, durationSec int, logFunc func(string)) (string, error) {
 		return a.RunCPUAcceptancePackCtx(ctx, baseDir, durationSec, logFunc)
 	}

bible-local/architecture/api-surface.md

@@ -41,7 +41,7 @@ All SAT run endpoints enqueue an async task. Response: `{"task_id": "..."}`.
 | POST   | `/api/sat/nvidia-targeted-power/run`       | NVIDIA targeted power             |
 | POST   | `/api/sat/nvidia-pulse/run`                | NVIDIA pulse test                 |
 | POST   | `/api/sat/nvidia-interconnect/run`         | NCCL all_reduce_perf              |
-| POST   | `/api/sat/nvidia-bandwidth/run`            | NVBandwidth test                  |
+| POST   | `/api/sat/nvidia-bandwidth/run`            | NVBandwidth + PCIe link under load |
 | POST   | `/api/sat/nvidia-stress/run`               | NVIDIA stress pack                |
 | POST   | `/api/sat/memory/run`                      | Memory acceptance                 |
 | POST   | `/api/sat/storage/run`                     | Storage acceptance (smartctl)     |

bible-local/architecture/runtime-flows.md

@@ -285,12 +285,12 @@ Web UI: Acceptance Tests page -> Run Test button
 Web UI: "Run All" button -> POST /api/sat/run-all
   body: operator intent only { stress_mode, amd_targets[], nvidia_gpu_indices[] }
   server (handler.planSATRunAll):
-    1. always: cpu, memory, storage, pcie-link
+    1. always: cpu, memory, storage
     2. tpm            - only if App.TPMPresent() finds tpm_version_major=2
     3. nvidia-config  - if DetectGPUPresence().Nvidia || NvidiaInitializing
     4. wait for NVIDIA enumeration: repeat fresh ListNvidiaGPUs queries until
        at least one GPU is returned, NvidiaGSPMode=="gsp-stuck", or 75s
-    5. nvidia / nvidia-interconnect / nvidia-bandwidth / nvidia-pcie-bandwidth
+    5. nvidia / nvidia-interconnect / nvidia-bandwidth
        (+ targeted-stress/power/pulse when stress_mode) - only once ready,
        -i = App.ListNvidiaGPUs() indices (intersected with the requested subset)
     6. amd / amd-mem / amd-bandwidth - if DetectGPUPresence().AMD and selected
@@ -306,6 +306,9 @@ Web UI: "Run All" button -> POST /api/sat/run-all
   vendor detection plus an lspci display-class fallback).
   `/api/gpu/presence`, `/api/gpu/tools` and the planner all use it.
 - `bee-gpu-burn` / `bee-john-gpu-stress` use `exec.CommandContext`: killed on job context cancel.
+- `nvidia-bandwidth` samples every selected GPU's negotiated PCIe speed and
+  width immediately after its final all-GPU `dcgmi diag -r nvbandwidth`
+  command exits. Idle and forced-retrain readings are not health verdicts.
 - Metric goroutine uses stopCh/doneCh pattern; main goroutine waits `<-doneCh` before reading rows (no mutex needed).
 - SVG chart is fully offline: no JS, no external CSS, pure inline SVG.
 - `RunNvidiaBandwidthPack` runs one all-GPU `nvbandwidth` pass in Validate; the

bible-local/decisions/2026-08-24-pcie-gpu-gen1-idle-warning.md

@@ -1,7 +1,7 @@
 # PCIe Gen1-at-idle GPU warning: history of attempts, and the fix
 
 **Date:** 2026-08-24
-**Status:** active — implemented (see "Resolution" at the end)
+**Status:** superseded in part by `2026-09-03-pcie-link-verdict-under-real-traffic.md`
 
 ## Symptom
 
@@ -116,6 +116,11 @@ Pick one (or combine):
 
 ## Resolution (2026-08-24)
 
+Historical note: the two SAT targets described below were the resolution at
+the time. The 2026-09-03 follow-up removes the forced-retrain target and folds
+the GPU link verdict into the existing bandwidth SAT after field evidence
+showed that retraining without traffic can remain at Gen1.
+
 Landed a variant of options 1–3 that turned out simpler than any of them
 individually once we stopped trying to make the idle reading recoverable:
 

bible-local/decisions/2026-09-03-pcie-link-verdict-under-real-traffic.md

@@ -0,0 +1,51 @@
+# PCIe link verdict comes from real traffic, not forced retraining
+
+**Date:** 2026-09-03
+**Status:** active
+
+## Evidence
+
+On `bee@172.16.41.97`, build `14.02-1-g642e686`, three independent runs of
+`PCIe Link Check (forced retrain)` reported all eight NVIDIA GPUs and their
+eight upstream bridges at Gen1 x16 after retraining, against Gen5 x16
+capability. In the same application history, every `dcgmi diag -r
+nvbandwidth` run brought every GPU to Gen5 x16 and passed. There were no AER
+or NVIDIA Xid events supporting a physical link fault.
+
+This disproves the premise that setting the PCIe Retrain Link bit is a
+generic substitute for device traffic. On this hardware and driver, a link
+can complete a forced retrain and remain in its idle Gen1 state. Counting both
+the endpoint and its upstream bridge also reports one physical link twice.
+
+## Decision
+
+- Remove the standalone `pcie-link` forced-retrain SAT from code, API, UI,
+  and Run All planning.
+- Remove the separate `nvidia-pcie-bandwidth` SAT. It duplicated the same
+  approximately 45-minute `nvbandwidth` workload already run by
+  `nvidia-bandwidth`.
+- Extend `nvidia-bandwidth` to record a sysfs baseline before the test and
+  sample negotiated speed and width synchronously when the final all-GPU
+  `dcgmi diag -r nvbandwidth` command exits. The sample runs before later
+  health probes and artifact processing can let the link return to idle.
+- Append `pcie_link_under_load_status`, per-GPU PCIe status, and the degraded
+  count to the existing summary. A supported GPU that is still below its own
+  maximum speed or width after real traffic fails the overall SAT. Missing
+  sysfs attributes produce `UNSUPPORTED`, not a false failure.
+- Keep non-GPU PCIe inspection read-only and diagnostic. Without a
+  device-specific traffic generator, current speed is inventory rather than
+  a health verdict.
+- When a new `nvidia-bandwidth` result is recorded, recompute the NVIDIA GPU
+  component status while ignoring the retired `sat:pcie-link` and
+  `sat:nvidia-pcie-bandwidth` sources. Their history remains auditable. Other
+  failures, such as `nvidia-config` or Xid-derived warnings, remain sticky.
+
+## Consequences
+
+Run All performs one fewer short but misleading test and one fewer duplicate
+long GPU bandwidth test. A Gen1 idle baseline no longer alarms when the same
+link reaches Gen5 x16 under traffic. A genuine under-load speed or width
+downgrade still fails the bandwidth SAT and reaches `pcie:gpu:nvidia`.
+
+Old task directories and component history remain readable. The removed API
+targets are no longer accepted for new tasks.

bible-local/decisions/README.md

@@ -11,8 +11,9 @@ One file per decision, named `YYYY-MM-DD-short-topic.md`.
 | 2026-07-27 | Split the NVIDIA Bandwidth SAT into per-socket passes before the all-GPU pass | active |
 | 2026-07-27 | Stream SAT job output live to disk and kick blackbox sync on job completion | active |
 | 2026-07-28 | Move pci=realloc out of the default/toram/no-GUI GRUB entries | active |
-| 2026-08-24 | PCIe Gen1-at-idle GPU warning: load-bearing link check, not idle sysfs | active |
+| 2026-08-24 | PCIe Gen1-at-idle GPU warning: load-bearing link check, not idle sysfs | superseded in part |
 | 2026-08-31 | nvbandwidth: single all-GPU pass in Validate, per-socket matrix only in deep tier | active |
 | 2026-08-31 | bee-nvidia.service: never blocking `systemctl restart` on units ordered After= itself | active |
 | 2026-08-31 | "Run All" SAT planning happens on the backend, not the browser | active |
 | 2026-08-31 | Support bundle uses private staging and unique atomic output | active |
+| 2026-09-03 | PCIe link verdict comes only from the existing real-traffic GPU bandwidth SAT | active |