diff --git a/bible-local/02-architecture.md b/bible-local/02-architecture.md index e2735f5..0b9c06a 100644 --- a/bible-local/02-architecture.md +++ b/bible-local/02-architecture.md @@ -26,10 +26,17 @@ internal/analyzer/ shared analysis helpers internal/parser/ archive extraction and parser dispatch internal/exporter/ CSV and Reanimator conversion internal/chart/ vendored `reanimator/chart` viewer submodule +internal/topology/ standalone topology.json builder and static SVG renderer internal/models/ stable data contracts web/ embedded UI assets ``` +The topology view is owned by LOGPile, not by the chart submodule. The main UI +opens `GET /topology/current` in a separate browser tab. The renderer consumes a +standalone `topology.json` document built from the normalized Reanimator +projection plus optional static evidence imported from Bee support bundles. It +does not poll telemetry or execute host commands. + ## Server state `internal/server.Server` stores: diff --git a/bible-local/03-api.md b/bible-local/03-api.md index c0697b7..463dc8d 100644 --- a/bible-local/03-api.md +++ b/bible-local/03-api.md @@ -201,6 +201,17 @@ The server first converts the current result to Reanimator JSON, then passes tha Serves embedded `reanimator/chart` static assets. +### `GET /topology/current` + +Renders the current server topology as a standalone static HTML/SVG page in a +separate browser tab. Rich Bee bundles add DIMM, storage-controller, and NVLink +edges from persisted diagnostic files. No live commands or polling are used. + +### `GET /api/topology` + +Returns the versioned semantic `topology.json` document. It contains nodes and +relationships, not SVG coordinates. + ## Export endpoints ### `GET /api/export/csv` @@ -215,6 +226,7 @@ Current implementation emits a ZIP bundle containing: - `collect.log` - `parser_fields.json` - `privacy_report.json` (only when the customer-data scan produced findings) +- `topology.json` (when topology nodes are available) ### `GET /api/export/reanimator` diff --git a/bible-local/04-data-models.md b/bible-local/04-data-models.md index 033ca97..c9a63e7 100644 --- a/bible-local/04-data-models.md +++ b/bible-local/04-data-models.md @@ -58,6 +58,17 @@ Required rules: 3. UI/export mismatches are bugs, not accepted divergence 4. New shared device fields belong in `HardwareDevice` first +### NUMA affinity + +NUMA affinity is nullable throughout the internal and Reanimator export path. +In Go it is represented as `*int`: `nil` means not reported / not applicable, +while a pointer to `0` means the real NUMA node 0. Numeric zero must never be +used as an absence sentinel. + +Topology relationships outside the Reanimator ingest contract live in the +separate versioned `topology.json` document and are not added to +`reanimator.json`. + Deduplication priority: | Priority | Key | diff --git a/bible-local/06-parsers.md b/bible-local/06-parsers.md index 8fa93e2..40513f6 100644 --- a/bible-local/06-parsers.md +++ b/bible-local/06-parsers.md @@ -59,7 +59,7 @@ serial from its socket, model, board serial, or another component; reject source | Vendor ID | Input family | Notes | |-----------|--------------|-------| | `dell` | TSR ZIP archives | Broad hardware, firmware, sensors, lifecycle events | -| `easy_bee` | `bee-support-*.tar.gz` | Imports embedded `export/bee-audit.json` snapshot from reanimator-easy-bee bundles | +| `easy_bee` | `bee-support-*.tar.gz` | Imports the embedded hardware snapshot and static DIMM/storage/NVIDIA evidence used to build `topology.json` | | `h3c_g5` | H3C SDS G5 bundles | INI/XML/CSV-driven hardware and event parsing | | `h3c_g6` | H3C SDS G6 bundles | Similar flow with G6-specific files | | `hpe_ilo_ahs` | HPE iLO Active Health System (`.ahs`) | Proprietary `ABJR` container with gzip-compressed `zbb` members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON | diff --git a/bible-local/10-decisions.md b/bible-local/10-decisions.md index c74cf8a..45bcb12 100644 --- a/bible-local/10-decisions.md +++ b/bible-local/10-decisions.md @@ -572,6 +572,8 @@ both CPUs or have no CPU affinity). This data is needed for topology-aware diagn provide CPUAffinity at all. - `TestParseDellCPUAffinity` verifies numeric values parsed correctly and "Not Applicable"→0. +The zero-as-absence representation in this decision is superseded by ADL-068. + --- ## ADL-023 — Reanimator export must match ingest contract exactly @@ -1983,3 +1985,38 @@ with **same-length neutral fillers**, in place. `TestSanitize_Zip_*`, `TestSanitize_BinaryMemberFlagged`, `TestNeutralZonesAreValidAndSameLength`, `TestTZFiller`), `internal/server` (`TestHandleSanitize_*`). + +--- + +## ADL-068 — Standalone topology document and nullable NUMA affinity + +**Date:** 2026-09-14 +**Context:** A server topology cannot distinguish the real NUMA node 0 from an +unknown affinity while the Go model uses numeric zero as its absence sentinel. +Bee support bundles also contain useful relationships (DIMM-to-socket, +disk-to-controller, and GPU/NVLink links) which are not part of the Reanimator +hardware ingest schema. Adding those relationships to `reanimator.json` would +couple a standard inventory document to one viewer. + +**Decision:** +- Represent `numa_node` as `*int` throughout LOGPile's PCIe, network, canonical + device, and Reanimator projection models. `nil` means absent/N/A; `&0` means + NUMA node 0. +- Keep `reanimator.json` schema-preserving. Store semantic topology as a + separate, versioned `topology.json` document containing nodes and edges, but + no presentation coordinates. +- Render `/topology/current` as a LOGPile-owned, static SVG page opened in a + separate browser tab. Do not modify the `internal/chart` submodule. +- The Easy Bee parser may retain the small static evidence files required to + enrich topology. Rendering must never execute commands, poll metrics, or + depend on live state. + +**Consequences:** +- JSON can round-trip both an absent NUMA affinity and the real value 0. +- Ordinary inputs get the topology supported by their normalized inventory; + rich Bee bundles additionally get memory, storage-controller, and NVLink + relationships. +- Raw-export ZIP bundles include `topology.json` separately from + `reanimator.json`. +- Fan RPM animation, PSU live load, and other Bee live-only presentation are + intentionally out of scope. diff --git a/bible-local/docs/hardware-ingest-contract.md b/bible-local/docs/hardware-ingest-contract.md index ceae3f3..bf1749c 100644 --- a/bible-local/docs/hardware-ingest-contract.md +++ b/bible-local/docs/hardware-ingest-contract.md @@ -477,6 +477,9 @@ GET /ingest/hardware/jobs/{job_id} | + общие поля статуса | | | см. раздел выше | `numa_node` передавайте для NIC / InfiniBand / RAID / GPU, когда источник знает CPU/NUMA affinity. Поле сохраняется в snapshot-атрибутах PCIe-компонента и дублируется в telemetry для topology use cases. + +`numa_node: 0` is a valid affinity value. Unknown or not-applicable affinity is +represented by an absent field, never by numeric zero. Поля `temperature_c` и `power_w` используйте для device-level telemetry GPU / accelerator / smart PCIe devices. Они не влияют на идентификацию компонента. `vendor_id`/`device_id` используются в первую очередь при разрешении канонического вендора/модели устройства (см. аналогичное примечание в разделе `storage`). diff --git a/internal/exporter/reanimator_converter.go b/internal/exporter/reanimator_converter.go index edd509a..c024510 100644 --- a/internal/exporter/reanimator_converter.go +++ b/internal/exporter/reanimator_converter.go @@ -678,6 +678,9 @@ func mergeCanonicalDevice(primary, secondary models.HardwareDevice) models.Hardw fillString(&primary.LinkSpeed, secondary.LinkSpeed) fillInt(&primary.MaxLinkWidth, secondary.MaxLinkWidth) fillString(&primary.MaxLinkSpeed, secondary.MaxLinkSpeed) + if primary.NUMANode == nil && secondary.NUMANode != nil { + primary.NUMANode = secondary.NUMANode + } fillInt(&primary.WattageW, secondary.WattageW) fillString(&primary.InputType, secondary.InputType) fillInt(&primary.InputPowerW, secondary.InputPowerW) diff --git a/internal/exporter/reanimator_converter_test.go b/internal/exporter/reanimator_converter_test.go index 09eba20..2ebc096 100644 --- a/internal/exporter/reanimator_converter_test.go +++ b/internal/exporter/reanimator_converter_test.go @@ -9,6 +9,33 @@ import ( "git.mchus.pro/mchus/logpile/internal/models" ) +func intPtr(v int) *int { return &v } + +func TestReanimatorExportPreservesNUMAZeroAndOmitsUnknown(t *testing.T) { + result := &models.AnalysisResult{Hardware: &models.HardwareConfig{ + BoardInfo: models.BoardInfo{SerialNumber: "SN-NUMA"}, + Devices: []models.HardwareDevice{ + {ID: "gpu:0", Kind: models.DeviceKindGPU, Slot: "0000:01:00.0", DeviceClass: "VideoController", Model: "GPU 0", NUMANode: intPtr(0)}, + {ID: "gpu:unknown", Kind: models.DeviceKindGPU, Slot: "0000:02:00.0", DeviceClass: "VideoController", Model: "GPU unknown"}, + }, + }} + out, err := ConvertToReanimator(result) + if err != nil { + t.Fatal(err) + } + body, err := json.Marshal(out) + if err != nil { + t.Fatal(err) + } + text := string(body) + if !strings.Contains(text, `"slot":"0000:01:00.0","numa_node":0`) { + t.Fatalf("NUMA zero was lost: %s", text) + } + if strings.Contains(text, `"slot":"0000:02:00.0","numa_node"`) { + t.Fatalf("unknown NUMA was serialized: %s", text) + } +} + func TestConvertToReanimator(t *testing.T) { tests := []struct { name string @@ -1786,7 +1813,7 @@ func TestConvertToReanimator_ExportsContractV24Telemetry(t *testing.T) { Slot: "PCIeCard2", SerialNumber: "NIC-001", DeviceClass: "EthernetController", - NUMANode: 1, + NUMANode: intPtr(1), Details: map[string]any{ "temperature_c": 48.5, "power_w": 18.2, @@ -1828,7 +1855,7 @@ func TestConvertToReanimator_ExportsContractV24Telemetry(t *testing.T) { if got := out.Hardware.Storage[0]; got.TemperatureC != 38.5 || got.PowerOnHours != 12450 || got.LifeRemainingPct != 91.0 { t.Fatalf("unexpected storage telemetry: %#v", got) } - if got := out.Hardware.PCIeDevices[0]; got.NUMANode != 1 || got.TemperatureC != 48.5 || got.PowerW != 18.2 || got.SFPTemperatureC != 36.2 { + if got := out.Hardware.PCIeDevices[0]; got.NUMANode == nil || *got.NUMANode != 1 || got.TemperatureC != 48.5 || got.PowerW != 18.2 || got.SFPTemperatureC != 36.2 { t.Fatalf("unexpected PCIe telemetry: %#v", got) } if got := out.Hardware.PowerSupplies[0]; got.TemperatureC != 39 || got.LifeRemainingPct != 97.0 || got.LifeUsedPct != 3.0 { diff --git a/internal/exporter/reanimator_models.go b/internal/exporter/reanimator_models.go index 1d0d4f2..e21cf30 100644 --- a/internal/exporter/reanimator_models.go +++ b/internal/exporter/reanimator_models.go @@ -145,7 +145,7 @@ type ReanimatorPCIe struct { Slot string `json:"slot"` VendorID int `json:"vendor_id,omitempty"` DeviceID int `json:"device_id,omitempty"` - NUMANode int `json:"numa_node,omitempty"` + NUMANode *int `json:"numa_node,omitempty"` IOMMUGroup *int `json:"iommu_group,omitempty"` TemperatureC float64 `json:"temperature_c,omitempty"` PowerW float64 `json:"power_w,omitempty"` diff --git a/internal/models/models.go b/internal/models/models.go index bbdf360..933a2d7 100644 --- a/internal/models/models.go +++ b/internal/models/models.go @@ -23,6 +23,7 @@ type AnalysisResult struct { Sensors []SensorReading `json:"sensors"` Hardware *HardwareConfig `json:"hardware"` PrivacyScan *PrivacyScan `json:"privacy_scan,omitempty"` // customer-data / anonymization scan of the source files + TopologyEvidence *TopologyEvidence `json:"-"` // source artifacts used to build the separate topology.json document } // PrivacyScan is the result of scanning the source files for customer-identifying @@ -232,7 +233,7 @@ type HardwareDevice struct { InputVoltage float64 `json:"input_voltage,omitempty"` TemperatureC int `json:"temperature_c,omitempty"` RemainingEndurancePct *int `json:"remaining_endurance_pct,omitempty"` // 0-100 %; nil = not reported - NUMANode int `json:"numa_node,omitempty"` // 0 = not reported/N/A + NUMANode *int `json:"numa_node,omitempty"` // nil = not reported/N/A; 0 is a valid NUMA node Status string `json:"status,omitempty"` StatusCheckedAt *time.Time `json:"status_checked_at,omitempty"` @@ -376,7 +377,7 @@ type PCIeDevice struct { PartNumber string `json:"part_number,omitempty"` SerialNumber string `json:"serial_number,omitempty"` MACAddresses []string `json:"mac_addresses,omitempty"` - NUMANode int `json:"numa_node,omitempty"` // 0 = not reported/N/A + NUMANode *int `json:"numa_node,omitempty"` // nil = not reported/N/A; 0 is a valid NUMA node Present *bool `json:"present,omitempty"` IOMMUGroup *int `json:"iommu_group,omitempty"` Status string `json:"status,omitempty"` @@ -514,7 +515,7 @@ type NetworkAdapter struct { LinkSpeed string `json:"link_speed,omitempty"` MaxLinkWidth int `json:"max_link_width,omitempty"` MaxLinkSpeed string `json:"max_link_speed,omitempty"` - NUMANode int `json:"numa_node,omitempty"` // 0 = not reported/N/A + NUMANode *int `json:"numa_node,omitempty"` // nil = not reported/N/A; 0 is a valid NUMA node Status string `json:"status,omitempty"` StatusCheckedAt *time.Time `json:"status_checked_at,omitempty"` diff --git a/internal/models/topology.go b/internal/models/topology.go new file mode 100644 index 0000000..8b3ab04 --- /dev/null +++ b/internal/models/topology.go @@ -0,0 +1,47 @@ +package models + +// TopologyEvidence contains static source artifacts which add relationships +// that the Reanimator inventory schema cannot express. It is deliberately not +// serialized into reanimator.json. +type TopologyEvidence struct { + DIMMType17 string + StorageMap string + NVIDIAQueryCSV string + NVIDIATopology string + NVLinkStatus string + NVLinkErrors string +} + +// TopologyDocument is the standalone, versioned topology.json contract. +// Coordinates are presentation details and are intentionally not persisted. +type TopologyDocument struct { + Version string `json:"version"` + Title string `json:"title,omitempty"` + Nodes []TopologyNode `json:"nodes,omitempty"` + Edges []TopologyEdge `json:"edges,omitempty"` +} + +type TopologyNode struct { + ID string `json:"id"` + Kind string `json:"kind"` + Label string `json:"label"` + Model string `json:"model,omitempty"` + Status string `json:"status,omitempty"` + Socket *int `json:"socket,omitempty"` + NUMANode *int `json:"numa_node,omitempty"` + BDF string `json:"bdf,omitempty"` + Count int `json:"count,omitempty"` + SizeMB int `json:"size_mb,omitempty"` + WattageW int `json:"wattage_w,omitempty"` + LinkSpeed string `json:"link_speed,omitempty"` + MaxLinkSpeed string `json:"max_link_speed,omitempty"` +} + +type TopologyEdge struct { + From string `json:"from"` + To string `json:"to"` + Kind string `json:"kind"` + Status string `json:"status,omitempty"` + Label string `json:"label,omitempty"` + Links int `json:"links,omitempty"` +} diff --git a/internal/parser/vendors/dell/parser.go b/internal/parser/vendors/dell/parser.go index 68b1ca2..9d29b15 100644 --- a/internal/parser/vendors/dell/parser.go +++ b/internal/parser/vendors/dell/parser.go @@ -488,7 +488,7 @@ func parsePCIeDeviceView(props map[string]string, result *models.AnalysisResult) DeviceID: deviceID, BDF: formatBDF(props["busnumber"], props["devicenumber"], props["functionnumber"]), Manufacturer: manufacturer, - NUMANode: parseIntLoose(props["cpuaffinity"]), + NUMANode: parseOptionalIntLoose(props["cpuaffinity"]), Status: normalizeStatus(props["primarystatus"]), } result.Hardware.PCIeDevices = append(result.Hardware.PCIeDevices, p) @@ -535,7 +535,7 @@ func parseNICView(props map[string]string, result *models.AnalysisResult) { props["controllerbiosversion"], )), PortCount: inferPortCountFromFQDD(fqdd), - NUMANode: parseIntLoose(props["cpuaffinity"]), + NUMANode: parseOptionalIntLoose(props["cpuaffinity"]), Status: normalizeStatus(props["primarystatus"]), } if mac != "" { @@ -589,7 +589,7 @@ func parseControllerView(props map[string]string, result *models.AnalysisResult) DeviceClass: "storage-controller", Manufacturer: strings.TrimSpace(firstNonEmpty(props["devicecardmanufacturer"], props["manufacturer"])), PartNumber: strings.TrimSpace(firstNonEmpty(props["ppid"], props["boardpartnumber"])), - NUMANode: parseIntLoose(props["cpuaffinity"]), + NUMANode: parseOptionalIntLoose(props["cpuaffinity"]), Status: normalizeStatus(props["primarystatus"]), }) @@ -983,6 +983,25 @@ func parseIntLoose(v string) int { return n } +// parseOptionalIntLoose preserves zero as a real value while keeping missing +// and textual N/A values distinct. CPUAffinity is the motivating field: NUMA +// node 0 is valid, whereas "Not Applicable" means there is no affinity. +func parseOptionalIntLoose(v string) *int { + v = strings.TrimSpace(v) + if v == "" { + return nil + } + num := firstNumberRE.FindString(v) + if num == "" { + return nil + } + n, err := strconv.Atoi(num) + if err != nil { + return nil + } + return &n +} + func parseInt64Loose(v string) int64 { v = strings.TrimSpace(v) if v == "" { diff --git a/internal/parser/vendors/dell/parser_test.go b/internal/parser/vendors/dell/parser_test.go index 9a06300..1fca830 100644 --- a/internal/parser/vendors/dell/parser_test.go +++ b/internal/parser/vendors/dell/parser_test.go @@ -390,7 +390,7 @@ func TestParseDellInfiniBandView(t *testing.T) { } // TestParseDellCPUAffinity verifies that CPUAffinity is parsed into NUMANode for -// NIC, PCIe, and controller views. "Not Applicable" must result in NUMANode=0. +// NIC, PCIe, and controller views. Zero is valid; "Not Applicable" is nil. func TestParseDellCPUAffinity(t *testing.T) { const viewXML = ` @@ -439,27 +439,38 @@ func TestParseDellCPUAffinity(t *testing.T) { } // NIC CPUAffinity=1 → NUMANode=1 - nicBySlot := make(map[string]int) + nicBySlot := make(map[string]*int) for _, nic := range result.Hardware.NetworkAdapters { nicBySlot[nic.Slot] = nic.NUMANode } - if nicBySlot["NIC.Slot.2-1-1"] != 1 { - t.Errorf("NIC.Slot.2-1-1 NUMANode = %d, want 1", nicBySlot["NIC.Slot.2-1-1"]) + if got := nicBySlot["NIC.Slot.2-1-1"]; got == nil || *got != 1 { + t.Errorf("NIC.Slot.2-1-1 NUMANode = %v, want 1", got) } - if nicBySlot["InfiniBand.Slot.1-1"] != 2 { - t.Errorf("InfiniBand.Slot.1-1 NUMANode = %d, want 2", nicBySlot["InfiniBand.Slot.1-1"]) + if got := nicBySlot["InfiniBand.Slot.1-1"]; got == nil || *got != 2 { + t.Errorf("InfiniBand.Slot.1-1 NUMANode = %v, want 2", got) } - // PCIe device CPUAffinity=2 → NUMANode=2; controller CPUAffinity="Not Applicable" → NUMANode=0 - pcieBySlot := make(map[string]int) + // PCIe device CPUAffinity=2 → NUMANode=2; controller CPUAffinity="Not Applicable" → nil. + pcieBySlot := make(map[string]*int) for _, pcie := range result.Hardware.PCIeDevices { pcieBySlot[pcie.Slot] = pcie.NUMANode } - if pcieBySlot["Slot.7-1"] != 2 { - t.Errorf("Slot.7-1 NUMANode = %d, want 2", pcieBySlot["Slot.7-1"]) + if got := pcieBySlot["Slot.7-1"]; got == nil || *got != 2 { + t.Errorf("Slot.7-1 NUMANode = %v, want 2", got) } - if pcieBySlot["RAID.Slot.1-1"] != 0 { - t.Errorf("RAID.Slot.1-1 NUMANode = %d, want 0 (Not Applicable)", pcieBySlot["RAID.Slot.1-1"]) + if got := pcieBySlot["RAID.Slot.1-1"]; got != nil { + t.Errorf("RAID.Slot.1-1 NUMANode = %v, want nil (Not Applicable)", got) + } +} + +func TestParseOptionalIntLoosePreservesZero(t *testing.T) { + if got := parseOptionalIntLoose("0"); got == nil || *got != 0 { + t.Fatalf("parseOptionalIntLoose(0) = %v, want pointer to 0", got) + } + for _, value := range []string{"", "N/A", "Not Applicable"} { + if got := parseOptionalIntLoose(value); got != nil { + t.Fatalf("parseOptionalIntLoose(%q) = %v, want nil", value, got) + } } } diff --git a/internal/parser/vendors/easy_bee/parser.go b/internal/parser/vendors/easy_bee/parser.go index 6a91da1..953edc7 100644 --- a/internal/parser/vendors/easy_bee/parser.go +++ b/internal/parser/vendors/easy_bee/parser.go @@ -11,7 +11,7 @@ import ( "git.mchus.pro/mchus/logpile/internal/parser" ) -const parserVersion = "1.0" +const parserVersion = "1.1" func init() { parser.Register(&Parser{}) @@ -126,6 +126,7 @@ func (p *Parser) Parse(files []parser.ExtractedFile) (*models.AnalysisResult, er PowerSupply: normalizePSUSlots(snapshot.Hardware.PowerSupply), }, } + result.TopologyEvidence = collectTopologyEvidence(files) if pn := chassisProductPartNumber(files, result.Hardware.BoardInfo.ProductName); pn != "" { result.Hardware.BoardInfo.PartNumber = pn @@ -156,6 +157,39 @@ func (p *Parser) Parse(files []parser.ExtractedFile) (*models.AnalysisResult, er return result, nil } +func collectTopologyEvidence(files []parser.ExtractedFile) *models.TopologyEvidence { + read := func(names ...string) string { + for _, name := range names { + for _, f := range files { + if strings.EqualFold(pathBase(f.Path), name) { + return string(f.Content) + } + } + } + return "" + } + e := &models.TopologyEvidence{ + DIMMType17: read("dmidecode-type17.txt"), + StorageMap: read("storage-controllers.txt"), + NVIDIAQueryCSV: read("nvidia-smi-query-fresh.csv", "nvidia-smi-query.csv"), + NVIDIATopology: read("nvidia-smi-topo-fresh.txt", "nvidia-smi-topo.txt"), + NVLinkStatus: read("nvidia-smi-nvlink-status-fresh.txt", "nvidia-smi-nvlink-status.txt"), + NVLinkErrors: read("nvidia-smi-nvlink-errors-fresh.txt", "nvidia-smi-nvlink-errors.txt"), + } + if e.DIMMType17 == "" && e.StorageMap == "" && e.NVIDIAQueryCSV == "" && e.NVIDIATopology == "" && e.NVLinkStatus == "" && e.NVLinkErrors == "" { + return nil + } + return e +} + +func pathBase(path string) string { + path = strings.ReplaceAll(strings.TrimSpace(path), `\`, "/") + if i := strings.LastIndexByte(path, '/'); i >= 0 { + return path[i+1:] + } + return path +} + type beeSnapshot struct { SourceType string `json:"source_type,omitempty"` Protocol string `json:"protocol,omitempty"` diff --git a/internal/parser/vendors/easy_bee/parser_test.go b/internal/parser/vendors/easy_bee/parser_test.go index e6b71dd..2bb8ac8 100644 --- a/internal/parser/vendors/easy_bee/parser_test.go +++ b/internal/parser/vendors/easy_bee/parser_test.go @@ -30,6 +30,23 @@ func TestDetectBeeSupportArchive(t *testing.T) { } } +func TestCollectTopologyEvidenceSupportsCategorizedBundleAndPrefersFresh(t *testing.T) { + files := []parser.ExtractedFile{ + {Path: "bundle/export/memory/dmidecode-type17.txt", Content: []byte("memory")}, + {Path: "bundle/export/storage/storage-controllers.txt", Content: []byte("storage")}, + {Path: "bundle/export/gpu/nvidia-smi-query.csv", Content: []byte("query")}, + {Path: "bundle/export/gpu/nvidia-smi-topo.txt", Content: []byte("old")}, + {Path: "bundle/export/gpu/nvidia-smi-topo-fresh.txt", Content: []byte("fresh")}, + } + got := collectTopologyEvidence(files) + if got == nil { + t.Fatal("expected topology evidence") + } + if got.DIMMType17 != "memory" || got.StorageMap != "storage" || got.NVIDIAQueryCSV != "query" || got.NVIDIATopology != "fresh" { + t.Fatalf("unexpected topology evidence: %+v", got) + } +} + // TestDetectBeeSupportArchiveV12Layout covers the real bundle layout produced // by bee-support v12.x, where bee-audit.json lives under tasks/_state/ and // runtime-health.json under status/, not export/. See CLAUDE.md memory on diff --git a/internal/parser/vendors/inspur/component.go b/internal/parser/vendors/inspur/component.go index 95f40a6..68d1277 100644 --- a/internal/parser/vendors/inspur/component.go +++ b/internal/parser/vendors/inspur/component.go @@ -952,7 +952,7 @@ func mergeInspurNIC(dst *models.NetworkAdapter, src models.NetworkAdapter) { if strings.TrimSpace(dst.MaxLinkSpeed) == "" { dst.MaxLinkSpeed = src.MaxLinkSpeed } - if dst.NUMANode == 0 { + if dst.NUMANode == nil { dst.NUMANode = src.NUMANode } if strings.TrimSpace(dst.Status) == "" { diff --git a/internal/server/device_repository.go b/internal/server/device_repository.go index 36bc560..7751115 100644 --- a/internal/server/device_repository.go +++ b/internal/server/device_repository.go @@ -177,6 +177,7 @@ func BuildHardwareDevices(hw *models.HardwareConfig) []models.HardwareDevice { LinkSpeed: p.LinkSpeed, MaxLinkWidth: p.MaxLinkWidth, MaxLinkSpeed: p.MaxLinkSpeed, + NUMANode: p.NUMANode, Status: p.Status, StatusCheckedAt: p.StatusCheckedAt, StatusChangedAt: p.StatusChangedAt, @@ -568,6 +569,9 @@ func mergeDevices(primary, secondary models.HardwareDevice) models.HardwareDevic fillString(&primary.LinkSpeed, secondary.LinkSpeed) fillInt(&primary.MaxLinkWidth, secondary.MaxLinkWidth) fillString(&primary.MaxLinkSpeed, secondary.MaxLinkSpeed) + if primary.NUMANode == nil && secondary.NUMANode != nil { + primary.NUMANode = secondary.NUMANode + } fillInt(&primary.WattageW, secondary.WattageW) fillString(&primary.InputType, secondary.InputType) fillInt(&primary.InputPowerW, secondary.InputPowerW) diff --git a/internal/server/device_repository_test.go b/internal/server/device_repository_test.go index 042b9fa..ccb816e 100644 --- a/internal/server/device_repository_test.go +++ b/internal/server/device_repository_test.go @@ -9,6 +9,8 @@ import ( "git.mchus.pro/mchus/logpile/internal/models" ) +func intPtr(v int) *int { return &v } + func TestBuildHardwareDevices_DedupSerialThenBDF(t *testing.T) { hw := &models.HardwareConfig{ PCIeDevices: []models.PCIeDevice{ @@ -138,7 +140,7 @@ func TestBuildHardwareDevices_NetworkAdapterPreservesPCIeMetadata(t *testing.T) MACAddresses: []string{"44:1A:4C:16:E8:03", "44:1A:4C:16:E8:04"}, LinkWidth: 16, LinkSpeed: "32 GT/s", - NUMANode: 1, + NUMANode: intPtr(1), Status: "ok", }, }, @@ -149,7 +151,7 @@ func TestBuildHardwareDevices_NetworkAdapterPreservesPCIeMetadata(t *testing.T) if d.Kind != models.DeviceKindNetwork { continue } - if d.BDF != "0000:27:00.0" || d.LinkWidth != 16 || d.LinkSpeed != "32 GT/s" || d.NUMANode != 1 { + if d.BDF != "0000:27:00.0" || d.LinkWidth != 16 || d.LinkSpeed != "32 GT/s" || d.NUMANode == nil || *d.NUMANode != 1 { t.Fatalf("expected network PCIe metadata to be preserved, got %+v", d) } return diff --git a/internal/server/raw_export.go b/internal/server/raw_export.go index 66ab47e..09a0ae3 100644 --- a/internal/server/raw_export.go +++ b/internal/server/raw_export.go @@ -16,10 +16,11 @@ import ( const rawExportFormatV1 = "logpile.raw-export.v1" const ( - rawExportBundlePackageFile = "raw_export.json" - rawExportBundleLogFile = "collect.log" - rawExportBundleFieldsFile = "parser_fields.json" - rawExportBundlePrivacyFile = "privacy_report.json" + rawExportBundlePackageFile = "raw_export.json" + rawExportBundleLogFile = "collect.log" + rawExportBundleFieldsFile = "parser_fields.json" + rawExportBundlePrivacyFile = "privacy_report.json" + rawExportBundleTopologyFile = "topology.json" ) type RawExportPackage struct { @@ -165,6 +166,20 @@ func buildRawExportBundle(pkg *RawExportPackage, result *models.AnalysisResult, } } + if topologyDoc, topologyErr := currentTopologyDocument(result); topologyErr == nil && len(topologyDoc.Nodes) > 0 { + tf, err := zw.Create(rawExportBundleTopologyFile) + if err != nil { + return nil, err + } + topologyJSON, err := json.MarshalIndent(topologyDoc, "", " ") + if err != nil { + return nil, err + } + if _, err := tf.Write(topologyJSON); err != nil { + return nil, err + } + } + if err := zw.Close(); err != nil { return nil, err } diff --git a/internal/server/server.go b/internal/server/server.go index 541f803..87189e9 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -77,6 +77,7 @@ func (s *Server) setupRoutes() { // Pages s.mux.HandleFunc("/", s.handleIndex) s.mux.HandleFunc("GET /chart/current", s.handleChartCurrent) + s.mux.HandleFunc("GET /topology/current", s.handleTopologyCurrent) // API endpoints s.mux.HandleFunc("POST /api/upload", s.handleUpload) @@ -86,6 +87,7 @@ func (s *Server) setupRoutes() { s.mux.HandleFunc("GET /api/events", s.handleGetEvents) s.mux.HandleFunc("GET /api/sensors", s.handleGetSensors) s.mux.HandleFunc("GET /api/config", s.handleGetConfig) + s.mux.HandleFunc("GET /api/topology", s.handleGetTopology) s.mux.HandleFunc("GET /api/serials", s.handleGetSerials) s.mux.HandleFunc("GET /api/firmware", s.handleGetFirmware) s.mux.HandleFunc("GET /api/parse-errors", s.handleGetParseErrors) diff --git a/internal/server/topology_view.go b/internal/server/topology_view.go new file mode 100644 index 0000000..03ddacf --- /dev/null +++ b/internal/server/topology_view.go @@ -0,0 +1,43 @@ +package server + +import ( + "encoding/json" + "net/http" + + "git.mchus.pro/mchus/logpile/internal/exporter" + "git.mchus.pro/mchus/logpile/internal/models" + "git.mchus.pro/mchus/logpile/internal/topology" +) + +func currentTopologyDocument(result *models.AnalysisResult) (*models.TopologyDocument, error) { + if result == nil || result.Hardware == nil { + return &models.TopologyDocument{Version: topology.ContractVersion}, nil + } + snapshot, err := exporter.ConvertToReanimator(result) + if err != nil { + return nil, err + } + return topology.Build(snapshot, result.TopologyEvidence), nil +} + +func (s *Server) handleTopologyCurrent(w http.ResponseWriter, r *http.Request) { + doc, err := currentTopologyDocument(s.GetResult()) + if err != nil { + s.htmlError(w, "failed to build topology", http.StatusInternalServerError) + return + } + w.Header().Set("Content-Type", "text/html; charset=utf-8") + _, _ = w.Write(topology.RenderHTML(doc)) +} + +func (s *Server) handleGetTopology(w http.ResponseWriter, r *http.Request) { + doc, err := currentTopologyDocument(s.GetResult()) + if err != nil { + jsonError(w, "Failed to build topology", http.StatusInternalServerError) + return + } + w.Header().Set("Content-Type", "application/json; charset=utf-8") + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + _ = enc.Encode(doc) +} diff --git a/internal/server/topology_view_test.go b/internal/server/topology_view_test.go new file mode 100644 index 0000000..42ad6b1 --- /dev/null +++ b/internal/server/topology_view_test.go @@ -0,0 +1,71 @@ +package server + +import ( + "archive/zip" + "bytes" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "git.mchus.pro/mchus/logpile/internal/models" +) + +func topologyTestResult() *models.AnalysisResult { + return &models.AnalysisResult{Filename: "bee.tar.gz", Hardware: &models.HardwareConfig{ + BoardInfo: models.BoardInfo{ProductName: "SYS-TEST", SerialNumber: "SN123"}, + CPUs: []models.CPU{{Socket: 0, Model: "Xeon", Status: "OK"}}, + PCIeDevices: []models.PCIeDevice{{ + Slot: "0000:31:00.0", BDF: "0000:31:00.0", DeviceClass: "VideoController", + Model: "NVIDIA GPU", NUMANode: intPtr(0), Status: "OK", + }}, + }} +} + +func TestTopologyEndpointsPreserveNUMAZero(t *testing.T) { + s := New(Config{}) + s.SetResult(topologyTestResult()) + for _, tc := range []struct{ path, want string }{{"/api/topology", `"numa_node": 0`}, {"/topology/current", "SYS-TEST - SN123"}} { + rec := httptest.NewRecorder() + s.mux.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, tc.path, nil)) + if rec.Code != http.StatusOK { + t.Fatalf("%s returned %d: %s", tc.path, rec.Code, rec.Body.String()) + } + if !strings.Contains(rec.Body.String(), tc.want) { + t.Fatalf("%s missing %q: %s", tc.path, tc.want, rec.Body.String()) + } + } +} + +func TestRawExportBundleContainsSeparateTopologyJSON(t *testing.T) { + result := topologyTestResult() + pkg := newRawExportFromUploadedFile(result.Filename, "application/gzip", []byte("source"), result) + body, err := buildRawExportBundle(pkg, result, "test") + if err != nil { + t.Fatal(err) + } + zr, err := zip.NewReader(bytes.NewReader(body), int64(len(body))) + if err != nil { + t.Fatal(err) + } + for _, f := range zr.File { + if f.Name != rawExportBundleTopologyFile { + continue + } + r, err := f.Open() + if err != nil { + t.Fatal(err) + } + data, err := io.ReadAll(r) + _ = r.Close() + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(data), `"version": "1.0"`) { + t.Fatalf("unexpected topology.json: %s", data) + } + return + } + t.Fatal("raw export bundle has no topology.json") +} diff --git a/internal/topology/build.go b/internal/topology/build.go new file mode 100644 index 0000000..7d558f0 --- /dev/null +++ b/internal/topology/build.go @@ -0,0 +1,405 @@ +package topology + +import ( + "fmt" + "regexp" + "sort" + "strconv" + "strings" + + "git.mchus.pro/mchus/logpile/internal/exporter" + "git.mchus.pro/mchus/logpile/internal/models" +) + +const ContractVersion = "1.0" + +var ( + cpuLocatorRE = regexp.MustCompile(`(?i)(?:^|[_ -])cpu\s*0*(\d+)`) + bankNodeRE = regexp.MustCompile(`(?i)node\s*0*(\d+)`) + nvLinkRE = regexp.MustCompile(`(?i)^NV(\d+)$`) +) + +// Build creates the standalone topology.json document from the same +// normalized projection used by the chart, augmented by optional static Bee +// support-bundle evidence. It never executes commands or reads live state. +func Build(snapshot *exporter.ReanimatorExport, evidence *models.TopologyEvidence) *models.TopologyDocument { + doc := &models.TopologyDocument{Version: ContractVersion} + if snapshot == nil { + return doc + } + doc.Title = topologyTitle(snapshot) + hw := snapshot.Hardware + + cpuIDs := make([]string, 0, len(hw.CPUs)) + socketByCPU := make([]int, 0, len(hw.CPUs)) + for i, cpu := range hw.CPUs { + socket := cpu.Socket + id := fmt.Sprintf("cpu:%d", socket) + if containsNode(doc.Nodes, id) { + id = fmt.Sprintf("cpu:%d:%d", socket, i) + } + s := socket + doc.Nodes = append(doc.Nodes, models.TopologyNode{ID: id, Kind: "cpu", Label: fmt.Sprintf("CPU %d", socket), Model: cleanCPUModel(cpu.Model), Status: cpu.Status, Socket: &s}) + cpuIDs = append(cpuIDs, id) + socketByCPU = append(socketByCPU, socket) + } + + numaToCPU := buildNUMAMap(hw.PCIeDevices, cpuIDs, socketByCPU) + bdfToNode := map[string]string{} + gpuNodeByBDF := map[string]string{} + for i, dev := range hw.PCIeDevices { + kind := pcieKind(dev) + if kind == "" { + continue + } + bdf := normalizeBDF(firstNonEmpty(dev.Slot, dev.BDF)) + id := fmt.Sprintf("%s:%s", kind, bdf) + if bdf == "" { + id = fmt.Sprintf("%s:%d", kind, i) + } + node := models.TopologyNode{ID: id, Kind: kind, Label: strings.ToUpper(kind), Model: dev.Model, Status: dev.Status, NUMANode: dev.NUMANode, BDF: bdf, LinkSpeed: dev.LinkSpeed, MaxLinkSpeed: dev.MaxLinkSpeed} + doc.Nodes = append(doc.Nodes, node) + if bdf != "" { + bdfToNode[bdf] = id + if kind == "gpu" { + gpuNodeByBDF[bdf] = id + } + } + if dev.NUMANode != nil { + if cpuID := numaToCPU[*dev.NUMANode]; cpuID != "" { + doc.Edges = append(doc.Edges, models.TopologyEdge{From: cpuID, To: id, Kind: "pcie", Status: linkStatus(dev.LinkSpeed, dev.MaxLinkSpeed)}) + } + } + } + + bankNodes := parseDIMMBankNodes(evidenceString(evidence, func(e *models.TopologyEvidence) string { return e.DIMMType17 })) + for i, mem := range hw.Memory { + id := fmt.Sprintf("memory:%s", strings.TrimSpace(mem.Slot)) + if strings.TrimSpace(mem.Slot) == "" || containsNode(doc.Nodes, id) { + id = fmt.Sprintf("memory:%d", i) + } + node := models.TopologyNode{ID: id, Kind: "memory", Label: "Memory", Model: mem.PartNumber, Status: mem.Status, SizeMB: mem.SizeMB} + doc.Nodes = append(doc.Nodes, node) + if rawNode, ok := memoryNode(mem.Slot, mem.Location, bankNodes); ok { + if cpuID := memoryCPU(rawNode, cpuIDs, socketByCPU); cpuID != "" { + doc.Edges = append(doc.Edges, models.TopologyEdge{From: cpuID, To: id, Kind: "memory"}) + } + } + } + + controllerByHCTL := parseStorageMap(evidenceString(evidence, func(e *models.TopologyEvidence) string { return e.StorageMap })) + for i, disk := range hw.Storage { + id := fmt.Sprintf("storage:%s", strings.TrimSpace(disk.Slot)) + if strings.TrimSpace(disk.Slot) == "" || containsNode(doc.Nodes, id) { + id = fmt.Sprintf("storage:%d", i) + } + doc.Nodes = append(doc.Nodes, models.TopologyNode{ID: id, Kind: "storage", Label: firstNonEmpty(disk.Type, "Disk"), Model: disk.Model, Status: disk.Status}) + if controller := bdfToNode[controllerByHCTL[strings.TrimSpace(disk.Slot)]]; controller != "" { + doc.Edges = append(doc.Edges, models.TopologyEdge{From: controller, To: id, Kind: "storage"}) + } + } + + for i, psu := range hw.PowerSupplies { + label := strings.TrimSpace(psu.Slot) + if label == "" { + label = fmt.Sprintf("PSU %d", i+1) + } + doc.Nodes = append(doc.Nodes, models.TopologyNode{ID: fmt.Sprintf("psu:%d", i), Kind: "psu", Label: label, Model: psu.Model, Status: psu.Status, WattageW: psu.WattageW}) + } + for i, fw := range hw.Firmware { + doc.Nodes = append(doc.Nodes, models.TopologyNode{ID: fmt.Sprintf("firmware:%d", i), Kind: "firmware", Label: fw.DeviceName, Model: "fw " + fw.Version}) + } + + addNVLinkEdges(doc, evidence, gpuNodeByBDF) + return doc +} + +func topologyTitle(s *exporter.ReanimatorExport) string { + parts := []string{strings.TrimSpace(s.Hardware.Board.ProductName), strings.TrimSpace(s.Hardware.Board.SerialNumber)} + var out []string + for _, p := range parts { + if p != "" { + out = append(out, p) + } + } + if len(out) == 0 { + return "Server topology" + } + return strings.Join(out, " - ") +} + +func buildNUMAMap(devices []exporter.ReanimatorPCIe, cpuIDs []string, sockets []int) map[int]string { + result := map[int]string{} + if len(cpuIDs) == 0 { + return result + } + seen, hasZero := map[int]bool{}, false + for _, d := range devices { + if d.NUMANode != nil { + seen[*d.NUMANode] = true + hasZero = hasZero || *d.NUMANode == 0 + } + } + var nodes []int + for n := range seen { + nodes = append(nodes, n) + } + sort.Ints(nodes) + for rank, n := range nodes { + if hasZero && rank < len(cpuIDs) { + result[n] = cpuIDs[rank] + continue + } + for i, socket := range sockets { + if socket == n { + result[n] = cpuIDs[i] + break + } + } + if result[n] == "" && rank < len(cpuIDs) { + result[n] = cpuIDs[rank] + } + } + return result +} + +func pcieKind(d exporter.ReanimatorPCIe) string { + c := strings.ToLower(strings.TrimSpace(d.DeviceClass)) + model := strings.ToLower(d.Model + " " + d.Manufacturer) + switch { + case strings.Contains(c, "video"), strings.Contains(c, "display"), strings.Contains(c, "3d"), strings.Contains(c, "processingaccelerator"), strings.Contains(model, "nvidia") && strings.Contains(model, "gpu"): + return "gpu" + case strings.Contains(c, "network"), strings.Contains(c, "ethernet"), strings.Contains(c, "fibrechannel"), len(d.MACAddresses) > 0: + return "nic" + case strings.Contains(c, "storage"), strings.Contains(c, "raid"): + return "raid" + default: + return "" + } +} + +func linkStatus(current, maximum string) string { + a, b := speedRank(current), speedRank(maximum) + if a == 0 || b == 0 { + return "Unknown" + } + if a < b { + return "Warning" + } + return "OK" +} + +func speedRank(v string) float64 { + v = strings.ToLower(strings.TrimSpace(v)) + v = strings.TrimPrefix(v, "gen") + for _, suffix := range []string{" gt/s", "gts", "gtps"} { + v = strings.TrimSuffix(v, suffix) + } + n, _ := strconv.ParseFloat(strings.TrimSpace(v), 64) + return n +} + +func parseDIMMBankNodes(raw string) map[string]int { + out := map[string]int{} + for _, sec := range strings.Split(raw, "Memory Device") { + locator, node := "", -1 + for _, line := range strings.Split(sec, "\n") { + line = strings.TrimSpace(line) + if v, ok := strings.CutPrefix(line, "Locator:"); ok { + locator = strings.TrimSpace(v) + } + if v, ok := strings.CutPrefix(line, "Bank Locator:"); ok { + if m := bankNodeRE.FindStringSubmatch(v); m != nil { + node, _ = strconv.Atoi(m[1]) + } + } + } + if locator != "" && node >= 0 { + out[locator] = node + } + } + return out +} + +func memoryNode(slot, location string, bank map[string]int) (int, bool) { + for _, value := range []string{slot, location} { + if m := cpuLocatorRE.FindStringSubmatch(strings.TrimSpace(value)); m != nil { + n, _ := strconv.Atoi(m[1]) + return n, true + } + if n, ok := bank[strings.TrimSpace(value)]; ok { + return n, true + } + if m := bankNodeRE.FindStringSubmatch(value); m != nil { + n, _ := strconv.Atoi(m[1]) + return n, true + } + } + return 0, false +} + +func memoryCPU(node int, cpuIDs []string, sockets []int) string { + for i, s := range sockets { + if s == node { + return cpuIDs[i] + } + } + if node >= 0 && node < len(cpuIDs) { + return cpuIDs[node] + } + if node > 0 && node <= len(cpuIDs) { + return cpuIDs[node-1] + } + return "" +} + +func parseStorageMap(raw string) map[string]string { + out := map[string]string{} + for _, line := range strings.Split(raw, "\n") { + hctl, ctrl := "", "" + for _, field := range strings.Fields(line) { + if v, ok := strings.CutPrefix(field, "hctl="); ok { + hctl = v + } + if v, ok := strings.CutPrefix(field, "ctrl="); ok { + ctrl = normalizeBDF(v) + } + } + if hctl != "" && ctrl != "" { + out[hctl] = ctrl + } + } + return out +} + +func addNVLinkEdges(doc *models.TopologyDocument, evidence *models.TopologyEvidence, gpuByBDF map[string]string) { + if evidence == nil || evidence.NVIDIATopology == "" || evidence.NVIDIAQueryCSV == "" { + return + } + idxToNode := map[int]string{} + for _, line := range strings.Split(evidence.NVIDIAQueryCSV, "\n") { + parts := strings.Split(line, ",") + if len(parts) < 2 { + continue + } + idx, err := strconv.Atoi(strings.TrimSpace(parts[0])) + if err != nil { + continue + } + if id := gpuByBDF[normalizeBDF(parts[1])]; id != "" { + idxToNode[idx] = id + } + } + for idx, id := range idxToNode { + for i := range doc.Nodes { + if doc.Nodes[i].ID == id { + doc.Nodes[i].Label = fmt.Sprintf("GPU %d", idx) + break + } + } + } + lines := strings.Split(evidence.NVIDIATopology, "\n") + var gpuCols []int + header := -1 + for i, line := range lines { + fields := strings.Fields(line) + if len(fields) > 1 && fields[0] == "GPU0" { + for _, f := range fields { + if strings.HasPrefix(f, "GPU") { + n, err := strconv.Atoi(strings.TrimPrefix(f, "GPU")) + if err == nil { + gpuCols = append(gpuCols, n) + } + } + } + header = i + break + } + } + if header < 0 { + return + } + seen := map[[2]int]bool{} + nvStatus := "OK" + if strings.Contains(strings.ToLower(evidence.NVLinkStatus), "") { + nvStatus = "Warning" + } + if nvlinkHasErrors(evidence.NVLinkErrors) { + nvStatus = "Critical" + } + for _, line := range lines[header+1:] { + fields := strings.Fields(line) + if len(fields) < 2 || !strings.HasPrefix(fields[0], "GPU") { + continue + } + row, err := strconv.Atoi(strings.TrimPrefix(fields[0], "GPU")) + if err != nil { + continue + } + for col, gpu := range gpuCols { + if row == gpu || col+1 >= len(fields) { + continue + } + m := nvLinkRE.FindStringSubmatch(fields[col+1]) + if m == nil { + continue + } + links, _ := strconv.Atoi(m[1]) + a, b := row, gpu + if a > b { + a, b = b, a + } + key := [2]int{a, b} + if seen[key] || idxToNode[a] == "" || idxToNode[b] == "" { + continue + } + seen[key] = true + doc.Edges = append(doc.Edges, models.TopologyEdge{From: idxToNode[a], To: idxToNode[b], Kind: "nvlink", Status: nvStatus, Label: fmt.Sprintf("NV%d", links), Links: links}) + } + } +} + +func nvlinkHasErrors(raw string) bool { + for _, line := range strings.Split(raw, "\n") { + lower := strings.ToLower(line) + if !strings.Contains(lower, "error") { + continue + } + fields := strings.Fields(line) + if len(fields) == 0 { + continue + } + last := strings.Trim(fields[len(fields)-1], " ,") + if n, err := strconv.ParseInt(last, 10, 64); err == nil && n > 0 { + return true + } + } + return false +} + +func evidenceString(e *models.TopologyEvidence, get func(*models.TopologyEvidence) string) string { + if e == nil { + return "" + } + return get(e) +} +func containsNode(nodes []models.TopologyNode, id string) bool { + for _, n := range nodes { + if n.ID == id { + return true + } + } + return false +} +func normalizeBDF(v string) string { return strings.ToLower(strings.TrimSpace(v)) } +func firstNonEmpty(values ...string) string { + for _, v := range values { + if strings.TrimSpace(v) != "" { + return strings.TrimSpace(v) + } + } + return "" +} +func cleanCPUModel(v string) string { + return strings.Join(strings.Fields(strings.NewReplacer("(R)", "", "(TM)", "", " CPU", "", " Processor", "").Replace(v)), " ") +} diff --git a/internal/topology/build_test.go b/internal/topology/build_test.go new file mode 100644 index 0000000..bc34cf5 --- /dev/null +++ b/internal/topology/build_test.go @@ -0,0 +1,84 @@ +package topology + +import ( + "encoding/json" + "strings" + "testing" + + "git.mchus.pro/mchus/logpile/internal/exporter" + "git.mchus.pro/mchus/logpile/internal/models" +) + +func intp(v int) *int { return &v } + +func TestBuildDistinguishesNUMAZeroFromUnknown(t *testing.T) { + snapshot := &exporter.ReanimatorExport{Hardware: exporter.ReanimatorHardware{ + Board: exporter.ReanimatorBoard{SerialNumber: "SN1"}, + CPUs: []exporter.ReanimatorCPU{{Socket: 0}, {Socket: 1}}, + PCIeDevices: []exporter.ReanimatorPCIe{ + {Slot: "0000:01:00.0", DeviceClass: "VideoController", Model: "GPU A", NUMANode: intp(0)}, + {Slot: "0000:02:00.0", DeviceClass: "NetworkController", Model: "NIC A", NUMANode: nil}, + }, + }} + doc := Build(snapshot, nil) + var gpuAttached, nicAttached bool + for _, e := range doc.Edges { + gpuAttached = gpuAttached || (e.From == "cpu:0" && e.To == "gpu:0000:01:00.0") + nicAttached = nicAttached || strings.Contains(e.To, "nic:") + } + if !gpuAttached { + t.Fatal("NUMA node 0 GPU was not attached to CPU 0") + } + if nicAttached { + t.Fatal("unknown-affinity NIC must not be attached") + } + body, err := json.Marshal(doc) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(body), `"numa_node":0`) { + t.Fatalf("topology JSON lost NUMA zero: %s", body) + } +} + +func TestBuildUsesBeeStaticEvidence(t *testing.T) { + snapshot := &exporter.ReanimatorExport{Hardware: exporter.ReanimatorHardware{ + Board: exporter.ReanimatorBoard{SerialNumber: "SN1"}, + CPUs: []exporter.ReanimatorCPU{{Socket: 0}, {Socket: 1}}, + Memory: []exporter.ReanimatorMemory{{Slot: "DIMM000(A)", SerialNumber: "M1", Status: "OK"}}, + Storage: []exporter.ReanimatorStorage{{Slot: "0:0:0:0", SerialNumber: "D1", Model: "Disk"}}, + PCIeDevices: []exporter.ReanimatorPCIe{ + {Slot: "0000:01:00.0", DeviceClass: "StorageController", NUMANode: intp(0)}, + {Slot: "0000:31:00.0", DeviceClass: "VideoController", Model: "GPU 0", NUMANode: intp(0)}, + {Slot: "0000:32:00.0", DeviceClass: "VideoController", Model: "GPU 1", NUMANode: intp(0)}, + }, + }} + e := &models.TopologyEvidence{ + DIMMType17: "Memory Device\n Locator: DIMM000(A)\n Bank Locator: Node0_Channel0", + StorageMap: "sda hctl=0:0:0:0 ctrl=0000:01:00.0", + NVIDIAQueryCSV: "0, 0000:31:00.0\n1, 0000:32:00.0\n", + NVIDIATopology: " GPU0 GPU1\nGPU0 X NV4\nGPU1 NV4 X\n", + } + doc := Build(snapshot, e) + kinds := map[string]int{} + for _, edge := range doc.Edges { + kinds[edge.Kind]++ + } + for _, kind := range []string{"memory", "storage", "nvlink"} { + if kinds[kind] == 0 { + t.Fatalf("missing %s edge: %+v", kind, doc.Edges) + } + } +} + +func TestRenderHTMLIsStaticAndEscapesLabels(t *testing.T) { + body := string(RenderHTML(&models.TopologyDocument{Version: ContractVersion, Title: ``, Nodes: []models.TopologyNode{{ID: "cpu:0", Kind: "cpu", Label: "CPU 0", Socket: intp(0)}}})) + if strings.Contains(body, ``) { + t.Fatal("title was not escaped") + } + for _, forbidden := range []string{"setInterval(", "fetch(", "nvidia-smi"} { + if strings.Contains(body, forbidden) { + t.Fatalf("rendered page contains live behavior %q", forbidden) + } + } +} diff --git a/internal/topology/render.go b/internal/topology/render.go new file mode 100644 index 0000000..63e9e7b --- /dev/null +++ b/internal/topology/render.go @@ -0,0 +1,399 @@ +package topology + +import ( + "fmt" + "html" + "sort" + "strings" + + "git.mchus.pro/mchus/logpile/internal/models" +) + +type card struct { + label, model, status string + count int +} + +type branch struct { + info card + edgeStatus string + subs []card +} + +type block struct { + head card + branches []branch +} + +// RenderHTML renders a self-contained, static topology page. It intentionally +// contains no polling, live telemetry, shell integration, or chart dependency. +func RenderHTML(doc *models.TopologyDocument) []byte { + if doc == nil { + doc = &models.TopologyDocument{Version: ContractVersion} + } + title := doc.Title + if strings.TrimSpace(title) == "" { + title = "Server topology" + } + var b strings.Builder + b.WriteString(``) + b.WriteString(html.EscapeString(title)) + b.WriteString(` — Topology
LOGPile

`) + b.WriteString(html.EscapeString(title)) + b.WriteString(`

Server topology

`) + if len(doc.Nodes) == 0 { + b.WriteString(`
Topology
No topology data loaded.
`) + } else { + b.WriteString(`
Topology
`) + b.WriteString(renderDiagram(doc)) + b.WriteString(renderLooseRows(doc)) + b.WriteString(`
`) + if nv := renderNVLink(doc); nv != "" { + b.WriteString(nv) + } + } + b.WriteString(`
`) + return []byte(b.String()) +} + +func renderDiagram(doc *models.TopologyDocument) string { + nodeByID := map[string]models.TopologyNode{} + for _, n := range doc.Nodes { + nodeByID[n.ID] = n + } + children := map[string][]models.TopologyNode{} + edgeStatus := map[string]string{} + storageChildren := map[string][]models.TopologyNode{} + attached := map[string]bool{} + for _, e := range doc.Edges { + if child, ok := nodeByID[e.To]; ok { + switch e.Kind { + case "pcie", "memory": + children[e.From] = append(children[e.From], child) + edgeStatus[e.To] = e.Status + attached[e.To] = true + case "storage": + storageChildren[e.From] = append(storageChildren[e.From], child) + attached[e.To] = true + } + } + } + var cpus []models.TopologyNode + for _, n := range doc.Nodes { + if n.Kind == "cpu" { + cpus = append(cpus, n) + } + } + sort.SliceStable(cpus, func(i, j int) bool { return ptrValue(cpus[i].Socket) < ptrValue(cpus[j].Socket) }) + var blocks []block + for _, cpu := range cpus { + blocks = append(blocks, block{head: nodeCard(cpu), branches: groupBranches(children[cpu.ID], edgeStatus, storageChildren)}) + } + var other []models.TopologyNode + for _, n := range doc.Nodes { + if (n.Kind == "gpu" || n.Kind == "nic" || n.Kind == "raid" || n.Kind == "memory" || n.Kind == "storage") && !attached[n.ID] { + other = append(other, n) + } + } + if len(other) > 0 { + blocks = append(blocks, block{head: card{label: "Other", model: "no socket affinity", status: "Unknown", count: 1}, branches: groupBranches(other, edgeStatus, storageChildren)}) + } + if len(blocks) == 0 { + return `
No CPU or attachable device topology.
` + } + + const barW, branchW, subW, gap, boxH, boxGap, minBarH, midGap = 118, 208, 172, 46, 70, 14, 92, 72 + hasSubs := false + for _, blk := range blocks { + for _, br := range blk.branches { + hasSubs = hasSubs || len(br.subs) > 0 + } + } + leftReach := barW + gap + branchW + if hasSubs { + leftReach += gap + subW + } + width := 24 + leftReach + 24 + if len(blocks) > 1 { + width = 24 + leftReach + midGap + leftReach + 24 + } + type box struct { + x, y, w, h int + c card + } + type edge struct { + x1, y1, x2, y2 int + status string + } + var boxes []box + var edges []edge + rowTop := 30 + for i := 0; i < len(blocks); { + rowBottom := rowTop + for side := 0; side < 2 && i < len(blocks); side++ { + blk := blocks[i] + barX, branchX, subX := 24, 24+barW+gap, 24+barW+gap+branchW+gap + if side == 1 { + barX = width - 24 - barW + branchX = barX - gap - branchW + subX = branchX - gap - subW + } + y := rowTop + for _, br := range blk.branches { + boxes = append(boxes, box{x: branchX, y: y, w: branchW, h: boxH, c: br.info}) + x1, x2 := barX+barW, branchX + if side == 1 { + x1 = barX + x2 = branchX + branchW + } + edges = append(edges, edge{x1: x1, y1: y + boxH/2, x2: x2, y2: y + boxH/2, status: br.edgeStatus}) + advance := boxH + boxGap + stackLayers(br.info.count)*4 + if len(br.subs) > 0 { + sy := y + branchSubX := branchX + branchW + if side == 1 { + branchSubX = branchX + } + for _, sub := range br.subs { + boxes = append(boxes, box{x: subX, y: sy, w: subW, h: boxH, c: sub}) + sx := subX + if side == 1 { + sx = subX + subW + } + subStatus := sub.status + if severity(subStatus) == 0 { + subStatus = "OK" + } + edges = append(edges, edge{x1: branchSubX, y1: y + boxH/2, x2: sx, y2: sy + boxH/2, status: subStatus}) + sy += boxH + boxGap + stackLayers(sub.count)*4 + } + if sy-y > advance { + advance = sy - y + } + } + y += advance + } + bottom := y - boxGap + if bottom < rowTop+minBarH { + bottom = rowTop + minBarH + } + boxes = append(boxes, box{x: barX, y: rowTop, w: barW, h: bottom - rowTop, c: blk.head}) + if bottom > rowBottom { + rowBottom = bottom + } + i++ + } + rowTop = rowBottom + 30 + } + height := rowTop + 10 + var b strings.Builder + fmt.Fprintf(&b, `
`, width, height, width, height) + for _, e := range edges { + fmt.Fprintf(&b, ``, e.x1, e.y1, e.x2, e.y2, statusClass(e.status)) + } + for _, bx := range boxes { + writeBox(&b, bx.x, bx.y, bx.w, bx.h, bx.c) + } + b.WriteString(`
`) + return b.String() +} + +func groupCards(nodes []models.TopologyNode) []card { + order := []string{"memory", "gpu", "nic", "raid", "storage"} + groups := map[string][]models.TopologyNode{} + for _, n := range nodes { + groups[n.Kind] = append(groups[n.Kind], n) + } + var out []card + for _, kind := range order { + items := groups[kind] + if len(items) == 0 { + continue + } + model := items[0].Model + status := "Unknown" + for _, n := range items { + if severity(n.Status) > severity(status) { + status = n.Status + } + } + label := map[string]string{"memory": "Memory", "gpu": "GPU", "nic": "NIC", "raid": "RAID", "storage": "Storage"}[kind] + if kind == "memory" { + total := 0 + for _, n := range items { + total += n.SizeMB + } + if total > 0 { + model = fmt.Sprintf("%d GB total", total/1024) + } + } + out = append(out, card{label: label, model: model, status: status, count: len(items)}) + } + return out +} + +func groupBranches(nodes []models.TopologyNode, edgeStatuses map[string]string, storageChildren map[string][]models.TopologyNode) []branch { + cards := groupCards(nodes) + byKind := map[string][]models.TopologyNode{} + for _, n := range nodes { + byKind[n.Kind] = append(byKind[n.Kind], n) + } + var out []branch + for _, c := range cards { + kind := strings.ToLower(c.label) + if kind == "storage" { + kind = "storage" + } + status := "OK" + for _, n := range byKind[kind] { + if s := edgeStatuses[n.ID]; severity(s) > severity(status) { + status = s + } + } + br := branch{info: c, edgeStatus: status} + if kind == "raid" { + var disks []models.TopologyNode + for _, n := range byKind[kind] { + disks = append(disks, storageChildren[n.ID]...) + } + br.subs = groupCards(disks) + } + out = append(out, br) + } + return out +} + +func nodeCard(n models.TopologyNode) card { + return card{label: n.Label, model: n.Model, status: n.Status, count: max(1, n.Count)} +} + +func writeBox(b *strings.Builder, x, y, w, h int, c card) { + cls := statusClass(c.status) + for i := stackLayers(c.count); i >= 1; i-- { + off := i * 4 + fmt.Fprintf(b, ``, x+off, y+off, w, h, cls) + } + fmt.Fprintf(b, ``, x, y, w, h, cls) + label := c.label + if c.count > 1 { + label = fmt.Sprintf("%s ×%d", label, c.count) + } + fmt.Fprintf(b, `%s`, x+10, y+20, cls, html.EscapeString(label)) + if c.model != "" { + fmt.Fprintf(b, `%s`, x+10, y+37, cls, html.EscapeString(truncate(c.model, 28))) + } + fmt.Fprintf(b, `%s`, x+10, y+h-10, cls, html.EscapeString(displayStatus(c.status))) +} + +func renderLooseRows(doc *models.TopologyDocument) string { + var firmware, psus []models.TopologyNode + for _, n := range doc.Nodes { + if n.Kind == "firmware" { + firmware = append(firmware, n) + } + if n.Kind == "psu" { + psus = append(psus, n) + } + } + var b strings.Builder + writeRow := func(title string, nodes []models.TopologyNode) { + if len(nodes) == 0 { + return + } + b.WriteString(`

` + html.EscapeString(title) + `

`) + for _, n := range nodes { + b.WriteString(`
` + html.EscapeString(n.Label) + ``) + detail := n.Model + if n.WattageW > 0 { + detail = fmt.Sprintf("%d W", n.WattageW) + } + if detail != "" { + b.WriteString(`` + html.EscapeString(detail) + ``) + } + b.WriteString(`
`) + } + b.WriteString(`
`) + } + writeRow("Firmware", firmware) + writeRow("Power supplies", psus) + return b.String() +} + +func renderNVLink(doc *models.TopologyDocument) string { + byID := map[string]models.TopologyNode{} + for _, n := range doc.Nodes { + byID[n.ID] = n + } + var links []models.TopologyEdge + for _, e := range doc.Edges { + if e.Kind == "nvlink" { + links = append(links, e) + } + } + if len(links) == 0 { + return "" + } + var b strings.Builder + b.WriteString(`
NVLink Topology
`) + for _, e := range links { + a, bn := byID[e.From], byID[e.To] + b.WriteString(`
` + html.EscapeString(firstNonEmpty(a.Label, a.ID)) + `` + html.EscapeString(firstNonEmpty(bn.Label, bn.ID)) + `` + html.EscapeString(e.Label) + `
`) + } + b.WriteString(`
`) + return b.String() +} + +func severity(s string) int { + switch strings.ToUpper(strings.TrimSpace(s)) { + case "CRITICAL", "FAIL", "FAILED", "ERROR": + return 3 + case "WARNING", "WARN", "PARTIAL": + return 2 + case "OK": + return 1 + default: + return 0 + } +} +func statusClass(s string) string { + switch severity(s) { + case 3: + return "critical" + case 2: + return "warning" + case 1: + return "ok" + default: + return "unknown" + } +} +func displayStatus(s string) string { + if strings.TrimSpace(s) == "" { + return "Unknown" + } + return s +} +func ptrValue(v *int) int { + if v == nil { + return 1 << 30 + } + return *v +} +func stackLayers(n int) int { + if n <= 1 { + return 0 + } + if n > 2 { + return 2 + } + return n - 1 +} +func truncate(s string, n int) string { + r := []rune(strings.TrimSpace(s)) + if len(r) <= n { + return string(r) + } + return string(r[:n-1]) + "…" +} + +const pageCSS = `:root{--ink:#172033;--muted:#687386;--border:#d9dee8;--surface:#fff;--surface2:#f5f7fa;--ok-bg:#edf7e8;--ok-fg:#316523;--ok-border:#a3c293;--warn-bg:#fff7df;--warn-fg:#7a5700;--warn-border:#d5bd75;--crit-bg:#fff0f0;--crit-fg:#a12626;--crit-border:#dc9b9b}*{box-sizing:border-box}body{margin:0;background:#f3f5f8;color:var(--ink);font:14px/1.5 Lato,"Helvetica Neue",Arial,sans-serif}header{display:flex;justify-content:space-between;align-items:center;padding:18px 28px;background:#182231;color:#fff}header h1{font-size:20px;margin:2px 0}header p{margin:0;color:#bdc7d4}.eyebrow{font-size:11px;text-transform:uppercase;letter-spacing:.12em;color:#8fbbe9}button{padding:7px 12px;border:1px solid #718096;border-radius:4px;background:transparent;color:#fff;cursor:pointer}main{max-width:1320px;margin:24px auto;padding:0 20px}.card{background:#fff;border:1px solid var(--border);border-radius:4px;box-shadow:0 1px 3px #18223112;margin-bottom:18px}.card-head{padding:10px 14px;background:var(--surface2);border-bottom:1px solid var(--border);font-weight:700}.card-body,.empty{padding:16px}.empty{color:var(--muted)}.diagram-scroll{overflow-x:auto}.edge{stroke-width:2}.edge.ok{stroke:var(--ok-fg)}.edge.warning{stroke:var(--warn-fg)}.edge.critical{stroke:var(--crit-fg)}.edge.unknown{stroke:var(--muted)}.node,.node-back{stroke-width:1}.node-back{opacity:.55}.node.ok,.node-back.ok,.tile.ok{fill:var(--ok-bg);background:var(--ok-bg);stroke:var(--ok-border);border-color:var(--ok-border);color:var(--ok-fg)}.node.warning,.node-back.warning,.tile.warning{fill:var(--warn-bg);background:var(--warn-bg);stroke:var(--warn-border);border-color:var(--warn-border);color:var(--warn-fg)}.node.critical,.node-back.critical,.tile.critical{fill:var(--crit-bg);background:var(--crit-bg);stroke:var(--crit-border);border-color:var(--crit-border);color:var(--crit-fg)}.node.unknown,.node-back.unknown,.tile.unknown{fill:var(--surface2);background:var(--surface2);stroke:var(--border);border-color:var(--border);color:var(--muted)}.node-label{font-size:13px;font-weight:700}.node-model{font-size:11px;opacity:.86}.node-status{font-size:10px;font-weight:600}.node-label.ok,.node-model.ok,.node-status.ok{fill:var(--ok-fg)}.node-label.warning,.node-model.warning,.node-status.warning{fill:var(--warn-fg)}.node-label.critical,.node-model.critical,.node-status.critical{fill:var(--crit-fg)}.node-label.unknown,.node-model.unknown,.node-status.unknown{fill:var(--muted)}.row-title{font-size:11px;color:var(--muted);text-transform:uppercase;letter-spacing:.06em;margin:18px 0 7px}.tile-row{display:flex;flex-wrap:wrap;gap:9px}.tile{display:flex;flex-direction:column;min-width:150px;padding:9px 11px;border:1px solid;border-radius:6px}.tile span{font-size:11px;opacity:.86}.nvlinks{display:grid;gap:10px}.nvpair{display:grid;grid-template-columns:auto minmax(80px,1fr) auto auto;align-items:center;gap:10px}.nvpair i{height:2px;background:var(--muted)}.nvpair i.ok{background:var(--ok-fg)}.nvpair i.warning{background:var(--warn-fg)}.nvpair i.critical{background:var(--crit-fg)}.nvpair small{color:var(--muted)}@media(max-width:600px){header{padding:14px 16px}main{padding:0 10px;margin:12px auto}}` diff --git a/web/static/css/style.css b/web/static/css/style.css index 792a945..b8fc852 100644 --- a/web/static/css/style.css +++ b/web/static/css/style.css @@ -129,6 +129,7 @@ a:hover { font-size: 13px; font-weight: 700; transition: background 0.12s ease, opacity 0.12s ease; + text-decoration: none; } .header-action:hover { @@ -1046,4 +1047,3 @@ code { font-size: 0.82em; color: var(--muted); } - diff --git a/web/static/js/app.js b/web/static/js/app.js index 7204411..341ed5e 100644 --- a/web/static/js/app.js +++ b/web/static/js/app.js @@ -1409,6 +1409,7 @@ async function loadData(vendor, filename) { document.getElementById('data-section').classList.remove('hidden'); document.getElementById('header-raw-btn').classList.remove('hidden'); document.getElementById('header-reanimator-btn').classList.remove('hidden'); + document.getElementById('header-topology-btn').classList.remove('hidden'); document.getElementById('header-logs-csv-btn').classList.remove('hidden'); document.getElementById('header-log-meta').classList.remove('hidden'); diff --git a/web/templates/index.html b/web/templates/index.html index 8d1fe1a..a3e3047 100644 --- a/web/templates/index.html +++ b/web/templates/index.html @@ -18,6 +18,7 @@ +