diff --git a/bible-local/01-overview.md b/bible-local/01-overview.md index 332e750..69dcfee 100644 --- a/bible-local/01-overview.md +++ b/bible-local/01-overview.md @@ -30,6 +30,7 @@ All modes converge on the same normalized hardware model and exporter pipeline. - Reanimator Easy Bee support bundles - H3C SDS G5/G6 - Inspur / Kaytus +- HPE iLO AHS - NVIDIA HGX Field Diagnostics - NVIDIA Bug Report - Unraid diff --git a/bible-local/06-parsers.md b/bible-local/06-parsers.md index 42e5cf8..bda6b50 100644 --- a/bible-local/06-parsers.md +++ b/bible-local/06-parsers.md @@ -53,6 +53,7 @@ When `vendor_id` and `device_id` are known but the model name is missing or gene | `easy_bee` | `bee-support-*.tar.gz` | Imports embedded `export/bee-audit.json` snapshot from reanimator-easy-bee bundles | | `h3c_g5` | H3C SDS G5 bundles | INI/XML/CSV-driven hardware and event parsing | | `h3c_g6` | H3C SDS G6 bundles | Similar flow with G6-specific files | +| `hpe_ilo_ahs` | HPE iLO Active Health System (`.ahs`) | Proprietary `ABJR` container with gzip-compressed `zbb` members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON | | `inspur` | onekeylog archives | FRU/SDR plus optional Redis enrichment | | `nvidia` | HGX Field Diagnostics | GPU- and fabric-heavy diagnostic input | | `nvidia_bug_report` | `nvidia-bug-report-*.log.gz` | dmidecode, lspci, NVIDIA driver sections | @@ -121,6 +122,32 @@ with content markers (e.g. `Unraid kernel build`, parity data markers). --- +### HPE iLO AHS (`hpe_ilo_ahs`) + +**Status:** Ready (v1.0.0). Tested on HPE ProLiant Gen11 `.ahs` export from iLO 6. + +**Archive format:** `.ahs` single-file Active Health System export. + +**Detection:** Single-file input with `ABJR` container header and HPE AHS member names +such as `CUST_INFO.DAT`, `*.zbb`, `ilo_boot_support.zbb`. + +**Extracted data (current):** +- System board identity (manufacturer, model, serial, part number) +- iLO / System ROM / SPS top-level firmware +- CPU inventory (model-level) +- Memory DIMM inventory for populated slots +- PSU inventory +- PCIe / OCP NIC inventory from SMBIOS-style slot records +- Storage controller and physical drives from embedded Redfish JSON inside `zbb` members +- Basic iLO event log entries with timestamps when present + +**Implementation note:** The format is proprietary. Parser support is intentionally hybrid: +container parsing (`ABJR` + gzip) plus structured extraction from embedded Redfish objects and +printable SMBIOS/FRU payloads. This is sufficient for inventory-grade parsing without decoding the +entire internal `zbb` schema. + +--- + ### Generic text fallback (`generic`) **Status:** Ready (v1.0.0). @@ -141,6 +168,7 @@ with content markers (e.g. `Unraid kernel build`, parity data markers). |--------|----|--------|-----------| | Dell TSR | `dell` | Ready | TSR nested zip archives | | Reanimator Easy Bee | `easy_bee` | Ready | `bee-support-*.tar.gz` support bundles | +| HPE iLO AHS | `hpe_ilo_ahs` | Ready | iLO 6 `.ahs` exports | | Inspur / Kaytus | `inspur` | Ready | KR4268X2 onekeylog | | NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers | | NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems | diff --git a/bible-local/10-decisions.md b/bible-local/10-decisions.md index ac46625..eeab8ac 100644 --- a/bible-local/10-decisions.md +++ b/bible-local/10-decisions.md @@ -258,6 +258,9 @@ at parse time before storing in any model struct. Use the regex **Date:** 2026-03-12 **Context:** `shouldAdaptiveNVMeProbe` was introduced in `2fa4a12` to recover NVMe drives on Supermicro BMCs that expose empty `Drives` collections but serve disks at direct `Disk.Bay.N` + +--- + paths. The function returns `true` for any chassis with an empty `Members` array. On Supermicro HGX systems (SYS-A21GE-NBRT and similar) ~35 sub-chassis (GPU, NVSwitch, PCIeRetimer, ERoT, IRoT, BMC, FPGA) all carry `ChassisType=Module/Component/Zone` and @@ -976,3 +979,24 @@ the producer utility and archive importer. - Adding support required only a thin archive adapter instead of a full hardware parser. - If the upstream utility changes the embedded snapshot schema, the `easy_bee` adapter is the only place that must be updated. + +--- + +## ADL-038 — HPE AHS parser uses hybrid extraction instead of full `zbb` schema decoding + +**Date:** 2026-03-30 +**Context:** HPE iLO Active Health System exports (`.ahs`) are proprietary `ABJR` containers +with gzip-compressed `zbb` payloads. The sample inventory data contains two practical signal +families: printable SMBIOS/FRU-style strings and embedded Redfish JSON subtrees, especially for +storage controllers and drives. Full `zbb` binary schema decoding is not documented and would add +significant complexity before proving user value. +**Decision:** Support HPE AHS with a hybrid parser: +- decode the outer `ABJR` container +- gunzip embedded members when applicable +- extract inventory from printable SMBIOS/FRU payloads +- extract storage/controller details from embedded Redfish JSON objects +- do not attempt complete semantic decoding of the internal `zbb` record format +**Consequences:** +- Parser reaches inventory-grade usefulness quickly for HPE `.ahs` uploads. +- Storage inventory is stronger than text-only parsing because it reuses structured Redfish data when present. +- Future deeper `zbb` decoding can be added incrementally without replacing the current parser contract. diff --git a/internal/collector/redfish.go b/internal/collector/redfish.go index 54da9bf..a5d4b72 100644 --- a/internal/collector/redfish.go +++ b/internal/collector/redfish.go @@ -1606,6 +1606,7 @@ func (c *RedfishConnector) collectRawRedfishTree(ctx context.Context, client *ht crawlStart := time.Now() memoryClient := c.httpClientWithTimeout(req, redfishSnapshotMemoryRequestTimeout()) memoryGate := make(chan struct{}, redfishSnapshotMemoryConcurrency()) + postProbeClient := c.httpClientWithTimeout(req, redfishSnapshotPostProbeRequestTimeout()) branchLimiter := newRedfishSnapshotBranchLimiter(redfishSnapshotBranchConcurrency()) branchRetryPause := redfishSnapshotBranchRequeueBackoff() timings := newRedfishPathTimingCollector(4) @@ -1908,7 +1909,7 @@ func (c *RedfishConnector) collectRawRedfishTree(ctx context.Context, client *ht ETASeconds: int(estimateProgressETA(postProbeStart, i, len(postProbeCollections), 3*time.Second).Seconds()), }) } - for childPath, doc := range c.probeDirectRedfishCollectionChildren(ctx, client, req, baseURL, path) { + for childPath, doc := range c.probeDirectRedfishCollectionChildren(ctx, postProbeClient, req, baseURL, path) { if _, exists := out[childPath]; exists { continue } @@ -2158,6 +2159,12 @@ func shouldAdaptivePostProbeCollectionPath(path string, collectionDoc map[string if len(memberRefs) == 0 { return true } + // If the collection reports an explicit non-zero member count that already + // matches the number of discovered member refs, every member is accounted + // for and numeric probing cannot find anything new. + if odataCount := asInt(collectionDoc["Members@odata.count"]); odataCount > 0 && odataCount == len(memberRefs) { + return false + } return redfishCollectionHasNumericMemberRefs(memberRefs) } @@ -2350,6 +2357,18 @@ func redfishSnapshotRequestTimeout() time.Duration { return 12 * time.Second } +func redfishSnapshotPostProbeRequestTimeout() time.Duration { + if v := strings.TrimSpace(os.Getenv("LOGPILE_REDFISH_POSTPROBE_TIMEOUT")); v != "" { + if d, err := time.ParseDuration(v); err == nil && d > 0 { + return d + } + } + // Post-probe probes non-existent numeric paths expecting fast 404s. + // A short timeout prevents BMCs that hang on unknown paths from stalling + // the entire collection for minutes (e.g. HPE iLO on NetworkAdapters Ports). + return 4 * time.Second +} + func redfishSnapshotWorkers(tuning redfishprofile.AcquisitionTuning) int { if tuning.SnapshotWorkers >= 1 && tuning.SnapshotWorkers <= 16 { return tuning.SnapshotWorkers @@ -2852,6 +2871,8 @@ func shouldCrawlPath(path string) bool { "/GetServerAllUSBStatus", "/Oem/Public/KVM", "/SecureBoot/SecureBootDatabases", + // HPE iLO WorkloadPerformanceAdvisor — operational/advisory data, not inventory. + "/WorkloadPerformanceAdvisor", } { if strings.Contains(normalized, part) { return false @@ -5047,6 +5068,16 @@ func isVirtualStorageDrive(doc map[string]interface{}) bool { return false } +// isAbsentDriveDoc returns true when the drive document represents an empty bay +// with no installed media (Status.State == "Absent"). These should be excluded +// from the storage inventory. +func isAbsentDriveDoc(doc map[string]interface{}) bool { + if status, ok := doc["Status"].(map[string]interface{}); ok { + return strings.EqualFold(asString(status["State"]), "Absent") + } + return strings.EqualFold(asString(doc["Status"]), "Absent") +} + func looksLikeDrive(doc map[string]interface{}) bool { if asString(doc["MediaType"]) != "" { return true diff --git a/internal/collector/redfish_replay.go b/internal/collector/redfish_replay.go index f9cf189..5adbf8b 100644 --- a/internal/collector/redfish_replay.go +++ b/internal/collector/redfish_replay.go @@ -96,6 +96,7 @@ func ReplayRedfishFromRawPayloads(rawPayloads map[string]any, emit ProgressFn) ( networkProtocolDoc, _ := r.getJSON(joinPath(primaryManager, "/NetworkProtocol")) firmware := parseFirmware(systemDoc, biosDoc, managerDoc, networkProtocolDoc) firmware = dedupeFirmwareInfo(append(firmware, r.collectFirmwareInventory()...)) + firmware = filterStorageDriveFirmware(firmware, storageDevices) bmcManagementSummary := r.collectBMCManagementSummary(managerPaths) boardInfo.BMCMACAddress = strings.TrimSpace(firstNonEmpty( asString(bmcManagementSummary["mac_address"]), @@ -498,6 +499,10 @@ func (r redfishSnapshotReader) collectFirmwareInventory() []models.FirmwareInfo if strings.TrimSpace(version) == "" { continue } + // Skip placeholder version strings that carry no useful information. + if strings.EqualFold(strings.TrimSpace(version), "N/A") { + continue + } name := firmwareInventoryDeviceName(doc) name = strings.TrimSpace(name) if name == "" { @@ -550,6 +555,32 @@ func dedupeFirmwareInfo(items []models.FirmwareInfo) []models.FirmwareInfo { return out } +// filterStorageDriveFirmware removes from fw any entries whose DeviceName+Version +// already appear as a storage drive's Model+Firmware. Drive firmware is already +// represented in the Storage section and should not be duplicated in the general +// firmware list. +func filterStorageDriveFirmware(fw []models.FirmwareInfo, storage []models.Storage) []models.FirmwareInfo { + if len(storage) == 0 { + return fw + } + driveFW := make(map[string]struct{}, len(storage)) + for _, d := range storage { + model := strings.ToLower(strings.TrimSpace(d.Model)) + rev := strings.ToLower(strings.TrimSpace(d.Firmware)) + if model != "" && rev != "" { + driveFW[model+"|"+rev] = struct{}{} + } + } + out := fw[:0:0] + for _, f := range fw { + key := strings.ToLower(strings.TrimSpace(f.DeviceName)) + "|" + strings.ToLower(strings.TrimSpace(f.Version)) + if _, skip := driveFW[key]; !skip { + out = append(out, f) + } + } + return out +} + func (r redfishSnapshotReader) collectThresholdSensors(chassisPaths []string) []models.SensorReading { out := make([]models.SensorReading, 0) seen := make(map[string]struct{}) @@ -1261,6 +1292,12 @@ func (r redfishSnapshotReader) collectProcessors(systemPath string) []models.CPU !strings.EqualFold(pt, "CPU") && !strings.EqualFold(pt, "General") { continue } + // Skip absent processor sockets — empty slots with no CPU installed. + if status, ok := doc["Status"].(map[string]interface{}); ok { + if strings.EqualFold(asString(status["State"]), "Absent") { + continue + } + } cpu := parseCPUs([]map[string]interface{}{doc})[0] if cpu.Socket == 0 && socketIdx > 0 && strings.TrimSpace(asString(doc["Socket"])) == "" { cpu.Socket = socketIdx @@ -1287,6 +1324,10 @@ func (r redfishSnapshotReader) collectMemory(systemPath string) []models.MemoryD out := make([]models.MemoryDIMM, 0, len(memberDocs)) for _, doc := range memberDocs { dimm := parseMemory([]map[string]interface{}{doc})[0] + // Skip empty DIMM slots — no installed memory. + if !dimm.Present { + continue + } supplementalDocs := r.getLinkedSupplementalDocs(doc, "MemoryMetrics", "EnvironmentMetrics", "Metrics") if len(supplementalDocs) > 0 { dimm.Details = mergeGenericDetails(dimm.Details, redfishMemoryDetailsAcrossDocs(doc, supplementalDocs...)) diff --git a/internal/collector/redfish_replay_storage.go b/internal/collector/redfish_replay_storage.go index 54f9bf1..2ee27c3 100644 --- a/internal/collector/redfish_replay_storage.go +++ b/internal/collector/redfish_replay_storage.go @@ -14,13 +14,16 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro driveDocs, err := r.getCollectionMembers(driveCollectionPath) if err == nil { for _, driveDoc := range driveDocs { - if !isVirtualStorageDrive(driveDoc) { + if !isAbsentDriveDoc(driveDoc) && !isVirtualStorageDrive(driveDoc) { supplementalDocs := r.getLinkedSupplementalDocs(driveDoc, "DriveMetrics", "EnvironmentMetrics", "Metrics") out = append(out, parseDriveWithSupplementalDocs(driveDoc, supplementalDocs...)) } } if len(driveDocs) == 0 { for _, driveDoc := range r.probeDirectDiskBayChildren(driveCollectionPath) { + if isAbsentDriveDoc(driveDoc) { + continue + } supplementalDocs := r.getLinkedSupplementalDocs(driveDoc, "DriveMetrics", "EnvironmentMetrics", "Metrics") out = append(out, parseDriveWithSupplementalDocs(driveDoc, supplementalDocs...)) } @@ -43,7 +46,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro if err != nil { continue } - if !isVirtualStorageDrive(driveDoc) { + if !isAbsentDriveDoc(driveDoc) && !isVirtualStorageDrive(driveDoc) { supplementalDocs := r.getLinkedSupplementalDocs(driveDoc, "DriveMetrics", "EnvironmentMetrics", "Metrics") out = append(out, parseDriveWithSupplementalDocs(driveDoc, supplementalDocs...)) } @@ -51,7 +54,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro continue } if looksLikeDrive(member) { - if isVirtualStorageDrive(member) { + if isAbsentDriveDoc(member) || isVirtualStorageDrive(member) { continue } supplementalDocs := r.getLinkedSupplementalDocs(member, "DriveMetrics", "EnvironmentMetrics", "Metrics") @@ -63,14 +66,14 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro driveDocs, err := r.getCollectionMembers(joinPath(enclosurePath, "/Drives")) if err == nil { for _, driveDoc := range driveDocs { - if looksLikeDrive(driveDoc) && !isVirtualStorageDrive(driveDoc) { + if looksLikeDrive(driveDoc) && !isAbsentDriveDoc(driveDoc) && !isVirtualStorageDrive(driveDoc) { supplementalDocs := r.getLinkedSupplementalDocs(driveDoc, "DriveMetrics", "EnvironmentMetrics", "Metrics") out = append(out, parseDriveWithSupplementalDocs(driveDoc, supplementalDocs...)) } } if len(driveDocs) == 0 { for _, driveDoc := range r.probeDirectDiskBayChildren(joinPath(enclosurePath, "/Drives")) { - if isVirtualStorageDrive(driveDoc) { + if isAbsentDriveDoc(driveDoc) || isVirtualStorageDrive(driveDoc) { continue } out = append(out, parseDrive(driveDoc)) @@ -83,7 +86,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro if len(plan.KnownStorageDriveCollections) > 0 { for _, driveDoc := range r.collectKnownStorageMembers(systemPath, plan.KnownStorageDriveCollections) { - if looksLikeDrive(driveDoc) && !isVirtualStorageDrive(driveDoc) { + if looksLikeDrive(driveDoc) && !isAbsentDriveDoc(driveDoc) && !isVirtualStorageDrive(driveDoc) { supplementalDocs := r.getLinkedSupplementalDocs(driveDoc, "DriveMetrics", "EnvironmentMetrics", "Metrics") out = append(out, parseDriveWithSupplementalDocs(driveDoc, supplementalDocs...)) } @@ -98,7 +101,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro } for _, devAny := range devices { devDoc, ok := devAny.(map[string]interface{}) - if !ok || !looksLikeDrive(devDoc) || isVirtualStorageDrive(devDoc) { + if !ok || !looksLikeDrive(devDoc) || isAbsentDriveDoc(devDoc) || isVirtualStorageDrive(devDoc) { continue } out = append(out, parseDrive(devDoc)) @@ -112,7 +115,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro continue } for _, driveDoc := range driveDocs { - if !looksLikeDrive(driveDoc) || isVirtualStorageDrive(driveDoc) { + if !looksLikeDrive(driveDoc) || isAbsentDriveDoc(driveDoc) || isVirtualStorageDrive(driveDoc) { continue } out = append(out, parseDrive(driveDoc)) @@ -124,7 +127,7 @@ func (r redfishSnapshotReader) collectStorage(systemPath string, plan redfishpro continue } for _, driveDoc := range r.probeSupermicroNVMeDiskBays(chassisPath) { - if !looksLikeDrive(driveDoc) || isVirtualStorageDrive(driveDoc) { + if !looksLikeDrive(driveDoc) || isAbsentDriveDoc(driveDoc) || isVirtualStorageDrive(driveDoc) { continue } out = append(out, parseDrive(driveDoc)) diff --git a/internal/collector/redfishprofile/profile_hpe.go b/internal/collector/redfishprofile/profile_hpe.go new file mode 100644 index 0000000..cb98ca9 --- /dev/null +++ b/internal/collector/redfishprofile/profile_hpe.go @@ -0,0 +1,67 @@ +package redfishprofile + +func hpeProfile() Profile { + return staticProfile{ + name: "hpe", + priority: 20, + safeForFallback: true, + matchFn: func(s MatchSignals) int { + score := 0 + if containsFold(s.SystemManufacturer, "hpe") || + containsFold(s.SystemManufacturer, "hewlett packard") || + containsFold(s.ChassisManufacturer, "hpe") || + containsFold(s.ChassisManufacturer, "hewlett packard") { + score += 80 + } + for _, ns := range s.OEMNamespaces { + if containsFold(ns, "hpe") { + score += 30 + break + } + } + if containsFold(s.ServiceRootProduct, "ilo") { + score += 30 + } + if containsFold(s.ManagerManufacturer, "hpe") || containsFold(s.ManagerManufacturer, "ilo") { + score += 20 + } + return min(score, 100) + }, + extendAcquisition: func(plan *AcquisitionPlan, _ MatchSignals) { + // HPE ProLiant SmartStorage RAID controller inventory is not reachable + // via standard Redfish Storage paths — it requires the HPE OEM SmartStorage tree. + ensureScopedPathPolicy(plan, AcquisitionScopedPathPolicy{ + SystemCriticalSuffixes: []string{ + "/SmartStorage", + "/SmartStorageConfig", + }, + ManagerCriticalSuffixes: []string{ + "/LicenseService", + }, + }) + // HPE iLO responds more slowly than average BMCs under load; give the + // ETA estimator a realistic baseline so progress reports are accurate. + ensureETABaseline(plan, AcquisitionETABaseline{ + DiscoverySeconds: 12, + SnapshotSeconds: 180, + PrefetchSeconds: 30, + CriticalPlanBSeconds: 40, + ProfilePlanBSeconds: 25, + }) + ensureRecoveryPolicy(plan, AcquisitionRecoveryPolicy{ + EnableProfilePlanB: true, + }) + // HPE iLO starts throttling under high request rates. Setting a higher + // latency tolerance prevents the adaptive throttler from treating normal + // iLO slowness as a reason to stall the collection. + ensureRatePolicy(plan, AcquisitionRatePolicy{ + TargetP95LatencyMS: 1200, + ThrottleP95LatencyMS: 2500, + MinSnapshotWorkers: 2, + MinPrefetchWorkers: 1, + DisablePrefetchOnErrors: true, + }) + addPlanNote(plan, "hpe ilo acquisition extensions enabled") + }, + } +} diff --git a/internal/collector/redfishprofile/profiles_common.go b/internal/collector/redfishprofile/profiles_common.go index 55f2e7b..154931d 100644 --- a/internal/collector/redfishprofile/profiles_common.go +++ b/internal/collector/redfishprofile/profiles_common.go @@ -55,6 +55,7 @@ func BuiltinProfiles() []Profile { msiProfile(), supermicroProfile(), dellProfile(), + hpeProfile(), inspurGroupOEMPlatformsProfile(), hgxProfile(), xfusionProfile(), diff --git a/internal/parser/archive.go b/internal/parser/archive.go index d495938..e34a72c 100644 --- a/internal/parser/archive.go +++ b/internal/parser/archive.go @@ -19,6 +19,7 @@ const maxZipArchiveSize = 50 * 1024 * 1024 const maxGzipDecompressedSize = 50 * 1024 * 1024 var supportedArchiveExt = map[string]struct{}{ + ".ahs": {}, ".gz": {}, ".tgz": {}, ".tar": {}, @@ -45,6 +46,8 @@ func ExtractArchive(archivePath string) ([]ExtractedFile, error) { ext := strings.ToLower(filepath.Ext(archivePath)) switch ext { + case ".ahs": + return extractSingleFile(archivePath) case ".gz", ".tgz": return extractTarGz(archivePath) case ".tar", ".sds": @@ -66,6 +69,8 @@ func ExtractArchiveFromReader(r io.Reader, filename string) ([]ExtractedFile, er ext := strings.ToLower(filepath.Ext(filename)) switch ext { + case ".ahs": + return extractSingleFileFromReader(r, filename) case ".gz", ".tgz": return extractTarGzFromReader(r, filename) case ".tar", ".sds": diff --git a/internal/parser/archive_test.go b/internal/parser/archive_test.go index 0dc56c7..4d9ceb9 100644 --- a/internal/parser/archive_test.go +++ b/internal/parser/archive_test.go @@ -76,6 +76,7 @@ func TestIsSupportedArchiveFilename(t *testing.T) { name string want bool }{ + {name: "HPE_CZ2D1X0GS3_20260330.ahs", want: true}, {name: "dump.tar.gz", want: true}, {name: "nvidia-bug-report-1651124000923.log.gz", want: true}, {name: "snapshot.zip", want: true}, @@ -124,3 +125,20 @@ func TestExtractArchiveFromReaderSDS(t *testing.T) { t.Fatalf("expected bmc/pack.info, got %q", files[0].Path) } } + +func TestExtractArchiveFromReaderAHS(t *testing.T) { + payload := []byte("ABJRtest") + files, err := ExtractArchiveFromReader(bytes.NewReader(payload), "sample.ahs") + if err != nil { + t.Fatalf("extract ahs from reader: %v", err) + } + if len(files) != 1 { + t.Fatalf("expected 1 extracted file, got %d", len(files)) + } + if files[0].Path != "sample.ahs" { + t.Fatalf("expected sample.ahs, got %q", files[0].Path) + } + if string(files[0].Content) != string(payload) { + t.Fatalf("content mismatch") + } +} diff --git a/internal/parser/vendors/hpe_ilo_ahs/parser.go b/internal/parser/vendors/hpe_ilo_ahs/parser.go new file mode 100644 index 0000000..20557f9 --- /dev/null +++ b/internal/parser/vendors/hpe_ilo_ahs/parser.go @@ -0,0 +1,1287 @@ +package hpe_ilo_ahs + +import ( + "bytes" + "compress/gzip" + "encoding/binary" + "encoding/json" + "fmt" + "io" + "path/filepath" + "regexp" + "sort" + "strconv" + "strings" + "time" + + "git.mchus.pro/mchus/logpile/internal/models" + "git.mchus.pro/mchus/logpile/internal/parser" +) + +const ( + parserVersion = "1.0.0" + ahsHeaderSize = 116 + maxGzipSize = 50 * 1024 * 1024 +) + +var ( + partNumberPattern = regexp.MustCompile(`(?i)^[a-z0-9]{1,4}\d{4,6}-[a-z0-9]{2,4}$`) + serverSerialRE = regexp.MustCompile(`(?i)(?:^|[_-])([a-z0-9]{10})(?:[_-]|\.)`) + dimmSlotRE = regexp.MustCompile(`^PROC\s+(\d+)\s+DIMM\s+(\d+)$`) + procSlotRE = regexp.MustCompile(`^Proc\s+(\d+)$`) + psuSlotRE = regexp.MustCompile(`^Power Supply\s+(\d+)$`) + eventTimeRE = regexp.MustCompile(`^\d{2}/\d{2}/\d{4} \d{2}:\d{2}:\d{2}$`) +) + +func init() { + parser.Register(&Parser{}) +} + +type Parser struct{} + +func (p *Parser) Name() string { return "HPE iLO AHS Parser" } +func (p *Parser) Vendor() string { return "hpe_ilo_ahs" } +func (p *Parser) Version() string { return parserVersion } + +func (p *Parser) Detect(files []parser.ExtractedFile) int { + if len(files) != 1 { + return 0 + } + + file := files[0] + if len(file.Content) < ahsHeaderSize || !bytes.HasPrefix(file.Content, []byte("ABJR")) { + return 0 + } + + score := 55 + name := strings.ToLower(file.Path) + if strings.HasSuffix(name, ".ahs") { + score += 30 + } + if bytes.Contains(file.Content, []byte("CUST_INFO.DAT")) { + score += 10 + } + if bytes.Contains(file.Content, []byte(".zbb")) || bytes.Contains(file.Content, []byte("ilo_boot_support.zbb")) { + score += 10 + } + if score > 100 { + score = 100 + } + return score +} + +func (p *Parser) Parse(files []parser.ExtractedFile) (*models.AnalysisResult, error) { + if len(files) == 0 { + return emptyResult(), nil + } + + entries, err := parseAHSContainer(files[0].Content) + if err != nil { + return nil, fmt.Errorf("parse ahs container: %w", err) + } + + result := emptyResult() + result.SourceType = models.SourceTypeArchive + + tokens := make([]string, 0, 2048) + redfishDocs := make(map[string]map[string]any) + rawMetadata := make([]map[string]any, 0, len(entries)) + + for _, entry := range entries { + rawMetadata = append(rawMetadata, map[string]any{ + "name": entry.Name, + "compressed": entry.Compressed, + "compressed_size": len(entry.Payload), + "uncompressed_size": len(entry.Content), + "flag": entry.Flag, + }) + if len(entry.Content) == 0 { + continue + } + tokens = append(tokens, printableTokens(entry.Content, 3)...) + for path, doc := range extractEmbeddedRedfishDocs(entry.Content) { + redfishDocs[path] = doc + } + } + + if len(rawMetadata) > 0 { + result.RawPayloads = map[string]any{ + "hpe_ahs_entries": rawMetadata, + } + } + + board := parseBoardInfo(tokens, files[0].Path) + result.Hardware.BoardInfo = board + if board.ProductName != "" || board.SerialNumber != "" || board.PartNumber != "" { + result.FRU = append(result.FRU, models.FRUInfo{ + Description: "System", + Manufacturer: board.Manufacturer, + ProductName: board.ProductName, + SerialNumber: board.SerialNumber, + PartNumber: board.PartNumber, + Version: board.Version, + }) + } + + result.Hardware.CPUs = dedupeCPUs(parseCPUs(tokens)) + result.Hardware.Memory = dedupeMemory(parseDIMMs(tokens)) + result.Hardware.PowerSupply = dedupePSUs(parsePSUs(tokens)) + result.Hardware.NetworkAdapters = dedupeNetworkAdapters(parseNetworkAdapters(tokens)) + result.Hardware.Firmware = dedupeFirmware(parseFirmware(tokens)) + + storage, volumes, controllerDevices, controllerFW := parseRedfishStorage(redfishDocs) + result.Hardware.Storage = dedupeStorage(storage) + result.Hardware.Volumes = volumes + result.Hardware.Firmware = dedupeFirmware(append(result.Hardware.Firmware, controllerFW...)) + + result.Events = dedupeEvents(parseEvents(tokens)) + if result.CollectedAt.IsZero() { + for _, ev := range result.Events { + if ev.Timestamp.After(result.CollectedAt) { + result.CollectedAt = ev.Timestamp.UTC() + } + } + } + + result.Hardware.Devices = buildDevices( + result.Hardware.BoardInfo, + result.Hardware.CPUs, + result.Hardware.Memory, + result.Hardware.Storage, + result.Hardware.NetworkAdapters, + result.Hardware.PowerSupply, + controllerDevices, + ) + + return result, nil +} + +type ahsEntry struct { + Name string + Flag uint32 + Payload []byte + Content []byte + Compressed bool +} + +func emptyResult() *models.AnalysisResult { + return &models.AnalysisResult{ + Events: make([]models.Event, 0), + FRU: make([]models.FRUInfo, 0), + Sensors: make([]models.SensorReading, 0), + Hardware: &models.HardwareConfig{ + Firmware: make([]models.FirmwareInfo, 0), + Devices: make([]models.HardwareDevice, 0), + CPUs: make([]models.CPU, 0), + Memory: make([]models.MemoryDIMM, 0), + Storage: make([]models.Storage, 0), + Volumes: make([]models.StorageVolume, 0), + PCIeDevices: make([]models.PCIeDevice, 0), + GPUs: make([]models.GPU, 0), + NetworkCards: make([]models.NIC, 0), + NetworkAdapters: make([]models.NetworkAdapter, 0), + PowerSupply: make([]models.PSU, 0), + }, + } +} + +func parseAHSContainer(data []byte) ([]ahsEntry, error) { + entries := make([]ahsEntry, 0, 8) + offset := 0 + + for offset < len(data) { + if offset+ahsHeaderSize > len(data) { + return nil, fmt.Errorf("truncated header at offset %d", offset) + } + if !bytes.Equal(data[offset:offset+4], []byte("ABJR")) { + return nil, fmt.Errorf("invalid magic at offset %d", offset) + } + + size := int(binary.LittleEndian.Uint32(data[offset+8 : offset+12])) + flag := binary.LittleEndian.Uint32(data[offset+16 : offset+20]) + name := strings.TrimRight(string(data[offset+20:offset+52]), "\x00") + start := offset + ahsHeaderSize + end := start + size + if size < 0 || end > len(data) { + return nil, fmt.Errorf("invalid payload size for %q", name) + } + + payload := append([]byte(nil), data[start:end]...) + content := payload + compressed := len(payload) >= 2 && payload[0] == 0x1f && payload[1] == 0x8b + if compressed { + decoded, err := gunzipLimited(payload) + if err == nil { + content = decoded + } + } + + entries = append(entries, ahsEntry{ + Name: name, + Flag: flag, + Payload: payload, + Content: content, + Compressed: compressed, + }) + offset = end + } + + return entries, nil +} + +func gunzipLimited(payload []byte) ([]byte, error) { + gr, err := gzip.NewReader(bytes.NewReader(payload)) + if err != nil { + return nil, err + } + defer gr.Close() + + buf, err := io.ReadAll(io.LimitReader(gr, maxGzipSize+1)) + if err != nil { + return nil, err + } + if len(buf) > maxGzipSize { + return nil, fmt.Errorf("gzip payload exceeded %d bytes", maxGzipSize) + } + return buf, nil +} + +func printableTokens(data []byte, minLen int) []string { + out := make([]string, 0, 256) + start := -1 + for i, b := range data { + if b >= 32 && b <= 126 { + if start == -1 { + start = i + } + continue + } + if start != -1 && i-start >= minLen { + token := strings.TrimSpace(string(data[start:i])) + if token != "" { + out = append(out, token) + } + } + start = -1 + } + if start != -1 && len(data)-start >= minLen { + token := strings.TrimSpace(string(data[start:])) + if token != "" { + out = append(out, token) + } + } + return out +} + +func extractEmbeddedRedfishDocs(data []byte) map[string]map[string]any { + out := make(map[string]map[string]any) + marker := []byte(`{"@odata`) + for offset := 0; offset < len(data); { + idx := bytes.Index(data[offset:], marker) + if idx < 0 { + break + } + start := offset + idx + end, ok := findBalancedJSONObject(data, start) + if !ok { + offset = start + 1 + continue + } + + var doc map[string]any + if err := json.Unmarshal(data[start:end], &doc); err == nil { + path := strings.TrimSpace(asString(doc["@odata.id"])) + if strings.HasPrefix(path, "/redfish/") { + out[path] = doc + } + } + offset = end + } + return out +} + +func findBalancedJSONObject(data []byte, start int) (int, bool) { + if start >= len(data) || data[start] != '{' { + return 0, false + } + depth := 0 + inString := false + escaped := false + + for i := start; i < len(data); i++ { + c := data[i] + if inString { + switch { + case escaped: + escaped = false + case c == '\\': + escaped = true + case c == '"': + inString = false + } + continue + } + + switch c { + case '"': + inString = true + case '{': + depth++ + case '}': + depth-- + if depth == 0 { + return i + 1, true + } + } + } + + return 0, false +} + +func parseBoardInfo(tokens []string, path string) models.BoardInfo { + var board models.BoardInfo + + for i := 0; i+3 < len(tokens); i++ { + manufacturer := strings.TrimSpace(tokens[i]) + model := sanitizeModel(tokens[i+1]) + if !isHPEManufacturer(manufacturer) || !looksLikeServerModel(model) { + continue + } + board.Manufacturer = "HPE" + board.ProductName = model + if isLikelySerial(tokens[i+2]) { + board.SerialNumber = tokens[i+2] + } + if looksLikePartNumber(tokens[i+3]) { + board.PartNumber = tokens[i+3] + } + break + } + + if board.Manufacturer == "" && strings.Contains(strings.ToUpper(filepath.Base(path)), "HPE") { + board.Manufacturer = "HPE" + } + if board.SerialNumber == "" { + if match := serverSerialRE.FindStringSubmatch(strings.ToUpper(filepath.Base(path))); len(match) == 2 { + board.SerialNumber = match[1] + } + } + if board.ProductName == "" { + for _, token := range tokens { + if looksLikeServerModel(token) { + board.ProductName = sanitizeModel(token) + break + } + } + } + return board +} + +func parseCPUs(tokens []string) []models.CPU { + out := make([]models.CPU, 0, 2) + for i := 0; i+2 < len(tokens); i++ { + match := procSlotRE.FindStringSubmatch(tokens[i]) + if len(match) != 2 { + continue + } + socket, _ := strconv.Atoi(match[1]) + model := "" + manufacturer := "" + for j := i + 1; j < len(tokens) && j <= i+5; j++ { + if strings.HasPrefix(tokens[j], "PROC ") || procSlotRE.MatchString(tokens[j]) { + break + } + if manufacturer == "" && looksLikeCPUVendor(tokens[j]) { + manufacturer = tokens[j] + continue + } + if looksLikeCPUModel(tokens[j]) { + model = tokens[j] + break + } + } + if model == "" { + continue + } + cpu := models.CPU{ + Socket: socket, + Model: model, + Description: manufacturer, + Status: "ok", + } + out = append(out, cpu) + } + return out +} + +func parseDIMMs(tokens []string) []models.MemoryDIMM { + out := make([]models.MemoryDIMM, 0, 16) + for i := 0; i+3 < len(tokens); i++ { + match := dimmSlotRE.FindStringSubmatch(tokens[i]) + if len(match) != 3 { + continue + } + slot := tokens[i] + manufacturer := tokens[i+1] + partNumber := tokens[i+2] + serial := tokens[i+3] + if isUnavailable(partNumber) || isUnavailable(serial) { + continue + } + if isUnavailable(manufacturer) || strings.EqualFold(manufacturer, "unknown") { + manufacturer = "" + } + out = append(out, models.MemoryDIMM{ + Slot: slot, + Location: slot, + Present: true, + Manufacturer: manufacturer, + PartNumber: partNumber, + SerialNumber: serial, + Status: "ok", + }) + } + return out +} + +func parsePSUs(tokens []string) []models.PSU { + out := make([]models.PSU, 0, 4) + for i := 0; i+2 < len(tokens); i++ { + match := psuSlotRE.FindStringSubmatch(tokens[i]) + if len(match) != 2 { + continue + } + slot := "PSU " + match[1] + serial := tokens[i+1] + partNumber := tokens[i+2] + if isUnavailable(serial) && isUnavailable(partNumber) { + continue + } + psu := models.PSU{ + Slot: slot, + Present: true, + Model: valueOr(partNumber, "Power Supply"), + Vendor: "HPE", + SerialNumber: cleanUnavailable(serial), + PartNumber: cleanUnavailable(partNumber), + Status: "ok", + } + out = append(out, psu) + } + return out +} + +type pcieSequence struct { + UEFIPath string + Code string + Fields []string +} + +func parseNetworkAdapters(tokens []string) []models.NetworkAdapter { + sequences := collectPCIeSequences(tokens) + out := make([]models.NetworkAdapter, 0, 4) + + for _, seq := range sequences { + if strings.Contains(seq.Code, "DriveBay") { + continue + } + if len(seq.Fields) == 0 { + continue + } + + title := seq.Fields[0] + if strings.Contains(strings.ToLower(title), "empty") { + continue + } + + if !looksLikeNetworkTitle(seq.Code, title, seq.Fields) { + continue + } + + location := "" + model := title + description := "" + partNumber := "" + serial := "" + firmware := "" + + for _, field := range seq.Fields[1:] { + switch { + case location == "" && looksLikeLocation(field): + location = field + case partNumber == "" && looksLikePartNumber(field): + partNumber = field + case serial == "" && isLikelySerial(field): + serial = field + case firmware == "" && looksLikeVersion(field): + firmware = field + case model == title && looksLikeConcreteModel(field): + model = field + case description == "" && field != model: + description = field + } + } + + if model == "Network Controller" && description != "" { + model, description = description, title + } + + out = append(out, models.NetworkAdapter{ + Slot: slotLabelFromCode(seq.Code), + Location: valueOr(location, slotLabelFromCode(seq.Code)), + Present: true, + Model: model, + Description: description, + Vendor: inferVendor(model), + SerialNumber: serial, + PartNumber: partNumber, + Firmware: firmware, + Status: "ok", + Details: map[string]any{ + "uefi_path": seq.UEFIPath, + "source": "smbios_slot_inventory", + }, + }) + } + + return out +} + +func collectPCIeSequences(tokens []string) []pcieSequence { + out := make([]pcieSequence, 0, 16) + for i := 0; i < len(tokens); i++ { + if !strings.HasPrefix(tokens[i], "PciRoot(") { + continue + } + if i+1 >= len(tokens) { + continue + } + seq := pcieSequence{ + UEFIPath: tokens[i], + Code: tokens[i+1], + Fields: make([]string, 0, 6), + } + for j := i + 2; j < len(tokens) && len(seq.Fields) < 6; j++ { + if strings.HasPrefix(tokens[j], "PciRoot(") || dimmSlotRE.MatchString(tokens[j]) || procSlotRE.MatchString(tokens[j]) || psuSlotRE.MatchString(tokens[j]) { + break + } + seq.Fields = append(seq.Fields, tokens[j]) + } + out = append(out, seq) + } + return out +} + +func parseFirmware(tokens []string) []models.FirmwareInfo { + out := make([]models.FirmwareInfo, 0, 8) + seen := make(map[string]bool) + + for _, token := range tokens { + if strings.HasPrefix(token, "iLO ") && strings.Contains(token, " built on ") { + version := token + build := "" + if idx := strings.Index(token, " built on "); idx > 0 { + version = strings.TrimSpace(token[:idx]) + build = strings.TrimSpace(token[idx+10:]) + } + name := version + if fields := strings.Fields(version); len(fields) >= 2 { + name = strings.Join(fields[:2], " ") + version = strings.TrimSpace(strings.TrimPrefix(version, name)) + } + appendFirmware(&out, seen, models.FirmwareInfo{ + DeviceName: name, + Version: strings.TrimSpace(version), + BuildTime: build, + }) + } + } + + for i := 0; i+1 < len(tokens); i++ { + name := tokens[i] + version := tokens[i+1] + if !isTopLevelFirmwareLabel(name) || !looksLikeVersion(version) { + continue + } + appendFirmware(&out, seen, models.FirmwareInfo{ + DeviceName: name, + Version: version, + }) + } + + return out +} + +func parseRedfishStorage(docs map[string]map[string]any) ([]models.Storage, []models.StorageVolume, []models.HardwareDevice, []models.FirmwareInfo) { + paths := make([]string, 0, len(docs)) + for path := range docs { + paths = append(paths, path) + } + sort.Strings(paths) + + storage := make([]models.Storage, 0, 8) + volumes := make([]models.StorageVolume, 0, 4) + devices := make([]models.HardwareDevice, 0, 4) + firmware := make([]models.FirmwareInfo, 0, 4) + + for _, path := range paths { + doc := docs[path] + docType := asString(doc["@odata.type"]) + switch { + case strings.Contains(docType, "#StorageController."): + slot := redfishServiceLabel(doc, "Location", "PartLocation", "ServiceLabel") + model := valueOr(asString(doc["Model"]), asString(doc["Name"])) + partNumber := strings.TrimSpace(asString(doc["PartNumber"])) + sku := strings.TrimSpace(asString(doc["SKU"])) + serial := strings.TrimSpace(asString(doc["SerialNumber"])) + fw := strings.TrimSpace(asString(doc["FirmwareVersion"])) + device := models.HardwareDevice{ + ID: "hpe-ctrl-" + redfishID(path), + Kind: models.DeviceKindStorage, + Source: "redfish", + Slot: slot, + Location: slot, + DeviceClass: "storage_controller", + Model: model, + PartNumber: valueOr(partNumber, sku), + Manufacturer: strings.TrimSpace(asString(doc["Manufacturer"])), + SerialNumber: serial, + Firmware: fw, + Status: redfishStatus(doc["Status"]), + Details: map[string]any{ + "odata_id": path, + "part_number": partNumber, + "sku": sku, + }, + } + if width := asInt(doc, "PCIeInterface", "LanesInUse"); width > 0 { + device.LinkWidth = width + } + if speed := strings.TrimSpace(asString(nested(doc, "PCIeInterface", "PCIeType"))); speed != "" { + device.LinkSpeed = speed + } + devices = append(devices, device) + if fw != "" { + firmware = append(firmware, models.FirmwareInfo{ + DeviceName: model, + Description: slot, + Version: fw, + }) + } + + case strings.Contains(docType, "#Drive."): + if strings.EqualFold(redfishStatus(doc["Status"]), "absent") { + continue + } + capacity := asInt64(doc["CapacityBytes"]) + slot := redfishServiceLabel(doc, "PhysicalLocation", "PartLocation", "ServiceLabel") + if slot == "" { + slot = redfishServiceLabel(doc, "Location", "PartLocation", "ServiceLabel") + } + endurance := asOptionalInt(doc["PredictedMediaLifeLeftPercent"]) + entry := models.Storage{ + Slot: slot, + Type: valueOr(asString(doc["MediaType"]), "Drive"), + Model: valueOr(asString(doc["Model"]), asString(doc["Name"])), + Description: strings.TrimSpace(asString(doc["Name"])), + SizeGB: bytesToDecimalGB(capacity), + SerialNumber: strings.TrimSpace(asString(doc["SerialNumber"])), + Firmware: strings.TrimSpace(asString(doc["Revision"])), + Interface: valueOr(asString(doc["Protocol"]), asString(doc["MediaType"])), + Present: true, + RemainingEndurancePct: endurance, + Status: redfishStatus(doc["Status"]), + Details: map[string]any{ + "odata_id": path, + "capacity_bytes": capacity, + }, + } + storage = append(storage, entry) + + case strings.Contains(docType, "#Volume.") && !strings.HasSuffix(path, "/Capabilities"): + volumes = append(volumes, models.StorageVolume{ + ID: strings.TrimSpace(asString(doc["Id"])), + Name: strings.TrimSpace(asString(doc["Name"])), + RAIDLevel: strings.TrimSpace(asString(doc["RAIDType"])), + CapacityBytes: asInt64(doc["CapacityBytes"]), + SizeGB: bytesToDecimalGB(asInt64(doc["CapacityBytes"])), + Status: redfishStatus(doc["Status"]), + }) + } + } + + return storage, dedupeVolumes(volumes), dedupeDevices(devices), dedupeFirmware(firmware) +} + +func buildDevices(board models.BoardInfo, cpus []models.CPU, memory []models.MemoryDIMM, storage []models.Storage, adapters []models.NetworkAdapter, psus []models.PSU, extras []models.HardwareDevice) []models.HardwareDevice { + devices := make([]models.HardwareDevice, 0, 1+len(cpus)+len(memory)+len(storage)+len(adapters)+len(psus)+len(extras)) + + if board.ProductName != "" || board.SerialNumber != "" { + devices = append(devices, models.HardwareDevice{ + ID: "hpe-board", + Kind: models.DeviceKindBoard, + Source: "smbios", + Model: board.ProductName, + Manufacturer: board.Manufacturer, + SerialNumber: board.SerialNumber, + PartNumber: board.PartNumber, + Status: "ok", + }) + } + + for _, cpu := range cpus { + devices = append(devices, models.HardwareDevice{ + ID: fmt.Sprintf("hpe-cpu-%d", cpu.Socket), + Kind: models.DeviceKindCPU, + Source: "smbios", + Slot: fmt.Sprintf("CPU %d", cpu.Socket), + Model: cpu.Model, + Manufacturer: strings.TrimSpace(cpu.Description), + Cores: cpu.Cores, + Threads: cpu.Threads, + FrequencyMHz: cpu.FrequencyMHz, + MaxFreqMHz: cpu.MaxFreqMHz, + Status: cpu.Status, + }) + } + + for _, dimm := range memory { + devices = append(devices, models.HardwareDevice{ + ID: "hpe-mem-" + sanitizeID(dimm.Slot), + Kind: models.DeviceKindMemory, + Source: "smbios", + Slot: dimm.Slot, + Location: dimm.Location, + Model: dimm.PartNumber, + Manufacturer: dimm.Manufacturer, + SerialNumber: dimm.SerialNumber, + PartNumber: dimm.PartNumber, + Present: boolPtr(dimm.Present), + Status: dimm.Status, + }) + } + + for _, disk := range storage { + devices = append(devices, models.HardwareDevice{ + ID: "hpe-disk-" + sanitizeID(valueOr(disk.SerialNumber, disk.Slot)), + Kind: models.DeviceKindStorage, + Source: "redfish", + Slot: disk.Slot, + Location: disk.Location, + Model: disk.Model, + Manufacturer: disk.Manufacturer, + SerialNumber: disk.SerialNumber, + Firmware: disk.Firmware, + Type: disk.Type, + Interface: disk.Interface, + Present: boolPtr(disk.Present), + SizeGB: disk.SizeGB, + Status: disk.Status, + RemainingEndurancePct: disk.RemainingEndurancePct, + }) + } + + for _, nic := range adapters { + devices = append(devices, models.HardwareDevice{ + ID: "hpe-net-" + sanitizeID(valueOr(nic.SerialNumber, nic.Slot+"-"+nic.Model)), + Kind: models.DeviceKindNetwork, + Source: "smbios", + Slot: nic.Slot, + Location: nic.Location, + Model: nic.Model, + Manufacturer: nic.Vendor, + SerialNumber: nic.SerialNumber, + PartNumber: nic.PartNumber, + Firmware: nic.Firmware, + PortCount: nic.PortCount, + PortType: nic.PortType, + MACAddresses: append([]string(nil), nic.MACAddresses...), + Present: boolPtr(nic.Present), + Status: nic.Status, + }) + } + + for _, psu := range psus { + devices = append(devices, models.HardwareDevice{ + ID: "hpe-psu-" + sanitizeID(valueOr(psu.SerialNumber, psu.Slot)), + Kind: models.DeviceKindPSU, + Source: "smbios", + Slot: psu.Slot, + Model: psu.Model, + Manufacturer: psu.Vendor, + SerialNumber: psu.SerialNumber, + PartNumber: psu.PartNumber, + Firmware: psu.Firmware, + WattageW: psu.WattageW, + InputType: psu.InputType, + Present: boolPtr(psu.Present), + Status: psu.Status, + }) + } + + devices = append(devices, extras...) + return dedupeDevices(devices) +} + +func parseEvents(tokens []string) []models.Event { + out := make([]models.Event, 0, 16) + for i := 0; i+1 < len(tokens); i++ { + if !eventTimeRE.MatchString(tokens[i]) { + continue + } + ts, err := time.ParseInLocation("01/02/2006 15:04:05", tokens[i], time.UTC) + if err != nil { + continue + } + + message := "" + for j := i + 1; j < len(tokens) && j <= i+4; j++ { + if eventTimeRE.MatchString(tokens[j]) { + break + } + if looksLikeEventMessage(tokens[j]) { + message = tokens[j] + break + } + } + if message == "" { + continue + } + + out = append(out, models.Event{ + Timestamp: ts.UTC(), + Source: "HPE iLO", + EventType: inferEventType(message), + Severity: inferSeverity(message), + Description: message, + RawData: message, + }) + } + return out +} + +func appendFirmware(dst *[]models.FirmwareInfo, seen map[string]bool, item models.FirmwareInfo) { + item.DeviceName = strings.TrimSpace(item.DeviceName) + item.Version = strings.TrimSpace(item.Version) + if item.DeviceName == "" || item.Version == "" { + return + } + key := item.DeviceName + "|" + item.Version + "|" + item.Description + if seen[key] { + return + } + seen[key] = true + *dst = append(*dst, item) +} + +func dedupeCPUs(items []models.CPU) []models.CPU { + seen := make(map[string]bool) + out := make([]models.CPU, 0, len(items)) + for _, item := range items { + key := fmt.Sprintf("%d|%s", item.Socket, item.Model) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeMemory(items []models.MemoryDIMM) []models.MemoryDIMM { + seen := make(map[string]bool) + out := make([]models.MemoryDIMM, 0, len(items)) + for _, item := range items { + key := valueOr(item.SerialNumber, item.Slot+"|"+item.PartNumber) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupePSUs(items []models.PSU) []models.PSU { + seen := make(map[string]bool) + out := make([]models.PSU, 0, len(items)) + for _, item := range items { + key := valueOr(item.SerialNumber, item.Slot+"|"+item.PartNumber) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeNetworkAdapters(items []models.NetworkAdapter) []models.NetworkAdapter { + seen := make(map[string]bool) + out := make([]models.NetworkAdapter, 0, len(items)) + for _, item := range items { + key := valueOr(item.SerialNumber, item.Slot+"|"+item.Model) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeStorage(items []models.Storage) []models.Storage { + seen := make(map[string]bool) + out := make([]models.Storage, 0, len(items)) + for _, item := range items { + key := valueOr(item.SerialNumber, item.Slot+"|"+item.Model) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeFirmware(items []models.FirmwareInfo) []models.FirmwareInfo { + seen := make(map[string]bool) + out := make([]models.FirmwareInfo, 0, len(items)) + for _, item := range items { + key := item.DeviceName + "|" + item.Version + "|" + item.Description + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeVolumes(items []models.StorageVolume) []models.StorageVolume { + seen := make(map[string]bool) + out := make([]models.StorageVolume, 0, len(items)) + for _, item := range items { + key := valueOr(item.ID, item.Name+"|"+item.Controller) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeDevices(items []models.HardwareDevice) []models.HardwareDevice { + seen := make(map[string]bool) + out := make([]models.HardwareDevice, 0, len(items)) + for _, item := range items { + key := valueOr(item.SerialNumber, item.Kind+"|"+item.Slot+"|"+item.Model) + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func dedupeEvents(items []models.Event) []models.Event { + seen := make(map[string]bool) + out := make([]models.Event, 0, len(items)) + for _, item := range items { + key := item.Timestamp.Format(time.RFC3339) + "|" + item.Description + if seen[key] { + continue + } + seen[key] = true + out = append(out, item) + } + return out +} + +func isHPEManufacturer(v string) bool { + v = strings.TrimSpace(strings.ToUpper(v)) + return v == "HPE" || v == "HP" +} + +func looksLikeServerModel(v string) bool { + v = sanitizeModel(v) + if v == "" { + return false + } + lower := strings.ToLower(v) + return strings.Contains(lower, "proliant") || strings.Contains(lower, "apollo") || strings.Contains(lower, "synergy") || strings.Contains(lower, "edgeline") +} + +func looksLikeCPUVendor(v string) bool { + return strings.Contains(v, "Intel") || strings.Contains(v, "AMD") +} + +func looksLikeCPUModel(v string) bool { + return strings.Contains(v, "Xeon") || strings.Contains(v, "EPYC") || strings.Contains(v, "Opteron") +} + +func isUnavailable(v string) bool { + v = strings.TrimSpace(strings.ToUpper(v)) + return v == "" || v == "NOT AVAILABLE" || v == "UNKNOWN" || v == "N/A" +} + +func cleanUnavailable(v string) string { + if isUnavailable(v) { + return "" + } + return strings.TrimSpace(v) +} + +func looksLikePartNumber(v string) bool { + return partNumberPattern.MatchString(strings.TrimSpace(v)) +} + +func isLikelySerial(v string) bool { + v = strings.TrimSpace(v) + if len(v) < 6 || len(v) > 24 || strings.Contains(v, "-") || isUnavailable(v) { + return false + } + for _, r := range v { + if (r < '0' || r > '9') && (r < 'A' || r > 'Z') && (r < 'a' || r > 'z') { + return false + } + } + return true +} + +func looksLikeLocation(v string) bool { + lower := strings.ToLower(strings.TrimSpace(v)) + return strings.HasPrefix(lower, "slot ") || strings.HasPrefix(lower, "ocp slot") || strings.HasPrefix(lower, "pci-e slot") || strings.HasPrefix(lower, "pci-e") || strings.HasPrefix(lower, "nvme drive") +} + +func looksLikeVersion(v string) bool { + v = strings.TrimSpace(v) + if len(v) < 3 || len(v) > 48 || isUnavailable(v) { + return false + } + if strings.HasPrefix(v, "v") && len(v) > 1 && v[1] >= '0' && v[1] <= '9' { + return true + } + digit := false + for _, r := range v { + if r >= '0' && r <= '9' { + digit = true + break + } + } + if !digit { + return false + } + return strings.Contains(v, ".") || strings.Contains(strings.ToLower(v), "build") +} + +func looksLikeConcreteModel(v string) bool { + if isUnavailable(v) || looksLikeVersion(v) || looksLikePartNumber(v) || isLikelySerial(v) { + return false + } + if looksLikeLocation(v) { + return false + } + return true +} + +func looksLikeNetworkTitle(code, title string, fields []string) bool { + lower := strings.ToLower(code + " " + title + " " + strings.Join(fields, " ")) + return strings.Contains(lower, "nic.") || strings.Contains(lower, "network controller") || strings.Contains(lower, "ethernet") || strings.Contains(lower, "broadcom") || strings.Contains(lower, "connectx") || strings.Contains(lower, "mellanox") || strings.Contains(lower, "ocp.slot.15") +} + +func isTopLevelFirmwareLabel(v string) bool { + switch strings.TrimSpace(v) { + case "System ROM", "Redundant System ROM", "Server Platform Services (SPS) Firmware", "Intelligent Platform Abstraction Data": + return true + default: + return false + } +} + +func inferVendor(model string) string { + lower := strings.ToLower(model) + switch { + case strings.Contains(lower, "broadcom"): + return "Broadcom" + case strings.Contains(lower, "mellanox"), strings.Contains(lower, "connectx"), strings.Contains(lower, "mcx"): + return "NVIDIA" + case strings.Contains(lower, "hpe"): + return "HPE" + default: + return "" + } +} + +func slotLabelFromCode(code string) string { + parts := strings.Split(code, ".") + if len(parts) < 3 { + return code + } + switch parts[0] { + case "NIC": + return "Slot " + parts[2] + case "OCP": + return "OCP Slot " + parts[2] + case "PCI": + return "PCI-E Slot " + parts[2] + default: + return code + } +} + +func inferSeverity(message string) models.Severity { + lower := strings.ToLower(message) + switch { + case strings.Contains(lower, " down"), strings.Contains(lower, "warning"), strings.Contains(lower, "fail"), strings.Contains(lower, "error"): + return models.SeverityWarning + default: + return models.SeverityInfo + } +} + +func inferEventType(message string) string { + lower := strings.ToLower(message) + switch { + case strings.Contains(lower, "login"): + return "Login" + case strings.Contains(lower, "logout"): + return "Logout" + case strings.Contains(lower, "network"): + return "Network" + case strings.Contains(lower, "license"): + return "License" + default: + return "Event" + } +} + +func looksLikeEventMessage(v string) bool { + if len(v) < 8 || strings.HasPrefix(v, "src/") || strings.HasPrefix(v, "PciRoot(") { + return false + } + lower := strings.ToLower(v) + return strings.Contains(lower, "login") || strings.Contains(lower, "logout") || strings.Contains(lower, "link") || strings.Contains(lower, "license") || strings.Contains(lower, "security state") +} + +func sanitizeModel(v string) string { + return strings.TrimSuffix(strings.TrimSpace(v), ":") +} + +func sanitizeID(v string) string { + v = strings.ToLower(strings.TrimSpace(v)) + v = strings.ReplaceAll(v, " ", "-") + v = strings.ReplaceAll(v, "/", "-") + v = strings.ReplaceAll(v, ".", "-") + return v +} + +func bytesToDecimalGB(size int64) int { + if size <= 0 { + return 0 + } + return int((size + 500_000_000) / 1_000_000_000) +} + +func redfishServiceLabel(doc map[string]any, path ...string) string { + return strings.TrimSpace(asString(nested(doc, path...))) +} + +func redfishStatus(v any) string { + status, _ := v.(map[string]any) + state := strings.TrimSpace(asString(status["State"])) + health := strings.TrimSpace(asString(status["Health"])) + if strings.EqualFold(state, "Absent") { + return "absent" + } + if strings.EqualFold(health, "Warning") || strings.EqualFold(health, "Critical") { + return strings.ToLower(health) + } + if state != "" { + return strings.ToLower(state) + } + if health != "" { + return strings.ToLower(health) + } + return "" +} + +func redfishID(path string) string { + parts := strings.Split(strings.Trim(path, "/"), "/") + if len(parts) == 0 { + return "unknown" + } + return sanitizeID(parts[len(parts)-1]) +} + +func nested(v any, path ...string) any { + cur := v + for _, key := range path { + m, ok := cur.(map[string]any) + if !ok { + return nil + } + cur = m[key] + } + return cur +} + +func asString(v any) string { + switch value := v.(type) { + case string: + return value + case fmt.Stringer: + return value.String() + default: + return "" + } +} + +func asInt(doc map[string]any, path ...string) int { + return int(asInt64(nested(doc, path...))) +} + +func asInt64(v any) int64 { + switch value := v.(type) { + case float64: + return int64(value) + case float32: + return int64(value) + case int: + return int64(value) + case int64: + return value + case json.Number: + n, _ := value.Int64() + return n + default: + return 0 + } +} + +func asOptionalInt(v any) *int { + switch value := v.(type) { + case float64: + out := int(value) + return &out + case int: + out := value + return &out + default: + return nil + } +} + +func valueOr(v, fallback string) string { + if strings.TrimSpace(v) != "" { + return strings.TrimSpace(v) + } + return strings.TrimSpace(fallback) +} + +func boolPtr(v bool) *bool { + out := v + return &out +} diff --git a/internal/parser/vendors/hpe_ilo_ahs/parser_test.go b/internal/parser/vendors/hpe_ilo_ahs/parser_test.go new file mode 100644 index 0000000..21f0f27 --- /dev/null +++ b/internal/parser/vendors/hpe_ilo_ahs/parser_test.go @@ -0,0 +1,210 @@ +package hpe_ilo_ahs + +import ( + "bytes" + "compress/gzip" + "encoding/binary" + "testing" + + "git.mchus.pro/mchus/logpile/internal/parser" +) + +func TestDetectAHS(t *testing.T) { + p := &Parser{} + score := p.Detect([]parser.ExtractedFile{{ + Path: "HPE_CZ2D1X0GS3_20260330.ahs", + Content: makeAHSArchive(t, []ahsTestEntry{{Name: "CUST_INFO.DAT", Payload: []byte("x")}}), + }}) + if score < 80 { + t.Fatalf("expected high confidence detect, got %d", score) + } +} + +func TestParseAHSInventory(t *testing.T) { + p := &Parser{} + content := makeAHSArchive(t, []ahsTestEntry{ + {Name: "CUST_INFO.DAT", Payload: make([]byte, 16)}, + {Name: "0000088-2026-03-30.zbb", Payload: gzipBytes(t, []byte(sampleInventoryBlob()))}, + }) + + result, err := p.Parse([]parser.ExtractedFile{{ + Path: "HPE_CZ2D1X0GS3_20260330.ahs", + Content: content, + }}) + if err != nil { + t.Fatalf("parse failed: %v", err) + } + if result.Hardware == nil { + t.Fatalf("expected hardware section") + } + + board := result.Hardware.BoardInfo + if board.Manufacturer != "HPE" { + t.Fatalf("unexpected board manufacturer: %q", board.Manufacturer) + } + if board.ProductName != "ProLiant DL380 Gen11" { + t.Fatalf("unexpected board product: %q", board.ProductName) + } + if board.SerialNumber != "CZ2D1X0GS3" { + t.Fatalf("unexpected board serial: %q", board.SerialNumber) + } + if board.PartNumber != "P52560-421" { + t.Fatalf("unexpected board part number: %q", board.PartNumber) + } + + if len(result.Hardware.CPUs) != 1 || result.Hardware.CPUs[0].Model != "Intel(R) Xeon(R) Gold 6444Y" { + t.Fatalf("unexpected CPUs: %+v", result.Hardware.CPUs) + } + if len(result.Hardware.Memory) != 1 { + t.Fatalf("expected one DIMM, got %d", len(result.Hardware.Memory)) + } + if result.Hardware.Memory[0].PartNumber != "HMCG88AEBRA115N" { + t.Fatalf("unexpected DIMM part number: %q", result.Hardware.Memory[0].PartNumber) + } + + if len(result.Hardware.NetworkAdapters) != 2 { + t.Fatalf("expected two network adapters, got %d", len(result.Hardware.NetworkAdapters)) + } + if len(result.Hardware.PowerSupply) != 1 { + t.Fatalf("expected one PSU, got %d", len(result.Hardware.PowerSupply)) + } + if result.Hardware.PowerSupply[0].SerialNumber != "5XUWB0C4DJG4BV" { + t.Fatalf("unexpected PSU serial: %q", result.Hardware.PowerSupply[0].SerialNumber) + } + + if len(result.Hardware.Storage) != 1 { + t.Fatalf("expected one physical drive, got %d", len(result.Hardware.Storage)) + } + drive := result.Hardware.Storage[0] + if drive.Model != "SAMSUNGMZ7L3480HCHQ-00A07" { + t.Fatalf("unexpected drive model: %q", drive.Model) + } + if drive.SerialNumber != "S664NC0Y502720" { + t.Fatalf("unexpected drive serial: %q", drive.SerialNumber) + } + if drive.SizeGB != 480 { + t.Fatalf("unexpected drive size: %d", drive.SizeGB) + } + + if len(result.Hardware.Firmware) == 0 { + t.Fatalf("expected firmware inventory") + } + foundILO := false + foundControllerFW := false + for _, item := range result.Hardware.Firmware { + if item.DeviceName == "iLO 6" && item.Version == "v1.63p20" { + foundILO = true + } + if item.DeviceName == "HPE MR408i-o Gen11" && item.Version == "52.26.3-5379" { + foundControllerFW = true + } + } + if !foundILO { + t.Fatalf("expected iLO firmware entry") + } + if !foundControllerFW { + t.Fatalf("expected controller firmware entry") + } + + if len(result.Hardware.Devices) < 6 { + t.Fatalf("expected canonical devices, got %d", len(result.Hardware.Devices)) + } + if len(result.Events) == 0 { + t.Fatalf("expected parsed events") + } +} + +type ahsTestEntry struct { + Name string + Payload []byte + Flag uint32 +} + +func makeAHSArchive(t *testing.T, entries []ahsTestEntry) []byte { + t.Helper() + + var buf bytes.Buffer + for _, entry := range entries { + header := make([]byte, ahsHeaderSize) + copy(header[:4], []byte("ABJR")) + binary.LittleEndian.PutUint16(header[4:6], 0x0300) + binary.LittleEndian.PutUint16(header[6:8], 0x0002) + binary.LittleEndian.PutUint32(header[8:12], uint32(len(entry.Payload))) + flag := entry.Flag + if flag == 0 { + flag = 0x80000002 + if len(entry.Payload) >= 2 && entry.Payload[0] == 0x1f && entry.Payload[1] == 0x8b { + flag = 0x80000001 + } + } + binary.LittleEndian.PutUint32(header[16:20], flag) + copy(header[20:52], []byte(entry.Name)) + buf.Write(header) + buf.Write(entry.Payload) + } + return buf.Bytes() +} + +func gzipBytes(t *testing.T, payload []byte) []byte { + t.Helper() + + var buf bytes.Buffer + zw := gzip.NewWriter(&buf) + if _, err := zw.Write(payload); err != nil { + t.Fatalf("gzip payload: %v", err) + } + if err := zw.Close(); err != nil { + t.Fatalf("close gzip writer: %v", err) + } + return buf.Bytes() +} + +func sampleInventoryBlob() string { + return stringsJoin( + "iLO 6 v1.63p20 built on Sep 13 2024", + "HPE", + "ProLiant DL380 Gen11", + "CZ2D1X0GS3", + "P52560-421", + "Proc 1", + "Intel(R) Corporation", + "Intel(R) Xeon(R) Gold 6444Y", + "PROC 1 DIMM 3", + "Hynix", + "HMCG88AEBRA115N", + "2B5F92C6", + "Power Supply 1", + "5XUWB0C4DJG4BV", + "P03178-B21", + "PciRoot(0x1)/Pci(0x5,0x0)/Pci(0x0,0x0)", + "NIC.Slot.1.1", + "Network Controller", + "Slot 1", + "MCX512A-ACAT", + "MT2230478382", + "PciRoot(0x3)/Pci(0x1,0x0)/Pci(0x0,0x0)", + "OCP.Slot.15.1", + "Broadcom NetXtreme Gigabit Ethernet - NIC", + "OCP Slot 15", + "P51183-001", + "1CH0150001", + "20.28.41", + "System ROM", + "v2.22 (06/19/2024)", + "03/30/2026 09:47:33", + "iLO network link down.", + `{"@odata.id":"/redfish/v1/Systems/1/Storage/DE00A000/Controllers/0","@odata.type":"#StorageController.v1_7_0.StorageController","Id":"0","Name":"HPE MR408i-o Gen11","FirmwareVersion":"52.26.3-5379","Manufacturer":"HPE","Model":"HPE MR408i-o Gen11","PartNumber":"P58543-001","SKU":"P58335-B21","SerialNumber":"PXSFQ0BBIJY3B3","Status":{"State":"Enabled","Health":"OK"},"Location":{"PartLocation":{"ServiceLabel":"Slot=14","LocationType":"Slot","LocationOrdinalValue":14}},"PCIeInterface":{"PCIeType":"Gen4","LanesInUse":8}}`, + `{"@odata.id":"/redfish/v1/Chassis/DE00A000/Drives/0","@odata.type":"#Drive.v1_17_0.Drive","Id":"0","Name":"480GB 6G SATA SSD","Status":{"State":"StandbyOffline","Health":"OK"},"PhysicalLocation":{"PartLocation":{"ServiceLabel":"Slot=14:Port=1:Box=3:Bay=1","LocationType":"Bay","LocationOrdinalValue":1}},"CapacityBytes":480103981056,"MediaType":"SSD","Model":"SAMSUNGMZ7L3480HCHQ-00A07","Protocol":"SATA","Revision":"JXTC604Q","SerialNumber":"S664NC0Y502720","PredictedMediaLifeLeftPercent":100}`, + `{"@odata.id":"/redfish/v1/Chassis/DE00A000/Drives/64515","@odata.type":"#Drive.v1_17_0.Drive","Id":"64515","Name":"Empty Bay","Status":{"State":"Absent","Health":"OK"}}`, + ) +} + +func stringsJoin(parts ...string) string { + return string(bytes.Join(func() [][]byte { + out := make([][]byte, 0, len(parts)) + for _, part := range parts { + out = append(out, []byte(part)) + } + return out + }(), []byte{0})) +} diff --git a/internal/parser/vendors/vendors.go b/internal/parser/vendors/vendors.go index 61ff125..9b1536c 100644 --- a/internal/parser/vendors/vendors.go +++ b/internal/parser/vendors/vendors.go @@ -7,6 +7,7 @@ import ( _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/dell" _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/easy_bee" _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/h3c" + _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/hpe_ilo_ahs" _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/inspur" _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/nvidia" _ "git.mchus.pro/mchus/logpile/internal/parser/vendors/nvidia_bug_report" diff --git a/web/templates/index.html b/web/templates/index.html index 8834945..c67b3c1 100644 --- a/web/templates/index.html +++ b/web/templates/index.html @@ -36,9 +36,9 @@

Перетащите архив, TXT/LOG или JSON snapshot сюда

- + -

Поддерживаемые форматы: tar.gz, tar, tgz, sds, zip, json, txt, log

+

Поддерживаемые форматы: ahs, tar.gz, tar, tgz, sds, zip, json, txt, log