314 lines
15 KiB
Markdown
314 lines
15 KiB
Markdown
# 06 — Parsers
|
|
|
|
## Framework
|
|
|
|
Parsers live in `internal/parser/` and vendor implementations live in `internal/parser/vendors/`.
|
|
|
|
Core behavior:
|
|
- registration uses `init()` side effects
|
|
- all registered parsers run `Detect()`
|
|
- the highest-confidence parser wins
|
|
- a confidence tie is broken by the optional `PrioritizedParser` interface
|
|
(`DetectPriority() int`, default 0); used so `inspur_legacy` beats the broader
|
|
`inspur` parser on shared `onekeylog/` archives (see ADL-065)
|
|
- generic fallback stays last and low-confidence
|
|
|
|
`VendorParser` contract:
|
|
|
|
```go
|
|
type VendorParser interface {
|
|
Name() string
|
|
Vendor() string
|
|
Version() string
|
|
Detect(files []ExtractedFile) int
|
|
Parse(files []ExtractedFile) (*models.AnalysisResult, error)
|
|
}
|
|
```
|
|
|
|
## Adding a parser
|
|
|
|
1. Create `internal/parser/vendors/<vendor>/`
|
|
2. Start from `internal/parser/vendors/template/parser.go.template`
|
|
3. Implement `Detect()` and `Parse()`
|
|
4. Add a blank import in `internal/parser/vendors/vendors.go`
|
|
5. Add at least one positive and one negative detection test
|
|
|
|
## Data quality rules
|
|
|
|
### System firmware only in `hardware.firmware`
|
|
|
|
`hardware.firmware` must contain system-level firmware only.
|
|
Device-bound firmware belongs on the device record and must not be duplicated at the top level.
|
|
|
|
### Strip embedded MAC addresses from model names
|
|
|
|
If a source embeds ` - XX:XX:XX:XX:XX:XX` in a model/name field, remove that suffix before storing it.
|
|
|
|
### Use `pci.ids` for empty or generic PCI model names
|
|
|
|
When `vendor_id` and `device_id` are known but the model name is missing or generic, resolve the name via `internal/parser/vendors/pciids`.
|
|
|
|
### Preserve source-backed CPU identity
|
|
|
|
Keep a source PPIN in `CPU.PPIN`. If the source has no separate processor serial number, use
|
|
`models.ResolveCPUSerialNumber` to expose the valid PPIN as `CPU.SerialNumber`. Never derive a CPU
|
|
serial from its socket, model, board serial, or another component; reject source placeholders.
|
|
|
|
## Active vendor coverage
|
|
|
|
| Vendor ID | Input family | Notes |
|
|
|-----------|--------------|-------|
|
|
| `dell` | TSR ZIP archives | Broad hardware, firmware, sensors, lifecycle events |
|
|
| `easy_bee` | `bee-support-*.tar.gz` | Imports the embedded hardware snapshot and static DIMM/storage/NVIDIA evidence used to build `topology.json` |
|
|
| `h3c_g5` | H3C SDS G5 bundles | INI/XML/CSV-driven hardware and event parsing |
|
|
| `h3c_g6` | H3C SDS G6 bundles | Similar flow with G6-specific files |
|
|
| `hpe_ilo_ahs` | HPE iLO Active Health System (`.ahs`) | Proprietary `ABJR` container with gzip-compressed `zbb` members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON |
|
|
| `inspur` | onekeylog archives (classic `component.log` and per-file `component/*.txt` D-Bus layouts) | FRU/SDR, optional Redis enrichment, normalized IDL/SEL/syslog events (see ADL-056) |
|
|
| `inspur_legacy` | Legacy AMI-BMC onekeylog (NF5466M5/NF5280M5 gen: `Inspur_AssetInfoInventory.log`, binary `FRU.bin`, `Inspur_*_IDL`, `sel.log`, flat `<sev>.log`) | Binary IPMI FRU decode + text inventory + IDL/SEL/blackbox/MegaRAID/syslog events; no live sensors. Wins the `onekeylog/` detect tie via `DetectPriority`. See ADL-065 |
|
|
| `lenovo_xcc` | Lenovo XCC mini-log ZIP archives | JSON inventory + platform event logs |
|
|
| `nvidia` | HGX Field Diagnostics | GPU- and fabric-heavy diagnostic input |
|
|
| `nvidia_bug_report` | `nvidia-bug-report-*.log.gz` | dmidecode, lspci, NVIDIA driver sections; Xid/SXid GPU error events (see ADL-055, `bible-local/docs/nvidia-bug-report-analysis.md`) |
|
|
| `redfish_walk` | Any archive carrying a captured Redfish directory-tree walk (tar.gz/zip of `<path>/index.json` docs) | Low-confidence (35) vendor-agnostic fallback; see `dell` for the first known source and `redfishtree` helper package |
|
|
| `unraid` | Unraid diagnostics/log bundles | Server and storage-focused parsing |
|
|
| `xfusion` | xFusion iBMC `tar.gz` dump / file export | AppDump + RTOSDump + LogDump merge for hardware and firmware |
|
|
| `xigmanas` | XigmaNAS plain logs | FreeBSD/NAS-oriented inventory |
|
|
| `generic` | fallback | Low-confidence text fallback when nothing else matches |
|
|
|
|
## Practical guidance
|
|
|
|
- Be conservative with high detect scores
|
|
- Prefer filling missing fields over overwriting stronger source data
|
|
- Keep parser version constants current when behavior changes
|
|
- Any new vendor-specific filtering or dedup logic must ship with tests for that vendor format
|
|
|
|
**Archive format:** Unraid diagnostics archive contents (text-heavy diagnostics directories).
|
|
|
|
**Detection:** Combines filename/path markers (`diagnostics-*`, `unraid-*.txt`, `vars.txt`)
|
|
with content markers (e.g. `Unraid kernel build`, parity data markers).
|
|
|
|
**Extracted data (current):**
|
|
- Board / BIOS metadata (from motherboard/system files)
|
|
- CPU summary (from `lscpu.txt`)
|
|
- Memory modules (from diagnostics memory file)
|
|
- Storage devices (from `vars.txt` + SMART files)
|
|
- Syslog events
|
|
|
|
---
|
|
|
|
### H3C SDS G5 (`h3c_g5`)
|
|
|
|
**Status:** Ready (v2.2). Tested on H3C UniServer R4900 G5 SDS archives.
|
|
|
|
**Archive format:** `.sds` (tar archive)
|
|
|
|
**Detection:** `hardware_info.ini`, `hardware.info`, `firmware_version.ini`, `user/test*.csv`, plus H3C markers.
|
|
|
|
**Extracted data (current):**
|
|
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
|
|
- Firmware list (`firmware_version.ini`)
|
|
- CPU inventory (`hardware_info.ini`)
|
|
- Memory DIMM inventory (`hardware_info.ini`)
|
|
- Storage inventory (`hardware.info`, `storage_disk.ini`, `NVMe_info.txt`, RAID text enrichments)
|
|
- Logical RAID volumes (`raid.json`, `Storage_RAID-*.txt`)
|
|
- Sensor snapshot (`sensor_info.ini`)
|
|
- SEL events (`user/test.csv`, `user/test1.csv`, fallback `Sel.json` / `sel_list.txt`)
|
|
|
|
---
|
|
|
|
### H3C SDS G6 (`h3c_g6`)
|
|
|
|
**Status:** Ready (v2.2). Tested on H3C UniServer R4700 G6 SDS archives.
|
|
|
|
**Archive format:** `.sds` (tar archive)
|
|
|
|
**Detection:** `CPUDetailInfo.xml`, `MemoryDetailInfo.xml`, `firmware_version.json`, `Sel.json`, plus H3C markers.
|
|
|
|
**Extracted data (current):**
|
|
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
|
|
- Firmware list (`firmware_version.json`)
|
|
- CPU inventory (`CPUDetailInfo.xml`)
|
|
- Memory DIMM inventory (`MemoryDetailInfo.xml`)
|
|
- Storage inventory + capacity/model/interface (`storage_disk.ini`, `Storage_RAID-*.txt`, `NVMe_info.txt`)
|
|
- Logical RAID volumes (`raid.json`, fallback from `Storage_RAID-*.txt` when available)
|
|
- Sensor snapshot (`sensor_info.ini`)
|
|
- SEL events (`user/Sel.json`, fallback `user/sel_list.txt`)
|
|
|
|
---
|
|
|
|
### HPE iLO AHS (`hpe_ilo_ahs`)
|
|
|
|
**Status:** Ready (v1.0.0). Tested on HPE ProLiant Gen11 `.ahs` export from iLO 6.
|
|
|
|
**Archive format:** `.ahs` single-file Active Health System export.
|
|
|
|
**Detection:** Single-file input with `ABJR` container header and HPE AHS member names
|
|
such as `CUST_INFO.DAT`, `*.zbb`, `ilo_boot_support.zbb`.
|
|
|
|
**Extracted data (current):**
|
|
- System board identity (manufacturer, model, serial, part number)
|
|
- iLO / System ROM / SPS top-level firmware
|
|
- CPU inventory (model-level)
|
|
- Memory DIMM inventory for populated slots
|
|
- PSU inventory
|
|
- PCIe / OCP NIC inventory from SMBIOS-style slot records
|
|
- Storage controller and physical drives from embedded Redfish JSON inside `zbb` members
|
|
- Basic iLO event log entries with timestamps when present
|
|
|
|
**Implementation note:** The format is proprietary. Parser support is intentionally hybrid:
|
|
container parsing (`ABJR` + gzip) plus structured extraction from embedded Redfish objects and
|
|
printable SMBIOS/FRU payloads. This is sufficient for inventory-grade parsing without decoding the
|
|
entire internal `zbb` schema.
|
|
|
|
---
|
|
|
|
### Inspur / Kaytus (`inspur`)
|
|
|
|
**Status:** Ready (v2.2). Tested on Inspur NF5468M7 / Kaytus KR4268X2 onekeylog archives.
|
|
|
|
**Archive format:** onekeylog BMC diagnostic dump. Two known on-disk layouts:
|
|
1. Classic layout: top-level `onekeylog/` directory, single combined `component/component.log`
|
|
with `RESTful <Section> info:` JSON blocks (PSU, fan, HDD, PCIe, network, ...).
|
|
2. Per-file layout (seen on newer/OEM firmware): archive root is `dump_<serial>_<timestamp>/`
|
|
(not `onekeylog/`), and `component.log` is absent. Each component is instead dumped separately
|
|
under `component/*.txt` as a raw D-Bus `GetAll` transcript: `GETALL <object> OBJect` blocks with
|
|
tab-separated `"field"\t"type":"x"\t"data":value` triples. This is not valid JSON and needs its
|
|
own line-oriented extraction (`internal/parser/vendors/inspur/component_dbus.go`).
|
|
|
|
**Detection:** `onekeylog/` path segment, `devicefrusdr.log`, `component/component.log`,
|
|
`onekeylog_dreport.log` filename (per-file layout marker), `component/PowerSupplyInfo.txt`
|
|
(per-file layout marker), plus `asset.json` content markers (`VersionInfo` + `CpuInfo` +
|
|
`MemInfo`).
|
|
|
|
**Extracted data (current):**
|
|
- CPU / Memory / PCIe / Storage inventory from `asset.json` (both layouts)
|
|
- FRU + SDR from `devicefrusdr.log` (classic layout)
|
|
- PSU inventory: from `component.log`'s `RESTful PSU info:` block (classic layout) or
|
|
`component/PowerSupplyInfo.txt` GETALL transcript (per-file layout)
|
|
- Fan RPM/PWM sensors: from `component.log`'s `RESTful fan info:` block (classic layout) or
|
|
`component/FanInfo.txt` GETALL transcript (per-file layout)
|
|
- Redis snapshot enrichment for serials/firmware/telemetry when `redis-dump.rdb` is present
|
|
|
|
**Known gaps (per-file layout only):**
|
|
- `component/NetworkAdapter.txt` and `component/HDDBpListInfo.txt` use a different
|
|
`busctl --verbose` object-tree dump (not the GETALL format) and are not currently parsed. NIC
|
|
identity/MAC data is still available via PCIe device inventory from `asset.json`, so this is not
|
|
a data-loss gap for NICs.
|
|
- `component/RAID.txt` mixes formats per RAID controller family (`getLsiStorageCollection`,
|
|
`getPMCStorageCollection`); not currently parsed.
|
|
|
|
**Implementation note:** the same GETALL object name can recur across multiple command sections in
|
|
one file with different field subsets (e.g. `Pwm_N` appears once under the FanPWM sensor query
|
|
with a real `Value` reading, and again later under FanControl with only a `Target` setpoint).
|
|
`parseDBusGetAllObjects` unions fields across recurring object names, first-seen wins per field, so
|
|
a later content-free duplicate cannot silently blank out an earlier real reading.
|
|
|
|
---
|
|
|
|
### xFusion iBMC Dump / File Export (`xfusion`)
|
|
|
|
**Status:** Ready (v1.1.0). Tested on xFusion G5500 V7 `tar.gz` exports.
|
|
|
|
**Archive format:** `tar.gz` dump exported from the iBMC UI, including `AppDump/`, `RTOSDump/`,
|
|
and `LogDump/` trees.
|
|
|
|
**Detection:** `AppDump/FruData/fruinfo.txt`, `AppDump/card_manage/card_info`,
|
|
`RTOSDump/versioninfo/app_revision.txt`, and `LogDump/netcard/netcard_info.txt`.
|
|
|
|
**Extracted data (current):**
|
|
- Board / FRU inventory from `fruinfo.txt`
|
|
- CPU inventory from `CpuMem/cpu_info`
|
|
- Memory DIMM inventory from `CpuMem/mem_info`
|
|
- GPU inventory from `card_info`
|
|
- OCP NIC inventory by merging `card_info` with `LogDump/netcard/netcard_info.txt`
|
|
- PSU inventory from `BMC/psu_info.txt`
|
|
- Physical storage from `StorageMgnt/PhysicalDrivesInfo/*/disk_info`
|
|
- System firmware entries from `RTOSDump/versioninfo/app_revision.txt`
|
|
- Maintenance events from `LogDump/maintenance_log`
|
|
|
|
---
|
|
|
|
### Dell TSR (`dell`)
|
|
|
|
**Status:** Ready (v3.1). Two inventory source generations, both handled by the same parser.
|
|
|
|
**Archive format:** TSR nested zip (`signature` + `TSR*.pl.zip`).
|
|
|
|
**Extracted data:**
|
|
- Older/DCIM-XML generation: `sysinfo_dcim_view.xml` (board/CPU/NIC/etc.), `sysinfo_dcim_softwareidentity.xml`
|
|
(firmware), `sysinfo_cim_sensor.xml` (sensors), `curr_lclog.xml` (Lifecycle Controller events),
|
|
`tsr/metadata.json` (board identity fallback).
|
|
- iDRAC10-generation: same `metadata.json`/`curr_lclog.xml`, but no DCIM-XML files — instead
|
|
`tsr/hardware/sysinfo/inventory/redfishidracwalk.tar.gz` holds a captured Redfish directory-tree
|
|
walk. Parsed via the shared `redfishtree` helper package and replayed through
|
|
`collector.ReplayRedfishFromRawPayloads` (`internal/parser/vendors/dell/redfish_walk.go`), then
|
|
append-merged into the same `Hardware`/`Sensors`/`FRU`/`Events` slices the DCIM-XML path fills, so
|
|
the existing dedupe passes resolve any overlap in favor of DCIM-derived data. See ADL-048.
|
|
- Licenses (`hardware.licenses[]`, contract v2.12): the Redfish-walk replay reads the standard DMTF
|
|
`/redfish/v1/LicenseService/Licenses` collection (`internal/collector/redfish_replay_licenses.go`),
|
|
present on iDRAC10-generation firmware — feature-on-demand/advanced licenses such as
|
|
"iDRAC10 17G Enterprise License", "Secure Enterprise Key Manager". System-level licenses have no
|
|
`component_ref`; a license with `AuthorizationScope: "Device"` gets `component_ref` from
|
|
`Links.AuthorizedDevices`. Not sourced from DCIM-XML — only available on the Redfish-walk path.
|
|
See ADL-050.
|
|
|
|
---
|
|
|
|
### Generic Redfish walk fallback (`redfish_walk`)
|
|
|
|
**Status:** Ready (v1.0.0).
|
|
|
|
**Confidence:** 35 (above the `generic` fallback's 15, below every dedicated vendor parser — wins
|
|
`Detect()` only when no dedicated parser also recognizes the archive).
|
|
|
|
**Purpose:** Vendor-agnostic catch-all for any archive that carries a captured Redfish
|
|
directory-tree walk (one JSON document per resource, e.g.
|
|
`redfish/v1/Systems/System.Embedded.1/index.json`) but isn't otherwise claimed by a dedicated vendor
|
|
parser. First known source is Dell iDRAC10 TSR bundles (handled directly by `dell`, see above); this
|
|
exists so any other vendor shipping the same kind of raw Redfish walk is picked up automatically.
|
|
|
|
**Detection (`internal/parser/vendors/redfishtree`):** two-step, vendor-independent —
|
|
1. Path hint: an archive member path containing `redfish` and ending in `.tar.gz`/`.tgz`/`.zip`.
|
|
2. Structural confirmation: the unpacked tree must contain a document whose own `@odata.id` is
|
|
exactly `/redfish/v1`, plus a `/redfish/v1/Systems` or `/redfish/v1/Chassis` collection.
|
|
|
|
Documents are keyed by their own `@odata.id`, not the on-disk directory name (some resource names
|
|
are URL-encoded on disk, e.g. `Assembly%23`, but not in the JSON payload).
|
|
|
|
**Extracted data:** whatever `collector.ReplayRedfishFromRawPayloads` produces from the replayed
|
|
tree — board identity, firmware, CPUs, memory, storage, PCIe, GPUs, NICs, PSUs, sensors, events.
|
|
|
|
---
|
|
|
|
### Generic text fallback (`generic`)
|
|
|
|
**Status:** Ready (v1.0.0).
|
|
|
|
**Confidence:** 15 (lowest — only matches if no other parser scores higher)
|
|
|
|
**Purpose:** Fallback for any text file or single `.gz` file not matching a specific vendor.
|
|
|
|
**Behavior:**
|
|
- If filename matches `nvidia-bug-report-*.log.gz`: extracts driver version and GPU list.
|
|
- Otherwise: confirms file is text (not binary) and records a basic "Text File" event.
|
|
|
|
---
|
|
|
|
## Supported vendor matrix
|
|
|
|
| Vendor | ID | Status | Tested on |
|
|
|--------|----|--------|-----------|
|
|
| Dell TSR | `dell` | Ready | TSR nested zip archives (DCIM-XML and iDRAC10 Redfish-walk generations) |
|
|
| Generic Redfish walk | `redfish_walk` | Ready (fallback) | Any archive with a captured Redfish tree dump |
|
|
| Reanimator Easy Bee | `easy_bee` | Ready | `bee-support-*.tar.gz` support bundles |
|
|
| HPE iLO AHS | `hpe_ilo_ahs` | Ready | iLO 6 `.ahs` exports |
|
|
| Inspur / Kaytus | `inspur` | Ready | KR4268X2 onekeylog |
|
|
| Lenovo XCC mini-log | `lenovo_xcc` | Ready | ThinkSystem SR650 V3 XCC mini-log ZIP |
|
|
| NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers |
|
|
| NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems |
|
|
| Unraid | `unraid` | Ready | Unraid diagnostics archives |
|
|
| xFusion iBMC dump | `xfusion` | Ready | G5500 V7 file-export `tar.gz` bundles |
|
|
| XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs |
|
|
| H3C SDS G5 | `h3c_g5` | Ready | H3C UniServer R4900 G5 SDS archives |
|
|
| H3C SDS G6 | `h3c_g6` | Ready | H3C UniServer R4700 G6 SDS archives |
|
|
| Generic fallback | `generic` | Ready | Any text file |
|