Mikhail ChusavitinandClaude Sonnet 5
5677c49998
Some onekeylog BMC firmware variants split the combined component.log into per-file D-Bus GetAll transcripts under component/ (e.g. PowerSupplyInfo.txt, FanInfo.txt), which the inspur parser did not read, leaving PSU and fan data empty. Add a GETALL block parser and wire it as a fallback for PSU and fan telemetry when component.log is absent; document the layout and known gaps in bible-local. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
250 lines
11 KiB
Markdown
250 lines
11 KiB
Markdown
# 06 — Parsers
|
|
|
|
## Framework
|
|
|
|
Parsers live in `internal/parser/` and vendor implementations live in `internal/parser/vendors/`.
|
|
|
|
Core behavior:
|
|
- registration uses `init()` side effects
|
|
- all registered parsers run `Detect()`
|
|
- the highest-confidence parser wins
|
|
- generic fallback stays last and low-confidence
|
|
|
|
`VendorParser` contract:
|
|
|
|
```go
|
|
type VendorParser interface {
|
|
Name() string
|
|
Vendor() string
|
|
Version() string
|
|
Detect(files []ExtractedFile) int
|
|
Parse(files []ExtractedFile) (*models.AnalysisResult, error)
|
|
}
|
|
```
|
|
|
|
## Adding a parser
|
|
|
|
1. Create `internal/parser/vendors/<vendor>/`
|
|
2. Start from `internal/parser/vendors/template/parser.go.template`
|
|
3. Implement `Detect()` and `Parse()`
|
|
4. Add a blank import in `internal/parser/vendors/vendors.go`
|
|
5. Add at least one positive and one negative detection test
|
|
|
|
## Data quality rules
|
|
|
|
### System firmware only in `hardware.firmware`
|
|
|
|
`hardware.firmware` must contain system-level firmware only.
|
|
Device-bound firmware belongs on the device record and must not be duplicated at the top level.
|
|
|
|
### Strip embedded MAC addresses from model names
|
|
|
|
If a source embeds ` - XX:XX:XX:XX:XX:XX` in a model/name field, remove that suffix before storing it.
|
|
|
|
### Use `pci.ids` for empty or generic PCI model names
|
|
|
|
When `vendor_id` and `device_id` are known but the model name is missing or generic, resolve the name via `internal/parser/vendors/pciids`.
|
|
|
|
## Active vendor coverage
|
|
|
|
| Vendor ID | Input family | Notes |
|
|
|-----------|--------------|-------|
|
|
| `dell` | TSR ZIP archives | Broad hardware, firmware, sensors, lifecycle events |
|
|
| `easy_bee` | `bee-support-*.tar.gz` | Imports embedded `export/bee-audit.json` snapshot from reanimator-easy-bee bundles |
|
|
| `h3c_g5` | H3C SDS G5 bundles | INI/XML/CSV-driven hardware and event parsing |
|
|
| `h3c_g6` | H3C SDS G6 bundles | Similar flow with G6-specific files |
|
|
| `hpe_ilo_ahs` | HPE iLO Active Health System (`.ahs`) | Proprietary `ABJR` container with gzip-compressed `zbb` members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON |
|
|
| `inspur` | onekeylog archives (classic `component.log` and per-file `component/*.txt` D-Bus layouts) | FRU/SDR plus optional Redis enrichment |
|
|
| `lenovo_xcc` | Lenovo XCC mini-log ZIP archives | JSON inventory + platform event logs |
|
|
| `nvidia` | HGX Field Diagnostics | GPU- and fabric-heavy diagnostic input |
|
|
| `nvidia_bug_report` | `nvidia-bug-report-*.log.gz` | dmidecode, lspci, NVIDIA driver sections |
|
|
| `unraid` | Unraid diagnostics/log bundles | Server and storage-focused parsing |
|
|
| `xfusion` | xFusion iBMC `tar.gz` dump / file export | AppDump + RTOSDump + LogDump merge for hardware and firmware |
|
|
| `xigmanas` | XigmaNAS plain logs | FreeBSD/NAS-oriented inventory |
|
|
| `generic` | fallback | Low-confidence text fallback when nothing else matches |
|
|
|
|
## Practical guidance
|
|
|
|
- Be conservative with high detect scores
|
|
- Prefer filling missing fields over overwriting stronger source data
|
|
- Keep parser version constants current when behavior changes
|
|
- Any new vendor-specific filtering or dedup logic must ship with tests for that vendor format
|
|
|
|
**Archive format:** Unraid diagnostics archive contents (text-heavy diagnostics directories).
|
|
|
|
**Detection:** Combines filename/path markers (`diagnostics-*`, `unraid-*.txt`, `vars.txt`)
|
|
with content markers (e.g. `Unraid kernel build`, parity data markers).
|
|
|
|
**Extracted data (current):**
|
|
- Board / BIOS metadata (from motherboard/system files)
|
|
- CPU summary (from `lscpu.txt`)
|
|
- Memory modules (from diagnostics memory file)
|
|
- Storage devices (from `vars.txt` + SMART files)
|
|
- Syslog events
|
|
|
|
---
|
|
|
|
### H3C SDS G5 (`h3c_g5`)
|
|
|
|
**Status:** Ready (v1.0.0). Tested on H3C UniServer R4900 G5 SDS archives.
|
|
|
|
**Archive format:** `.sds` (tar archive)
|
|
|
|
**Detection:** `hardware_info.ini`, `hardware.info`, `firmware_version.ini`, `user/test*.csv`, plus H3C markers.
|
|
|
|
**Extracted data (current):**
|
|
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
|
|
- Firmware list (`firmware_version.ini`)
|
|
- CPU inventory (`hardware_info.ini`)
|
|
- Memory DIMM inventory (`hardware_info.ini`)
|
|
- Storage inventory (`hardware.info`, `storage_disk.ini`, `NVMe_info.txt`, RAID text enrichments)
|
|
- Logical RAID volumes (`raid.json`, `Storage_RAID-*.txt`)
|
|
- Sensor snapshot (`sensor_info.ini`)
|
|
- SEL events (`user/test.csv`, `user/test1.csv`, fallback `Sel.json` / `sel_list.txt`)
|
|
|
|
---
|
|
|
|
### H3C SDS G6 (`h3c_g6`)
|
|
|
|
**Status:** Ready (v1.0.0). Tested on H3C UniServer R4700 G6 SDS archives.
|
|
|
|
**Archive format:** `.sds` (tar archive)
|
|
|
|
**Detection:** `CPUDetailInfo.xml`, `MemoryDetailInfo.xml`, `firmware_version.json`, `Sel.json`, plus H3C markers.
|
|
|
|
**Extracted data (current):**
|
|
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
|
|
- Firmware list (`firmware_version.json`)
|
|
- CPU inventory (`CPUDetailInfo.xml`)
|
|
- Memory DIMM inventory (`MemoryDetailInfo.xml`)
|
|
- Storage inventory + capacity/model/interface (`storage_disk.ini`, `Storage_RAID-*.txt`, `NVMe_info.txt`)
|
|
- Logical RAID volumes (`raid.json`, fallback from `Storage_RAID-*.txt` when available)
|
|
- Sensor snapshot (`sensor_info.ini`)
|
|
- SEL events (`user/Sel.json`, fallback `user/sel_list.txt`)
|
|
|
|
---
|
|
|
|
### HPE iLO AHS (`hpe_ilo_ahs`)
|
|
|
|
**Status:** Ready (v1.0.0). Tested on HPE ProLiant Gen11 `.ahs` export from iLO 6.
|
|
|
|
**Archive format:** `.ahs` single-file Active Health System export.
|
|
|
|
**Detection:** Single-file input with `ABJR` container header and HPE AHS member names
|
|
such as `CUST_INFO.DAT`, `*.zbb`, `ilo_boot_support.zbb`.
|
|
|
|
**Extracted data (current):**
|
|
- System board identity (manufacturer, model, serial, part number)
|
|
- iLO / System ROM / SPS top-level firmware
|
|
- CPU inventory (model-level)
|
|
- Memory DIMM inventory for populated slots
|
|
- PSU inventory
|
|
- PCIe / OCP NIC inventory from SMBIOS-style slot records
|
|
- Storage controller and physical drives from embedded Redfish JSON inside `zbb` members
|
|
- Basic iLO event log entries with timestamps when present
|
|
|
|
**Implementation note:** The format is proprietary. Parser support is intentionally hybrid:
|
|
container parsing (`ABJR` + gzip) plus structured extraction from embedded Redfish objects and
|
|
printable SMBIOS/FRU payloads. This is sufficient for inventory-grade parsing without decoding the
|
|
entire internal `zbb` schema.
|
|
|
|
---
|
|
|
|
### Inspur / Kaytus (`inspur`)
|
|
|
|
**Status:** Ready (v2.2). Tested on Inspur NF5468M7 / Kaytus KR4268X2 onekeylog archives.
|
|
|
|
**Archive format:** onekeylog BMC diagnostic dump. Two known on-disk layouts:
|
|
1. Classic layout: top-level `onekeylog/` directory, single combined `component/component.log`
|
|
with `RESTful <Section> info:` JSON blocks (PSU, fan, HDD, PCIe, network, ...).
|
|
2. Per-file layout (seen on newer/OEM firmware): archive root is `dump_<serial>_<timestamp>/`
|
|
(not `onekeylog/`), and `component.log` is absent. Each component is instead dumped separately
|
|
under `component/*.txt` as a raw D-Bus `GetAll` transcript: `GETALL <object> OBJect` blocks with
|
|
tab-separated `"field"\t"type":"x"\t"data":value` triples. This is not valid JSON and needs its
|
|
own line-oriented extraction (`internal/parser/vendors/inspur/component_dbus.go`).
|
|
|
|
**Detection:** `onekeylog/` path segment, `devicefrusdr.log`, `component/component.log`,
|
|
`onekeylog_dreport.log` filename (per-file layout marker), `component/PowerSupplyInfo.txt`
|
|
(per-file layout marker), plus `asset.json` content markers (`VersionInfo` + `CpuInfo` +
|
|
`MemInfo`).
|
|
|
|
**Extracted data (current):**
|
|
- CPU / Memory / PCIe / Storage inventory from `asset.json` (both layouts)
|
|
- FRU + SDR from `devicefrusdr.log` (classic layout)
|
|
- PSU inventory: from `component.log`'s `RESTful PSU info:` block (classic layout) or
|
|
`component/PowerSupplyInfo.txt` GETALL transcript (per-file layout)
|
|
- Fan RPM/PWM sensors: from `component.log`'s `RESTful fan info:` block (classic layout) or
|
|
`component/FanInfo.txt` GETALL transcript (per-file layout)
|
|
- Redis snapshot enrichment for serials/firmware/telemetry when `redis-dump.rdb` is present
|
|
|
|
**Known gaps (per-file layout only):**
|
|
- `component/NetworkAdapter.txt` and `component/HDDBpListInfo.txt` use a different
|
|
`busctl --verbose` object-tree dump (not the GETALL format) and are not currently parsed. NIC
|
|
identity/MAC data is still available via PCIe device inventory from `asset.json`, so this is not
|
|
a data-loss gap for NICs.
|
|
- `component/RAID.txt` mixes formats per RAID controller family (`getLsiStorageCollection`,
|
|
`getPMCStorageCollection`); not currently parsed.
|
|
|
|
**Implementation note:** the same GETALL object name can recur across multiple command sections in
|
|
one file with different field subsets (e.g. `Pwm_N` appears once under the FanPWM sensor query
|
|
with a real `Value` reading, and again later under FanControl with only a `Target` setpoint).
|
|
`parseDBusGetAllObjects` unions fields across recurring object names, first-seen wins per field, so
|
|
a later content-free duplicate cannot silently blank out an earlier real reading.
|
|
|
|
---
|
|
|
|
### xFusion iBMC Dump / File Export (`xfusion`)
|
|
|
|
**Status:** Ready (v1.1.0). Tested on xFusion G5500 V7 `tar.gz` exports.
|
|
|
|
**Archive format:** `tar.gz` dump exported from the iBMC UI, including `AppDump/`, `RTOSDump/`,
|
|
and `LogDump/` trees.
|
|
|
|
**Detection:** `AppDump/FruData/fruinfo.txt`, `AppDump/card_manage/card_info`,
|
|
`RTOSDump/versioninfo/app_revision.txt`, and `LogDump/netcard/netcard_info.txt`.
|
|
|
|
**Extracted data (current):**
|
|
- Board / FRU inventory from `fruinfo.txt`
|
|
- CPU inventory from `CpuMem/cpu_info`
|
|
- Memory DIMM inventory from `CpuMem/mem_info`
|
|
- GPU inventory from `card_info`
|
|
- OCP NIC inventory by merging `card_info` with `LogDump/netcard/netcard_info.txt`
|
|
- PSU inventory from `BMC/psu_info.txt`
|
|
- Physical storage from `StorageMgnt/PhysicalDrivesInfo/*/disk_info`
|
|
- System firmware entries from `RTOSDump/versioninfo/app_revision.txt`
|
|
- Maintenance events from `LogDump/maintenance_log`
|
|
|
|
---
|
|
|
|
### Generic text fallback (`generic`)
|
|
|
|
**Status:** Ready (v1.0.0).
|
|
|
|
**Confidence:** 15 (lowest — only matches if no other parser scores higher)
|
|
|
|
**Purpose:** Fallback for any text file or single `.gz` file not matching a specific vendor.
|
|
|
|
**Behavior:**
|
|
- If filename matches `nvidia-bug-report-*.log.gz`: extracts driver version and GPU list.
|
|
- Otherwise: confirms file is text (not binary) and records a basic "Text File" event.
|
|
|
|
---
|
|
|
|
## Supported vendor matrix
|
|
|
|
| Vendor | ID | Status | Tested on |
|
|
|--------|----|--------|-----------|
|
|
| Dell TSR | `dell` | Ready | TSR nested zip archives |
|
|
| Reanimator Easy Bee | `easy_bee` | Ready | `bee-support-*.tar.gz` support bundles |
|
|
| HPE iLO AHS | `hpe_ilo_ahs` | Ready | iLO 6 `.ahs` exports |
|
|
| Inspur / Kaytus | `inspur` | Ready | KR4268X2 onekeylog |
|
|
| Lenovo XCC mini-log | `lenovo_xcc` | Ready | ThinkSystem SR650 V3 XCC mini-log ZIP |
|
|
| NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers |
|
|
| NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems |
|
|
| Unraid | `unraid` | Ready | Unraid diagnostics archives |
|
|
| xFusion iBMC dump | `xfusion` | Ready | G5500 V7 file-export `tar.gz` bundles |
|
|
| XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs |
|
|
| H3C SDS G5 | `h3c_g5` | Ready | H3C UniServer R4900 G5 SDS archives |
|
|
| H3C SDS G6 | `h3c_g6` | Ready | H3C UniServer R4700 G6 SDS archives |
|
|
| Generic fallback | `generic` | Ready | Any text file |
|