logpile/bible-local/06-parsers.md

# 06 — Parsers

## Framework

Parsers live in `internal/parser/` and vendor implementations live in `internal/parser/vendors/`.

Core behavior:
- registration uses `init()` side effects
- all registered parsers run `Detect()`
- the highest-confidence parser wins
- generic fallback stays last and low-confidence

`VendorParser` contract:

```go
type VendorParser interface {
    Name() string
    Vendor() string
    Version() string
    Detect(files []ExtractedFile) int
    Parse(files []ExtractedFile) (*models.AnalysisResult, error)
}
```

## Adding a parser

1. Create `internal/parser/vendors/<vendor>/`
2. Start from `internal/parser/vendors/template/parser.go.template`
3. Implement `Detect()` and `Parse()`
4. Add a blank import in `internal/parser/vendors/vendors.go`
5. Add at least one positive and one negative detection test

## Data quality rules

### System firmware only in `hardware.firmware`

`hardware.firmware` must contain system-level firmware only.
Device-bound firmware belongs on the device record and must not be duplicated at the top level.

### Strip embedded MAC addresses from model names

If a source embeds ` - XX:XX:XX:XX:XX:XX` in a model/name field, remove that suffix before storing it.

### Use `pci.ids` for empty or generic PCI model names

When `vendor_id` and `device_id` are known but the model name is missing or generic, resolve the name via `internal/parser/vendors/pciids`.

### Preserve source-backed CPU identity

Keep a source PPIN in `CPU.PPIN`. If the source has no separate processor serial number, use
`models.ResolveCPUSerialNumber` to expose the valid PPIN as `CPU.SerialNumber`. Never derive a CPU
serial from its socket, model, board serial, or another component; reject source placeholders.

## Active vendor coverage

| Vendor ID | Input family | Notes |
|-----------|--------------|-------|
| `dell` | TSR ZIP archives | Broad hardware, firmware, sensors, lifecycle events |
| `easy_bee` | `bee-support-*.tar.gz` | Imports embedded `export/bee-audit.json` snapshot from reanimator-easy-bee bundles |
| `h3c_g5` | H3C SDS G5 bundles | INI/XML/CSV-driven hardware and event parsing |
| `h3c_g6` | H3C SDS G6 bundles | Similar flow with G6-specific files |
| `hpe_ilo_ahs` | HPE iLO Active Health System (`.ahs`) | Proprietary `ABJR` container with gzip-compressed `zbb` members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON |
| `inspur` | onekeylog archives (classic `component.log` and per-file `component/*.txt` D-Bus layouts) | FRU/SDR, optional Redis enrichment, normalized IDL/SEL/syslog events (see ADL-056) |
| `lenovo_xcc` | Lenovo XCC mini-log ZIP archives | JSON inventory + platform event logs |
| `nvidia` | HGX Field Diagnostics | GPU- and fabric-heavy diagnostic input |
| `nvidia_bug_report` | `nvidia-bug-report-*.log.gz` | dmidecode, lspci, NVIDIA driver sections; Xid/SXid GPU error events (see ADL-055, `bible-local/docs/nvidia-bug-report-analysis.md`) |
| `redfish_walk` | Any archive carrying a captured Redfish directory-tree walk (tar.gz/zip of `<path>/index.json` docs) | Low-confidence (35) vendor-agnostic fallback; see `dell` for the first known source and `redfishtree` helper package |
| `unraid` | Unraid diagnostics/log bundles | Server and storage-focused parsing |
| `xfusion` | xFusion iBMC `tar.gz` dump / file export | AppDump + RTOSDump + LogDump merge for hardware and firmware |
| `xigmanas` | XigmaNAS plain logs | FreeBSD/NAS-oriented inventory |
| `generic` | fallback | Low-confidence text fallback when nothing else matches |

## Practical guidance

- Be conservative with high detect scores
- Prefer filling missing fields over overwriting stronger source data
- Keep parser version constants current when behavior changes
- Any new vendor-specific filtering or dedup logic must ship with tests for that vendor format

**Archive format:** Unraid diagnostics archive contents (text-heavy diagnostics directories).

**Detection:** Combines filename/path markers (`diagnostics-*`, `unraid-*.txt`, `vars.txt`)
with content markers (e.g. `Unraid kernel build`, parity data markers).

**Extracted data (current):**
- Board / BIOS metadata (from motherboard/system files)
- CPU summary (from `lscpu.txt`)
- Memory modules (from diagnostics memory file)
- Storage devices (from `vars.txt` + SMART files)
- Syslog events

---

### H3C SDS G5 (`h3c_g5`)

**Status:** Ready (v2.2). Tested on H3C UniServer R4900 G5 SDS archives.

**Archive format:** `.sds` (tar archive)

**Detection:** `hardware_info.ini`, `hardware.info`, `firmware_version.ini`, `user/test*.csv`, plus H3C markers.

**Extracted data (current):**
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
- Firmware list (`firmware_version.ini`)
- CPU inventory (`hardware_info.ini`)
- Memory DIMM inventory (`hardware_info.ini`)
- Storage inventory (`hardware.info`, `storage_disk.ini`, `NVMe_info.txt`, RAID text enrichments)
- Logical RAID volumes (`raid.json`, `Storage_RAID-*.txt`)
- Sensor snapshot (`sensor_info.ini`)
- SEL events (`user/test.csv`, `user/test1.csv`, fallback `Sel.json` / `sel_list.txt`)

---

### H3C SDS G6 (`h3c_g6`)

**Status:** Ready (v2.2). Tested on H3C UniServer R4700 G6 SDS archives.

**Archive format:** `.sds` (tar archive)

**Detection:** `CPUDetailInfo.xml`, `MemoryDetailInfo.xml`, `firmware_version.json`, `Sel.json`, plus H3C markers.

**Extracted data (current):**
- Board/FRU inventory (`FRUInfo.ini`, `board_info.ini`)
- Firmware list (`firmware_version.json`)
- CPU inventory (`CPUDetailInfo.xml`)
- Memory DIMM inventory (`MemoryDetailInfo.xml`)
- Storage inventory + capacity/model/interface (`storage_disk.ini`, `Storage_RAID-*.txt`, `NVMe_info.txt`)
- Logical RAID volumes (`raid.json`, fallback from `Storage_RAID-*.txt` when available)
- Sensor snapshot (`sensor_info.ini`)
- SEL events (`user/Sel.json`, fallback `user/sel_list.txt`)

---

### HPE iLO AHS (`hpe_ilo_ahs`)

**Status:** Ready (v1.0.0). Tested on HPE ProLiant Gen11 `.ahs` export from iLO 6.

**Archive format:** `.ahs` single-file Active Health System export.

**Detection:** Single-file input with `ABJR` container header and HPE AHS member names
such as `CUST_INFO.DAT`, `*.zbb`, `ilo_boot_support.zbb`.

**Extracted data (current):**
- System board identity (manufacturer, model, serial, part number)
- iLO / System ROM / SPS top-level firmware
- CPU inventory (model-level)
- Memory DIMM inventory for populated slots
- PSU inventory
- PCIe / OCP NIC inventory from SMBIOS-style slot records
- Storage controller and physical drives from embedded Redfish JSON inside `zbb` members
- Basic iLO event log entries with timestamps when present

**Implementation note:** The format is proprietary. Parser support is intentionally hybrid:
container parsing (`ABJR` + gzip) plus structured extraction from embedded Redfish objects and
printable SMBIOS/FRU payloads. This is sufficient for inventory-grade parsing without decoding the
entire internal `zbb` schema.

---

### Inspur / Kaytus (`inspur`)

**Status:** Ready (v2.2). Tested on Inspur NF5468M7 / Kaytus KR4268X2 onekeylog archives.

**Archive format:** onekeylog BMC diagnostic dump. Two known on-disk layouts:
1. Classic layout: top-level `onekeylog/` directory, single combined `component/component.log`
   with `RESTful <Section> info:` JSON blocks (PSU, fan, HDD, PCIe, network, ...).
2. Per-file layout (seen on newer/OEM firmware): archive root is `dump_<serial>_<timestamp>/`
   (not `onekeylog/`), and `component.log` is absent. Each component is instead dumped separately
   under `component/*.txt` as a raw D-Bus `GetAll` transcript: `GETALL <object> OBJect` blocks with
   tab-separated `"field"\t"type":"x"\t"data":value` triples. This is not valid JSON and needs its
   own line-oriented extraction (`internal/parser/vendors/inspur/component_dbus.go`).

**Detection:** `onekeylog/` path segment, `devicefrusdr.log`, `component/component.log`,
`onekeylog_dreport.log` filename (per-file layout marker), `component/PowerSupplyInfo.txt`
(per-file layout marker), plus `asset.json` content markers (`VersionInfo` + `CpuInfo` +
`MemInfo`).

**Extracted data (current):**
- CPU / Memory / PCIe / Storage inventory from `asset.json` (both layouts)
- FRU + SDR from `devicefrusdr.log` (classic layout)
- PSU inventory: from `component.log`'s `RESTful PSU info:` block (classic layout) or
  `component/PowerSupplyInfo.txt` GETALL transcript (per-file layout)
- Fan RPM/PWM sensors: from `component.log`'s `RESTful fan info:` block (classic layout) or
  `component/FanInfo.txt` GETALL transcript (per-file layout)
- Redis snapshot enrichment for serials/firmware/telemetry when `redis-dump.rdb` is present

**Known gaps (per-file layout only):**
- `component/NetworkAdapter.txt` and `component/HDDBpListInfo.txt` use a different
  `busctl --verbose` object-tree dump (not the GETALL format) and are not currently parsed. NIC
  identity/MAC data is still available via PCIe device inventory from `asset.json`, so this is not
  a data-loss gap for NICs.
- `component/RAID.txt` mixes formats per RAID controller family (`getLsiStorageCollection`,
  `getPMCStorageCollection`); not currently parsed.

**Implementation note:** the same GETALL object name can recur across multiple command sections in
one file with different field subsets (e.g. `Pwm_N` appears once under the FanPWM sensor query
with a real `Value` reading, and again later under FanControl with only a `Target` setpoint).
`parseDBusGetAllObjects` unions fields across recurring object names, first-seen wins per field, so
a later content-free duplicate cannot silently blank out an earlier real reading.

---

### xFusion iBMC Dump / File Export (`xfusion`)

**Status:** Ready (v1.1.0). Tested on xFusion G5500 V7 `tar.gz` exports.

**Archive format:** `tar.gz` dump exported from the iBMC UI, including `AppDump/`, `RTOSDump/`,
and `LogDump/` trees.

**Detection:** `AppDump/FruData/fruinfo.txt`, `AppDump/card_manage/card_info`,
`RTOSDump/versioninfo/app_revision.txt`, and `LogDump/netcard/netcard_info.txt`.

**Extracted data (current):**
- Board / FRU inventory from `fruinfo.txt`
- CPU inventory from `CpuMem/cpu_info`
- Memory DIMM inventory from `CpuMem/mem_info`
- GPU inventory from `card_info`
- OCP NIC inventory by merging `card_info` with `LogDump/netcard/netcard_info.txt`
- PSU inventory from `BMC/psu_info.txt`
- Physical storage from `StorageMgnt/PhysicalDrivesInfo/*/disk_info`
- System firmware entries from `RTOSDump/versioninfo/app_revision.txt`
- Maintenance events from `LogDump/maintenance_log`

---

### Dell TSR (`dell`)

**Status:** Ready (v3.1). Two inventory source generations, both handled by the same parser.

**Archive format:** TSR nested zip (`signature` + `TSR*.pl.zip`).

**Extracted data:**
- Older/DCIM-XML generation: `sysinfo_dcim_view.xml` (board/CPU/NIC/etc.), `sysinfo_dcim_softwareidentity.xml`
  (firmware), `sysinfo_cim_sensor.xml` (sensors), `curr_lclog.xml` (Lifecycle Controller events),
  `tsr/metadata.json` (board identity fallback).
- iDRAC10-generation: same `metadata.json`/`curr_lclog.xml`, but no DCIM-XML files — instead
  `tsr/hardware/sysinfo/inventory/redfishidracwalk.tar.gz` holds a captured Redfish directory-tree
  walk. Parsed via the shared `redfishtree` helper package and replayed through
  `collector.ReplayRedfishFromRawPayloads` (`internal/parser/vendors/dell/redfish_walk.go`), then
  append-merged into the same `Hardware`/`Sensors`/`FRU`/`Events` slices the DCIM-XML path fills, so
  the existing dedupe passes resolve any overlap in favor of DCIM-derived data. See ADL-048.
- Licenses (`hardware.licenses[]`, contract v2.12): the Redfish-walk replay reads the standard DMTF
  `/redfish/v1/LicenseService/Licenses` collection (`internal/collector/redfish_replay_licenses.go`),
  present on iDRAC10-generation firmware — feature-on-demand/advanced licenses such as
  "iDRAC10 17G Enterprise License", "Secure Enterprise Key Manager". System-level licenses have no
  `component_ref`; a license with `AuthorizationScope: "Device"` gets `component_ref` from
  `Links.AuthorizedDevices`. Not sourced from DCIM-XML — only available on the Redfish-walk path.
  See ADL-050.

---

### Generic Redfish walk fallback (`redfish_walk`)

**Status:** Ready (v1.0.0).

**Confidence:** 35 (above the `generic` fallback's 15, below every dedicated vendor parser — wins
`Detect()` only when no dedicated parser also recognizes the archive).

**Purpose:** Vendor-agnostic catch-all for any archive that carries a captured Redfish
directory-tree walk (one JSON document per resource, e.g.
`redfish/v1/Systems/System.Embedded.1/index.json`) but isn't otherwise claimed by a dedicated vendor
parser. First known source is Dell iDRAC10 TSR bundles (handled directly by `dell`, see above); this
exists so any other vendor shipping the same kind of raw Redfish walk is picked up automatically.

**Detection (`internal/parser/vendors/redfishtree`):** two-step, vendor-independent —
1. Path hint: an archive member path containing `redfish` and ending in `.tar.gz`/`.tgz`/`.zip`.
2. Structural confirmation: the unpacked tree must contain a document whose own `@odata.id` is
   exactly `/redfish/v1`, plus a `/redfish/v1/Systems` or `/redfish/v1/Chassis` collection.

Documents are keyed by their own `@odata.id`, not the on-disk directory name (some resource names
are URL-encoded on disk, e.g. `Assembly%23`, but not in the JSON payload).

**Extracted data:** whatever `collector.ReplayRedfishFromRawPayloads` produces from the replayed
tree — board identity, firmware, CPUs, memory, storage, PCIe, GPUs, NICs, PSUs, sensors, events.

---

### Generic text fallback (`generic`)

**Status:** Ready (v1.0.0).

**Confidence:** 15 (lowest — only matches if no other parser scores higher)

**Purpose:** Fallback for any text file or single `.gz` file not matching a specific vendor.

**Behavior:**
- If filename matches `nvidia-bug-report-*.log.gz`: extracts driver version and GPU list.
- Otherwise: confirms file is text (not binary) and records a basic "Text File" event.

---

## Supported vendor matrix

| Vendor | ID | Status | Tested on |
|--------|----|--------|-----------|
| Dell TSR | `dell` | Ready | TSR nested zip archives (DCIM-XML and iDRAC10 Redfish-walk generations) |
| Generic Redfish walk | `redfish_walk` | Ready (fallback) | Any archive with a captured Redfish tree dump |
| Reanimator Easy Bee | `easy_bee` | Ready | `bee-support-*.tar.gz` support bundles |
| HPE iLO AHS | `hpe_ilo_ahs` | Ready | iLO 6 `.ahs` exports |
| Inspur / Kaytus | `inspur` | Ready | KR4268X2 onekeylog |
| Lenovo XCC mini-log | `lenovo_xcc` | Ready | ThinkSystem SR650 V3 XCC mini-log ZIP |
| NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers |
| NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems |
| Unraid | `unraid` | Ready | Unraid diagnostics archives |
| xFusion iBMC dump | `xfusion` | Ready | G5500 V7 file-export `tar.gz` bundles |
| XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs |
| H3C SDS G5 | `h3c_g5` | Ready | H3C UniServer R4900 G5 SDS archives |
| H3C SDS G6 | `h3c_g6` | Ready | H3C UniServer R4700 G6 SDS archives |
| Generic fallback | `generic` | Ready | Any text file |