logpile/bible-local/06-parsers.md

06 — Parsers

Framework

Parsers live in internal/parser/ and vendor implementations live in internal/parser/vendors/.

Core behavior:

• registration uses init() side effects
• all registered parsers run Detect()
• the highest-confidence parser wins
• generic fallback stays last and low-confidence

VendorParser contract:

type VendorParser interface {
    Name() string
    Vendor() string
    Version() string
    Detect(files []ExtractedFile) int
    Parse(files []ExtractedFile) (*models.AnalysisResult, error)
}

Adding a parser

1. Create internal/parser/vendors/<vendor>/
2. Start from internal/parser/vendors/template/parser.go.template
3. Implement Detect() and Parse()
4. Add a blank import in internal/parser/vendors/vendors.go
5. Add at least one positive and one negative detection test

Data quality rules

System firmware only in hardware.firmware

hardware.firmware must contain system-level firmware only. Device-bound firmware belongs on the device record and must not be duplicated at the top level.

Strip embedded MAC addresses from model names

If a source embeds - XX:XX:XX:XX:XX:XX in a model/name field, remove that suffix before storing it.

Use pci.ids for empty or generic PCI model names

When vendor_id and device_id are known but the model name is missing or generic, resolve the name via internal/parser/vendors/pciids.

Preserve source-backed CPU identity

Keep a source PPIN in CPU.PPIN. If the source has no separate processor serial number, use models.ResolveCPUSerialNumber to expose the valid PPIN as CPU.SerialNumber. Never derive a CPU serial from its socket, model, board serial, or another component; reject source placeholders.

Active vendor coverage

Vendor ID	Input family	Notes
dell	TSR ZIP archives	Broad hardware, firmware, sensors, lifecycle events
easy_bee	bee-support-*.tar.gz	Imports embedded export/bee-audit.json snapshot from reanimator-easy-bee bundles
h3c_g5	H3C SDS G5 bundles	INI/XML/CSV-driven hardware and event parsing
h3c_g6	H3C SDS G6 bundles	Similar flow with G6-specific files
hpe_ilo_ahs	HPE iLO Active Health System (.ahs)	Proprietary ABJR container with gzip-compressed zbb members; parser combines SMBIOS-style inventory strings and embedded Redfish storage JSON
inspur	onekeylog archives (classic component.log and per-file component/*.txt D-Bus layouts)	FRU/SDR, optional Redis enrichment, normalized IDL/SEL/syslog events (see ADL-056)
lenovo_xcc	Lenovo XCC mini-log ZIP archives	JSON inventory + platform event logs
nvidia	HGX Field Diagnostics	GPU- and fabric-heavy diagnostic input
nvidia_bug_report	nvidia-bug-report-*.log.gz	dmidecode, lspci, NVIDIA driver sections; Xid/SXid GPU error events (see ADL-055, bible-local/docs/nvidia-bug-report-analysis.md)
redfish_walk	Any archive carrying a captured Redfish directory-tree walk (tar.gz/zip of <path>/index.json docs)	Low-confidence (35) vendor-agnostic fallback; see dell for the first known source and redfishtree helper package
unraid	Unraid diagnostics/log bundles	Server and storage-focused parsing
xfusion	xFusion iBMC tar.gz dump / file export	AppDump + RTOSDump + LogDump merge for hardware and firmware
xigmanas	XigmaNAS plain logs	FreeBSD/NAS-oriented inventory
generic	fallback	Low-confidence text fallback when nothing else matches

Practical guidance

• Be conservative with high detect scores
• Prefer filling missing fields over overwriting stronger source data
• Keep parser version constants current when behavior changes
• Any new vendor-specific filtering or dedup logic must ship with tests for that vendor format

Archive format: Unraid diagnostics archive contents (text-heavy diagnostics directories).

Detection: Combines filename/path markers (diagnostics-*, unraid-*.txt, vars.txt) with content markers (e.g. Unraid kernel build, parity data markers).

Extracted data (current):

• Board / BIOS metadata (from motherboard/system files)
• CPU summary (from lscpu.txt)
• Memory modules (from diagnostics memory file)
• Storage devices (from vars.txt + SMART files)
• Syslog events

---

H3C SDS G5 (h3c_g5)

Status: Ready (v2.2). Tested on H3C UniServer R4900 G5 SDS archives.

Archive format: .sds (tar archive)

Detection: hardware_info.ini, hardware.info, firmware_version.ini, user/test*.csv, plus H3C markers.

Extracted data (current):

• Board/FRU inventory (FRUInfo.ini, board_info.ini)
• Firmware list (firmware_version.ini)
• CPU inventory (hardware_info.ini)
• Memory DIMM inventory (hardware_info.ini)
• Storage inventory (hardware.info, storage_disk.ini, NVMe_info.txt, RAID text enrichments)
• Logical RAID volumes (raid.json, Storage_RAID-*.txt)
• Sensor snapshot (sensor_info.ini)
• SEL events (user/test.csv, user/test1.csv, fallback Sel.json / sel_list.txt)

---

H3C SDS G6 (h3c_g6)

Status: Ready (v2.2). Tested on H3C UniServer R4700 G6 SDS archives.

Archive format: .sds (tar archive)

Detection: CPUDetailInfo.xml, MemoryDetailInfo.xml, firmware_version.json, Sel.json, plus H3C markers.

Extracted data (current):

• Board/FRU inventory (FRUInfo.ini, board_info.ini)
• Firmware list (firmware_version.json)
• CPU inventory (CPUDetailInfo.xml)
• Memory DIMM inventory (MemoryDetailInfo.xml)
• Storage inventory + capacity/model/interface (storage_disk.ini, Storage_RAID-*.txt, NVMe_info.txt)
• Logical RAID volumes (raid.json, fallback from Storage_RAID-*.txt when available)
• Sensor snapshot (sensor_info.ini)
• SEL events (user/Sel.json, fallback user/sel_list.txt)

---

HPE iLO AHS (hpe_ilo_ahs)

Status: Ready (v1.0.0). Tested on HPE ProLiant Gen11 .ahs export from iLO 6.

Archive format: .ahs single-file Active Health System export.

Detection: Single-file input with ABJR container header and HPE AHS member names such as CUST_INFO.DAT, *.zbb, ilo_boot_support.zbb.

Extracted data (current):

• System board identity (manufacturer, model, serial, part number)
• iLO / System ROM / SPS top-level firmware
• CPU inventory (model-level)
• Memory DIMM inventory for populated slots
• PSU inventory
• PCIe / OCP NIC inventory from SMBIOS-style slot records
• Storage controller and physical drives from embedded Redfish JSON inside zbb members
• Basic iLO event log entries with timestamps when present

Implementation note: The format is proprietary. Parser support is intentionally hybrid: container parsing (ABJR + gzip) plus structured extraction from embedded Redfish objects and printable SMBIOS/FRU payloads. This is sufficient for inventory-grade parsing without decoding the entire internal zbb schema.

---

Inspur / Kaytus (inspur)

Status: Ready (v2.2). Tested on Inspur NF5468M7 / Kaytus KR4268X2 onekeylog archives.

Archive format: onekeylog BMC diagnostic dump. Two known on-disk layouts:

1. Classic layout: top-level onekeylog/ directory, single combined component/component.log with RESTful <Section> info: JSON blocks (PSU, fan, HDD, PCIe, network, ...).
2. Per-file layout (seen on newer/OEM firmware): archive root is dump_<serial>_<timestamp>/ (not onekeylog/), and component.log is absent. Each component is instead dumped separately under component/*.txt as a raw D-Bus GetAll transcript: GETALL <object> OBJect blocks with tab-separated "field"\t"type":"x"\t"data":value triples. This is not valid JSON and needs its own line-oriented extraction (internal/parser/vendors/inspur/component_dbus.go).

Detection: onekeylog/ path segment, devicefrusdr.log, component/component.log, onekeylog_dreport.log filename (per-file layout marker), component/PowerSupplyInfo.txt (per-file layout marker), plus asset.json content markers (VersionInfo + CpuInfo + MemInfo).

Extracted data (current):

• CPU / Memory / PCIe / Storage inventory from asset.json (both layouts)
• FRU + SDR from devicefrusdr.log (classic layout)
• PSU inventory: from component.log's RESTful PSU info: block (classic layout) or component/PowerSupplyInfo.txt GETALL transcript (per-file layout)
• Fan RPM/PWM sensors: from component.log's RESTful fan info: block (classic layout) or component/FanInfo.txt GETALL transcript (per-file layout)
• Redis snapshot enrichment for serials/firmware/telemetry when redis-dump.rdb is present

Known gaps (per-file layout only):

• component/NetworkAdapter.txt and component/HDDBpListInfo.txt use a different busctl --verbose object-tree dump (not the GETALL format) and are not currently parsed. NIC identity/MAC data is still available via PCIe device inventory from asset.json, so this is not a data-loss gap for NICs.
• component/RAID.txt mixes formats per RAID controller family (getLsiStorageCollection, getPMCStorageCollection); not currently parsed.

Implementation note: the same GETALL object name can recur across multiple command sections in one file with different field subsets (e.g. Pwm_N appears once under the FanPWM sensor query with a real Value reading, and again later under FanControl with only a Target setpoint). parseDBusGetAllObjects unions fields across recurring object names, first-seen wins per field, so a later content-free duplicate cannot silently blank out an earlier real reading.

---

xFusion iBMC Dump / File Export (xfusion)

Status: Ready (v1.1.0). Tested on xFusion G5500 V7 tar.gz exports.

Archive format: tar.gz dump exported from the iBMC UI, including AppDump/, RTOSDump/, and LogDump/ trees.

Detection: AppDump/FruData/fruinfo.txt, AppDump/card_manage/card_info, RTOSDump/versioninfo/app_revision.txt, and LogDump/netcard/netcard_info.txt.

Extracted data (current):

• Board / FRU inventory from fruinfo.txt
• CPU inventory from CpuMem/cpu_info
• Memory DIMM inventory from CpuMem/mem_info
• GPU inventory from card_info
• OCP NIC inventory by merging card_info with LogDump/netcard/netcard_info.txt
• PSU inventory from BMC/psu_info.txt
• Physical storage from StorageMgnt/PhysicalDrivesInfo/*/disk_info
• System firmware entries from RTOSDump/versioninfo/app_revision.txt
• Maintenance events from LogDump/maintenance_log

---

Dell TSR (dell)

Status: Ready (v3.1). Two inventory source generations, both handled by the same parser.

Archive format: TSR nested zip (signature + TSR*.pl.zip).

Extracted data:

• Older/DCIM-XML generation: sysinfo_dcim_view.xml (board/CPU/NIC/etc.), sysinfo_dcim_softwareidentity.xml (firmware), sysinfo_cim_sensor.xml (sensors), curr_lclog.xml (Lifecycle Controller events), tsr/metadata.json (board identity fallback).
• iDRAC10-generation: same metadata.json/curr_lclog.xml, but no DCIM-XML files — instead tsr/hardware/sysinfo/inventory/redfishidracwalk.tar.gz holds a captured Redfish directory-tree walk. Parsed via the shared redfishtree helper package and replayed through collector.ReplayRedfishFromRawPayloads (internal/parser/vendors/dell/redfish_walk.go), then append-merged into the same Hardware/Sensors/FRU/Events slices the DCIM-XML path fills, so the existing dedupe passes resolve any overlap in favor of DCIM-derived data. See ADL-048.
• Licenses (hardware.licenses[], contract v2.12): the Redfish-walk replay reads the standard DMTF /redfish/v1/LicenseService/Licenses collection (internal/collector/redfish_replay_licenses.go), present on iDRAC10-generation firmware — feature-on-demand/advanced licenses such as "iDRAC10 17G Enterprise License", "Secure Enterprise Key Manager". System-level licenses have no component_ref; a license with AuthorizationScope: "Device" gets component_ref from Links.AuthorizedDevices. Not sourced from DCIM-XML — only available on the Redfish-walk path. See ADL-050.

---

Generic Redfish walk fallback (redfish_walk)

Status: Ready (v1.0.0).

Confidence: 35 (above the generic fallback's 15, below every dedicated vendor parser — wins Detect() only when no dedicated parser also recognizes the archive).

Purpose: Vendor-agnostic catch-all for any archive that carries a captured Redfish directory-tree walk (one JSON document per resource, e.g. redfish/v1/Systems/System.Embedded.1/index.json) but isn't otherwise claimed by a dedicated vendor parser. First known source is Dell iDRAC10 TSR bundles (handled directly by dell, see above); this exists so any other vendor shipping the same kind of raw Redfish walk is picked up automatically.

Detection (internal/parser/vendors/redfishtree): two-step, vendor-independent —

1. Path hint: an archive member path containing redfish and ending in .tar.gz/.tgz/.zip.
2. Structural confirmation: the unpacked tree must contain a document whose own @odata.id is exactly /redfish/v1, plus a /redfish/v1/Systems or /redfish/v1/Chassis collection.

Documents are keyed by their own @odata.id, not the on-disk directory name (some resource names are URL-encoded on disk, e.g. Assembly%23, but not in the JSON payload).

Extracted data: whatever collector.ReplayRedfishFromRawPayloads produces from the replayed tree — board identity, firmware, CPUs, memory, storage, PCIe, GPUs, NICs, PSUs, sensors, events.

---

Generic text fallback (generic)

Status: Ready (v1.0.0).

Confidence: 15 (lowest — only matches if no other parser scores higher)

Purpose: Fallback for any text file or single .gz file not matching a specific vendor.

Behavior:

• If filename matches nvidia-bug-report-*.log.gz: extracts driver version and GPU list.
• Otherwise: confirms file is text (not binary) and records a basic "Text File" event.

---

Supported vendor matrix

Vendor	ID	Status	Tested on
Dell TSR	dell	Ready	TSR nested zip archives (DCIM-XML and iDRAC10 Redfish-walk generations)
Generic Redfish walk	redfish_walk	Ready (fallback)	Any archive with a captured Redfish tree dump
Reanimator Easy Bee	easy_bee	Ready	bee-support-*.tar.gz support bundles
HPE iLO AHS	hpe_ilo_ahs	Ready	iLO 6 .ahs exports
Inspur / Kaytus	inspur	Ready	KR4268X2 onekeylog
Lenovo XCC mini-log	lenovo_xcc	Ready	ThinkSystem SR650 V3 XCC mini-log ZIP
NVIDIA HGX Field Diag	nvidia	Ready	Various HGX servers
NVIDIA Bug Report	nvidia_bug_report	Ready	H100 systems
Unraid	unraid	Ready	Unraid diagnostics archives
xFusion iBMC dump	xfusion	Ready	G5500 V7 file-export tar.gz bundles
XigmaNAS	xigmanas	Ready	FreeBSD NAS logs
H3C SDS G5	h3c_g5	Ready	H3C UniServer R4900 G5 SDS archives
H3C SDS G6	h3c_g6	Ready	H3C UniServer R4700 G6 SDS archives
Generic fallback	generic	Ready	Any text file