Commit Graph
230 Commits
Author SHA1 Message Date
Mikhail Chusavitin 9ac38c2914 feat: add standalone server topology view v1.37 2026-09-14 10:29:36 +03:00
mchus 5b8be0e464 Merge branch 'test/full-coverage' 2026-09-12 17:16:29 +03:00
mchus 176ee61606 test: cover archive file selection helpers 2026-09-12 16:47:39 +03:00
mchus c9a9eaf1dc test: cover cpu audit reanimator workflow 2026-09-12 16:46:36 +03:00
mchus 95d83ea818 test: cover unraid inventory integration 2026-09-12 16:45:49 +03:00
mchus 615a655dd8 test: cover nvidia bug report inventory parsing 2026-09-12 16:44:33 +03:00
mchus cf9c6b8909 test: make portable fixtures optional and cover xigmanas 2026-09-12 16:42:13 +03:00
mchusandClaude Sonnet 5 a5c77db9c6 docs(releases): v1.36 release notes (cumulative since v1.35)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.36
2026-09-10 12:00:11 +03:00
mchusandClaude Sonnet 5 ccd91c902b chore(chart): bump submodule to 52c372c (PrintMode on latest main)
Rebases the PrintMode viewer change onto the current chart main
(consolidated rendering / licenses section).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:58:33 +03:00
mchusandClaude Sonnet 5 71f4c237fc fix(inspur): match main board FRU on word boundary
A bare "board" substring test let compound module names like
"FANBOARD0_FRU" qualify as the main system board, overwriting the
real board info with fan-board serial/part numbers. Switch to a
word-boundary regex.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:58:33 +03:00
Mikhail ChusavitinandClaude Sonnet 5 95f552a0e5 feat(privacy): fold sanitize into the header "Remove personal data" button
Drops the in-panel sanitize sub-block (button + preview table + "already
sanitized" note) that duplicated affordances across the page. The header
"Clear Data" button is renamed "Remove personal data" and now sanitizes the
uploaded file and downloads the clean copy in one click - no preview, no
confirmation. It is shown only when the source can be sanitized, and reads
"Personal data removed" (disabled) when the file already looks de-identified.
The old clearData() teardown is covered by Restart.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.35
2026-09-03 10:19:44 +03:00
Mikhail ChusavitinandClaude Sonnet 5 21670801f3 docs(releases): v1.35 release notes (cumulative since v1.34)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jDYM1nnoZZ3vFz23DDaV1
2026-09-03 10:11:07 +03:00
Mikhail ChusavitinandClaude Sonnet 5 f38fb2de69 feat(privacy): detect an already-sanitized source
Scan now reports PrivacyScan.Sanitized {detected, markers, strong, files,
evidence}. SanitizationMarkers recognises a value slot filled with one
repeated placeholder + separators (xxxxx.xxxx.xx, x@xxxx.xxxx.xx,
000.00.00.0, a decoy timezone) - it matches the shape, not the literal "x",
so evolving the redaction mechanism still trips it.

detected requires corroboration: strong>=2, or strong>=1 && markers>=3, or
markers>=4. A single filler-looking token is reported (markers:1) but never
asserted as sanitized, so a partial future pass or a coincidence does not
read as "done". 0.0.0.0 / 000 / UTC / Etc/UTC are too plausibly intentional
and do not count.

UI: the Customer-data panel shows "файл уже обезличен" and hides the
sanitize button when detected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jDYM1nnoZZ3vFz23DDaV1
2026-09-03 10:02:02 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification
Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 18:05:27 +03:00
Mikhail ChusavitinandClaude Sonnet 5 e74e01ad05 docs(releases): note the RMA-corpus privacy calibration in v1.34
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.34
2026-09-02 16:48:47 +03:00
Mikhail ChusavitinandClaude Sonnet 5 2be215fca1 fix(privacy): calibrate against the RMA log corpus (130k files)
Cross-checked internal/privacy against the batch companion-app report over
project/rma. Fixes for the dominant false-positive classes:

- Real-TLD gate (tld.go): FQDN/e-mail must end in a curated TLD or a
  pseudo-TLD with a >=3-char label; two-letter file/code suffixes
  (.sh .so .md .id .service ...) are a hard denylist. Kills "0.linux"
  (45996 hits), "mountall.sh", "libc.so", "@odata.id",
  "serial-getty@ttyAMA0.service".
- Clean-token boundary + Title-case reject: "auth.backend.gssapi.store-creds",
  "OS.It" are code, not hosts.
- Kernel ring-buffer ("[ 8.07][ T1] ...") and Go stack-trace lines skipped.
- resolv domain/search values must contain a dot ("domain 53" -> out).
- IPv4: skip comment lines, version/spec lines (X.Org, IEEE Std, l0fw_ver),
  "0."/"1."/".0" quads; allowlist Yandex resolvers + RFC3849 2001:db8::/32.
- fru_location: drop all-digit / serial-like / field-name-echo values.
- Drop the hostname rule (zero real hits, only "bmc-state-manager" noise).
- domain category: high -> low, medium at 3+ labels. Real customer signal
  now comes from resolv/nsupdate/ad_ldap/mgmt, which the corpus confirms
  catches every actual customer (netwell.local, tcsbank.ru).
- Allowlist smartmontools.org, openib.org, apache.org, freebsd.org,
  golang.org, ipxe.org, nvidia.com and other FOSS/vendor infra; skip
  LOGPile's own raw_export.json / parser_fields.json / collect.log members.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:45:35 +03:00
Mikhail ChusavitinandClaude Sonnet 5 551d8e450d fix(privacy): drop nvidia-bug-report / dmidecode false positives
- Allowlist nvidia.com, mellanox.com, gnu.org, debian.org, ubuntu.com; the
  common "not specified / not available" FRU placeholders.
- Reject matches with no alphanumeric or <2 chars (stray ":" from a dumped
  resolv line), fru_location values that echo the field name ("Base Board
  Asset Tag", "P1-DIMMA1_AssetTag"), IPv4 embedded in a version string
  ("18:6.1.4.5"), and e-mail on kernel ring-buffer lines (driver copyright).
- Customer guess: don't report a single-hit low-confidence domain at all -
  "unidentified" beats guessing nvidia.com from a driver comment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:27:17 +03:00
Mikhail ChusavitinandClaude Sonnet 5 c9748f3830 docs(releases): v1.34 release notes (cumulative since v1.31)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:21:24 +03:00
Mikhail ChusavitinandClaude Sonnet 5 fb0b0e3c55 feat(privacy): surface the customer-data scan above the hardware report
- Move the "Customer data" panel to the top of the data section (above the
  chart iframe); header + customer guess always visible, findings table
  collapsed by default and expandable.
- Add a chart top-notice (above Board/CPUs) summarizing the scan via the
  viewer's standard NoticeTitle/NoticeBody - interim until chart custom panels.
- Allowlist ieisystem.com (IEI = Inspur brand infrastructure).
- Add chart-custom-panels-spec.md: a reusable, versioned contract proposal for
  host-supplied panels across every app embedding reanimator/chart.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 16:19:57 +03:00
Mikhail ChusavitinandClaude Sonnet 5 4a4910f207 feat(privacy): scan ingested sources for customer-identifying data
Detection-only scan (internal/privacy) attached to every AnalysisResult:
a customer-domain guess plus a findings list (category, file, line, match,
hint), ported from the KB grep playbook. Runs on archive uploads and the
serialized Redfish tree; gated by LOGPILE_PRIVACY_SCAN (default on).

Surfaced at GET /api/privacy-scan, in the "Customer data" UI panel, and as
privacy_report.json in the raw-export bundle. IP policy keeps RFC1918 and
example ranges out of findings; allowlist covers standards-body and vendor
infrastructure domains. No customer tokens in the repo. See ADL-066 and
bible-local/docs/privacy-scan.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.33
2026-09-02 15:24:32 +03:00
Mikhail ChusavitinandClaude Sonnet 5 3311bafd8e feat(inspur_legacy): new parser for pre-Kaytus Inspur onekeylog
Older AMI-BMC Inspur onekeylog archives (NF5466M5 / NF5280M5 generation)
opened to an empty result: they carry none of the files the inspur parser
keys on (no asset.json, devicefrusdr.log, selelist.csv or component.log).

New package internal/parser/vendors/inspur_legacy (vendor id inspur_legacy),
separate from inspur:
- binary IPMI FRU decode (FRU.bin) -> board identity
- Inspur_AssetInfoInventory.log -> CPU / memory / PCIe / PSU inventory
- events from Inspur_<model>_<serial>_IDL, sel.log, blackbox.log,
  MegaRAID raid0.log, and the flat AMI <severity>.log files
- no live sensors in this archive class -> recorded as a collection error
- BMC clock timestamps before 2010 dropped as un-set (1970 / ~2005 RTC)

Registry: add optional PrioritizedParser { DetectPriority() int } so a
confidence tie is broken by specificity. inspur_legacy returns 10 and also
declines (Detect 0) when modern Kaytus markers are present, so the two
Inspur parsers never fight over a newer dump.

Docs: ADL-065, 06-parsers.md, releases/v1.32.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017VL8wLGD6Lnp6hZCpqT6cZ
v1.32
2026-09-02 12:51:53 +03:00
Mikhail ChusavitinandClaude Sonnet 5 be37f1852b docs(releases): fill in v1.31 release notes (cumulative since v1.26)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LffAvostt3uMkiUbVUiyM
2026-09-01 12:13:15 +03:00
Mikhail ChusavitinandClaude Sonnet 5 8278c77d98 fix(inspur): strip iBMC firmware build-timestamp suffix
A second NF5280M6 BMC-dump/live-CD pair showed RESTful version info
firmware versions carrying a build stamp ("08.05.01 (02/21/2024
16:51:27)") the live-CD does not, which churns a FIRMWARE_CHANGED event
on every source switch. cleanFirmwareVersion strips a trailing " (...)"
so BIOS/BMC match the bare live-CD version. See ADL-064 amendment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LffAvostt3uMkiUbVUiyM
v1.31
2026-09-01 12:02:19 +03:00
Mikhail ChusavitinandClaude Sonnet 5 2fa0f78f94 fix(inspur): read RESTful FRU info and float fans_power from component.log
Diffing an NF5280M6 BMC dump against its BEE-SP live-CD bundle found two
blind spots in the combined-component.log onekeylog layout (no
devicefrusdr.log / asset.json):

- board manufacturer/product/part/uuid empty and stats.fru 0: the
  "RESTful FRU info:" JSON block was never parsed. New component_fru.go
  (ParseComponentLogFRU) flattens it to []models.FRUInfo, prefers the
  product-area system serial over the board PCB serial, and sets
  BoardInfo.UUID from system_uuid. Wired as a fallback only when
  result.FRU is still empty.
- zero fan sensors: FanRESTInfo.FansPower was int but this firmware
  writes "fans_power": 12.000000, so json.Unmarshal of the whole fan
  block failed. Changed to float64.

Also included: SOL smartd SCSI/SAS device-line parsing and diagnose.go
gofmt from concurrent work on the same live-CD-diff task. See ADL-064.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LffAvostt3uMkiUbVUiyM
2026-09-01 11:58:55 +03:00
Mikhail ChusavitinandClaude Sonnet 5 ab8636da04 fix(xfusion): parse disk_info Capacity in TB/binary units for size_gb
parseDiskInfo read capacity with Sscanf(cap, "%f GB"), so every iBMC
"Capacity : 6.986 TB" NVMe line failed to match and size_gb stayed 0
while the BEE-SP live-CD export of the same drive had the real size.
The iBMC number is also binary (GiB/TiB) despite the GB/TB label.

parseDiskCapacityGB parses number + unit (TB/GB/MB), treats it as
binary, and emits decimal GB, matching the drive's marketed capacity
and the live-CD inventory. Found by diffing a G5500 V7 BMC dump against
its live-CD bundle. See ADL-063.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LffAvostt3uMkiUbVUiyM
2026-09-01 11:58:46 +03:00
Mikhail ChusavitinandClaude Sonnet 5 8e7f22077d fix(xfusion): emit OCP NIC ports with no serial so reanimator keys them by MAC
Verified against reanimator/core: it keys a NIC component on serial → MAC →
synthetic(slot). A BMC dump carrying the card serial collapses both ports into
one serial-keyed component, while a live-CD bundle (no serial) keeps two
MAC-keyed components, so alternating the two sources produced a spurious
COMPONENT_REMOVED + 2x COMPONENT_INSTALLED every time. Emitting the per-port
NetworkAdapter entries with an empty serial makes both sources key on the port
MAC and yield the same two components. The card serial still reaches the export
via the FRU record and the legacy NIC entry.

Also records ADL-062 findings from reading the ingesting audit tool: it persists
only vendor/model/serial per component (no size/speed/link/clock/wattage), so the
remaining one-sided field diffs between the two bundles cannot generate change
events.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VFy4m7cVv4cqp25jJh2gSB
v1.30
2026-08-31 22:55:45 +03:00
Mikhail ChusavitinandClaude Sonnet 5 9b2c654182 fix(parser,exporter): reconcile BMC-dump and live-CD exports of the same server
The audit tool that ingests Reanimator exports treats any per-component field
change between imports as a component replacement. Importing an xFusion BMC dump
and an easy_bee BEE-SP bundle for one server produced large spurious diffs.

xfusion:
- parseMemInfo tolerates a stray 0x0A inside the binary SPD "bom number" column
  (it was splitting a DIMM record in two and emitting a phantom "slot s" module)
- DIMM slot from the "dimm name" column ("DIMM071"), mainboard "location" dropped
- GPU slot = BDF (Reanimator contract)
- NIC emitted per PCI function from netcard_info.txt (BDF + per-port MAC, shared
  card serial) instead of one card-level adapter with the wrong BDF, which had
  been colliding with a GPU and vanishing in dedup
- NIC manufacturer left blank when it is the system OEM so the exporter resolves
  the silicon vendor from pci.ids
- "(U6216)" chip designator stripped from firmware versions

easy_bee:
- PSU bay numbers rebased 0-indexed -> 1-indexed; bare single-letter PSU
  "firmware" (a leaked FRU version) cleared
- board.part_number taken from the bundle's ipmitool-fru.txt chassis
  "Product Part Number" to match the BMC value

exporter (cross-vendor):
- canonicalMemorySlot: drop dmidecode "(J)" channel tag, Memory111 -> DIMM111
- canonicalGPUModel: NVIDIA DC GPUs reduce to the bare chip token
- canonicalStorageMediaAndInterface: NVMe is a bus not a medium
- manufacturerFromStorageModel: fill blank drive vendor from the model prefix
- isRemovableUSBStorageDevice: drop live-CD boot sticks
- isOnboardControllerPCIeDevice: drop SATA/NVMe/MegaRAID/PCIe-switch controller
  functions that only an lspci scan reports (keep add-in cards with an identity)

For the reference server every physical component now appears in both exports
keyed identically; residual diffs are one-sided enrichment only.

Refs ADL-061, ADL-062.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VFy4m7cVv4cqp25jJh2gSB
2026-08-31 22:50:07 +03:00
Mikhail ChusavitinandClaude Sonnet 5 5b65c99b0e fix(server): build ping address with net.JoinHostPort for IPv6 hosts
fmt.Sprintf("%s:%d", host, port) produces a malformed address for IPv6
literal hosts (go vet: "address format does not work with IPv6").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:08:51 +03:00
Mikhail ChusavitinandClaude Sonnet 5 70c939ea4b docs(bible-local): record ADL-054..060 and parser/exporter notes
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.29
2026-08-27 17:05:56 +03:00
Mikhail ChusavitinandClaude Sonnet 5 fc68134ed9 feat(cmd): add logpile-cpu-audit offline CPU identity audit CLI
Non-server entry point over the parser registry: recursively discovers
supported dumps, parses them with bounded concurrency, and writes one
path-mirrored CPU identity report per input plus a deterministic summary.
Failures are isolated as JSON. A -reanimator mode emits a flat CPU-only
dataset via the production exporter.

See ADL-059.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:52 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a430a8c46f feat(nvidia_bug_report): extract Xid/SXid GPU error events
Parse Xid and SXid entries from the kernel log into GPU error events with
severity and the decoded failure reason.

See ADL-055.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:52 +03:00
Mikhail ChusavitinandClaude Sonnet 5 e7b8a8badc feat(inspur): source-PRI timezone timeline, PPIN serial, NVMe fault storage status
Event ingestion now uses the source syslog PRI and an explicit-offset
timezone timeline instead of assuming host-local time. CPU PPIN is
exported as the source-backed CPU serial, and an active NVMe fault SEL
event promotes the matching drive's storage status.

See ADL-056, ADL-057.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:45 +03:00
Mikhail ChusavitinandClaude Sonnet 5 07c270cda2 feat(models): resolve CPU serial from source PPIN as common fallback
Centralize CPU identity in models.ResolveCPUSerialNumber: an explicit
source serial wins, otherwise a valid source PPIN is used, placeholders
rejected. Dell, H3C and Redfish apply it while parsing; canonical-device
and Reanimator conversion apply it again at the output boundary. No
identity is synthesized from socket/model/board serial.

See ADL-058.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:40 +03:00
Mikhail ChusavitinandClaude Sonnet 5 9d701885da feat(parser): bound single-file gzip by ratio guard instead of fixed byte cap
Plain gzipped logs (nvidia-bug-report-*.log.gz) routinely exceed the old
50MB decompression cap, which silently dropped the tail. Replace it with a
decompression-ratio bomb guard plus a 1GB absolute memory ceiling.

See ADL-054.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:33 +03:00
Mikhail ChusavitinandClaude Sonnet 5 632067fef5 fix(parser): dedupe HPE AHS DIMM inventory per slot, newest snapshot wins
The AHS blackbox stores a DIMM inventory snapshot per POST cycle and
parseDIMMs flattens all of them. dedupeMemory keyed on serial, so a slot
whose module was swapped between captures survived once per historical
occupant — a 24-slot board reported 26 modules and "same P/N" vs
"mixed P/N" configs looked identical.

dedupeMemory now collapses to one entry per non-empty slot with the last
(most recent, since token order follows chronological blackbox-record
order) occurrence winning. Slotless entries keep the serial/part fallback.

See ADL-060.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 16:49:12 +03:00
Mikhail ChusavitinandClaude Sonnet 5 99d8f098ac fix(parser): match easy_bee bee-audit.json/runtime-health.json outside export/
bee-support v12.x bundles place bee-audit.json under tasks/_state/ and
runtime-health.json under status/, not export/. Detection was path-locked
to export/, dropping confidence to 50 and relying on a coincidental
export/reanimator.json duplicate to find the snapshot at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.28
2026-08-24 17:22:24 +03:00
Mikhail ChusavitinandClaude Sonnet 5 6599ab49c2 fix(collector): surface RAID/HBA controller cards from Storage.StorageControllers[]
The RAID controller (XFusion XC170-M-8i / Broadcom SAS3808) never appeared
in inventory at all: it isn't listed in any Chassis/Systems PCIeDevices
collection on this BMC, and its dedicated Board resource link 404s (id
contains parentheses, same class of bug as the OCP NIC fixed earlier). Its
full identity -- model, firmware, BDF, vendor/device IDs -- was sitting
unread in the Storage resource's embedded StorageControllers[] array the
whole time.

Added parseStorageControllerPCIeDevice + collectStorageControllers to read
that array and surface the controller as a PCIeDevice entry, merged into
the existing pcie_devices list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-18 14:30:56 +03:00
Mikhail ChusavitinandClaude Sonnet 5 1e4ec513e1 fix(collector): dedup GPUs and resolve model text on xFusion boards without PCI identity
A re-run against the G5500 V7 (previous fix commit a3567dd) surfaced two
follow-on bugs once the GPUs started being classified correctly:

- gpuDocDedupKey fell straight through to the full @odata.id path when a
  GPU had neither SerialNumber nor BDF, so the same physical H100 exposed
  under both Systems/1/PCIeDevices and Chassis/1/PCIeDevices (identical
  resource Id, different collection root) was kept twice, doubling the
  reported GPU count. Added an Id + resolved VendorID/DeviceID fallback
  tier that collapses same-Id duplicates without collapsing genuinely
  distinct GPUs that happen to share a vendor/device pair.

- isMissingOrRawPCIModel/isGenericRedfishInventoryName didn't recognize
  xFusion's generic "PCIeCardN"/"OCPCardN" slot labels, so when the doc's
  actual Model field was empty and GPU.Model fell back to the slot-like
  Name field, the pci.ids VendorId/DeviceId resolution never fired and the
  GPU surfaced with a meaningless model like "PCIeCard1" instead of the
  real chip name.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-18 14:24:49 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a3567dd5f6 fix(collector): recover GPUs/NICs dropped on xFusion G5500 Redfish exports
looksLikeGPU now falls back to resolving VendorId/DeviceId through the
pci.ids database when the BMC leaves Name/Model/Manufacturer/ClassCode
empty, so GPUs identifiable only by raw PCI IDs (e.g. NVIDIA H100 SXM5
0x10de/0x2330) are no longer misclassified as generic PCIe devices.

The replay pipeline's "backed by canonical NIC" dedup used to trust a
PCIeDevice's Links.NetworkDeviceFunctions reference at face value and
drop the device, assuming a NetworkAdapters record existed elsewhere.
On BMCs that expose resource IDs with characters (parentheses) that
404 on fetch, that canonical NIC never gets captured, so the device
carrying its actual hardware identity vanished from the export
entirely. hasResolvableLinkedMember now verifies the linked resource
is actually present in the snapshot before treating it as authoritative.

Also normalize PartNumber through normalizeRedfishIdentityField in the
GPU/PCIe parsers so a BMC-supplied literal "null" string doesn't leak
into exports verbatim.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-18 14:17:25 +03:00
Mikhail ChusavitinandClaude Sonnet 5 2c3072cf10 fix(exporter): fix memory dedup collapsing distinct DIMMs, disambiguate slots
dedupeMemory keyed on slot before serial, unlike dedupeStorage/dedupePSUs
which both prefer serial. Some collectors (observed on MSI CG480-S6053 with
older BEE-SP versions — confirmed reproducible across multiple servers,
fixed in newer BEE-SP versions) report every memory module at the same slot
label ("DIMM 0") even though each has a distinct real serial. Slot-first
keying collapsed all of them into a single record on export — 16 real 32GB
DIMMs (512GB) reduced to 1, discarding 15 physical modules' worth of data.

Serial now takes priority, matching storage/PSU. Also add
disambiguateMemorySlots (same pattern as the existing PSU slot fix): when
multiple already-distinct DIMMs still share a slot label, renumber them to
"DIMM {n}" by ascending serial order so Reanimator can track them by
position and results are deterministic across re-imports.

Updated the existing dedup test (was asserting the old collapse-by-slot
behavior on a same-slot/different-serial case, which was exactly the bug)
and added a dedicated regression test for the collision case.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.27
2026-08-17 11:05:19 +03:00
Mikhail ChusavitinandClaude Sonnet 5 6827c1fe20 fix(exporter): assign PSU slot positions by serial order, not encounter order
disambiguatePSUSlots previously renumbered colliding PSU slots in whatever
order the devices happened to appear in after parsing/dedup, which isn't
guaranteed stable across runs. Sort each colliding group by serial number
ascending before assigning 0-based positions, so the same PSU always lands
in the same slot on re-conversion regardless of incidental ordering upstream.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 09:56:13 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a590242382 fix(exporter): disambiguate PSU slot collisions to avoid Reanimator "other"
Some collectors (BEE-SP observed so far) report every PSU at slot "0". The
export itself already keeps every PSU as a separate record (dedupePSUs keys
on serial first), but Reanimator relies on slot to track installed position,
so PSUs sharing a slot value land in "other components" instead of being
tracked as PSU installations.

Renumber colliding slots to stable 0-based positions by encounter order.
This doesn't invent a serial or other identity — the PSUs are already
distinguished by serial — it only assigns the position field Reanimator
needs when the source failed to. Files where every PSU already has a
unique slot are unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 09:52:33 +03:00
Mikhail ChusavitinandClaude Sonnet 5 48baae41f4 fix(exporter): key PCIe canonical dedup by BDF before serial
canonicalKey used serial_number as the primary merge key for pcie-class
devices (PCIe/GPU/Network), falling back to BDF only when serial was empty.
Multi-port NICs commonly report one serial for the whole physical card
across all of its ports/BDFs (e.g. a dual-port ConnectX-5: same serial,
distinct BDFs 0000:65:00.0/.1, distinct MACs). Keying on serial first
collapsed every port sharing that serial into a single canonical device,
silently dropping all but one port from the export.

BDF now takes priority for pcie-class devices, since it uniquely identifies
one physical PCI function; other device kinds (storage, memory, PSU, etc.)
keep the existing serial-first behavior, which is correct there since a
serial genuinely maps 1:1 to a physical unit for those.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-17 09:42:34 +03:00
Mikhail ChusavitinandClaude Sonnet 5 8665f79fd6 fix(parser,exporter): fix large tar.gz truncation and empty device_class defaulting
- archive.go: extractTarGzFromReader truncated decompressed tar.gz content
  at a hard 50MB byte boundary before checking whether it was a tar archive,
  corrupting the tar structure mid-entry for any legitimately large archive
  (e.g. a 200MB decompressed NVIDIA bug-report bundle) and causing the whole
  file to fail with "tar read: unexpected EOF" instead of extracting what's
  there. Now peeks the first 512-byte tar block to detect tar vs. single
  gzipped file without consuming the stream, and streams tar entries with a
  cumulative (not raw-byte) size limit that only ever stops at an entry
  boundary. The byte-level cap still applies to the single-gzipped-file case,
  where it's safe since there's no container structure to corrupt.

- reanimator_converter.go: normalizeLegacyPCIeDeviceClass mapped an empty
  device_class to "NetworkController" by accident (grouped into the same
  case as "network"/"ethernet" aliases). Sources that never populate a class
  at all (e.g. Dell's DCIM_PCIDeviceView) got every such device — including
  NVMe drives and SATA controllers — mislabeled as network controllers.
  Empty now stays empty instead of being guessed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-15 14:00:55 +03:00
Mikhail Chusavitin 4fa16c78a5 perf(convert): parallelize batch conversion across worker pool
Batch convert processed files strictly sequentially. Parsing/converting
is CPU-bound and stateless per file, so run it across a GOMAXPROCS-sized
worker pool; zip writing and progress updates stay serialized in
original file order.
2026-08-15 12:43:21 +03:00
Mikhail ChusavitinandClaude Sonnet 5 42cfa3aa94 fix(exporter): drop dead bmc_event_summary field, fix status loss for inspur CPU/GPU
bmc_event_summary was a derived Assert/Deassert summary added in 4409594
alongside real fixes for GPU fault handling. It's not part of the Reanimator
hardware-ingest contract (event_logs is the only accepted log channel) and
was silently dropped on import — pure dead weight, removed.

Three related status bugs surfaced while auditing converted exports against
the contract, all specific to Inspur/onekeylog dumps:
- CPU status from RESTful CPU info was parsed but never assigned to
  models.CPU, and was skipped entirely whenever asset.json already supplied
  a CPU list (its own inventory has no status field) — CPUs always exported
  as Unknown even when the source reported OK.
- PCIe device status (RESTful "status": 1) was parsed but never mapped onto
  models.PCIeDevice, so RESTful-only devices always lost status.
- For GPUs specifically, asset.go emits two device records per physical GPU
  (a generic pcie_devices entry enriched with real status, and a separate
  gpus entry with a resolved model name but no status). dedupePCIe picks a
  single winner by quality score, and a better model name outweighed having
  a real status — the winner kept "Unknown" even when a losing duplicate had
  the real value. dedupePCIe now backfills status onto the winner from a
  losing duplicate when the winner's is Unknown.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-15 12:11:16 +03:00
Mikhail ChusavitinandClaude Sonnet 5 0867123a91 feat(exporter): flag PCIe/GPU/NIC devices with degraded link width as Warning
Applies a single universal check (negotiated LinkWidth < MaxLinkWidth) in
convertPCIeFromDevices, the shared conversion path all vendor parsers feed
into, so a narrower-than-supported link (bad seat, bent connector, wrong
riser) surfaces as a Warning status regardless of which parser produced the
reading — instead of the previous OtrdDiagnoseComponent.json-only check that
only covered the newer HGX dump layout.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-13 18:15:08 +03:00
Mikhail ChusavitinandClaude Sonnet 5 93f0897b81 fix(exporter): dedup NICs by normalized MAC, fix garbage redis serial fallback
MAC-format mismatches between collector sources (dash vs colon separators)
were preventing duplicate NIC/PCIe entries from merging in the canonical
device dedup pass. Add MAC address normalization and merge devices that
share a normalized MAC before the existing serial/BDF-based dedup runs.

Also fix a bug in the Inspur redis-dump serial fallback parser: when a
field's inline value was the placeholder "N/A", the code incorrectly fell
through to a window-scan fallback that could pick up an unrelated adjacent
Redis key name (e.g. "AssetInfoPCIEMMIOSpace") as a fake serial number.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-13 18:09:49 +03:00
Mikhail ChusavitinandClaude Sonnet 5 399eca5f49 feat(exporter): collect and export storage.vendor_id/device_id (contract v2.13)
Add PCI Vendor ID / Device ID to hardware.storage[] per the updated Reanimator
ingest contract, mirroring the existing pcie_devices[] fields. Populated for
Redfish-collected NVMe drives (live + TSR replay, via linked PCIeFunctions)
and for Inspur (from asset.json's own PcieInfo[], joined by PcieSlot). Also
fixes canonicalDevicesForExport dropping the fields when converting Storage
into the canonical HardwareDevice list used by the actual export path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-11 16:04:54 +03:00
Mikhail Chusavitin 063587958e Merge remote-tracking branch 'origin/main'
# Conflicts:
#	bible-local/10-decisions.md
#	internal/chart
#	internal/models/models.go
v1.26
2026-08-11 12:12:03 +03:00