26 Commits
Author SHA1 Message Date
Mikhail Chusavitin 78e1a473cc Update published parser dependency to v31 and expose FFDC archives 2026-10-22 21:21:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 c4870cfb36 fix(topology): attach SNC/NPS NUMA nodes to their real socket
Topology mapped one NUMA node per CPU, so on a 2-socket SNC2 host
(nodes 0-3) node 1 went to CPU1 and nodes 2-3 got no CPU: the second
BlueField-3, four GPUs and several NICs rendered unattached.

Use numa-nodes.txt (node=N package=P), now carried by parser v26 as
TopologyEvidence.NUMANodes; without it, fall back to contiguous
per-socket blocks of zero-based node ids. Pin parser v26.0.0. ADL-082.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:41:22 +03:00
Mikhail Chusavitin 23cda0fc83 chore: update parser module to v20.0.0 2026-10-17 00:53:53 +03:00
Mikhail ChusavitinandClaude Sonnet 5 475318fc2b refactor: extract parsers and Redfish replay/analysis to external module
logpile no longer carries its own archive/vendor parsers or Redfish
raw-payload replay/analysis engine. Both now live in
git.mchus.pro/reanimator/parser/v13 (already used by the sibling `core`
project), reached through parser.NewService().AnalyzeArchivePayload /
AnalyzeRedfishRawPayloads. logpile keeps live BMC/Redfish collection,
the DB, exporter, server and UI.

- Deleted internal/parser/ (interface, registry, archive extraction,
  all vendor packages), internal/collector/redfish_replay*.go (replay
  engine, confirmed near-identical to the module's redfish/ package),
  internal/ingest/ (pure duplicate of the module's Service),
  internal/models/ (repointed every import to the module's models
  package instead), and internal/rar5 (only used by the deleted
  archive extractor). Removed the now-unused third_party/pciids
  submodule.
- internal/collector/redfish.go: Collect() now calls
  parser.NewService().AnalyzeRedfishRawPayloads(...) at the one place
  it used to call the local ReplayRedfishFromRawPayloads (an existing
  "unified tunnel" between live collection and raw import). A
  deadcode-driven pass then removed ~150 doc-to-model conversion
  helpers in redfish.go that duplicated the module's redfish/ package
  and were already unreachable before this change; kept the small
  generic dedup/scoring helpers the live crawler still calls.
- internal/collector/redfishprofile/ stays whole: Collect() depends on
  it directly for acquisition-time tuning. Its analysis-only methods
  (ApplyAnalysisProfiles/PostAnalyze) are now dead but left in place
  rather than splitting the package's single Profile interface.
- pciids moved with the module (now a public package there); logpile's
  live collector and exporter import it from there instead of keeping
  a local copy.
- go.mod: new require on git.mchus.pro/reanimator/parser/v13 v13.0.0
  (plain module dependency, not a submodule like reanimator/chart,
  since this module is independently versioned and shared with core).
- bible-local docs rewritten for the new boundary; see ADL-080.

See also the companion change in git.mchus.pro/reanimator/parser
(tagged v13.0.0): new top-level exports
(IsSupportedArchiveFilename/SupportedArchiveExtensions/
ParseInDefaultArchiveLocation/PrivacyScanEnabled) and pciids moved to
a public package, both needed to complete this migration.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 12:59:39 +03:00
Mikhail ChusavitinandClaude Sonnet 5 2b0a2cb562 feat(raw-export): accept AUDIT-SP's signed audit.raw-export.v1 format
The AUDIT-SP collector (rebrand of the logpile collection tool) now stamps
raw_export.json with format "audit.raw-export.v1" instead of
"logpile.raw-export.v1" and ships a raw_export.sig alongside it, which broke
import entirely ("no parser found for this archive format").

- accept both format strings when parsing a raw-export bundle
- for the audit format, verify raw_export.sig (Ed25519, 64 bytes) against the
  exact bytes of raw_export.json using a trusted public key embedded in the
  binary (keys/audit-signing-public.pem); raw_export.sha256 is not consulted
- reject bundles with duplicate raw_export.json or raw_export.sig entries
- import proceeds even if the signature is missing or invalid, but the result
  is marked unverified rather than silently trusted
- expose GET /api/signature-status and show a "Signed" badge in the header
  when the signature verifies

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 11:55:42 +03:00
Mikhail Chusavitin 9ac38c2914 feat: add standalone server topology view 2026-09-14 10:29:36 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification
Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 18:05:27 +03:00
Mikhail ChusavitinandClaude Sonnet 5 4a4910f207 feat(privacy): scan ingested sources for customer-identifying data
Detection-only scan (internal/privacy) attached to every AnalysisResult:
a customer-domain guess plus a findings list (category, file, line, match,
hint), ported from the KB grep playbook. Runs on archive uploads and the
serialized Redfish tree; gated by LOGPILE_PRIVACY_SCAN (default on).

Surfaced at GET /api/privacy-scan, in the "Customer data" UI panel, and as
privacy_report.json in the raw-export bundle. IP policy keeps RFC1918 and
example ranges out of findings; allowlist covers standards-body and vendor
infrastructure domains. No customer tokens in the repo. See ADL-066 and
bible-local/docs/privacy-scan.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 15:24:32 +03:00
Mikhail ChusavitinandClaude Sonnet 4.6 3e3c48bc08 feat(hpe-ilo): parse AHS files, fix event logs export, add logs CSV export
- HPE iLO AHS parser: handle truncated last entry gracefully, recognize
  Alletra product line, expand event type/severity inference, trim iLO
  frame separators from event messages
- Fix event_logs always 0 in Reanimator export: normalizeEventLogSource
  now maps "HPE iLO" → "bmc"
- Fix chart JS not loading in LOGPile: rewriteChartStaticPaths now also
  rewrites src="/static/view.js" → /chart/static/view.js
- Add "Logs Export" button (CSV, semicolon-delimited, UTF-8 BOM) and
  remove PDF button
- Fix collector test broken by pciids rename of Intel VMD device
- Update submodules: chart v2.7, pciids, bible

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 15:11:46 +03:00
mchusandClaude Sonnet 4.6 aba7a54990 feat(parser): lenovo xcc vroc volume parsing - v1.2
Parse inventory_volume.log: Intel VROC (VMD) RAID volumes including
RAID level, capacity (GiB/TiB support added), status and member drives.
Add Drives []string to StorageVolume model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 17:00:27 +03:00
Mikhail ChusavitinandClaude Sonnet 4.6 9df13327aa feat(collect): remove power-on/off, add skip-hung for Redfish collection
Remove power-on and power-off functionality from the Redfish collector;
keep host power-state detection and show a warning in the UI when the
host is powered off before collection starts.

Add a "Пропустить зависшие" (skip hung) button that lets the user abort
stuck Redfish collection phases without losing already-collected data.
Introduces a two-level context model in Collect(): the outer job context
covers the full lifecycle including replay; an inner collectCtx covers
snapshot, prefetch, and plan-B phases only. Closing the skipCh cancels
collectCtx immediately — aborts all in-flight HTTP requests and exits
plan-B loops — then replay runs on whatever rawTree was collected.

Signal path: UI → POST /api/collect/{id}/skip → JobManager.SkipJob()
→ close(skipCh) → goroutine in Collect() → cancelCollect().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 13:12:38 +03:00
Mikhail ChusavitinandClaude Sonnet 4.6 d650a6ba1c refactor: unified ingest pipeline + modular Redfish profile framework
Implement the full architectural plan: unified ingest.Service entry point
for archive and Redfish payloads, modular redfishprofile package with
composable profiles (generic, ami-family, msi, supermicro, dell,
hgx-topology), score-based profile matching with fallback expansion mode,
and profile-driven acquisition/analysis plans.

Vendor-specific logic moved out of common executors and into profile hooks.
GPU chassis lookup strategies and known storage recovery collections
(IntelVROC/HA-RAID/MRVL) now live in ResolvedAnalysisPlan, populated by
profiles at analysis time. Replay helpers read from the plan; no hardcoded
path lists remain in generic code.

Also splits redfish_replay.go into domain modules (gpu, storage, inventory,
fru, profiles) and adds full fixture/matcher/directive test coverage
including Dell, AMI, unknown-vendor fallback, and deterministic ordering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 08:48:58 +03:00
Mikhail Chusavitin 057a222288 ui: embed reanimator chart viewer 2026-03-16 00:20:11 +03:00
Mikhail Chusavitin 476630190d export: align reanimator contract v2.7 2026-03-15 23:27:32 +03:00
mchus 4940cd9645 sync file-type support across upload/convert and fix collected_at timezone handling 2026-02-28 23:27:49 +03:00
mchus 25e3b8bb42 Add convert mode batch workflow with full progress 2026-02-28 21:44:36 +03:00
Mikhail Chusavitin 000199fbdc Add parse errors tab and improve error diagnostics UI 2026-02-25 13:28:19 +03:00
Mikhail Chusavitin ce30f943df Export raw bundles with collection logs and parser field snapshot 2026-02-24 17:36:44 +03:00
Mikhail Chusavitin 810c4b5ff9 Add raw export reanalyze flow for Redfish snapshots 2026-02-24 17:23:26 +03:00
mchus 5e49adaf05 Update parser and project changes 2026-02-15 22:02:07 +03:00
mchusandClaude Sonnet 4.5 1b1bc74fc7 Add Reanimator format export support
Implement export to Reanimator format for asset tracking integration.

Features:
- New API endpoint: GET /api/export/reanimator
- Web UI button "Экспорт Reanimator" in Configuration tab
- Auto-detect CPU manufacturer (Intel/AMD/ARM/Ampere)
- Generate PCIe serial numbers if missing
- Merge GPUs and NetworkAdapters into pcie_devices
- Filter components without serial numbers
- RFC3339 timestamp format
- Full compliance with Reanimator specification

Changes:
- Add reanimator_models.go: data models for Reanimator format
- Add reanimator_converter.go: conversion functions
- Add reanimator_converter_test.go: unit tests
- Add reanimator_integration_test.go: integration tests
- Update handlers.go: add handleExportReanimator
- Update server.go: register /api/export/reanimator route
- Update index.html: add export button
- Update CLAUDE.md: document export behavior
- Add REANIMATOR_EXPORT.md: implementation summary

Tests: All tests passing (15+ new tests)
Format spec: example/docs/INTEGRATION_GUIDE.md

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:54:37 +03:00
Mikhail Chusavitin c89ee0118f Add pluggable live collectors and simplify API connect form 2026-02-04 19:00:03 +03:00
Mikhail Chusavitin d38d0c9d30 feat(backend): add in-memory collect job manager and mock executor 2026-02-04 10:01:51 +03:00
Mikhail Chusavitin aa3c82d9ba feat(api): add live collection contract endpoints 2026-02-04 09:54:48 +03:00
mchusandClaude Opus 4.5 c7422e95aa v1.1.0: Parser versioning, server info, auto-browser, section overviews
- Add parser versioning with Version() method and version display on main screen
- Add server model and serial number to Configuration tab and TXT export
- Add auto-browser opening on startup with --no-browser flag
- Add Restart and Exit buttons with graceful shutdown
- Add section overview stats (CPU, Power, Storage, GPU, Network)
- Change PCIe Link display to "x16 PCIe Gen4" format
- Add Location column to Serials section
- Extract BoardInfo from FRU and PlatformId from ThermalConfig

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 13:49:43 +03:00
mchusandClaude Opus 4.5 512957545a Add LOGPile BMC diagnostic log analyzer
Features:
- Modular parser architecture for vendor-specific formats
- Inspur/Kaytus parser supporting asset.json, devicefrusdr.log,
  component.log, idl.log, and syslog files
- PCI Vendor/Device ID lookup for hardware identification
- Web interface with tabs: Events, Sensors, Config, Serials, Firmware
- Server specification summary with component grouping
- Export to CSV, JSON, TXT formats
- BMC alarm parsing from IDL logs (memory errors, PSU events, etc.)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 04:11:23 +03:00