Commit Graph
12 Commits
Author SHA1 Message Date
Mikhail Chusavitin 23cda0fc83 chore: update parser module to v20.0.0 2026-10-17 00:53:53 +03:00
Mikhail ChusavitinandClaude Sonnet 5 475318fc2b refactor: extract parsers and Redfish replay/analysis to external module
logpile no longer carries its own archive/vendor parsers or Redfish
raw-payload replay/analysis engine. Both now live in
git.mchus.pro/reanimator/parser/v13 (already used by the sibling `core`
project), reached through parser.NewService().AnalyzeArchivePayload /
AnalyzeRedfishRawPayloads. logpile keeps live BMC/Redfish collection,
the DB, exporter, server and UI.

- Deleted internal/parser/ (interface, registry, archive extraction,
  all vendor packages), internal/collector/redfish_replay*.go (replay
  engine, confirmed near-identical to the module's redfish/ package),
  internal/ingest/ (pure duplicate of the module's Service),
  internal/models/ (repointed every import to the module's models
  package instead), and internal/rar5 (only used by the deleted
  archive extractor). Removed the now-unused third_party/pciids
  submodule.
- internal/collector/redfish.go: Collect() now calls
  parser.NewService().AnalyzeRedfishRawPayloads(...) at the one place
  it used to call the local ReplayRedfishFromRawPayloads (an existing
  "unified tunnel" between live collection and raw import). A
  deadcode-driven pass then removed ~150 doc-to-model conversion
  helpers in redfish.go that duplicated the module's redfish/ package
  and were already unreachable before this change; kept the small
  generic dedup/scoring helpers the live crawler still calls.
- internal/collector/redfishprofile/ stays whole: Collect() depends on
  it directly for acquisition-time tuning. Its analysis-only methods
  (ApplyAnalysisProfiles/PostAnalyze) are now dead but left in place
  rather than splitting the package's single Profile interface.
- pciids moved with the module (now a public package there); logpile's
  live collector and exporter import it from there instead of keeping
  a local copy.
- go.mod: new require on git.mchus.pro/reanimator/parser/v13 v13.0.0
  (plain module dependency, not a submodule like reanimator/chart,
  since this module is independently versioned and shared with core).
- bible-local docs rewritten for the new boundary; see ADL-080.

See also the companion change in git.mchus.pro/reanimator/parser
(tagged v13.0.0): new top-level exports
(IsSupportedArchiveFilename/SupportedArchiveExtensions/
ParseInDefaultArchiveLocation/PrivacyScanEnabled) and pciids moved to
a public package, both needed to complete this migration.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 12:59:39 +03:00
mchus c9a9eaf1dc test: cover cpu audit reanimator workflow 2026-09-12 16:46:36 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification
Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 18:05:27 +03:00
Mikhail ChusavitinandClaude Sonnet 5 fc68134ed9 feat(cmd): add logpile-cpu-audit offline CPU identity audit CLI
Non-server entry point over the parser registry: recursively discovers
supported dumps, parses them with bounded concurrency, and writes one
path-mirrored CPU identity report per input plus a deterministic summary.
Failures are isolated as JSON. A -reanimator mode emits a flat CPU-only
dataset via the production exporter.

See ADL-059.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 17:05:52 +03:00
mchusandClaude Sonnet 4.6 57de3ba6eb chore: align codebase with bible engineering contracts
- identifier-normalization: use strings.EqualFold in h3c/parser.go
- import-export: CSV now uses UTF-8 BOM and semicolon delimiter
- go-code-style: translate all Russian source strings to English (ADL-007)
- go-background-tasks: add Type, Message, Result fields to Job struct
- go-api: wrap list endpoints in {items, total_count, page, per_page, total_pages}
- module-structure: rename helpers.go → context_sleep.go
- build-version-display: htmlError renders version footer on error pages
- go-logging: migrate all log.Printf calls to log/slog with structured attrs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 14:35:39 +03:00
mchusandClaude Sonnet 4.6 aba7a54990 feat(parser): lenovo xcc vroc volume parsing - v1.2
Parse inventory_volume.log: Intel VROC (VMD) RAID volumes including
RAID level, capacity (GiB/TiB support added), status and member drives.
Add Drives []string to StorageVolume model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 17:00:27 +03:00
Mikhail Chusavitin ce30f943df Export raw bundles with collection logs and parser field snapshot 2026-02-24 17:36:44 +03:00
Mikhail Chusavitin bb48b03677 Redfish snapshot/export overhaul and portable release build 2026-02-04 19:43:51 +03:00
Mikhail ChusavitinandClaude Sonnet 4.5 21f4e5a67e v1.2.0: Enhanced Inspur/Kaytus parser with GPU, PCIe, and storage support
Major improvements:
- Add CSV SEL event parser for Kaytus firmware format
- Add PCIe device parser with link speed/width detection
- Add GPU temperature and PCIe link monitoring
- Add disk backplane parser for storage bay information
- Fix memory module detection (only show installed DIMMs)

Parser enhancements:
- Parse RESTful PCIe Device info (max/current link width/speed)
- Parse GPU sensor data (core and memory temperatures)
- Parse diskbackplane info (slot count, installed drives)
- Parse SEL events from CSV format (selelist.csv)
- Fix memory Present status logic (check mem_mod_status)

Web interface improvements:
- Add PCIe link degradation highlighting (red when current < max)
- Add storage table with Present status and location
- Update memory specification to show only installed modules with frequency
- Sort events from newest to oldest
- Filter out N/A serial numbers from display

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-30 12:30:18 +03:00
mchusandClaude Opus 4.5 c7422e95aa v1.1.0: Parser versioning, server info, auto-browser, section overviews
- Add parser versioning with Version() method and version display on main screen
- Add server model and serial number to Configuration tab and TXT export
- Add auto-browser opening on startup with --no-browser flag
- Add Restart and Exit buttons with graceful shutdown
- Add section overview stats (CPU, Power, Storage, GPU, Network)
- Change PCIe Link display to "x16 PCIe Gen4" format
- Add Location column to Serials section
- Extract BoardInfo from FRU and PlatformId from ThermalConfig

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 13:49:43 +03:00
mchusandClaude Opus 4.5 512957545a Add LOGPile BMC diagnostic log analyzer
Features:
- Modular parser architecture for vendor-specific formats
- Inspur/Kaytus parser supporting asset.json, devicefrusdr.log,
  component.log, idl.log, and syslog files
- PCI Vendor/Device ID lookup for hardware identification
- Web interface with tabs: Events, Sensors, Config, Serials, Firmware
- Server specification summary with component grouping
- Export to CSV, JSON, TXT formats
- BMC alarm parsing from IDL logs (memory errors, PSU events, etc.)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 04:11:23 +03:00