Commit Graph
6 Commits
Author SHA1 Message Date
Mikhail ChusavitinandClaude Opus 5.5 cb2e38223e chore: update parser module to v38.0.0, script parser bumps
Move logpile to git.mchus.pro/reanimator/parser/v38 v38.0.0 (ADL-085).
Add scripts/bump-parser.sh, which bumps the parser, tests, commits, tags
the release as v<counter>.<parser major>, pushes and builds (ADL-086).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 18:07:28 +03:00
Mikhail Chusavitin 78e1a473cc Update published parser dependency to v31 and expose FFDC archives 2026-10-22 21:21:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 c4870cfb36 fix(topology): attach SNC/NPS NUMA nodes to their real socket
Topology mapped one NUMA node per CPU, so on a 2-socket SNC2 host
(nodes 0-3) node 1 went to CPU1 and nodes 2-3 got no CPU: the second
BlueField-3, four GPUs and several NICs rendered unattached.

Use numa-nodes.txt (node=N package=P), now carried by parser v26 as
TopologyEvidence.NUMANodes; without it, fall back to contiguous
per-socket blocks of zero-based node ids. Pin parser v26.0.0. ADL-082.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:41:22 +03:00
Mikhail Chusavitin 23cda0fc83 chore: update parser module to v20.0.0 2026-10-17 00:53:53 +03:00
Mikhail ChusavitinandClaude Sonnet 5 475318fc2b refactor: extract parsers and Redfish replay/analysis to external module
logpile no longer carries its own archive/vendor parsers or Redfish
raw-payload replay/analysis engine. Both now live in
git.mchus.pro/reanimator/parser/v13 (already used by the sibling `core`
project), reached through parser.NewService().AnalyzeArchivePayload /
AnalyzeRedfishRawPayloads. logpile keeps live BMC/Redfish collection,
the DB, exporter, server and UI.

- Deleted internal/parser/ (interface, registry, archive extraction,
  all vendor packages), internal/collector/redfish_replay*.go (replay
  engine, confirmed near-identical to the module's redfish/ package),
  internal/ingest/ (pure duplicate of the module's Service),
  internal/models/ (repointed every import to the module's models
  package instead), and internal/rar5 (only used by the deleted
  archive extractor). Removed the now-unused third_party/pciids
  submodule.
- internal/collector/redfish.go: Collect() now calls
  parser.NewService().AnalyzeRedfishRawPayloads(...) at the one place
  it used to call the local ReplayRedfishFromRawPayloads (an existing
  "unified tunnel" between live collection and raw import). A
  deadcode-driven pass then removed ~150 doc-to-model conversion
  helpers in redfish.go that duplicated the module's redfish/ package
  and were already unreachable before this change; kept the small
  generic dedup/scoring helpers the live crawler still calls.
- internal/collector/redfishprofile/ stays whole: Collect() depends on
  it directly for acquisition-time tuning. Its analysis-only methods
  (ApplyAnalysisProfiles/PostAnalyze) are now dead but left in place
  rather than splitting the package's single Profile interface.
- pciids moved with the module (now a public package there); logpile's
  live collector and exporter import it from there instead of keeping
  a local copy.
- go.mod: new require on git.mchus.pro/reanimator/parser/v13 v13.0.0
  (plain module dependency, not a submodule like reanimator/chart,
  since this module is independently versioned and shared with core).
- bible-local docs rewritten for the new boundary; see ADL-080.

See also the companion change in git.mchus.pro/reanimator/parser
(tagged v13.0.0): new top-level exports
(IsSupportedArchiveFilename/SupportedArchiveExtensions/
ParseInDefaultArchiveLocation/PrivacyScanEnabled) and pciids moved to
a public package, both needed to complete this migration.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 12:59:39 +03:00
Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification
Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 18:05:27 +03:00