Files

T

Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification

Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-02 18:05:27 +03:00

1.7 KiB

Raw Blame History

01 — Overview

Purpose

LOGPile is a standalone Go application for BMC diagnostics analysis with an embedded web UI. It runs as a single binary and normalizes hardware data from archives or live Redfish collection.

Operating modes

Mode	Entry point	Outcome
Archive upload	`POST /api/upload`	Parse a supported archive, raw export bundle, or JSON snapshot into `AnalysisResult`
Live collection	`POST /api/collect`	Collect from a live BMC via Redfish and store the result in memory
Batch convert	`POST /api/convert`	Convert multiple supported input files into Reanimator JSON in a ZIP artifact

All modes converge on the same normalized hardware model and exporter pipeline.

In scope

Single-binary desktop/server utility with embedded UI
Vendor archive parsing and live Redfish collection
Canonical hardware inventory across UI and exports
Reopenable raw export bundles for future re-analysis
Reanimator export and batch conversion workflows
Embedded pci.ids lookup for vendor/device name enrichment
Customer-data / anonymization scan of ingested sources (detection only; see docs/privacy-scan.md)
In-place, length-preserving de-identification of an uploaded dump (docs/log-sanitization.md)

Current vendor coverage

Dell TSR
Reanimator Easy Bee support bundles
H3C SDS G5/G6
Inspur / Kaytus
HPE iLO AHS
NVIDIA HGX Field Diagnostics
NVIDIA Bug Report
Unraid
xFusion iBMC dump / file export
XigmaNAS
Generic fallback parser

Non-goals

Persistent storage or multi-user state
Production IPMI collection
Authentication/authorization on the built-in HTTP server
Long-term server-side job history beyond in-memory process lifetime

1.7 KiB Raw Blame History