Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.
- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
"00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
and all embedded timestamps preserved; untouched zip entries copied raw;
member payload length unchanged so tar headers stay byte-identical; only the
.gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
fillers are recognised by isRedactionFiller so a re-scan / second pass is a
no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
"MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
и скачать копию" button in the Customer-data panel, and
logpile -sanitize <file> (restores mtime/atime).
Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Detection-only scan (internal/privacy) attached to every AnalysisResult:
a customer-domain guess plus a findings list (category, file, line, match,
hint), ported from the KB grep playbook. Runs on archive uploads and the
serialized Redfish tree; gated by LOGPILE_PRIVACY_SCAN (default on).
Surfaced at GET /api/privacy-scan, in the "Customer data" UI panel, and as
privacy_report.json in the raw-export bundle. IP policy keeps RFC1918 and
example ranges out of findings; allowlist covers standards-body and vendor
infrastructure domains. No customer tokens in the repo. See ADL-066 and
bible-local/docs/privacy-scan.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Add HPE iLO Redfish profile (priority 20): matches on manufacturer/OEM/iLO signals,
adds SmartStorage/SmartStorageConfig to critical paths, sets realistic ETA baseline
and rate policy for iLO's known slowness
- Fix post-probe hang on HPE iLO: skip numeric probing of collections where
Members@odata.count == len(Members); add 4s postProbeClient timeout as safety net
- Exclude /WorkloadPerformanceAdvisor from crawl paths
- Fix replay parser: skip absent CPU sockets, absent DIMM slots, absent drive bays
- Filter N/A version entries from firmware inventory
- Remove drive firmware from general firmware list (already in Storage[].Firmware)
- Add HPE AHS (.ahs) archive parser with hybrid SMBIOS/Redfish extraction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parser / archive:
- Add .sds extension as tar-format alias (archive.go)
- Add tests for multipart upload size limits (multipart_limits_test.go)
- Remove supermicro crashdump parser (ADL-015)
Dell parser:
- Remove GPU duplicates from PCIeDevices (DCIM_VideoView vs DCIM_PCIDeviceView
both list the same GPU; VideoView record is authoritative)
Server:
- Add LOGPILE_CONVERT_MAX_MB env var for independent convert batch size limit
- Improve "file too large" error message with current limit value
Web:
- Add CONVERT_MAX_FILES_PER_BATCH = 1000 cap
- Minor UI copy and CSS fixes
Bible:
- bible-local/06-parsers.md: add pci.ids enrichment rule (enrich model from
pciids when name is empty but vendor_id+device_id are present)
- Sync bible submodule and local overview/architecture docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add bible.git as submodule at bible/
- Move docs/bible/ → bible-local/ (project-specific architecture)
- Update CLAUDE.md to reference both bible/ and bible-local/
- Add AGENTS.md for Codex with same structure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mchusandClaude Sonnet 4.6