Scan now reports PrivacyScan.Sanitized {detected, markers, strong, files,
evidence}. SanitizationMarkers recognises a value slot filled with one
repeated placeholder + separators (xxxxx.xxxx.xx, x@xxxx.xxxx.xx,
000.00.00.0, a decoy timezone) - it matches the shape, not the literal "x",
so evolving the redaction mechanism still trips it.
detected requires corroboration: strong>=2, or strong>=1 && markers>=3, or
markers>=4. A single filler-looking token is reported (markers:1) but never
asserted as sanitized, so a partial future pass or a coincidence does not
read as "done". 0.0.0.0 / 000 / UTC / Etc/UTC are too plausibly intentional
and do not count.
UI: the Customer-data panel shows "файл уже обезличен" and hides the
sanitize button when detected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jDYM1nnoZZ3vFz23DDaV1
Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.
- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
"00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
and all embedded timestamps preserved; untouched zip entries copied raw;
member payload length unchanged so tar headers stay byte-identical; only the
.gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
fillers are recognised by isRedactionFiller so a re-scan / second pass is a
no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
"MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
и скачать копию" button in the Customer-data panel, and
logpile -sanitize <file> (restores mtime/atime).
Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Cross-checked internal/privacy against the batch companion-app report over
project/rma. Fixes for the dominant false-positive classes:
- Real-TLD gate (tld.go): FQDN/e-mail must end in a curated TLD or a
pseudo-TLD with a >=3-char label; two-letter file/code suffixes
(.sh .so .md .id .service ...) are a hard denylist. Kills "0.linux"
(45996 hits), "mountall.sh", "libc.so", "@odata.id",
"serial-getty@ttyAMA0.service".
- Clean-token boundary + Title-case reject: "auth.backend.gssapi.store-creds",
"OS.It" are code, not hosts.
- Kernel ring-buffer ("[ 8.07][ T1] ...") and Go stack-trace lines skipped.
- resolv domain/search values must contain a dot ("domain 53" -> out).
- IPv4: skip comment lines, version/spec lines (X.Org, IEEE Std, l0fw_ver),
"0."/"1."/".0" quads; allowlist Yandex resolvers + RFC3849 2001:db8::/32.
- fru_location: drop all-digit / serial-like / field-name-echo values.
- Drop the hostname rule (zero real hits, only "bmc-state-manager" noise).
- domain category: high -> low, medium at 3+ labels. Real customer signal
now comes from resolv/nsupdate/ad_ldap/mgmt, which the corpus confirms
catches every actual customer (netwell.local, tcsbank.ru).
- Allowlist smartmontools.org, openib.org, apache.org, freebsd.org,
golang.org, ipxe.org, nvidia.com and other FOSS/vendor infra; skip
LOGPile's own raw_export.json / parser_fields.json / collect.log members.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Allowlist nvidia.com, mellanox.com, gnu.org, debian.org, ubuntu.com; the
common "not specified / not available" FRU placeholders.
- Reject matches with no alphanumeric or <2 chars (stray ":" from a dumped
resolv line), fru_location values that echo the field name ("Base Board
Asset Tag", "P1-DIMMA1_AssetTag"), IPv4 embedded in a version string
("18:6.1.4.5"), and e-mail on kernel ring-buffer lines (driver copyright).
- Customer guess: don't report a single-hit low-confidence domain at all -
"unidentified" beats guessing nvidia.com from a driver comment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Move the "Customer data" panel to the top of the data section (above the
chart iframe); header + customer guess always visible, findings table
collapsed by default and expandable.
- Add a chart top-notice (above Board/CPUs) summarizing the scan via the
viewer's standard NoticeTitle/NoticeBody - interim until chart custom panels.
- Allowlist ieisystem.com (IEI = Inspur brand infrastructure).
- Add chart-custom-panels-spec.md: a reusable, versioned contract proposal for
host-supplied panels across every app embedding reanimator/chart.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Detection-only scan (internal/privacy) attached to every AnalysisResult:
a customer-domain guess plus a findings list (category, file, line, match,
hint), ported from the KB grep playbook. Runs on archive uploads and the
serialized Redfish tree; gated by LOGPILE_PRIVACY_SCAN (default on).
Surfaced at GET /api/privacy-scan, in the "Customer data" UI panel, and as
privacy_report.json in the raw-export bundle. IP policy keeps RFC1918 and
example ranges out of findings; allowlist covers standards-body and vendor
infrastructure domains. No customer tokens in the repo. See ADL-066 and
bible-local/docs/privacy-scan.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Parse Xid and SXid entries from the kernel log into GPU error events with
severity and the decoded failure reason.
See ADL-055.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add PCI Vendor ID / Device ID to hardware.storage[] per the updated Reanimator
ingest contract, mirroring the existing pcie_devices[] fields. Populated for
Redfish-collected NVMe drives (live + TSR replay, via linked PCIeFunctions)
and for Inspur (from asset.json's own PcieInfo[], joined by PcieSlot). Also
fixes canonicalDevicesForExport dropping the fields when converting Storage
into the canonical HardwareDevice list used by the actual export path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Implements the hardware.licenses[] contract section (v2.12, refreshed from
reanimator/core's hardware-ingest-contract.md — was v2.11 locally).
- models.License / HardwareConfig.Licenses mirror the contract field set.
- collector.collectLicenses() reads the standard DMTF
/redfish/v1/LicenseService/Licenses collection during Redfish-walk replay;
it's a generic DMTF resource, not Dell-specific, so any future vendor's
Redfish walk gets license collection for free through
ReplayRedfishFromRawPayloads.
- vendors/dell merges replayed Licenses like every other category.
- exporter.convertLicenses/dedupeLicenses wire hw.Licenses into the
reanimator export directly (no canonical-devices merge — licenses have no
physical identity to merge on), setting Present on every record from the
start (per the ADL-049 round-trip lesson).
- chart viewer renders a licenses section in /chart/current.
Verified end-to-end on the PowerEdge R7715 (1TVFYL4) TSR: 3 system-level
licenses extracted and correctly exported/rendered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Collect hardware event logs (last 7 days) from Systems and Managers/SEL LogServices
- Parse AMI raw IPMI dump messages into readable descriptions (Sensor_Type: Event_Type)
- Filter out audit/journal/non-hardware log services; only SEL from Managers
- MSI ghost GPU filter: exclude processor GPU entries with temperature=0 when host is powered on
- Reanimator collected_at uses InventoryData/Status.LastModifiedTime (30-day fallback)
- Invalidate Redfish inventory CRC groups before host power-on
- Log inventory LastModifiedTime age in collection logs
- Drop SecureBoot collection (SecureBootMode, SecureBootDatabases) — not hardware inventory
- Add build version to UI footer via template
- Add MSI Redfish API reference doc to bible-local/docs/
ADL-032–ADL-035
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
