feat(sanitize): in-place, length-preserving log de-identification
Adds internal/sanitize: rewrites the customer-identifying spans that internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with same-length neutral fillers, in place, without changing the file format. - Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP -> "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed. - Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes, and all embedded timestamps preserved; untouched zip entries copied raw; member payload length unchanged so tar headers stay byte-identical; only the .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output. - privacy.FindSpans is the one matcher shared by detection and redaction; fillers are recognised by isRedactionFiller so a re-scan / second pass is a no-op. New privacy FPs fixed along the way: syslog selectors (local7.info), "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer domains. - Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and unreadable nested archives are reported in Result.SkippedBinary, never edited. - Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить и скачать копию" button in the Customer-data panel, and logpile -sanitize <file> (restores mtime/atime). Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C .sds yields the identical hardware inventory; re-scan is clean. ADL-067, bible-local/docs/log-sanitization.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
e74e01ad05
commit
a63bb17438
@@ -159,6 +159,27 @@ Returns the customer-data scan for the current dataset (`models.PrivacyScan`:
|
||||
Returns `{ "loaded": false }` when nothing is loaded or the scan produced
|
||||
nothing. Detection only; see `docs/privacy-scan.md`.
|
||||
|
||||
### `POST /api/sanitize`
|
||||
|
||||
Builds a de-identified copy of the current uploaded file (customer-data spans
|
||||
replaced with same-length neutral fillers, format and timestamps preserved) and
|
||||
returns the change preview:
|
||||
|
||||
```json
|
||||
{ "filename": "...", "input_size": 0, "output_size": 0,
|
||||
"total_replaced": 0, "changes": [{"path","category","count","sample_before","sample_after"}],
|
||||
"skipped_binary": ["member (binary - sanitize manually)"] }
|
||||
```
|
||||
|
||||
`422` when the current dataset is not an uploaded file (live Redfish / snapshot)
|
||||
or the format cannot be rebuilt. See `docs/log-sanitization.md`.
|
||||
|
||||
### `GET /api/sanitize/download`
|
||||
|
||||
Streams the file produced by the last `POST /api/sanitize`
|
||||
(`Content-Disposition: attachment; filename="<original>"`,
|
||||
`Content-Type: application/octet-stream`). `404` if none is ready.
|
||||
|
||||
### `GET /api/parsers`
|
||||
|
||||
Returns registered parser metadata.
|
||||
|
||||
Reference in New Issue
Block a user