feat(sanitize): in-place, length-preserving log de-identification

Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This commit is contained in:

Mikhail Chusavitin

2026-09-02 18:05:27 +03:00

co-authored by

Claude Sonnet 5

parent e74e01ad05

commit a63bb17438

25 changed files with 1742 additions and 66 deletions

									
										bible-local/08-build-release.md
									
		+2
		
												View File
												
				@@ -8,6 +8,8 @@ Defined in `cmd/logpile/main.go`:

				|------|---------|---------|

				| `--port` | `8082` | HTTP server port |

				| `--file` | empty | Preload archive file |

				| `--sanitize` | empty | De-identify customer data in this file and exit (`docs/log-sanitization.md`) |

				| `--sanitize-out` | empty | Where `--sanitize` writes its result (default: overwrite the input) |

				| `--version` | `false` | Print version and exit |

				| `--no-browser` | `false` | Do not auto-open browser |

				| `--hold-on-crash` | `true` on Windows | Keep console open after fatal crash |