feat(privacy): detect an already-sanitized source
Scan now reports PrivacyScan.Sanitized {detected, markers, strong, files,
evidence}. SanitizationMarkers recognises a value slot filled with one
repeated placeholder + separators (xxxxx.xxxx.xx, x@xxxx.xxxx.xx,
000.00.00.0, a decoy timezone) - it matches the shape, not the literal "x",
so evolving the redaction mechanism still trips it.
detected requires corroboration: strong>=2, or strong>=1 && markers>=3, or
markers>=4. A single filler-looking token is reported (markers:1) but never
asserted as sanitized, so a partial future pass or a coincidence does not
read as "done". 0.0.0.0 / 000 / UTC / Etc/UTC are too plausibly intentional
and do not count.
UI: the Customer-data panel shows "файл уже обезличен" and hides the
sanitize button when detected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jDYM1nnoZZ3vFz23DDaV1
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
a63bb17438
commit
f38fb2de69
@@ -155,9 +155,11 @@ Returns normalized parse and collection issues combined from:
|
||||
### `GET /api/privacy-scan`
|
||||
|
||||
Returns the customer-data scan for the current dataset (`models.PrivacyScan`:
|
||||
`files_scanned`, `customers[]`, `findings[]`, `summary`).
|
||||
Returns `{ "loaded": false }` when nothing is loaded or the scan produced
|
||||
nothing. Detection only; see `docs/privacy-scan.md`.
|
||||
`files_scanned`, `customers[]`, `findings[]`, `summary`), plus `sanitizable`
|
||||
(bool) and, when the source already looks de-identified, `sanitized`
|
||||
(`{detected, markers, strong, files, evidence}`).
|
||||
Returns `{ "loaded": false }` when nothing is loaded. Detection only; see
|
||||
`docs/privacy-scan.md`.
|
||||
|
||||
### `POST /api/sanitize`
|
||||
|
||||
|
||||
@@ -1974,6 +1974,10 @@ with **same-length neutral fillers**, in place.
|
||||
reported, not edited.
|
||||
- Re-parsing a sanitized dump yields the identical hardware inventory
|
||||
(verified on Dell TSR, xFusion, Inspur onekeylog, H3C `.sds`).
|
||||
- The scan detects an already-sanitized source
|
||||
(`PrivacyScan.Sanitized`, `privacy/sanitized.go`): it matches the *shape*
|
||||
of a filled value slot, not the literal `x`, and needs corroborating
|
||||
markers so a partial future redaction pass does not read as "done".
|
||||
- Full contract and rule list: `bible-local/docs/log-sanitization.md`.
|
||||
- Tests: `internal/sanitize` (`TestRedactText_*`, `TestSanitize_Tar*`,
|
||||
`TestSanitize_Zip_*`, `TestSanitize_BinaryMemberFlagged`,
|
||||
|
||||
@@ -21,6 +21,11 @@ scan (see `privacy-scan.md`). ADL-067.
|
||||
- **No marker.** Nothing is stamped into the file.
|
||||
- **Idempotent.** The fillers are on the privacy allowlist (`isRedactionFiller`),
|
||||
so a re-scan finds nothing and a second `Sanitize` is a no-op.
|
||||
- **Detectable.** The privacy scan reports an already-sanitized file in
|
||||
`PrivacyScan.Sanitized` - see `privacy-scan.md`. It needs corroborating
|
||||
markers, so a single filler-looking token (or a partial future redaction
|
||||
pass) does not read as "done", and the UI hides the sanitize button when a
|
||||
file already looks de-identified.
|
||||
|
||||
## Entry point
|
||||
|
||||
|
||||
@@ -115,6 +115,23 @@ default values (`Asia/Shanghai`, `To Be Filled By O.E.M.`, `NULL`, `0.0.0.0`).
|
||||
neutral fillers (see `log-sanitization.md`). The fillers are recognised by
|
||||
`isRedactionFiller` so a scan of a sanitized file is clean.
|
||||
|
||||
## Already-sanitized detection (`sanitized.go`)
|
||||
|
||||
`Scan` also reports whether the source already looks de-identified, in
|
||||
`PrivacyScan.Sanitized` (`{detected, markers, strong, files, evidence}`).
|
||||
|
||||
`SanitizationMarkers(line)` recognises a **value slot filled with one repeated
|
||||
placeholder + separators** - `xxxxx.xxxx.xx`, `x@xxxx.xxxx.xx`, `000.00.00.0`,
|
||||
a decoy timezone (`Antarctica/McMurdo`, `Etc/Universal`, ...). It matches the
|
||||
*shape*, not the literal `x`, so changing the filler character later still
|
||||
trips it. `0.0.0.0`, `000`, `UTC`, `Etc/UTC` are too plausibly intentional and
|
||||
do not count.
|
||||
|
||||
`detected` requires corroboration - `strong >= 2`, or `strong >= 1 && markers
|
||||
>= 3`, or `markers >= 4`. A single filler-looking token is reported
|
||||
(`markers: 1`) but never asserted as sanitized, so a partial future redaction
|
||||
pass or a coincidence does not read as "done".
|
||||
|
||||
## Customer guess (`customer.go`)
|
||||
|
||||
Findings in `domain`, `resolv`, `ad_ldap`, `cert`, `nsupdate`,
|
||||
|
||||
Reference in New Issue
Block a user