Mikhail ChusavitinandClaude Sonnet 5
fb0b0e3c55
- Move the "Customer data" panel to the top of the data section (above the chart iframe); header + customer guess always visible, findings table collapsed by default and expandable. - Add a chart top-notice (above Board/CPUs) summarizing the scan via the viewer's standard NoticeTitle/NoticeBody - interim until chart custom panels. - Allowlist ieisystem.com (IEI = Inspur brand infrastructure). - Add chart-custom-panels-spec.md: a reusable, versioned contract proposal for host-supplied panels across every app embedding reanimator/chart. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
110 lines
5.7 KiB
Markdown
110 lines
5.7 KiB
Markdown
# Privacy / customer-data scan
|
|
|
|
`internal/privacy` scans ingested source files for customer-identifying and
|
|
site-identifying data and attaches a `models.PrivacyScan` to `AnalysisResult`.
|
|
Detection only - it never modifies the source. See ADL-066.
|
|
|
|
## Entry point
|
|
|
|
```go
|
|
privacy.Scan([]privacy.File{{Path, Content}}) *models.PrivacyScan
|
|
```
|
|
|
|
Returns `nil` when there is nothing to scan (no text files). Callers:
|
|
|
|
| Source | Hook |
|
|
|--------|------|
|
|
| Archive upload | `parser.BMCParser.parseFiles` (`internal/parser/parser.go`) |
|
|
| Live Redfish / replay | `ingest.Service.AnalyzeRedfishRawPayloads` scans the JSON-serialized `raw_payloads.redfish_tree` |
|
|
|
|
Gated by `LOGPILE_PRIVACY_SCAN` (`0`/`false`/`off`/`no` disable; default on),
|
|
via `parser.PrivacyScanEnabled()`.
|
|
|
|
## Output
|
|
|
|
- `GET /api/privacy-scan` - the `PrivacyScan` object, or `{"loaded": false}`.
|
|
- UI: the "Customer data" panel (`#privacy-section`) at the top of the data
|
|
section, above the chart iframe. Header + customer guess are always shown; the
|
|
findings table is collapsed by default and expands on click. Hidden entirely
|
|
when there are no findings and no customer guess.
|
|
- The chart viewer also gets a top notice (above Board/CPUs) summarizing the
|
|
scan, via the standard `viewer.RenderOptions.NoticeTitle/NoticeBody`
|
|
(`privacyNotice` in `internal/server/handlers.go`) - interim until the chart
|
|
"custom panels" mechanism ships (see `chart-custom-panels-spec.md`).
|
|
- `privacy_report.json` in the raw-export ZIP (`GET /api/export/json`), omitted
|
|
when the scan produced nothing.
|
|
|
|
## Scan mechanics
|
|
|
|
- Files whose name contains a vendor factory-template marker are skipped whole
|
|
(`_tencent`, `_jingdong`, `_pdd`, `_baidu`, `_kuaishou`, `_tianyiyun`, `_jd.`,
|
|
`syslog_jd`, `snmptrapcfg`, `.json_bak`, `ntp_auto`).
|
|
- Binary content (NUL byte / invalid UTF-8 in the first 8 KiB) is skipped.
|
|
- Line-oriented; lines longer than 8 KiB are truncated for matching.
|
|
- Findings are deduped by `category|match|path`; global cap 1000.
|
|
- A bare-FQDN or e-mail match is dropped when a more precise rule already
|
|
matched the same token on the same line (e.g. `resolv.conf` `domain x` yields
|
|
a `resolv` finding, not also a `domain` one).
|
|
- A match that parses as an IP but is not sensitive (see IP policy) is dropped
|
|
regardless of the rule that produced it.
|
|
|
|
## Rule catalogue (`rules.go`)
|
|
|
|
| Category | Severity | Matches | Remediation target |
|
|
|----------|----------|---------|--------------------|
|
|
| `domain` | high | bare FQDN with a known TLD | replace with `example.local` |
|
|
| `resolv` | high | `domain` / `search` / `nameserver` lines | `resolv.conf` |
|
|
| `ad_ldap` | high | `racdomain=`, `adfilterdcN=`, `rolegroupN{name,domain}=`, `binddn`, `bindpw` | `activedir.conf`, `ldap.conf` |
|
|
| `mgmt_subdomain` | high | `*.mgmt/oob/ipmi/drac/idrac/ilo/bmc.*` | management-network subdomain |
|
|
| `nsupdate` | high | `update add|delete <fqdn> ... A|AAAA|PTR|CNAME` | `runningdata/var/tmp/nsupdate_temp` |
|
|
| `cert` | high | `Subject:`/`Issuer:`/`CN=`/`DNS:` in `*.pem`/`*.csr`/`*.crt`/`*.cer` | certificates |
|
|
| `timezone` | medium/low | `TimeZone=`, `SELTimeUTCOffset=`, `Continent/City`, localized TZ abbreviation before a year | `Etc/UTC`, offset 0 |
|
|
| `email` | medium | `local@host.tld` (OData/Redfish annotations and non-TLD hosts filtered) | EventService / LDAP / certs |
|
|
| `public_ip` | medium | routable IPv4/IPv6 (see IP policy) | RFC 5737 doc ranges |
|
|
| `collector` | medium | `SyslogHostname=`, `@ip:port`, JSON `"Destination"` | syslog / SNMP trap receiver |
|
|
| `fru_location` | medium | `Asset Tag`, `Product Location`, `Chassis Location`, `Board Extra` | FRU fields |
|
|
| `dhcp` | medium | `option domain-name` | customer DHCP lease |
|
|
| `hostname` | low | `sn-`/`bmc-`/`ilo-`/`idrac-`/`drac-`/`srv-` prefixed names | customer naming scheme |
|
|
|
|
## IP policy (`ip.go`)
|
|
|
|
Not sensitive: RFC1918 / RFC4193 ULA, loopback, link-local, unspecified,
|
|
multicast, `0.0.0.0/8`, `255.0.0.0/8` and up; RFC 5737 (`192.0.2/24`,
|
|
`198.51.100/24`, `203.0.113/24`), RFC 2544 (`198.18/15`), RFC 6598 CGNAT
|
|
(`100.64/10`), 6to4 relay anycast; well-known public resolvers
|
|
(`8.8.8.8`, `1.1.1.1`, `9.9.9.9`, `208.67.222.222`, `1.2.3.4`, ...).
|
|
|
|
Everything else that is global unicast is a finding.
|
|
|
|
## Allowlist (`allowlist.go`)
|
|
|
|
Reference data, not vendor-detection logic: RFC 2606/5737 names, `pool.ntp.org`
|
|
/ `nist.gov`, standards bodies (`dmtf.org`, `oasis-open.org`, `w3.org`,
|
|
`iana.org`, `ietf.org`), BMC-stack and vendor infrastructure
|
|
(`megarac.com`, `ami.com`, `openssh.com`, `libssh.org`, `rsyslog.com`,
|
|
`inspur.com`, `inservice-iq.com`, `kaytus.com`, `jd.com`, `jdcloud.com`),
|
|
`foobar.edu` / `issue.net` (stock `hosts.allow` / sshd banner), and factory
|
|
default values (`Asia/Shanghai`, `To Be Filled By O.E.M.`, `NULL`, `0.0.0.0`).
|
|
|
|
## Customer guess (`customer.go`)
|
|
|
|
Findings in `domain`, `resolv`, `ad_ldap`, `cert`, `nsupdate`,
|
|
`mgmt_subdomain`, `dhcp`, `email` are reduced to a registrable domain
|
|
(`eTLD+1`, using a small built-in multi-label suffix list). Score =
|
|
`hits + 3*distinct_files (+10 if seen in a strong category)`. Confidence:
|
|
`high` = strong category and >=2 files; `medium` = one of those; `low`
|
|
otherwise. Top 3 returned, weak single-hit domains dropped unless nothing else
|
|
qualifies. Up to 3 `path:line: excerpt` evidence strings per domain - the domain
|
|
itself is kept in the output (identifying the customer is the point); nothing is
|
|
written to the repo.
|
|
|
|
## Limitations
|
|
|
|
- No sanitization. The report guides a manual cleanup
|
|
(KB: "Очистка BMC-дампа от данных заказчика").
|
|
- Public-suffix handling is a heuristic, not the full PSL.
|
|
- Binary and over-size (>10 MB extractor cap) files are not scanned - a clean
|
|
report is a floor, not a proof.
|
|
- The Redfish path scans only the serialized tree, the one text corpus a
|
|
live/replayed collection carries.
|