Mikhail ChusavitinandClaude Sonnet 5
a63bb17438
Adds internal/sanitize: rewrites the customer-identifying spans that internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with same-length neutral fillers, in place, without changing the file format. - Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP -> "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed. - Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes, and all embedded timestamps preserved; untouched zip entries copied raw; member payload length unchanged so tar headers stay byte-identical; only the .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output. - privacy.FindSpans is the one matcher shared by detection and redaction; fillers are recognised by isRedactionFiller so a re-scan / second pass is a no-op. New privacy FPs fixed along the way: syslog selectors (local7.info), "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer domains. - Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and unreadable nested archives are reported in Result.SkippedBinary, never edited. - Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить и скачать копию" button in the Customer-data panel, and logpile -sanitize <file> (restores mtime/atime). Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C .sds yields the identical hardware inventory; re-scan is clean. ADL-067, bible-local/docs/log-sanitization.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
45 lines
2.0 KiB
Markdown
45 lines
2.0 KiB
Markdown
# LOGPile Bible
|
|
|
|
`bible-local/` is the project-specific source of truth for LOGPile.
|
|
Keep top-level docs minimal and put maintained architecture/API contracts here.
|
|
|
|
## Rules
|
|
|
|
- Documentation language: English only
|
|
- Update relevant bible files in the same change as the code
|
|
- Record significant architectural decisions in [`10-decisions.md`](10-decisions.md)
|
|
- Do not duplicate shared rules from `bible/`
|
|
|
|
## Read order
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| [01-overview.md](01-overview.md) | Product scope, modes, non-goals |
|
|
| [02-architecture.md](02-architecture.md) | Runtime structure, state, main flows |
|
|
| [04-data-models.md](04-data-models.md) | Stable data contracts and canonical inventory |
|
|
| [03-api.md](03-api.md) | HTTP endpoints and response contracts |
|
|
| [05-collectors.md](05-collectors.md) | Live collection behavior |
|
|
| [06-parsers.md](06-parsers.md) | Archive parser framework and vendor coverage |
|
|
| [07-exporters.md](07-exporters.md) | Raw export, Reanimator export, batch convert |
|
|
| [docs/hardware-ingest-contract.md](docs/hardware-ingest-contract.md) | Reanimator ingest schema mirrored locally |
|
|
| [docs/privacy-scan.md](docs/privacy-scan.md) | Customer-data / anonymization scan of ingested sources |
|
|
| [docs/log-sanitization.md](docs/log-sanitization.md) | In-place, length-preserving redaction of customer data |
|
|
| [08-build-release.md](08-build-release.md) | Build and release workflow |
|
|
| [09-testing.md](09-testing.md) | Test expectations and regression rules |
|
|
| [10-decisions.md](10-decisions.md) | Architectural Decision Log |
|
|
|
|
## Fast orientation
|
|
|
|
- Entry point: `cmd/logpile/main.go`
|
|
- HTTP layer: `internal/server/`
|
|
- Core contracts: `internal/models/models.go`
|
|
- Live collection: `internal/collector/`
|
|
- Archive parsing: `internal/parser/`
|
|
- Export conversion: `internal/exporter/`
|
|
- Frontend consumer: `web/static/js/app.js`
|
|
|
|
## Maintenance rule
|
|
|
|
If a document becomes stale, either fix it immediately or delete it.
|
|
Stale docs are worse than missing docs.
|