Mikhail ChusavitinandClaude Sonnet 5
2be215fca1
Cross-checked internal/privacy against the batch companion-app report over project/rma. Fixes for the dominant false-positive classes: - Real-TLD gate (tld.go): FQDN/e-mail must end in a curated TLD or a pseudo-TLD with a >=3-char label; two-letter file/code suffixes (.sh .so .md .id .service ...) are a hard denylist. Kills "0.linux" (45996 hits), "mountall.sh", "libc.so", "@odata.id", "serial-getty@ttyAMA0.service". - Clean-token boundary + Title-case reject: "auth.backend.gssapi.store-creds", "OS.It" are code, not hosts. - Kernel ring-buffer ("[ 8.07][ T1] ...") and Go stack-trace lines skipped. - resolv domain/search values must contain a dot ("domain 53" -> out). - IPv4: skip comment lines, version/spec lines (X.Org, IEEE Std, l0fw_ver), "0."/"1."/".0" quads; allowlist Yandex resolvers + RFC3849 2001:db8::/32. - fru_location: drop all-digit / serial-like / field-name-echo values. - Drop the hostname rule (zero real hits, only "bmc-state-manager" noise). - domain category: high -> low, medium at 3+ labels. Real customer signal now comes from resolv/nsupdate/ad_ldap/mgmt, which the corpus confirms catches every actual customer (netwell.local, tcsbank.ru). - Allowlist smartmontools.org, openib.org, apache.org, freebsd.org, golang.org, ipxe.org, nvidia.com and other FOSS/vendor infra; skip LOGPile's own raw_export.json / parser_fields.json / collect.log members. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
LOGPile Bible
bible-local/ is the project-specific source of truth for LOGPile.
Keep top-level docs minimal and put maintained architecture/API contracts here.
Rules
- Documentation language: English only
- Update relevant bible files in the same change as the code
- Record significant architectural decisions in
10-decisions.md - Do not duplicate shared rules from
bible/
Read order
| File | Purpose |
|---|---|
| 01-overview.md | Product scope, modes, non-goals |
| 02-architecture.md | Runtime structure, state, main flows |
| 04-data-models.md | Stable data contracts and canonical inventory |
| 03-api.md | HTTP endpoints and response contracts |
| 05-collectors.md | Live collection behavior |
| 06-parsers.md | Archive parser framework and vendor coverage |
| 07-exporters.md | Raw export, Reanimator export, batch convert |
| docs/hardware-ingest-contract.md | Reanimator ingest schema mirrored locally |
| docs/privacy-scan.md | Customer-data / anonymization scan of ingested sources |
| 08-build-release.md | Build and release workflow |
| 09-testing.md | Test expectations and regression rules |
| 10-decisions.md | Architectural Decision Log |
Fast orientation
- Entry point:
cmd/logpile/main.go - HTTP layer:
internal/server/ - Core contracts:
internal/models/models.go - Live collection:
internal/collector/ - Archive parsing:
internal/parser/ - Export conversion:
internal/exporter/ - Frontend consumer:
web/static/js/app.js
Maintenance rule
If a document becomes stale, either fix it immediately or delete it. Stale docs are worse than missing docs.