Files

T

Mikhail ChusavitinandClaude Sonnet 5 a63bb17438 feat(sanitize): in-place, length-preserving log de-identification

Adds internal/sanitize: rewrites the customer-identifying spans that
internal/privacy detects (domain/FQDN/e-mail/AD/public-IP/timezone) with
same-length neutral fillers, in place, without changing the file format.

- Fillers keep byte length: "sigma.sbrf.ru" -> "xxxxx.xxxx.xx", IP ->
  "00.000.000.00", "Europe/Moscow" -> "Etc/Universal" (same-length valid
  neutral IANA zone), offset "180" -> "000". Timestamps are not recomputed.
- Lossless recursive archive walk (tar/.sds/gz/tgz/zip): entry names, modes,
  and all embedded timestamps preserved; untouched zip entries copied raw;
  member payload length unchanged so tar headers stay byte-identical; only the
  .gz/.zip compression layer is rebuilt. 0 redactions -> byte-identical output.
- privacy.FindSpans is the one matcher shared by detection and redaction;
  fillers are recognised by isRedactionFiller so a re-scan / second pass is a
  no-op. New privacy FPs fixed along the way: syslog selectors (local7.info),
  "MEVersion" firmware quads, *.conf_bak vendor templates, bundled viewer
  domains.
- Binary members (FRU.bin, localtime, redis-dump.rdb, SOL captures) and
  unreadable nested archives are reported in Result.SkippedBinary, never edited.
- Surfaces: POST /api/sanitize (+ GET /api/sanitize/download), the "Обезличить
  и скачать копию" button in the Customer-data panel, and
  logpile -sanitize <file> (restores mtime/atime).

Verified: re-parsing a sanitized Dell TSR / xFusion / Inspur onekeylog / H3C
.sds yields the identical hardware inventory; re-scan is clean. ADL-067,
bible-local/docs/log-sanitization.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-02 18:05:27 +03:00

2.4 KiB

Raw Blame History

08 — Build & Release

CLI flags

Defined in cmd/logpile/main.go:

Flag	Default	Purpose
`--port`	`8082`	HTTP server port
`--file`	empty	Preload archive file
`--sanitize`	empty	De-identify customer data in this file and exit (`docs/log-sanitization.md`)
`--sanitize-out`	empty	Where `--sanitize` writes its result (default: overwrite the input)
`--version`	`false`	Print version and exit
`--no-browser`	`false`	Do not auto-open browser
`--hold-on-crash`	`true` on Windows	Keep console open after fatal crash

Common commands

make build
make build-all
make test
make fmt
make update-pci-ids

Notes:

make build outputs bin/logpile
make build-all builds the supported cross-platform binaries
make build and make build-all run scripts/update-pci-ids.sh --best-effort unless SKIP_PCI_IDS_UPDATE=1

PCI IDs

Source submodule: third_party/pciids Embedded copy: internal/parser/vendors/pciids/pci.ids

Typical setup after clone:

git submodule update --init third_party/pciids

Release script

Run:

./scripts/release.sh

Current behavior:

Reads version from git describe --tags
Refuses a dirty tree unless ALLOW_DIRTY=1
Sets stable Go cache/toolchain environment
Creates releases/{VERSION}/
Creates a release-notes template if missing
Builds darwin-arm64 and windows-amd64
Packages any already-present binaries from bin/
Generates SHA256SUMS.txt

Release tag format:

project release tags use vN.M
do not create vN.M.P tags for LOGPile releases
release artifacts and main.version inherit the exact git tag string

Important limitation:

scripts/release.sh does not run make build-all for you
if you want Linux or additional macOS archives in the release directory, build them before running the script

Toolchain note:

scripts/release.sh defaults GOTOOLCHAIN=local to use the already installed Go toolchain and avoid implicit network downloads during release builds
if you intentionally want another toolchain, pass it explicitly, for example GOTOOLCHAIN=go1.24.0 ./scripts/release.sh

Run locally

./bin/logpile
./bin/logpile --port 9090
./bin/logpile --no-browser
./bin/logpile --version

macOS Gatekeeper

xattr -d com.apple.quarantine /path/to/logpile-darwin-arm64

2.4 KiB Raw Blame History