131 lines
4.4 KiB
Markdown
131 lines
4.4 KiB
Markdown
# LOGPile - Engineering Notes (for Claude/Codex)
|
|
|
|
## Project summary
|
|
|
|
LOGPile is a standalone Go app for BMC diagnostics analysis with embedded web UI.
|
|
|
|
Current product modes:
|
|
1. Upload and parse vendor archives / JSON snapshots.
|
|
2. Collect live data via Redfish and analyze/export it.
|
|
|
|
## Runtime architecture
|
|
|
|
- Go + `net/http` (`http.ServeMux`)
|
|
- Embedded UI (`web/embed.go`, `//go:embed templates static`)
|
|
- In-memory state (`Server.result`, `Server.detectedVendor`)
|
|
- Job manager for live collect status/logs
|
|
|
|
Default port: `8082`.
|
|
|
|
## Key flows
|
|
|
|
### Upload flow (`POST /api/upload`)
|
|
- Accepts multipart file field `archive`.
|
|
- If file looks like JSON, parsed as `models.AnalysisResult` snapshot.
|
|
- Otherwise passed to archive parser (`parser.NewBMCParser().ParseFromReader(...)`).
|
|
- Result stored in memory and exposed by API/UI.
|
|
|
|
### Live flow (`POST /api/collect`)
|
|
- Validates request (`host/protocol/port/username/auth_type/tls_mode`).
|
|
- Runs collector asynchronously with progress callback.
|
|
- On success:
|
|
- source metadata set (`source_type=api`, protocol/host/date),
|
|
- result becomes current in-memory dataset.
|
|
- On failed/canceled previous dataset stays unchanged.
|
|
|
|
## Collectors
|
|
|
|
Registry: `internal/collector/registry.go`
|
|
|
|
- `redfish` (real collector):
|
|
- dynamic discovery of Systems/Chassis/Managers,
|
|
- CPU/RAM/Storage/GPU/PSU/NIC/PCIe/Firmware mapping,
|
|
- raw Redfish snapshot (`result.RawPayloads["redfish_tree"]`) for offline future analysis,
|
|
- progress logs include active collection stage and snapshot progress.
|
|
- `ipmi` is currently a mock collector scaffold.
|
|
|
|
## Inspur/Kaytus parser notes
|
|
|
|
- Base hardware inventory comes from `asset.json` + `component.log` + `devicefrusdr.log`.
|
|
- Additional runtime enrichment is applied from `redis-dump.rdb` (if present):
|
|
- GPU serial/firmware/UUID and selected runtime metrics;
|
|
- NIC firmware/serial/part fields where text logs are incomplete.
|
|
- GPU/NIC enrichment from Redis is conservative (fills missing fields, avoids unsafe remapping).
|
|
|
|
### External PCI IDs lookup (no hardcoded model mapping)
|
|
|
|
`internal/parser/vendors/pciids` now loads IDs from a repo file
|
|
(`internal/parser/vendors/pciids/pci.ids`, embedded at build time) plus optional external overrides.
|
|
|
|
Lookup priority:
|
|
1. embedded `internal/parser/vendors/pciids/pci.ids`,
|
|
2. `./pci.ids`,
|
|
3. `/usr/share/hwdata/pci.ids`,
|
|
4. `/usr/share/misc/pci.ids`,
|
|
5. `/opt/homebrew/share/pciids/pci.ids`,
|
|
6. `LOGPILE_PCI_IDS_PATH` (highest priority overrides; supports path list).
|
|
|
|
Implication:
|
|
- for unknown device IDs (e.g. new NVIDIA GPU IDs), model naming can be updated via `pci.ids`
|
|
without changing parser code.
|
|
|
|
## Export behavior
|
|
|
|
Endpoints:
|
|
- `/api/export/csv`
|
|
- `/api/export/json`
|
|
- `/api/export/reanimator`
|
|
|
|
Filename pattern for all exports:
|
|
`YYYY-MM-DD (SERVER MODEL) - SERVER SN.<ext>`
|
|
|
|
Notes:
|
|
- JSON export contains full `AnalysisResult`, including `raw_payloads`.
|
|
- **Reanimator export** (`/api/export/reanimator`):
|
|
- Exports hardware data in Reanimator format for integration with asset tracking systems.
|
|
- Format specification: `example/docs/INTEGRATION_GUIDE.md`
|
|
- Requires `hardware.board.serial_number` to be present.
|
|
- Key features:
|
|
- Infers CPU manufacturer from model name (Intel/AMD/ARM/Ampere).
|
|
- Generates PCIe serial numbers if missing: `{board_serial}-PCIE-{slot}`.
|
|
- Adds status fields (defaults to "OK").
|
|
- RFC3339 timestamp format.
|
|
- Includes GPUs and NetworkAdapters as PCIe devices.
|
|
- Filters out storage devices and PSUs without serial numbers.
|
|
|
|
## CLI flags (`cmd/logpile/main.go`)
|
|
|
|
- `--port`
|
|
- `--file` (reserved/preload, not active workflow)
|
|
- `--version`
|
|
- `--no-browser`
|
|
- `--hold-on-crash` (default true on Windows) — keeps console open on fatal crash for debugging.
|
|
|
|
## Build / release
|
|
|
|
- `make build` -> single local binary (`CGO_ENABLED=0`).
|
|
- `make build-all` -> cross-platform binaries.
|
|
- Tags/releases are published with `tea`.
|
|
- Release notes live in `docs/releases/<tag>.md`.
|
|
|
|
## Testing expectations
|
|
|
|
Before merge:
|
|
|
|
```bash
|
|
go test ./...
|
|
```
|
|
|
|
If touching collectors/handlers, prefer adding or updating tests in:
|
|
- `internal/collector/*_test.go`
|
|
- `internal/server/*_test.go`
|
|
|
|
## Practical coding guidance
|
|
|
|
- Keep API contracts stable with frontend (`web/static/js/app.js`).
|
|
- When adding Redfish mappings, prefer tolerant/fallback parsing:
|
|
- alternate collection paths,
|
|
- `@odata.id` references and embedded members,
|
|
- deduping by serial/BDF/slot+model.
|
|
- Avoid breaking snapshot backward compatibility (`AnalysisResult` JSON shape).
|