docs: introduce project Bible and consolidate all architecture documentation
- Create docs/bible/ with 10 structured chapters (overview, architecture, API, data models, collectors, parsers, exporters, build, testing, decisions) - All documentation in English per ADL-007 - Record all existing architectural decisions in docs/bible/10-decisions.md - Slim README.md to user-facing quick start only - Replace CLAUDE.md with a single directive to read and follow the Bible - Remove absorbed files: REANIMATOR_EXPORT.md, docs/INTEGRATION_GUIDE.md, and all vendor parser README.md files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Mikhail Chusavitin
co-authored by
Claude Sonnet 4.6
Claude Sonnet 4.6
parent
82ee513835
commit
fcd57c1ba9
@@ -1,150 +1,11 @@
|
||||
# LOGPile - Engineering Notes (for Claude/Codex)
|
||||
# LOGPile — Instructions for Claude
|
||||
|
||||
## Project summary
|
||||
Read and follow the project Bible before making any changes:
|
||||
|
||||
LOGPile is a standalone Go app for BMC diagnostics analysis with embedded web UI.
|
||||
**[`docs/bible/README.md`](docs/bible/README.md)**
|
||||
|
||||
Current product modes:
|
||||
1. Upload and parse vendor archives / JSON snapshots.
|
||||
2. Collect live data via Redfish and analyze/export it.
|
||||
The Bible is the single source of truth for architecture, data models, API contracts,
|
||||
parsers, exporters, build process, and testing expectations.
|
||||
|
||||
## Runtime architecture
|
||||
|
||||
- Go + `net/http` (`http.ServeMux`)
|
||||
- Embedded UI (`web/embed.go`, `//go:embed templates static`)
|
||||
- In-memory state (`Server.result`, `Server.detectedVendor`)
|
||||
- Job manager for live collect status/logs
|
||||
|
||||
Default port: `8082`.
|
||||
|
||||
## Key flows
|
||||
|
||||
### Upload flow (`POST /api/upload`)
|
||||
- Accepts multipart file field `archive`.
|
||||
- If file looks like JSON, parsed as `models.AnalysisResult` snapshot.
|
||||
- Otherwise passed to archive parser (`parser.NewBMCParser().ParseFromReader(...)`).
|
||||
- Result stored in memory and exposed by API/UI.
|
||||
|
||||
### Live flow (`POST /api/collect`)
|
||||
- Validates request (`host/protocol/port/username/auth_type/tls_mode`).
|
||||
- Runs collector asynchronously with progress callback.
|
||||
- On success:
|
||||
- source metadata set (`source_type=api`, protocol/host/date),
|
||||
- result becomes current in-memory dataset.
|
||||
- On failed/canceled previous dataset stays unchanged.
|
||||
|
||||
## Collectors
|
||||
|
||||
Registry: `internal/collector/registry.go`
|
||||
|
||||
- `redfish` (real collector):
|
||||
- dynamic discovery of Systems/Chassis/Managers,
|
||||
- CPU/RAM/Storage/GPU/PSU/NIC/PCIe/Firmware mapping,
|
||||
- raw Redfish snapshot (`result.RawPayloads["redfish_tree"]`) for offline future analysis,
|
||||
- progress logs include active collection stage and snapshot progress.
|
||||
- `ipmi` is currently a mock collector scaffold.
|
||||
|
||||
## Inspur/Kaytus parser notes
|
||||
|
||||
- Base hardware inventory comes from `asset.json` + `component.log` + `devicefrusdr.log`.
|
||||
- Additional runtime enrichment is applied from `redis-dump.rdb` (if present):
|
||||
- GPU serial/firmware/UUID and selected runtime metrics;
|
||||
- NIC firmware/serial/part fields where text logs are incomplete.
|
||||
- GPU/NIC enrichment from Redis is conservative (fills missing fields, avoids unsafe remapping).
|
||||
|
||||
### External PCI IDs lookup (no hardcoded model mapping)
|
||||
|
||||
`internal/parser/vendors/pciids` now loads IDs from a repo file
|
||||
(`internal/parser/vendors/pciids/pci.ids`, embedded at build time) plus optional external overrides.
|
||||
|
||||
Lookup priority:
|
||||
1. embedded `internal/parser/vendors/pciids/pci.ids`,
|
||||
2. `./pci.ids`,
|
||||
3. `/usr/share/hwdata/pci.ids`,
|
||||
4. `/usr/share/misc/pci.ids`,
|
||||
5. `/opt/homebrew/share/pciids/pci.ids`,
|
||||
6. `LOGPILE_PCI_IDS_PATH` (highest priority overrides; supports path list).
|
||||
|
||||
Implication:
|
||||
- for unknown device IDs (e.g. new NVIDIA GPU IDs), model naming can be updated via `pci.ids`
|
||||
without changing parser code.
|
||||
|
||||
## Export behavior
|
||||
|
||||
Endpoints:
|
||||
- `/api/export/csv`
|
||||
- `/api/export/json`
|
||||
- `/api/export/reanimator`
|
||||
|
||||
Filename pattern for all exports:
|
||||
`YYYY-MM-DD (SERVER MODEL) - SERVER SN.<ext>`
|
||||
|
||||
Notes:
|
||||
- JSON export contains full `AnalysisResult`, including `raw_payloads`.
|
||||
- **Reanimator export** (`/api/export/reanimator`):
|
||||
- Exports hardware data in Reanimator format for integration with asset tracking systems.
|
||||
- Format specification: `example/docs/INTEGRATION_GUIDE.md`
|
||||
- Requires `hardware.board.serial_number` to be present.
|
||||
- Key features:
|
||||
- Infers CPU manufacturer from model name (Intel/AMD/ARM/Ampere).
|
||||
- Generates PCIe serial numbers if missing: `{board_serial}-PCIE-{slot}`.
|
||||
- Adds status fields (defaults to "OK").
|
||||
- RFC3339 timestamp format.
|
||||
- Includes GPUs and NetworkAdapters as PCIe devices.
|
||||
- Filters out storage devices and PSUs without serial numbers.
|
||||
|
||||
## Canonical device repository (AI memory)
|
||||
|
||||
Single source of truth for hardware inventory is `hardware.devices`.
|
||||
|
||||
Rules:
|
||||
- UI tabs must read hardware records from `hardware.devices`.
|
||||
- Device Inventory tab includes kinds: `pcie`, `storage`, `gpu`, `network`.
|
||||
- Reanimator export must use the same canonical repository (`hardware.devices`).
|
||||
- Any UI vs Reanimator mismatch is a bug.
|
||||
|
||||
Canonical dedupe (applied once in repository builder):
|
||||
1. usable `serial_number`,
|
||||
2. fallback `bdf` (PCI ID in BDF format),
|
||||
3. if both are absent, keep records distinct (no forced merge).
|
||||
|
||||
Implementation guidance:
|
||||
- Keep `hardware.devices` schema as close as possible to Reanimator JSON fields.
|
||||
- Exporter should mainly group/filter canonical records by section, not rebuild data from multiple sources.
|
||||
- New hardware attributes must be added to canonical device schema first, then mapped to Reanimator/UI.
|
||||
|
||||
## CLI flags (`cmd/logpile/main.go`)
|
||||
|
||||
- `--port`
|
||||
- `--file` (reserved/preload, not active workflow)
|
||||
- `--version`
|
||||
- `--no-browser`
|
||||
- `--hold-on-crash` (default true on Windows) — keeps console open on fatal crash for debugging.
|
||||
|
||||
## Build / release
|
||||
|
||||
- `make build` -> single local binary (`CGO_ENABLED=0`).
|
||||
- `make build-all` -> cross-platform binaries.
|
||||
- Tags/releases are published with `tea`.
|
||||
- Release notes live in `docs/releases/<tag>.md`.
|
||||
|
||||
## Testing expectations
|
||||
|
||||
Before merge:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
If touching collectors/handlers, prefer adding or updating tests in:
|
||||
- `internal/collector/*_test.go`
|
||||
- `internal/server/*_test.go`
|
||||
|
||||
## Practical coding guidance
|
||||
|
||||
- Keep API contracts stable with frontend (`web/static/js/app.js`).
|
||||
- When adding Redfish mappings, prefer tolerant/fallback parsing:
|
||||
- alternate collection paths,
|
||||
- `@odata.id` references and embedded members,
|
||||
- deduping by serial/BDF/slot+model.
|
||||
- Avoid breaking snapshot backward compatibility (`AnalysisResult` JSON shape).
|
||||
Every significant architectural decision must be recorded in
|
||||
[`docs/bible/10-decisions.md`](docs/bible/10-decisions.md).
|
||||
|
||||
Reference in New Issue
Block a user