docs: consolidate architecture docs into bible

This commit is contained in:
2026-02-23 17:51:25 +03:00
parent fcd57c1ba9
commit 38cc051f23
11 changed files with 161 additions and 123 deletions
+1 -11
View File
@@ -1,11 +1 @@
# LOGPile — Instructions for Claude Read and follow [`docs/bible/README.md`](docs/bible/README.md) as the single source of truth, and do not contradict the Bible.
Read and follow the project Bible before making any changes:
**[`docs/bible/README.md`](docs/bible/README.md)**
The Bible is the single source of truth for architecture, data models, API contracts,
parsers, exporters, build process, and testing expectations.
Every significant architectural decision must be recorded in
[`docs/bible/10-decisions.md`](docs/bible/10-decisions.md).
+5 -62
View File
@@ -1,68 +1,11 @@
# LOGPile # LOGPile
LOGPile — standalone Go-приложение для анализа диагностических данных BMC. Standalone Go application for BMC diagnostics analysis with an embedded web UI.
Поддерживает два сценария: ## Documentation
1. Загрузка архивов/снапшотов и оффлайн-анализ в веб-интерфейсе.
2. Live-сбор через Redfish API с последующим экспортом.
## Сборка - Architecture and technical documentation (single source of truth): [`docs/bible/README.md`](docs/bible/README.md)
```bash ## License
make build # bin/logpile (текущая платформа)
make build-all # все платформы в bin/
```
Требования: Go 1.22+ MIT (see `LICENSE`)
## Запуск
```bash
./bin/logpile
./bin/logpile --port 8082
./bin/logpile --no-browser
./bin/logpile --version
```
На Windows `--hold-on-crash` включён по умолчанию (консоль не закрывается при падении).
## macOS: снятие карантина
```bash
xattr -d com.apple.quarantine /path/to/logpile-darwin-arm64
```
## Форматы загрузки
`POST /api/upload` принимает:
- архивы: `.tar`, `.tar.gz`, `.tgz`
- JSON snapshot (`AnalysisResult`)
## Live Redfish
```bash
POST /api/collect
{
"host": "bmc01.example.local",
"protocol": "redfish",
"port": 443,
"username": "admin",
"auth_type": "password",
"password": "secret",
"tls_mode": "insecure"
}
```
## Экспорт
- `GET /api/export/csv` — серийные номера
- `GET /api/export/json` — полный AnalysisResult
- `GET /api/export/reanimator` — формат Reanimator
## Архитектурная документация
→ [`docs/bible/`](docs/bible/README.md)
## Лицензия
MIT — см. `LICENSE`.
+1 -1
View File
@@ -20,7 +20,7 @@ through the same API and UI.
- Single self-contained binary with embedded HTML/JS/CSS (no static file serving required). - Single self-contained binary with embedded HTML/JS/CSS (no static file serving required).
- Vendor archive parsing: Inspur/Kaytus, Supermicro, NVIDIA HGX Field Diagnostics, - Vendor archive parsing: Inspur/Kaytus, Supermicro, NVIDIA HGX Field Diagnostics,
NVIDIA Bug Report, XigmaNAS, Generic text fallback. NVIDIA Bug Report, Unraid, XigmaNAS, Generic text fallback.
- Live Redfish collection with async progress tracking. - Live Redfish collection with async progress tracking.
- Normalized hardware inventory: CPU / RAM / Storage / GPU / PSU / NIC / PCIe / Firmware. - Normalized hardware inventory: CPU / RAM / Storage / GPU / PSU / NIC / PCIe / Firmware.
- Raw `redfish_tree` snapshot stored in `RawPayloads` for future offline re-analysis. - Raw `redfish_tree` snapshot stored in `RawPayloads` for future offline re-analysis.
+14 -8
View File
@@ -19,16 +19,21 @@ cmd/logpile/main.go # Binary entry point, CLI flag parsing
internal/ internal/
collector/ # Live data collectors collector/ # Live data collectors
registry.go # Collector registration registry.go # Collector registration
redfish/ # Redfish collector (real) redfish.go # Redfish connector (real implementation)
ipmi/ # IPMI collector (mock scaffold) ipmi_mock.go # IPMI mock connector (scaffold)
types.go # Connector request/progress contracts
parser/ # Archive parsers parser/ # Archive parsers
bmc_parser.go # Top-level parser dispatcher parser.go # BMCParser (dispatcher) + parse orchestration
archive.go # Archive extraction helpers
registry.go # Parser registry + detect/selection
interface.go # VendorParser interface
vendors/ # Vendor-specific parser modules vendors/ # Vendor-specific parser modules
vendors.go # Import-side-effect registrations vendors.go # Import-side-effect registrations
inspur/ inspur/
supermicro/ supermicro/
nvidia/ nvidia/
nvidia_bug_report/ nvidia_bug_report/
unraid/
xigmanas/ xigmanas/
generic/ generic/
pciids/ # PCI IDs lookup (embedded pci.ids) pciids/ # PCI IDs lookup (embedded pci.ids)
@@ -36,9 +41,9 @@ internal/
server.go # Server struct, route registration server.go # Server struct, route registration
handlers.go # All HTTP handler functions handlers.go # All HTTP handler functions
exporter/ # Export formatters exporter/ # Export formatters
exporter.go # CSV + JSON exporters
reanimator_models.go reanimator_models.go
reanimator_converter.go reanimator_converter.go
csv.go / json.go
models/ # Shared data contracts models/ # Shared data contracts
web/ web/
embed.go # go:embed directive embed.go # go:embed directive
@@ -55,7 +60,8 @@ The `Server` struct in `internal/server/server.go` holds:
|-------|------|-------------| |-------|------|-------------|
| `result` | `*models.AnalysisResult` | Current parsed/collected dataset | | `result` | `*models.AnalysisResult` | Current parsed/collected dataset |
| `detectedVendor` | `string` | Vendor identifier from last parse | | `detectedVendor` | `string` | Vendor identifier from last parse |
| Job manager | internal | Tracks live collect job status/logs | | `jobManager` | `*JobManager` | Tracks live collect job status/logs |
| `collectors` | `*collector.Registry` | Registered live collection connectors |
State is replaced atomically on successful upload or collect. State is replaced atomically on successful upload or collect.
On a failed/canceled collect, the previous `result` is preserved unchanged. On a failed/canceled collect, the previous `result` is preserved unchanged.
@@ -96,14 +102,14 @@ Job lifecycle states: `queued → running → success | failed | canceled`
## PCI IDs lookup ## PCI IDs lookup
Lookup order (first match wins, `LOGPILE_PCI_IDS_PATH` highest priority): Load/override order (`LOGPILE_PCI_IDS_PATH` has highest priority because it is loaded last):
1. Embedded `internal/parser/vendors/pciids/pci.ids` (compiled into binary) 1. Embedded `internal/parser/vendors/pciids/pci.ids` (base dataset compiled into binary)
2. `./pci.ids` 2. `./pci.ids`
3. `/usr/share/hwdata/pci.ids` 3. `/usr/share/hwdata/pci.ids`
4. `/usr/share/misc/pci.ids` 4. `/usr/share/misc/pci.ids`
5. `/opt/homebrew/share/pciids/pci.ids` 5. `/opt/homebrew/share/pciids/pci.ids`
6. `LOGPILE_PCI_IDS_PATH` env var (colon-separated list; overrides all above) 6. Paths from `LOGPILE_PCI_IDS_PATH` (colon-separated on Unix; later loaded, override same IDs)
This means unknown GPU/NIC model strings can be updated by refreshing `pci.ids` This means unknown GPU/NIC model strings can be updated by refreshing `pci.ids`
without any code change. without any code change.
+37 -7
View File
@@ -16,6 +16,7 @@
Upload a vendor diagnostic archive or a JSON snapshot. Upload a vendor diagnostic archive or a JSON snapshot.
**Request:** `multipart/form-data`, field name `archive`. **Request:** `multipart/form-data`, field name `archive`.
Server-side multipart limit: **100 MiB**.
Accepted inputs: Accepted inputs:
- `.tar`, `.tar.gz`, `.tgz` — vendor diagnostic archives - `.tar`, `.tar.gz`, `.tgz` — vendor diagnostic archives
@@ -30,7 +31,7 @@ Accepted inputs:
### `POST /api/collect` ### `POST /api/collect`
Start a live Redfish collection job. Start a live collection job (`redfish` or `ipmi`).
**Request body:** **Request body:**
```json ```json
@@ -45,9 +46,24 @@ Start a live Redfish collection job.
} }
``` ```
`tls_mode` values: `insecure` | `verify` Supported values:
- `protocol`: `redfish` | `ipmi`
- `auth_type`: `password` | `token`
- `tls_mode`: `strict` | `insecure`
**Response:** `202 Accepted` with `{ "id": "<job-id>" }` **Response:** `202 Accepted`
```json
{
"job_id": "job_a1b2c3d4e5f6",
"status": "queued",
"message": "Collection job accepted",
"created_at": "2026-02-23T12:00:00Z"
}
```
Validation behavior:
- `400 Bad Request` for invalid JSON
- `422 Unprocessable Entity` for semantic validation errors (missing/invalid fields)
### `GET /api/collect/{id}` ### `GET /api/collect/{id}`
@@ -56,9 +72,12 @@ Poll job status and progress log.
**Response:** **Response:**
```json ```json
{ {
"id": "...", "job_id": "job_a1b2c3d4e5f6",
"status": "running", "status": "running",
"logs": ["Connecting to BMC...", "Collecting CPUs..."] "progress": 55,
"logs": ["..."],
"created_at": "2026-02-23T12:00:00Z",
"updated_at": "2026-02-23T12:00:10Z"
} }
``` ```
@@ -78,18 +97,28 @@ Returns source metadata for the current dataset.
```json ```json
{ {
"loaded": true,
"filename": "redfish://bmc01.example.local",
"vendor": "redfish",
"source_type": "api", "source_type": "api",
"protocol": "redfish", "protocol": "redfish",
"target_host": "bmc01.example.local", "target_host": "bmc01.example.local",
"collected_at": "2026-02-10T15:30:00Z" "collected_at": "2026-02-10T15:30:00Z",
"stats": { "events": 0, "sensors": 0, "fru": 0 }
} }
``` ```
`source_type`: `archive` | `api` `source_type`: `archive` | `api`
When no dataset is loaded, response is `{ "loaded": false }`.
### `GET /api/config` ### `GET /api/config`
Returns the full canonical hardware inventory (`hardware.devices`) plus board info. Returns source metadata plus:
- `hardware.board`
- `hardware.firmware`
- canonical `hardware.devices`
- computed `specification` summary lines
### `GET /api/events` ### `GET /api/events`
@@ -139,6 +168,7 @@ Clear current in-memory dataset.
### `POST /api/shutdown` ### `POST /api/shutdown`
Gracefully shut down the server process. Gracefully shut down the server process.
This endpoint terminates the current process after responding.
--- ---
+15 -9
View File
@@ -11,25 +11,31 @@ Key top-level fields:
| Field | Type | Description | | Field | Type | Description |
|-------|------|-------------| |-------|------|-------------|
| `hardware` | `Hardware` | All normalized hardware inventory | | `filename` | `string` | Uploaded filename or generated live source identifier |
| `source_type` | `string` | `archive` or `api` |
| `protocol` | `string` | `redfish`, `ipmi`, or empty for archive uploads |
| `target_host` | `string` | BMC host for live collection |
| `collected_at` | `time.Time` | Upload/collection timestamp |
| `hardware` | `*HardwareConfig` | All normalized hardware inventory |
| `events` | `[]Event` | Diagnostic events from parsers | | `events` | `[]Event` | Diagnostic events from parsers |
| `sensors` | `[]Sensor` | Sensor readings | | `fru` | `[]FRUInfo` | FRU/SDR-derived inventory details |
| `sensors` | `[]SensorReading` | Sensor readings |
| `raw_payloads` | `map[string]any` | Raw vendor data (e.g. `redfish_tree`) | | `raw_payloads` | `map[string]any` | Raw vendor data (e.g. `redfish_tree`) |
| `source` | `SourceMeta` | Origin metadata (type, protocol, host, date) |
### Hardware sub-structure ### Hardware sub-structure
``` ```
Hardware HardwareConfig
├── board BoardInfo — server/motherboard identity ├── board BoardInfo — server/motherboard identity
├── devices []Device — CANONICAL INVENTORY (see below) ├── devices []HardwareDevice — CANONICAL INVENTORY (see below)
├── cpus []CPU ├── cpus []CPU
├── memory []MemoryDIMM ├── memory []MemoryDIMM
├── storage []Storage ├── storage []Storage
├── pcie_devices []PCIeDevice
├── gpus []GPU ├── gpus []GPU
├── psus []PSU ├── network_adapters []NetworkAdapter
├── nics []NetworkAdapter ├── network_cards []NIC (legacy/alternate source field)
├── pcie []PCIeDevice ├── power_supplies []PSU
└── firmware []FirmwareInfo └── firmware []FirmwareInfo
``` ```
@@ -65,7 +71,7 @@ This minimizes translation logic in the exporter and prevents drift.
--- ---
## Source metadata (`SourceMeta`) ## Source metadata fields (stored directly on `AnalysisResult`)
Carried by both `/api/status` and `/api/config`: Carried by both `/api/status` and `/api/config`:
+14 -1
View File
@@ -1,7 +1,12 @@
# 05 — Collectors # 05 — Collectors
Collectors live in `internal/collector/`. Collectors live in `internal/collector/`.
Registration: `internal/collector/registry.go`
Core files:
- `internal/collector/registry.go` — connector registry (`redfish`, `ipmi`)
- `internal/collector/redfish.go` — real Redfish connector
- `internal/collector/ipmi_mock.go` — IPMI mock connector scaffold
- `internal/collector/types.go` — request/progress contracts
--- ---
@@ -9,6 +14,13 @@ Registration: `internal/collector/registry.go`
**Status:** Production-ready. **Status:** Production-ready.
### Request contract (from server)
Passed through from `/api/collect` after validation:
- `host`, `port`, `username`
- `auth_type=password|token` (+ matching credential field)
- `tls_mode=strict|insecure`
### Discovery ### Discovery
Dynamic — does not assume fixed paths. Discovers: Dynamic — does not assume fixed paths. Discovers:
@@ -47,6 +59,7 @@ When adding Redfish mappings, follow these principles:
The collector emits progress log entries at each stage (connecting, enumerating systems, The collector emits progress log entries at each stage (connecting, enumerating systems,
collecting CPUs, etc.) so the UI can display meaningful status. collecting CPUs, etc.) so the UI can display meaningful status.
Current progress message strings are user-facing and may be localized.
--- ---
+20 -3
View File
@@ -21,6 +21,7 @@ import (
type VendorParser interface { type VendorParser interface {
Name() string // human-readable name Name() string // human-readable name
Vendor() string // vendor identifier string Vendor() string // vendor identifier string
Version() string // parser version (increment on logic changes)
Detect(files []ExtractedFile) int // confidence 0100 Detect(files []ExtractedFile) int // confidence 0100
Parse(files []ExtractedFile) (*models.AnalysisResult, error) Parse(files []ExtractedFile) (*models.AnalysisResult, error)
} }
@@ -195,6 +196,24 @@ inspur/
--- ---
### Unraid (`unraid`)
**Status:** Ready (v1.0.0).
**Archive format:** Unraid diagnostics archive contents (text-heavy diagnostics directories).
**Detection:** Combines filename/path markers (`diagnostics-*`, `unraid-*.txt`, `vars.txt`)
with content markers (e.g. `Unraid kernel build`, parity data markers).
**Extracted data (current):**
- Board / BIOS metadata (from motherboard/system files)
- CPU summary (from `lscpu.txt`)
- Memory modules (from diagnostics memory file)
- Storage devices (from `vars.txt` + SMART files)
- Syslog events
---
### Generic text fallback (`generic`) ### Generic text fallback (`generic`)
**Status:** Ready (v1.0.0). **Status:** Ready (v1.0.0).
@@ -217,8 +236,6 @@ inspur/
| Supermicro | `supermicro` | Ready | SYS-821GE-TNHR crashdump | | Supermicro | `supermicro` | Ready | SYS-821GE-TNHR crashdump |
| NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers | | NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers |
| NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems | | NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems |
| Unraid | `unraid` | Ready | Unraid diagnostics archives |
| XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs | | XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs |
| Generic fallback | `generic` | Ready | Any text file | | Generic fallback | `generic` | Ready | Any text file |
| Dell iDRAC | `dell` | Planned | — |
| HPE iLO | `hpe` | Planned | — |
| Lenovo XCC | `lenovo` | Planned | — |
+2 -3
View File
@@ -67,10 +67,9 @@ What it does:
6. Builds `darwin-arm64` and `windows-amd64` binaries 6. Builds `darwin-arm64` and `windows-amd64` binaries
7. Packages all binaries found in `bin/` as `.tar.gz` / `.zip` 7. Packages all binaries found in `bin/` as `.tar.gz` / `.zip`
8. Generates `SHA256SUMS.txt` 8. Generates `SHA256SUMS.txt`
9. Prints next steps (tag, push, create release on git.mchus.pro) 9. Prints next steps (tag, push, create release manually)
Release notes live in `docs/releases/<tag>.md`. Release notes template is created in `releases/{VERSION}/RELEASE_NOTES.md`.
Tags and releases are published with `tea`.
## Running ## Running
+16 -3
View File
@@ -89,13 +89,26 @@ Exporter groups/filters canonical records by section; does not rebuild from sub-
**Date:** 2026-02-20 **Date:** 2026-02-20
**Context:** Codebase documentation was mixed Russian/English, reducing clarity for **Context:** Codebase documentation was mixed Russian/English, reducing clarity for
international contributors and AI assistants. international contributors and AI assistants.
**Decision:** All documentation in `docs/bible/` and new inline code documentation **Decision:** All maintained project documentation (`docs/bible/`, `README.md`,
must be written in English. Existing Russian-language user-facing README may remain `CLAUDE.md`, and new technical docs) must be written in English.
in Russian but architecture docs migrate to English.
**Consequences:** **Consequences:**
- Bible is authoritative in English. - Bible is authoritative in English.
- AI assistants get consistent, unambiguous context. - AI assistants get consistent, unambiguous context.
--- ---
## ADL-008 — Bible is the single source of truth for architecture docs
**Date:** 2026-02-23
**Context:** Architecture information was duplicated across `README.md`, `CLAUDE.md`,
and the Bible, creating drift risk and stale guidance for humans and AI agents.
**Decision:** Keep architecture and technical design documentation only in `docs/bible/`.
Top-level `README.md` and `CLAUDE.md` must remain minimal pointers/instructions.
**Consequences:**
- Reduces documentation drift and duplicate updates.
- AI assistants are directed to one authoritative source before making changes.
- Documentation updates that affect architecture must include Bible changes (and ADL entries when significant).
---
<!-- Add new decisions below this line using the format above --> <!-- Add new decisions below this line using the format above -->
+35 -14
View File
@@ -1,34 +1,55 @@
# LOGPile Bible # LOGPile Bible
> **Documentation language:** English only. All new documentation must be written in English. > **Documentation language:** English only. All maintained project documentation must be written in English.
> >
> **Architectural decisions:** Every significant architectural decision **must** be recorded in > **Architectural decisions:** Every significant architectural decision **must** be recorded in
> [`10-decisions.md`](10-decisions.md) before or alongside the code change. > [`10-decisions.md`](10-decisions.md) before or alongside the code change.
>
> **Single source of truth:** Architecture and technical design documentation belongs in `docs/bible/`.
> Keep `README.md` and `CLAUDE.md` minimal to avoid duplicate documentation.
This directory is the single source of truth for LOGPile's architecture, design, and integration contracts. This directory is the single source of truth for LOGPile's architecture, design, and integration contracts.
It is structured so that both humans and AI assistants can navigate it quickly. It is structured so that both humans and AI assistants can navigate it quickly.
--- ---
## Contents ## Reading Map (Hierarchical)
| # | File | What it covers | ### 1. Foundations (read first)
|---|------|----------------|
| 01 | [overview.md](01-overview.md) | Product goals, operating modes, high-level concept | | File | What it covers |
| 02 | [architecture.md](02-architecture.md) | Runtime structure, key flows, in-memory state | |------|----------------|
| 03 | [api.md](03-api.md) | All HTTP endpoints — contracts, request/response shapes | | [01-overview.md](01-overview.md) | Product purpose, operating modes, scope |
| 04 | [data-models.md](04-data-models.md) | `AnalysisResult`, canonical `hardware.devices` repository | | [02-architecture.md](02-architecture.md) | Runtime structure, control flow, in-memory state |
| 05 | [collectors.md](05-collectors.md) | Live data collectors (Redfish, IPMI scaffold) | | [04-data-models.md](04-data-models.md) | Core contracts (`AnalysisResult`, canonical `hardware.devices`) |
| 06 | [parsers.md](06-parsers.md) | Archive parser framework + all vendor parsers |
| 07 | [exporters.md](07-exporters.md) | CSV / JSON / Reanimator export + full Reanimator integration spec | ### 2. Runtime Interfaces
| 08 | [build-release.md](08-build-release.md) | Build system, CLI flags, release process |
| 09 | [testing.md](09-testing.md) | Testing expectations and guidelines | | File | What it covers |
| 10 | [decisions.md](10-decisions.md) | Architectural decision log (ADL) | |------|----------------|
| [03-api.md](03-api.md) | HTTP API contracts and endpoint behavior |
| [05-collectors.md](05-collectors.md) | Live collection connectors (Redfish, IPMI mock) |
| [06-parsers.md](06-parsers.md) | Archive parser framework and vendor parsers |
| [07-exporters.md](07-exporters.md) | CSV / JSON / Reanimator exports and integration mapping |
### 3. Delivery & Quality
| File | What it covers |
|------|----------------|
| [08-build-release.md](08-build-release.md) | Build, packaging, release workflow |
| [09-testing.md](09-testing.md) | Testing expectations and verification guidance |
### 4. Governance (always current)
| File | What it covers |
|------|----------------|
| [10-decisions.md](10-decisions.md) | Architectural Decision Log (ADL) |
--- ---
## Quick orientation for AI assistants ## Quick orientation for AI assistants
- Read order for most changes: `01``02``04` → relevant interface doc(s) → `10`
- Entry point: `cmd/logpile/main.go` - Entry point: `cmd/logpile/main.go`
- HTTP server: `internal/server/` — handlers in `handlers.go`, routes in `server.go` - HTTP server: `internal/server/` — handlers in `handlers.go`, routes in `server.go`
- Data contracts: `internal/models/` — never break `AnalysisResult` JSON shape - Data contracts: `internal/models/` — never break `AnalysisResult` JSON shape