docs: consolidate architecture docs into bible

2026-02-23 17:51:25 +03:00
parent fcd57c1ba9
commit 38cc051f23
11 changed files with 161 additions and 123 deletions
--- a/docs/bible/01-overview.md
+++ b/docs/bible/01-overview.md
@@ -20,7 +20,7 @@ through the same API and UI.

 - Single self-contained binary with embedded HTML/JS/CSS (no static file serving required).
 - Vendor archive parsing: Inspur/Kaytus, Supermicro, NVIDIA HGX Field Diagnostics,
-  NVIDIA Bug Report, XigmaNAS, Generic text fallback.
+  NVIDIA Bug Report, Unraid, XigmaNAS, Generic text fallback.
 - Live Redfish collection with async progress tracking.
 - Normalized hardware inventory: CPU / RAM / Storage / GPU / PSU / NIC / PCIe / Firmware.
 - Raw `redfish_tree` snapshot stored in `RawPayloads` for future offline re-analysis.
--- a/docs/bible/02-architecture.md
+++ b/docs/bible/02-architecture.md
@@ -19,16 +19,21 @@ cmd/logpile/main.go          # Binary entry point, CLI flag parsing
 internal/
  collector/                 # Live data collectors
    registry.go              # Collector registration
-    redfish/                 # Redfish collector (real)
-    ipmi/                    # IPMI collector (mock scaffold)
+    redfish.go               # Redfish connector (real implementation)
+    ipmi_mock.go             # IPMI mock connector (scaffold)
+    types.go                 # Connector request/progress contracts
  parser/                    # Archive parsers
-    bmc_parser.go            # Top-level parser dispatcher
+    parser.go                # BMCParser (dispatcher) + parse orchestration
+    archive.go               # Archive extraction helpers
+    registry.go              # Parser registry + detect/selection
+    interface.go             # VendorParser interface
    vendors/                 # Vendor-specific parser modules
      vendors.go             # Import-side-effect registrations
      inspur/
      supermicro/
      nvidia/
      nvidia_bug_report/
+      unraid/
      xigmanas/
      generic/
      pciids/                # PCI IDs lookup (embedded pci.ids)
@@ -36,9 +41,9 @@ internal/
    server.go                # Server struct, route registration
    handlers.go              # All HTTP handler functions
  exporter/                  # Export formatters
+    exporter.go              # CSV + JSON exporters
    reanimator_models.go
    reanimator_converter.go
-    csv.go / json.go
  models/                    # Shared data contracts
 web/
  embed.go                   # go:embed directive
@@ -55,7 +60,8 @@ The `Server` struct in `internal/server/server.go` holds:
 |-------|------|-------------|
 | `result` | `*models.AnalysisResult` | Current parsed/collected dataset |
 | `detectedVendor` | `string` | Vendor identifier from last parse |
-| Job manager | internal | Tracks live collect job status/logs |
+| `jobManager` | `*JobManager` | Tracks live collect job status/logs |
+| `collectors` | `*collector.Registry` | Registered live collection connectors |

 State is replaced atomically on successful upload or collect.
 On a failed/canceled collect, the previous `result` is preserved unchanged.
@@ -96,14 +102,14 @@ Job lifecycle states: `queued → running → success | failed | canceled`

 ## PCI IDs lookup

-Lookup order (first match wins, `LOGPILE_PCI_IDS_PATH` highest priority):
+Load/override order (`LOGPILE_PCI_IDS_PATH` has highest priority because it is loaded last):

-1. Embedded `internal/parser/vendors/pciids/pci.ids` (compiled into binary)
+1. Embedded `internal/parser/vendors/pciids/pci.ids` (base dataset compiled into binary)
 2. `./pci.ids`
 3. `/usr/share/hwdata/pci.ids`
 4. `/usr/share/misc/pci.ids`
 5. `/opt/homebrew/share/pciids/pci.ids`
-6. `LOGPILE_PCI_IDS_PATH` env var (colon-separated list; overrides all above)
+6. Paths from `LOGPILE_PCI_IDS_PATH` (colon-separated on Unix; later loaded, override same IDs)

 This means unknown GPU/NIC model strings can be updated by refreshing `pci.ids`
 without any code change.
--- a/docs/bible/03-api.md
+++ b/docs/bible/03-api.md
@@ -16,6 +16,7 @@
 Upload a vendor diagnostic archive or a JSON snapshot.

 **Request:** `multipart/form-data`, field name `archive`.
+Server-side multipart limit: **100 MiB**.

 Accepted inputs:
 - `.tar`, `.tar.gz`, `.tgz` — vendor diagnostic archives
@@ -30,7 +31,7 @@ Accepted inputs:

 ### `POST /api/collect`

-Start a live Redfish collection job.
+Start a live collection job (`redfish` or `ipmi`).

 **Request body:**
 ```json
@@ -45,9 +46,24 @@ Start a live Redfish collection job.
 }
 ```

-`tls_mode` values: `insecure` | `verify`
+Supported values:
+- `protocol`: `redfish` | `ipmi`
+- `auth_type`: `password` | `token`
+- `tls_mode`: `strict` | `insecure`

-**Response:** `202 Accepted` with `{ "id": "<job-id>" }`
+**Response:** `202 Accepted`
+```json
+{
+  "job_id": "job_a1b2c3d4e5f6",
+  "status": "queued",
+  "message": "Collection job accepted",
+  "created_at": "2026-02-23T12:00:00Z"
+}
+```
+
+Validation behavior:
+- `400 Bad Request` for invalid JSON
+- `422 Unprocessable Entity` for semantic validation errors (missing/invalid fields)

 ### `GET /api/collect/{id}`

@@ -56,9 +72,12 @@ Poll job status and progress log.
 **Response:**
 ```json
 {
-  "id": "...",
+  "job_id": "job_a1b2c3d4e5f6",
  "status": "running",
-  "logs": ["Connecting to BMC...", "Collecting CPUs..."]
+  "progress": 55,
+  "logs": ["..."],
+  "created_at": "2026-02-23T12:00:00Z",
+  "updated_at": "2026-02-23T12:00:10Z"
 }
 ```

@@ -78,18 +97,28 @@ Returns source metadata for the current dataset.

 ```json
 {
+  "loaded": true,
+  "filename": "redfish://bmc01.example.local",
+  "vendor": "redfish",
  "source_type": "api",
  "protocol": "redfish",
  "target_host": "bmc01.example.local",
-  "collected_at": "2026-02-10T15:30:00Z"
+  "collected_at": "2026-02-10T15:30:00Z",
+  "stats": { "events": 0, "sensors": 0, "fru": 0 }
 }
 ```

 `source_type`: `archive` | `api`

+When no dataset is loaded, response is `{ "loaded": false }`.
+
 ### `GET /api/config`

-Returns the full canonical hardware inventory (`hardware.devices`) plus board info.
+Returns source metadata plus:
+- `hardware.board`
+- `hardware.firmware`
+- canonical `hardware.devices`
+- computed `specification` summary lines

 ### `GET /api/events`

@@ -139,6 +168,7 @@ Clear current in-memory dataset.
 ### `POST /api/shutdown`

 Gracefully shut down the server process.
+This endpoint terminates the current process after responding.

 ---

--- a/docs/bible/04-data-models.md
+++ b/docs/bible/04-data-models.md
@@ -11,25 +11,31 @@ Key top-level fields:

 | Field | Type | Description |
 |-------|------|-------------|
-| `hardware` | `Hardware` | All normalized hardware inventory |
+| `filename` | `string` | Uploaded filename or generated live source identifier |
+| `source_type` | `string` | `archive` or `api` |
+| `protocol` | `string` | `redfish`, `ipmi`, or empty for archive uploads |
+| `target_host` | `string` | BMC host for live collection |
+| `collected_at` | `time.Time` | Upload/collection timestamp |
+| `hardware` | `*HardwareConfig` | All normalized hardware inventory |
 | `events` | `[]Event` | Diagnostic events from parsers |
-| `sensors` | `[]Sensor` | Sensor readings |
+| `fru` | `[]FRUInfo` | FRU/SDR-derived inventory details |
+| `sensors` | `[]SensorReading` | Sensor readings |
 | `raw_payloads` | `map[string]any` | Raw vendor data (e.g. `redfish_tree`) |
-| `source` | `SourceMeta` | Origin metadata (type, protocol, host, date) |

 ### Hardware sub-structure

 ```
-Hardware
+HardwareConfig
  ├── board        BoardInfo       — server/motherboard identity
-  ├── devices      []Device        — CANONICAL INVENTORY (see below)
+  ├── devices      []HardwareDevice — CANONICAL INVENTORY (see below)
  ├── cpus         []CPU
  ├── memory       []MemoryDIMM
  ├── storage      []Storage
+  ├── pcie_devices []PCIeDevice
  ├── gpus         []GPU
-  ├── psus         []PSU
-  ├── nics         []NetworkAdapter
-  ├── pcie         []PCIeDevice
+  ├── network_adapters []NetworkAdapter
+  ├── network_cards    []NIC           (legacy/alternate source field)
+  ├── power_supplies   []PSU
  └── firmware     []FirmwareInfo
 ```

@@ -65,7 +71,7 @@ This minimizes translation logic in the exporter and prevents drift.

 ---

-## Source metadata (`SourceMeta`)
+## Source metadata fields (stored directly on `AnalysisResult`)

 Carried by both `/api/status` and `/api/config`:

--- a/docs/bible/05-collectors.md
+++ b/docs/bible/05-collectors.md
@@ -1,7 +1,12 @@
 # 05 — Collectors

 Collectors live in `internal/collector/`.
-Registration: `internal/collector/registry.go`
+
+Core files:
+- `internal/collector/registry.go` — connector registry (`redfish`, `ipmi`)
+- `internal/collector/redfish.go` — real Redfish connector
+- `internal/collector/ipmi_mock.go` — IPMI mock connector scaffold
+- `internal/collector/types.go` — request/progress contracts

 ---

@@ -9,6 +14,13 @@ Registration: `internal/collector/registry.go`

 **Status:** Production-ready.

+### Request contract (from server)
+
+Passed through from `/api/collect` after validation:
+- `host`, `port`, `username`
+- `auth_type=password|token` (+ matching credential field)
+- `tls_mode=strict|insecure`
+
 ### Discovery

 Dynamic — does not assume fixed paths. Discovers:
@@ -47,6 +59,7 @@ When adding Redfish mappings, follow these principles:

 The collector emits progress log entries at each stage (connecting, enumerating systems,
 collecting CPUs, etc.) so the UI can display meaningful status.
+Current progress message strings are user-facing and may be localized.

 ---

--- a/docs/bible/06-parsers.md
+++ b/docs/bible/06-parsers.md
@@ -19,8 +19,9 @@ import (

 ```go
 type VendorParser interface {
-    Name()   string                              // human-readable name
+    Name() string                                // human-readable name
    Vendor() string                              // vendor identifier string
+    Version() string                             // parser version (increment on logic changes)
    Detect(files []ExtractedFile) int            // confidence 0–100
    Parse(files []ExtractedFile) (*models.AnalysisResult, error)
 }
@@ -195,6 +196,24 @@ inspur/

 ---

+### Unraid (`unraid`)
+
+**Status:** Ready (v1.0.0).
+
+**Archive format:** Unraid diagnostics archive contents (text-heavy diagnostics directories).
+
+**Detection:** Combines filename/path markers (`diagnostics-*`, `unraid-*.txt`, `vars.txt`)
+with content markers (e.g. `Unraid kernel build`, parity data markers).
+
+**Extracted data (current):**
+- Board / BIOS metadata (from motherboard/system files)
+- CPU summary (from `lscpu.txt`)
+- Memory modules (from diagnostics memory file)
+- Storage devices (from `vars.txt` + SMART files)
+- Syslog events
+
+---
+
 ### Generic text fallback (`generic`)

 **Status:** Ready (v1.0.0).
@@ -217,8 +236,6 @@ inspur/
 | Supermicro | `supermicro` | Ready | SYS-821GE-TNHR crashdump |
 | NVIDIA HGX Field Diag | `nvidia` | Ready | Various HGX servers |
 | NVIDIA Bug Report | `nvidia_bug_report` | Ready | H100 systems |
+| Unraid | `unraid` | Ready | Unraid diagnostics archives |
 | XigmaNAS | `xigmanas` | Ready | FreeBSD NAS logs |
 | Generic fallback | `generic` | Ready | Any text file |
-| Dell iDRAC | `dell` | Planned | — |
-| HPE iLO | `hpe` | Planned | — |
-| Lenovo XCC | `lenovo` | Planned | — |
--- a/docs/bible/08-build-release.md
+++ b/docs/bible/08-build-release.md
@@ -67,10 +67,9 @@ What it does:
 6. Builds `darwin-arm64` and `windows-amd64` binaries
 7. Packages all binaries found in `bin/` as `.tar.gz` / `.zip`
 8. Generates `SHA256SUMS.txt`
-9. Prints next steps (tag, push, create release on git.mchus.pro)
+9. Prints next steps (tag, push, create release manually)

-Release notes live in `docs/releases/<tag>.md`.
-Tags and releases are published with `tea`.
+Release notes template is created in `releases/{VERSION}/RELEASE_NOTES.md`.

 ## Running

--- a/docs/bible/10-decisions.md
+++ b/docs/bible/10-decisions.md
@@ -89,13 +89,26 @@ Exporter groups/filters canonical records by section; does not rebuild from sub-
 **Date:** 2026-02-20
 **Context:** Codebase documentation was mixed Russian/English, reducing clarity for
 international contributors and AI assistants.
-**Decision:** All documentation in `docs/bible/` and new inline code documentation
-must be written in English. Existing Russian-language user-facing README may remain
-in Russian but architecture docs migrate to English.
+**Decision:** All maintained project documentation (`docs/bible/`, `README.md`,
+`CLAUDE.md`, and new technical docs) must be written in English.
 **Consequences:**
 - Bible is authoritative in English.
 - AI assistants get consistent, unambiguous context.

 ---

+## ADL-008 — Bible is the single source of truth for architecture docs
+
+**Date:** 2026-02-23
+**Context:** Architecture information was duplicated across `README.md`, `CLAUDE.md`,
+and the Bible, creating drift risk and stale guidance for humans and AI agents.
+**Decision:** Keep architecture and technical design documentation only in `docs/bible/`.
+Top-level `README.md` and `CLAUDE.md` must remain minimal pointers/instructions.
+**Consequences:**
+- Reduces documentation drift and duplicate updates.
+- AI assistants are directed to one authoritative source before making changes.
+- Documentation updates that affect architecture must include Bible changes (and ADL entries when significant).
+
+---
+
 <!-- Add new decisions below this line using the format above -->
--- a/docs/bible/README.md
+++ b/docs/bible/README.md
@@ -1,34 +1,55 @@
 # LOGPile Bible

-> **Documentation language:** English only. All new documentation must be written in English.
+> **Documentation language:** English only. All maintained project documentation must be written in English.
 >
 > **Architectural decisions:** Every significant architectural decision **must** be recorded in
 > [`10-decisions.md`](10-decisions.md) before or alongside the code change.
+>
+> **Single source of truth:** Architecture and technical design documentation belongs in `docs/bible/`.
+> Keep `README.md` and `CLAUDE.md` minimal to avoid duplicate documentation.

 This directory is the single source of truth for LOGPile's architecture, design, and integration contracts.
 It is structured so that both humans and AI assistants can navigate it quickly.

 ---

-## Contents
+## Reading Map (Hierarchical)

-| # | File | What it covers |
-|---|------|----------------|
-| 01 | [overview.md](01-overview.md) | Product goals, operating modes, high-level concept |
-| 02 | [architecture.md](02-architecture.md) | Runtime structure, key flows, in-memory state |
-| 03 | [api.md](03-api.md) | All HTTP endpoints — contracts, request/response shapes |
-| 04 | [data-models.md](04-data-models.md) | `AnalysisResult`, canonical `hardware.devices` repository |
-| 05 | [collectors.md](05-collectors.md) | Live data collectors (Redfish, IPMI scaffold) |
-| 06 | [parsers.md](06-parsers.md) | Archive parser framework + all vendor parsers |
-| 07 | [exporters.md](07-exporters.md) | CSV / JSON / Reanimator export + full Reanimator integration spec |
-| 08 | [build-release.md](08-build-release.md) | Build system, CLI flags, release process |
-| 09 | [testing.md](09-testing.md) | Testing expectations and guidelines |
-| 10 | [decisions.md](10-decisions.md) | Architectural decision log (ADL) |
+### 1. Foundations (read first)
+
+| File | What it covers |
+|------|----------------|
+| [01-overview.md](01-overview.md) | Product purpose, operating modes, scope |
+| [02-architecture.md](02-architecture.md) | Runtime structure, control flow, in-memory state |
+| [04-data-models.md](04-data-models.md) | Core contracts (`AnalysisResult`, canonical `hardware.devices`) |
+
+### 2. Runtime Interfaces
+
+| File | What it covers |
+|------|----------------|
+| [03-api.md](03-api.md) | HTTP API contracts and endpoint behavior |
+| [05-collectors.md](05-collectors.md) | Live collection connectors (Redfish, IPMI mock) |
+| [06-parsers.md](06-parsers.md) | Archive parser framework and vendor parsers |
+| [07-exporters.md](07-exporters.md) | CSV / JSON / Reanimator exports and integration mapping |
+
+### 3. Delivery & Quality
+
+| File | What it covers |
+|------|----------------|
+| [08-build-release.md](08-build-release.md) | Build, packaging, release workflow |
+| [09-testing.md](09-testing.md) | Testing expectations and verification guidance |
+
+### 4. Governance (always current)
+
+| File | What it covers |
+|------|----------------|
+| [10-decisions.md](10-decisions.md) | Architectural Decision Log (ADL) |

 ---

 ## Quick orientation for AI assistants

+- Read order for most changes: `01` → `02` → `04` → relevant interface doc(s) → `10`
 - Entry point: `cmd/logpile/main.go`
 - HTTP server: `internal/server/` — handlers in `handlers.go`, routes in `server.go`
 - Data contracts: `internal/models/` — never break `AnalysisResult` JSON shape