docs: refresh project documentation
This commit is contained in:
Mikhail Chusavitin
@@ -3,107 +3,69 @@
|
||||
Collectors live in `internal/collector/`.
|
||||
|
||||
Core files:
|
||||
- `internal/collector/registry.go` — connector registry (`redfish`, `ipmi`)
|
||||
- `internal/collector/redfish.go` — real Redfish connector
|
||||
- `internal/collector/ipmi_mock.go` — IPMI mock connector scaffold
|
||||
- `internal/collector/types.go` — request/progress contracts
|
||||
- `registry.go` for protocol registration
|
||||
- `redfish.go` for live collection
|
||||
- `redfish_replay.go` for replay from raw payloads
|
||||
- `ipmi_mock.go` for the placeholder IPMI implementation
|
||||
- `types.go` for request/progress contracts
|
||||
|
||||
---
|
||||
## Redfish collector
|
||||
|
||||
## Redfish Collector (`redfish`)
|
||||
Status: active production path.
|
||||
|
||||
**Status:** Production-ready.
|
||||
Request fields passed from the server:
|
||||
- `host`
|
||||
- `port`
|
||||
- `username`
|
||||
- `auth_type`
|
||||
- credential field (`password` or token)
|
||||
- `tls_mode`
|
||||
|
||||
### Request contract (from server)
|
||||
### Core rule
|
||||
|
||||
Passed through from `/api/collect` after validation:
|
||||
- `host`, `port`, `username`
|
||||
- `auth_type=password|token` (+ matching credential field)
|
||||
- `tls_mode=strict|insecure`
|
||||
Live collection and replay must stay behaviorally aligned.
|
||||
If the collector adds a fallback, probe, or normalization rule, replay must mirror it.
|
||||
|
||||
### Discovery
|
||||
### Discovery model
|
||||
|
||||
Dynamic — does not assume fixed paths. Discovers:
|
||||
- `Systems` collection → per-system resources
|
||||
- `Chassis` collection → enclosure/board data
|
||||
- `Managers` collection → BMC/firmware info
|
||||
The collector does not rely on one fixed vendor tree.
|
||||
It discovers and follows Redfish resources dynamically from root collections such as:
|
||||
- `Systems`
|
||||
- `Chassis`
|
||||
- `Managers`
|
||||
|
||||
### Collected data
|
||||
### Stored raw data
|
||||
|
||||
| Category | Notes |
|
||||
|----------|-------|
|
||||
| CPU | Model, cores, threads, socket, status |
|
||||
| Memory | DIMM slot, size, type, speed, serial, manufacturer |
|
||||
| Storage | Slot, type, model, serial, firmware, interface, status |
|
||||
| GPU | Detected via PCIe class + NVIDIA vendor ID |
|
||||
| PSU | Model, serial, wattage, firmware, telemetry (input/output power, voltage) |
|
||||
| NIC | Model, serial, port count, BDF |
|
||||
| PCIe | Slot, vendor_id, device_id, BDF, link width/speed |
|
||||
| Firmware | BIOS, BMC versions |
|
||||
Important raw payloads:
|
||||
- `raw_payloads.redfish_tree`
|
||||
- `raw_payloads.redfish_fetch_errors`
|
||||
- `raw_payloads.source_timezone` when available
|
||||
|
||||
### Raw snapshot
|
||||
### Snapshot crawler rules
|
||||
|
||||
Full Redfish response tree is stored in `result.RawPayloads["redfish_tree"]`.
|
||||
This allows future offline re-analysis without re-collecting from a live BMC.
|
||||
- bounded by `LOGPILE_REDFISH_SNAPSHOT_MAX_DOCS`
|
||||
- prioritized toward high-value inventory paths
|
||||
- tolerant of expected vendor-specific failures
|
||||
- normalizes `@odata.id` values before queueing
|
||||
|
||||
### Unified Redfish analysis pipeline (live == replay)
|
||||
### Redfish implementation guidance
|
||||
|
||||
LOGPile uses a **single Redfish analyzer path**:
|
||||
When changing collection logic:
|
||||
|
||||
1. Live collector crawls the Redfish API and builds `raw_payloads.redfish_tree`
|
||||
2. Parsed result is produced by replaying that tree through the same analyzer used by raw import
|
||||
1. Prefer alternate-path support over vendor hardcoding
|
||||
2. Keep expensive probing bounded
|
||||
3. Deduplicate by serial, then BDF, then location/model fallbacks
|
||||
4. Preserve replay determinism from saved raw payloads
|
||||
5. Add tests for both the motivating topology and a negative case
|
||||
|
||||
This guarantees that live collection and `Export Raw Data` re-open/re-analyze produce the same
|
||||
normalized output for the same `redfish_tree`.
|
||||
### Known vendor fallbacks
|
||||
|
||||
### Snapshot crawler behavior (important)
|
||||
- empty standard drive collections may trigger bounded `Disk.Bay` probing
|
||||
- `Storage.Links.Enclosures[*]` may be followed to recover physical drives
|
||||
- `PowerSubsystem/PowerSupplies` is preferred over legacy `Power` when available
|
||||
|
||||
The Redfish snapshot crawler is intentionally:
|
||||
- **bounded** (`LOGPILE_REDFISH_SNAPSHOT_MAX_DOCS`)
|
||||
- **prioritized** (PCIe, Fabrics, FirmwareInventory, Storage, PowerSubsystem, ThermalSubsystem)
|
||||
- **tolerant** (skips noisy expected failures, strips `#fragment` from `@odata.id`)
|
||||
## IPMI collector
|
||||
|
||||
Design notes:
|
||||
- Queue capacity is sized to snapshot cap to avoid worker deadlocks on large trees.
|
||||
- UI progress is coarse and human-readable; detailed per-request diagnostics are available via debug logs.
|
||||
- `LOGPILE_REDFISH_DEBUG=1` and `LOGPILE_REDFISH_SNAPSHOT_DEBUG=1` enable console diagnostics.
|
||||
Status: mock scaffold only.
|
||||
|
||||
### Parsing guidelines
|
||||
|
||||
When adding Redfish mappings, follow these principles:
|
||||
- Support alternate collection paths (resources may appear at different odata URLs).
|
||||
- Follow `@odata.id` references and handle embedded `Members` arrays.
|
||||
- Prefer **raw-tree replay compatibility**: if live collector adds a fallback/probe, replay analyzer must mirror it.
|
||||
- Deduplicate by serial / BDF / slot+model (in that priority order).
|
||||
- Prefer tolerant/fallback parsing — missing fields should be silently skipped,
|
||||
not cause the whole collection to fail.
|
||||
|
||||
### Vendor-specific storage fallbacks (Supermicro and similar)
|
||||
|
||||
When standard `Storage/.../Drives` collections are empty, collector/replay may recover drives via:
|
||||
- `Storage.Links.Enclosures[*] -> .../Drives`
|
||||
- direct probing of finite `Disk.Bay` candidates (`Disk.Bay.0`, `Disk.Bay0`, `.../0`)
|
||||
|
||||
This is required for some BMCs that publish drive inventory in vendor-specific paths while leaving
|
||||
standard collections empty.
|
||||
|
||||
### PSU source preference (newer Redfish)
|
||||
|
||||
PSU inventory source order:
|
||||
1. `Chassis/*/PowerSubsystem/PowerSupplies` (preferred on X14+/newer Redfish)
|
||||
2. `Chassis/*/Power` (legacy fallback)
|
||||
|
||||
### Progress reporting
|
||||
|
||||
The collector emits progress log entries at each stage (connecting, enumerating systems,
|
||||
collecting CPUs, etc.) so the UI can display meaningful status.
|
||||
Current progress message strings are user-facing and may be localized.
|
||||
|
||||
---
|
||||
|
||||
## IPMI Collector (`ipmi`)
|
||||
|
||||
**Status:** Mock scaffold only — not implemented.
|
||||
|
||||
Registered in the collector registry but returns placeholder data.
|
||||
Real IPMI support is a future work item.
|
||||
It remains registered for protocol completeness, but it is not a real collection path.
|
||||
|
||||
Reference in New Issue
Block a user