Add asset host logs ingest and UI

This commit is contained in:
Mikhail Chusavitin
2026-03-15 21:38:20 +03:00
parent 5370c1a698
commit f4cd15f0c4
19 changed files with 1374 additions and 222 deletions

View File

@@ -25,11 +25,17 @@ Registry invariants:
- `POST /ingest/hardware`
- `POST /ingest/manual/csv`
- `GET /ingest/manual/csv/jobs/{id}`
- `GET /api/assets/{asset_id}/logs`
- returns deduplicated asset event logs (`host`, `bmc`, `redfish`) with filters by source, severity, activity state, and date range
Manual CSV ingest contract:
- `POST /ingest/manual/csv` is asynchronous and returns `202 Accepted` with `job_id`.
- Final result is read from `GET /ingest/manual/csv/jobs/{id}`.
Hardware ingest notes:
- `POST /ingest/hardware` may accept optional `event_logs` inside the snapshot payload.
- `event_logs` are persisted into dedicated asset log storage and are not mapped to history/timeline events.
## Timeline
- `GET /assets/{id}/timeline`

View File

@@ -18,6 +18,7 @@
- Failures: `failure_events`
- Asset firmware state: `machine_firmware_states`
- Asset sensor readings: `machine_sensor_readings`
- Asset logs: `machine_event_logs`
- Component history (canonical): `component_change_events`, `component_state_snapshots`
- Asset history (canonical): `asset_change_events`, `asset_state_snapshots`
- History jobs/audit: `history_recompute_jobs`, `history_admin_audit`
@@ -32,6 +33,34 @@
- Updated on every ingest — not history-backed (no events, no snapshots).
- `collected_at` reflects the source observation time, not ingest time.
## Asset Event Logs
`machine_event_logs` is a deduplicated append/update read model for host/BMC/Redfish logs per asset.
- Scope: operational log entries shown in dedicated asset log views, not timeline/history.
- Logical identity is a normalized log fingerprint, not the raw ingest row.
- Required stored dimensions:
- `machine_id`
- `log_source` (`host`, `bmc`, `redfish`)
- `fingerprint`
- `event_time`
- `severity`
- `message_id`
- `message`
- `component_ref`
- `raw_payload_json`
- `first_seen_at`
- `last_seen_at`
- `seen_count`
- Upsert rule:
- new fingerprint for same asset/source -> insert new log row
- repeated fingerprint -> update `last_seen_at`, increment `seen_count`, refresh mutable presentation fields if needed
- Not history-backed:
- no `asset_change_events`
- no `component_change_events`
- no `timeline_events`
- `raw_payload_json` is retained for diagnostics and vendor-specific fields.
Out of active model:
- `projects` is legacy and not used by runtime handlers/contracts.

View File

@@ -25,6 +25,12 @@ Direct projection writes are allowed only inside history recompute/rebuild flows
- runtime current-state reads (UI/API/history projection metadata) must use history + projections (`parts`, `machines`, `installations`, `timeline_events`, `machine_firmware_states`)
- `observations` may be used for ingest forensics, audits, and backfill/migration only
Asset event logs are a separate ingest/read-model flow:
- accepted from hardware JSON ingest as optional `event_logs`
- persisted outside history/timeline in dedicated asset log storage
- shown in dedicated UI sections/pages (`Host Logs` / asset logs), not in movement/history timeline
- may be used later for derived diagnostics rules, but ingesting a log entry alone must not mutate canonical asset/component state
## Event Time Source Priority
Use component event time over ingest time whenever possible.
@@ -162,6 +168,19 @@ All asset page component actions are history-backed and transactional.
- Hardware log collection is represented in history as `ASSET_LOG_COLLECTED`.
- UI/API timeline renders it as `LOG_COLLECTED`.
## Asset Event Log Ingest Flow
- Hardware ingest may include normalized raw event logs from `host`, `bmc`, and `redfish`.
- Each incoming log row is normalized and matched by asset + source + fingerprint.
- Duplicate rows update the existing log record (`last_seen_at`, `seen_count`) instead of creating duplicates.
- Asset event logs are queryable independently from timeline/history and must keep source-specific diagnostic payload.
- Log ingest is append/update only; it does not create:
- history events
- snapshots
- timeline projection rows
- failure projections by default
- Any future rule that derives state/failure signals from logs must be explicit and separately documented.
## Delete / Rollback / Hard Restore Flows
Mutating history operations are asynchronous and DB-backed (`history_recompute_jobs`).

View File

@@ -6,10 +6,32 @@ Required section order:
1. `Current Components`
2. `Previous Components` (directly below Current Components)
3. `Movement & Events` (timeline cards)
3. `Host Logs`
4. `Movement & Events` (timeline cards)
`Previous Components` lists components that were previously installed on the asset and are currently removed.
`Host Logs` is a dedicated asset log section for operational logs ingested from `host`, `bmc`, and `redfish`.
- It is separate from timeline/history cards.
- Default presentation is a filterable table/list ordered by newest `event_time` first.
- Filters:
- source (`host`, `bmc`, `redfish`)
- severity
- active/repeated state
- date range
- free-text search over message / message id / component ref
- Rows show at minimum:
- event time
- source
- severity
- message id/code
- message
- component/device reference when available
- repeat counter (`seen_count`) when greater than 1
- Row detail expands raw vendor payload / normalized fields without navigating away.
- Host log rows must not be merged into `Movement & Events`.
`Current Components` interaction contract:
- Section header uses the same right-aligned button row pattern as `Server Card`.