268 lines
11 KiB
Markdown
268 lines
11 KiB
Markdown
# 03 — Database
|
|
|
|
## SQLite (local, client-side)
|
|
|
|
File: `qfs.db` in the user-state directory (see [05-config.md](05-config.md)).
|
|
|
|
### Tables
|
|
|
|
#### Components and Reference Data
|
|
|
|
| Table | Purpose | Key Fields |
|
|
|-------|---------|------------|
|
|
| `local_components` | Component metadata (NO prices) | `lot_name` (PK), `lot_description`, `category`, `model` |
|
|
| `connection_settings` | MariaDB connection settings | key-value store |
|
|
| `app_settings` | Application settings | `key` (PK), `value`, `updated_at` |
|
|
|
|
Read-only cache contract:
|
|
|
|
- `local_components`, `local_pricelists`, `local_pricelist_items`, `local_partnumber_books`, and `local_partnumber_book_items` are synchronized caches, not user-authored data.
|
|
- Startup must prefer application availability over preserving a broken cache schema.
|
|
- If one of these tables cannot be migrated safely, the client may quarantine or drop it and recreate it empty; the next sync repopulates it.
|
|
|
|
#### Pricelists
|
|
|
|
| Table | Purpose | Key Fields |
|
|
|-------|---------|------------|
|
|
| `local_pricelists` | Pricelist headers | `id`, `server_id` (unique), `source`, `version`, `created_at` |
|
|
| `local_pricelist_items` | Pricelist line items ← **sole source of prices** | `id`, `pricelist_id` (FK), `lot_name`, `price`, `lot_category` |
|
|
|
|
#### Partnumber Books (PN → LOT mapping, pull-only from PriceForge)
|
|
|
|
| Table | Purpose | Key Fields |
|
|
|-------|---------|------------|
|
|
| `local_partnumber_books` | Version snapshots of PN→LOT mappings | `id`, `server_id` (unique), `version`, `created_at`, `is_active` |
|
|
| `local_partnumber_book_items` | Canonical PN catalog rows | `id`, `partnumber`, `lots_json`, `description` |
|
|
|
|
Active book: `WHERE is_active=1 ORDER BY created_at DESC, id DESC LIMIT 1`
|
|
|
|
#### Configurations and Projects
|
|
|
|
| Table | Purpose | Key Fields |
|
|
|-------|---------|------------|
|
|
| `local_configurations` | Saved configurations | `id`, `uuid` (unique), `items` (JSON), `vendor_spec` (JSON: PN/qty/description + canonical `lot_mappings[]`), `line_no`, `pricelist_id`, `warehouse_pricelist_id`, `competitor_pricelist_id`, `current_version_id`, `sync_status` |
|
|
| `local_configuration_versions` | Immutable snapshots (revisions) | `id`, `configuration_id` (FK), `version_no`, `data` (JSON), `change_note`, `created_at` |
|
|
| `local_projects` | Projects | `id`, `uuid` (unique), `name`, `code`, `sync_status` |
|
|
|
|
#### Sync
|
|
|
|
| Table | Purpose |
|
|
|-------|---------|
|
|
| `pending_changes` | Queue of changes to push to MariaDB |
|
|
| `local_schema_migrations` | Applied migrations (idempotency guard) |
|
|
|
|
---
|
|
|
|
### Key SQLite Indexes
|
|
|
|
```sql
|
|
-- Pricelists
|
|
INDEX local_pricelist_items(pricelist_id)
|
|
UNIQUE INDEX local_pricelists(server_id)
|
|
INDEX local_pricelists(source, created_at) -- used for "latest by source" queries
|
|
-- latest-by-source runtime query also applies deterministic tie-break by id DESC
|
|
|
|
-- Configurations
|
|
INDEX local_configurations(pricelist_id)
|
|
INDEX local_configurations(warehouse_pricelist_id)
|
|
INDEX local_configurations(competitor_pricelist_id)
|
|
INDEX local_configurations(project_uuid, line_no) -- project ordering (Line column)
|
|
UNIQUE INDEX local_configurations(uuid)
|
|
```
|
|
|
|
---
|
|
|
|
### `items` JSON Structure in Configurations
|
|
|
|
```json
|
|
{
|
|
"items": [
|
|
{
|
|
"lot_name": "CPU_AMD_9654",
|
|
"quantity": 2,
|
|
"unit_price": 123456.78,
|
|
"section": "Processors"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Prices are stored inside the `items` JSON field and refreshed from the pricelist on configuration refresh.
|
|
|
|
---
|
|
|
|
## MariaDB (server-side, sync-only)
|
|
|
|
Database: `RFQ_LOG`
|
|
|
|
### Tables and Permissions
|
|
|
|
| Table | Purpose | Permissions |
|
|
|-------|---------|-------------|
|
|
| `lot` | Component catalog | SELECT |
|
|
| `qt_lot_metadata` | Extended component data | SELECT |
|
|
| `qt_categories` | Component categories | SELECT |
|
|
| `qt_pricelists` | Pricelists | SELECT |
|
|
| `qt_pricelist_items` | Pricelist line items | SELECT |
|
|
| `stock_log` | Latest stock qty by partnumber (pricelist enrichment during sync only) | SELECT |
|
|
| `qt_configurations` | Saved configurations (includes `line_no`) | SELECT, INSERT, UPDATE |
|
|
| `qt_projects` | Projects | SELECT, INSERT, UPDATE |
|
|
| `qt_client_schema_state` | Applied migrations state + client operational status per `username + hostname` | SELECT, INSERT, UPDATE |
|
|
| `qt_pricelist_sync_status` | Pricelist sync status | SELECT, INSERT, UPDATE |
|
|
| `qt_partnumber_books` | Partnumber book headers with snapshot membership in `partnumbers_json` (written by PriceForge) | SELECT |
|
|
| `qt_partnumber_book_items` | Canonical PN catalog with `lots_json` composition (written by PriceForge) | SELECT |
|
|
| `qt_vendor_partnumber_seen` | Vendor PN tracking for unresolved/ignored BOM rows (`is_ignored`) | INSERT only for new `partnumber`; existing rows must not be modified |
|
|
|
|
Legacy server tables not used by QuoteForge runtime anymore:
|
|
|
|
- `qt_bom`
|
|
- `qt_lot_bundles`
|
|
- `qt_lot_bundle_items`
|
|
|
|
QuoteForge canonical BOM storage is:
|
|
|
|
- `qt_configurations.vendor_spec`
|
|
- row-level PN -> multiple LOT decomposition in `vendor_spec[].lot_mappings[]`
|
|
|
|
Partnumber book server read contract:
|
|
|
|
1. Read active or target book from `qt_partnumber_books`.
|
|
2. Parse `partnumbers_json`.
|
|
3. Load payloads from `qt_partnumber_book_items WHERE partnumber IN (...)`.
|
|
|
|
Pricelist stock enrichment contract:
|
|
|
|
1. Sync pulls base pricelist rows from `qt_pricelist_items`.
|
|
2. Sync reads latest stock quantities from `stock_log`.
|
|
3. Sync resolves `partnumber -> lot` through the local mirror of `qt_partnumber_book_items` (`local_partnumber_book_items.lots_json`).
|
|
4. Sync stores enriched `available_qty` and `partnumbers` into `local_pricelist_items`.
|
|
|
|
Runtime rule:
|
|
|
|
- pricelist UI and quote logic read only `local_pricelist_items`;
|
|
- runtime code must not query `stock_log`, `qt_pricelist_items`, or `qt_partnumber_book_items` directly outside sync.
|
|
|
|
`qt_partnumber_book_items` no longer contains `book_id` or `lot_name`.
|
|
It stores one row per `partnumber` with:
|
|
|
|
- `partnumber`
|
|
- `lots_json` as `[{"lot_name":"CPU_X","qty":2}, ...]`
|
|
- `description`
|
|
|
|
`qt_client_schema_state` current contract:
|
|
|
|
- identity key: `username + hostname`
|
|
- client/runtime state:
|
|
`app_version`, `last_checked_at`, `updated_at`
|
|
- operational state:
|
|
`last_sync_at`, `last_sync_status`
|
|
- queue health:
|
|
`pending_changes_count`, `pending_errors_count`
|
|
- local dataset size:
|
|
`configurations_count`, `projects_count`
|
|
- price context:
|
|
`estimate_pricelist_version`, `warehouse_pricelist_version`, `competitor_pricelist_version`
|
|
- last known sync problem:
|
|
`last_sync_error_code`, `last_sync_error_text`
|
|
|
|
`last_sync_error_*` source priority:
|
|
|
|
1. blocked readiness state from `local_sync_guard_state`
|
|
2. latest non-empty `pending_changes.last_error`
|
|
3. `NULL` when no known sync problem exists
|
|
|
|
### Grant Permissions to Existing User
|
|
|
|
```sql
|
|
GRANT SELECT ON RFQ_LOG.lot TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_lot_metadata TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_categories TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_pricelists TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_pricelist_items TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.stock_log TO '<DB_USER>'@'%';
|
|
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_configurations TO '<DB_USER>'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_projects TO '<DB_USER>'@'%';
|
|
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_client_schema_state TO '<DB_USER>'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_pricelist_sync_status TO '<DB_USER>'@'%';
|
|
|
|
GRANT SELECT ON RFQ_LOG.qt_partnumber_books TO '<DB_USER>'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_partnumber_book_items TO '<DB_USER>'@'%';
|
|
GRANT INSERT, UPDATE ON RFQ_LOG.qt_vendor_partnumber_seen TO '<DB_USER>'@'%';
|
|
|
|
FLUSH PRIVILEGES;
|
|
```
|
|
|
|
### Create a New User
|
|
|
|
```sql
|
|
CREATE USER IF NOT EXISTS 'quote_user'@'%' IDENTIFIED BY '<DB_PASSWORD>';
|
|
|
|
GRANT SELECT ON RFQ_LOG.lot TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_lot_metadata TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_categories TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_pricelists TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_pricelist_items TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.stock_log TO 'quote_user'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_configurations TO 'quote_user'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_projects TO 'quote_user'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_client_schema_state TO 'quote_user'@'%';
|
|
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_pricelist_sync_status TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_partnumber_books TO 'quote_user'@'%';
|
|
GRANT SELECT ON RFQ_LOG.qt_partnumber_book_items TO 'quote_user'@'%';
|
|
GRANT INSERT, UPDATE ON RFQ_LOG.qt_vendor_partnumber_seen TO 'quote_user'@'%';
|
|
|
|
FLUSH PRIVILEGES;
|
|
SHOW GRANTS FOR 'quote_user'@'%';
|
|
```
|
|
|
|
**Note:** If pricelists sync but stock enrichment is empty, verify `SELECT` on `qt_pricelist_items`, `qt_partnumber_books`, `qt_partnumber_book_items`, and `stock_log`.
|
|
|
|
**Note:** If you see `Access denied for user ...@'<ip>'`, check for conflicting user entries (user@localhost vs user@'%').
|
|
|
|
---
|
|
|
|
## Migrations
|
|
|
|
### SQLite Migrations (local) — два уровня, выполняются при каждом старте
|
|
|
|
**1. GORM AutoMigrate** (`internal/localdb/localdb.go`) — первый и основной уровень.
|
|
Список Go-моделей передаётся в `db.AutoMigrate(...)`. GORM создаёт отсутствующие таблицы и добавляет новые колонки. Колонки и таблицы **не удаляет**.
|
|
|
|
Local SQLite partnumber book cache contract:
|
|
|
|
- `local_partnumber_books.partnumbers_json` stores PN membership for a pulled book.
|
|
- `local_partnumber_book_items` is a deduplicated local catalog by `partnumber`.
|
|
- `local_partnumber_book_items.lots_json` mirrors the server `lots_json` payload.
|
|
- SQLite migration `2026_03_07_local_partnumber_book_catalog` rebuilds old `book_id + lot_name` rows into the new local cache shape.
|
|
→ Для добавления новой таблицы или колонки достаточно добавить модель/поле и включить модель в AutoMigrate.
|
|
|
|
**2. `runLocalMigrations`** (`internal/localdb/migrations.go`) — второй уровень, для операций которые AutoMigrate не умеет: backfill данных, пересоздание таблиц, создание индексов.
|
|
Каждая функция выполняется один раз — идемпотентность через запись `id` в `local_schema_migrations`.
|
|
|
|
QuoteForge does not use centralized server-driven SQLite migrations.
|
|
All local SQLite schema/data migrations live in the client codebase.
|
|
|
|
### MariaDB Migrations (server-side)
|
|
|
|
- Stored in `migrations/` (SQL files)
|
|
- Applied via `-migrate` flag
|
|
- `min_app_version` — minimum app version required for the migration
|
|
|
|
---
|
|
|
|
## DB Debugging
|
|
|
|
```bash
|
|
# Inspect schema
|
|
sqlite3 ~/.local/state/quoteforge/qfs.db ".schema local_components"
|
|
sqlite3 ~/.local/state/quoteforge/qfs.db ".schema local_configurations"
|
|
|
|
# Check pricelist item count
|
|
sqlite3 ~/.local/state/quoteforge/qfs.db "SELECT COUNT(*) FROM local_pricelist_items"
|
|
|
|
# Check pending sync queue
|
|
sqlite3 ~/.local/state/quoteforge/qfs.db "SELECT COUNT(*) FROM pending_changes"
|
|
```
|