Add shared bible submodule, rename local bible to bible-local
- Add bible.git as submodule at bible/ - Rename bible/ → bible-local/ (project-specific architecture) - Update CLAUDE.md to reference both bible/ and bible-local/ - Add AGENTS.md for Codex with same structure Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
204
bible-local/03-database.md
Normal file
204
bible-local/03-database.md
Normal file
@@ -0,0 +1,204 @@
|
||||
# 03 — Database
|
||||
|
||||
## SQLite (local, client-side)
|
||||
|
||||
File: `qfs.db` in the user-state directory (see [05-config.md](05-config.md)).
|
||||
|
||||
### Tables
|
||||
|
||||
#### Components and Reference Data
|
||||
|
||||
| Table | Purpose | Key Fields |
|
||||
|-------|---------|------------|
|
||||
| `local_components` | Component metadata (NO prices) | `lot_name` (PK), `lot_description`, `category`, `model` |
|
||||
| `connection_settings` | MariaDB connection settings | key-value store |
|
||||
| `app_settings` | Application settings | `key` (PK), `value`, `updated_at` |
|
||||
|
||||
#### Pricelists
|
||||
|
||||
| Table | Purpose | Key Fields |
|
||||
|-------|---------|------------|
|
||||
| `local_pricelists` | Pricelist headers | `id`, `server_id` (unique), `source`, `version`, `created_at` |
|
||||
| `local_pricelist_items` | Pricelist line items ← **sole source of prices** | `id`, `pricelist_id` (FK), `lot_name`, `price`, `lot_category` |
|
||||
|
||||
#### Partnumber Books (PN → LOT mapping, pull-only from PriceForge)
|
||||
|
||||
| Table | Purpose | Key Fields |
|
||||
|-------|---------|------------|
|
||||
| `local_partnumber_books` | Version snapshots of PN→LOT mappings | `id`, `server_id` (unique), `version`, `created_at`, `is_active` |
|
||||
| `local_partnumber_book_items` | PN→LOT mapping rows | `id`, `book_id` (FK), `partnumber`, `lot_name`, `is_primary_pn` |
|
||||
|
||||
Active book: `WHERE is_active=1 ORDER BY created_at DESC, id DESC LIMIT 1`
|
||||
|
||||
`is_primary_pn=1` means this PN's quantity in the vendor BOM determines qty(LOT). If only non-primary PNs are present for a LOT, qty defaults to 1.
|
||||
|
||||
#### Configurations and Projects
|
||||
|
||||
| Table | Purpose | Key Fields |
|
||||
|-------|---------|------------|
|
||||
| `local_configurations` | Saved configurations | `id`, `uuid` (unique), `items` (JSON), `vendor_spec` (JSON: PN/qty/description + canonical `lot_mappings[]`), `line_no`, `pricelist_id`, `warehouse_pricelist_id`, `competitor_pricelist_id`, `current_version_id`, `sync_status` |
|
||||
| `local_configuration_versions` | Immutable snapshots (revisions) | `id`, `configuration_id` (FK), `version_no`, `data` (JSON), `change_note`, `created_at` |
|
||||
| `local_projects` | Projects | `id`, `uuid` (unique), `name`, `code`, `sync_status` |
|
||||
|
||||
#### Sync
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `pending_changes` | Queue of changes to push to MariaDB |
|
||||
| `local_schema_migrations` | Applied migrations (idempotency guard) |
|
||||
|
||||
---
|
||||
|
||||
### Key SQLite Indexes
|
||||
|
||||
```sql
|
||||
-- Pricelists
|
||||
INDEX local_pricelist_items(pricelist_id)
|
||||
UNIQUE INDEX local_pricelists(server_id)
|
||||
INDEX local_pricelists(source, created_at) -- used for "latest by source" queries
|
||||
-- latest-by-source runtime query also applies deterministic tie-break by id DESC
|
||||
|
||||
-- Configurations
|
||||
INDEX local_configurations(pricelist_id)
|
||||
INDEX local_configurations(warehouse_pricelist_id)
|
||||
INDEX local_configurations(competitor_pricelist_id)
|
||||
INDEX local_configurations(project_uuid, line_no) -- project ordering (Line column)
|
||||
UNIQUE INDEX local_configurations(uuid)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `items` JSON Structure in Configurations
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"lot_name": "CPU_AMD_9654",
|
||||
"quantity": 2,
|
||||
"unit_price": 123456.78,
|
||||
"section": "Processors"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Prices are stored inside the `items` JSON field and refreshed from the pricelist on configuration refresh.
|
||||
|
||||
---
|
||||
|
||||
## MariaDB (server-side, sync-only)
|
||||
|
||||
Database: `RFQ_LOG`
|
||||
|
||||
### Tables and Permissions
|
||||
|
||||
| Table | Purpose | Permissions |
|
||||
|-------|---------|-------------|
|
||||
| `lot` | Component catalog | SELECT |
|
||||
| `qt_lot_metadata` | Extended component data | SELECT |
|
||||
| `qt_categories` | Component categories | SELECT |
|
||||
| `qt_pricelists` | Pricelists | SELECT |
|
||||
| `qt_pricelist_items` | Pricelist line items | SELECT |
|
||||
| `lot_partnumbers` | Partnumber → lot mapping (pricelist enrichment) | SELECT |
|
||||
| `stock_log` | Latest stock qty by partnumber (pricelist enrichment) | SELECT |
|
||||
| `qt_configurations` | Saved configurations (includes `line_no`) | SELECT, INSERT, UPDATE |
|
||||
| `qt_projects` | Projects | SELECT, INSERT, UPDATE |
|
||||
| `qt_client_local_migrations` | Migration catalog | SELECT only |
|
||||
| `qt_client_schema_state` | Applied migrations state | SELECT, INSERT, UPDATE |
|
||||
| `qt_pricelist_sync_status` | Pricelist sync status | SELECT, INSERT, UPDATE |
|
||||
| `qt_partnumber_books` | Partnumber book snapshots (written by PriceForge) | SELECT |
|
||||
| `qt_partnumber_book_items` | PN→LOT mapping rows (written by PriceForge) | SELECT |
|
||||
| `qt_vendor_partnumber_seen` | Vendor PN tracking for unresolved/ignored BOM rows (`is_ignored`) | INSERT, UPDATE |
|
||||
|
||||
### Grant Permissions to Existing User
|
||||
|
||||
```sql
|
||||
GRANT SELECT ON RFQ_LOG.lot TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_lot_metadata TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_categories TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_pricelists TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_pricelist_items TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.lot_partnumbers TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.stock_log TO '<DB_USER>'@'%';
|
||||
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_configurations TO '<DB_USER>'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_projects TO '<DB_USER>'@'%';
|
||||
|
||||
GRANT SELECT ON RFQ_LOG.qt_client_local_migrations TO '<DB_USER>'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_client_schema_state TO '<DB_USER>'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_pricelist_sync_status TO '<DB_USER>'@'%';
|
||||
|
||||
GRANT SELECT ON RFQ_LOG.qt_partnumber_books TO '<DB_USER>'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_partnumber_book_items TO '<DB_USER>'@'%';
|
||||
GRANT INSERT, UPDATE ON RFQ_LOG.qt_vendor_partnumber_seen TO '<DB_USER>'@'%';
|
||||
|
||||
FLUSH PRIVILEGES;
|
||||
```
|
||||
|
||||
### Create a New User
|
||||
|
||||
```sql
|
||||
CREATE USER IF NOT EXISTS 'quote_user'@'%' IDENTIFIED BY '<DB_PASSWORD>';
|
||||
|
||||
GRANT SELECT ON RFQ_LOG.lot TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_lot_metadata TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_categories TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_pricelists TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_pricelist_items TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.lot_partnumbers TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.stock_log TO 'quote_user'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_configurations TO 'quote_user'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_projects TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_client_local_migrations TO 'quote_user'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_client_schema_state TO 'quote_user'@'%';
|
||||
GRANT SELECT, INSERT, UPDATE ON RFQ_LOG.qt_pricelist_sync_status TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_partnumber_books TO 'quote_user'@'%';
|
||||
GRANT SELECT ON RFQ_LOG.qt_partnumber_book_items TO 'quote_user'@'%';
|
||||
GRANT INSERT, UPDATE ON RFQ_LOG.qt_vendor_partnumber_seen TO 'quote_user'@'%';
|
||||
|
||||
FLUSH PRIVILEGES;
|
||||
SHOW GRANTS FOR 'quote_user'@'%';
|
||||
```
|
||||
|
||||
**Note:** If pricelists sync but show `0` positions (or logs contain `enriching pricelist items with stock` + `SELECT denied`), verify `SELECT` on `lot_partnumbers` and `stock_log` in addition to `qt_pricelist_items`.
|
||||
|
||||
**Note:** If you see `Access denied for user ...@'<ip>'`, check for conflicting user entries (user@localhost vs user@'%').
|
||||
|
||||
---
|
||||
|
||||
## Migrations
|
||||
|
||||
### SQLite Migrations (local) — три уровня, выполняются при каждом старте
|
||||
|
||||
**1. GORM AutoMigrate** (`internal/localdb/localdb.go`) — первый и основной уровень.
|
||||
Список Go-моделей передаётся в `db.AutoMigrate(...)`. GORM создаёт отсутствующие таблицы и добавляет новые колонки. Колонки и таблицы **не удаляет**.
|
||||
→ Для добавления новой таблицы или колонки достаточно добавить модель/поле и включить модель в AutoMigrate.
|
||||
|
||||
**2. `runLocalMigrations`** (`internal/localdb/migrations.go`) — второй уровень, для операций которые AutoMigrate не умеет: backfill данных, пересоздание таблиц, создание индексов.
|
||||
Каждая функция выполняется один раз — идемпотентность через запись `id` в `local_schema_migrations`.
|
||||
|
||||
**3. Централизованные (server-side)** — третий уровень, при проверке готовности к синку.
|
||||
SQL-тексты хранятся в `qt_client_local_migrations` (MariaDB, пишет только PriceForge). Клиент читает, применяет к локальной SQLite, записывает в `local_remote_migrations_applied` + `qt_client_schema_state`.
|
||||
|
||||
### MariaDB Migrations (server-side)
|
||||
|
||||
- Stored in `migrations/` (SQL files)
|
||||
- Applied via `-migrate` flag
|
||||
- `min_app_version` — minimum app version required for the migration
|
||||
|
||||
---
|
||||
|
||||
## DB Debugging
|
||||
|
||||
```bash
|
||||
# Inspect schema
|
||||
sqlite3 ~/.local/state/quoteforge/qfs.db ".schema local_components"
|
||||
sqlite3 ~/.local/state/quoteforge/qfs.db ".schema local_configurations"
|
||||
|
||||
# Check pricelist item count
|
||||
sqlite3 ~/.local/state/quoteforge/qfs.db "SELECT COUNT(*) FROM local_pricelist_items"
|
||||
|
||||
# Check pending sync queue
|
||||
sqlite3 ~/.local/state/quoteforge/qfs.db "SELECT COUNT(*) FROM pending_changes"
|
||||
```
|
||||
Reference in New Issue
Block a user