mchus/bible
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add agent bootstrap and contract read router

Browse Source
This commit is contained in:
Mikhail Chusavitin
2026-04-02 13:48:36 +03:00
parent 688b87e98d
commit 1d89a4918e
22 changed files with 883 additions and 1284 deletions
Download Patch File Download Diff File Expand all files Collapse all files

253
rules/patterns/go-database/contract.md
View File

@@ -2,225 +2,56 @@
Version: 1.9
## MySQL Transaction Cursor Safety (CRITICAL)
See `README.md` for examples, migration snippets, and Docker test commands.
**Never execute SQL on the same transaction while iterating over a query result cursor.**
## Query and Startup Rules
This is the most common source of `invalid connection` and `unexpected EOF` driver panics.
### Rule
Use a two-phase approach: read all rows first, close the cursor, then execute writes.
```go
// WRONG — executes SQL inside rows.Next() loop on the same tx
rows, _ := tx.Query("SELECT id FROM machines")
for rows.Next() {
var id string
rows.Scan(&id)
tx.Exec("UPDATE machines SET processed=1 WHERE id=?", id) // DEADLOCK / driver panic
}
// CORRECT — collect IDs first, then write
rows, _ := tx.Query("SELECT id FROM machines")
var ids []string
for rows.Next() {
var id string
rows.Scan(&id)
ids = append(ids, id)
}
rows.Close() // explicit close before any write
for _, id := range ids {
tx.Exec("UPDATE machines SET processed=1 WHERE id=?", id)
}
```
This applies to:
- `database/sql` with manual transactions
- GORM `db.Raw().Scan()` inside a `db.Transaction()` callback
- Any loop that calls a repository method while a cursor is open
## Soft Delete / Archive Pattern
Do not use hard deletes for user-visible records. Use an archive flag.
```go
// Schema: is_active bool DEFAULT true
// "Delete" = set is_active = false
// Restore = set is_active = true
// All list queries must filter:
WHERE is_active = true
```
- Never physically delete rows that have foreign key references or history.
- Hard delete is only acceptable for orphaned/temporary data with no audit trail requirement.
- Never execute SQL on the same transaction while iterating an open result cursor. Use a two-phase flow: read all rows, close the cursor, then execute writes.
- This rule applies to `database/sql`, GORM transactions, and any repository call made while another cursor in the same transaction is still open.
- User-visible records use soft delete or archive flags. Do not hard-delete records with history or foreign-key references.
- Archive operations must be reversible from the UI.
- Use `gorm:"-"` only for fields that must be ignored entirely. Use `gorm:"-:migration"` for fields populated by queries but excluded from migrations.
- Always verify the DB connection before starting the HTTP server. Never serve traffic with an unverified DB connection.
- Prevent N+1 queries. Do not query inside loops over rows from another query; use JOINs or batched `IN (...)` queries.
## GORM Virtual Fields
## Migration and Backup Rules
Use the correct tag based on whether the field should exist in the DB schema:
- The migration engine owns backup creation. The operator must never be required to take a manual pre-migration backup.
- Backup storage, retention, archive format, and restore-readiness must follow `backup-management`.
- Before applying any unapplied migrations, take and verify a full DB backup.
- Before applying a migration step that changes a table, take a targeted backup of each affected table.
- Before writing any backup, verify that the output path resolves outside the git worktree and is not tracked or staged in git.
- If any migration step in a session fails, roll back all steps applied in that session in reverse order.
- If rollback is not sufficient, restore from the targeted backup taken before the failing step.
- After rollback or restore, the DB must be back in the same state it had before the session started.
- Migration failures must emit structured diagnostics naming the failed step, rollback actions, and final DB state.
```go
// Field computed at runtime, column must NOT exist in DB (excludes from migrations AND queries)
Count int `gorm:"-"`
## Migration Authoring Rules
// Field computed at query time via JOIN/SELECT, column must NOT be in migrations
// but IS populated from query results
DisplayName string `gorm:"-:migration"`
```
- For local-first desktop apps, migration recovery must also follow `local-first-recovery`.
- Migrations are sequential and immutable after merge.
- Each migration should be reversible where possible.
- Do not rename a column in one step. Add new, backfill, and drop old across separate deploys.
- Auto-apply on startup is allowed for internal tools only if the behavior is documented.
- Every `.sql` migration file must start with:
- `-- Tables affected: ...`
- `-- recovery.not-started: ...`
- `-- recovery.partial: ...`
- `-- recovery.completed: ...`
- one or more `-- verify: <description> | <SQL>` checks
- Verify queries must return rows only when something is wrong.
- Verify queries must exclude NULL and empty values when those would create false positives.
- A migration is recorded as applied only after all verify checks pass.
- `gorm:"-"` — fully ignored: no migration, no read, no write.
- `gorm:"-:migration"` — skip migration only; GORM will still read/write if the column exists.
- Do not use `gorm:"-"` for JOIN-populated fields — the value will always be zero.
## Pre-Production Validation Rules
## Fail-Fast DB Check on Startup
- Test pending migrations on a dump of the current production DB, not on fixtures.
- Use a local MariaDB Docker container matching the production version and collation.
- Execute each migration file as one DB session so session variables such as `SET FOREIGN_KEY_CHECKS = 0` remain in effect for the whole file.
- If migrations fail in Docker, fix them before touching production.
Always verify the database connection before starting the HTTP server.
## Common Pitfalls
```go
sqlDB, err := db.DB()
if err != nil || sqlDB.Ping() != nil {
log.Fatal("database unavailable, refusing to start")
}
// then: run migrations, then: start gin/http server
```
Never start serving traffic with an unverified DB connection. Fail loudly at boot.
## N+1 Query Prevention
Use JOINs or batch IN queries. Never query inside a loop over rows from another query.
```go
// WRONG
for _, pricelist := range pricelists {
items, _ := repo.GetItems(pricelist.ID) // N queries
}
// CORRECT
items, _ := repo.GetItemsByPricelistIDs(ids) // 1 query with WHERE id IN (...)
// then group in Go
```
## Automatic Backup During Migration
The migration engine is responsible for all backup steps. The operator must never be required to take a backup manually.
Backup naming, storage, archive format, retention, and restore-readiness must follow the `backup-management` contract.
### Full DB Backup on New Migrations
When the migration engine detects that new (unapplied) migrations exist, it must take a full database backup before applying any of them.
Rules:
- The full backup must complete and be verified before the first migration step runs.
- The backup must be triggered by the application's own backup mechanism; do not assume `mysql`, `mysqldump`, `pg_dump`, or any other external DB client tool is present on the operator's machine.
- Before creating the backup, verify that the backup output path resolves outside the git worktree and is not tracked or staged in git.
### Per-Table Backup Before Each Table Migration
Before applying a migration step that affects a specific table, take a targeted backup of that table.
Rules:
- A per-table backup must be created immediately before the migration step that modifies that table.
- If a single migration step touches multiple tables, back up each affected table before the step runs.
- Per-table backups are in addition to the full DB backup; they are not a substitute for it.
### Session Rollback on Failure
If any migration step fails during a session, the engine must roll back all migrations applied in that session.
Rules:
- "Session" means all migration steps started in a single run of the migration engine.
- On failure, roll back every step applied in the current session in reverse order before surfacing the error.
- If rollback of a step is not possible (e.g., the operation is not reversible in MySQL without the per-table backup), restore from the per-table backup taken before that step.
- After rollback or restore, the database must be in the same state as before the session started.
- The engine must emit structured diagnostics that identify which step failed, which steps were rolled back, and the final database state.
## Migration Policy
- For local-first desktop applications, startup and migration recovery must follow the `local-first-recovery` contract.
- Migrations are numbered sequentially and never modified after merge.
- Each migration must be reversible where possible (document rollback in a comment).
- Never rename a column in one migration step — add new, backfill, drop old across separate deploys.
- Auto-apply migrations on startup is acceptable for internal tools; document if used.
## SQL Migration File Format
Every `.sql` migration file must begin with a structured header block:
```sql
-- Tables affected: supplier, lot_log
-- recovery.not-started: No action required.
-- recovery.partial: DELETE FROM parts_log WHERE created_by = 'migration';
-- recovery.completed: Same as partial.
-- verify: No orphaned supplier_code | SELECT supplier_code FROM parts_log pl LEFT JOIN supplier s ON s.supplier_code = pl.supplier_code WHERE s.supplier_code IS NULL LIMIT 1
-- verify: No empty supplier_code | SELECT supplier_name FROM supplier WHERE supplier_code = '' LIMIT 1
```
**`-- Tables affected:`** — comma-separated list of tables the migration touches. Used by the backup engine to take a targeted pre-migration backup. Omit only if no table can be identified; the engine falls back to full DB backup.
**`-- recovery.*:`** — human-readable rollback SQL for each migration state (`not-started`, `partial`, `completed`). Executed manually by an operator if automatic restore fails. Must be correct, copy-pasteable SQL.
**`-- verify:`** — post-migration assertion query. Format: `-- verify: <description> | <SQL>`. The engine runs the query after all statements in the file succeed. If the query returns **any row**, the migration is considered failed and is rolled back. Write the query so it returns a row only when something is **wrong**:
```sql
-- verify: Orphaned FK refs | SELECT id FROM child c LEFT JOIN parent p ON p.id = c.parent_id WHERE p.id IS NULL LIMIT 1
-- ^ returns a row = bad ^ returns nothing = good
```
- Verify queries must filter out NULL/empty values that would cause false positives: add `AND col IS NOT NULL AND col != ''`.
- A migration is only recorded as applied after all verify checks pass.
- Verify checks are not a substitute for testing; they are a last-resort safety net on production.
## Pre-Production Migration Testing in Docker
Before applying a set of new migrations to production, always validate them against a copy of the production database in a local MariaDB Docker container that matches the production version and collation.
```bash
# Start container matching production (MariaDB 11.8, utf8mb4_uca1400_ai_ci)
docker run -d --name pf_test \
-e MYSQL_ROOT_PASSWORD=test -e MYSQL_DATABASE=RFQ_LOG \
mariadb:11.8 --character-set-server=utf8mb4 --collation-server=utf8mb4_uca1400_ai_ci
# Load production dump
docker exec -i pf_test mariadb -uroot -ptest RFQ_LOG < prod_dump.sql
# Run migrations via pfs (uses real migration engine + verify checks, no backup)
./pfs -migrate-dsn "root:test@tcp(127.0.0.1:3306)/RFQ_LOG?parseTime=true&charset=utf8mb4&multiStatements=true" \
-no-backup -verbose
```
The `-migrate-dsn` flag connects to the given DSN, runs all pending migrations, runs verify checks, and exits. No config file, no server, no browser.
**Rules:**
- Always test on a dump of the **current production database**, not a fixture — schema drift and real data distributions expose bugs that fixtures miss.
- The Docker container must use the same MariaDB version and `--collation-server` as production.
- Each migration file is executed as a **single session** so `SET FOREIGN_KEY_CHECKS = 0` applies to all its statements. Never test by running statements from a migration file individually across separate sessions — the session variable will reset between them.
- If any migration fails in Docker, fix the SQL before touching production. Do not rely on "it will be different in production."
## SQL Migration Authoring — Common Pitfalls
**Semicolons inside string literals break naive splitters.**
The migration engine uses a quote-aware statement splitter. Do not rely on external tools that split on bare `;`. When writing supplier/product names with punctuation, use commas — not semicolons — as separators in string literals. A semicolon inside `'COMPANY; LTD'` will break any naive `split(";")` approach.
**`SET FOREIGN_KEY_CHECKS = 0` only applies to the current session.**
This is a session variable. If statements run in separate connections (e.g. via individual subprocess calls), FK checks are re-enabled for each new connection. Always run an entire migration file as one session. The pfs migration engine runs all statements in a file on the same GORM db handle, which reuses the same connection.
**Verify queries must exclude NULL values.**
A query like `SELECT c.col FROM child c LEFT JOIN parent p ON p.id = c.id WHERE p.id IS NULL` will return rows with `c.col = NULL` if the child table has rows with a NULL FK value. Add `AND c.col IS NOT NULL AND c.col != ''` to avoid false failures.
**Catch-all INSERT for referential integrity before adding FK constraints.**
When adding a FK constraint to a table that previously had no FK (legacy data may have orphaned references), add a catch-all step before the constraint:
```sql
-- Ensure every value referenced in child table exists in parent before adding FK.
INSERT IGNORE INTO parent (name)
SELECT DISTINCT c.fk_col FROM child c
LEFT JOIN parent p ON p.name = c.fk_col
WHERE p.name IS NULL AND c.fk_col IS NOT NULL AND c.fk_col != '';
```
This is not a hack — it repairs data that was valid before the constraint existed. Never delete orphaned child rows unless data loss is acceptable.
- Do not use tools that naively split SQL on bare `;`. String literals may contain semicolons.
- `SET FOREIGN_KEY_CHECKS = 0` is session-scoped. If the file is split across multiple sessions, FK checks come back on.
- When adding a new FK to legacy data, repair missing parent rows before enforcing the constraint unless data loss is explicitly acceptable.