diff --git a/rules/patterns/go-database/contract.md b/rules/patterns/go-database/contract.md index 54ab940..2caed9b 100644 --- a/rules/patterns/go-database/contract.md +++ b/rules/patterns/go-database/contract.md @@ -1,6 +1,6 @@ # Contract: Database Patterns (Go / MySQL / MariaDB) -Version: 1.8 +Version: 1.9 ## MySQL Transaction Cursor Safety (CRITICAL) @@ -146,3 +146,81 @@ Rules: - Each migration must be reversible where possible (document rollback in a comment). - Never rename a column in one migration step — add new, backfill, drop old across separate deploys. - Auto-apply migrations on startup is acceptable for internal tools; document if used. + +## SQL Migration File Format + +Every `.sql` migration file must begin with a structured header block: + +```sql +-- Tables affected: supplier, lot_log +-- recovery.not-started: No action required. +-- recovery.partial: DELETE FROM parts_log WHERE created_by = 'migration'; +-- recovery.completed: Same as partial. +-- verify: No orphaned supplier_code | SELECT supplier_code FROM parts_log pl LEFT JOIN supplier s ON s.supplier_code = pl.supplier_code WHERE s.supplier_code IS NULL LIMIT 1 +-- verify: No empty supplier_code | SELECT supplier_name FROM supplier WHERE supplier_code = '' LIMIT 1 +``` + +**`-- Tables affected:`** — comma-separated list of tables the migration touches. Used by the backup engine to take a targeted pre-migration backup. Omit only if no table can be identified; the engine falls back to full DB backup. + +**`-- recovery.*:`** — human-readable rollback SQL for each migration state (`not-started`, `partial`, `completed`). Executed manually by an operator if automatic restore fails. Must be correct, copy-pasteable SQL. + +**`-- verify:`** — post-migration assertion query. Format: `-- verify: | `. The engine runs the query after all statements in the file succeed. If the query returns **any row**, the migration is considered failed and is rolled back. Write the query so it returns a row only when something is **wrong**: + +```sql +-- verify: Orphaned FK refs | SELECT id FROM child c LEFT JOIN parent p ON p.id = c.parent_id WHERE p.id IS NULL LIMIT 1 +-- ^ returns a row = bad ^ returns nothing = good +``` + +- Verify queries must filter out NULL/empty values that would cause false positives: add `AND col IS NOT NULL AND col != ''`. +- A migration is only recorded as applied after all verify checks pass. +- Verify checks are not a substitute for testing; they are a last-resort safety net on production. + +## Pre-Production Migration Testing in Docker + +Before applying a set of new migrations to production, always validate them against a copy of the production database in a local MariaDB Docker container that matches the production version and collation. + +```bash +# Start container matching production (MariaDB 11.8, utf8mb4_uca1400_ai_ci) +docker run -d --name pf_test \ + -e MYSQL_ROOT_PASSWORD=test -e MYSQL_DATABASE=RFQ_LOG \ + mariadb:11.8 --character-set-server=utf8mb4 --collation-server=utf8mb4_uca1400_ai_ci + +# Load production dump +docker exec -i pf_test mariadb -uroot -ptest RFQ_LOG < prod_dump.sql + +# Run migrations via pfs (uses real migration engine + verify checks, no backup) +./pfs -migrate-dsn "root:test@tcp(127.0.0.1:3306)/RFQ_LOG?parseTime=true&charset=utf8mb4&multiStatements=true" \ + -no-backup -verbose +``` + +The `-migrate-dsn` flag connects to the given DSN, runs all pending migrations, runs verify checks, and exits. No config file, no server, no browser. + +**Rules:** +- Always test on a dump of the **current production database**, not a fixture — schema drift and real data distributions expose bugs that fixtures miss. +- The Docker container must use the same MariaDB version and `--collation-server` as production. +- Each migration file is executed as a **single session** so `SET FOREIGN_KEY_CHECKS = 0` applies to all its statements. Never test by running statements from a migration file individually across separate sessions — the session variable will reset between them. +- If any migration fails in Docker, fix the SQL before touching production. Do not rely on "it will be different in production." + +## SQL Migration Authoring — Common Pitfalls + +**Semicolons inside string literals break naive splitters.** +The migration engine uses a quote-aware statement splitter. Do not rely on external tools that split on bare `;`. When writing supplier/product names with punctuation, use commas — not semicolons — as separators in string literals. A semicolon inside `'COMPANY; LTD'` will break any naive `split(";")` approach. + +**`SET FOREIGN_KEY_CHECKS = 0` only applies to the current session.** +This is a session variable. If statements run in separate connections (e.g. via individual subprocess calls), FK checks are re-enabled for each new connection. Always run an entire migration file as one session. The pfs migration engine runs all statements in a file on the same GORM db handle, which reuses the same connection. + +**Verify queries must exclude NULL values.** +A query like `SELECT c.col FROM child c LEFT JOIN parent p ON p.id = c.id WHERE p.id IS NULL` will return rows with `c.col = NULL` if the child table has rows with a NULL FK value. Add `AND c.col IS NOT NULL AND c.col != ''` to avoid false failures. + +**Catch-all INSERT for referential integrity before adding FK constraints.** +When adding a FK constraint to a table that previously had no FK (legacy data may have orphaned references), add a catch-all step before the constraint: + +```sql +-- Ensure every value referenced in child table exists in parent before adding FK. +INSERT IGNORE INTO parent (name) +SELECT DISTINCT c.fk_col FROM child c +LEFT JOIN parent p ON p.name = c.fk_col +WHERE p.name IS NULL AND c.fk_col IS NOT NULL AND c.fk_col != ''; +``` + +This is not a hack — it repairs data that was valid before the constraint existed. Never delete orphaned child rows unless data loss is acceptable.