Compare commits

...
Author SHA1 Message Date
Mikhail Chusavitin 8569871a17 fix: verify server snapshots and recover whole migration sessions (#3) 2026-10-26 22:48:07 +03:00
Mikhail Chusavitin 3d4101ecc0 test: enforce release failure gates and record platform builds (#6) 2026-10-26 21:22:34 +03:00
Mikhail Chusavitin 46d7ccda4d test: verify TLS policies against disposable MariaDB servers (#5) 2026-10-26 21:10:24 +03:00
Mikhail Chusavitin 2d60fcfb39 test: verify reset CLI and restored startup preserve local work (#7) 2026-10-26 21:07:41 +03:00
Mikhail Chusavitin ffdbe4fbd0 test: exercise XSS regressions in headless Chrome (#4) 2026-10-26 21:03:36 +03:00
Mikhail Chusavitin e8dd03c4f5 test: verify browser trust boundary on both application routers (#2) 2026-10-26 20:59:35 +03:00
Mikhail ChusavitinandClaude Opus 5.5 fa09513e26 docs: release notes v3.51
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:52:03 +03:00
Mikhail ChusavitinandClaude Opus 5.5 44a09d4420 fix(sync): stop MariaDB connection leak and cut per-cycle sync waste
- ConnectionManager reuses a stale-but-healthy pool instead of dialing a new
  one every 30s, and closes every pool it replaces (one client held ~30
  connections). Pool capped at 2 connections, idle released after 1 min.
- Background sync every 1 min with ±30% jitter, scheduled from cycle start;
  users-status online threshold = interval × 1.3. One heartbeat per cycle.
- Partnumber books: read headers only, fetch partnumbers_json per book when
  new/incomplete; empty book counts as complete; an incomplete book is
  re-pulled once per process (server book lists BOT_AGIBOT_G2_PERF missing
  from qt_partnumber_book_items).
- NeedSync in one UNION ALL query; SyncQtSettings skips unchanged rewrite.
- Migration 035: server-maintained server_updated_at on qt_configurations and
  qt_projects. Clients pull only changed rows after one full pull per process;
  servers without 035 and old clients keep full pulls.
- Migrator: fresh GORM statement per verify query and history insert; files
  with two verify checks failed to record.

ADL: bible-local/decisions/2026-10-26-mariadb-single-pool.md,
bible-local/decisions/2026-10-26-incremental-config-project-pull.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:46:59 +03:00
Mikhail ChusavitinandCodex 8a08270b79 docs: release notes v3.50
Co-Authored-By: Codex <noreply@openai.com>
2026-10-26 10:58:39 +03:00
mchus cab5d13f16 Merge pull request 'Исправления шести issue: безопасность API, защита данных, TLS и миграции' (#8) from fix/open-issues-security-recovery into main 2026-10-26 10:56:07 +03:00
Mikhail Chusavitin d6c498f0cf docs: record issue implementation plan and validation results 2026-10-26 10:54:22 +03:00
Mikhail Chusavitin aa55dc621f build: require patched Go and gate releases on govulncheck (#6) 2026-10-26 10:53:14 +03:00
Mikhail Chusavitin a59cc7ccb7 fix: verify SQL migrations before recording and preserve sessions (#3) 2026-10-26 10:49:23 +03:00
Mikhail Chusavitin 370407d3ba feat: configure MariaDB TLS for setup and runtime connections (#5) 2026-10-26 10:47:46 +03:00
Mikhail Chusavitin 15f81e053c fix: require verified pre-reset backup and protect unsynced work (#7) 2026-10-26 10:41:25 +03:00
Mikhail Chusavitin 1ada8c12e2 fix: render LOT values and sync errors without executable interpolation (#4) 2026-10-26 10:38:41 +03:00
Mikhail Chusavitin c0c7804494 fix: protect local HTTP boundary and saved setup credentials (#2) 2026-10-26 10:35:37 +03:00
Mikhail ChusavitinandClaude Opus 5.5 134685d482 docs: release notes v3.49
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:31:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 88629a14da fix: «Цена покупки» показывает суммы по строке (цена × кол-во), как «Цена продажи»
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:30:46 +03:00
mchusandClaude Sonnet 5.5 2f30dd0d27 docs: release notes v3.48
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 10:45:54 +03:00
mchusandClaude Sonnet 5.5 d48865869a fix: платформа поддержки HGX только для SXM/HGX GPU; актуализированы цены HGX по регламенту 2026-10-07
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 10:45:51 +03:00
Mikhail ChusavitinandClaude Sonnet 5 8b073e63b1 docs: release notes v3.47
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 11:54:11 +03:00
Mikhail ChusavitinandClaude Sonnet 5 cccb8890a4 fix: requeue orphaned pending configurations; independent background sync streams
- Configurations stuck in sync_status=pending without a pending_changes row are
  re-enqueued as an update with the current snapshot on each push cycle.
- Background cycle no longer aborts pricelist/partbook/settings sync when push or
  readiness fails; each stream logs its own error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 11:53:44 +03:00
Mikhail ChusavitinandClaude Opus 5.5 310b296045 docs: release notes v3.46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 16:34:17 +03:00
Mikhail ChusavitinandClaude Opus 5.5 746e5663fb fix: история автозаполнения браузера перекрывала выпадашку LOT
Поля с собственной выпадашкой в конфигураторе не имели autocomplete="off",
и браузер рисовал поверх неё нативный список ранее введённых значений.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 16:33:54 +03:00
Mikhail ChusavitinandClaude Opus 5.5 ae26db6624 docs: release notes v3.45
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:24:09 +03:00
Mikhail ChusavitinandClaude Opus 5.5 673a207079 feat: вкладки конфигуратора только из qt_settings, без JS-копии
- кнопки вкладок строятся из tab_config (порядок, label); удалённая на
  сервере вкладка (SW) больше не висит статической кнопкой;
- TAB_CONFIG на странице содержит только «Other», остальное — из
  /api/configurator-settings; убраны захардкоженные фильтры категорий по
  типу и ALWAYS_VISIBLE_TABS; тип без allowlist видит все категории;
- настройки грузятся до категорий и до фильтрации по типу конфигурации;
- единственный fallback — Go-дефолты в handlers/component.go;
  always_visible_tabs по умолчанию ["base"].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:23:48 +03:00
Mikhail ChusavitinandClaude Opus 5.5 aefe167590 docs: release notes v3.44
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:00:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 a939a5f36c feat: типы конфигураций из qt_settings в модалке проекта, копия сохраняет config_type
- модалка «Новая конфигурация в проекте» строит кнопки типа из
  qt_settings.config_types (новый тип «Сеть» появляется без релиза);
- ручная полная синхронизация (/api/sync/all) тянет qt_settings;
- копирование конфигурации (в т.ч. из версии) сохраняет config_type —
  раньше копия СХД/сети становилась «сервером».

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 16:59:42 +03:00
Mikhail ChusavitinandClaude Opus 5.5 54a7c7cc8d docs: release notes v3.43
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 14:34:45 +03:00
Mikhail ChusavitinandClaude Opus 5.5 a2a1904d91 feat: make migrate — серверные SQL-миграции вместо AutoMigrate
`qfs -migrate` теперь применяет только migrations/*.sql (встроены в бинарник)
с учётом в qt_schema_migrations и завершается, не запуская приложение.
Новые команды: make migrate-status (read-only список applied/pending),
make migrate, make migrate-baseline UPTO=<file> (пометить как применённые
без выполнения). migrate отказывается работать, если в истории нет ни одной
применённой миграции — иначе все файлы пошли бы заново.

Удалён GORM AutoMigrate против MariaDB (models.Migrate, SeedCategories):
устаревший бинарник v1.18 с -migrate прогнал по боевой базе AutoMigrate
своих моделей. Миграции идут через отдельное соединение без read/write
таймаутов — 3s-таймаут sync-соединения обрывал backfill в 032.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 12:21:53 +03:00
Mikhail ChusavitinandClaude Opus 5.5 259bd48d2e docs: release notes v3.42
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 12:11:48 +03:00
103 changed files with 5579 additions and 694 deletions

No files matched your search

+27 -5
View File
@@ -1,4 +1,4 @@
.PHONY: build build-release clean test run version install-hooks
.PHONY: build build-release clean test run version install-hooks migrate migrate-status migrate-baseline security-check
# Get version from git
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
@@ -13,20 +13,20 @@ build:
go build -o bin/$(BINARY) ./cmd/qfs
# Build for release (optimized, with version)
build-release:
build-release: security-check
@echo "Building $(BINARY) version $(VERSION)..."
CGO_ENABLED=0 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY) ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)"
@./bin/$(BINARY) -version
# Build release for Linux (cross-compile)
build-linux:
build-linux: security-check
@echo "Building $(BINARY) for Linux..."
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-linux-amd64 ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)-linux-amd64"
# Build release for macOS (cross-compile)
build-macos:
build-macos: security-check
@echo "Building $(BINARY) for macOS..."
CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-darwin-amd64 ./cmd/qfs
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-darwin-arm64 ./cmd/qfs
@@ -34,7 +34,7 @@ build-macos:
@echo "✓ Built: bin/$(BINARY)-darwin-arm64"
# Build release for Windows (cross-compile)
build-windows:
build-windows: security-check
@echo "Building $(BINARY) for Windows..."
CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-windows-amd64.exe ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)-windows-amd64.exe"
@@ -55,10 +55,32 @@ clean:
rm -rf bin/
rm -f $(BINARY)
# Server (MariaDB) migrations — migrations/*.sql tracked in qt_schema_migrations.
# Always built from the current source (go run), never a possibly stale bin/ binary.
# Uses the connection saved in the local app settings.
# List migrations with applied/pending state (read-only)
migrate-status:
go run ./cmd/qfs -migrate-status
# Apply pending migrations
migrate:
go run ./cmd/qfs -migrate
# Record migrations up to UPTO as applied WITHOUT running them (one-time, for a server
# whose schema is already current): make migrate-baseline UPTO=033_add_price_quality_to_pricelist_items.sql
migrate-baseline:
@test -n "$(UPTO)" || (echo "usage: make migrate-baseline UPTO=<file.sql>"; exit 1)
go run ./cmd/qfs -migrate-baseline "$(UPTO)"
# Run tests
test:
go test -v ./...
# Pin the scanner so releases do not depend on a locally installed executable.
security-check:
go run golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...
# Run development server
run:
go run ./cmd/qfs
+2 -1
View File
@@ -24,7 +24,8 @@ go run ./cmd/qfs
Useful commands:
```bash
go run ./cmd/qfs -migrate
make migrate-status
make migrate
go test ./...
go vet ./...
make build-release
+11
View File
@@ -19,3 +19,14 @@ var StaticFiles embed.FS
func StaticFS() (fs.FS, error) {
return fs.Sub(StaticFiles, "web/static")
}
// MigrationFiles contains the MariaDB server migrations (migrations/*.sql), so
// `qfs -migrate` works from any directory. Applied via models.RunSQLMigrations.
//
//go:embed migrations/*.sql
var MigrationFiles embed.FS
// MigrationsFS returns a filesystem rooted at migrations/.
func MigrationsFS() (fs.FS, error) {
return fs.Sub(MigrationFiles, "migrations")
}
+16
View File
@@ -0,0 +1,16 @@
package quoteforge
import (
"io/fs"
"testing"
)
func TestMigrationsEmbedded(t *testing.T) {
migrations, err := MigrationsFS()
if err != nil {
t.Fatal(err)
}
if _, err := fs.Stat(migrations, "034_add_feature_flags_to_configurations.sql"); err != nil {
t.Fatalf("migration 034 not embedded: %v", err)
}
}
+11
View File
@@ -68,3 +68,14 @@ bible/ shared engineering rules
bible-local/ project-specific architecture
releases/ release artifacts and notes
```
## Local HTTP trust boundary
Loopback binding does not make a browser client trusted. Both setup and normal
routers reject non-loopback Host, Origin and Referer values, opaque origins and
cross-site Fetch Metadata requests before any handler runs. CLI clients may omit
browser headers. API requests carrying a body require application/json, except
the project vendor-import endpoint, which also accepts multipart/form-data.
Setup retains HTML forms; an empty password reuses the saved credential only
when host, port and user are unchanged. Changing the destination requires an
explicit password.
+28 -4
View File
@@ -72,6 +72,11 @@ Buy table column order:
PN вендора | Описание | LOT | Кол-во | Estimate | Склад | Конкуренты | Ручная цена
```
Buy table money cells (Estimate / Склад / Конкуренты / Ручная цена) show the row's line
total (unit × qty), the same as the Sale table's "Цена" — never a unit price; the footer
"Итого" is the sum of those cells. A "Ручная цена" share shown in a row is that row's share
of the manual total, not divided by qty.
Sale table column order:
```
@@ -203,12 +208,17 @@ Configurations have a `config_type` field: `"server"` (default) or `"storage"`.
Rules:
- `config_type` defaults to `"server"` for all existing and new configurations unless explicitly set;
- `config_type` is a free-form code (no enum on client or server); every write path — update, clone,
clone-from-version, rollback, sync push/pull — must carry it unchanged, including codes this client
does not know;
- the configurator page is shared for both types; the SW tab is always visible regardless of type;
- storage configurations use the same vendor_spec + PN→LOT + pricing flow as server configurations;
- storage component categories map to existing tabs: `ENC`/`DKC`/`CTL` → Base, `HIC` → PCI (HIC-карты СХД; `HBA`/`NIC` — серверные, не смешивать), `SSD`/`HDD` → Storage (используют существующие серверные LOT), `ACC` → Accessories (используют существующие серверные LOT), `SW` → SW.
- `DKC` = контроллерная полка (модель СХД + тип дисков + кол-во слотов + кол-во контроллеров); `CTL` = контроллер (кэш + встроенные порты); `ENC` = дисковая полка без контроллера.
- the available config types and their localized names flow from `qt_settings.config_types` on the server;
QF falls back to hardcoded "server/Сервер" and "storage/СХД" when the setting is absent.
QF falls back to hardcoded "server/Сервер" and "storage/СХД" when the setting is absent;
- every "new configuration" type picker (`/configs` and the project page) renders its buttons from
`config_types` — a new type added on the server appears without a client release.
## Server-driven configurator settings (`qt_settings`)
@@ -225,9 +235,12 @@ full contract and JSON schemas.
| `support_pricing` | Price data for the Base tab's support-level picker (see below) |
Rules:
- sync runs as part of the pricelist pull; failure is non-fatal (Warn log only);
- sync runs in the background sync worker (after the pricelist pull) and in the manual full sync
(`POST /api/sync/all`); failure is non-fatal (Warn log only);
- `local_qt_settings` is a read-only cache — never written by user actions;
- absent or unparseable settings: QF uses hardcoded fallbacks for that key only;
- absent or unparseable settings: `GET /api/configurator-settings` serves the Go-side default for that
key only (`internal/handlers/component.go`); this is the single fallback — the configurator page
holds no copy of tabs/categories and builds its tab buttons from the served `tab_config`;
- `config_types[].categories` is an allowlist: a category absent from all types is shown everywhere;
- `qt_categories.name` and `qt_categories.name_ru` are not used by QF runtime; do not depend on them.
@@ -281,7 +294,7 @@ rental calc the same way any BOM line does, with no special-casing needed in tho
a QuoteForge release — see `bible-local/server-contract-qt-settings.md` for the schema; the
fixed list of offerable level×duration codes stays in the frontend, only the price is
server-driven;
- platform (x86 vs HGX-H100/H200 vs HGX-B200 vs HGX-B300) is auto-detected client-side from the
- platform (x86 vs HGX-H100/H200 vs HGX-B200 vs HGX-B300) is auto-detected client-side from the (HGX only for SXM/HGX-named GPUs or H100/H200/B200/B300 non-PCIe; PCIe GPUs like A30 stay x86)
cart's GPU components, the same chip-generation classification used by the rental
depreciation calc (`internal/services/rental.go`), so the picker only offers combinations
valid for the current configuration;
@@ -329,3 +342,14 @@ Rules:
- PN to LOT resolution uses the active local partnumber book;
- canonical persisted mapping is `lot_mappings[]`;
- QuoteForge does not use legacy BOM tables such as `qt_bom`, `qt_lot_bundles`, or `qt_lot_bundle_items`.
## Browser rendering
`web/static/html-escape.js` provides the shared `escapeHtml` for HTML text and
quoted attributes. Never interpolate data into JavaScript event attributes.
Cart and rental controls carry escaped LOT values in `data-lot` and dispatch
through delegated listeners. Sync errors are escaped as text; toast messages use
`textContent`. Run `node scripts/tests/html-security.cjs` for the fast regression check and
`node scripts/tests/html-security-browser.cjs` for real DOM parsing and delegated
events in headless Chrome (`CHROME_BIN` overrides the executable). The browser
check uses synthetic data, controlled API responses and a temporary profile.
+60 -3
View File
@@ -57,7 +57,7 @@ Server-side only (not queried by client runtime):
- `qt_lot_metadata` — component metadata / price settings; the Go server-side component/category management layer (`ComponentRepository`, `CategoryRepository`, `ComponentService`) was removed — no client code reads this table
- `qt_component_usage_stats` — aggregated component popularity stats (written by server jobs)
- `qt_pricing_alerts` — price anomaly alerts (models exist in Go; feature disabled in runtime)
- `qt_schema_migrations` — server migration history (applied via `go run ./cmd/qfs -migrate`)
- `qt_schema_migrations` — server migration history (applied via `make migrate`, see [Migrations](#migrations))
- `qt_scheduler_runs` — server background job tracking (no Go code references it in this repo)
### Competitor subsystem (server-side only, not used by QuoteForge Go code)
@@ -196,6 +196,7 @@ PK: lot_name
| rental_items | JSON | per-lot New/БУ condition for the Аренда tab; see [02-architecture.md](02-architecture.md#rental--paid-testing-pricing-contract) |
| rental_uplift_percent | decimal(8,2) DEFAULT 0 | uplift applied to Estimate buy price for rental "Цена" |
| feature_flags | JSON NULL | added by migration 034. Data features the configuration uses (`["sale_row_markups"]`); derived on save, a client refuses to write a config with a flag it doesn't know. See [decisions/2026-09-24-configuration-feature-flags.md](decisions/2026-09-24-configuration-feature-flags.md) |
| server_updated_at | timestamp(6) NOT NULL | added by migration 035. Server-maintained change marker (`DEFAULT`/`ON UPDATE CURRENT_TIMESTAMP(6)`, index `(server_updated_at, id)`); never written by clients, used for incremental pulls. See [decisions/2026-10-26-incremental-config-project-pull.md](decisions/2026-10-26-incremental-config-project-pull.md) |
### qt_lot_metadata
PK: lot_name
@@ -306,6 +307,7 @@ PK: username
| rental_enabled | tinyint(1) DEFAULT 0 | shows the Аренда tab on this project's configurations |
| created_at | timestamp | |
| updated_at | timestamp ON UPDATE | |
| server_updated_at | timestamp(6) NOT NULL | added by migration 035. Server-maintained change marker (`DEFAULT`/`ON UPDATE CURRENT_TIMESTAMP(6)`, index `(server_updated_at, id)`); never written by clients, used for incremental pulls. See [decisions/2026-10-26-incremental-config-project-pull.md](decisions/2026-10-26-incremental-config-project-pull.md) |
### qt_schema_migrations
| Column | Type | Notes |
@@ -427,5 +429,60 @@ SQLite:
- local migration state is tracked in `local_schema_migrations`.
MariaDB:
- SQL files live in `migrations/`;
- they are applied by `go run ./cmd/qfs -migrate`.
- SQL files live in `migrations/` (embedded into the binary), one file per change,
`NNN_short_topic.sql`, with the recovery/verify header used by the existing files;
filenames containing `local_` are SQLite-only and skipped;
- applied state is tracked per filename in `qt_schema_migrations`;
- commands (always from current source — never run a `bin/` binary of unknown age):
- `make migrate-status` (`qfs -migrate-status`) — list applied/pending, read-only;
- `make migrate` (`qfs -migrate`) — apply pending files in filename order, then exit;
refuses when `qt_schema_migrations` has no applied rows at all (every file would re-run);
- `make migrate-baseline UPTO=<file.sql>` (`qfs -migrate-baseline`) — record files up to
`UPTO` as applied **without running them**, for a server whose schema is already current;
- migrations use their own connection without read/write timeouts (`openMigrationDB`),
not the app's 3s-timeout sync connection — a long backfill must not be cut mid-file;
- server migration commands finish before runtime connection workers start;
- `-migrate` requires a DBA maintenance window with every other writer stopped,
directly granted global ALL PRIVILEGES, and no other connection using the schema.
Do not expand runtime-user privileges; use the existing developer DBA workflow;
- the runner holds a schema advisory lock and one connection for the entire session.
It automatically writes a full consistent snapshot before changes and per-table
snapshots before every file; each archive is restored into a temporary schema
and compared before any corresponding original-schema change. Snapshot or
verification failure stops the run. See [06-backup.md](06-backup.md);
- on a statement/verify failure, per-step snapshots are restored in reverse order,
including migration history; a full-session snapshot is the fallback. A persistent
marker refuses retry after interruption until `qfs -migrate-restore <archive.zip>`
restores and verifies the original state. Connection/recovery failure retains the
marker and archives and emits an explicit restore diagnostic;
- recovery supports InnoDB base tables and their triggers. Views, routines,
events and cross-schema FKs block migrations before changes; new migrations must
stay within supported table SQL. Existing triggers are captured with their
definer, body, order and SQL/charset context, then restored after all data; Verification requires CREATE/DROP DATABASE.
The runner refuses NO_BACKSLASH_ESCAPES sessions;
- each file, its `-- verify: <description> | <SQL>` queries and history insertion run
on one database session, preserving session variables;
- every verify query and the history insert use a fresh GORM statement
(`Session{NewDB: true}`) on that session; chaining them on one statement leaked state
and broke files with two or more verify checks (fixed 2026-10-26 with migration 035);
- a file is recorded only after statements and all verify checks succeed. Verify
queries return rows only for broken invariants; a row or SQL error stops the run
without marking that file applied. Malformed verify headers fail before execution;
- only duplicate column/key names and already-existing objects are tolerated.
FK creation errors (including errno 150/121) remain fatal;
- each file must be safe to re-run after a partial failure (`ADD COLUMN IF NOT EXISTS`,
idempotent `UPDATE ... WHERE <not yet done>`). The statement parser preserves
semicolons in literals/quoted identifiers and strips ordinary comments. Malformed
quotes, executable comments and DELIMITER directives are rejected before execution;
- there is no GORM `AutoMigrate` against MariaDB: it retypes production columns to match the
running binary's models. On 2026-09-24 a stale v1.18 binary run with the old AutoMigrate
`-migrate` did exactly that, which is why it was removed.
## Connection transport
`connection_settings.tls_mode` defaults to `preferred`; `tls_ca_pem` stores an
optional public CA certificate bundle. SQLite startup adds these columns for
legacy settings. Every runtime/setup MariaDB DSN uses `internal/mariadb` to
apply the policy documented in [05-config.md](05-config.md). Private keys are
never part of the CA field. No mode silently falls back to plaintext except
`preferred`.
+24 -2
View File
@@ -62,8 +62,12 @@ They are stored in SQLite and encrypted with `local_encryption.key` unless `QUOT
| --- | --- |
| `-config <path>` | config file path |
| `-localdb <path>` | SQLite path |
| `-reset-localdb` | destructive local DB reset |
| `-migrate` | apply server migrations and exit |
| `-reset-localdb` | reset after mandatory verified backup; refuses unsynced work |
| `-force-reset-localdb` | with `-reset-localdb`, allow clearing unsynced work; backup remains mandatory |
| `-migrate` | apply pending server SQL migrations (`migrations/*.sql`) and exit |
| `-migrate-restore <archive.zip>` | restore and verify a server migration snapshot during DBA maintenance |
| `-migrate-status` | list server SQL migrations with applied/pending state and exit |
| `-migrate-baseline <file>` | record server migrations up to `<file>` as applied without running them, and exit |
| `-version` | print app version and exit |
## First run
@@ -72,3 +76,21 @@ They are stored in SQLite and encrypted with `local_encryption.key` unless `QUOT
2. runtime opens the local SQLite database;
3. if no stored MariaDB credentials exist, `/setup` is served;
4. after setup, runtime works locally and sync uses saved DB settings in the background.
`QFS_RESET_LOCAL_DB` is no longer supported. Reset requires an explicit CLI flag
on each launch. A forced reset still stops if the safety backup cannot be created
and verified. See [06-backup.md](06-backup.md).
## MariaDB TLS
`/setup` stores `tls_mode` and `tls_ca_pem` in SQLite `connection_settings`,
alongside encrypted credentials. Existing and new connections default to
`preferred`, which encrypts when advertised by MariaDB but can fall back to
plaintext and does not authenticate the server. `required` mandates encryption
without verifying the certificate. `verify-ca` requires a PEM CA and validates
the server certificate chain without checking its hostname. `verify-full`
validates both chain and hostname, using the supplied PEM or system roots.
Only the verification modes accept CA input; invalid modes/PEM are rejected.
Use `verify-full` for authenticated server connections. The setup form tests
the selected policy before saving. Sync, setup checks and migration tooling
share the same TLS configuration builder.
+82
View File
@@ -53,3 +53,85 @@ Archive name:
4. replace `config.yaml` if needed;
5. restart the app;
6. re-enter MariaDB credentials if the original encryption key is unavailable.
## Pre-reset safety backup
Before clearing any table, `-reset-localdb` creates a fresh ZIP under
`<backup root>/pre-reset/qfs-pre-reset-v1-<UTC timestamp>-<unique>.zip`, outside
normal retention and regardless of `QFS_BACKUP_DISABLE`. The archive contains
a consistent SQLite snapshot and the selected config when present. Every ZIP
entry is read back for CRC validation; the extracted database must pass SQLite
`integrity_check`. A backup or verification failure prevents the reset. The
backup directory must also resolve outside git after following symlinks.
Reset refuses queued changes and any project/configuration not marked `synced`,
including NULL/legacy states. A missing sync-status column also blocks reset.
Only `-reset-localdb -force-reset-localdb` bypasses this data check. Deletions run
in one transaction and preserve connection settings. Missing legacy tables are
skipped. Use the restore procedure above for the pre-reset archive; retain it
until the reset has been verified. The environment reset switch was removed to
prevent repeated destructive startup.
## Server migration snapshots
`qfs -migrate` writes mandatory archives under
`<backup root>/server-migrations/qfs-mariadb-v1-<label>-<UTC timestamp>-<unique>.zip`.
`QFS_BACKUP_DIR` overrides the root; `QFS_BACKUP_DISABLE` never disables safety
snapshots. Paths are checked outside git, including symlink targets, before
creation and restore. Archives have private filesystem permissions and contain
server table schemas and data, which may be sensitive; never put them in git.
Connection passwords and the local encryption key are not copied into metadata.
One full snapshot precedes the session. Before each file, a separate snapshot
contains a schema definition and row stream for **every** table: this includes
all affected tables even when a legacy Tables-affected header is incomplete.
This intentionally favors recovery over storage/time efficiency for the manual
DBA tool. Retain safety snapshots outside normal rotation until the migration
has been verified; remove verified-session artifacts as operational cleanup.
The versioned ZIP manifest records table schemas, writable column order, row
counts and order-independent row digests. Row streams preserve NULL, empty
strings, binary values and fractional timestamps. Restore temporarily uses
NO_AUTO_VALUE_ON_ZERO to preserve explicit zero IDs and legacy zero dates;
row digests detect coercion and the original session mode is restored. Generated columns are rebuilt
from schema, not inserted. Trigger bodies, definers, firing order and creation
SQL/charset context are preserved. Triggers are recreated after row loading so
restore does not duplicate audit data or alter restored values. CRC/format/digests are checked, then each archive is
restored into a unique temporary schema on the same server and compared for
schema and data equality. The temporary schema is dropped afterward. A failed
or interrupted check must be investigated before retry; abandoned `qfs_verify_`
schemas are DBA cleanup artifacts, never application data.
Stop every other writer for the entire migration/recovery window, including
clients that could reconnect. The runner detects other current schema connections
and uses an advisory lock to exclude a second runner; the lock does not stop
unrelated SQL clients. An active client appearing during failure recovery causes
recovery refusal rather than overwriting its work. InnoDB base tables and triggers are supported; views, routines, events and
cross-schema FKs require explicit recovery support before migration. Triggers
whose recorded database collation differs from the current database collation
also cause refusal until that legacy state has supported recovery. Trigger
routine calls and qualified references to other schemas are refused; source
SQL must have no effects outside the captured schema.
Unsupported schemas stop before changes. Runtime database-user grants stay as
listed in [03-database.md](03-database.md); the tool requires an existing DBA account.
Failure restores step snapshots in reverse order; the full-session archive is
used if step recovery is insufficient. Schema, writable data and migration
history must match the session start before recovery is reported successful.
The persistent `.pending` marker is keyed by configured endpoint and schema, so
server restarts do not hide an interrupted session. Backups survive connection
loss; a failed recovery retains its marker and reports the full snapshot path.
After an interruption, keep clients stopped and use the snapshot path reported
by the runner:
```bash
go run ./cmd/qfs -migrate-restore /outside/repository/server-migrations/qfs-mariadb-v1-session-<timestamp>-<unique>.zip
make migrate-status
make migrate
```
Restore uses the currently saved DBA connection, requires the same database name,
checks the archive by restoring it into a temporary schema first, restores the
original schema, verifies equality and clears the matching pending marker. It
does not require mysql/mysqldump or any operator-installed database utility.
+25 -1
View File
@@ -4,10 +4,18 @@
```bash
go run ./cmd/qfs
go run ./cmd/qfs -migrate
make migrate-status # server SQL migrations: applied/pending
make migrate # apply pending server SQL migrations
go run ./cmd/migrate_project_updated_at
go test ./...
go vet ./...
make security-check # pinned govulncheck, blocks reachable vulnerabilities
# Optional live transport checks; owns disposable Docker containers:
go test -tags integration ./internal/mariadb -run TestMariaDBTLSIntegration -count=1
# Snapshot/DDL recovery checks; owns a disposable MariaDB 11.8 container:
go test -tags integration ./internal/models -run TestSQLMigrationSafetyMariaDB -count=1
# Production-dump rehearsal; source path via QFS_TEST_PRODUCTION_DUMP:
go test -tags integration ./internal/models -run TestProductionDumpMigrationMariaDB -count=1 -v
make build-release
make install-hooks
```
@@ -33,3 +41,19 @@ make install-hooks
Release history belongs under `releases/<version>/RELEASE_NOTES.md`.
Do not keep temporary change summaries in the repository root.
## Toolchain and release security
`go.mod` requires Go 1.27.3 or newer, including security fixes in the standard
library. This minimum also rejects older compilers with `GOTOOLCHAIN=local`;
normal Go toolchain selection downloads a suitable compiler when necessary.
`make security-check` runs `golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...`.
The scanner version is pinned in the Makefile. All release build targets and
`scripts/release.sh` require this check to succeed before producing artifacts.
A scanner/database/download failure also stops release.
`node scripts/tests/release-security.cjs` verifies failure stops all release
entry points before artifact creation using an isolated failing scanner.
When updating dependencies: run `go mod tidy`, `go test ./...`, `go vet ./...`,
`make security-check` and the JavaScript regression check documented in
[02-architecture.md](02-architecture.md).
+1
View File
@@ -16,6 +16,7 @@ Project-specific architecture and operational contracts.
| [09-vendor-spec.md](09-vendor-spec.md) | Vendor BOM and CFXML import contract |
| [10-agent-api-guide.md](10-agent-api-guide.md) | End-to-end API guide for agents pricing servers from a TZ |
| [11-lot-suggestions.md](11-lot-suggestions.md) | lot_suggestion column in qt_vendor_partnumber_seen — write/read contract for manual UI mappings |
| [issue-plan-2026-10-26.md](issue-plan-2026-10-26.md) | Open issue plan, implementation status and validation |
| [decisions/](decisions/README.md) | Architectural decision log, one file per decision |
## Rules
@@ -0,0 +1,52 @@
# Decision: incremental pull of configurations and projects via server_updated_at
**Date:** 2026-10-26
**Status:** active
## Context
Opening the configurations or projects list (`GET /api/configs`, `/api/projects`,
`/api/projects/:uuid/configs`, on-demand with a 30s cooldown) and the manual sync ran
`ImportConfigurationsToLocal` / `ImportProjectsToLocal`, which read **every** server row
and rewrote it locally. Measured on production (2026-10-26): configurations 22.2s
(4158 rows, ~3.2 MB, 4078 rewritten for one actual change), projects 4.6s (1692 rows),
holding a pool connection the whole time. `qt_configurations` had no change column;
`qt_projects.updated_at` is set by clients (`SaveProjectPreservingUpdatedAt`), so a change
made offline and pushed later carries an old timestamp and cannot drive a filter.
## Decision
- Server migration `035_add_server_updated_at.sql` adds
`server_updated_at TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6)`
plus index `(server_updated_at, id)` to `qt_configurations` and `qt_projects`. MariaDB
maintains it; no client writes it and it is **not** in `models.Configuration` /
`models.Project`, so writes from any client version (GORM `Create`/`Save`, raw
`UPDATE` with column lists) keep working and bump the marker on every real change.
- Client (`internal/services/sync/pull_cursor.go`):
- if the column is missing (server not migrated yet) every pull is full, exactly as
before; presence is re-checked until found, then cached for the process;
- the first pull of each table per process is full (restores rows lost locally once per
app start), later pulls read `server_updated_at >= cursor - 120s` in one query;
- the cursor is the server's `NOW(6)` read *before* the pull, stored in `app_settings`
(`server_pull_cursor:<table>`) together with `host:port/database`; a cursor from a
different server is ignored;
- the per-row merge (skip `pending`, keep local deactivation sticky, keep `line`) is
unchanged; only the set of rows read differs.
- Old clients keep doing full pulls and are unaffected.
## Consequences
- Idle pulls go from ~27s / 3.2 MB to one indexed range query.
- Hard deletes on the server are not propagated — same as the full pull before.
- Changes applied by InnoDB `ON UPDATE CASCADE` (e.g. a `qt_projects.uuid` change
cascading to `qt_configurations.project_uuid`) do not bump `server_updated_at`; they are
picked up by the next full pull (app start).
- Cursor comparison happens in the server session time zone. Production is `SYSTEM`/MSK
(no DST); on a server with DST, rows changed around the autumn transition could be
missed until the next app start.
- Applying 035 adds the column and builds an index on both tables (~4k and ~2k rows on
production, seconds); `qfs -migrate` is run once by the developer as usual.
- Tests: `internal/services/sync/incremental_pull_mariadb_test.go` (needs
`QF_TEST_MARIADB_DSN`, a disposable MariaDB with the production table DDL; verified on
MariaDB 11.8, same as production), migrator fix in
`internal/models/sql_migration_verify_test.go` (`two checks pass`).
@@ -0,0 +1,54 @@
# Decision: one MariaDB pool per client — reuse, never orphan
**Date:** 2026-10-26
**Status:** active
## Context
A single client held ~30 MariaDB connections during sync. `ConnectionManager.GetDB`
treated a pool older than `pingInterval` (30s) as unusable and dialed a brand-new
`*gorm.DB` without closing the previous one. Sync calls `getDB()` per step, so every
~30s a new pool appeared while the old one kept its idle connections until
`ConnMaxLifetime` (5 min): ~10 orphaned pools × 2 idle + up to 10 active ≈ 30. Each
orphaned `*sql.DB` also leaked its opener goroutine. `TryConnect`, the background
ping-failure path and a failed ping inside `dial` dropped pools the same way.
## Decision
- `GetDB` pings the existing pool when it is merely stale and returns it; it dials only
when there is no pool or the ping fails.
- Whenever the manager replaces or drops a pool (`GetDB`, `TryConnect`,
`checkOnlineNow`, `MarkOffline`, `Disconnect`, failed ping in `dial`) the old
`*sql.DB` is closed.
- Pool limits: `MaxOpenConns=2`, `MaxIdleConns=2`, `ConnMaxLifetime=5m`,
`ConnMaxIdleTime=1m` — one client holds at most 2 connections, idle ones are released
within a minute. Sync is sequential (one query at a time); the second connection lets
UI server queries (active viewers, users status) run alongside a long sync step.
- Background sync interval (`backgroundSyncInterval` in `cmd/qfs/main.go`) is 1 minute
with ±30% random jitter (`sync.WorkerJitter`), so clients do not hit MariaDB in
lockstep. The next cycle is scheduled from the *start* of the previous one, so
heartbeats (`qt_client_schema_state.last_checked_at`) are at most 1.3 × interval apart
while a cycle is shorter than that.
- `/api/sync/users-status` online threshold = interval × (1 + `WorkerJitter`).
- `PullPartnumberBooks` reads only book headers (`id, version, created_at, is_active`)
each cycle; `partnumbers_json` (~5 MB for all books) is fetched per book only when the
book is new or its local items are incomplete. An empty book counts as complete. An
incomplete book is re-pulled at most once per process per version: a server book can
list partnumbers missing from `qt_partnumber_book_items` (seen: `BOT_AGIBOT_G2_PERF`
in `PNBOOK-2026-10-21-001`), which a re-pull cannot fix; this is logged as `WARN` with
the missing partnumbers.
- A worker cycle sends one heartbeat: the worker calls `EnsureReadinessForSync` once and
then the unexported `pushPendingChanges` (the exported `PushPendingChanges` still checks
readiness itself for the manual-sync handlers).
- `NeedSync` asks for the latest active pricelist of all three sources in one
`UNION ALL` query (`GetLatestActiveIDsBySources`): 0.71s → 0.24s.
- `SyncQtSettings` skips the local `DELETE` + `INSERT` when the server rows equal the cache.
- Measured against the production server: a cold sync took 171s (pricelists 105s,
partnumber books 38s, configurations 23s) and is bound by server reads, not pool
size. A warm cycle takes ~1.4s (was ~2.5s plus a 5 MB download).
## Consequences
- Code that keeps a `*gorm.DB` across a reconnect gets `sql: database is closed`; sync
steps must call `getDB()` per operation instead of caching the handle.
- Tests: `internal/db/connection_test.go`, `internal/services/sync/worker_jitter_test.go`.
@@ -0,0 +1,62 @@
# Decision: verified snapshots and whole-session recovery for server migrations
**Date:** 2026-10-26
**Status:** active
## Context
Issue #3 fixes FK error propagation, session variables and verify execution.
Its proposed manual-backup exception conflicts with the shared go-database
contract. MariaDB DDL cannot be made rollback-safe by wrapping the migration
files in an ordinary transaction: see [MariaDB implicit commits](https://mariadb.com/docs/server/reference/sql-statements/transactions/sql-statements-that-cause-an-implicit-commit).
## Decision
The migration engine, rather than the operator, owns full-session and pre-file
per-table backups. A Go ZIP/JSON snapshot format avoids external database-client
dependencies. Every archive is restored into a temporary schema and compared
before the corresponding original-schema changes. Per-file archives include all
tables to cover incomplete legacy affected-table headers and dynamic SQL.
On failure, restore each step snapshot in reverse order; use the full-session
snapshot as fallback. Include migration history in recovery. Persist a recovery
marker before applying any changes, refuse an interrupted session on retry and
ship an explicit `-migrate-restore` path. Restore failures retain archives and
produce diagnostics instead of claiming success.
The production dump contains 27 triggers, so the implementation preserves
InnoDB tables and trigger bodies, definers, order and SQL/charset context.
Triggers are recreated after row loading to avoid firing them during restore;
verification binds them to the temporary schema and performs no later DML.
Views, routines, events, cross-schema FKs and legacy trigger/database collation
mismatches are refused before changes. It requires an existing
DBA account with directly granted global ALL PRIVILEGES so object/client
visibility cannot silently hide unsupported state, plus a maintenance window
without other writers. Do not grant these privileges to the runtime user. The
advisory migration lock excludes other runners, not arbitrary SQL clients.
The parser supports SQL literals and quoted identifiers containing semicolons,
escaped/doubled quotes and ordinary comments. It rejects malformed input,
executable comments and DELIMITER rather than silently misinterpreting them.
Existing merged SQL files are unchanged.
## Consequences
Safety snapshots and restore verification add disk usage, temporary server schema
creation and runtime to the developer-only migration command. Normal local-first
CRUD and sync gain no MariaDB dependency. Migration commands start no background
connection worker. Before production use, rehearse on a current production dump
in MariaDB matching production version and collation; synthetic tests alone do
not satisfy that gate. Unsupported schema features need explicit recovery support
before such schemas may be migrated by this tool.
## Acceptance
Rehearsed on the supplied 2026-10-26 production dump (45 tables, 27 triggers),
using an isolated MariaDB 11.8.3 container and utf8mb4_uca1400_ai_ci. The test
verifies full restore, actual migration 035 on the reconstructed pre-035 clone,
reverse recovery after a later DDL/verify failure, successful retry and exact
restoration of the original dump state. Synthetic tests additionally exercise
FK errno 150, interrupted-session recovery, NULL/empty/binary/decimal/timestamp
values, zero auto-increment IDs and legacy zero dates. No production changes
are performed by these checks.
+130
View File
@@ -0,0 +1,130 @@
# Issue implementation plan - 2026-10-26
Repository: https://git.mchus.pro/mchus/QuoteForge
Scope: issues #2-#7. Initial fixes were merged into `main` in
[PR #8](https://git.mchus.pro/mchus/QuoteForge/pulls/8), with release notes in v3.50.
The follow-up adds completed acceptance checks and server migration recovery
required by the shared rules. Each issue has its own implementation/acceptance commit.
| Order | Issue | Implementation and acceptance | Status |
| --- | --- | --- | --- |
| 1 | [#2](https://git.mchus.pro/mchus/QuoteForge/issues/2) | Block foreign Host/Origin/Referer and cross-site browser requests in both routers; validate API body types; refuse saved-password reuse for a changed host/port/user. Middleware and real setup/runtime router tests reject attacks without changing saved credentials or user data. | Acceptance passed |
| 2 | [#4](https://git.mchus.pro/mchus/QuoteForge/issues/4) | Move LOT-bearing cart/rental events to delegated listeners and escaped data attributes; escape BOM and sync output; share HTML escaping; render toast as text. Malicious LOT regression checks and headless Chrome DOM checks pass. | Acceptance passed |
| 3 | [#7](https://git.mchus.pro/mchus/QuoteForge/issues/7) | Require a fresh verified archive outside rotation before reset, refuse unsynced/legacy states without explicit force, use transactional deletion, remove environment-triggered reset. Verified refusal, ZIP restore, backup failure, rollback and CLI startup after restore. | Acceptance passed |
| 4 | [#5](https://git.mchus.pro/mchus/QuoteForge/issues/5) | Persist TLS mode/CA in SQLite, expose setup controls, share policy across setup/sync/migration DSNs. Verified certificate trust, hostname checks, fallback policy, settings round-trip and live MariaDB 11.8 transport. | Acceptance passed |
| 5 | [#3](https://git.mchus.pro/mchus/QuoteForge/issues/3) | Keep each migration file in one session, execute verify headers before history insertion, propagate FK failures. Failed queries/checks restore the entire session. Verified snapshots, literal-aware SQL parsing, trigger preservation and crash recovery pass synthetic tests and production-dump rehearsal on MariaDB 11.8.3. | Acceptance passed |
| 6 | [#6](https://git.mchus.pro/mchus/QuoteForge/issues/6) | Require Go 1.27.3, update vulnerable dependencies, block release on pinned govulncheck. Tests, vet, security scan, cross-platform builds and release failure checks pass. | Acceptance passed |
## Decisions
- HTTP forms remain supported on setup and multipart vendor import; JSON APIs
reject non-JSON bodies. CLI clients may omit browser origin headers.
- CSP without inline scripts remains outside this patch; the issue lists it as optional.
- Forced reset still requires a verified backup. `QFS_RESET_LOCAL_DB` is ignored,
with a startup warning; only explicit CLI flags can initiate reset.
- TLS defaults to `preferred` for existing-server compatibility, as requested in
the issue. It permits plaintext fallback; use `verify-full` for authenticated
encryption. The setup form explains this distinction.
- `go 1.27.3` enforces the minimum compiler, including with `GOTOOLCHAIN=local`.
A redundant `toolchain go1.27.3` is removed by Go tooling and is unnecessary.
## Validation and handoff
The original implementation PR is merged. Current acceptance results are listed
below. Govulncheck reports zero reachable vulnerabilities and one module-level
advisory whose affected code is not called. Initial cross-platform builds and
all Go/JavaScript checks pass; final verification also covers the recovery change.
Production-dump rehearsal also passes on MariaDB 11.8.3 with the production
collation. No production migration or new release is performed by this work.
## Acceptance follow-up
### Issue #2
`go test ./cmd/qfs -run TestRequestBoundaryOnApplicationRouters -count=1`
exercises the production setup and runtime route constructors with temporary
SQLite state. Foreign setup forms, opaque origins and rebinding hosts are
rejected, text/plain JSON receives 415, and both setup handlers reject saved
password reuse for a changed destination. Local reads still succeed. Saved
credentials, project/configuration counts and the pending queue remain unchanged.
The setup router construction was extracted without changing its routes so
this regression exercises the same middleware registration as startup.
### Issue #4
`node scripts/tests/html-security-browser.cjs` passes in headless Chrome with
an isolated temporary profile. It executes the actual template rendering
functions in a browser DOM with controlled API responses. Both cart layouts,
rental, parsed BOM, toast, pending sync errors, error history and repair errors
preserve malicious strings as text. Real click/change/focus events preserve
LOT values through delegated handlers; no injected elements or payload execution
are observed. This is a rendering integration check, not a full application
navigation test. Set `CHROME_BIN` outside macOS to select a Chrome executable.
### Issue #7
`go test ./cmd/qfs ./internal/localdb -run TestReset -count=1` covers reset
safety and a subprocess executing the real `main` flag/startup path. With
rotating backups disabled, the environment switch does not reset pending work,
normal reset refuses it, and explicit forced reset creates a verified archive.
The test restores ZIP contents and reopens the database through normal startup:
connection settings and the pending queue survive. It deliberately selects
`-migrate-status` with an unavailable loopback database to stop startup before
HTTP/browser launch; no production database or user state is touched.
### Issue #5
`go test -tags integration ./internal/mariadb -run TestMariaDBTLSIntegration -count=1`
starts its own disposable MariaDB 11.8 containers on random loopback ports, with
TLS enabled and disabled, and removes both afterward. The test verifies the
actual session cipher: preferred encrypts with a TLS server and falls back only
with a plaintext server; required and verification modes reject plaintext.
Correct trust succeeds, unrelated roots and wrong hostname fail. Certificates,
private keys and database storage are temporary; no saved user DSN is used.
The matching image must already exist locally (`docker pull mariadb:11.8`).
Setup policy, local settings and synthetic TLS tests also pass.
### Issue #6
Go 1.27.3: `go test ./...`, `go vet ./...` and `make security-check` pass.
CGO-free builds produce macOS arm64 Mach-O and Windows amd64 PE executables
outside the repository. `node scripts/tests/release-security.cjs` substitutes a
failing Go scanner in an isolated directory and confirms all five release build
targets and `scripts/release.sh` stop before compilation or artifact creation.
The scanner reports zero reachable vulnerabilities and one uncalled module-level
advisory. No release is published by these acceptance checks.
### Issue #3
The parser, mandatory server snapshots, restore-readiness checks, reverse step
recovery and explicit crash restore are implemented; see
[server migration recovery decision](decisions/2026-10-26-server-migration-recovery.md).
`go test -tags integration ./internal/models -run TestSQLMigrationSafetyMariaDB -count=1`
passes on a disposable MariaDB 11.8: session variables and two verify checks,
semicolon literals, a later failed verify, a real FK errno 150 after successful
DDL, reverse recovery of prior files, interrupted-session refusal/restore/retry,
unsupported-schema refusal, active-client refusal and backup failure.
Restored schema, binary/NULL/empty/decimal/timestamp data and history are compared
to the original snapshot. Unit tests cover snapshot format/digests, symlink path
refusal and all embedded migration SQL syntax.
The supplied `2026-10-26_RFQ_LOG.sql` dump (45 tables, 27 triggers) is loaded
read-only into a new disposable MariaDB **11.8.3** container with
`utf8mb4_uca1400_ai_ci`, matching the production version and collation.
`TestProductionDumpMigrationMariaDB` verifies a full snapshot restore, rebuilds
the pre-035 schema only in that clone, applies the real embedded migration 035,
forces a subsequent DDL/verify failure, verifies reverse recovery against the
complete pre-session snapshot, then applies 035 successfully and restores the
exact original dump state. Trigger bodies, definers and order are preserved;
triggers never fire during row loading. No production schema or dump file is changed.
Run this optional, destructive-to-its-own-container rehearsal with an existing
`mariadb:11.8.3` image and `QFS_TEST_PRODUCTION_DUMP=/outside/repository/dump.sql`:
```bash
go test -tags integration ./internal/models -run TestProductionDumpMigrationMariaDB -count=1 -v
```
The environment variable supplies the source file only; no externally supplied
DSN is accepted. The test owns and removes its container and temporary archives.
+2
View File
@@ -75,6 +75,8 @@ Update this file in the same commit as any change to the flows below.
- `POST /api/sync/push` drains the queue by writing to MariaDB.
- If a push fails, `increment_attempts` and `last_error` are updated; row stays in queue.
- `RepairPendingChanges` reconciles orphaned changes (configuration/project deleted locally).
- Background cycle (`sync.Worker.runSync`) runs independent streams: push (gated by readiness guard), then pricelists, partnumber books, qt_settings. A failure in one stream is logged and never stops the others.
- Configurations left `sync_status=pending` with no `pending_changes` row (queue row dropped, e.g. a stale event skipped without marking synced) are re-enqueued as an `update` with the current snapshot at the start of each push (`requeueOrphanedPendingConfigurations`).
---
+37 -17
View File
@@ -43,7 +43,7 @@ category codes that are allowed for each type. QF uses this for:
"MB","CPU","MEM","RAID",
"SSD","HDD","M2","EDSFF","HHHL",
"GPU","NIC","HCA","DPU","HBA",
"PSU","PS","ACC","RISERS","CARD","BB"
"PSU","PS","ACC","RISERS","CARD","BB","SFP"
]
},
{
@@ -53,7 +53,7 @@ category codes that are allowed for each type. QF uses this for:
"categories": [
"DKC","CPU","MEM","PS",
"SSD","HDD","M2","EDSFF","HHHL",
"NIC","HBA","HCA","ACC","CARD"
"NIC","HBA","HCA","ACC","CARD","SFP"
]
}
]
@@ -109,7 +109,16 @@ single-select mode.
]
},
{ "key": "power", "label": "Power", "single_select": false, "categories": ["PS","PSU"] },
{ "key": "accessories", "label": "Accessories", "single_select": false, "categories": ["ACC","CARD"] },
{
"key": "accessories",
"label": "Accessories",
"single_select": false,
"categories": ["ACC","CARD","SFP"],
"sections": [
{ "title": "Аксессуары", "categories": ["ACC","CARD"] },
{ "title": "Трансиверы", "categories": ["SFP"] }
]
},
{ "key": "sw", "label": "SW", "single_select": false, "categories": ["SW"] }
]
```
@@ -126,9 +135,15 @@ contain any items. Other tabs are hidden when empty.
**Value format:** JSON string array.
```json
["base", "storage", "pci"]
["base"]
```
"Empty" is evaluated after the per-`config_type` category filter, so with only
`base` listed a tab such as Storage/PCI shows for a server configuration (its
categories have components) and hides for a type whose allowlist excludes all of
its categories (e.g. networking). Listing a tab here forces it visible for every
config type.
---
### `required_categories`
@@ -173,19 +188,19 @@ the regulation and the picker won't show it.
},
"hgx_price": {
"HGX-H200": {
"B": {"1": 20000, "3": 55000, "5": 80000},
"S": {"1": 28000, "3": 70000, "5": 100000},
"P": {"1": 63000, "3": 105000}
"B": {"1": 28000, "3": 77000, "5": 112000},
"S": {"1": 39500, "3": 98000, "5": 140000},
"P": {"1": 88500, "3": 147000}
},
"HGX-B200": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 101000, "3": 176000}
"B": {"1": 158500, "3": 230500, "5": 284500},
"S": {"1": 176500, "3": 266500, "5": 332500},
"P": {"1": 232000, "3": 322000}
},
"HGX-B300": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 112000, "3": 187000}
"B": {"1": 192000, "3": 270000, "5": 328500},
"S": {"1": 211500, "3": 309000, "5": 380500},
"P": {"1": 286000, "3": 383500}
}
}
}
@@ -200,10 +215,15 @@ current as the regulation evolves.
## Backward compatibility
- If `qt_settings` does not exist (old server): QF logs `Warn` during sync and
leaves `local_qt_settings` empty. The frontend falls back to hardcoded defaults
for all four settings. No crash, no data loss.
- If a specific key is absent from `qt_settings`: QF falls back to the hardcoded
default for that key only.
leaves `local_qt_settings` empty. `GET /api/configurator-settings` then serves
the Go-side defaults (`internal/handlers/component.go`, `default*()`) for every
setting. No crash, no data loss.
- If a specific key is absent from `qt_settings`: the same Go-side default is served
for that key only.
- The Go-side defaults are the **only** fallback and are not kept in step with the
server — change tabs/categories in `qt_settings`, not in code. The configurator page
has no JS copy: `TAB_CONFIG` (except "Other") and the tab buttons are built from the
served `tab_config`; a config type with no `config_types` allowlist sees all categories.
- Old QF clients that do not know about `local_qt_settings` continue to use their
hardcoded JS constants unchanged.
+40 -78
View File
@@ -46,7 +46,7 @@ import (
var Version = "dev"
var errVendorImportTooLarge = errors.New("vendor workspace file exceeds 1 GiB limit")
const backgroundSyncInterval = 5 * time.Minute
const backgroundSyncInterval = time.Minute
const onDemandPullCooldown = 30 * time.Second
const startupConsoleWarning = "Не закрывайте консоль иначе приложение не будет работать"
@@ -59,8 +59,12 @@ func main() {
configPath := flag.String("config", "", "path to config file (default: user state dir or QFS_CONFIG_PATH)")
localDBPath := flag.String("localdb", "", "path to local SQLite database (default: user state dir or QFS_DB_PATH)")
resetLocalDB := flag.Bool("reset-localdb", false, "reset local SQLite data on startup (keeps connection settings)")
migrate := flag.Bool("migrate", false, "run database migrations")
resetLocalDB := flag.Bool("reset-localdb", false, "reset local SQLite data after a verified safety backup (keeps connection settings)")
forceResetLocalDB := flag.Bool("force-reset-localdb", false, "allow -reset-localdb to clear unsynced work after a verified backup")
migrate := flag.Bool("migrate", false, "apply pending server (MariaDB) SQL migrations and exit")
migrateRestore := flag.String("migrate-restore", "", "restore a verified server migration snapshot during maintenance and exit")
migrateStatus := flag.Bool("migrate-status", false, "list server SQL migrations with applied/pending state and exit")
migrateBaseline := flag.String("migrate-baseline", "", "record server SQL migrations up to and including this file as applied, without running them, and exit")
version := flag.Bool("version", false, "show version information")
flag.Parse()
@@ -110,8 +114,11 @@ func main() {
}
}
if shouldResetLocalDB(*resetLocalDB) {
if err := localdb.ResetData(resolvedLocalDBPath); err != nil {
if os.Getenv("QFS_RESET_LOCAL_DB") != "" {
slog.Warn("QFS_RESET_LOCAL_DB is ignored; use -reset-localdb explicitly")
}
if *resetLocalDB {
if err := localdb.ResetDataWithOptions(resolvedLocalDBPath, resolvedConfigPath, *forceResetLocalDB); err != nil {
slog.Error("failed to reset local database", "error", err)
os.Exit(1)
}
@@ -126,6 +133,10 @@ func main() {
// Check if running in setup mode (no connection settings)
if !local.HasSettings() {
if *migrate || *migrateStatus || *migrateBaseline != "" || *migrateRestore != "" {
slog.Error("server migration command requires saved DBA connection settings")
os.Exit(1)
}
slog.Info("no database settings found, starting setup mode")
runSetupMode(local)
return
@@ -168,13 +179,6 @@ func main() {
// Create connection manager. Runtime stays local-first; MariaDB is used on demand by sync/setup only.
connMgr := db.NewConnectionManager(local)
// Keep the online-status cache fresh in the background so request-handling
// goroutines (health checks, middleware, etc.) never block on the MySQL
// dial/read timeout themselves.
connMgrCtx, connMgrCancel := context.WithCancel(context.Background())
defer connMgrCancel()
connMgr.Start(connMgrCtx)
dbUser := local.GetDBUser()
slog.Info("starting QuoteForge server",
@@ -185,28 +189,36 @@ func main() {
"online", false,
)
if *migrate {
mariaDB, err := connMgr.GetDB()
if *migrate || *migrateStatus || *migrateBaseline != "" || *migrateRestore != "" {
mariaDB, err := openMigrationDB(local)
if err != nil {
slog.Error("cannot run migrations: database not available", "error", err)
os.Exit(1)
}
if mariaDB == nil {
slog.Error("cannot run migrations: database not available")
switch {
case *migrateRestore != "":
err = models.RestoreSQLMigrationSession(mariaDB, *migrateRestore)
case *migrateStatus:
err = runMigrateStatus(mariaDB)
case *migrateBaseline != "":
err = runMigrateBaseline(mariaDB, *migrateBaseline)
default:
err = runMigrateApply(mariaDB, resolvedLocalDBPath)
}
if err != nil {
slog.Error("migration command failed", "error", err)
os.Exit(1)
}
slog.Info("running database migrations...")
if err := models.Migrate(mariaDB); err != nil {
slog.Error("migration failed", "error", err)
os.Exit(1)
}
if err := models.SeedCategories(mariaDB); err != nil {
slog.Error("seeding categories failed", "error", err)
os.Exit(1)
}
slog.Info("migrations completed")
os.Exit(0)
}
// Keep the online-status cache fresh in the background so request-handling
// goroutines (health checks, middleware, etc.) never block on the MySQL
// dial/read timeout themselves.
connMgrCtx, connMgrCancel := context.WithCancel(context.Background())
defer connMgrCancel()
connMgr.Start(connMgrCtx)
gin.SetMode(cfg.Server.Mode)
restartSig := make(chan struct{}, 1)
@@ -303,22 +315,6 @@ func showStartupConsoleWarning() {
fmt.Printf("\033]0;%s\007", startupConsoleWarning)
}
func shouldResetLocalDB(flagValue bool) bool {
if flagValue {
return true
}
value := strings.TrimSpace(os.Getenv("QFS_RESET_LOCAL_DB"))
if value == "" {
return false
}
switch strings.ToLower(value) {
case "1", "true", "yes", "y":
return true
default:
return false
}
}
func derefString(value *string) string {
if value == nil {
return ""
@@ -553,46 +549,12 @@ func nextBackupTime(now time.Time, hour, minute int) time.Time {
func runSetupMode(local *localdb.LocalDB) {
restartSig := make(chan struct{}, 1)
// In setup mode, we don't have a connection manager yet (will restart after setup)
templatesPath := filepath.Join("web", "templates")
setupHandler, err := handlers.NewSetupHandler(local, nil, templatesPath, restartSig)
router, err := setupSetupRouter(local, restartSig)
if err != nil {
slog.Error("failed to create setup handler", "error", err)
slog.Error("failed to create setup router", "error", err)
os.Exit(1)
}
gin.SetMode(gin.ReleaseMode)
router := gin.New()
router.Use(gin.Recovery())
if staticFS, err := qfassets.StaticFS(); err == nil {
router.StaticFS("/static", http.FS(staticFS))
} else {
slog.Error("failed to load embedded static assets", "error", err)
os.Exit(1)
}
// Setup routes only
router.GET("/", func(c *gin.Context) {
c.Redirect(http.StatusFound, "/setup")
})
router.GET("/setup", setupHandler.ShowSetup)
router.POST("/setup", setupHandler.SaveConnection)
router.POST("/setup/test", setupHandler.TestConnection)
router.GET("/setup/status", setupHandler.GetStatus)
settingsHandler := handlers.NewSettingsHandler(local)
router.GET("/api/settings/ui", settingsHandler.GetUI)
router.PUT("/api/settings/ui", settingsHandler.PutUI)
// Health check
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "setup_required",
"time": time.Now().UTC().Format(time.RFC3339),
})
})
addr := "127.0.0.1:8080"
slog.Info("starting setup mode server", "address", addr, "version", Version)
+134
View File
@@ -0,0 +1,134 @@
package main
import (
"fmt"
"io/fs"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/models"
mysqlDriver "github.com/go-sql-driver/mysql"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// Server (MariaDB) migration commands. Each runs only the embedded migrations/*.sql
// files tracked in qt_schema_migrations — never GORM AutoMigrate, which can silently
// retype production columns to match whatever models the running binary was built with.
// See bible-local/03-database.md "Migrations".
// openMigrationDB opens a dedicated MariaDB connection for migrations. The app's shared
// connection (localdb.GetDSN) has 3s read/write timeouts for offline-first sync; a
// backfill UPDATE over qt_pricelist_items needs minutes, and a client-side timeout
// mid-statement leaves the file half-applied and unrecorded.
func openMigrationDB(local *localdb.LocalDB) (*gorm.DB, error) {
dsn, err := local.GetDSN()
if err != nil {
return nil, fmt.Errorf("server connection settings: %w", err)
}
cfg, err := mysqlDriver.ParseDSN(dsn)
if err != nil {
return nil, fmt.Errorf("parse DSN: %w", err)
}
cfg.Timeout = 10 * time.Second
cfg.ReadTimeout = 0
cfg.WriteTimeout = 0
db, err := gorm.Open(mysql.Open(cfg.FormatDSN()), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return nil, fmt.Errorf("connect to server: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return nil, fmt.Errorf("migration pool: %w", err)
}
sqlDB.SetMaxOpenConns(1)
sqlDB.SetMaxIdleConns(1)
return db, nil
}
// runMigrateStatus prints every server migration and whether it is applied.
func runMigrateStatus(db *gorm.DB) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
states, err := models.SQLMigrationStatus(db, migrations)
if err != nil {
return err
}
pending := 0
for _, st := range states {
if st.AppliedAt != nil {
fmt.Printf(" applied %s (%s)\n", st.Filename, st.AppliedAt.Format("2006-01-02 15:04"))
} else {
fmt.Printf(" PENDING %s\n", st.Filename)
pending++
}
}
fmt.Printf("%d migrations, %d pending\n", len(states), pending)
return nil
}
// runMigrateApply applies pending server migrations. Refuses on a server with no
// migration history at all: its schema was built by other means, so every file would be
// re-run — record a baseline with -migrate-baseline first.
func runMigrateApply(db *gorm.DB, localDBPath string) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
if err := requireMigrationHistory(db, migrations); err != nil {
return err
}
backupRoot, err := appstate.ServerMigrationBackupDir(localDBPath)
if err != nil {
return err
}
applied, err := models.RunSQLMigrations(db, migrations, func(filename string) {
fmt.Printf(" applying %s ...\n", filename)
}, backupRoot)
if err != nil {
return err
}
if len(applied) == 0 {
fmt.Println("server schema is up to date — nothing to apply")
return nil
}
fmt.Printf("applied %d migration(s)\n", len(applied))
return nil
}
// runMigrateBaseline records migrations up to and including upto as applied without
// running them.
func runMigrateBaseline(db *gorm.DB, upto string) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
recorded, err := models.BaselineSQLMigrations(db, migrations, upto)
for _, f := range recorded {
fmt.Printf(" recorded %s (not executed)\n", f)
}
if err != nil {
return err
}
fmt.Printf("baseline: %d migration(s) recorded as applied, up to %s\n", len(recorded), upto)
return nil
}
func requireMigrationHistory(db *gorm.DB, migrations fs.FS) error {
states, err := models.SQLMigrationStatus(db, migrations)
if err != nil {
return err
}
for _, st := range states {
if st.AppliedAt != nil {
return nil
}
}
return fmt.Errorf("qt_schema_migrations has no applied migrations, so all %d files would run against this server. "+
"Record what the server already has first: qfs -migrate-baseline <last-applied-file.sql> (make migrate-baseline UPTO=...)", len(states))
}
+72
View File
@@ -0,0 +1,72 @@
package main
import (
"net/http/httptest"
"reflect"
"strings"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/config"
"github.com/gin-gonic/gin"
)
func TestRequestBoundaryOnApplicationRouters(t *testing.T) {
moveToRepoRoot(t)
local, connMgr, _ := newAPITestStack(t)
if err := local.SaveSettings("db.example", 3306, "example", "example", "example"); err != nil {
t.Fatal(err)
}
cfg := &config.Config{}
setConfigDefaults(cfg)
runtimeRouter, _, err := setupRouter(cfg, local, connMgr, "example", nil)
if err != nil {
t.Fatal(err)
}
setupRouter, err := setupSetupRouter(local, nil)
if err != nil {
t.Fatal(err)
}
for name, router := range map[string]*gin.Engine{"setup": setupRouter, "runtime": runtimeRouter} {
t.Run(name, func(t *testing.T) {
before, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
projects, configs, pending := local.CountProjects(), local.CountConfigurations(), local.CountPendingChanges()
for _, tt := range []struct {
name, method, path, host, origin, contentType, body string
want int
}{
{"foreign setup save", "POST", "/setup", "", "https://attacker.example", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 403},
{"foreign setup test", "POST", "/setup/test", "", "https://attacker.example", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 403},
{"rebind settings read", "GET", "/setup/status", "attacker.example:8080", "", "", "", 403},
{"opaque origin", "PUT", "/api/settings/ui", "", "null", "application/json", `{"indicator_mode":"accessible"}`, 403},
{"plain JSON", "PUT", "/api/settings/ui", "", "", "text/plain", `{"indicator_mode":"accessible"}`, 415},
{"saved password destination save", "POST", "/setup", "", "", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 422},
{"saved password destination test", "POST", "/setup/test", "", "", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 422},
{"local read", "GET", "/api/settings/ui", "", "http://127.0.0.1:8080", "", "", 200},
} {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, "http://127.0.0.1:8080"+tt.path, strings.NewReader(tt.body))
if tt.host != "" {
req.Host = tt.host
}
req.Header.Set("Origin", tt.origin)
req.Header.Set("Content-Type", tt.contentType)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != tt.want {
t.Fatalf("status=%d want=%d body=%s", w.Code, tt.want, w.Body.String())
}
})
}
after, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(before, after) || projects != local.CountProjects() || configs != local.CountConfigurations() || pending != local.CountPendingChanges() {
t.Fatal("rejected requests changed local state")
}
})
}
}
+130
View File
@@ -0,0 +1,130 @@
package main
import (
"archive/zip"
"context"
"flag"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
)
func TestResetCLIHelper(t *testing.T) {
if os.Getenv("QFS_RESET_CLI_HELPER") != "1" {
return
}
for i, arg := range os.Args {
if arg == "--" {
os.Args = append([]string{"qfs"}, os.Args[i+1:]...)
flag.CommandLine = flag.NewFlagSet("qfs", flag.ExitOnError)
main()
return
}
}
t.Fatal("missing CLI arguments")
}
func TestResetCLIAndRestore(t *testing.T) {
dir := t.TempDir()
t.Setenv("QFS_STATE_DIR", dir)
t.Setenv("QFS_BACKUP_DISABLE", "1")
t.Setenv("QFS_RESET_LOCAL_DB", "yes")
t.Setenv("QFS_BACKUP_DIR", filepath.Join(dir, "backups"))
dbPath := filepath.Join(dir, "qfs.db")
configPath := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(configPath, []byte("server:\n host: 127.0.0.1\n"), 0600); err != nil {
t.Fatal(err)
}
local, err := localdb.New(dbPath)
if err != nil {
t.Fatal(err)
}
if err := local.SaveSettings("127.0.0.1", 1, "example", "example", "example"); err != nil {
t.Fatal(err)
}
if err := local.AddPendingChange("configuration", "example", "update", "{}"); err != nil {
t.Fatal(err)
}
if err := local.Close(); err != nil {
t.Fatal(err)
}
run := func(args ...string) string {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
argv := append([]string{"-test.run=^TestResetCLIHelper$", "--", "-localdb", dbPath, "-config", configPath, "-migrate-status"}, args...)
cmd := exec.CommandContext(ctx, os.Args[0], argv...)
cmd.Env = append(os.Environ(), "QFS_RESET_CLI_HELPER=1")
output, err := cmd.CombinedOutput()
if ctx.Err() != nil {
t.Fatalf("CLI timed out: %s", output)
}
if err == nil {
t.Fatalf("expected reset refusal or offline migration-status exit: %s", output)
}
return string(output)
}
assertPending := func(want int64) {
t.Helper()
local, err := localdb.New(dbPath)
if err != nil {
t.Fatal(err)
}
defer local.Close()
if got := local.CountPendingChanges(); got != want {
t.Fatalf("pending=%d want=%d", got, want)
}
if !local.HasSettings() {
t.Fatal("connection settings lost")
}
}
if output := run(); !strings.Contains(output, "QFS_RESET_LOCAL_DB is ignored") {
t.Fatalf("environment reset not diagnosed: %s", output)
}
assertPending(1)
if output := run("-reset-localdb"); !strings.Contains(output, "reset refused") {
t.Fatalf("pending reset not refused: %s", output)
}
assertPending(1)
if output := run("-reset-localdb", "-force-reset-localdb"); !strings.Contains(output, "verified pre-reset backup") {
t.Fatalf("missing safety backup: %s", output)
}
assertPending(0)
archives, err := filepath.Glob(filepath.Join(dir, "backups", "pre-reset", "*.zip"))
if err != nil || len(archives) != 1 {
t.Fatalf("archives=%v err=%v", archives, err)
}
archive, err := zip.OpenReader(archives[0])
if err != nil {
t.Fatal(err)
}
defer archive.Close()
for _, entry := range archive.File {
if entry.Name != "qfs.db" && entry.Name != "config.yaml" {
t.Fatalf("unexpected archive entry: %s", entry.Name)
}
in, err := entry.Open()
if err != nil {
t.Fatal(err)
}
data, readErr := io.ReadAll(in)
closeErr := in.Close()
if readErr != nil || closeErr != nil {
t.Fatalf("restore entry: read=%v close=%v", readErr, closeErr)
}
if err := os.WriteFile(filepath.Join(dir, entry.Name), data, 0600); err != nil {
t.Fatal(err)
}
}
assertPending(1)
if output := run(); !strings.Contains(output, "cannot run migrations: database not available") {
t.Fatalf("restored startup did not reach migration status: %s", output)
}
assertPending(1)
}
+57
View File
@@ -0,0 +1,57 @@
package main
import (
"fmt"
"net/http"
"path/filepath"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/handlers"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/middleware"
"github.com/gin-gonic/gin"
)
func setupSetupRouter(local *localdb.LocalDB, restartSig chan struct{}) (*gin.Engine, error) {
// In setup mode, we don't have a connection manager yet (will restart after setup)
templatesPath := filepath.Join("web", "templates")
setupHandler, err := handlers.NewSetupHandler(local, nil, templatesPath, restartSig)
if err != nil {
return nil, fmt.Errorf("create setup handler: %w", err)
}
gin.SetMode(gin.ReleaseMode)
router := gin.New()
router.Use(gin.Recovery())
router.Use(middleware.CORS())
if staticFS, err := qfassets.StaticFS(); err == nil {
router.StaticFS("/static", http.FS(staticFS))
} else {
return nil, fmt.Errorf("load embedded static assets: %w", err)
}
// Setup routes only
router.GET("/", func(c *gin.Context) {
c.Redirect(http.StatusFound, "/setup")
})
router.GET("/setup", setupHandler.ShowSetup)
router.POST("/setup", setupHandler.SaveConnection)
router.POST("/setup/test", setupHandler.TestConnection)
router.GET("/setup/status", setupHandler.GetStatus)
settingsHandler := handlers.NewSettingsHandler(local)
router.GET("/api/settings/ui", settingsHandler.GetUI)
router.PUT("/api/settings/ui", settingsHandler.PutUI)
// Health check
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "setup_required",
"time": time.Now().UTC().Format(time.RFC3339),
})
})
return router, nil
}
+23 -23
View File
@@ -45,7 +45,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// list versions happy path
listReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions?limit=10&offset=0", nil)
listReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions?limit=10&offset=0", nil)
listRec := httptest.NewRecorder()
router.ServeHTTP(listRec, listReq)
if listRec.Code != http.StatusOK {
@@ -53,7 +53,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// get version happy path
getReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/1", nil)
getReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/1", nil)
getRec := httptest.NewRecorder()
router.ServeHTTP(getRec, getReq)
if getRec.Code != http.StatusOK {
@@ -62,7 +62,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
// rollback happy path
body := []byte(`{"target_version":1,"note":"api rollback"}`)
rbReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/rollback", bytes.NewReader(body))
rbReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/rollback", bytes.NewReader(body))
rbReq.Header.Set("Content-Type", "application/json")
rbRec := httptest.NewRecorder()
router.ServeHTTP(rbRec, rbReq)
@@ -84,7 +84,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 404: version missing
notFoundReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/999", nil)
notFoundReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/999", nil)
notFoundRec := httptest.NewRecorder()
router.ServeHTTP(notFoundRec, notFoundReq)
if notFoundRec.Code != http.StatusNotFound {
@@ -92,7 +92,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 400: invalid version number
invalidReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/abc", nil)
invalidReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/abc", nil)
invalidRec := httptest.NewRecorder()
router.ServeHTTP(invalidRec, invalidReq)
if invalidRec.Code != http.StatusBadRequest {
@@ -100,7 +100,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 400: rollback invalid target_version
badRollbackReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/rollback", bytes.NewReader([]byte(`{"target_version":0}`)))
badRollbackReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/rollback", bytes.NewReader([]byte(`{"target_version":0}`)))
badRollbackReq.Header.Set("Content-Type", "application/json")
badRollbackRec := httptest.NewRecorder()
router.ServeHTTP(badRollbackRec, badRollbackReq)
@@ -109,28 +109,28 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// archive + reactivate flow
delReq := httptest.NewRequest(http.MethodDelete, "/api/configs/"+created.UUID, nil)
delReq := httptest.NewRequest(http.MethodDelete, "http://127.0.0.1:8080/api/configs/"+created.UUID, nil)
delRec := httptest.NewRecorder()
router.ServeHTTP(delRec, delReq)
if delRec.Code != http.StatusOK {
t.Fatalf("archive status=%d body=%s", delRec.Code, delRec.Body.String())
}
archivedListReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=archived&page=1&per_page=20", nil)
archivedListReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=archived&page=1&per_page=20", nil)
archivedListRec := httptest.NewRecorder()
router.ServeHTTP(archivedListRec, archivedListReq)
if archivedListRec.Code != http.StatusOK {
t.Fatalf("archived list status=%d body=%s", archivedListRec.Code, archivedListRec.Body.String())
}
reactivateReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/reactivate", nil)
reactivateReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/reactivate", nil)
reactivateRec := httptest.NewRecorder()
router.ServeHTTP(reactivateRec, reactivateReq)
if reactivateRec.Code != http.StatusOK {
t.Fatalf("reactivate status=%d body=%s", reactivateRec.Code, reactivateRec.Body.String())
}
activeListReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=active&page=1&per_page=20", nil)
activeListReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=active&page=1&per_page=20", nil)
activeListRec := httptest.NewRecorder()
router.ServeHTTP(activeListRec, activeListReq)
if activeListRec.Code != http.StatusOK {
@@ -151,7 +151,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"P1","code":"P1"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"P1","code":"P1"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -164,7 +164,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
}
createCfgBody := []byte(`{"name":"Cfg A","items":[{"lot_name":"CPU","quantity":1,"unit_price":100}],"server_count":1}`)
createCfgReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/configs", bytes.NewReader(createCfgBody))
createCfgReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs", bytes.NewReader(createCfgBody))
createCfgReq.Header.Set("Content-Type", "application/json")
createCfgRec := httptest.NewRecorder()
router.ServeHTTP(createCfgRec, createCfgReq)
@@ -179,7 +179,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected config project_uuid=%s got=%v", project.UUID, createdCfg.ProjectUUID)
}
cloneReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/configs/"+createdCfg.UUID+"/clone", bytes.NewReader([]byte(`{"name":"Cfg A Clone"}`)))
cloneReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs/"+createdCfg.UUID+"/clone", bytes.NewReader([]byte(`{"name":"Cfg A Clone"}`)))
cloneReq.Header.Set("Content-Type", "application/json")
cloneRec := httptest.NewRecorder()
router.ServeHTTP(cloneRec, cloneReq)
@@ -194,7 +194,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected clone project_uuid=%s got=%v", project.UUID, cloneCfg.ProjectUUID)
}
projectConfigsReq := httptest.NewRequest(http.MethodGet, "/api/projects/"+project.UUID+"/configs", nil)
projectConfigsReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs", nil)
projectConfigsRec := httptest.NewRecorder()
router.ServeHTTP(projectConfigsRec, projectConfigsReq)
if projectConfigsRec.Code != http.StatusOK {
@@ -210,14 +210,14 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected 2 project configs after clone, got %d", len(projectConfigsResp.Configurations))
}
archiveReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/archive", nil)
archiveReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/archive", nil)
archiveRec := httptest.NewRecorder()
router.ServeHTTP(archiveRec, archiveReq)
if archiveRec.Code != http.StatusOK {
t.Fatalf("archive project status=%d body=%s", archiveRec.Code, archiveRec.Body.String())
}
activeReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=active&page=1&per_page=20", nil)
activeReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=active&page=1&per_page=20", nil)
activeRec := httptest.NewRecorder()
router.ServeHTTP(activeRec, activeReq)
if activeRec.Code != http.StatusOK {
@@ -245,7 +245,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"Move Project","code":"MOVE"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"Move Project","code":"MOVE"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -257,7 +257,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("unmarshal project: %v", err)
}
createConfigReq := httptest.NewRequest(http.MethodPost, "/api/configs", bytes.NewReader([]byte(`{"name":"Move Me","items":[],"notes":"","server_count":1}`)))
createConfigReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs", bytes.NewReader([]byte(`{"name":"Move Me","items":[],"notes":"","server_count":1}`)))
createConfigReq.Header.Set("Content-Type", "application/json")
createConfigRec := httptest.NewRecorder()
router.ServeHTTP(createConfigRec, createConfigReq)
@@ -269,7 +269,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("unmarshal config: %v", err)
}
moveReq := httptest.NewRequest(http.MethodPatch, "/api/configs/"+created.UUID+"/project", bytes.NewReader([]byte(`{"project_uuid":"`+project.UUID+`"}`)))
moveReq := httptest.NewRequest(http.MethodPatch, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/project", bytes.NewReader([]byte(`{"project_uuid":"`+project.UUID+`"}`)))
moveReq.Header.Set("Content-Type", "application/json")
moveRec := httptest.NewRecorder()
router.ServeHTTP(moveRec, moveReq)
@@ -277,7 +277,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("move config status=%d body=%s", moveRec.Code, moveRec.Body.String())
}
getReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID, nil)
getReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID, nil)
getRec := httptest.NewRecorder()
router.ServeHTTP(getRec, getReq)
if getRec.Code != http.StatusOK {
@@ -307,7 +307,7 @@ func TestVendorImportRejectsOversizedUpload(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"Import Project","code":"IMP"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"Import Project","code":"IMP"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -334,7 +334,7 @@ func TestVendorImportRejectsOversizedUpload(t *testing.T) {
t.Fatalf("close multipart writer: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/vendor-import", &body)
req := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/vendor-import", &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
@@ -358,7 +358,7 @@ func TestCreateConfigMalformedJSONReturnsGenericError(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/configs", bytes.NewReader([]byte(`{"name":`)))
req := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs", bytes.NewReader([]byte(`{"name":`)))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
+27 -21
View File
@@ -1,11 +1,11 @@
module git.mchus.pro/mchus/quoteforge
go 1.24.0
go 1.27.3
require (
github.com/gin-gonic/gin v1.9.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-sql-driver/mysql v1.7.1
github.com/go-sql-driver/mysql v1.10.1
github.com/google/uuid v1.6.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/mysql v1.5.2
@@ -13,35 +13,41 @@ require (
)
require (
github.com/bytedance/sonic v1.9.1 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic v1.15.0 // indirect
github.com/bytedance/sonic/loader v0.5.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.12 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/glebarez/go-sqlite v1.21.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/go-playground/validator/v10 v10.30.1 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.59.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/stretchr/testify v1.11.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/net v0.46.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/text v0.30.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
github.com/ugorji/go/codec v1.3.1 // indirect
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
golang.org/x/arch v0.22.0 // indirect
golang.org/x/crypto v0.58.0 // indirect
golang.org/x/net v0.61.0 // indirect
golang.org/x/sys v0.49.0 // indirect
golang.org/x/text v0.43.0 // indirect
google.golang.org/protobuf v1.36.10 // indirect
modernc.org/libc v1.22.5 // indirect
modernc.org/mathutil v1.5.0 // indirect
modernc.org/memory v1.5.0 // indirect
+67 -56
View File
@@ -1,20 +1,24 @@
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
github.com/bytedance/sonic v1.9.1 h1:6iJ6NqdoxCDr6mbY8h18oSO+cShGSMRGCEo7F2h0x8s=
github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZXU064P/U=
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 h1:qSGYFH7+jGhDF8vLC+iwCD4WpbV1EBDSzWkJODFLams=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k=
github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE=
github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg=
github.com/gin-gonic/gin v1.9.1/go.mod h1:hPrL7YrpYKXt5YId3A/Tnip5kqbEAP+KLuI3SUcPTeU=
github.com/gabriel-vasile/mimetype v1.4.12 h1:e9hWvmLYvtp846tLHam2o++qitpguFiYCKbn0w9jyqw=
github.com/gabriel-vasile/mimetype v1.4.12/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/glebarez/go-sqlite v1.21.2 h1:3a6LFC4sKahUunAmynQKLZceZCOzUthkRkEAl9gAXWo=
github.com/glebarez/go-sqlite v1.21.2/go.mod h1:sfxdZyhQjTM2Wry3gVYWaW072Ri1WMdWJi0k6+3382k=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
@@ -25,16 +29,17 @@ github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/o
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.14.0 h1:vgvQWe3XCz3gIeFDm/HnTIbj6UGmg/+t63MyGU2n5js=
github.com/go-playground/validator/v10 v10.14.0/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU=
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
github.com/go-sql-driver/mysql v1.7.0/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=
github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/go-sql-driver/mysql v1.10.1 h1:arlSnNLq6a5yxGxV7qg9lF4j0C+KwD6NbQyKr9QL6ME=
github.com/go-sql-driver/mysql v1.10.1/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
@@ -46,62 +51,69 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pelletier/go-toml/v2 v2.0.8 h1:0ctb6s9mE31h0/lhu+J6OPmVeDxJn+kYnJc2jZR9tGQ=
github.com/pelletier/go-toml/v2 v2.0.8/go.mod h1:vuYfssBdrU2XDZ9bYydBu6t+6a6PYNcZljzZR9VXg+4=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4=
golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210=
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
@@ -118,4 +130,3 @@ modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds=
modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/sqlite v1.23.1 h1:nrSBg4aRQQwq59JpvGEQ15tNxoO5pX/kUjcRNwSAGQM=
modernc.org/sqlite v1.23.1/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
+8 -2
View File
@@ -273,7 +273,10 @@ func createBackupArchive(destPath, dbPath, configPath string) error {
}
if strings.TrimSpace(configPath) != "" {
_ = addZipOptionalFile(zipWriter, configPath)
if err := addZipOptionalFile(zipWriter, configPath); err != nil {
_ = zipWriter.Close()
return fmt.Errorf("archive config: %w", err)
}
}
if err := zipWriter.Close(); err != nil {
@@ -355,7 +358,10 @@ func isSQLiteBusyError(err error) bool {
func addZipOptionalFile(writer *zip.Writer, path string) error {
if _, err := os.Stat(path); err != nil {
return nil
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("stat optional backup file: %w", err)
}
return addZipFile(writer, path)
}
+111
View File
@@ -0,0 +1,111 @@
package appstate
import (
"archive/zip"
"fmt"
"io"
"os"
"path/filepath"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// CreatePreResetBackup is mandatory and independent of recurring backup settings.
func CreatePreResetBackup(dbPath, configPath string) (string, error) {
root := filepath.Join(resolveBackupRoot(dbPath), "pre-reset")
if err := validateBackupRoot(root); err != nil {
return "", err
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create pre-reset backup directory: %w", err)
}
resolved, err := filepath.EvalSymlinks(root)
if err != nil {
return "", fmt.Errorf("resolve pre-reset backup directory: %w", err)
}
if err := validateBackupRoot(resolved); err != nil {
return "", err
}
file, err := os.CreateTemp(resolved, "qfs-pre-reset-v1-"+time.Now().UTC().Format("20060102T150405Z")+"-*.zip")
if err != nil {
return "", fmt.Errorf("reserve pre-reset backup: %w", err)
}
path := file.Name()
if err := file.Close(); err != nil {
return "", fmt.Errorf("close pre-reset backup: %w", err)
}
if err := createBackupArchive(path, dbPath, configPath); err != nil {
return "", fmt.Errorf("write pre-reset archive: %w", err)
}
if err := verifyResetArchive(path, filepath.Base(dbPath)); err != nil {
return "", fmt.Errorf("verify pre-reset archive: %w", err)
}
return path, nil
}
func verifyResetArchive(path, dbName string) error {
archive, err := zip.OpenReader(path)
if err != nil {
return fmt.Errorf("open archive: %w", err)
}
defer archive.Close()
dir, err := os.MkdirTemp("", "qfs-verify-reset-*")
if err != nil {
return fmt.Errorf("create verification directory: %w", err)
}
defer os.RemoveAll(dir)
found := false
for _, entry := range archive.File {
in, err := entry.Open()
if err != nil {
return fmt.Errorf("open archive entry: %w", err)
}
var out io.Writer = io.Discard
var restored *os.File
if entry.Name == dbName {
restored, err = os.OpenFile(filepath.Join(dir, "verify.db"), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
in.Close()
return fmt.Errorf("create verification database: %w", err)
}
out = restored
found = true
}
_, copyErr := io.Copy(out, in)
closeErr := in.Close()
if restored != nil {
if err := restored.Close(); err != nil {
return fmt.Errorf("close verification database: %w", err)
}
}
if copyErr != nil {
return fmt.Errorf("read archive entry: %w", copyErr)
}
if closeErr != nil {
return fmt.Errorf("close archive entry: %w", closeErr)
}
}
if !found {
return fmt.Errorf("archive has no database snapshot")
}
db, err := gorm.Open(sqlite.Open(filepath.Join(dir, "verify.db")), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return fmt.Errorf("open restored snapshot: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return fmt.Errorf("get restored database: %w", err)
}
defer sqlDB.Close()
var result string
if err := db.Raw("PRAGMA integrity_check").Scan(&result).Error; err != nil {
return fmt.Errorf("check restored snapshot: %w", err)
}
if result != "ok" {
return fmt.Errorf("snapshot integrity check failed: %s", result)
}
return nil
}
+51
View File
@@ -0,0 +1,51 @@
package appstate
import (
"fmt"
"os"
"path/filepath"
)
// ServerMigrationBackupDir is separate from local backup retention.
func ServerMigrationBackupDir(dbPath string) (string, error) {
root := filepath.Join(resolveBackupRoot(dbPath), "server-migrations")
if err := ValidateBackupLocation(root); err != nil {
return "", err
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create server backup directory: %w", err)
}
resolved, err := filepath.EvalSymlinks(root)
if err != nil {
return "", fmt.Errorf("resolve server backup directory: %w", err)
}
if err := ValidateBackupLocation(resolved); err != nil {
return "", err
}
return resolved, nil
}
// ValidateBackupLocation checks the destination, including existing symlinks.
func ValidateBackupLocation(path string) error {
if err := validateBackupRoot(path); err != nil {
return err
}
existing := path
for {
if _, err := os.Lstat(existing); err == nil {
break
} else if !os.IsNotExist(err) {
return fmt.Errorf("inspect backup path: %w", err)
}
parent := filepath.Dir(existing)
if parent == existing {
return fmt.Errorf("backup path has no existing parent")
}
existing = parent
}
resolved, err := filepath.EvalSymlinks(existing)
if err != nil {
return fmt.Errorf("resolve backup path: %w", err)
}
return validateBackupRoot(resolved)
}
@@ -0,0 +1,29 @@
package appstate
import (
"os"
"path/filepath"
"testing"
)
func TestServerBackupRejectsGitAndSymlinks(t *testing.T) {
root := t.TempDir()
repo := filepath.Join(root, "repo")
if err := os.MkdirAll(filepath.Join(repo, ".git"), 0700); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "link")
if err := os.Symlink(repo, link); err != nil {
t.Fatal(err)
}
for _, path := range []string{filepath.Join(repo, "backups"), filepath.Join(link, "backups")} {
t.Setenv("QFS_BACKUP_DIR", path)
if _, err := ServerMigrationBackupDir(filepath.Join(root, "state", "qfs.db")); err == nil {
t.Fatalf("accepted repository backup: %s", path)
}
}
t.Setenv("QFS_BACKUP_DIR", filepath.Join(root, "backups"))
if _, err := ServerMigrationBackupDir(filepath.Join(root, "state", "qfs.db")); err != nil {
t.Fatal(err)
}
}
+67 -18
View File
@@ -22,9 +22,10 @@ const (
// goroutines must never pay the MySQL dial/read timeout themselves.
defaultStatusCheckInterval = 15 * time.Second
maxOpenConns = 10
maxOpenConns = 2
maxIdleConns = 2
connMaxLifetime = 5 * time.Minute
connMaxIdleTime = time.Minute
)
// ConnectionStatus represents the current status of the database connection
@@ -49,11 +50,13 @@ type ConnectionManager struct {
statusCheckInterval time.Duration // background prober cadence (default: 15s)
stopStatusLoop chan struct{} // closed by Stop() to end the background loop
dialFn func() (*gorm.DB, error) // opens a new pool; cm.dial unless overridden in tests
}
// NewConnectionManager creates a new ConnectionManager instance
func NewConnectionManager(localDB *localdb.LocalDB) *ConnectionManager {
return &ConnectionManager{
cm := &ConnectionManager{
localDB: localDB,
connectTimeout: defaultConnectTimeout,
pingInterval: defaultPingInterval,
@@ -63,6 +66,8 @@ func NewConnectionManager(localDB *localdb.LocalDB) *ConnectionManager {
lastError: nil,
lastCheck: time.Time{},
}
cm.dialFn = cm.dial
return cm
}
// Start launches a background goroutine that keeps the online-status cache
@@ -137,25 +142,66 @@ func (cm *ConnectionManager) GetDB() (*gorm.DB, error) {
return db, err
}
newDB, err := cm.dial()
// The cached pool is only stale, not broken: ping it instead of dialing a
// new one. Re-dialing here used to orphan the old *sql.DB together with its
// idle connections, leaking a pool every pingInterval during sync.
cm.mu.RLock()
existing := cm.db
cm.mu.RUnlock()
if existing != nil {
if err := cm.ping(existing); err == nil {
cm.mu.Lock()
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
return existing, nil
}
}
newDB, err := cm.dialFn()
cm.mu.Lock()
old := cm.db
if err != nil {
// Drop stale handle so callers don't treat it as an active connection.
cm.db = nil
cm.lastError = err
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return nil, err
}
cm.db = newDB
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return newDB, nil
}
// ping checks an existing pool with the connect timeout. No locks held.
func (cm *ConnectionManager) ping(db *gorm.DB) error {
ctx, cancel := context.WithTimeout(context.Background(), cm.connectTimeout)
defer cancel()
sqlDB, err := db.DB()
if err != nil {
return err
}
return sqlDB.PingContext(ctx)
}
// closeDB releases a pool that is no longer referenced by the manager.
func closeDB(db *gorm.DB) {
if db == nil {
return
}
if sqlDB, err := db.DB(); err == nil {
sqlDB.Close()
}
}
// cachedResult returns (db, err, true) if the cached state is still fresh
// enough to answer without a new network round-trip: either a live
// connection within pingInterval, or a recent failure still within
@@ -200,15 +246,17 @@ func (cm *ConnectionManager) dial() (*gorm.DB, error) {
return nil, fmt.Errorf("getting sql.DB: %w", err)
}
// Ping with timeout
if err = sqlDB.PingContext(ctx); err != nil {
return nil, fmt.Errorf("pinging database: %w", err)
}
// Set connection pool settings
sqlDB.SetMaxOpenConns(maxOpenConns)
sqlDB.SetMaxIdleConns(maxIdleConns)
sqlDB.SetConnMaxLifetime(connMaxLifetime)
sqlDB.SetConnMaxIdleTime(connMaxIdleTime)
// Ping with timeout
if err = sqlDB.PingContext(ctx); err != nil {
sqlDB.Close()
return nil, fmt.Errorf("pinging database: %w", err)
}
return db, nil
}
@@ -264,20 +312,17 @@ func (cm *ConnectionManager) checkOnlineNow() bool {
}
// Perform ping with timeout — no locks held here.
ctx, cancel := context.WithTimeout(context.Background(), cm.connectTimeout)
defer cancel()
sqlDB, err := db.DB()
if err == nil {
err = sqlDB.PingContext(ctx)
}
err := cm.ping(db)
cm.mu.Lock()
defer cm.mu.Unlock()
if err != nil {
cm.lastError = err
cm.lastCheck = time.Now()
cm.db = nil
if cm.db == db {
cm.db = nil
}
closeDB(db)
return false
}
cm.lastCheck = time.Now()
@@ -292,19 +337,23 @@ func (cm *ConnectionManager) TryConnect() error {
cm.connMu.Lock()
defer cm.connMu.Unlock()
newDB, err := cm.dial()
newDB, err := cm.dialFn()
cm.mu.Lock()
defer cm.mu.Unlock()
old := cm.db
if err != nil {
cm.db = nil
cm.lastError = err
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return err
}
cm.db = newDB
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return nil
}
+74
View File
@@ -0,0 +1,74 @@
package db
import (
"path/filepath"
"testing"
"time"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func newTestConnectionManager(t *testing.T) (*ConnectionManager, *int) {
t.Helper()
local, err := localdb.New(filepath.Join(t.TempDir(), "qfs.db"))
if err != nil {
t.Fatalf("open local db: %v", err)
}
cm := NewConnectionManager(local)
dials := 0
cm.dialFn = func() (*gorm.DB, error) {
dials++
return gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "remote.db")), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
}
return cm, &dials
}
func TestGetDBReusesStalePoolInsteadOfRedialing(t *testing.T) {
cm, dials := newTestConnectionManager(t)
first, err := cm.GetDB()
if err != nil {
t.Fatalf("first GetDB: %v", err)
}
// Simulate pingInterval elapsing between sync steps.
cm.mu.Lock()
cm.lastCheck = time.Now().Add(-2 * cm.pingInterval)
cm.mu.Unlock()
second, err := cm.GetDB()
if err != nil {
t.Fatalf("second GetDB: %v", err)
}
if second != first {
t.Fatalf("expected stale but healthy pool to be reused")
}
if *dials != 1 {
t.Fatalf("expected 1 dial, got %d", *dials)
}
}
func TestTryConnectClosesReplacedPool(t *testing.T) {
cm, _ := newTestConnectionManager(t)
first, err := cm.GetDB()
if err != nil {
t.Fatalf("GetDB: %v", err)
}
if err := cm.TryConnect(); err != nil {
t.Fatalf("TryConnect: %v", err)
}
sqlDB, err := first.DB()
if err != nil {
t.Fatalf("sql.DB: %v", err)
}
if err := sqlDB.Ping(); err == nil {
t.Fatalf("expected replaced pool to be closed")
}
}
+15 -10
View File
@@ -118,6 +118,11 @@ func (h *ComponentHandler) GetCategories(c *gin.Context) {
c.JSON(http.StatusOK, models.DefaultCategories)
}
// GetConfiguratorSettings serves the synced qt_settings. The default*() values
// below are the only client-side fallback, used per key when the local cache has
// no such setting (e.g. before the first sync); they are not kept in step with
// the server — edit qt_settings in MariaDB, not these. The configurator page has
// no fallback of its own.
func (h *ComponentHandler) GetConfiguratorSettings(c *gin.Context) {
s, _ := h.localDB.GetConfiguratorSettings()
if s == nil {
@@ -131,7 +136,7 @@ func (h *ComponentHandler) GetConfiguratorSettings(c *gin.Context) {
s.TabConfig = defaultTabConfig()
}
if len(s.AlwaysVisibleTabs) == 0 {
s.AlwaysVisibleTabs = []string{"base", "storage", "pci"}
s.AlwaysVisibleTabs = []string{"base"}
}
if len(s.RequiredCategories) == 0 {
s.RequiredCategories = map[string][]string{"server": {"CPU", "MEM", "BB"}}
@@ -158,19 +163,19 @@ func defaultSupportPricing() *localdb.SupportPricingTable {
},
HGXPrice: map[string]map[string]map[string]float64{
"HGX-H200": {
"B": {"1": 20000, "3": 55000, "5": 80000},
"S": {"1": 28000, "3": 70000, "5": 100000},
"P": {"1": 63000, "3": 105000},
"B": {"1": 28000, "3": 77000, "5": 112000},
"S": {"1": 39500, "3": 98000, "5": 140000},
"P": {"1": 88500, "3": 147000},
},
"HGX-B200": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 101000, "3": 176000},
"B": {"1": 158500, "3": 230500, "5": 284500},
"S": {"1": 176500, "3": 266500, "5": 332500},
"P": {"1": 232000, "3": 322000},
},
"HGX-B300": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 112000, "3": 187000},
"B": {"1": 192000, "3": 270000, "5": 328500},
"S": {"1": 211500, "3": 309000, "5": 380500},
"P": {"1": 286000, "3": 383500},
},
},
}
+54 -17
View File
@@ -1,6 +1,7 @@
package handlers
import (
"errors"
"fmt"
"html/template"
"log/slog"
@@ -13,8 +14,10 @@ import (
"git.mchus.pro/mchus/quoteforge/internal/appmeta"
"git.mchus.pro/mchus/quoteforge/internal/db"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"github.com/gin-gonic/gin"
mysqlDriver "github.com/go-sql-driver/mysql"
"gorm.io/gorm"
)
type SetupHandler struct {
@@ -82,14 +85,24 @@ func (h *SetupHandler) TestConnection(c *gin.Context) {
port = p
}
// If password is empty, try to use saved password
if password == "" {
if settings, err := h.localDB.GetSettings(); err == nil && settings != nil {
password = settings.PasswordEncrypted // GetSettings returns decrypted password in this field
}
settings, err := h.localDB.GetSettings()
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
_ = c.Error(err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read settings"})
return
}
password, err = setupPassword(settings, host, port, user, password)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
dsn := buildMySQLDSN(host, port, database, user, password, 5*time.Second)
tlsMode, caPEM := c.DefaultPostForm("tls_mode", "preferred"), c.PostForm("tls_ca_pem")
dsn, err := buildMySQLDSNWithTLS(host, port, database, user, password, 5*time.Second, tlsMode, caPEM)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
lotCount, canWrite, err := db.ValidateMariaDBConnection(dsn)
if err != nil {
_ = c.Error(err)
@@ -110,7 +123,12 @@ func (h *SetupHandler) TestConnection(c *gin.Context) {
// SaveConnection saves the connection settings and signals restart
func (h *SetupHandler) SaveConnection(c *gin.Context) {
existingSettings, _ := h.localDB.GetSettings()
existingSettings, settingsErr := h.localDB.GetSettings()
if settingsErr != nil && !errors.Is(settingsErr, gorm.ErrRecordNotFound) {
_ = c.Error(settingsErr)
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read settings"})
return
}
host := c.PostForm("host")
portStr := c.PostForm("port")
@@ -123,15 +141,19 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
port = p
}
// If password is empty, use saved password
if password == "" {
if settings, err := h.localDB.GetSettings(); err == nil && settings != nil {
password = settings.PasswordEncrypted // GetSettings returns decrypted password in this field
}
password, err := setupPassword(existingSettings, host, port, user, password)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
// Test connection first
dsn := buildMySQLDSN(host, port, database, user, password, 5*time.Second)
tlsMode, caPEM := c.DefaultPostForm("tls_mode", "preferred"), c.PostForm("tls_ca_pem")
dsn, err := buildMySQLDSNWithTLS(host, port, database, user, password, 5*time.Second, tlsMode, caPEM)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
if _, _, err := db.ValidateMariaDBConnection(dsn); err != nil {
_ = c.Error(err)
c.JSON(http.StatusBadRequest, gin.H{
@@ -142,7 +164,7 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
}
// Save settings
if err := h.localDB.SaveSettings(host, port, database, user, password); err != nil {
if err := h.localDB.SaveSettingsWithTLS(host, port, database, user, password, tlsMode, caPEM); err != nil {
_ = c.Error(err)
c.JSON(http.StatusInternalServerError, gin.H{
"success": false,
@@ -165,7 +187,8 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
existingSettings.Port != port ||
existingSettings.Database != database ||
existingSettings.User != user ||
existingSettings.PasswordEncrypted != password
existingSettings.PasswordEncrypted != password ||
existingSettings.TLSMode != tlsMode || existingSettings.TLSCAPEM != caPEM
restartQueued := settingsChanged && h.restartSig != nil
c.JSON(http.StatusOK, gin.H{
@@ -195,7 +218,7 @@ func (h *SetupHandler) GetStatus(c *gin.Context) {
})
}
func buildMySQLDSN(host string, port int, database, user, password string, timeout time.Duration) string {
func buildMySQLDSNWithTLS(host string, port int, database, user, password string, timeout time.Duration, tlsMode, caPEM string) (string, error) {
cfg := mysqlDriver.NewConfig()
cfg.User = user
cfg.Passwd = password
@@ -208,5 +231,19 @@ func buildMySQLDSN(host string, port int, database, user, password string, timeo
cfg.Params = map[string]string{
"charset": "utf8mb4",
}
return cfg.FormatDSN()
if err := mariadb.ConfigureTLS(cfg, tlsMode, caPEM); err != nil {
return "", fmt.Errorf("configure TLS: %w", err)
}
return cfg.FormatDSN(), nil
}
// setupPassword reuses a credential only for its saved destination and account.
func setupPassword(saved *localdb.ConnectionSettings, host string, port int, user, password string) (string, error) {
if password != "" || saved == nil {
return password, nil
}
if saved.Host != host || saved.Port != port || saved.User != user {
return "", fmt.Errorf("Password is required when changing host, port or user")
}
return saved.PasswordEncrypted, nil
}
+31
View File
@@ -0,0 +1,31 @@
package handlers
import (
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"testing"
)
func TestSetupPassword(t *testing.T) {
saved := &localdb.ConnectionSettings{Host: "db.example", Port: 3306, User: "example", PasswordEncrypted: "example"}
for _, tt := range []struct {
name, host, user, password string
port int
wantError bool
}{
{"same destination", "db.example", "example", "", 3306, false},
{"changed host", "other.example", "example", "", 3306, true},
{"changed port", "db.example", "example", "", 3307, true},
{"changed user", "db.example", "other", "", 3306, true},
{"explicit password", "other.example", "other", "example", 3307, false},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := setupPassword(saved, tt.host, tt.port, tt.user, tt.password)
if (err != nil) != tt.wantError {
t.Fatalf("unexpected error: %v", err)
}
if !tt.wantError && got != "example" {
t.Fatal("credential not preserved")
}
})
}
}
+37
View File
@@ -0,0 +1,37 @@
package handlers
import (
"net/http"
"net/http/httptest"
"net/url"
"path/filepath"
"strings"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"github.com/gin-gonic/gin"
)
func TestSetupFreshSettingsValidateTLS(t *testing.T) {
local, err := localdb.New(filepath.Join(t.TempDir(), "qfs.db"))
if err != nil {
t.Fatal(err)
}
defer local.Close()
handler := &SetupHandler{localDB: local}
for _, test := range []struct {
path string
handler gin.HandlerFunc
}{{"/setup/test", handler.TestConnection}, {"/setup", handler.SaveConnection}} {
router := gin.New()
router.POST(test.path, test.handler)
body := url.Values{"host": {"db.example"}, "user": {"example"}, "tls_mode": {"invalid"}}.Encode()
req := httptest.NewRequest(http.MethodPost, test.path, strings.NewReader(body))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusUnprocessableEntity {
t.Fatalf("%s: status=%d body=%s", test.path, w.Code, w.Body.String())
}
}
}
+8 -2
View File
@@ -43,7 +43,7 @@ func NewSyncHandler(localDB *localdb.LocalDB, syncService *sync.Service, connMgr
syncService: syncService,
connMgr: connMgr,
autoSyncInterval: autoSyncInterval,
onlineGraceFactor: 1.10,
onlineGraceFactor: 1 + sync.WorkerJitter, // longest gap between jittered sync cycles
tmpl: tmpl,
}, nil
}
@@ -240,7 +240,7 @@ type SyncAllResponse struct {
// SyncAll performs full bidirectional sync:
// - push pending local changes (projects/configurations) to server
// - pull components, pricelists, projects, and configurations from server
// - pull components, pricelists, qt_settings, projects, and configurations from server
// POST /api/sync/all
func (h *SyncHandler) SyncAll(c *gin.Context) {
if !h.ensureSyncReadiness(c) {
@@ -282,6 +282,12 @@ func (h *SyncHandler) SyncAll(c *gin.Context) {
slog.Warn("partnumber books pull failed during full sync", "error", err)
}
// Configurator settings (config_types, tab_config, ...): non-fatal, the
// local cache is kept on failure.
if err := h.syncService.SyncQtSettings(); err != nil {
slog.Warn("qt_settings sync failed during full sync", "error", err)
}
projectsResult, err := h.syncService.ImportProjectsToLocal()
if err != nil {
slog.Error("project import failed during full sync", "error", err)
+27 -44
View File
@@ -14,6 +14,7 @@ import (
"git.mchus.pro/mchus/quoteforge/internal/appmeta"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"github.com/glebarez/sqlite"
mysqlDriver "github.com/go-sql-driver/mysql"
uuidpkg "github.com/google/uuid"
@@ -24,6 +25,8 @@ import (
// ConnectionSettings stores MariaDB connection credentials
type ConnectionSettings struct {
TLSMode string `gorm:"default:preferred"`
TLSCAPEM string
ID uint `gorm:"primaryKey"`
Host string `gorm:"not null"`
Port int `gorm:"not null;default:3306"`
@@ -50,47 +53,6 @@ var localReadOnlyCacheTables = []string{
"local_partnumber_books",
}
// ResetData clears local data tables while keeping connection settings.
// It does not drop schema or connection_settings.
func ResetData(dbPath string) error {
if strings.TrimSpace(dbPath) == "" {
return nil
}
if _, err := os.Stat(dbPath); err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return fmt.Errorf("stat local db: %w", err)
}
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
return fmt.Errorf("opening sqlite database: %w", err)
}
// Order does not matter because we use DELETEs without FK constraints in SQLite.
tables := []string{
"local_projects",
"local_configurations",
"local_configuration_versions",
"local_pricelists",
"local_pricelist_items",
"local_sync_guard_state",
"pending_changes",
"app_settings",
}
for _, table := range tables {
if err := db.Exec("DELETE FROM " + table).Error; err != nil {
return fmt.Errorf("clear %s: %w", table, err)
}
}
slog.Info("local database data reset", "path", dbPath)
return nil
}
// New creates a new LocalDB instance
func New(dbPath string) (*LocalDB, error) {
// Ensure directory exists
@@ -548,6 +510,14 @@ func (l *LocalDB) GetSettings() (*ConnectionSettings, error) {
// SaveSettings saves connection settings with encrypted password
func (l *LocalDB) SaveSettings(host string, port int, database, user, password string) error {
return l.SaveSettingsWithTLS(host, port, database, user, password, "preferred", "")
}
func (l *LocalDB) SaveSettingsWithTLS(host string, port int, database, user, password, tlsMode, caPEM string) error {
if err := mariadb.ConfigureTLS(mysqlDriver.NewConfig(), tlsMode, caPEM); err != nil {
return fmt.Errorf("validate connection TLS: %w", err)
}
// Encrypt password
encrypted, err := Encrypt(password)
if err != nil {
@@ -556,6 +526,8 @@ func (l *LocalDB) SaveSettings(host string, port int, database, user, password s
settings := ConnectionSettings{
ID: 1, // Always use ID=1 for single settings row
TLSMode: tlsMode,
TLSCAPEM: caPEM,
Host: host,
Port: port,
Database: database,
@@ -601,6 +573,9 @@ func (l *LocalDB) GetDSN() (string, error) {
"charset": "utf8mb4",
}
if err := mariadb.ConfigureTLS(cfg, settings.TLSMode, settings.TLSCAPEM); err != nil {
return "", fmt.Errorf("configure connection TLS: %w", err)
}
return cfg.FormatDSN(), nil
}
@@ -1285,7 +1260,6 @@ func (l *LocalDB) GetLastComponentSyncError() string {
return strings.TrimSpace(value)
}
const openConfigUUIDsKey = "open_config_uuids"
// GetOpenConfigUUIDs returns UUIDs of all configurations currently open in the configurator.
@@ -1361,7 +1335,6 @@ func (l *LocalDB) CountAllPricelistItems() int64 {
return count
}
// DBFilePath returns the path to the SQLite database file.
func (l *LocalDB) DBFilePath() string {
return l.path
@@ -1862,6 +1835,17 @@ func (l *LocalDB) MarkChangesSynced(ids []int64) error {
return l.db.Where("id IN ?", ids).Delete(&PendingChange{}).Error
}
// ListConfigurationsPendingWithoutQueue returns configurations still marked pending
// whose pending_changes rows are gone, so nothing will ever push them.
func (l *LocalDB) ListConfigurationsPendingWithoutQueue() ([]LocalConfiguration, error) {
var configs []LocalConfiguration
err := l.db.Where(
"sync_status = ? AND uuid NOT IN (SELECT entity_uuid FROM pending_changes WHERE entity_type = ?)",
"pending", "configuration",
).Find(&configs).Error
return configs, err
}
// PurgeOrphanConfigurationPendingChanges removes configuration pending changes
// whose entity_uuid no longer exists in local_configurations.
func (l *LocalDB) PurgeOrphanConfigurationPendingChanges() (int64, error) {
@@ -2084,7 +2068,6 @@ func (l *LocalDB) GetLocalPricelistItemsPage(pricelistID uint, search string, pa
return items, total, nil
}
// GetSchemaMigrations returns all applied local schema migrations ordered by applied_at.
func (l *LocalDB) GetSchemaMigrations() ([]LocalSchemaMigration, error) {
var migrations []LocalSchemaMigration
+51
View File
@@ -0,0 +1,51 @@
package localdb
import (
"encoding/json"
"fmt"
"time"
)
// serverPullCursor is the server-side NOW(6) taken before the last successful
// incremental pull of a table, bound to the server it was taken from.
type serverPullCursor struct {
Scope string `json:"scope"`
ServerTime string `json:"server_time"`
}
func serverPullCursorKey(table string) string {
return "server_pull_cursor:" + table
}
// ServerPullScope identifies the MariaDB server+database a pull cursor belongs to,
// so a cursor never survives a switch to another server.
func (l *LocalDB) ServerPullScope() (string, error) {
settings, err := l.GetSettings()
if err != nil {
return "", err
}
return fmt.Sprintf("%s:%d/%s", settings.Host, settings.Port, settings.Database), nil
}
// GetServerPullCursor returns the stored server time for table if it was taken
// on the server identified by scope.
func (l *LocalDB) GetServerPullCursor(table, scope string) (string, bool) {
raw, ok := l.getAppSettingValue(serverPullCursorKey(table))
if !ok {
return "", false
}
var cursor serverPullCursor
if err := json.Unmarshal([]byte(raw), &cursor); err != nil || cursor.Scope != scope || cursor.ServerTime == "" {
return "", false
}
return cursor.ServerTime, true
}
// SetServerPullCursor stores the server time a pull of table started at.
func (l *LocalDB) SetServerPullCursor(table, scope, serverTime string) error {
raw, err := json.Marshal(serverPullCursor{Scope: scope, ServerTime: serverTime})
if err != nil {
return err
}
return l.upsertAppSetting(l.db, serverPullCursorKey(table), string(raw), time.Now())
}
+21
View File
@@ -72,6 +72,11 @@ func (l *LocalDB) SyncQtSettings(mariaDB *gorm.DB) error {
return nil
}
var current []LocalQtSetting
if err := l.db.Select("name, value").Find(&current).Error; err == nil && sameQtSettings(current, rows) {
return nil
}
return l.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Exec("DELETE FROM local_qt_settings").Error; err != nil {
return fmt.Errorf("clearing local_qt_settings: %w", err)
@@ -142,3 +147,19 @@ func (l *LocalDB) GetConfiguratorSettings() (*ConfiguratorSettings, error) {
return out, nil
}
func sameQtSettings(a, b []LocalQtSetting) bool {
if len(a) != len(b) {
return false
}
values := make(map[string]string, len(a))
for _, row := range a {
values[row.Name] = row.Value
}
for _, row := range b {
if v, ok := values[row.Name]; !ok || v != row.Value {
return false
}
}
return true
}
+85
View File
@@ -0,0 +1,85 @@
package localdb
import (
"fmt"
"log/slog"
"os"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func ResetData(dbPath string) error {
configPath, err := appstate.ResolveConfigPathNearDB("", dbPath)
if err != nil {
return fmt.Errorf("resolve reset config: %w", err)
}
return ResetDataWithOptions(dbPath, configPath, false)
}
// ResetDataWithOptions requires a fresh verified backup even for forced resets.
func ResetDataWithOptions(dbPath, configPath string, force bool) error {
if strings.TrimSpace(dbPath) == "" {
return nil
}
if _, err := os.Stat(dbPath); err != nil {
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("stat reset database: %w", err)
}
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return fmt.Errorf("open reset database: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return fmt.Errorf("get reset database: %w", err)
}
defer sqlDB.Close()
return db.Transaction(func(tx *gorm.DB) error {
for _, table := range []string{"pending_changes", "local_projects", "local_configurations"} {
if !tx.Migrator().HasTable(table) {
continue
}
query := tx.Table(table)
if table != "pending_changes" {
if !tx.Migrator().HasColumn(table, "sync_status") {
if !force {
return fmt.Errorf("reset refused: %s has no sync_status; use -reset-localdb -force-reset-localdb after reviewing local work", table)
}
continue
}
query = query.Where("sync_status IS NULL OR sync_status <> ?", "synced")
}
var count int64
if err := query.Count(&count).Error; err != nil {
return fmt.Errorf("check reset data in %s: %w", table, err)
}
if count > 0 && !force {
return fmt.Errorf("reset refused: %s contains %d unsynced rows; sync first or use -reset-localdb -force-reset-localdb", table, count)
}
}
archivePath, err := appstate.CreatePreResetBackup(dbPath, configPath)
if err != nil {
return fmt.Errorf("reset safety backup: %w", err)
}
slog.Info("verified pre-reset backup", "path", archivePath)
for _, table := range []string{
"local_projects", "local_configurations", "local_configuration_versions",
"local_pricelists", "local_pricelist_items", "local_sync_guard_state", "pending_changes", "app_settings",
} {
if !tx.Migrator().HasTable(table) {
continue
}
if err := tx.Exec("DELETE FROM " + table).Error; err != nil {
return fmt.Errorf("clear %s: %w", table, err)
}
}
slog.Info("local database data reset", "path", dbPath, "forced", force)
return nil
})
}
+154
View File
@@ -0,0 +1,154 @@
package localdb
import (
"archive/zip"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestResetProtectsUnsyncedData(t *testing.T) {
for _, table := range []string{"pending_changes", "local_projects", "local_configurations"} {
t.Run(table, func(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("INSERT INTO " + table + " (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
err := ResetDataWithOptions(path, "", false)
if err == nil || !strings.Contains(err.Error(), table) {
t.Fatalf("expected refusal naming %s, got %v", table, err)
}
assertResetRows(t, db, table, 1)
})
}
}
func TestResetBackupAndRollback(t *testing.T) {
for _, failDelete := range []bool{false, true} {
t.Run(map[bool]string{false: "reset", true: "rollback"}[failDelete], func(t *testing.T) {
path, db := resetFixture(t)
t.Setenv("QFS_BACKUP_DISABLE", "1")
if err := db.Exec("INSERT INTO local_projects (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
if failDelete {
if err := db.Exec("CREATE TRIGGER block_reset BEFORE DELETE ON local_configurations BEGIN SELECT RAISE(ABORT, 'blocked'); END").Error; err != nil {
t.Fatal(err)
}
if err := db.Exec("INSERT INTO local_configurations (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
}
cfg := filepath.Join(filepath.Dir(path), "config.yaml")
if err := os.WriteFile(cfg, []byte("server: {}"), 0600); err != nil {
t.Fatal(err)
}
err := ResetDataWithOptions(path, cfg, true)
if (err != nil) != failDelete {
t.Fatalf("reset: %v", err)
}
want := int64(0)
if failDelete {
want = 1
}
assertResetRows(t, db, "local_projects", want)
archives, err := filepath.Glob(filepath.Join(filepath.Dir(path), "backups", "pre-reset", "*.zip"))
if err != nil || len(archives) != 1 {
t.Fatalf("archives=%v err=%v", archives, err)
}
archive, err := zip.OpenReader(archives[0])
if err != nil {
t.Fatal(err)
}
defer archive.Close()
for _, entry := range archive.File {
if entry.Name != filepath.Base(path) {
continue
}
in, err := entry.Open()
if err != nil {
t.Fatal(err)
}
data, err := io.ReadAll(in)
in.Close()
if err != nil {
t.Fatal(err)
}
restoredPath := filepath.Join(t.TempDir(), "restored.db")
if err := os.WriteFile(restoredPath, data, 0600); err != nil {
t.Fatal(err)
}
restored, err := gorm.Open(sqlite.Open(restoredPath), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := restored.DB()
defer sqlDB.Close()
assertResetRows(t, restored, "local_projects", 1)
}
assertResetRows(t, db, "connection_settings", 1)
})
}
}
func TestResetRefusesFailedBackup(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("INSERT INTO local_projects (sync_status) VALUES ('synced')").Error; err != nil {
t.Fatal(err)
}
t.Setenv("QFS_BACKUP_DIR", path) // A database file cannot be a backup directory.
if err := ResetDataWithOptions(path, "", true); err == nil {
t.Fatal("expected backup refusal")
}
assertResetRows(t, db, "local_projects", 1)
}
func TestResetLegacySchema(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("ALTER TABLE local_projects DROP COLUMN sync_status").Error; err != nil {
t.Fatal(err)
}
if err := ResetDataWithOptions(path, "", false); err == nil {
t.Fatal("expected legacy-state refusal")
}
if err := ResetDataWithOptions(path, "", true); err != nil {
t.Fatal(err)
}
}
func resetFixture(t *testing.T) (string, *gorm.DB) {
t.Helper()
t.Setenv("QFS_BACKUP_DIR", "")
path := filepath.Join(t.TempDir(), "qfs.db")
db, err := gorm.Open(sqlite.Open(path), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := db.DB()
t.Cleanup(func() { sqlDB.Close() })
for _, table := range []string{"pending_changes", "local_projects", "local_configurations", "connection_settings"} {
if err := db.Exec("CREATE TABLE " + table + " (id INTEGER PRIMARY KEY, sync_status TEXT)").Error; err != nil {
t.Fatal(err)
}
}
if err := db.Exec("INSERT INTO connection_settings (sync_status) VALUES ('synced')").Error; err != nil {
t.Fatal(err)
}
return path, db
}
func assertResetRows(t *testing.T, db *gorm.DB, table string, want int64) {
t.Helper()
var count int64
if err := db.Table(table).Count(&count).Error; err != nil {
t.Fatal(err)
}
if count != want {
t.Fatalf("%s has %d rows, want %d", table, count, want)
}
}
+34
View File
@@ -0,0 +1,34 @@
package localdb
import (
mysql "github.com/go-sql-driver/mysql"
"testing"
)
func TestSettingsTLSRoundTrip(t *testing.T) {
t.Setenv("QFS_STATE_DIR", t.TempDir())
local := newModeDB(t)
for _, mode := range []string{"preferred", "required", "verify-full"} {
if err := local.SaveSettingsWithTLS("db.example", 3306, "example", "example", "example", mode, ""); err != nil {
t.Fatal(err)
}
settings, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
if settings.TLSMode != mode {
t.Fatalf("mode=%s, want %s", settings.TLSMode, mode)
}
dsn, err := local.GetDSN()
if err != nil {
t.Fatal(err)
}
cfg, err := mysql.ParseDSN(dsn)
if err != nil {
t.Fatal(err)
}
if cfg.TLS == nil || cfg.AllowFallbackToPlaintext != (mode == "preferred") {
t.Fatalf("wrong TLS policy for %s", mode)
}
}
}
+185
View File
@@ -0,0 +1,185 @@
package mariadb
import (
"archive/zip"
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"gorm.io/gorm"
)
type snapshotTable struct {
Name, Create, Entry, Digest string
Columns []string
Rows int64
}
type snapshotManifest struct {
Format int
Schema, Collation string
Tables []snapshotTable
Triggers []snapshotTrigger
}
func freshDB(db *gorm.DB) *gorm.DB { return db.Session(&gorm.Session{NewDB: true}) }
func quoteIdentifier(s string) string { return "`" + strings.ReplaceAll(s, "`", "``") + "`" }
// CreateSnapshot writes a consistent InnoDB snapshot, without external DB tools.
// The caller supplies a dedicated connection and excludes concurrent writers
// for the entire migration session, including any recovery.
func CreateSnapshot(db *gorm.DB, root, label string) (string, error) {
if err := appstate.ValidateBackupLocation(root); err != nil {
return "", err
}
if filepath.Base(label) != label {
return "", fmt.Errorf("invalid snapshot label")
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create snapshot directory: %w", err)
}
if err := appstate.ValidateBackupLocation(root); err != nil {
return "", err
}
file, err := os.CreateTemp(root, "qfs-mariadb-v1-"+label+"-"+time.Now().UTC().Format("20060102T150405Z")+"-*.zip")
if err != nil {
return "", fmt.Errorf("create snapshot: %w", err)
}
path := file.Name()
archive := zip.NewWriter(file)
err = freshDB(db).Transaction(func(tx *gorm.DB) error {
manifest, err := snapshotInventory(tx)
if err != nil {
return err
}
for i := range manifest.Tables {
table := &manifest.Tables[i]
table.Entry = fmt.Sprintf("tables/%06d.jsonl", i)
out, err := archive.Create(table.Entry)
if err != nil {
return fmt.Errorf("create table entry %s: %w", table.Name, err)
}
if err := snapshotRows(tx, table, out); err != nil {
return err
}
}
out, err := archive.Create("manifest.json")
if err != nil {
return fmt.Errorf("create snapshot manifest: %w", err)
}
if err := json.NewEncoder(out).Encode(manifest); err != nil {
return fmt.Errorf("write snapshot manifest: %w", err)
}
return nil
}, &sql.TxOptions{Isolation: sql.LevelRepeatableRead, ReadOnly: true})
zipErr := archive.Close()
syncErr := file.Sync()
closeErr := file.Close()
for _, candidate := range []error{err, zipErr, syncErr, closeErr} {
if candidate != nil {
return "", fmt.Errorf("write snapshot %s: %w", path, candidate)
}
}
if _, err := readSnapshot(path, nil); err != nil {
return "", err
}
return path, nil
}
func snapshotInventory(db *gorm.DB) (*snapshotManifest, error) {
m := &snapshotManifest{Format: 1}
if err := freshDB(db).Raw("SELECT DATABASE(), @@collation_database").Row().Scan(&m.Schema, &m.Collation); err != nil {
return nil, fmt.Errorf("read snapshot schema: %w", err)
}
var unsupported int64
query := `SELECT
(SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND (TABLE_TYPE<>'BASE TABLE' OR ENGINE<>'InnoDB')) +
(SELECT COUNT(*) FROM information_schema.ROUTINES WHERE ROUTINE_SCHEMA=?) +
(SELECT COUNT(*) FROM information_schema.EVENTS WHERE EVENT_SCHEMA=?) +
(SELECT COUNT(*) FROM information_schema.KEY_COLUMN_USAGE WHERE
(TABLE_SCHEMA=? AND REFERENCED_TABLE_SCHEMA<>?) OR (REFERENCED_TABLE_SCHEMA=? AND TABLE_SCHEMA<>?))`
if err := freshDB(db).Raw(query, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema).Scan(&unsupported).Error; err != nil {
return nil, fmt.Errorf("check snapshot support: %w", err)
}
if unsupported != 0 {
return nil, fmt.Errorf("snapshot refused: views, non-InnoDB tables, routines, events or cross-schema foreign keys require supported recovery before migration")
}
var names []string
if err := freshDB(db).Raw("SELECT TABLE_NAME FROM information_schema.TABLES WHERE TABLE_SCHEMA=? ORDER BY TABLE_NAME", m.Schema).Scan(&names).Error; err != nil {
return nil, fmt.Errorf("list snapshot tables: %w", err)
}
for _, name := range names {
table := snapshotTable{Name: name}
var ignored string
if err := freshDB(db).Raw("SHOW CREATE TABLE "+quoteIdentifier(name)).Row().Scan(&ignored, &table.Create); err != nil {
return nil, fmt.Errorf("read schema for %s: %w", name, err)
}
if err := freshDB(db).Raw(`SELECT COLUMN_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=? AND EXTRA NOT LIKE '%VIRTUAL GENERATED%' AND EXTRA NOT LIKE '%STORED GENERATED%' ORDER BY ORDINAL_POSITION`, m.Schema, name).Scan(&table.Columns).Error; err != nil {
return nil, fmt.Errorf("read columns for %s: %w", name, err)
}
if len(table.Columns) == 0 {
return nil, fmt.Errorf("snapshot table %s has no writable columns", name)
}
m.Tables = append(m.Tables, table)
}
if err := readSnapshotTriggers(db, m); err != nil {
return nil, err
}
return m, nil
}
func snapshotRows(db *gorm.DB, table *snapshotTable, out io.Writer) error {
columns := make([]string, len(table.Columns))
for i, col := range table.Columns {
columns[i] = "CAST(" + quoteIdentifier(col) + " AS BINARY)"
}
rows, err := freshDB(db).Raw("SELECT " + strings.Join(columns, ",") + " FROM " + quoteIdentifier(table.Name)).Rows()
if err != nil {
return fmt.Errorf("read snapshot table %s: %w", table.Name, err)
}
defer rows.Close()
var digest [32]byte
encoder := json.NewEncoder(out)
table.Rows = 0
for rows.Next() {
values := make([]sql.RawBytes, len(columns))
args := make([]any, len(columns))
for i := range args {
args[i] = &values[i]
}
if err := rows.Scan(args...); err != nil {
return fmt.Errorf("read row in %s: %w", table.Name, err)
}
if err := encoder.Encode(values); err != nil {
return fmt.Errorf("write row in %s: %w", table.Name, err)
}
addRowDigest(&digest, values)
table.Rows++
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate %s: %w", table.Name, err)
}
if err := rows.Close(); err != nil {
return fmt.Errorf("close rows in %s: %w", table.Name, err)
}
table.Digest = hex.EncodeToString(digest[:])
return nil
}
func addRowDigest(sum *[32]byte, row []sql.RawBytes) {
encoded, _ := json.Marshal(row) // Raw bytes and nil always marshal successfully.
hash := sha256.Sum256(encoded)
carry := uint16(0)
for i := len(sum) - 1; i >= 0; i-- {
value := uint16(sum[i]) + uint16(hash[i]) + carry
sum[i], carry = byte(value), value>>8
}
}
+101
View File
@@ -0,0 +1,101 @@
package mariadb
import (
"archive/zip"
"database/sql"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
)
func readSnapshot(path string, visit func(snapshotTable, []sql.RawBytes) error) (*snapshotManifest, error) {
if err := appstate.ValidateBackupLocation(path); err != nil {
return nil, err
}
archive, err := zip.OpenReader(path)
if err != nil {
return nil, fmt.Errorf("open server snapshot: %w", err)
}
defer archive.Close()
files := map[string]*zip.File{}
for _, f := range archive.File {
if files[f.Name] != nil {
return nil, fmt.Errorf("duplicate snapshot entry")
}
files[f.Name] = f
}
manifestFile := files["manifest.json"]
if manifestFile == nil || manifestFile.UncompressedSize64 > 16<<20 {
return nil, fmt.Errorf("missing or oversized snapshot manifest")
}
in, err := manifestFile.Open()
if err != nil {
return nil, fmt.Errorf("open server manifest: %w", err)
}
data, readErr := io.ReadAll(in)
closeErr := in.Close()
if readErr != nil || closeErr != nil {
return nil, fmt.Errorf("read server manifest: %w", errors.Join(readErr, closeErr))
}
var m snapshotManifest
if err := json.Unmarshal(data, &m); err != nil {
return nil, fmt.Errorf("parse server manifest: %w", err)
}
if m.Format != 1 || m.Schema == "" || len(files) != len(m.Tables)+1 {
return nil, fmt.Errorf("invalid server snapshot manifest")
}
names, entries := map[string]bool{}, map[string]bool{}
for _, table := range m.Tables {
if table.Name == "" || names[table.Name] || entries[table.Entry] || files[table.Entry] == nil || len(table.Columns) == 0 || table.Rows < 0 || !strings.HasPrefix(table.Create, "CREATE TABLE "+quoteIdentifier(table.Name)+" (") {
return nil, fmt.Errorf("invalid snapshot table metadata")
}
names[table.Name], entries[table.Entry] = true, true
if err := readSnapshotTable(files[table.Entry], table, visit); err != nil {
return nil, err
}
}
return &m, nil
}
func readSnapshotTable(file *zip.File, table snapshotTable, visit func(snapshotTable, []sql.RawBytes) error) error {
in, err := file.Open()
if err != nil {
return fmt.Errorf("open snapshot table %s: %w", table.Name, err)
}
defer in.Close()
decoder := json.NewDecoder(in)
var rows int64
var digest [32]byte
for {
var values []sql.RawBytes
err := decoder.Decode(&values)
if err == io.EOF {
break
}
if err != nil {
return fmt.Errorf("read snapshot row in %s: %w", table.Name, err)
}
if len(values) != len(table.Columns) {
return fmt.Errorf("snapshot column count mismatch in %s", table.Name)
}
addRowDigest(&digest, values)
rows++
if visit != nil {
if err := visit(table, values); err != nil {
return err
}
}
}
if err := in.Close(); err != nil {
return fmt.Errorf("close snapshot table %s: %w", table.Name, err)
}
if rows != table.Rows || hex.EncodeToString(digest[:]) != table.Digest {
return fmt.Errorf("snapshot checksum mismatch in %s", table.Name)
}
return nil
}
+118
View File
@@ -0,0 +1,118 @@
package mariadb
import (
"database/sql"
"errors"
"fmt"
"strings"
"gorm.io/gorm"
)
// RestoreSnapshot restores and checks a snapshot on a dedicated connection.
// The same maintenance exclusion as migration is required throughout recovery.
func RestoreSnapshot(db *gorm.DB, path string) error {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
var schema string
if err := freshDB(db).Raw("SELECT DATABASE()").Scan(&schema).Error; err != nil {
return fmt.Errorf("read restore destination: %w", err)
}
if schema != m.Schema {
return fmt.Errorf("snapshot schema does not match restore destination")
}
if err := restoreSnapshotTables(db, path, m); err != nil {
return err
}
return compareSnapshot(db, m)
}
func restoreSnapshotTables(db *gorm.DB, path string, m *snapshotManifest) (err error) {
current, err := snapshotInventory(db)
if err != nil {
return err
}
var foreignKeys int
var sqlMode string
if err := freshDB(db).Raw("SELECT @@foreign_key_checks, @@sql_mode").Row().Scan(&foreignKeys, &sqlMode); err != nil {
return fmt.Errorf("read restore session policy: %w", err)
}
if err := freshDB(db).Exec("SET FOREIGN_KEY_CHECKS=0").Error; err != nil {
return fmt.Errorf("disable FK checks for restore: %w", err)
}
defer func() {
if restoreErr := freshDB(db).Exec("SET sql_mode=?", sqlMode).Error; restoreErr != nil {
err = errors.Join(err, fmt.Errorf("restore SQL mode: %w", restoreErr))
}
if restoreErr := freshDB(db).Exec("SET FOREIGN_KEY_CHECKS=?", foreignKeys).Error; restoreErr != nil {
err = errors.Join(err, fmt.Errorf("restore FK policy: %w", restoreErr))
}
}()
// Preserve explicit zero auto-increment IDs and legacy zero dates. Exact
// schema/data comparison detects any coercion before the source is changed.
if err := freshDB(db).Exec("SET sql_mode='NO_AUTO_VALUE_ON_ZERO'").Error; err != nil {
return fmt.Errorf("set snapshot insert mode: %w", err)
}
for i := len(current.Tables) - 1; i >= 0; i-- {
if err := freshDB(db).Exec("DROP TABLE " + quoteIdentifier(current.Tables[i].Name)).Error; err != nil {
return fmt.Errorf("drop restore table %s: %w", current.Tables[i].Name, err)
}
}
for _, table := range m.Tables {
if err := freshDB(db).Exec(table.Create).Error; err != nil {
return fmt.Errorf("restore schema for %s: %w", table.Name, err)
}
}
var prefix, rowSQL, tableName string
var args []any
var groups []string
var bytes int
flush := func() error {
if len(groups) == 0 {
return nil
}
// Bind through database/sql: GORM expands a byte slice after '('.
// Direct execution also keeps restored values out of SQL logs.
if _, err := db.Statement.ConnPool.ExecContext(db.Statement.Context, prefix+strings.Join(groups, ","), args...); err != nil {
return fmt.Errorf("restore rows in %s: %w", tableName, err)
}
args, groups, bytes = nil, nil, 0
return nil
}
_, err = readSnapshot(path, func(table snapshotTable, values []sql.RawBytes) error {
if tableName != table.Name {
if err := flush(); err != nil {
return err
}
tableName = table.Name
columns, placeholders := make([]string, len(values)), make([]string, len(values))
for i := range values {
columns[i], placeholders[i] = quoteIdentifier(table.Columns[i]), "?"
}
prefix = "INSERT INTO " + quoteIdentifier(tableName) + " (" + strings.Join(columns, ",") + ") VALUES "
rowSQL = "(" + strings.Join(placeholders, ",") + ")"
}
for _, value := range values {
if value == nil {
args = append(args, nil)
} else {
args = append(args, []byte(value))
bytes += len(value)
}
}
groups = append(groups, rowSQL)
if len(groups) >= 250 || len(args)+len(values) > 60000 || bytes >= 1<<20 {
return flush()
}
return nil
})
if err == nil {
err = flush()
}
if err != nil {
return err
}
return restoreSnapshotTriggers(db, m)
}
+87
View File
@@ -0,0 +1,87 @@
package mariadb
import (
"archive/zip"
"database/sql"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"testing"
)
func TestSnapshotRowsAndValidation(t *testing.T) {
row := []sql.RawBytes{[]byte{0, 1, 255}, {}, nil}
var sum [32]byte
addRowDigest(&sum, row)
for _, tt := range []struct {
name string
format int
rows int64
digest string
width int
valid bool
}{
{"valid", 1, 1, hex.EncodeToString(sum[:]), 3, true},
{"bad format", 2, 1, hex.EncodeToString(sum[:]), 3, false},
{"bad count", 1, 2, hex.EncodeToString(sum[:]), 3, false},
{"bad digest", 1, 1, "invalid", 3, false},
{"bad width", 1, 1, hex.EncodeToString(sum[:]), 2, false},
} {
t.Run(tt.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "backup.zip")
file, err := os.Create(path)
if err != nil {
t.Fatal(err)
}
archive := zip.NewWriter(file)
entry, err := archive.Create("table.jsonl")
if err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(entry).Encode(row); err != nil {
t.Fatal(err)
}
entry, err = archive.Create("manifest.json")
if err != nil {
t.Fatal(err)
}
table := snapshotTable{Name: "example", Create: "CREATE TABLE `example` (id INT)", Entry: "table.jsonl", Columns: []string{"a", "b", "c"}[:tt.width], Rows: tt.rows, Digest: tt.digest}
if err := json.NewEncoder(entry).Encode(snapshotManifest{Format: tt.format, Schema: "example", Tables: []snapshotTable{table}}); err != nil {
t.Fatal(err)
}
if err := archive.Close(); err != nil {
t.Fatal(err)
}
if err := file.Close(); err != nil {
t.Fatal(err)
}
_, err = readSnapshot(path, nil)
if (err == nil) != tt.valid {
t.Fatalf("valid=%v err=%v", tt.valid, err)
}
if !tt.valid {
if err := RestoreSnapshot(nil, path); err == nil {
t.Fatal("invalid archive reached restore without preflight refusal")
}
}
})
}
}
func TestSnapshotDigestPreservesValues(t *testing.T) {
a, b := []sql.RawBytes{[]byte("a")}, []sql.RawBytes{[]byte("b")}
var forward, reverse, null, empty [32]byte
addRowDigest(&forward, a)
addRowDigest(&forward, b)
addRowDigest(&reverse, b)
addRowDigest(&reverse, a)
if forward != reverse {
t.Fatal("digest depends on row order")
}
addRowDigest(&null, []sql.RawBytes{nil})
addRowDigest(&empty, []sql.RawBytes{{}})
if null == empty {
t.Fatal("digest conflates NULL and empty bytes")
}
}
@@ -0,0 +1,43 @@
package mariadb
import (
"fmt"
"regexp"
"strings"
)
// Recovery covers the selected schema, so external trigger effects are refused.
func validateSnapshotTriggerReferences(trigger snapshotTrigger, schema string) error {
body := trigger.Body
quoted := `'(?:''|\\.|[^'\\])*'`
if strings.Contains(trigger.SQLMode, "NO_BACKSLASH_ESCAPES") {
quoted = `'(?:''|[^'])*'`
}
body = regexp.MustCompile(quoted).ReplaceAllString(body, " ")
if !strings.Contains(trigger.SQLMode, "ANSI_QUOTES") {
double := `"(?:""|\\.|[^"\\])*"`
if strings.Contains(trigger.SQLMode, "NO_BACKSLASH_ESCAPES") {
double = `"(?:""|[^"])*"`
}
body = regexp.MustCompile(double).ReplaceAllString(body, " ")
}
if strings.Contains(body, "/*!") || strings.Contains(body, "/*M!") {
return fmt.Errorf("trigger %s has unsupported executable comments", trigger.Name)
}
body = regexp.MustCompile(`(?s)/\*.*?\*/|--[\t ][^\n]*|#[^\n]*`).ReplaceAllString(body, " ")
if regexp.MustCompile(`(?i)\bCALL\s+`).MatchString(body) {
return fmt.Errorf("trigger %s calls a routine outside supported recovery", trigger.Name)
}
identifier := "(?:`(?:``|[^`])+`|\"(?:\"\"|[^\"])+\"|[\\pL_$][\\pL\\pN_$]*)"
for _, pattern := range []string{
"(?i)\\b(?:FROM|JOIN|UPDATE|INTO)\\s+(?:(?:LOW_PRIORITY|HIGH_PRIORITY|DELAYED|IGNORE)\\s+)*(" + identifier + ")\\s*\\.",
"(?i)(" + identifier + ")\\s*\\.\\s*" + identifier + "\\s*\\(",
} {
for _, match := range regexp.MustCompile(pattern).FindAllStringSubmatch(body, -1) {
if !strings.EqualFold(strings.Trim(match[1], "`\""), schema) {
return fmt.Errorf("trigger %s references another schema outside supported recovery", trigger.Name)
}
}
}
return nil
}
@@ -0,0 +1,29 @@
package mariadb
import "testing"
func TestSnapshotTriggerReferences(t *testing.T) {
for _, tt := range []struct {
name, body, mode string
fail bool
}{
{"local", "INSERT INTO audit VALUES (NEW.id, OLD.value)", "", false},
{"local qualified", "UPDATE `example`.`audit` SET id=NEW.id", "", false},
{"alias", "SELECT 1 FROM audit a WHERE a.id=OLD.id", "", false},
{"literal", "INSERT INTO audit VALUES ('example.com', '$.field', 'UPDATE other.audit')", "", false},
{"comment", "/* UPDATE other.audit */ INSERT INTO audit VALUES (NEW.id)", "", false},
{"external write", "UPDATE other.audit SET id=NEW.id", "", true},
{"external insert", "INSERT INTO `other`.`audit` VALUES (NEW.id)", "", true},
{"ANSI external", `UPDATE "other"."audit" SET id=NEW.id`, "ANSI_QUOTES", true},
{"external function", "SET NEW.value=other.example(NEW.value)", "", true},
{"routine", "CALL example.change_rows()", "", true},
{"executable", "/*!50000 UPDATE other.audit SET id=1 */", "", true},
} {
t.Run(tt.name, func(t *testing.T) {
err := validateSnapshotTriggerReferences(snapshotTrigger{Name: "example", Body: tt.body, SQLMode: tt.mode}, "example")
if (err != nil) != tt.fail {
t.Fatalf("error=%v", err)
}
})
}
}
+83
View File
@@ -0,0 +1,83 @@
package mariadb
import (
"errors"
"fmt"
"strings"
"gorm.io/gorm"
)
type snapshotTrigger struct {
Name string `gorm:"column:TRIGGER_NAME"`
Table string `gorm:"column:EVENT_OBJECT_TABLE"`
Event string `gorm:"column:EVENT_MANIPULATION"`
Timing string `gorm:"column:ACTION_TIMING"`
Order int `gorm:"column:ACTION_ORDER"`
Body string `gorm:"column:ACTION_STATEMENT"`
SQLMode string `gorm:"column:SQL_MODE"`
Definer string `gorm:"column:DEFINER"`
Charset string `gorm:"column:CHARACTER_SET_CLIENT"`
Collation string `gorm:"column:COLLATION_CONNECTION"`
DatabaseCollation string `gorm:"column:DATABASE_COLLATION"`
}
func readSnapshotTriggers(db *gorm.DB, m *snapshotManifest) error {
query := `SELECT TRIGGER_NAME, EVENT_OBJECT_TABLE, EVENT_MANIPULATION, ACTION_TIMING,
ACTION_ORDER, ACTION_STATEMENT, SQL_MODE, DEFINER, CHARACTER_SET_CLIENT,
COLLATION_CONNECTION, DATABASE_COLLATION FROM information_schema.TRIGGERS
WHERE TRIGGER_SCHEMA=? ORDER BY EVENT_OBJECT_TABLE, ACTION_TIMING, EVENT_MANIPULATION, ACTION_ORDER`
if err := freshDB(db).Raw(query, m.Schema).Scan(&m.Triggers).Error; err != nil {
return fmt.Errorf("read snapshot triggers: %w", err)
}
for _, trigger := range m.Triggers {
if err := validateSnapshotTriggerReferences(trigger, m.Schema); err != nil {
return err
}
if trigger.DatabaseCollation != m.Collation {
return fmt.Errorf("trigger %s has a legacy database collation; restore support is required before migration", trigger.Name)
}
}
return nil
}
func restoreSnapshotTriggers(db *gorm.DB, m *snapshotManifest) (err error) {
if len(m.Triggers) == 0 {
return nil
}
var mode, charset, collation, schema string
if err := freshDB(db).Raw("SELECT @@sql_mode, @@character_set_client, @@collation_connection, DATABASE()").Row().Scan(&mode, &charset, &collation, &schema); err != nil {
return fmt.Errorf("read trigger restore session: %w", err)
}
defer func() {
for _, reset := range []struct{ sql, value string }{{"SET sql_mode=?", mode}, {"SET character_set_client=?", charset}, {"SET collation_connection=?", collation}} {
if e := freshDB(db).Exec(reset.sql, reset.value).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset trigger restore session: %w", e))
}
}
}()
for _, trigger := range m.Triggers {
if trigger.Timing != "BEFORE" && trigger.Timing != "AFTER" {
return fmt.Errorf("invalid trigger timing")
}
if trigger.Event != "INSERT" && trigger.Event != "UPDATE" && trigger.Event != "DELETE" {
return fmt.Errorf("invalid trigger event")
}
at := strings.LastIndex(trigger.Definer, "@")
if at < 1 || at == len(trigger.Definer)-1 {
return fmt.Errorf("invalid trigger definer")
}
for _, setting := range []struct{ sql, value string }{{"SET sql_mode=?", trigger.SQLMode}, {"SET character_set_client=?", trigger.Charset}, {"SET collation_connection=?", trigger.Collation}} {
if err := freshDB(db).Exec(setting.sql, setting.value).Error; err != nil {
return fmt.Errorf("set trigger restore session: %w", err)
}
}
// Explicitly bind name and target table to the destination. Trigger bodies
// retain their original SQL; verification performs no DML after creation.
statement := "CREATE DEFINER=" + quoteIdentifier(trigger.Definer[:at]) + "@" + quoteIdentifier(trigger.Definer[at+1:]) + " TRIGGER " + quoteIdentifier(schema) + "." + quoteIdentifier(trigger.Name) + " " + trigger.Timing + " " + trigger.Event + " ON " + quoteIdentifier(schema) + "." + quoteIdentifier(trigger.Table) + " FOR EACH ROW " + trigger.Body
if _, err := db.Statement.ConnPool.ExecContext(db.Statement.Context, statement); err != nil {
return fmt.Errorf("restore trigger %s: %w", trigger.Name, err)
}
}
return nil
}
+79
View File
@@ -0,0 +1,79 @@
package mariadb
import (
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"io"
"reflect"
"gorm.io/gorm"
)
// CheckSnapshot compares restored schema and all writable row values.
func CheckSnapshot(db *gorm.DB, path string) error {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
return compareSnapshot(db, m)
}
// VerifySnapshot proves restore readiness in an isolated temporary database.
func VerifySnapshot(db *gorm.DB, path string) (err error) {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
var random [12]byte
if _, err := rand.Read(random[:]); err != nil {
return fmt.Errorf("name verification schema: %w", err)
}
scratch := "qfs_verify_" + hex.EncodeToString(random[:])
var original string
if err := freshDB(db).Raw("SELECT DATABASE()").Scan(&original).Error; err != nil {
return fmt.Errorf("read original schema: %w", err)
}
if err := freshDB(db).Exec("CREATE DATABASE " + quoteIdentifier(scratch) + " COLLATE " + quoteIdentifier(m.Collation)).Error; err != nil {
return fmt.Errorf("create restore verification schema: %w", err)
}
defer func() {
if e := freshDB(db).Exec("USE " + quoteIdentifier(original)).Error; e != nil {
err = errors.Join(err, fmt.Errorf("return to original schema: %w", e))
}
if e := freshDB(db).Exec("DROP DATABASE " + quoteIdentifier(scratch)).Error; e != nil {
err = errors.Join(err, fmt.Errorf("remove verification schema %s: %w", scratch, e))
}
}()
if err := freshDB(db).Exec("USE " + quoteIdentifier(scratch)).Error; err != nil {
return fmt.Errorf("select verification schema: %w", err)
}
if err := restoreSnapshotTables(db, path, m); err != nil {
return err
}
return compareSnapshot(db, m)
}
func compareSnapshot(db *gorm.DB, m *snapshotManifest) error {
current, err := snapshotInventory(db)
if err != nil {
return err
}
if !reflect.DeepEqual(current.Triggers, m.Triggers) {
return fmt.Errorf("restored trigger metadata mismatch")
}
if len(current.Tables) != len(m.Tables) {
return fmt.Errorf("restored table count mismatch")
}
for i := range m.Tables {
want, got := m.Tables[i], current.Tables[i]
if err := snapshotRows(db, &got, io.Discard); err != nil {
return err
}
if want.Name != got.Name || want.Create != got.Create || want.Rows != got.Rows || want.Digest != got.Digest {
return fmt.Errorf("restored snapshot mismatch in %s (name=%t schema=%t rows=%t digest=%t)", want.Name, want.Name == got.Name, want.Create == got.Create, want.Rows == got.Rows, want.Digest == got.Digest)
}
}
return nil
}
+71
View File
@@ -0,0 +1,71 @@
package mariadb
import (
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"fmt"
"strings"
mysql "github.com/go-sql-driver/mysql"
)
// ConfigureTLS configures the DSN so setup and runtime use the same TLS policy.
func ConfigureTLS(cfg *mysql.Config, mode, caPEM string) error {
mode = strings.TrimSpace(mode)
if mode == "" {
mode = "preferred"
}
caPEM = strings.TrimSpace(caPEM)
if caPEM != "" && mode != "verify-ca" && mode != "verify-full" {
return fmt.Errorf("CA certificate requires verify-ca or verify-full")
}
switch mode {
case "preferred":
cfg.TLSConfig = "preferred"
return nil
case "required":
cfg.TLSConfig = "skip-verify"
return nil
case "verify-full":
if caPEM == "" {
cfg.TLSConfig = "true"
return nil
}
case "verify-ca":
if caPEM == "" {
return fmt.Errorf("verify-ca requires a CA certificate")
}
default:
return fmt.Errorf("unsupported MariaDB TLS mode")
}
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM([]byte(caPEM)) {
return fmt.Errorf("invalid CA certificate PEM")
}
tlsConfig := &tls.Config{MinVersion: tls.VersionTLS12, RootCAs: roots}
if mode == "verify-ca" {
// Verify the certificate chain explicitly; this mode omits hostname checks.
tlsConfig.InsecureSkipVerify = true
tlsConfig.VerifyConnection = func(state tls.ConnectionState) error {
if len(state.PeerCertificates) == 0 {
return fmt.Errorf("MariaDB did not provide a certificate")
}
intermediates := x509.NewCertPool()
for _, cert := range state.PeerCertificates[1:] {
intermediates.AddCert(cert)
}
_, err := state.PeerCertificates[0].Verify(x509.VerifyOptions{Roots: roots, Intermediates: intermediates, KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}})
if err != nil {
return fmt.Errorf("verify MariaDB certificate: %w", err)
}
return nil
}
}
name := fmt.Sprintf("qfs-%x", sha256.Sum256([]byte(mode+"\n"+caPEM)))
if err := mysql.RegisterTLSConfig(name, tlsConfig); err != nil {
return fmt.Errorf("register MariaDB TLS: %w", err)
}
cfg.TLSConfig = name
return nil
}
+128
View File
@@ -0,0 +1,128 @@
//go:build integration
package mariadb
import (
"context"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"database/sql"
"encoding/pem"
"math/big"
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
mysql "github.com/go-sql-driver/mysql"
)
func TestMariaDBTLSIntegration(t *testing.T) {
dir := t.TempDir()
if err := os.Chmod(dir, 0755); err != nil {
t.Fatal(err)
}
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatal(err)
}
cert := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "example"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}}
der, err := x509.CreateCertificate(rand.Reader, cert, cert, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
ca := string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}))
for name, data := range map[string][]byte{"server.crt": []byte(ca), "server.key": pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)})} {
// Docker's mysql user must read these disposable test files.
if err := os.WriteFile(filepath.Join(dir, name), data, 0644); err != nil {
t.Fatal(err)
}
}
otherCA, _ := testCertificate(t)
for _, tlsEnabled := range []bool{true, false} {
t.Run(map[bool]string{true: "TLS", false: "plaintext"}[tlsEnabled], func(t *testing.T) {
args := []string{"run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306",
"-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "-v", dir + ":/qfs-tls:ro", "mariadb:11.8"}
if tlsEnabled {
args = append(args, "--ssl-cert=/qfs-tls/server.crt", "--ssl-key=/qfs-tls/server.key")
} else {
args = append(args, "--skip-ssl")
}
out, err := exec.Command("docker", args...).CombinedOutput()
if err != nil {
t.Fatalf("start disposable MariaDB: %v: %s", err, out)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if out, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop test MariaDB: %v: %s", err, out)
}
})
out, err = exec.Command("docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatalf("test MariaDB port: %v: %s", err, out)
}
addr := strings.TrimSpace(string(out))
connect := func(mode, roots, serverName string) (string, error) {
cfg := mysql.NewConfig()
cfg.User, cfg.Net, cfg.Addr, cfg.Timeout = "root", "tcp", addr, time.Second
if err := ConfigureTLS(cfg, mode, roots); err != nil {
return "", err
}
cfg, err = mysql.ParseDSN(cfg.FormatDSN())
if err != nil {
return "", err
}
if serverName != "" {
cfg.TLS.ServerName = serverName
}
connector, err := mysql.NewConnector(cfg)
if err != nil {
return "", err
}
db := sql.OpenDB(connector)
defer db.Close()
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
var name, cipher string
err = db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&name, &cipher)
return cipher, err
}
deadline := time.Now().Add(45 * time.Second)
for {
if _, err := connect("preferred", "", ""); err == nil {
break
}
if time.Now().After(deadline) {
t.Fatal("test MariaDB did not become ready")
}
time.Sleep(100 * time.Millisecond)
}
for _, tt := range []struct {
mode, roots, name string
success bool
}{
{"preferred", "", "", true}, {"required", "", "", tlsEnabled},
{"verify-ca", ca, "", tlsEnabled}, {"verify-full", ca, "", tlsEnabled},
{"verify-ca", otherCA, "", false}, {"verify-full", otherCA, "", false},
{"verify-full", ca, "wrong.example", false},
} {
cipher, err := connect(tt.mode, tt.roots, tt.name)
if (err == nil) != tt.success {
t.Errorf("mode=%s name=%s success=%v error=%v", tt.mode, tt.name, tt.success, err)
}
if err == nil && (cipher != "") != tlsEnabled {
t.Errorf("mode=%s cipher=%q TLS=%v", tt.mode, cipher, tlsEnabled)
}
}
})
}
}
+97
View File
@@ -0,0 +1,97 @@
package mariadb
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"testing"
"time"
mysql "github.com/go-sql-driver/mysql"
)
func TestTLSPolicy(t *testing.T) {
ca, cert := testCertificate(t)
_, other := testCertificate(t)
for _, tt := range []struct {
mode, ca string
wantError, fallback, insecure bool
}{
{"", "", false, true, true},
{"preferred", "", false, true, true},
{"required", "", false, false, true},
{"verify-full", "", false, false, false},
{"verify-full", ca, false, false, false},
{"verify-ca", ca, false, false, true},
{"verify-ca", "", true, false, false},
{"verify-full", "invalid", true, false, false},
{"preferred", ca, true, false, false},
{"false", "", true, false, false},
} {
t.Run(tt.mode+tt.ca[:min(len(tt.ca), 5)], func(t *testing.T) {
cfg := mysql.NewConfig()
cfg.Net, cfg.Addr = "tcp", "db.example:3306"
err := ConfigureTLS(cfg, tt.mode, tt.ca)
if (err != nil) != tt.wantError {
t.Fatalf("configure: %v", err)
}
if tt.wantError {
return
}
parsed, err := mysql.ParseDSN(cfg.FormatDSN())
if err != nil {
t.Fatal(err)
}
if parsed.TLS == nil || parsed.AllowFallbackToPlaintext != tt.fallback || parsed.TLS.InsecureSkipVerify != tt.insecure {
t.Fatalf("unexpected TLS policy for %s", tt.mode)
}
if tt.mode == "verify-ca" {
if err := parsed.TLS.VerifyConnection(tls.ConnectionState{PeerCertificates: []*x509.Certificate{cert}}); err != nil {
t.Fatal(err)
}
for _, chain := range [][]*x509.Certificate{nil, {other}} {
if err := parsed.TLS.VerifyConnection(tls.ConnectionState{PeerCertificates: chain}); err == nil {
t.Fatal("untrusted certificate accepted")
}
}
}
if tt.mode == "verify-full" && tt.ca != "" {
if parsed.TLS.ServerName != "db.example" {
t.Fatalf("missing hostname verification: %s", parsed.TLS.ServerName)
}
if _, err := cert.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: parsed.TLS.ServerName}); err != nil {
t.Fatal(err)
}
if _, err := other.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: parsed.TLS.ServerName}); err == nil {
t.Fatal("untrusted certificate accepted")
}
if _, err := cert.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: "other.example"}); err == nil {
t.Fatal("wrong hostname accepted")
}
}
})
}
}
func testCertificate(t *testing.T) (string, *x509.Certificate) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "example"}, DNSNames: []string{"db.example"}, NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
cert, err := x509.ParseCertificate(der)
if err != nil {
t.Fatal(err)
}
return string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})), cert
}
+27 -1
View File
@@ -1,6 +1,7 @@
package middleware
import (
"mime"
"net"
"net/http"
"net/url"
@@ -11,6 +12,19 @@ import (
func CORS() gin.HandlerFunc {
return func(c *gin.Context) {
hostURL, hostErr := url.Parse("http://" + c.Request.Host)
if hostErr != nil || hostURL.Host != c.Request.Host || hostURL.Path != "" || hostURL.RawQuery != "" || !isLoopbackOrigin("http://"+c.Request.Host) {
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "Invalid request host"})
return
}
if c.GetHeader("Sec-Fetch-Site") == "cross-site" {
c.AbortWithStatus(http.StatusForbidden)
return
}
if referer := c.GetHeader("Referer"); referer != "" && !isLoopbackOrigin(referer) {
c.AbortWithStatus(http.StatusForbidden)
return
}
origin := strings.TrimSpace(c.GetHeader("Origin"))
if origin != "" {
if isLoopbackOrigin(origin) {
@@ -20,7 +34,7 @@ func CORS() gin.HandlerFunc {
c.Header("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization")
c.Header("Access-Control-Expose-Headers", "Content-Length, Content-Disposition")
c.Header("Access-Control-Max-Age", "86400")
} else if c.Request.Method == http.MethodOptions {
} else {
c.AbortWithStatus(http.StatusForbidden)
return
}
@@ -30,6 +44,15 @@ func CORS() gin.HandlerFunc {
c.AbortWithStatus(http.StatusNoContent)
return
}
if c.Request.Method != http.MethodGet && c.Request.Method != http.MethodHead &&
strings.HasPrefix(c.Request.URL.Path, "/api/") && c.Request.ContentLength != 0 {
mediaType, _, err := mime.ParseMediaType(c.GetHeader("Content-Type"))
isUpload := strings.HasPrefix(c.FullPath(), "/api/projects/") && strings.HasSuffix(c.FullPath(), "/vendor-import")
if err != nil || (mediaType != "application/json" && !(isUpload && mediaType == "multipart/form-data")) {
c.AbortWithStatusJSON(http.StatusUnsupportedMediaType, gin.H{"error": "Unsupported content type"})
return
}
}
c.Next()
}
@@ -43,6 +66,9 @@ func isLoopbackOrigin(origin string) bool {
if u.Scheme != "http" && u.Scheme != "https" {
return false
}
if u.User != nil || u.Opaque != "" {
return false
}
host := strings.TrimSpace(u.Hostname())
if host == "" {
return false
+66
View File
@@ -0,0 +1,66 @@
package middleware
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
)
func TestLocalRequestBoundary(t *testing.T) {
gin.SetMode(gin.TestMode)
tests := []struct {
name, method, path, host, origin, referer, fetchSite, contentType, body string
want int
}{
{name: "CLI JSON", method: "POST", contentType: "application/json", body: "{}", want: 200},
{name: "local browser", method: "POST", origin: "http://localhost:8080", contentType: "application/json; charset=utf-8", body: "{}", want: 200},
{name: "foreign form", method: "POST", path: "/setup", origin: "https://attacker.example", contentType: "application/x-www-form-urlencoded", body: "host=example", want: 403},
{name: "foreign JSON", method: "POST", origin: "https://attacker.example", contentType: "text/plain", body: "{}", want: 403},
{name: "rebind read", method: "GET", host: "attacker.example:8080", want: 403},
{name: "null origin", method: "POST", origin: "null", want: 403},
{name: "foreign referer", method: "POST", referer: "https://attacker.example/page", want: 403},
{name: "cross site", method: "POST", fetchSite: "cross-site", want: 403},
{name: "plain JSON", method: "POST", contentType: "text/plain", body: "{}", want: 415},
{name: "API form", method: "POST", contentType: "application/x-www-form-urlencoded", body: "x=y", want: 415},
{name: "setup form", method: "POST", path: "/setup", origin: "http://127.0.0.1:8080", contentType: "application/x-www-form-urlencoded", body: "host=example", want: 200},
{name: "upload", method: "POST", path: "/api/projects/id/vendor-import", contentType: "multipart/form-data; boundary=example", body: "example", want: 200},
{name: "multipart JSON route", method: "POST", contentType: "multipart/form-data; boundary=example", body: "example", want: 415},
{name: "bodyless action", method: "POST", want: 200},
{name: "local preflight", method: "OPTIONS", origin: "http://localhost:8080", want: 204},
{name: "foreign preflight", method: "OPTIONS", origin: "https://attacker.example", want: 403},
{name: "IPv6", method: "GET", host: "[::1]:8080", want: 200},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
router := gin.New()
router.Use(CORS())
called := false
handler := func(c *gin.Context) { called = true; c.Status(200) }
router.Any("/api/configs", handler)
router.Any("/setup", handler)
router.POST("/api/projects/:uuid/vendor-import", handler)
path := tt.path
if path == "" {
path = "/api/configs"
}
req := httptest.NewRequest(tt.method, path, strings.NewReader(tt.body))
req.Host = tt.host
if req.Host == "" {
req.Host = "127.0.0.1:8080"
}
for key, value := range map[string]string{"Origin": tt.origin, "Referer": tt.referer, "Sec-Fetch-Site": tt.fetchSite, "Content-Type": tt.contentType} {
if value != "" {
req.Header.Set(key, value)
}
}
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != tt.want || called != (tt.want == http.StatusOK) {
t.Fatalf("status=%d called=%v, want %d", w.Code, called, tt.want)
}
})
}
}
+4 -55
View File
@@ -1,57 +1,6 @@
package models
import (
"log/slog"
"strings"
"gorm.io/gorm"
)
// AllModels returns all models for auto-migration
func AllModels() []interface{} {
return []interface{}{
&Category{},
&Project{},
&Configuration{},
&Pricelist{},
&PricelistItem{},
}
}
// Migrate runs auto-migration for all QuoteForge tables
// Handles MySQL constraint errors gracefully for existing tables
func Migrate(db *gorm.DB) error {
for _, model := range AllModels() {
if err := db.AutoMigrate(model); err != nil {
// Skip known MySQL constraint errors for existing tables
errStr := err.Error()
if strings.Contains(errStr, "Can't DROP") ||
strings.Contains(errStr, "Duplicate key name") ||
strings.Contains(errStr, "check that it exists") ||
strings.Contains(errStr, "Cannot change column") ||
strings.Contains(errStr, "used in a foreign key constraint") {
slog.Warn("migration warning (skipped)", "model", model, "error", errStr)
continue
}
return err
}
}
return nil
}
// SeedCategories upserts default categories, updating display_order on existing rows.
func SeedCategories(db *gorm.DB) error {
for _, cat := range DefaultCategories {
var existing Category
if err := db.Where("code = ?", cat.Code).First(&existing).Error; err != nil {
if err := db.Create(&cat).Error; err != nil {
return err
}
} else {
if err := db.Model(&existing).Update("display_order", cat.DisplayOrder).Error; err != nil {
return err
}
}
}
return nil
}
// Server schema changes are SQL files in migrations/, applied by `qfs -migrate`
// (models.RunSQLMigrations). There is deliberately no GORM AutoMigrate against
// MariaDB: it retypes production columns to match whatever models the running binary
// was built with — an old binary run with -migrate once did exactly that.
@@ -0,0 +1,162 @@
//go:build integration
package models
import (
"context"
"os"
"os/exec"
"regexp"
"strings"
"testing"
"testing/fstest"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// Input is read-only. All SQL runs in a new, disposable local container.
func TestProductionDumpMigrationMariaDB(t *testing.T) {
dump := os.Getenv("QFS_TEST_PRODUCTION_DUMP")
if dump == "" {
t.Skip("QFS_TEST_PRODUCTION_DUMP is not set")
}
input, err := os.Open(dump)
if err != nil {
t.Fatal(err)
}
defer input.Close()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
image := os.Getenv("QFS_TEST_MARIADB_IMAGE")
if image == "" {
image = "mariadb:11.8.3"
}
out, err := exec.CommandContext(ctx, "docker", "run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306",
"-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "-e", "MARIADB_DATABASE=RFQ_LOG", image,
"--skip-ssl", "--character-set-server=utf8mb4", "--collation-server=utf8mb4_uca1400_ai_ci", "--max-allowed-packet=256M").CombinedOutput()
if err != nil {
t.Fatalf("start dump rehearsal MariaDB: %v", err)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if _, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop dump rehearsal MariaDB: %v", err)
}
})
out, err = exec.CommandContext(ctx, "docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatal(err)
}
dsn := "root@tcp(" + strings.TrimSpace(string(out)) + ")/RFQ_LOG?parseTime=true&timeout=1s&tls=false"
var db *gorm.DB
deadline := time.Now().Add(45 * time.Second)
for {
db, err = gorm.Open(mysql.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err == nil {
break
}
if time.Now().After(deadline) {
t.Fatal("dump rehearsal database did not become ready")
}
time.Sleep(150 * time.Millisecond)
}
pool, err := db.DB()
if err != nil {
t.Fatal(err)
}
defer pool.Close()
pool.SetMaxOpenConns(1)
pool.SetMaxIdleConns(1)
importer := exec.CommandContext(ctx, "docker", "exec", "-i", id, "mariadb", "--skip-ssl", "--binary-mode", "--default-character-set=utf8mb4", "RFQ_LOG")
importer.Stdin = input
if output, err := importer.CombinedOutput(); err != nil {
// Import errors can include entire data rows; report only error code/line.
location := regexp.MustCompile(`(?m)^ERROR[^\n]*`).Find(output)
location = regexp.MustCompile(`'[^']*'`).ReplaceAll(location, []byte("'<redacted>'"))
if len(location) > 250 {
location = location[:250]
}
t.Fatalf("load dump into disposable database: %v (%s)", err, location)
}
var version, collation string
if err := db.Raw("SELECT VERSION(), @@collation_database").Row().Scan(&version, &collation); err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(version, "11.8.3-") || collation != "utf8mb4_uca1400_ai_ci" {
t.Fatalf("unexpected rehearsal server: version=%s collation=%s", version, collation)
}
var tables, triggers int64
if err := db.Raw("SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA=DATABASE()").Scan(&tables).Error; err != nil {
t.Fatal(err)
}
if err := db.Raw("SELECT COUNT(*) FROM information_schema.TRIGGERS WHERE TRIGGER_SCHEMA=DATABASE()").Scan(&triggers).Error; err != nil {
t.Fatal(err)
}
t.Logf("production dump imported: MariaDB=%s collation=%s tables=%d triggers=%d", version, collation, tables, triggers)
if err := db.Exec("SET time_zone='+00:00'").Error; err != nil {
t.Fatal(err)
}
root := t.TempDir()
var original string
if err := db.Connection(func(conn *gorm.DB) error {
var err error
original, err = mariadb.CreateSnapshot(conn, root, "production-dump")
if err != nil {
return err
}
return mariadb.VerifySnapshot(conn, original)
}); err != nil {
t.Fatal(err)
}
t.Log("full dump snapshot restore verified")
const filename = "035_add_server_updated_at.sql"
if err := db.Where("filename=?", filename).Delete(&SQLSchemaMigration{}).Error; err != nil {
t.Fatal(err)
}
// Reconstruct the real pre-035 schema only in this disposable clone.
for _, table := range []string{"qt_configurations", "qt_projects"} {
if err := db.Exec("ALTER TABLE " + table + " DROP INDEX IF EXISTS idx_" + table + "_server_updated_at, DROP COLUMN IF EXISTS server_updated_at").Error; err != nil {
t.Fatal(err)
}
}
var before string
if err := db.Connection(func(conn *gorm.DB) error {
var err error
before, err = mariadb.CreateSnapshot(conn, root, "before-rehearsal")
return err
}); err != nil {
t.Fatal(err)
}
content, err := qfassets.MigrationFiles.ReadFile("migrations/" + filename)
if err != nil {
t.Fatal(err)
}
files := fstest.MapFS{filename: &fstest.MapFile{Data: content}, "036_recovery_probe.sql": &fstest.MapFile{Data: []byte("-- Tables affected: qt_configurations\n-- verify: deliberate rehearsal failure | SELECT 1\nALTER TABLE qt_configurations ADD COLUMN qfs_recovery_probe INT;")}}
if _, err := RunSQLMigrations(db, files, nil, root); err == nil || !strings.Contains(err.Error(), "all session changes restored") {
t.Fatalf("dump rollback rehearsal: %v", err)
}
if err := mariadb.CheckSnapshot(db, before); err != nil {
t.Fatalf("dump changed after rollback: %v", err)
}
t.Log("real migration plus later failed DDL/verify restored the complete dump")
delete(files, "036_recovery_probe.sql")
if applied, err := RunSQLMigrations(db, files, nil, root); err != nil || len(applied) != 1 {
t.Fatalf("real migration rehearsal: applied=%v error=%v", applied, err)
}
states, err := SQLMigrationStatus(db, files)
if err != nil || len(states) != 1 || states[0].AppliedAt == nil {
t.Fatalf("rehearsal history invalid: %v", err)
}
if err := RestoreSQLMigrationSession(db, original); err != nil {
t.Fatal(err)
}
if err := mariadb.CheckSnapshot(db, original); err != nil {
t.Fatal(err)
}
t.Logf("production dump rehearsal complete: %s, exact original restored", filename)
}
@@ -0,0 +1,44 @@
package models
import (
"crypto/sha256"
"fmt"
"strings"
"gorm.io/driver/mysql"
"gorm.io/gorm"
)
// migrationJournalName uses the configured endpoint, which survives server restarts.
func migrationJournalName(db *gorm.DB, schema string) (string, error) {
dialect, ok := db.Dialector.(*mysql.Dialector)
if !ok || dialect.DSNConfig == nil {
return "", fmt.Errorf("migration requires a MariaDB DSN")
}
identity := dialect.DSNConfig.Net + "/" + dialect.DSNConfig.Addr + "/" + schema
return fmt.Sprintf("qfs-migration-v1-%x.pending", sha256.Sum256([]byte(identity))), nil
}
func requireExclusiveMigrationSession(db *gorm.DB) error {
var grants []string
if err := db.Session(&gorm.Session{NewDB: true}).Raw("SHOW GRANTS FOR CURRENT_USER").Scan(&grants).Error; err != nil {
return fmt.Errorf("read maintenance privileges: %w", err)
}
allowed := false
for _, grant := range grants {
if strings.Contains(grant, " ON *.* ") && strings.Contains(grant, "ALL PRIVILEGES") {
allowed = true
}
}
if !allowed {
return fmt.Errorf("migration recovery requires a DBA with directly granted global ALL PRIVILEGES to inspect all objects and clients")
}
var clients int64
if err := db.Session(&gorm.Session{NewDB: true}).Raw("SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE ID<>CONNECTION_ID() AND DB=DATABASE()").Scan(&clients).Error; err != nil {
return fmt.Errorf("check maintenance clients: %w", err)
}
if clients != 0 {
return fmt.Errorf("migration requires maintenance window: %d other connections use this schema", clients)
}
return nil
}
+45
View File
@@ -0,0 +1,45 @@
package models
import (
"fmt"
"io/fs"
"regexp"
"gorm.io/gorm"
)
func planSQLMigrations(db *gorm.DB, migrations fs.FS) ([]plannedSQLMigration, error) {
states, err := SQLMigrationStatus(db, migrations)
if err != nil {
return nil, err
}
var plans []plannedSQLMigration
unsupported := regexp.MustCompile(`(?i)\b(?:CREATE|DROP|ALTER)\s+(?:DATABASE|SCHEMA|VIEW|TRIGGER|PROCEDURE|FUNCTION|EVENT)\b|\b(?:USE|DELIMITER|GRANT|REVOKE)\s|\bSET\s+(?:GLOBAL|SQL_MODE)\b|\bENGINE\s*=\s*(?:MyISAM|Aria|MEMORY|BLACKHOLE)\b`)
for _, state := range states {
if state.AppliedAt != nil {
continue
}
content, err := fs.ReadFile(migrations, state.Filename)
if err != nil {
return nil, fmt.Errorf("read migration %s: %w", state.Filename, err)
}
statements, err := splitSQLStatements(string(content))
if err != nil {
return nil, fmt.Errorf("parse migration %s: %w", state.Filename, err)
}
checks, err := parseMigrationVerifications(string(content))
if err != nil {
return nil, fmt.Errorf("migration %s verify headers: %w", state.Filename, err)
}
if len(checks) == 0 {
return nil, fmt.Errorf("migration %s needs verify headers", state.Filename)
}
for _, stmt := range statements {
if unsupported.MatchString(stmt) {
return nil, fmt.Errorf("migration %s contains SQL outside supported table recovery", state.Filename)
}
}
plans = append(plans, plannedSQLMigration{state.Filename, statements, checks})
}
return plans, nil
}
+67
View File
@@ -0,0 +1,67 @@
package models
import (
"crypto/sha256"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/gorm"
)
// RestoreSQLMigrationSession is the shipped recovery path after interruption.
func RestoreSQLMigrationSession(db *gorm.DB, snapshot string) error {
return db.Connection(func(conn *gorm.DB) (err error) {
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
var schema, zone string
var acquired int
if err := fresh().Raw("SELECT DATABASE(), @@time_zone").Row().Scan(&schema, &zone); err != nil {
return fmt.Errorf("read recovery session: %w", err)
}
lock := fmt.Sprintf("qfs-migrate-%x", sha256.Sum256([]byte(schema)))
if err := fresh().Raw("SELECT GET_LOCK(?,0)", lock).Scan(&acquired).Error; err != nil || acquired != 1 {
return fmt.Errorf("migration lock unavailable for recovery")
}
defer func() {
if e := fresh().Exec("SET time_zone=?", zone).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset recovery time zone: %w", e))
}
if e := fresh().Raw("SELECT RELEASE_LOCK(?)", lock).Scan(&acquired).Error; e != nil {
err = errors.Join(err, fmt.Errorf("release recovery lock: %w", e))
}
}()
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return err
}
journalName, err := migrationJournalName(conn, schema)
if err != nil {
return err
}
journal := filepath.Join(filepath.Dir(snapshot), journalName)
if content, e := os.ReadFile(journal); e == nil {
if strings.TrimSpace(string(content)) != snapshot {
return fmt.Errorf("pending recovery requires its original snapshot: %s", strings.TrimSpace(string(content)))
}
} else if !os.IsNotExist(e) {
return fmt.Errorf("read recovery marker: %w", e)
}
if err := fresh().Exec("SET time_zone='+00:00'").Error; err != nil {
return fmt.Errorf("set restore time zone: %w", err)
}
if err := mariadb.VerifySnapshot(fresh(), snapshot); err != nil {
return fmt.Errorf("verify recovery snapshot: %w", err)
}
if err := mariadb.RestoreSnapshot(fresh(), snapshot); err != nil {
return fmt.Errorf("restore migration session: %w", err)
}
if err := os.Remove(journal); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("clear recovery marker: %w", err)
}
slog.Info("server migration snapshot restored", "schema", schema, "backup", snapshot, "state", "verified")
return nil
})
}
+165
View File
@@ -0,0 +1,165 @@
package models
import (
"crypto/sha256"
"errors"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/gorm"
)
type plannedSQLMigration struct {
filename string
statements []string
checks []migrationVerification
}
// RunSQLMigrations requires a DBA maintenance window: stop all other writers.
// Backups are mandatory; each file gets per-table snapshots before any DDL.
func RunSQLMigrations(db *gorm.DB, migrations fs.FS, onApply func(string), backupRoot string) ([]string, error) {
var applied []string
err := db.Connection(func(conn *gorm.DB) (err error) {
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
var schema, mode, zone, engine string
var foreignKeys int
if err := fresh().Raw("SELECT DATABASE(), @@sql_mode, @@time_zone, @@foreign_key_checks, @@default_storage_engine").Row().Scan(&schema, &mode, &zone, &foreignKeys, &engine); err != nil {
return fmt.Errorf("read migration session: %w", err)
}
if schema == "" || strings.Contains(mode, "NO_BACKSLASH_ESCAPES") || engine != "InnoDB" {
return fmt.Errorf("migration requires a selected InnoDB schema and backslash escape SQL mode")
}
lock := fmt.Sprintf("qfs-migrate-%x", sha256.Sum256([]byte(schema)))
var acquired int
if err := fresh().Raw("SELECT GET_LOCK(?,0)", lock).Scan(&acquired).Error; err != nil || acquired != 1 {
return fmt.Errorf("migration lock unavailable for %s", schema)
}
defer func() {
for _, reset := range []struct {
sql string
value any
}{{"SET time_zone=?", zone}, {"SET sql_mode=?", mode}, {"SET foreign_key_checks=?", foreignKeys}} {
if e := fresh().Exec(reset.sql, reset.value).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset migration session: %w", e))
}
}
if e := fresh().Raw("SELECT RELEASE_LOCK(?)", lock).Scan(&acquired).Error; e != nil {
err = errors.Join(err, fmt.Errorf("release migration lock: %w", e))
}
}()
plans, err := planSQLMigrations(fresh(), migrations)
if err != nil {
return err
}
if err := appstate.ValidateBackupLocation(backupRoot); err != nil {
return err
}
if err := os.MkdirAll(backupRoot, 0700); err != nil {
return fmt.Errorf("create migration backup root: %w", err)
}
journalName, err := migrationJournalName(conn, schema)
if err != nil {
return err
}
journal := filepath.Join(backupRoot, journalName)
if content, e := os.ReadFile(journal); e == nil {
return fmt.Errorf("interrupted migration requires restore before retry; safety snapshot: %s", strings.TrimSpace(string(content)))
} else if !os.IsNotExist(e) {
return fmt.Errorf("read migration recovery marker: %w", e)
}
if len(plans) == 0 {
return nil
}
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return err
}
if err := fresh().Exec("SET time_zone='+00:00'").Error; err != nil {
return fmt.Errorf("set snapshot time zone: %w", err)
}
full, err := mariadb.CreateSnapshot(fresh(), backupRoot, "session")
if err != nil {
return fmt.Errorf("pre-migration snapshot: %w", err)
}
if err := mariadb.VerifySnapshot(fresh(), full); err != nil {
return fmt.Errorf("pre-migration restore verification: %w", err)
}
marker, err := os.OpenFile(journal, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
return fmt.Errorf("create migration recovery marker: %w", err)
}
_, writeErr := fmt.Fprintln(marker, full)
syncErr, closeErr := marker.Sync(), marker.Close()
if e := errors.Join(writeErr, syncErr, closeErr); e != nil {
return fmt.Errorf("persist migration recovery marker: %w", e)
}
slog.Info("verified server migration backup", "schema", schema, "path", full)
var steps []string
for _, plan := range plans {
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
step, err := mariadb.CreateSnapshot(fresh(), backupRoot, plan.filename)
if err == nil {
err = mariadb.VerifySnapshot(fresh(), step)
}
if err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
steps = append(steps, step)
if onApply != nil {
onApply(plan.filename)
}
if err := ensureSQLMigrationsTable(fresh()); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
if err := applySQLMigrationOnSession(fresh(), plan.filename, plan.statements, plan.checks); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
applied = append(applied, plan.filename)
}
if err := os.Remove(journal); err != nil {
return fmt.Errorf("remove completed migration marker: %w", err)
}
return nil
})
if err != nil {
return nil, err
}
return applied, nil
}
func recoverSQLMigration(db *gorm.DB, journal, full string, steps []string, filename string, cause error) error {
slog.Error("server migration failed", "step", filename, "error", cause, "backup", full)
if err := requireExclusiveMigrationSession(db); err != nil {
return errors.Join(cause, fmt.Errorf("recovery refused with active clients; keep maintenance window and restore %s: %w", full, err))
}
var recovery error
for i := len(steps) - 1; i >= 0; i-- {
slog.Warn("restoring migration step", "step", filename, "backup", steps[i])
if err := mariadb.RestoreSnapshot(db, steps[i]); err != nil {
recovery = err
break
}
}
if recovery == nil {
recovery = mariadb.CheckSnapshot(db, full)
}
if recovery != nil {
slog.Warn("restoring full migration session", "backup", full, "error", recovery)
recovery = mariadb.RestoreSnapshot(db, full)
}
if recovery != nil {
return errors.Join(cause, fmt.Errorf("recovery failed; database requires restore from %s: %w", full, recovery))
}
if err := os.Remove(journal); err != nil {
return errors.Join(cause, fmt.Errorf("database restored but recovery marker remains: %w", err))
}
slog.Warn("server migration session restored", "step", filename, "state", "before-session", "backup", full)
return fmt.Errorf("migration %s failed; all session changes restored: %w", filename, cause)
}
@@ -0,0 +1,227 @@
//go:build integration
package models
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"testing/fstest"
"time"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func TestSQLMigrationSafetyMariaDB(t *testing.T) {
out, err := exec.Command("docker", "run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306", "-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "mariadb:11.8", "--skip-ssl").CombinedOutput()
if err != nil {
t.Fatalf("start test MariaDB: %v: %s", err, out)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if out, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop test MariaDB: %v: %s", err, out)
}
})
out, err = exec.Command("docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatalf("get test MariaDB port: %v: %s", err, out)
}
addr := strings.TrimSpace(string(out))
dsn := "root@tcp(" + addr + ")/?parseTime=true&timeout=1s&tls=false"
var admin *gorm.DB
deadline := time.Now().Add(45 * time.Second)
for {
admin, err = gorm.Open(mysql.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err == nil {
break
}
if time.Now().After(deadline) {
t.Fatalf("test MariaDB unavailable: %v", err)
}
time.Sleep(150 * time.Millisecond)
}
pool, err := admin.DB()
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pool.Close() })
newDB := func(t *testing.T) *gorm.DB {
t.Helper()
schema := fmt.Sprintf("qfs_test_%d", time.Now().UnixNano())
if err := admin.Exec("CREATE DATABASE " + schema + " COLLATE utf8mb4_unicode_ci").Error; err != nil {
t.Fatal(err)
}
db, err := gorm.Open(mysql.Open(strings.Replace(dsn, ")/?", ")/"+schema+"?", 1)), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatal(err)
}
p, err := db.DB()
if err != nil {
t.Fatal(err)
}
p.SetMaxOpenConns(1)
p.SetMaxIdleConns(1)
t.Cleanup(func() {
p.Close()
if err := admin.Exec("DROP DATABASE " + schema).Error; err != nil {
t.Error(err)
}
})
for _, sql := range []string{
"SET time_zone='+00:00'",
"CREATE TABLE parent (id INT PRIMARY KEY AUTO_INCREMENT, value TEXT, empty_value TEXT, nullable TEXT, raw VARBINARY(32), amount DECIMAL(20,6), stamp TIMESTAMP(6), generated INT AS (id+1) VIRTUAL) ENGINE=InnoDB",
"INSERT INTO parent (id,value,empty_value,nullable,raw,amount,stamp) VALUES (1,'quote; and ''text''','',NULL,X'0001FF',123.456789,'2026-10-26 12:34:56.123456')",
"SET @saved_test_mode=@@sql_mode",
"SET sql_mode='NO_AUTO_VALUE_ON_ZERO'",
"INSERT INTO parent (id,value,stamp) VALUES (0,'legacy','0000-00-00 00:00:00')",
"SET sql_mode=@saved_test_mode",
"CREATE TABLE child (id INT PRIMARY KEY, parent_id INT, CONSTRAINT child_parent FOREIGN KEY (parent_id) REFERENCES parent(id)) ENGINE=InnoDB",
"INSERT INTO child VALUES (1,1)",
"CREATE TABLE audit (id INT PRIMARY KEY AUTO_INCREMENT, note TEXT) ENGINE=InnoDB",
"CREATE TRIGGER insert_value BEFORE INSERT ON parent FOR EACH ROW SET NEW.value=CONCAT(NEW.value,'!')",
"CREATE TRIGGER audit_z AFTER UPDATE ON parent FOR EACH ROW INSERT INTO audit (note) VALUES ('first')",
"CREATE TRIGGER audit_a AFTER UPDATE ON parent FOR EACH ROW INSERT INTO audit (note) VALUES ('second')",
} {
if err := db.Exec(sql).Error; err != nil {
t.Fatal(err)
}
}
return db
}
migration := func(sql, checks string) *fstest.MapFile {
return &fstest.MapFile{Data: []byte("-- Tables affected: parent, child\n" + checks + "\n" + sql)}
}
for _, failure := range []string{"none", "verify", "FK"} {
t.Run(failure, func(t *testing.T) {
db, root := newDB(t), t.TempDir()
var original string
err := db.Connection(func(conn *gorm.DB) error {
var e error
original, e = mariadb.CreateSnapshot(conn, root, "original")
return e
})
if err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("SET @example='value; quoted'; PREPARE q FROM 'SELECT 1'; EXECUTE q; DEALLOCATE PREPARE q; ALTER TABLE parent ADD COLUMN added INT; UPDATE parent SET value=@example;", "-- verify: added missing | SELECT 1 FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='parent' AND COLUMN_NAME='added' HAVING COUNT(*)=0\n-- verify: wrong value | SELECT 1 FROM parent WHERE value<>'value; quoted'")}
if failure == "verify" {
files["002_b.sql"] = migration("CREATE TABLE new_table (id INT); UPDATE parent SET value='changed';", "-- verify: deliberate failure | SELECT 1")
}
if failure == "FK" {
files["002_b.sql"] = migration("ALTER TABLE child ADD COLUMN added INT; ALTER TABLE child ADD CONSTRAINT broken_fk FOREIGN KEY (added) REFERENCES parent(missing);", "-- verify: should never run | SELECT 1 WHERE 0")
}
applied, err := RunSQLMigrations(db, files, nil, root)
if failure == "none" {
if err != nil || len(applied) != 1 {
t.Fatalf("apply=%v error=%v", applied, err)
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Count(&count).Error; err != nil || count != 1 {
t.Fatalf("history=%d err=%v", count, err)
}
} else {
if err == nil || !strings.Contains(err.Error(), "all session changes restored") || len(applied) != 0 {
t.Fatalf("apply=%v error=%v", applied, err)
}
if err := mariadb.CheckSnapshot(db, original); err != nil {
t.Fatalf("recovery changed original state: %v", err)
}
}
markers, err := filepath.Glob(filepath.Join(root, "*.pending"))
if err != nil || len(markers) != 0 {
t.Fatalf("markers=%v error=%v", markers, err)
}
})
}
t.Run("interrupted session", func(t *testing.T) {
db, root := newDB(t), t.TempDir()
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
func() {
defer func() {
if recover() == nil {
t.Error("expected interruption")
}
}()
RunSQLMigrations(db, files, func(string) { panic("interruption") }, root)
}()
if err := db.Exec("ALTER TABLE parent ADD COLUMN added INT").Error; err != nil {
t.Fatal(err)
}
if _, err := RunSQLMigrations(db, files, nil, root); err == nil || !strings.Contains(err.Error(), "interrupted migration") {
t.Fatalf("retry not refused: %v", err)
}
markers, err := filepath.Glob(filepath.Join(root, "*.pending"))
if err != nil || len(markers) != 1 {
t.Fatalf("markers=%v err=%v", markers, err)
}
data, err := os.ReadFile(markers[0])
if err != nil {
t.Fatal(err)
}
if err := RestoreSQLMigrationSession(db, strings.TrimSpace(string(data))); err != nil {
t.Fatal(err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("interrupted DDL not restored")
}
if _, err := RunSQLMigrations(db, files, nil, root); err != nil {
t.Fatalf("retry after restore: %v", err)
}
})
t.Run("unsupported object", func(t *testing.T) {
db := newDB(t)
if err := db.Exec("CREATE VIEW example_view AS SELECT id FROM parent").Error; err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, t.TempDir()); err == nil || !strings.Contains(err.Error(), "snapshot refused") {
t.Fatalf("unsupported schema accepted: %v", err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("unsupported schema was changed")
}
})
t.Run("other client", func(t *testing.T) {
db := newDB(t)
p, err := db.DB()
if err != nil {
t.Fatal(err)
}
p.SetMaxOpenConns(2)
other, err := p.Conn(context.Background())
if err != nil {
t.Fatal(err)
}
defer other.Close()
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, t.TempDir()); err == nil || !strings.Contains(err.Error(), "other connections") {
t.Fatalf("active client accepted: %v", err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("active schema changed")
}
})
t.Run("backup failure", func(t *testing.T) {
db := newDB(t)
path := filepath.Join(t.TempDir(), "not-directory")
if err := os.WriteFile(path, []byte("example"), 0600); err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, path); err == nil {
t.Fatal("backup failure ignored")
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("schema changed without backup")
}
})
}
+76
View File
@@ -0,0 +1,76 @@
package models
import (
"fmt"
"strings"
"gorm.io/gorm"
)
type migrationVerification struct {
description string
query string
}
func parseMigrationVerifications(script string) ([]migrationVerification, error) {
var checks []migrationVerification
for _, line := range strings.Split(script, "\n") {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "-- verify:") {
continue
}
description, query, ok := strings.Cut(strings.TrimPrefix(line, "-- verify:"), "|")
description, query = strings.TrimSpace(description), strings.TrimSpace(query)
if !ok || description == "" || query == "" {
return nil, fmt.Errorf("invalid migration verify header")
}
checks = append(checks, migrationVerification{description: description, query: query})
}
return checks, nil
}
func applySQLMigrationFile(db *gorm.DB, filename, content string) error {
checks, err := parseMigrationVerifications(content)
if err != nil {
return fmt.Errorf("parse migration %s: %w", filename, err)
}
statements, err := splitSQLStatements(content)
if err != nil {
return fmt.Errorf("parse migration %s SQL: %w", filename, err)
}
return db.Connection(func(conn *gorm.DB) error {
return applySQLMigrationOnSession(conn, filename, statements, checks)
})
}
func applySQLMigrationOnSession(conn *gorm.DB, filename string, statements []string, checks []migrationVerification) error {
if err := executeMigrationStatements(conn, filename, statements); err != nil {
return err
}
// Each query gets a fresh statement on the pinned connection: chaining
// Raw/Create on conn itself leaks the previous query's state (with two
// verify checks the history insert targeted a table named "rows").
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
for _, check := range checks {
rows, err := fresh().Raw(check.query).Rows()
if err != nil {
return fmt.Errorf("verify migration %s (%s): %w", filename, check.description, err)
}
failed := rows.Next()
rowErr := rows.Err()
closeErr := rows.Close()
if rowErr != nil {
return fmt.Errorf("read migration verification %s (%s): %w", filename, check.description, rowErr)
}
if closeErr != nil {
return fmt.Errorf("close migration verification %s (%s): %w", filename, check.description, closeErr)
}
if failed {
return fmt.Errorf("verify migration %s failed: %s", filename, check.description)
}
}
if err := fresh().Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record migration %s: %w", filename, err)
}
return nil
}
@@ -0,0 +1,70 @@
package models
import (
"path/filepath"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestMigrationVerification(t *testing.T) {
for _, tt := range []struct{ name, content, wantError string }{
{"valid", "-- verify: bad value | SELECT 1 FROM example WHERE value <> 1\nCREATE TEMP TABLE example (value INT); INSERT INTO example VALUES (1);", ""},
{"failed", "-- verify: bad value | SELECT 1 FROM example WHERE value <> 1\nCREATE TEMP TABLE example (value INT); INSERT INTO example VALUES (2);", "bad value"},
{"query error", "-- verify: missing table | SELECT 1 FROM missing\nCREATE TEMP TABLE example (value INT);", "missing table"},
{"malformed", "-- verify: no query\nCREATE TABLE example (value INT);", "invalid migration verify header"},
{"two checks pass", "-- verify: first | SELECT 1 WHERE 0\n-- verify: second | SELECT 1 WHERE 0\nCREATE TEMP TABLE example (value INT);", ""},
{"second check fails", "-- verify: first | SELECT 1 WHERE 0\n-- verify: second | SELECT 1\nCREATE TEMP TABLE example (value INT);", "second"},
{"execution failure", "-- verify: none | SELECT 1 WHERE 0\nINVALID SQL;", "exec migration"},
} {
t.Run(tt.name, func(t *testing.T) {
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "migration.db")), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := db.DB()
defer sqlDB.Close()
if err := db.AutoMigrate(&SQLSchemaMigration{}); err != nil {
t.Fatal(err)
}
err = applySQLMigrationFile(db, "001_example.sql", tt.content)
if tt.wantError != "" && (err == nil || !strings.Contains(err.Error(), tt.wantError)) {
t.Fatalf("want %q, got %v", tt.wantError, err)
}
if tt.wantError == "" && err != nil {
t.Fatal(err)
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Count(&count).Error; err != nil {
t.Fatal(err)
}
if (count == 1) != (tt.wantError == "") {
t.Fatalf("incorrect history count=%d", count)
}
if tt.name == "malformed" && db.Migrator().HasTable("example") {
t.Fatal("malformed header mutated database")
}
})
}
}
func TestMigrationErrorAllowlist(t *testing.T) {
for _, tt := range []struct {
message string
want bool
}{
{"Duplicate column name 'example'", true},
{"Duplicate key name 'example'", true},
{"Table 'example' already exists", true},
{"Can't create table example (errno: 150 Foreign key constraint is incorrectly formed)", false},
{"Can't create table example (errno: 121 Duplicate key)", false},
{"Duplicate foreign key constraint name 'example'", false},
{"Access denied", false},
} {
if got := isIgnorableMigrationError(tt.message); got != tt.want {
t.Errorf("%s: got %v", tt.message, got)
}
}
}
+85 -88
View File
@@ -1,11 +1,9 @@
package models
import (
"bufio"
"errors"
"fmt"
"os"
"path/filepath"
"io/fs"
"sort"
"strings"
"time"
@@ -24,10 +22,87 @@ func (SQLSchemaMigration) TableName() string {
return "qt_schema_migrations"
}
// NeedsSQLMigrations reports whether at least one SQL migration from migrationsDir
// SQLMigrationState is one server migration file and when it was recorded as applied
// (nil = pending).
type SQLMigrationState struct {
Filename string
AppliedAt *time.Time
}
// SQLMigrationStatus lists every server migration in migrations with its applied state.
// Read-only: does not create qt_schema_migrations.
func SQLMigrationStatus(db *gorm.DB, migrations fs.FS) ([]SQLMigrationState, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return nil, err
}
applied := map[string]time.Time{}
if db.Migrator().HasTable(&SQLSchemaMigration{}) {
var rows []SQLSchemaMigration
if err := db.Find(&rows).Error; err != nil {
return nil, fmt.Errorf("read qt_schema_migrations: %w", err)
}
for _, r := range rows {
applied[r.Filename] = r.AppliedAt
}
}
states := make([]SQLMigrationState, 0, len(files))
for _, f := range files {
st := SQLMigrationState{Filename: f}
if at, ok := applied[f]; ok {
at := at
st.AppliedAt = &at
}
states = append(states, st)
}
return states, nil
}
// BaselineSQLMigrations records every migration up to and including upto as applied
// WITHOUT executing it — for a server whose schema was brought up to date by other
// means before qt_schema_migrations was used. Returns the filenames newly recorded.
func BaselineSQLMigrations(db *gorm.DB, migrations fs.FS, upto string) ([]string, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return nil, err
}
found := false
for _, f := range files {
if f == upto {
found = true
break
}
}
if !found {
return nil, fmt.Errorf("baseline migration %q not found", upto)
}
if err := ensureSQLMigrationsTable(db); err != nil {
return nil, fmt.Errorf("migrate qt_schema_migrations table: %w", err)
}
var recorded []string
for _, f := range files {
if f > upto {
break
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Where("filename = ?", f).Count(&count).Error; err != nil {
return recorded, fmt.Errorf("check migration %s: %w", f, err)
}
if count > 0 {
continue
}
if err := db.Create(&SQLSchemaMigration{Filename: f}).Error; err != nil {
return recorded, fmt.Errorf("record baseline migration %s: %w", f, err)
}
recorded = append(recorded, f)
}
return recorded, nil
}
// NeedsSQLMigrations reports whether at least one SQL migration from migrations
// is not yet recorded in qt_schema_migrations.
func NeedsSQLMigrations(db *gorm.DB, migrationsDir string) (bool, error) {
files, err := listSQLMigrationFiles(migrationsDir)
func NeedsSQLMigrations(db *gorm.DB, migrations fs.FS) (bool, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return false, err
}
@@ -48,52 +123,6 @@ func NeedsSQLMigrations(db *gorm.DB, migrationsDir string) (bool, error) {
return count < int64(len(files)), nil
}
// RunSQLMigrations applies SQL files from migrationsDir once and records them in qt_schema_migrations.
// Local SQLite-only scripts are skipped automatically.
func RunSQLMigrations(db *gorm.DB, migrationsDir string) error {
if err := ensureSQLMigrationsTable(db); err != nil {
return fmt.Errorf("migrate qt_schema_migrations table: %w", err)
}
files, err := listSQLMigrationFiles(migrationsDir)
if err != nil {
return err
}
for _, filename := range files {
var count int64
if err := db.Model(&SQLSchemaMigration{}).Where("filename = ?", filename).Count(&count).Error; err != nil {
return fmt.Errorf("check migration %s: %w", filename, err)
}
if count > 0 {
continue
}
path := filepath.Join(migrationsDir, filename)
content, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("read migration %s: %w", filename, err)
}
statements := splitSQLStatements(string(content))
if len(statements) == 0 {
if err := db.Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record empty migration %s: %w", filename, err)
}
continue
}
if err := executeMigrationStatements(db, filename, statements); err != nil {
return err
}
if err := db.Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record migration %s: %w", filename, err)
}
}
return nil
}
// IsMigrationPermissionError returns true if err indicates insufficient privileges
// to create/alter/read migration metadata or target schema objects.
func IsMigrationPermissionError(err error) bool {
@@ -158,9 +187,6 @@ func isIgnorableMigrationError(message string) bool {
"duplicate column name",
"duplicate key name",
"already exists",
"can't create table",
"duplicate foreign key constraint name",
"errno 121",
}
for _, pattern := range ignorable {
if strings.Contains(lower, pattern) {
@@ -170,42 +196,13 @@ func isIgnorableMigrationError(message string) bool {
return false
}
func splitSQLStatements(script string) []string {
scanner := bufio.NewScanner(strings.NewReader(script))
scanner.Buffer(make([]byte, 1024), 1024*1024)
lines := make([]string, 0, 128)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
if strings.HasPrefix(line, "--") {
continue
}
lines = append(lines, scanner.Text())
}
combined := strings.Join(lines, "\n")
raw := strings.Split(combined, ";")
stmts := make([]string, 0, len(raw))
for _, stmt := range raw {
trimmed := strings.TrimSpace(stmt)
if trimmed == "" {
continue
}
stmts = append(stmts, trimmed)
}
return stmts
}
func listSQLMigrationFiles(migrationsDir string) ([]string, error) {
entries, err := os.ReadDir(migrationsDir)
func listSQLMigrationFiles(migrations fs.FS) ([]string, error) {
entries, err := fs.ReadDir(migrations, ".")
if err != nil {
if errors.Is(err, os.ErrNotExist) {
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return nil, fmt.Errorf("read migrations dir %s: %w", migrationsDir, err)
return nil, fmt.Errorf("read migrations dir: %w", err)
}
files := make([]string, 0, len(entries))
+56
View File
@@ -0,0 +1,56 @@
package models
import (
"path/filepath"
"testing"
"testing/fstest"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestSQLMigrationStatusAndListing(t *testing.T) {
migrations := fstest.MapFS{
"002_b.sql": {Data: []byte("ALTER TABLE x ADD COLUMN b INT;")},
"001_a.sql": {Data: []byte("ALTER TABLE x ADD COLUMN a INT;")},
"006_local_only_versions.sql": {Data: []byte("-- sqlite only")},
"README.md": {Data: []byte("not a migration")},
}
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "m.db")), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
// No history table yet: everything pending, nothing created.
states, err := SQLMigrationStatus(db, migrations)
if err != nil {
t.Fatal(err)
}
if len(states) != 2 || states[0].Filename != "001_a.sql" || states[1].Filename != "002_b.sql" {
t.Fatalf("unexpected listing %+v", states)
}
if states[0].AppliedAt != nil || db.Migrator().HasTable(&SQLSchemaMigration{}) {
t.Fatal("status must be read-only and report pending")
}
if err := db.AutoMigrate(&SQLSchemaMigration{}); err != nil {
t.Fatal(err)
}
if err := db.Create(&SQLSchemaMigration{Filename: "001_a.sql"}).Error; err != nil {
t.Fatal(err)
}
states, _ = SQLMigrationStatus(db, migrations)
if states[0].AppliedAt == nil || states[1].AppliedAt != nil {
t.Fatalf("want 001 applied, 002 pending: %+v", states)
}
}
func TestSplitSQLStatementsSkipsComments(t *testing.T) {
got, err := splitSQLStatements("-- header; with semicolon\nALTER TABLE a ADD COLUMN x INT;\n\nALTER TABLE b\n ADD COLUMN y INT;\n")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[1] != "ALTER TABLE b\n ADD COLUMN y INT" {
t.Fatalf("unexpected statements %q", got)
}
}
+70
View File
@@ -0,0 +1,70 @@
package models
import (
"fmt"
"strings"
)
// splitSQLStatements supports MariaDB literals, quoted identifiers and comments.
func splitSQLStatements(script string) ([]string, error) {
var statements []string
var current strings.Builder
var quote byte
for i := 0; i < len(script); i++ {
ch := script[i]
if quote != 0 {
current.WriteByte(ch)
if ch == '\\' && quote != '`' && i+1 < len(script) {
i++
current.WriteByte(script[i])
} else if ch == quote {
if i+1 < len(script) && script[i+1] == quote {
i++
current.WriteByte(script[i])
} else {
quote = 0
}
}
continue
}
switch {
case ch == '\'' || ch == '"' || ch == '`':
quote = ch
current.WriteByte(ch)
case ch == '#' || (ch == '-' && i+2 < len(script) && script[i+1] == '-' && script[i+2] <= ' '):
for i < len(script) && script[i] != '\n' {
i++
}
current.WriteByte('\n')
case ch == '/' && i+1 < len(script) && script[i+1] == '*':
if strings.HasPrefix(script[i:], "/*!") || strings.HasPrefix(script[i:], "/*M!") {
return nil, fmt.Errorf("executable SQL comments are unsupported")
}
end := strings.Index(script[i+2:], "*/")
if end < 0 {
return nil, fmt.Errorf("unterminated SQL comment")
}
i += end + 3
current.WriteByte(' ')
case ch == ';':
if stmt := strings.TrimSpace(current.String()); stmt != "" {
statements = append(statements, stmt)
}
current.Reset()
default:
current.WriteByte(ch)
}
}
if quote != 0 {
return nil, fmt.Errorf("unterminated SQL quote")
}
if stmt := strings.TrimSpace(current.String()); stmt != "" {
statements = append(statements, stmt)
}
for _, stmt := range statements {
if strings.HasPrefix(strings.ToUpper(stmt), "DELIMITER ") {
return nil, fmt.Errorf("DELIMITER directives are unsupported")
}
}
return statements, nil
}
@@ -0,0 +1,70 @@
package models
import (
qfassets "git.mchus.pro/mchus/quoteforge"
"io/fs"
"reflect"
"strings"
"testing"
)
func TestSQLStatementParser(t *testing.T) {
for _, tt := range []struct {
name, sql string
want []string
fail bool
}{
{"empty", "-- comment;\n# more;", nil, false},
{"literal separators", "SELECT 'a;b', \"c;d\", `e;f`; SELECT 2;", []string{"SELECT 'a;b', \"c;d\", `e;f`", "SELECT 2"}, false},
{"doubled quotes", "SELECT 'a'';b', `a``;b`;", []string{"SELECT 'a'';b', `a``;b`"}, false},
{"backslash quote", "SELECT 'a\\';b'; SELECT 2", []string{"SELECT 'a\\';b'", "SELECT 2"}, false},
{"inline comments", "SELECT/* ; */ 1; -- end;\nSELECT 2# end;\n;", []string{"SELECT 1", "SELECT 2"}, false},
{"not a comment", "SELECT 4--2;", []string{"SELECT 4--2"}, false},
{"unclosed quote", "SELECT 'a;", nil, true},
{"unclosed block", "SELECT 1; /*", nil, true},
{"executable comment", "/*! SELECT 1 */", nil, true},
{"MariaDB comment", "/*M! SELECT 1 */", nil, true},
{"delimiter", "DELIMITER $$\nSELECT 1", nil, true},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := splitSQLStatements(tt.sql)
if (err != nil) != tt.fail {
t.Fatalf("error=%v", err)
}
if !tt.fail && !reflect.DeepEqual(got, tt.want) {
t.Fatalf("got=%q want=%q", got, tt.want)
}
})
}
long := "SELECT '" + strings.Repeat("x", 2<<20) + ";y';"
if got, err := splitSQLStatements(long); err != nil || len(got) != 1 {
t.Fatalf("large statement: count=%d err=%v", len(got), err)
}
}
func TestEmbeddedServerMigrationSQL(t *testing.T) {
migrations, err := qfassets.MigrationsFS()
if err != nil {
t.Fatal(err)
}
files, err := listSQLMigrationFiles(migrations)
if err != nil {
t.Fatal(err)
}
for _, name := range files {
t.Run(name, func(t *testing.T) {
content, err := fs.ReadFile(migrations, name)
if err != nil {
t.Fatal(err)
}
statements, err := splitSQLStatements(string(content))
if err != nil || len(statements) == 0 {
t.Fatalf("count=%d err=%v", len(statements), err)
}
checks, err := parseMigrationVerifications(string(content))
if err != nil || len(checks) == 0 {
t.Fatalf("checks=%d err=%v", len(checks), err)
}
})
}
}
+25
View File
@@ -141,6 +141,31 @@ func (r *PartnumberBookRepository) HasAllBookItems(bookID uint) bool {
return count == int64(len(book.PartnumbersJSON))
}
// MissingBookItems returns the book's partnumbers that have no local catalog row.
func (r *PartnumberBookRepository) MissingBookItems(bookID uint) []string {
book, err := r.getBook(bookID)
if err != nil || len(book.PartnumbersJSON) == 0 {
return nil
}
var present []string
if err := r.db.Model(&localdb.LocalPartnumberBookItem{}).
Where("partnumber IN ?", []string(book.PartnumbersJSON)).
Pluck("partnumber", &present).Error; err != nil {
return nil
}
have := make(map[string]struct{}, len(present))
for _, pn := range present {
have[pn] = struct{}{}
}
var missing []string
for _, pn := range book.PartnumbersJSON {
if _, ok := have[pn]; !ok {
missing = append(missing, pn)
}
}
return missing
}
func (r *PartnumberBookRepository) getBook(bookID uint) (*localdb.LocalPartnumberBook, error) {
var book localdb.LocalPartnumberBook
if err := r.db.First(&book, bookID).Error; err != nil {
+29
View File
@@ -156,6 +156,35 @@ func (r *PricelistRepository) GetLatestActiveBySource(source string) (*models.Pr
return &pricelist, nil
}
// GetLatestActiveIDsBySources returns, per source, the ID GetLatestActiveBySource
// would return, in one round trip. Sources with no such pricelist are absent.
func (r *PricelistRepository) GetLatestActiveIDsBySources(sources []string) (map[string]uint, error) {
result := make(map[string]uint, len(sources))
if len(sources) == 0 {
return result, nil
}
parts := make([]string, 0, len(sources))
args := make([]any, 0, len(sources))
for _, source := range sources {
parts = append(parts, `(SELECT id, source FROM qt_pricelists
WHERE is_active = 1 AND source = ?
AND EXISTS (SELECT 1 FROM qt_pricelist_items WHERE qt_pricelist_items.pricelist_id = qt_pricelists.id)
ORDER BY created_at DESC, id DESC LIMIT 1)`)
args = append(args, source)
}
var rows []struct {
ID uint
Source string
}
if err := r.db.Raw(strings.Join(parts, " UNION ALL "), args...).Scan(&rows).Error; err != nil {
return nil, fmt.Errorf("getting latest pricelists: %w", err)
}
for _, row := range rows {
result[row.Source] = row.ID
}
return result, nil
}
// Create creates a new pricelist
func (r *PricelistRepository) Create(pricelist *models.Pricelist) error {
if err := r.db.Create(pricelist).Error; err != nil {
+3
View File
@@ -334,6 +334,7 @@ func (s *LocalConfigurationService) CloneToProject(configUUID string, ownerUsern
PricelistID: original.PricelistID,
WarehousePricelistID: original.WarehousePricelistID,
CompetitorPricelistID: original.CompetitorPricelistID,
ConfigType: original.ConfigType,
DisablePriceRefresh: original.DisablePriceRefresh,
CreatedAt: time.Now(),
}
@@ -660,6 +661,7 @@ func (s *LocalConfigurationService) CloneNoAuthToProjectFromVersion(configUUID s
return nil, fmt.Errorf("decode version snapshot for clone: %w", decErr)
}
snapshotCfg := localdb.LocalToConfiguration(snapshot)
snapshotCfg.ConfigType = original.ConfigType // not part of version snapshots
original = snapshotCfg
original.UUID = configUUID // preserve original UUID for project resolution
}
@@ -695,6 +697,7 @@ func (s *LocalConfigurationService) CloneNoAuthToProjectFromVersion(configUUID s
PricelistID: original.PricelistID,
WarehousePricelistID: original.WarehousePricelistID,
CompetitorPricelistID: original.CompetitorPricelistID,
ConfigType: original.ConfigType,
DisablePriceRefresh: original.DisablePriceRefresh,
CreatedAt: time.Now(),
}
@@ -0,0 +1,51 @@
package services
import (
"testing"
"git.mchus.pro/mchus/quoteforge/internal/models"
)
// config_type is free-form (codes come from qt_settings.config_types), so an
// unknown code like "networking" must survive every write path unchanged.
func TestConfigTypeSurvivesUpdateAndClone(t *testing.T) {
service, local := newLocalConfigServiceForTest(t)
created, err := service.Create("tester", &CreateConfigRequest{
Name: "net",
Items: models.ConfigItems{{LotName: "CPU_A", Quantity: 1, UnitPrice: 1000}},
ConfigType: "networking",
})
if err != nil {
t.Fatalf("create: %v", err)
}
// The configurator's save payload carries no config_type.
if _, err := service.UpdateNoAuth(created.UUID, &CreateConfigRequest{
Name: "net",
Items: models.ConfigItems{{LotName: "CPU_A", Quantity: 2, UnitPrice: 1000}},
}); err != nil {
t.Fatalf("update: %v", err)
}
if stored, _ := local.GetConfigurationByUUID(created.UUID); stored.ConfigType != "networking" {
t.Fatalf("config_type after update = %q, want networking", stored.ConfigType)
}
for _, fromVersion := range []int{0, 1} {
clone, err := service.CloneNoAuthToProjectFromVersion(created.UUID, "copy", "tester", nil, fromVersion)
if err != nil {
t.Fatalf("clone from version %d: %v", fromVersion, err)
}
if stored, _ := local.GetConfigurationByUUID(clone.UUID); stored.ConfigType != "networking" {
t.Fatalf("clone from version %d: config_type = %q, want networking", fromVersion, stored.ConfigType)
}
}
clone, err := service.CloneToProject(created.UUID, "tester", "copy2", nil)
if err != nil {
t.Fatalf("clone to project: %v", err)
}
if stored, _ := local.GetConfigurationByUUID(clone.UUID); stored.ConfigType != "networking" {
t.Fatalf("CloneToProject: config_type = %q, want networking", stored.ConfigType)
}
}
@@ -0,0 +1,203 @@
package sync_test
import (
"os"
"strings"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/models"
syncsvc "git.mchus.pro/mchus/quoteforge/internal/services/sync"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// Runs only against a disposable MariaDB that already has the qt_projects and
// qt_configurations tables (e.g. a container loaded with SHOW CREATE TABLE from
// production). The test truncates them. Example:
//
// QF_TEST_MARIADB_DSN='root:test@tcp(127.0.0.1:33306)/qf?parseTime=true' go test ./internal/services/sync -run MariaDB
func newMariaDBForPullTest(t *testing.T) *gorm.DB {
t.Helper()
dsn := os.Getenv("QF_TEST_MARIADB_DSN")
if dsn == "" {
t.Skip("QF_TEST_MARIADB_DSN not set")
}
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatalf("open mariadb: %v", err)
}
mustExec(t, db, "SET FOREIGN_KEY_CHECKS=0")
mustExec(t, db, "TRUNCATE TABLE qt_configurations")
mustExec(t, db, "TRUNCATE TABLE qt_projects")
mustExec(t, db, "SET FOREIGN_KEY_CHECKS=1")
for _, table := range []string{"qt_configurations", "qt_projects"} {
mustExec(t, db, "ALTER TABLE "+table+" DROP INDEX IF EXISTS idx_"+table+"_server_updated_at")
mustExec(t, db, "ALTER TABLE "+table+" DROP COLUMN IF EXISTS server_updated_at")
}
return db
}
func mustExec(t *testing.T, db *gorm.DB, sql string, args ...any) {
t.Helper()
if err := db.Exec(sql, args...).Error; err != nil {
t.Fatalf("%s: %v", sql, err)
}
}
func applyServerUpdatedAtMigration(t *testing.T, db *gorm.DB) {
t.Helper()
raw, err := os.ReadFile("../../../migrations/035_add_server_updated_at.sql")
if err != nil {
t.Fatalf("read migration: %v", err)
}
var body []string
for _, line := range strings.Split(string(raw), "\n") {
if !strings.HasPrefix(strings.TrimSpace(line), "--") {
body = append(body, line)
}
}
for _, stmt := range strings.Split(strings.Join(body, "\n"), ";") {
if strings.TrimSpace(stmt) != "" {
mustExec(t, db, stmt)
}
}
}
// seedServerConfig writes the way any client version does: through the
// models.Configuration struct, which knows nothing about server_updated_at.
func seedServerConfig(t *testing.T, db *gorm.DB, uuid, name string) *models.Configuration {
t.Helper()
cfg := &models.Configuration{
UUID: uuid,
OwnerUsername: "tester",
Name: name,
Items: models.ConfigItems{{LotName: "CPU_PULL", Quantity: 1, UnitPrice: 100}},
ServerCount: 1,
Line: 10,
}
if err := db.Create(cfg).Error; err != nil {
t.Fatalf("seed config %s: %v", uuid, err)
}
return cfg
}
func TestIncrementalPullMariaDB(t *testing.T) {
serverDB := newMariaDBForPullTest(t)
local := newLocalDBForSyncTest(t)
if err := local.SaveSettings("127.0.0.1", 33306, "qf", "root", "test"); err != nil {
t.Fatalf("save settings: %v", err)
}
// Before migration 035: every pull is full, exactly as before.
seedServerConfig(t, serverDB, "cfg-a", "A")
seedServerConfig(t, serverDB, "cfg-b", "B")
svc := syncsvc.NewServiceWithDB(serverDB, local)
for i := 0; i < 2; i++ {
res, err := svc.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("pre-migration import: %v", err)
}
if res.Imported+res.Updated != 2 {
t.Fatalf("pre-migration pull %d: expected full pull of 2, got %+v", i, res)
}
}
applyServerUpdatedAtMigration(t, serverDB)
applyServerUpdatedAtMigration(t, serverDB) // re-runnable
// A client without the column in its model still inserts and updates fine.
seedServerConfig(t, serverDB, "cfg-c", "C")
// Age all rows past the overlap window.
mustExec(t, serverDB, "UPDATE qt_configurations SET server_updated_at = NOW(6) - INTERVAL 1 HOUR")
// First pull after the migration in this process: full, sets the cursor.
res, err := svc.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("first post-migration import: %v", err)
}
if res.Imported+res.Updated != 3 {
t.Fatalf("expected full pull of 3, got %+v", res)
}
// Nothing changed on the server: nothing is pulled.
res, err = svc.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("idle incremental import: %v", err)
}
if res.Imported+res.Updated != 0 {
t.Fatalf("expected empty incremental pull, got %+v", res)
}
// An old-style client update (gorm Save of the whole struct) bumps the marker.
var b models.Configuration
if err := serverDB.Where("uuid = ?", "cfg-b").First(&b).Error; err != nil {
t.Fatalf("load cfg-b: %v", err)
}
b.Name = "B renamed"
if err := serverDB.Save(&b).Error; err != nil {
t.Fatalf("save cfg-b: %v", err)
}
seedServerConfig(t, serverDB, "cfg-d", "D")
res, err = svc.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("incremental import: %v", err)
}
if res.Updated != 1 || res.Imported != 1 {
t.Fatalf("expected 1 updated + 1 imported, got %+v", res)
}
localB, err := local.GetConfigurationByUUID("cfg-b")
if err != nil || localB.Name != "B renamed" {
t.Fatalf("expected cfg-b renamed locally, got %+v err=%v", localB, err)
}
// A new process does one full pull again.
restarted := syncsvc.NewServiceWithDB(serverDB, local)
res, err = restarted.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("restart import: %v", err)
}
if res.Imported+res.Updated != 4 {
t.Fatalf("expected full pull of 4 after restart, got %+v", res)
}
// Switching to another server invalidates the cursor.
if err := local.SaveSettings("other-host", 3306, "qf", "root", "test"); err != nil {
t.Fatalf("save settings: %v", err)
}
res, err = restarted.ImportConfigurationsToLocal()
if err != nil {
t.Fatalf("import after server switch: %v", err)
}
if res.Imported+res.Updated != 4 {
t.Fatalf("expected full pull of 4 after server switch, got %+v", res)
}
}
func TestIncrementalProjectPullMariaDB(t *testing.T) {
serverDB := newMariaDBForPullTest(t)
local := newLocalDBForSyncTest(t)
if err := local.SaveSettings("127.0.0.1", 33306, "qf", "root", "test"); err != nil {
t.Fatalf("save settings: %v", err)
}
applyServerUpdatedAtMigration(t, serverDB)
code := "P-1"
project := &models.Project{UUID: "prj-1", OwnerUsername: "tester", Code: code, IsActive: true}
if err := serverDB.Create(project).Error; err != nil {
t.Fatalf("seed project: %v", err)
}
mustExec(t, serverDB, "UPDATE qt_projects SET server_updated_at = NOW(6) - INTERVAL 1 HOUR")
svc := syncsvc.NewServiceWithDB(serverDB, local)
if res, err := svc.ImportProjectsToLocal(); err != nil || res.Imported != 1 {
t.Fatalf("first project pull: %+v err=%v", res, err)
}
if res, err := svc.ImportProjectsToLocal(); err != nil || res.Imported+res.Updated != 0 {
t.Fatalf("idle project pull: %+v err=%v", res, err)
}
mustExec(t, serverDB, "UPDATE qt_projects SET name = 'renamed' WHERE uuid = 'prj-1'")
if res, err := svc.ImportProjectsToLocal(); err != nil || res.Updated != 1 {
t.Fatalf("incremental project pull: %+v err=%v", res, err)
}
}
+65 -21
View File
@@ -23,15 +23,16 @@ func (s *Service) PullPartnumberBooks() (int, error) {
localBookRepo := repository.NewPartnumberBookRepository(s.localDB.DB())
// Headers only: partnumbers_json is several MB in total and the worker runs
// every minute, so it is fetched per book only when that book needs items.
type serverBook struct {
ID int `gorm:"column:id"`
Version string `gorm:"column:version"`
CreatedAt time.Time `gorm:"column:created_at"`
IsActive bool `gorm:"column:is_active"`
PartnumbersJSON string `gorm:"column:partnumbers_json"`
ID int `gorm:"column:id"`
Version string `gorm:"column:version"`
CreatedAt time.Time `gorm:"column:created_at"`
IsActive bool `gorm:"column:is_active"`
}
var serverBooks []serverBook
if err := mariaDB.Raw("SELECT id, version, created_at, is_active, partnumbers_json FROM qt_partnumber_books ORDER BY created_at DESC, id DESC").Scan(&serverBooks).Error; err != nil {
if err := mariaDB.Raw("SELECT id, version, created_at, is_active FROM qt_partnumber_books ORDER BY created_at DESC, id DESC").Scan(&serverBooks).Error; err != nil {
return 0, fmt.Errorf("querying server partnumber books: %w", err)
}
slog.Info("partnumber books found on server", "count", len(serverBooks))
@@ -40,39 +41,58 @@ func (s *Service) PullPartnumberBooks() (int, error) {
for _, sb := range serverBooks {
var existing localdb.LocalPartnumberBook
err := s.localDB.DB().Where("server_id = ?", sb.ID).First(&existing).Error
partnumbers, errPartnumbers := decodeServerPartnumbers(sb.PartnumbersJSON)
if errPartnumbers != nil {
slog.Error("failed to decode server partnumbers_json", "server_id", sb.ID, "error", errPartnumbers)
continue
}
if err == nil {
existing.Version = sb.Version
existing.CreatedAt = sb.CreatedAt
existing.IsActive = sb.IsActive
if existing.Version != sb.Version || existing.IsActive != sb.IsActive {
existing.Version = sb.Version
existing.CreatedAt = sb.CreatedAt
existing.IsActive = sb.IsActive
if err := localBookRepo.SaveBook(&existing); err != nil {
slog.Error("failed to update local partnumber book header", "server_id", sb.ID, "error", err)
continue
}
}
if localBookRepo.HasAllBookItems(existing.ID) {
slog.Debug("partnumber book already synced, skipping", "server_id", sb.ID, "version", sb.Version)
continue
}
// A server book may list partnumbers that qt_partnumber_book_items
// lacks; re-pulling cannot fix that, so try once per process.
if !s.markPartnumberBookRepair(sb.ID, sb.Version) {
slog.Debug("partnumber book still incomplete after re-pull this session, skipping", "server_id", sb.ID, "version", sb.Version)
continue
}
slog.Info("partnumber book header exists but catalog items are missing, re-pulling items", "server_id", sb.ID, "version", sb.Version)
partnumbers, err := fetchServerBookPartnumbers(mariaDB, sb.ID)
if err != nil {
slog.Error("failed to fetch server partnumbers_json", "server_id", sb.ID, "error", err)
continue
}
existing.PartnumbersJSON = partnumbers
if err := localBookRepo.SaveBook(&existing); err != nil {
slog.Error("failed to update local partnumber book header", "server_id", sb.ID, "error", err)
continue
}
localItemCount := localBookRepo.CountBookItems(existing.ID)
if localItemCount > 0 && localBookRepo.HasAllBookItems(existing.ID) {
slog.Debug("partnumber book already synced, skipping", "server_id", sb.ID, "version", sb.Version, "items", localItemCount)
continue
}
slog.Info("partnumber book header exists but catalog items are missing, re-pulling items", "server_id", sb.ID, "version", sb.Version)
n, err := pullBookItems(mariaDB, localBookRepo, existing.PartnumbersJSON)
if err != nil {
slog.Error("failed to re-pull items for existing book", "server_id", sb.ID, "error", err)
} else {
slog.Info("re-pulled items for existing book", "server_id", sb.ID, "version", sb.Version, "items_saved", n)
pulled++
if missing := localBookRepo.MissingBookItems(existing.ID); len(missing) > 0 {
slog.Warn("partnumber book lists partnumbers absent from server qt_partnumber_book_items", "server_id", sb.ID, "version", sb.Version, "missing", missing)
}
}
continue
}
slog.Info("pulling new partnumber book", "server_id", sb.ID, "version", sb.Version, "is_active", sb.IsActive)
partnumbers, err := fetchServerBookPartnumbers(mariaDB, sb.ID)
if err != nil {
slog.Error("failed to fetch server partnumbers_json", "server_id", sb.ID, "error", err)
continue
}
localBook := &localdb.LocalPartnumberBook{
ServerID: sb.ID,
Version: sb.Version,
@@ -99,6 +119,30 @@ func (s *Service) PullPartnumberBooks() (int, error) {
return pulled, nil
}
// markPartnumberBookRepair records a re-pull attempt for a book version and
// reports whether this is the first attempt in the current process.
func (s *Service) markPartnumberBookRepair(serverID int, version string) bool {
s.pnRepairMu.Lock()
defer s.pnRepairMu.Unlock()
if s.pnRepairAttempts == nil {
s.pnRepairAttempts = map[int]string{}
}
if v, ok := s.pnRepairAttempts[serverID]; ok && v == version {
return false
}
s.pnRepairAttempts[serverID] = version
return true
}
// fetchServerBookPartnumbers loads and decodes partnumbers_json of one server book.
func fetchServerBookPartnumbers(mariaDB *gorm.DB, serverID int) (localdb.LocalStringList, error) {
var raw string
if err := mariaDB.Raw("SELECT partnumbers_json FROM qt_partnumber_books WHERE id = ?", serverID).Scan(&raw).Error; err != nil {
return nil, err
}
return decodeServerPartnumbers(raw)
}
// pullBookItems fetches catalog items for a partnumber list from MariaDB and saves them to SQLite.
// Returns the number of items saved.
func pullBookItems(mariaDB *gorm.DB, repo *repository.PartnumberBookRepository, partnumbers localdb.LocalStringList) (int, error) {
+105
View File
@@ -0,0 +1,105 @@
package sync
import (
"log/slog"
"gorm.io/gorm"
)
// serverPullOverlap re-reads rows changed shortly before the cursor, covering
// transactions that stamped server_updated_at before the cursor but committed
// after the previous pull read it.
const serverPullOverlap = "INTERVAL 120 SECOND"
// serverPull describes how one pull of a server table should run. See
// bible-local/decisions/2026-10-26-incremental-config-project-pull.md.
type serverPull struct {
table string
scope string
since string // cursor; empty = full pull
startedAt string // server NOW(6) before reading; becomes the next cursor
trackable bool // server has server_updated_at (migration 035)
}
// beginServerPull decides between a full and an incremental pull of table.
// The first pull of each table per process is full, so rows lost locally are
// restored at least once per app start; later pulls read only rows whose
// server_updated_at moved since the stored cursor.
func (s *Service) beginServerPull(mariaDB *gorm.DB, table string) serverPull {
pull := serverPull{table: table}
if !s.serverHasChangeMarker(mariaDB, table) {
return pull
}
if err := mariaDB.Raw("SELECT DATE_FORMAT(NOW(6), '%Y-%m-%d %H:%i:%s.%f')").Scan(&pull.startedAt).Error; err != nil || pull.startedAt == "" {
slog.Warn("server pull: failed to read server time, falling back to full pull", "table", table, "error", err)
return serverPull{table: table}
}
scope, err := s.localDB.ServerPullScope()
if err != nil {
return serverPull{table: table}
}
pull.scope = scope
pull.trackable = true
s.pullMu.Lock()
fullDone := s.fullPullDone[table]
s.pullMu.Unlock()
if fullDone {
if since, ok := s.localDB.GetServerPullCursor(table, scope); ok {
pull.since = since
}
}
return pull
}
// incrementalScope narrows query to rows changed since the cursor.
func (p serverPull) incrementalScope(query *gorm.DB) *gorm.DB {
return query.Where("server_updated_at >= CAST(? AS DATETIME(6)) - "+serverPullOverlap, p.since)
}
// finishServerPull records a successful pull: the cursor moves to the server
// time taken before reading, and a full pull marks the table as verified for
// this process.
func (s *Service) finishServerPull(pull serverPull) {
if !pull.trackable {
return
}
if err := s.localDB.SetServerPullCursor(pull.table, pull.scope, pull.startedAt); err != nil {
slog.Warn("server pull: failed to store cursor", "table", pull.table, "error", err)
return
}
if pull.since == "" {
s.pullMu.Lock()
if s.fullPullDone == nil {
s.fullPullDone = map[string]bool{}
}
s.fullPullDone[pull.table] = true
s.pullMu.Unlock()
}
}
// serverHasChangeMarker reports whether table has server_updated_at. A positive
// answer is cached for the process; a negative one is re-checked next time so a
// migration applied while the client runs is picked up.
func (s *Service) serverHasChangeMarker(mariaDB *gorm.DB, table string) bool {
s.pullMu.Lock()
known := s.changeMarker[table]
s.pullMu.Unlock()
if known {
return true
}
var n int64
if err := mariaDB.Raw(`
SELECT COUNT(*) FROM information_schema.COLUMNS
WHERE table_schema = DATABASE() AND table_name = ? AND column_name = 'server_updated_at'
`, table).Scan(&n).Error; err != nil || n == 0 {
return false
}
s.pullMu.Lock()
if s.changeMarker == nil {
s.changeMarker = map[string]bool{}
}
s.changeMarker[table] = true
s.pullMu.Unlock()
return true
}
+129 -25
View File
@@ -30,6 +30,13 @@ type Service struct {
directDB *gorm.DB
pricelistMu sync.Mutex // prevents concurrent pricelist syncs
schemaOnce sync.Once // ensures ensureClientSchemaStateTable runs at most once per process
pnRepairMu sync.Mutex
pnRepairAttempts map[int]string // server book id -> version already re-pulled this process
pullMu sync.Mutex
changeMarker map[string]bool // server table -> has server_updated_at
fullPullDone map[string]bool // server table -> full pull succeeded this process
}
// NewService creates a new sync service
@@ -118,13 +125,27 @@ func (s *Service) ImportConfigurationsToLocal() (*ConfigImportResult, error) {
configRepo := repository.NewConfigurationRepository(mariaDB)
result := &ConfigImportResult{}
pull := s.beginServerPull(mariaDB, "qt_configurations")
offset := 0
const limit = 200
for {
serverConfigs, _, err := configRepo.ListAll(offset, limit)
if err != nil {
return nil, fmt.Errorf("listing server configurations: %w", err)
var serverConfigs []models.Configuration
if pull.since != "" {
// Incremental: the changed set is small, read it in one go.
if offset > 0 {
break
}
if err := pull.incrementalScope(mariaDB.Model(&models.Configuration{})).
Order("created_at DESC, id DESC").
Find(&serverConfigs).Error; err != nil {
return nil, fmt.Errorf("listing changed server configurations: %w", err)
}
} else {
serverConfigs, _, err = configRepo.ListAll(offset, limit)
if err != nil {
return nil, fmt.Errorf("listing server configurations: %w", err)
}
}
if len(serverConfigs) == 0 {
break
@@ -171,6 +192,7 @@ func (s *Service) ImportConfigurationsToLocal() (*ConfigImportResult, error) {
offset += len(serverConfigs)
}
s.finishServerPull(pull)
return result, nil
}
@@ -184,13 +206,27 @@ func (s *Service) ImportProjectsToLocal() (*ProjectImportResult, error) {
projectRepo := repository.NewProjectRepository(mariaDB)
result := &ProjectImportResult{}
pull := s.beginServerPull(mariaDB, "qt_projects")
offset := 0
const limit = 200
for {
serverProjects, _, err := projectRepo.List(offset, limit, true)
if err != nil {
return nil, fmt.Errorf("listing server projects: %w", err)
var serverProjects []models.Project
if pull.since != "" {
// Incremental: the changed set is small, read it in one go.
if offset > 0 {
break
}
if err := pull.incrementalScope(mariaDB.Model(&models.Project{})).
Order("created_at DESC").
Find(&serverProjects).Error; err != nil {
return nil, fmt.Errorf("listing changed server projects: %w", err)
}
} else {
serverProjects, _, err = projectRepo.List(offset, limit, true)
if err != nil {
return nil, fmt.Errorf("listing server projects: %w", err)
}
}
if len(serverProjects) == 0 {
break
@@ -247,6 +283,7 @@ func (s *Service) ImportProjectsToLocal() (*ProjectImportResult, error) {
offset += len(serverProjects)
}
s.finishServerPull(pull)
return result, nil
}
@@ -291,28 +328,30 @@ func (s *Service) NeedSync() (bool, error) {
mariaDB, err := s.getDB()
if err == nil {
pricelistRepo := repository.NewPricelistRepository(mariaDB)
sources := []models.PricelistSource{
models.PricelistSourceEstimate,
models.PricelistSourceWarehouse,
models.PricelistSourceCompetitor,
sources := []string{
string(models.PricelistSourceEstimate),
string(models.PricelistSourceWarehouse),
string(models.PricelistSourceCompetitor),
}
allMatch := true
for _, source := range sources {
latestServer, err := pricelistRepo.GetLatestActiveBySource(string(source))
if err != nil {
continue
}
latestLocal, err := s.localDB.GetLatestLocalPricelistBySource(string(source))
if err != nil {
return true, nil
}
if latestServer.ID != latestLocal.ServerID {
return true, nil
}
}
if allMatch {
latestServer, err := pricelistRepo.GetLatestActiveIDsBySources(sources)
if err != nil {
slog.Warn("failed to check latest server pricelists", "error", err)
return false, nil
}
for _, source := range sources {
serverID, ok := latestServer[source]
if !ok {
continue
}
latestLocal, err := s.localDB.GetLatestLocalPricelistBySource(source)
if err != nil {
return true, nil
}
if serverID != latestLocal.ServerID {
return true, nil
}
}
return false, nil
}
}
@@ -941,6 +980,12 @@ func (s *Service) PushPendingChanges() (int, error) {
if _, err := s.EnsureReadinessForSync(); err != nil {
return 0, err
}
return s.pushPendingChanges()
}
// pushPendingChanges pushes without its own readiness check; the caller has
// just run EnsureReadinessForSync (and with it the client heartbeat).
func (s *Service) pushPendingChanges() (int, error) {
removed, err := s.localDB.PurgeOrphanConfigurationPendingChanges()
if err != nil {
@@ -957,6 +1002,8 @@ func (s *Service) PushPendingChanges() (int, error) {
slog.Info("auto-repaired errored pending changes", "repaired", repaired)
}
s.requeueOrphanedPendingConfigurations()
changes, err := s.localDB.GetPendingChanges()
if err != nil {
return 0, fmt.Errorf("getting pending changes: %w", err)
@@ -1019,6 +1066,63 @@ func (s *Service) PushPendingChanges() (int, error) {
return pushed, nil
}
// requeueOrphanedPendingConfigurations re-enqueues an update for configurations that are
// still sync_status=pending but have no queue row left (the row was dropped, e.g. a stale
// event was skipped without marking the config synced). The update carries the current
// local snapshot and version, so the server receives the latest state.
func (s *Service) requeueOrphanedPendingConfigurations() int {
orphans, err := s.localDB.ListConfigurationsPendingWithoutQueue()
if err != nil {
slog.Warn("failed to list orphaned pending configurations", "error", err)
return 0
}
requeued := 0
for _, orphan := range orphans {
cfg, versionID, versionNo, err := s.loadCurrentConfigurationState(orphan.UUID)
if err != nil {
slog.Warn("cannot requeue orphaned pending configuration", "uuid", orphan.UUID, "error", err)
continue
}
now := time.Now().UTC()
payload := ConfigurationChangePayload{
EventID: uuid.New().String(),
IdempotencyKey: fmt.Sprintf("%s:v%d:requeue", orphan.UUID, versionNo),
ConfigurationUUID: orphan.UUID,
ProjectUUID: cfg.ProjectUUID,
PricelistID: cfg.PricelistID,
Operation: "update",
CurrentVersionID: versionID,
CurrentVersionNo: versionNo,
ConflictPolicy: "last_write_wins",
Snapshot: cfg,
CreatedAt: now,
}
raw, err := json.Marshal(payload)
if err != nil {
slog.Warn("cannot marshal requeued configuration payload", "uuid", orphan.UUID, "error", err)
continue
}
if err := s.localDB.DB().Create(&localdb.PendingChange{
EntityType: "configuration",
EntityUUID: orphan.UUID,
Operation: "update",
Payload: string(raw),
CreatedAt: now,
}).Error; err != nil {
slog.Warn("cannot enqueue requeued configuration", "uuid", orphan.UUID, "error", err)
continue
}
requeued++
}
if requeued > 0 {
slog.Info("requeued orphaned pending configurations", "requeued", requeued)
}
return requeued
}
// pushSingleChange pushes a single pending change to the server
func (s *Service) pushSingleChange(change *localdb.PendingChange) error {
switch change.EntityType {
@@ -0,0 +1,66 @@
package sync_test
import (
"testing"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/models"
"git.mchus.pro/mchus/quoteforge/internal/services"
syncsvc "git.mchus.pro/mchus/quoteforge/internal/services/sync"
)
// Reproduces the state seen in the field bundle: configuration stays sync_status=pending
// while its pending_changes rows are gone. Push must not leave it pending forever.
func TestPushPendingChangesRequeuesOrphanedPendingConfiguration(t *testing.T) {
local := newLocalDBForSyncTest(t)
serverDB := newServerDBForSyncTest(t)
localSync := syncsvc.NewService(nil, local)
configService := services.NewLocalConfigurationService(local, localSync, &services.QuoteService{}, func() bool { return false })
pushService := syncsvc.NewServiceWithDB(serverDB, local)
created, err := configService.Create("tester", &services.CreateConfigRequest{
Name: "Orphan v1",
Items: models.ConfigItems{{LotName: "CPU_O", Quantity: 1, UnitPrice: 500}},
ServerCount: 1,
})
if err != nil {
t.Fatalf("create config: %v", err)
}
if _, err := configService.UpdateNoAuth(created.UUID, &services.CreateConfigRequest{
Name: "Orphan v2",
Items: models.ConfigItems{{LotName: "CPU_O", Quantity: 2, UnitPrice: 500}},
ServerCount: 1,
ProjectUUID: created.ProjectUUID,
}); err != nil {
t.Fatalf("update config: %v", err)
}
// Simulate the dropped queue: rows are gone, local config still marked pending.
if err := local.DB().Where("1 = 1").Delete(&localdb.PendingChange{}).Error; err != nil {
t.Fatalf("clear pending changes: %v", err)
}
if _, err := pushService.PushPendingChanges(); err != nil {
t.Fatalf("push: %v", err)
}
var serverCfg models.Configuration
if err := serverDB.Where("uuid = ?", created.UUID).First(&serverCfg).Error; err != nil {
t.Fatalf("configuration not on server: %v", err)
}
if serverCfg.Name != "Orphan v2" {
t.Fatalf("expected latest version on server, got %q", serverCfg.Name)
}
localCfg, err := local.GetConfigurationByUUID(created.UUID)
if err != nil {
t.Fatalf("get local config: %v", err)
}
if localCfg.SyncStatus != "synced" {
t.Fatalf("expected sync_status synced, got %q", localCfg.SyncStatus)
}
if got := local.CountPendingChanges(); got != 0 {
t.Fatalf("expected empty pending queue, got %d", got)
}
}
+32 -17
View File
@@ -3,12 +3,29 @@ package sync
import (
"context"
"log/slog"
"math/rand/v2"
"time"
"git.mchus.pro/mchus/quoteforge/internal/db"
)
// Worker performs background synchronization at regular intervals
// WorkerJitter spreads background sync cycles by ±30% of the interval so that
// clients started at the same time do not hit MariaDB in lockstep.
const WorkerJitter = 0.3
// jitteredInterval returns base scaled by a random factor in [1-WorkerJitter, 1+WorkerJitter].
func jitteredInterval(base time.Duration) time.Duration {
factor := 1 - WorkerJitter + rand.Float64()*2*WorkerJitter
return time.Duration(float64(base) * factor)
}
// nextCycleDelay returns how long to wait so the next cycle starts a jittered
// interval after started; zero if the cycle itself took longer than that.
func nextCycleDelay(started time.Time, base time.Duration) time.Duration {
return max(jitteredInterval(base)-time.Since(started), 0)
}
// Worker performs background synchronization at jittered intervals
type Worker struct {
service *Service
connMgr *db.ConnectionManager
@@ -37,12 +54,14 @@ func (w *Worker) isOnline() bool {
func (w *Worker) Start(ctx context.Context) {
w.logger.Info("starting background sync worker", "interval", w.interval)
ticker := time.NewTicker(w.interval)
defer ticker.Stop()
// Run once immediately
// Run once immediately. The next cycle is scheduled from the start of the
// previous one, so heartbeats are at most (1+WorkerJitter)*interval apart.
started := time.Now()
w.runSync()
timer := time.NewTimer(nextCycleDelay(started, w.interval))
defer timer.Stop()
for {
select {
case <-ctx.Done():
@@ -51,8 +70,10 @@ func (w *Worker) Start(ctx context.Context) {
case <-w.stopCh:
w.logger.Info("background sync worker stopped")
return
case <-ticker.C:
case <-timer.C:
started = time.Now()
w.runSync()
timer.Reset(nextCycleDelay(started, w.interval))
}
}
}
@@ -71,28 +92,22 @@ func (w *Worker) runSync() {
return
}
// Each stream is independent: a failure in one must not stop the others.
// Readiness only gates pushing local changes to the server.
if readiness, err := w.service.EnsureReadinessForSync(); err != nil {
w.logger.Warn("background sync: blocked by readiness guard",
w.logger.Warn("background sync: push blocked by readiness guard",
"error", err,
"reason_code", readiness.ReasonCode,
"reason_text", readiness.ReasonText,
)
return
}
// Push pending changes first
pushed, err := w.service.PushPendingChanges()
if err != nil {
} else if pushed, err := w.service.pushPendingChanges(); err != nil {
w.logger.Warn("background sync: failed to push pending changes", "error", err)
} else if pushed > 0 {
w.logger.Info("background sync: pushed pending changes", "count", pushed)
}
// Then check for new pricelists
err = w.service.SyncPricelistsIfNeeded()
if err != nil {
if err := w.service.SyncPricelistsIfNeeded(); err != nil {
w.logger.Warn("background sync: failed to sync pricelists", "error", err)
return
}
// Pull partnumber books together with pricelists
@@ -0,0 +1,23 @@
package sync
import (
"testing"
"time"
)
func TestJitteredIntervalStaysWithinBounds(t *testing.T) {
base := time.Minute
lo := time.Duration(float64(base) * (1 - WorkerJitter))
hi := time.Duration(float64(base) * (1 + WorkerJitter))
seen := map[time.Duration]struct{}{}
for i := 0; i < 1000; i++ {
d := jitteredInterval(base)
if d < lo || d > hi {
t.Fatalf("interval %v outside [%v, %v]", d, lo, hi)
}
seen[d] = struct{}{}
}
if len(seen) < 100 {
t.Fatalf("expected spread of intervals, got %d distinct values", len(seen))
}
}
+20
View File
@@ -0,0 +1,20 @@
-- Tables affected: qt_configurations, qt_projects
-- recovery.not-started: safe to re-run; ADD COLUMN IF NOT EXISTS / ADD INDEX IF NOT EXISTS
-- recovery.partial: ALTER TABLE qt_configurations DROP COLUMN server_updated_at; ALTER TABLE qt_projects DROP COLUMN server_updated_at;
-- recovery.completed: no action needed
-- verify: qt_configurations.server_updated_at column missing | SELECT 1 FROM information_schema.COLUMNS WHERE table_schema=DATABASE() AND table_name='qt_configurations' AND column_name='server_updated_at' HAVING COUNT(*)=0
-- verify: qt_projects.server_updated_at column missing | SELECT 1 FROM information_schema.COLUMNS WHERE table_schema=DATABASE() AND table_name='qt_projects' AND column_name='server_updated_at' HAVING COUNT(*)=0
-- Server-maintained change marker for incremental client pulls. Never written by
-- clients: DEFAULT on INSERT and ON UPDATE on any row change, so clients that do not
-- know the column keep working unchanged. See
-- bible-local/decisions/2026-10-26-incremental-config-project-pull.md.
ALTER TABLE qt_configurations
ADD COLUMN IF NOT EXISTS server_updated_at TIMESTAMP(6) NOT NULL
DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
ADD INDEX IF NOT EXISTS idx_qt_configurations_server_updated_at (server_updated_at, id);
ALTER TABLE qt_projects
ADD COLUMN IF NOT EXISTS server_updated_at TIMESTAMP(6) NOT NULL
DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
ADD INDEX IF NOT EXISTS idx_qt_projects_server_updated_at (server_updated_at, id);
+18
View File
@@ -0,0 +1,18 @@
# QuoteForge v3.42
Дата релиза: 2026-09-24
Тег: `v3.42`
Предыдущий релиз: `v3.41`
## Ключевые изменения
- отрицательная накидка по строке в «Цена продажи» теперь работает (скидка); итоговая цена строки не уходит ниже нуля — ни в таблице, ни в CSV;
- накидки больше не теряются при сохранении: клиент переносит незнакомые ему поля `notes` как есть. Клиенты до v3.41 по-прежнему стирают построчные накидки при сохранении, а v3.41 — отрицательные: всем нужно обновиться;
- фича-флаги конфигурации (`feature_flags`): если конфигурация использует функцию, неизвестную этой версии, клиент отказывается её сохранять и синхронизировать, показывает баннер со списком неизвестных флагов и просит обновиться. Первый флаг — `sale_row_markups`;
- **требуется серверная миграция `034_add_feature_flags_to_configurations.sql`** (`qfs -migrate`); без неё синхронизация работает, но флаги на сервере не сохраняются.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+19
View File
@@ -0,0 +1,19 @@
# QuoteForge v3.43
Дата релиза: 2026-09-24
Тег: `v3.43`
Предыдущий релиз: `v3.42`
## Ключевые изменения
- серверные миграции: `qfs -migrate` (`make migrate`) применяет только SQL-файлы `migrations/*.sql`, встроенные в бинарник, с учётом в `qt_schema_migrations`, и завершается без запуска приложения;
- новые команды `make migrate-status` (список применённых/ожидающих миграций, только чтение) и `make migrate-baseline UPTO=<file.sql>` (пометить миграции применёнными без выполнения);
- `-migrate` отказывается работать, если в истории нет ни одной применённой миграции, — иначе все файлы выполнились бы заново;
- удалён GORM AutoMigrate против MariaDB: он менял типы колонок боевой базы под модели запущенного бинарника;
- миграции идут через отдельное соединение без 3-секундного таймаута — длинный backfill (032) больше не обрывается.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+17
View File
@@ -0,0 +1,17 @@
# QuoteForge v3.44
Дата релиза: 2026-09-24
Тег: `v3.44`
Предыдущий релиз: `v3.43`
## Ключевые изменения
- модалка «Новая конфигурация в проекте» строит кнопки типа оборудования из серверной настройки `qt_settings.config_types` — новые типы (например, «Сеть») появляются без релиза клиента;
- кнопка полной синхронизации теперь сразу подтягивает серверные настройки конфигуратора (`qt_settings`), а не только фоновый воркер раз в 5 минут;
- копирование конфигурации (в т.ч. из версии) сохраняет тип оборудования — раньше копия СХД/сети становилась «сервером».
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+18
View File
@@ -0,0 +1,18 @@
# QuoteForge v3.45
Дата релиза: 2026-09-24
Тег: `v3.45`
Предыдущий релиз: `v3.44`
## Ключевые изменения
- вкладки конфигуратора (набор, порядок, названия) строятся из серверной настройки `qt_settings.tab_config` — вкладка, убранная на сервере (например, SW), больше не остаётся пустой кнопкой, новые вкладки появляются без релиза клиента;
- из страницы конфигуратора убраны захардкоженные вкладки и фильтры категорий по типу конфигурации; единственный запасной вариант — встроенные дефолты сервера приложения, используются только если настройки ещё ни разу не синхронизировались;
- по умолчанию всегда видна только вкладка Base, остальные скрываются, если для типа конфигурации в них нечего выбрать;
- настройки конфигуратора загружаются до списка категорий — вкладка «Other» больше не захватывает категории, которые относятся к серверным вкладкам.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+15
View File
@@ -0,0 +1,15 @@
# QuoteForge v3.46
Дата релиза: 2026-10-02
Тег: `v3.46`
Предыдущий релиз: `v3.45`
## Ключевые изменения
- в конфигураторе поверх выпадашки выбора LOT больше не появляется второй список — история автозаполнения браузера; она отключена на всех полях с собственной выпадашкой (вкладка Base, остальные вкладки, редактирование позиции, сопоставление BOM).
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+16
View File
@@ -0,0 +1,16 @@
# QuoteForge v3.47
Дата релиза: 2026-10-05
Тег: `v3.47`
Предыдущий релиз: `v3.46`
## Ключевые изменения
- конфигурации, застрявшие в статусе «pending» без записи в очереди синка, теперь автоматически ставятся в очередь заново и уходят на сервер при следующем цикле синка;
- фоновый синк разделён на независимые потоки: ошибка push, готовности или синка прайслистов больше не останавливает синк партбуков и настроек конфигуратора.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+16
View File
@@ -0,0 +1,16 @@
# QuoteForge v3.48
Дата релиза: 2026-10-14
Тег: `v3.48`
Предыдущий релиз: `v3.47`
## Ключевые изменения
- режим расчёта поддержки HGX теперь выбирается только для GPU с SXM/HGX в названии или H100/H200/B200/B300 не в PCIe-исполнении; серверы с PCIe GPU (например, A30) считаются как x86 — процент от цены;
- значения по умолчанию цен поддержки HGX обновлены по регламенту от 2026-10-07 (основной источник цен — `qt_settings["support_pricing"]` в MariaDB, его нужно обновить отдельно).
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+17
View File
@@ -0,0 +1,17 @@
# QuoteForge v3.49
Дата релиза: 2026-10-16
Тег: `v3.49`
Предыдущий релиз: `v3.48`
## Ключевые изменения
- вкладка «Ценообразование», таблица «Цена покупки»: колонки Estimate / Склад / Конкуренты / Ручная цена показывают сумму по строке (цена × кол-во), как «Цена продажи» — строки теперь складываются в «Итого»;
- доля «Ручной цены» и кнопка «BOM Цена» также показывают суммы по строке, а не цену за штуку;
- подпись над таблицами исправлена: «Суммы за всё количество» вместо «Цены указаны за 1 шт.».
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+26
View File
@@ -0,0 +1,26 @@
# QuoteForge v3.50
Дата релиза: 2026-10-26
Тег: `v3.50`
Предыдущий релиз: `v3.49`
## Ключевые изменения
- локальный API и первоначальная настройка защищены от CSRF и DNS rebinding; сохранённый пароль не отправляется при изменении хоста, порта или пользователя без явного ввода пароля (#2);
- устранены XSS через LOT, BOM и сообщения синхронизации; обработчики корзины и аренды получают LOT через data-атрибуты (#4);
- перед сбросом локальной базы создаётся отдельный проверенный архив; несинхронизированные данные защищены отказом без явного принудительного сброса, переменная QFS_RESET_LOCAL_DB больше не запускает сброс (#7);
- в настройках подключения MariaDB доступны TLS preferred, required, verify-ca и verify-full, а также CA-сертификат; preferred допускает соединение без TLS, verify-full проверяет сертификат и имя сервера (#5);
- SQL-миграции выполняются по одной сессии на файл, запускают verify-проверки до записи в историю и не скрывают ошибки внешних ключей (#3);
- минимальная версия Go повышена до 1.27.3, обновлены зависимости; релизная сборка блокируется при достижимых уязвимостях по govulncheck (#6).
## Проверки
Пройдены Go-тесты, go vet, проверка XSS и сборки macOS arm64 / Windows amd64.
Govulncheck: 0 достижимых уязвимостей; остаётся одна запись на уровне модуля,
уязвимый код которого приложение не вызывает.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+26
View File
@@ -0,0 +1,26 @@
# QuoteForge v3.51
Дата релиза: 2026-10-26
Тег: `v3.51`
Предыдущий релиз: `v3.50`
## Ключевые изменения
- устранена утечка подключений к MariaDB: при синхронизации клиент каждые 30 секунд открывал новый пул, не закрывая старый, и держал до ~30 соединений; теперь не больше 2, простаивающие закрываются через минуту;
- фоновая синхронизация раз в минуту со случайным разбросом ±30%, чтобы клиенты не обращались к серверу одновременно; статус «онлайн» в списке пользователей считается по интервалу +30%;
- списки проектов и конфигураций подгружают с сервера только изменённые записи (миграция `035`, колонка `server_updated_at`): ~0,35 с вместо ~27 с и 3 МБ; полная загрузка — один раз при запуске приложения; старые клиенты и сервер без миграции работают как раньше;
- PN-книги больше не скачиваются целиком на каждом цикле: забираются только заголовки, содержимое — для новых или неполных книг; неполная книга докачивается один раз за запуск с предупреждением в логе;
- проверка новых прайслистов одним запросом вместо трёх, одна отметка о клиенте на цикл синхронизации, настройки конфигуратора не перезаписываются без изменений;
- исправлен мигратор: файл с двумя и более `verify`-проверками применялся, но не записывался в историю миграций.
## Проверки
Пройдены Go-тесты и go vet. Инкрементальная загрузка проверена интеграционным тестом
на MariaDB 11.8 со схемой продакшена и на продакшен-сервере после миграции `035`.
Govulncheck: 0 достижимых уязвимостей.
## Запуск на macOS
Снимите карантинный атрибут через терминал: `xattr -d com.apple.quarantine /path/to/qfs-darwin-arm64`
После этого бинарник запустится без предупреждения Gatekeeper.
+3
View File
@@ -21,6 +21,9 @@ fi
echo -e "${GREEN}Building QuoteForge version: ${VERSION}${NC}"
echo ""
# Stop before creating release artifacts if reachable vulnerabilities are found.
make security-check
ensure_release_notes() {
local notes_path="$1"
if [ -f "${notes_path}" ]; then
+98
View File
@@ -0,0 +1,98 @@
const assert = require('node:assert/strict');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const {spawnSync} = require('node:child_process');
const index = fs.readFileSync('web/templates/index.html', 'utf8');
const base = fs.readFileSync('web/templates/base.html', 'utf8');
function extract(source, name) {
const match = source.match(new RegExp(`^( *)(?:async )?function ${name}\\(`, 'm'));
assert(match, `Missing function ${name}`);
const end = source.indexOf(`\n${match[1]}}`, match.index);
assert(end > match.index);
return source.slice(match.index, end + match[1].length + 2);
}
const functions = [
...['renderMultiSelectTab', 'renderMultiSelectTabWithSections', 'renderRentalTab', '_renderBOMParsedTable'].map(n => extract(index, n)),
...['showToast', 'loadSyncInfo', 'repairPendingChanges'].map(n => extract(base, n)),
].join('\n');
const ids = new Set([...functions.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
const fixtures = [...ids].map(id => id.endsWith('-body') ? `<table><tbody id="${id}"></tbody></table>` : `<div id="${id}"></div>`).join('');
const listeners = index.split('<script>')[1].split('// Tab configuration')[0];
const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'qfs-browser-security-'));
const target = path.join(directory, 'test.html');
const test = `
let cart = [], allComponents = [], currentTab = 'cpu', configUUID = 'example';
let rentalUpliftPercent = 0, bomRows = [], reply = {}, calls = [];
window.__injected = 0;
const getTabForCategory = () => 'cpu', getDisplayPrice = () => 1;
const qualityDotHtml = () => '', formatMoney = String, formatPriceOrNA = String, formatCurrency = String;
const ciStr = s => s.toLowerCase(), getComponentCategory = c => c.category;
const _setBomUIError = () => {}, _ensureBomDatalist = () => {}, _getRowLotQtyPerPN = () => 1;
const renderBOMLotAllocationsEditor = () => '', buildRentalItemsPayload = () => [];
const removeFromCart = lot => calls.push(['remove', lot]);
const updateMultiQuantity = (lot, value) => calls.push(['quantity', lot, value]);
const onRentalConditionChange = (lot, value) => calls.push(['condition', lot, value]);
const showAutocompleteEditItem = lot => calls.push(['edit', lot]);
window.fetch = async () => ({ok: true, json: async () => reply});
function check(ok, message) { if (!ok) throw new Error(message); }
function inert(id, payload) {
const el = document.getElementById(id);
check(el.textContent.includes(payload), id + ': text did not survive');
check(!el.querySelector('img,script,iframe'), id + ': injected element');
}
(async () => {
for (const lot of [
"LOT'\\\"><img src=x onerror=window.__injected++>",
'LOT&#X61;&#39;);window.__injected++;//',
'LOT\\\\&<>"',
]) {
cart = [{lot_name: lot, description: lot, category: 'cpu', quantity: 1}];
for (const render of [() => renderMultiSelectTab([]), () => renderMultiSelectTabWithSections([{title: 'CPU', categories: ['cpu']}])]) {
render();
const root = document.getElementById('tab-content');
check(!root.querySelector('img,script'), 'cart: injected element');
const quantity = root.querySelector('[data-cart-action="quantity"]');
check(quantity.dataset.lot === lot, 'cart: dataset changed');
quantity.value = '2'; quantity.dispatchEvent(new Event('change', {bubbles: true}));
root.querySelector('[data-cart-action="edit"]').dispatchEvent(new FocusEvent('focusin', {bubbles: true}));
root.querySelector('[data-cart-action="remove"] svg').dispatchEvent(new MouseEvent('click', {bubbles: true}));
check(calls.length === 3 && calls.every(c => c[1] === lot), 'cart: delegated action changed LOT');
calls = [];
}
reply = {items: [{...cart[0], condition: 'new', buy_price: 1, price: 1, one_time: 1, weekly: 1}]};
await renderRentalTab();
inert('rental-body', lot);
const condition = document.querySelector('[data-cart-action="condition"]');
condition.dispatchEvent(new Event('change', {bubbles: true}));
check(calls.length === 1 && calls[0][1] === lot, 'rental: delegated action changed LOT'); calls = [];
bomRows = [{resolved_lot: lot, vendor_pn: lot, description: lot, quantity: 1, resolution_source: 'manual'}];
_renderBOMParsedTable(); inert('bom-table-body', lot);
showToast(lot, 'error'); inert('toast', lot);
reply = {is_online: false, config_count: 0, project_count: 0,
pending_changes: [{entity_uuid: 'example', operation: lot, entity_type: lot, last_error: lot}],
errors: [{message: lot}]};
await loadSyncInfo(); inert('modal-pending-list', lot); inert('modal-errors-list', lot);
reply = {success: true, repaired: 0, remaining_errors: [lot]};
await repairPendingChanges(); inert('repair-result', lot);
}
await new Promise(resolve => setTimeout(resolve, 50));
check(window.__injected === 0, 'payload executed');
document.getElementById('result').textContent = 'PASS';
})().catch(error => {document.getElementById('result').textContent = 'FAIL: ' + error.stack;});
`;
fs.writeFileSync(target, `<!doctype html><meta charset="utf-8">${fixtures}<pre id="result">PENDING</pre><script>${fs.readFileSync('web/static/html-escape.js', 'utf8')}\n${functions}\n${listeners}\n${test}</script>`);
const chrome = process.env.CHROME_BIN || '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
try {
const result = spawnSync(chrome, ['--headless', '--disable-gpu', '--no-first-run', '--disable-background-networking',
'--user-data-dir=' + path.join(directory, 'profile'), '--virtual-time-budget=1000', '--dump-dom', 'file://' + target], {encoding: 'utf8', timeout: 30000});
assert.ifError(result.error);
assert.equal(result.status, 0, result.stderr);
const outcome = result.stdout.match(/<pre id="result">([\s\S]*?)<\/pre>/)?.[1];
assert.equal(outcome, 'PASS', outcome || 'Browser did not finish');
console.log('Browser HTML security regressions passed');
} finally {
fs.rmSync(directory, {recursive: true, force: true});
}
+55
View File
@@ -0,0 +1,55 @@
const assert = require('node:assert/strict');
const fs = require('node:fs');
const vm = require('node:vm');
const source = fs.readFileSync('web/templates/index.html', 'utf8');
const base = fs.readFileSync('web/templates/base.html', 'utf8');
const nodes = {};
const listeners = {};
const calls = [];
const context = vm.createContext({
document: {
getElementById: id => nodes[id] ??= {},
createElement: () => ({}),
addEventListener: (name, handler) => listeners[name] = handler,
},
setTimeout: () => {},
cart: [], allComponents: [], currentTab: 'cpu',
getTabForCategory: () => 'cpu', getDisplayPrice: () => 1,
qualityDotHtml: () => '', formatMoney: String, formatPriceOrNA: String,
removeFromCart: lot => calls.push(['remove', lot]),
updateMultiQuantity: (lot, value) => calls.push(['quantity', lot, value]),
onRentalConditionChange: (lot, value) => calls.push(['condition', lot, value]),
showAutocompleteEditItem: lot => calls.push(['edit', lot]),
});
function loadFunction(text, name) {
const start = text.indexOf(`function ${name}(`);
assert(start >= 0);
const end = text.indexOf('\n}', start);
// Functions in base.html have eight spaces of indentation.
const baseEnd = text.indexOf('\n }', start);
vm.runInContext(text.slice(start, (end < 0 ? baseEnd + 10 : end + 2)), context);
}
vm.runInContext(fs.readFileSync('web/static/html-escape.js', 'utf8'), context);
loadFunction(source, 'renderMultiSelectTab');
vm.runInContext(source.split('<script>')[1].split('// Tab configuration')[0], context);
loadFunction(base, 'showToast');
nodes.toast = {replaceChildren: child => nodes.toast.child = child};
for (const lot of [`LOT'\"<img src=x onerror=alert(1)>`, 'LOT&#X61;&#39;);alert(1)//', 'LOT\\\'()&<>']) {
context.cart = [{lot_name: lot, quantity: 1, category: 'cpu'}];
context.renderMultiSelectTab([]);
const html = nodes['tab-content'].innerHTML;
assert(html.includes(`data-lot="${context.escapeHtml(lot)}"`));
assert(!html.includes(lot));
assert(!html.includes('onclick="removeFromCart('));
listeners.click({target: {closest: () => ({dataset: {lot}})}});
listeners.change({target: {dataset: {lot, cartAction: 'quantity'}, value: '2'}});
listeners.change({target: {dataset: {lot, cartAction: 'condition'}, checked: true}});
listeners.focusin({target: {dataset: {lot, cartAction: 'edit'}}});
assert(calls.splice(0).every(call => call[1] === lot));
context.showToast(lot, 'error');
assert.equal(nodes.toast.child.textContent, lot);
assert.equal(nodes.toast.child.innerHTML, undefined);
}
assert.equal(context.escapeHtml(0), '0');
assert.equal(context.escapeHtml(null), '');
console.log('HTML security regressions passed');
+33
View File
@@ -0,0 +1,33 @@
const assert = require('node:assert/strict');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const {spawnSync} = require('node:child_process');
const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'qfs-release-gate-'));
try {
fs.copyFileSync('Makefile', path.join(directory, 'Makefile'));
fs.copyFileSync('scripts/release.sh', path.join(directory, 'release.sh'));
const tools = path.join(directory, 'tools');
fs.mkdirSync(tools);
fs.writeFileSync(path.join(tools, 'go'), '#!/bin/sh\nprintf "%s\\n" "$*" >> "$QFS_TEST_GO_CALLS"\nexit 23\n', {mode: 0o755});
const callsPath = path.join(directory, 'calls');
const env = {...process.env, PATH: tools + path.delimiter + process.env.PATH, QFS_TEST_GO_CALLS: callsPath};
const cases = [
...['build-release', 'build-linux', 'build-macos', 'build-windows', 'build-all'].map(target => ['make', [target]]),
['bash', ['release.sh']],
];
for (const [cmd, args] of cases) {
fs.writeFileSync(callsPath, '');
const result = spawnSync(cmd, args, {cwd: directory, env, encoding: 'utf8', timeout: 10000});
assert.ifError(result.error);
assert.notEqual(result.status, 0, `${cmd} ${args}: scanner failure was ignored`);
const calls = fs.readFileSync(callsPath, 'utf8').trim().split('\n');
assert.deepEqual(calls, ['run golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...']);
assert(!fs.existsSync(path.join(directory, 'bin')), 'Build artifacts created after failed scan');
assert(!fs.existsSync(path.join(directory, 'releases')), 'Release artifacts created after failed scan');
}
console.log('Release security failure checks passed');
} finally {
fs.rmSync(directory, {recursive: true, force: true});
}
+9
View File
@@ -0,0 +1,9 @@
// Escape text for HTML text nodes and quoted attributes, never JavaScript code.
function escapeHtml(value) {
return String(value ?? '')
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
+11 -7
View File
@@ -10,6 +10,7 @@
<script src="/static/vendor/htmx-1.9.10.min.js"></script>
<script>window.QF_INDICATOR_MODE = "{{ .IndicatorMode }}";</script>
<script src="/static/price-quality.js"></script>
<script src="/static/html-escape.js"></script>
<style>
.htmx-request { opacity: 0.5; }
.line-clamp-2 { display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; overflow: hidden; }
@@ -222,7 +223,10 @@
function showToast(msg, type) {
const colors = { success: 'bg-green-500', error: 'bg-red-500', info: 'bg-blue-500' };
const el = document.getElementById('toast');
el.innerHTML = '<div class="' + (colors[type] || colors.info) + ' text-white px-4 py-2 rounded shadow">' + msg + '</div>';
const message = document.createElement('div');
message.className = (colors[type] || colors.info) + ' text-white px-4 py-2 rounded shadow';
message.textContent = msg;
el.replaceChildren(message);
setTimeout(() => el.innerHTML = '', 3000);
}
@@ -340,11 +344,11 @@
const shortUUID = ch.entity_uuid.substring(0, 8);
const time = new Date(ch.created_at).toLocaleString('ru-RU');
const hasError = ch.last_error ? ' border-l-2 border-red-400 pl-2' : '';
const errorLine = ch.last_error ? `<div class="text-red-500 text-xs mt-0.5">${ch.last_error}</div>` : '';
const errorLine = ch.last_error ? `<div class="text-red-500 text-xs mt-0.5">${escapeHtml(ch.last_error)}</div>` : '';
return `<div class="bg-gray-50 rounded px-3 py-1.5${hasError}">
<span class="font-medium">${ch.operation}</span>
<span class="text-gray-500">${ch.entity_type}</span>
<span class="font-mono text-xs text-gray-400">${shortUUID}</span>
<span class="font-medium">${escapeHtml(ch.operation)}</span>
<span class="text-gray-500">${escapeHtml(ch.entity_type)}</span>
<span class="font-mono text-xs text-gray-400">${escapeHtml(shortUUID)}</span>
<span class="text-gray-400 text-xs ml-1">${time}</span>
${errorLine}
</div>`;
@@ -362,7 +366,7 @@
errorsList.innerHTML = data.errors.map(error => {
const time = new Date(error.timestamp).toLocaleString('ru-RU');
return `<div class="bg-red-50 text-red-700 rounded px-3 py-1.5">
<span class="text-xs text-red-400">${time}</span>: ${error.message}
<span class="text-xs text-red-400">${time}</span>: ${escapeHtml(error.message)}
</div>`;
}).join('');
} else {
@@ -409,7 +413,7 @@
resultDiv.className = 'mt-2 text-sm text-yellow-700 bg-yellow-50 rounded px-3 py-2';
resultDiv.textContent = 'Нечего исправлять или проблемы остались';
if (data.remaining_errors && data.remaining_errors.length > 0) {
resultDiv.innerHTML += '<div class="mt-1 text-xs">' + data.remaining_errors.join('<br>') + '</div>';
resultDiv.innerHTML += '<div class="mt-1 text-xs">' + data.remaining_errors.map(escapeHtml).join('<br>') + '</div>';
}
}
} else {
-5
View File
@@ -36,11 +36,6 @@ const configUUID = '{{.ConfigUUID}}';
let configData = null;
let projectData = null;
function escapeHtml(text) {
const div = document.createElement('div');
div.textContent = text || '';
return div.innerHTML;
}
function formatMoney(value) {
const num = Number(value);
-5
View File
@@ -408,11 +408,6 @@ async function resolveUniqueConfigName(baseName, projectUUID, excludeUUID) {
return {name: candidate, changed: true};
}
function escapeHtml(text) {
const div = document.createElement('div');
div.textContent = text;
return div.innerHTML;
}
async function deleteConfig(uuid) {
if (!confirm('Переместить конфигурацию в архив?')) return;
+88 -139
View File
@@ -103,34 +103,7 @@
<div class="bg-white rounded-lg shadow">
<div class="border-b">
<nav class="flex overflow-x-auto" id="category-tabs">
<button onclick="switchTab('base')" data-tab="base"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-blue-600 text-blue-600 whitespace-nowrap">
Base
</button>
<button onclick="switchTab('storage')" data-tab="storage"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
Storage
</button>
<button onclick="switchTab('pci')" data-tab="pci"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
PCI
</button>
<button onclick="switchTab('power')" data-tab="power"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
Power
</button>
<button onclick="switchTab('accessories')" data-tab="accessories"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
Accessories
</button>
<button onclick="switchTab('sw')" data-tab="sw"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
SW
</button>
<button onclick="switchTab('other')" data-tab="other"
class="tab-btn px-4 py-3 text-sm font-medium border-b-2 border-transparent text-gray-500 hover:text-gray-700 whitespace-nowrap">
Other
</button>
<!-- Buttons are rendered from TAB_CONFIG (qt_settings.tab_config) by renderTabButtons() -->
</nav>
</div>
@@ -227,7 +200,7 @@
<div class="bg-white rounded-lg shadow p-4">
<div class="flex items-baseline gap-3 mb-3">
<h3 class="text-base font-semibold text-gray-800">Цена покупки</h3>
<span class="text-xs text-gray-400">Цены указаны за 1 шт.</span>
<span class="text-xs text-gray-400">Суммы за всё количество</span>
</div>
<div class="overflow-x-auto">
<table class="w-full text-sm border-collapse">
@@ -276,7 +249,7 @@
<div class="bg-white rounded-lg shadow p-4">
<div class="flex items-baseline gap-3 mb-1">
<h3 class="text-base font-semibold text-gray-800">Цена продажи</h3>
<span class="text-xs text-gray-400">Цены указаны за 1 шт.</span>
<span class="text-xs text-gray-400">Суммы за всё количество</span>
</div>
<div class="overflow-x-auto">
<table class="w-full text-sm border-collapse">
@@ -431,48 +404,22 @@
</style>
<script>
// Tab configuration - will be populated dynamically
document.addEventListener('click', event => {
const button = event.target.closest('[data-cart-action="remove"]');
if (button) removeFromCart(button.dataset.lot);
});
document.addEventListener('change', event => {
const input = event.target;
if (input.dataset.cartAction === 'quantity') updateMultiQuantity(input.dataset.lot, input.value);
if (input.dataset.cartAction === 'condition') onRentalConditionChange(input.dataset.lot, input.checked);
});
document.addEventListener('focusin', event => {
if (event.target.dataset.cartAction === 'edit') showAutocompleteEditItem(event.target.dataset.lot, event.target);
});
// Tab configuration comes from /api/configurator-settings (qt_settings.tab_config,
// or the Go-side fallback when the key is absent) — see applyServerSettings().
// Only "other" is client-owned: it collects categories not listed in any tab.
let TAB_CONFIG = {
base: {
categories: ['MB', 'CPU', 'MEM', 'ENC', 'DKC', 'CTL'],
singleSelect: true,
label: 'Base'
},
storage: {
categories: ['RAID', 'M2', 'SSD', 'HDD', 'EDSFF', 'HHHL'],
singleSelect: false,
label: 'Storage',
sections: [
{ title: 'RAID Контроллеры', categories: ['RAID'] },
{ title: 'Диски', categories: ['M2', 'SSD', 'HDD', 'EDSFF', 'HHHL'] }
]
},
pci: {
categories: ['GPU', 'DPU', 'NIC', 'HCA', 'HBA', 'HIC'],
singleSelect: false,
label: 'PCI',
sections: [
{ title: 'GPU / DPU', categories: ['GPU', 'DPU'] },
{ title: 'NIC / HCA', categories: ['NIC', 'HCA'] },
{ title: 'HBA', categories: ['HBA'] },
{ title: 'HIC', categories: ['HIC'] }
]
},
power: {
categories: ['PS', 'PSU'],
singleSelect: false,
label: 'Power'
},
accessories: {
categories: ['ACC', 'CARD'],
singleSelect: false,
label: 'Accessories'
},
sw: {
categories: ['SW'],
singleSelect: false,
label: 'SW'
},
other: {
categories: [],
singleSelect: false,
@@ -579,7 +526,7 @@ let allComponents = [];
let cart = [];
let categoryOrderMap = {}; // Category code -> display_order mapping
let configTypeCategoryMap = {}; // configTypeCode → Set<UPPER_CODE> of allowed categories (from server)
let alwaysVisibleTabsSet = null; // Set<tabKey> — null means use hardcoded fallback
let alwaysVisibleTabsSet = new Set(); // Set<tabKey> from qt_settings.always_visible_tabs
let requiredCategoriesMap = {}; // configTypeCode → Set<UPPER_CODE> of required categories
let autoSaveTimeout = null; // Timeout for debounced autosave
let hasUnsavedChanges = false;
@@ -975,6 +922,7 @@ function applyServerSettings(settings) {
supportPricingTable = settings.support_pricing;
}
renderTabButtons();
applyConfigTypeToTabs();
updateTabVisibility();
updateRequiredCategoryBadges();
@@ -1017,9 +965,10 @@ document.addEventListener('DOMContentLoaded', async function() {
return;
}
// Load categories and configurator settings in background (defaults are usable immediately).
const categoriesPromise = loadCategoriesFromAPI().catch(() => {});
loadCfgSettings().then(s => applyServerSettings(s)).catch(() => {});
// Tabs come from configurator settings; "other" is derived from them, so
// categories load after settings.
const settingsPromise = loadCfgSettings().then(s => applyServerSettings(s)).catch(() => {});
const categoriesPromise = settingsPromise.then(() => loadCategoriesFromAPI()).catch(() => {});
try {
const resp = await fetch('/api/configs/' + configUUID);
@@ -1033,6 +982,7 @@ document.addEventListener('DOMContentLoaded', async function() {
const config = await resp.json();
configName = config.name;
configType = config.config_type || 'server';
await settingsPromise;
applyConfigTypeToTabs();
currentVersionNo = config.current_version_no || 1;
projectUUID = config.project_uuid || '';
@@ -1388,6 +1338,26 @@ function getTabForCategory(category) {
return 'other';
}
// Tab buttons mirror TAB_CONFIG (qt_settings.tab_config order, "other" last), so
// tabs added, removed or relabelled on the server need no client release.
function renderTabButtons() {
const nav = document.getElementById('category-tabs');
if (!nav) return;
const keys = Object.keys(TAB_CONFIG).filter(k => k !== 'other');
keys.push('other');
if (!TAB_CONFIG[currentTab]) currentTab = firstTabKey();
nav.replaceChildren(...keys.map(key => {
const btn = document.createElement('button');
btn.type = 'button';
btn.dataset.tab = key;
btn.className = 'tab-btn px-4 py-3 text-sm font-medium border-b-2 whitespace-nowrap hover:text-gray-700 ' +
(key === currentTab ? 'border-blue-600 text-blue-600' : 'border-transparent text-gray-500');
btn.textContent = TAB_CONFIG[key]?.label || key;
btn.addEventListener('click', () => switchTab(key));
return btn;
}));
}
function switchTab(tab) {
currentTab = tab;
document.querySelectorAll('.tab-btn').forEach(btn => {
@@ -1403,35 +1373,15 @@ function switchTab(tab) {
renderTab();
}
// Hardcoded fallback constants — used only when server has not provided config_types data
const ALWAYS_VISIBLE_TABS = new Set(['base', 'storage', 'pci']);
const STORAGE_ONLY_BASE_CATEGORIES = ['DKC', 'CTL', 'ENC'];
const SERVER_ONLY_BASE_CATEGORIES = ['MB', 'CPU', 'MEM'];
const STORAGE_HIDDEN_STORAGE_CATEGORIES = ['RAID'];
const STORAGE_HIDDEN_PCI_CATEGORIES = ['GPU', 'DPU'];
const STORAGE_HIDDEN_POWER_CATEGORIES = ['PS', 'PSU'];
// A config type without an allowlist in config_types sees every category.
function isCategoryVisibleForConfigType(code, cfgType) {
const allowed = configTypeCategoryMap[cfgType];
if (!allowed || allowed.size === 0) return _hardcodedCategoryVisible(code, cfgType);
if (!allowed || allowed.size === 0) return true;
return allowed.has(code.toUpperCase());
}
function _hardcodedCategoryVisible(code, cfgType) {
if (cfgType === 'storage') {
if (STORAGE_ONLY_BASE_CATEGORIES.includes(code)) return true;
if (SERVER_ONLY_BASE_CATEGORIES.includes(code)) return false;
if (STORAGE_HIDDEN_STORAGE_CATEGORIES.includes(code)) return false;
if (STORAGE_HIDDEN_PCI_CATEGORIES.includes(code)) return false;
if (STORAGE_HIDDEN_POWER_CATEGORIES.includes(code)) return false;
} else {
if (STORAGE_ONLY_BASE_CATEGORIES.includes(code)) return false;
}
return true;
}
function _effectiveAlwaysVisibleTabs() {
return alwaysVisibleTabsSet || ALWAYS_VISIBLE_TABS;
function firstTabKey() {
return Object.keys(TAB_CONFIG).find(k => k !== 'other') || 'other';
}
function applyConfigTypeToTabs() {
@@ -1466,7 +1416,7 @@ function applyConfigTypeToTabs() {
}
function updateTabVisibility() {
const visibleTabs = _effectiveAlwaysVisibleTabs();
const visibleTabs = alwaysVisibleTabsSet;
for (const tabId of Object.keys(TAB_CONFIG)) {
if (visibleTabs.has(tabId)) continue;
const btn = document.querySelector(`[data-tab="${tabId}"]`);
@@ -1477,15 +1427,16 @@ function updateTabVisibility() {
});
const visible = hasComponents || hasCartItems;
btn.classList.toggle('hidden', !visible);
// If the current tab just got hidden, fall back to base
// If the current tab just got hidden, fall back to the first tab
if (!visible && currentTab === tabId) {
switchTab('base');
switchTab(firstTabKey());
}
}
}
function getComponentsForTab(tab) {
const config = TAB_CONFIG[tab];
if (!config) return [];
return allComponents.filter(comp => {
const category = getComponentCategory(comp);
if (tab === 'other') {
@@ -1503,6 +1454,7 @@ function getComponentsForCategory(category) {
function renderTab() {
const config = TAB_CONFIG[currentTab];
if (!config) return; // tab_config not loaded yet
const components = getComponentsForTab(currentTab);
if (config.singleSelect) {
@@ -1583,10 +1535,10 @@ function renderSingleSelectTab(categories) {
<div class="autocomplete-wrapper relative">
<input type="text"
id="input-${cat}"
value="${selectedItem?.lot_name || ''}"
value="${escapeHtml(selectedItem?.lot_name || '')}"
placeholder="Введите артикул..."
class="w-full px-2 py-1 border rounded text-sm font-mono"
onfocus="showAutocomplete('${cat}', this)"
autocomplete="off" onfocus="showAutocomplete('${cat}', this)"
oninput="filterAutocomplete('${cat}', this.value)"
onkeydown="handleAutocompleteKey(event, '${cat}')">
</div>
@@ -1652,7 +1604,7 @@ function renderMultiSelectTab(components) {
<input type="text"
value="${escapeHtml(item.lot_name)}"
class="w-full px-2 py-1 border rounded text-sm font-mono"
onfocus="showAutocompleteEditItem('${item.lot_name}', this)"
autocomplete="off" data-lot="${escapeHtml(item.lot_name)}" data-cart-action="edit"
oninput="filterAutocompleteEditItem(this.value)"
onkeydown="handleAutocompleteKeyEditItem(event)">
</div>
@@ -1661,12 +1613,12 @@ function renderMultiSelectTab(components) {
<td class="px-3 py-2 text-sm text-right">${formatPriceOrNA(item.estimate_price ?? item.unit_price)}</td>
<td class="px-3 py-2 text-center">
<input type="number" min="1" value="${item.quantity}"
onchange="updateMultiQuantity('${item.lot_name}', this.value)"
data-lot="${escapeHtml(item.lot_name)}" data-cart-action="quantity"
class="w-16 px-2 py-1 border rounded text-center text-sm">
</td>
<td class="px-3 py-2 text-sm text-right font-medium">${formatMoney(total)}</td>
<td class="px-3 py-2 text-center">
<button onclick="removeFromCart('${item.lot_name}')" class="text-red-500 hover:text-red-700">
<button data-lot="${escapeHtml(item.lot_name)}" data-cart-action="remove" class="text-red-500 hover:text-red-700">
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
</svg>
@@ -1685,7 +1637,7 @@ function renderMultiSelectTab(components) {
id="input-new-item"
placeholder="Добавить компонент..."
class="w-full px-2 py-1 border rounded text-sm"
onfocus="showAutocompleteMulti(this)"
autocomplete="off" onfocus="showAutocompleteMulti(this)"
oninput="filterAutocompleteMulti(this.value)"
onkeydown="handleAutocompleteKeyMulti(event)">
</div>
@@ -1764,7 +1716,7 @@ function renderMultiSelectTabWithSections(sections) {
value="${escapeHtml(item.lot_name)}"
data-categories="${categoriesStr}"
class="w-full px-2 py-1 border rounded text-sm font-mono"
onfocus="showAutocompleteEditItem('${item.lot_name}', this)"
autocomplete="off" data-lot="${escapeHtml(item.lot_name)}" data-cart-action="edit"
oninput="filterAutocompleteEditItem(this.value)"
onkeydown="handleAutocompleteKeyEditItem(event)">
</div>
@@ -1773,12 +1725,12 @@ function renderMultiSelectTabWithSections(sections) {
<td class="px-3 py-2 text-sm text-right">${formatPriceOrNA(item.estimate_price ?? item.unit_price)}</td>
<td class="px-3 py-2 text-center">
<input type="number" min="1" value="${item.quantity}"
onchange="updateMultiQuantity('${item.lot_name}', this.value)"
data-lot="${escapeHtml(item.lot_name)}" data-cart-action="quantity"
class="w-16 px-2 py-1 border rounded text-center text-sm">
</td>
<td class="px-3 py-2 text-sm text-right font-medium">${formatMoney(total)}</td>
<td class="px-3 py-2 text-center">
<button onclick="removeFromCart('${item.lot_name}')" class="text-red-500 hover:text-red-700">
<button data-lot="${escapeHtml(item.lot_name)}" data-cart-action="remove" class="text-red-500 hover:text-red-700">
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
</svg>
@@ -1798,7 +1750,7 @@ function renderMultiSelectTabWithSections(sections) {
data-categories="${categoriesStr}"
placeholder="Добавить ${section.title.toLowerCase()}..."
class="w-full px-2 py-1 border rounded text-sm"
onfocus="showAutocompleteSection('${sectionId}', this)"
autocomplete="off" onfocus="showAutocompleteSection('${sectionId}', this)"
oninput="filterAutocompleteSection('${sectionId}', this.value, this)"
onkeydown="handleAutocompleteKeySection(event, '${sectionId}')">
</div>
@@ -2360,7 +2312,7 @@ function updateCartUI() {
</div>
<div class="flex items-center space-x-2">
<span>${formatMoney(itemTotal)}</span>
<button onclick="removeFromCart('${item.lot_name}')" class="text-red-500 hover:text-red-700">
<button data-lot="${escapeHtml(item.lot_name)}" data-cart-action="remove" class="text-red-500 hover:text-red-700">
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
</svg>
@@ -2425,19 +2377,22 @@ function findSupportCartIndex() {
return cart.findIndex(i => (i.lot_name || '').toUpperCase().startsWith('SVC_'));
}
// Mirrors the GPU chip-generation classification used by internal/services/rental.go,
// so the support picker and the rental calc agree on which platform a config is.
// HGX = SXM baseboard servers. Plain PCIe GPUs (A30, L40, H200 NVL PCIe, ...) in a regular
// server keep the x86 percent-of-price support model.
function detectSupportPlatform() {
const actualGenSubstrings = ['H200', 'B200', 'B300', 'BLACKWELL SE'];
let sawGPU = false;
let platform = 'x86';
for (const item of cart) {
if ((item.category || '').toUpperCase() !== 'GPU') continue;
sawGPU = true;
const upperLot = (item.lot_name || '').toUpperCase();
const isPCIe = upperLot.includes('PCIE');
const isHGX = upperLot.includes('HGX') || upperLot.includes('SXM') ||
(!isPCIe && /(H100|H200|B200|B300)/.test(upperLot));
if (!isHGX) continue;
if (upperLot.includes('B300')) return 'HGX-B300';
if (upperLot.includes('B200')) return 'HGX-B200';
if (upperLot.includes('B200')) platform = 'HGX-B200';
else if (platform === 'x86') platform = 'HGX-H200';
}
return sawGPU ? 'HGX-H200' : 'x86';
return platform;
}
// cartTotal excludes any existing support line so the x86 percent is based on hardware only.
@@ -3467,7 +3422,7 @@ async function renderRentalTab() {
return `<tr class="pricing-row-buy">
<td class="px-3 py-1.5 text-center text-xs border-t border-gray-200">
<input type="checkbox" ${isUsed ? 'checked' : ''}
onchange="onRentalConditionChange('${escapeHtml(item.lot_name).replace(/'/g, "\\'")}', this.checked)"
data-lot="${escapeHtml(item.lot_name)}" data-cart-action="condition"
class="rounded border-gray-300">
</td>
<td class="px-3 py-1.5 text-xs border-t border-gray-200">${escapeHtml(item.lot_name)}</td>
@@ -3804,7 +3759,7 @@ function _bomRawLotCell(rowIdx) {
return `<div class="autocomplete-wrapper relative"><input type="text" placeholder="Введите артикул..."
value="${escapeHtml(val)}"
class="w-full min-w-28 px-2 py-1 border rounded text-xs font-mono"
onfocus="showAutocompleteBOM(${rowIdx}, this)"
autocomplete="off" onfocus="showAutocompleteBOM(${rowIdx}, this)"
oninput="filterAutocompleteBOM(${rowIdx}, this.value); setBOMManualLotDraft(${rowIdx}, this.value, this)"
onkeydown="handleAutocompleteKeyBOM(event, ${rowIdx})"></div>
${renderBOMLotAllocationsEditor(rowIdx)}`;
@@ -4204,14 +4159,14 @@ function _renderBOMParsedTable() {
lotCell = `<div class="autocomplete-wrapper relative"><input type="text" placeholder="Введите артикул..."
value="${escapeHtml(row.manual_lot || '')}"
class="w-full px-2 py-1 border rounded text-sm font-mono focus:ring-1 focus:ring-blue-400"
onfocus="showAutocompleteBOM(${row.source_row_index}, this)"
autocomplete="off" onfocus="showAutocompleteBOM(${row.source_row_index}, this)"
oninput="filterAutocompleteBOM(${row.source_row_index}, this.value); bomRows.find(r=>r.source_row_index===${row.source_row_index}).manual_lot=this.value;"
onkeydown="handleAutocompleteKeyBOM(event, ${row.source_row_index})"></div>${renderBOMLotAllocationsEditor(idx)}`;
} else {
let suffix = '';
if (qtyMismatch) suffix = ` <span class="text-yellow-600 text-xs">≠est(${cartQty})</span>`;
else if (notInCart) suffix = ` <span class="text-orange-500 text-xs">новый</span>`;
lotCell = `<div><span class="font-mono text-xs">${row.resolved_lot}</span>${suffix}</div>${renderBOMLotAllocationsEditor(idx)}`;
lotCell = `<div><span class="font-mono text-xs">${escapeHtml(row.resolved_lot)}</span>${suffix}</div>${renderBOMLotAllocationsEditor(idx)}`;
}
tr.innerHTML = `
@@ -4788,10 +4743,10 @@ async function renderPricingTab() {
${pnDescHtml}
<td class="px-2 py-0.5 text-xs ${borderTop}">${r.lotCell}</td>
<td class="px-2 py-0.5 text-right text-xs ${borderTop}">${r.qty}</td>
<td class="px-2 py-0.5 text-right text-xs ${borderTop} ${worldCls(r.estWorld)}">${r.estUnit > 0 ? formatCurrency(r.estUnit) : '—'}${worldMark(r.estWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs stock-price-col ${borderTop} ${worldCls(r.whWorld)}">${r.warehouseUnit != null ? formatCurrency(r.warehouseUnit) : '—'}${worldMark(r.whWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs ${borderTop} ${worldCls(r.compWorld)}">${r.competitorUnit != null ? formatCurrency(r.competitorUnit) : '—'}${worldMark(r.compWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs pricing-vendor-price-buy ${borderTop} ${r.vendorOrigUnit == null ? 'text-gray-400' : ''}">${r.vendorOrigUnit != null ? formatCurrency(r.vendorOrigUnit) : '—'}</td>
<td class="px-2 py-0.5 text-right text-xs ${borderTop} ${worldCls(r.estWorld)}">${r.est > 0 ? formatCurrency(r.est) : '—'}${worldMark(r.estWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs stock-price-col ${borderTop} ${worldCls(r.whWorld)}">${r.warehouse != null ? formatCurrency(r.warehouse) : '—'}${worldMark(r.whWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs ${borderTop} ${worldCls(r.compWorld)}">${r.competitor != null ? formatCurrency(r.competitor) : '—'}${worldMark(r.compWorld)}</td>
<td class="px-2 py-0.5 text-right text-xs pricing-vendor-price-buy ${borderTop} ${r.vendorOrig == null ? 'text-gray-400' : ''}">${r.vendorOrig != null ? formatCurrency(r.vendorOrig) : '—'}</td>
`;
tbodyBuy.appendChild(tr);
});
@@ -5036,7 +4991,6 @@ function applyCustomPrice(table) {
let assigned = 0;
rows.forEach((tr, i) => {
const rowEst = parseFloat(tr.dataset.est) || 0;
const qty = Math.max(1, parseFloat(tr.dataset.qty) || 1);
const cell = vendorCells[i];
if (!cell) return;
let share;
@@ -5046,7 +5000,7 @@ function applyCustomPrice(table) {
share = Math.round((rowEst / estimateTotal) * customPrice * 100) / 100;
assigned += share;
}
cell.textContent = formatCurrency(share / qty);
cell.textContent = formatCurrency(share);
cell.className = cell.className.replace(/\btext-(?:gray|green|red|blue)-\d+\b/g, '').trim();
cell.classList.add(rowEst > 0 ? _pctClass(share, rowEst) : 'text-blue-700');
});
@@ -5060,8 +5014,8 @@ function applyCustomPrice(table) {
const cell = vendorCells[i];
if (!cell) return;
cell.className = cell.className.replace(/\btext-(?:gray|green|red|blue)-\d+\b/g, '').trim();
if (tr.dataset.vendorOrigUnit !== '') {
cell.textContent = formatCurrency(parseFloat(tr.dataset.vendorOrigUnit));
if (tr.dataset.vendorOrig !== '') {
cell.textContent = formatCurrency(parseFloat(tr.dataset.vendorOrig));
} else {
cell.textContent = '—';
cell.classList.add('text-gray-400');
@@ -5099,10 +5053,9 @@ function setPricingCustomPriceFromVendor() {
rows.forEach((tr, i) => {
const cell = vendorCells[i];
if (!cell) return;
const origUnit = tr.dataset.vendorOrigUnit;
const origTotal = tr.dataset.vendorOrig;
if (origUnit !== '') {
cell.textContent = formatCurrency(parseFloat(origUnit));
if (origTotal !== '') {
cell.textContent = formatCurrency(parseFloat(origTotal));
cell.className = cell.className.replace(/\btext-(?:gray|green|red|blue)-\d+\b/g, '').trim();
total += parseFloat(origTotal) || 0;
hasAny = true;
@@ -5173,10 +5126,6 @@ async function exportPricingCSV(table) {
}
}
function escapeHtml(str) {
if (!str) return '';
return str.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
}
function formatCurrency(val) {
if (val == null) return '—';
-9
View File
@@ -159,15 +159,6 @@
return 5;
}
function escapeHtml(text) {
if (text === null || text === undefined) return '';
return String(text)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#039;');
}
function renderItems(items) {
if (items.length === 0) {
Loaded 100 of 103 files, more files were not shown because too many files have changed in this diff. Show more