Compare commits

...
Author SHA1 Message Date
Mikhail Chusavitin 8569871a17 fix: verify server snapshots and recover whole migration sessions (#3) 2026-10-26 22:48:07 +03:00
Mikhail Chusavitin 3d4101ecc0 test: enforce release failure gates and record platform builds (#6) 2026-10-26 21:22:34 +03:00
Mikhail Chusavitin 46d7ccda4d test: verify TLS policies against disposable MariaDB servers (#5) 2026-10-26 21:10:24 +03:00
Mikhail Chusavitin 2d60fcfb39 test: verify reset CLI and restored startup preserve local work (#7) 2026-10-26 21:07:41 +03:00
Mikhail Chusavitin ffdbe4fbd0 test: exercise XSS regressions in headless Chrome (#4) 2026-10-26 21:03:36 +03:00
Mikhail Chusavitin e8dd03c4f5 test: verify browser trust boundary on both application routers (#2) 2026-10-26 20:59:35 +03:00
Mikhail ChusavitinandClaude Opus 5.5 fa09513e26 docs: release notes v3.51
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:52:03 +03:00
Mikhail ChusavitinandClaude Opus 5.5 44a09d4420 fix(sync): stop MariaDB connection leak and cut per-cycle sync waste
- ConnectionManager reuses a stale-but-healthy pool instead of dialing a new
  one every 30s, and closes every pool it replaces (one client held ~30
  connections). Pool capped at 2 connections, idle released after 1 min.
- Background sync every 1 min with ±30% jitter, scheduled from cycle start;
  users-status online threshold = interval × 1.3. One heartbeat per cycle.
- Partnumber books: read headers only, fetch partnumbers_json per book when
  new/incomplete; empty book counts as complete; an incomplete book is
  re-pulled once per process (server book lists BOT_AGIBOT_G2_PERF missing
  from qt_partnumber_book_items).
- NeedSync in one UNION ALL query; SyncQtSettings skips unchanged rewrite.
- Migration 035: server-maintained server_updated_at on qt_configurations and
  qt_projects. Clients pull only changed rows after one full pull per process;
  servers without 035 and old clients keep full pulls.
- Migrator: fresh GORM statement per verify query and history insert; files
  with two verify checks failed to record.

ADL: bible-local/decisions/2026-10-26-mariadb-single-pool.md,
bible-local/decisions/2026-10-26-incremental-config-project-pull.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:46:59 +03:00
Mikhail ChusavitinandCodex 8a08270b79 docs: release notes v3.50
Co-Authored-By: Codex <noreply@openai.com>
2026-10-26 10:58:39 +03:00
mchus cab5d13f16 Merge pull request 'Исправления шести issue: безопасность API, защита данных, TLS и миграции' (#8) from fix/open-issues-security-recovery into main 2026-10-26 10:56:07 +03:00
Mikhail Chusavitin d6c498f0cf docs: record issue implementation plan and validation results 2026-10-26 10:54:22 +03:00
Mikhail Chusavitin aa55dc621f build: require patched Go and gate releases on govulncheck (#6) 2026-10-26 10:53:14 +03:00
Mikhail Chusavitin a59cc7ccb7 fix: verify SQL migrations before recording and preserve sessions (#3) 2026-10-26 10:49:23 +03:00
Mikhail Chusavitin 370407d3ba feat: configure MariaDB TLS for setup and runtime connections (#5) 2026-10-26 10:47:46 +03:00
Mikhail Chusavitin 15f81e053c fix: require verified pre-reset backup and protect unsynced work (#7) 2026-10-26 10:41:25 +03:00
Mikhail Chusavitin 1ada8c12e2 fix: render LOT values and sync errors without executable interpolation (#4) 2026-10-26 10:38:41 +03:00
Mikhail Chusavitin c0c7804494 fix: protect local HTTP boundary and saved setup credentials (#2) 2026-10-26 10:35:37 +03:00
Mikhail ChusavitinandClaude Opus 5.5 134685d482 docs: release notes v3.49
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:31:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 88629a14da fix: «Цена покупки» показывает суммы по строке (цена × кол-во), как «Цена продажи»
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:30:46 +03:00
mchusandClaude Sonnet 5.5 2f30dd0d27 docs: release notes v3.48
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 10:45:54 +03:00
mchusandClaude Sonnet 5.5 d48865869a fix: платформа поддержки HGX только для SXM/HGX GPU; актуализированы цены HGX по регламенту 2026-10-07
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 10:45:51 +03:00
Mikhail ChusavitinandClaude Sonnet 5 8b073e63b1 docs: release notes v3.47
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 11:54:11 +03:00
Mikhail ChusavitinandClaude Sonnet 5 cccb8890a4 fix: requeue orphaned pending configurations; independent background sync streams
- Configurations stuck in sync_status=pending without a pending_changes row are
  re-enqueued as an update with the current snapshot on each push cycle.
- Background cycle no longer aborts pricelist/partbook/settings sync when push or
  readiness fails; each stream logs its own error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 11:53:44 +03:00
Mikhail ChusavitinandClaude Opus 5.5 310b296045 docs: release notes v3.46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 16:34:17 +03:00
Mikhail ChusavitinandClaude Opus 5.5 746e5663fb fix: история автозаполнения браузера перекрывала выпадашку LOT
Поля с собственной выпадашкой в конфигураторе не имели autocomplete="off",
и браузер рисовал поверх неё нативный список ранее введённых значений.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 16:33:54 +03:00
Mikhail ChusavitinandClaude Opus 5.5 ae26db6624 docs: release notes v3.45
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:24:09 +03:00
Mikhail ChusavitinandClaude Opus 5.5 673a207079 feat: вкладки конфигуратора только из qt_settings, без JS-копии
- кнопки вкладок строятся из tab_config (порядок, label); удалённая на
  сервере вкладка (SW) больше не висит статической кнопкой;
- TAB_CONFIG на странице содержит только «Other», остальное — из
  /api/configurator-settings; убраны захардкоженные фильтры категорий по
  типу и ALWAYS_VISIBLE_TABS; тип без allowlist видит все категории;
- настройки грузятся до категорий и до фильтрации по типу конфигурации;
- единственный fallback — Go-дефолты в handlers/component.go;
  always_visible_tabs по умолчанию ["base"].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:23:48 +03:00
Mikhail ChusavitinandClaude Opus 5.5 aefe167590 docs: release notes v3.44
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 17:00:15 +03:00
Mikhail ChusavitinandClaude Opus 5.5 a939a5f36c feat: типы конфигураций из qt_settings в модалке проекта, копия сохраняет config_type
- модалка «Новая конфигурация в проекте» строит кнопки типа из
  qt_settings.config_types (новый тип «Сеть» появляется без релиза);
- ручная полная синхронизация (/api/sync/all) тянет qt_settings;
- копирование конфигурации (в т.ч. из версии) сохраняет config_type —
  раньше копия СХД/сети становилась «сервером».

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 16:59:42 +03:00
Mikhail ChusavitinandClaude Opus 5.5 54a7c7cc8d docs: release notes v3.43
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 14:34:45 +03:00
Mikhail ChusavitinandClaude Opus 5.5 a2a1904d91 feat: make migrate — серверные SQL-миграции вместо AutoMigrate
`qfs -migrate` теперь применяет только migrations/*.sql (встроены в бинарник)
с учётом в qt_schema_migrations и завершается, не запуская приложение.
Новые команды: make migrate-status (read-only список applied/pending),
make migrate, make migrate-baseline UPTO=<file> (пометить как применённые
без выполнения). migrate отказывается работать, если в истории нет ни одной
применённой миграции — иначе все файлы пошли бы заново.

Удалён GORM AutoMigrate против MariaDB (models.Migrate, SeedCategories):
устаревший бинарник v1.18 с -migrate прогнал по боевой базе AutoMigrate
своих моделей. Миграции идут через отдельное соединение без read/write
таймаутов — 3s-таймаут sync-соединения обрывал backfill в 032.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 12:21:53 +03:00
Mikhail ChusavitinandClaude Opus 5.5 259bd48d2e docs: release notes v3.42
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 12:11:48 +03:00
Mikhail ChusavitinandClaude Opus 5.5 4e3b43dbd0 feat: фича-флаги конфигурации — клиент не сохраняет то, чего не понимает
Конфигурация хранит feature_flags (JSON-массив, NULL = нет) — какие данные
она использует. Флаги выводятся из содержимого при каждом сохранении
(internal/configfeatures). Если у конфигурации есть флаг, неизвестный этой
версии, клиент отказывается её изменять: все write-пути отдают 409
unsupported_config_features со списком флагов, конфигуратор показывает
баннер «обновите приложение» и блокирует сохранение. Push на сервер тоже
отказывается, если серверная версия несёт неизвестный флаг; такое изменение
остаётся в очереди до обновления.

Первый флаг: sale_row_markups. Серверная миграция 034 добавляет колонку;
до её применения запись идёт без колонки (с warning). Формат имён и реестр
флагов — bible-local/02-architecture.md, ADL 2026-09-24.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 12:10:58 +03:00
Mikhail ChusavitinandClaude Opus 5.5 6b9652e3ac fix: отрицательная накидка и сохранность накидок в notes
Отрицательная «Накидка, %» отбрасывалась в четырёх местах (ввод, сохранение,
восстановление, отображение) — теперь любое ненулевое значение сохраняется,
отрицательное работает как скидка. Коэффициент (аплифт + накидка/100)
ограничен снизу нулём и в UI, и в CSV-экспорте. Принимается «−» (U+2212).

Накидки пропадали после синхронизации: клиент до v3.41 собирает notes заново
из трёх старых полей и при сохранении стирает sale_row_markups на сервере,
а следующий pull перезаписывает локальную копию. Клиент теперь переносит
незнакомые ключи notes/pricing_ui как есть, чтобы будущие поля не терялись.
ADL исправлен (неверное утверждение «старые клиенты игнорируют ключ»).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 10:08:34 +03:00
Mikhail ChusavitinandClaude Sonnet 5 4501193423 docs: release notes v3.41
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:58:24 +03:00
Mikhail ChusavitinandClaude Sonnet 5 31b991e0f2 fix: длинное имя варианта корёжило кнопки в хедере конфигуратора
Имя варианта не усекалось в хлебных крошках (в отличие от имени
конфигурации), поэтому длинное значение распирало flex-контейнер и
сжимало кнопки «Обновить цены»/«Сохранить». Усекаю до 16 символов
(truncateBreadcrumbSpecName) + shrink-0/flex-wrap на хедере как защита.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:57:01 +03:00
Mikhail ChusavitinandClaude Sonnet 5 2f3bdc609a fix: невалидное имя варианта проекта — предупреждение вместо ошибки
Имя варианта "main" или с недопустимыми символами больше не отклоняет
запрос ошибкой 400 — теперь автоматически конвертируется в валидное
(sanitizeProjectVariantName), а API/UI показывают variant_warning вместо
блокировки создания/переименования варианта.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:56:55 +03:00
Mikhail ChusavitinandClaude Sonnet 5 006d7e87c5 fix: артикул — распознавание DPU/BlueField профиля (BF3/BF2)
DPU-лоты (DPU_BF3_B3220-32G) не подходили под regex parsePortSpeed
(рассчитан на NIC/HCA-суффиксы NpMGbE/pFC) и падали в заглушку
категории NET с предупреждением. Добавлен parseDPUModel — структурный
разбор DPU_{FAMILY}_{MODEL} без каталога вендоров.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:56:42 +03:00
Mikhail ChusavitinandClaude Sonnet 5 37def17c5c fix: накидка по строке в «Цена продажи» не пересчитывала цену
Инпут накидки не получал data-row-key, поэтому onSaleRowMarkupInput писал
значение в saleRowMarkups[undefined], а recomputeSalePrices() читал по
настоящему ключу строки — значение никогда не применялось. Цена теперь
пересчитывается мгновенно при вводе, без перезагрузки.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 18:42:29 +03:00
Mikhail ChusavitinandClaude Sonnet 5 497aee1e71 style: минимальная высота строк в таблицах вкладки «Ценообразование»
Заголовки/футеры py-2 -> py-1, тело обеих таблиц (Buy/Sale) py-1.5 -> py-0.5.
Элементы управления под таблицами (инпуты, кнопки) не тронуты.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 16:41:51 +03:00
Mikhail ChusavitinandClaude Sonnet 5 7263dd4572 feat: накидка по строке в таблице «Цена продажи» + минимальный CSV
Sale-таблица показывает LOT/Описание/Кол-во/Накидка,%/Цена вместо Estimate/
Склад/Конкуренты/Ручная цена; итоговая цена строки = база (raw estimate или
доля от общей «Ручная цена») × (Аплифт к estimate + своя Накидка%). Экспорт
CSV этой таблицы теперь выводит только LOT;Описание;Кол-во;Цена. Buy-таблица
и массовый экспорт по проекту не затронуты — новые поля запроса опциональны
и включаются только кнопкой «Экспорт CSV» у Sale-таблицы.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 16:39:54 +03:00
Mikhail ChusavitinandClaude Sonnet 5 fb412a4227 docs: release notes v3.40
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAhfF4P1ySRZ67yyUUeVw2
2026-09-01 09:02:36 +03:00
Mikhail ChusavitinandClaude Sonnet 5 d7d4ea74b6 fix: артикул — категории из component universe + видимость нераспознанных токенов
Генерация артикула резолвила lot_category из одного прайслиста конфигурации
(GetLocalLotCategoriesByServerPricelistID), поэтому world-only LOT (напр.
GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE, есть только в world) молча выпадал из
артикула. Теперь категории берутся через GetLocalComponentCategoriesByLotNames
(тот же world ∪ estimate, что и весь конфигуратор). BuildOptions.ServerPricelist
убран; preview-article принимает pricelist_id, но игнорирует.

Нераспознанные токены больше не пишутся как UNK: в артикул идёт lot_category как
плейсхолдер (4xGPU, 2xCPU), сегмент помечается Recognized=false, добавляется
warning с именем LOT. Конфигуратор подсвечивает такие сегменты (amber) и выводит
список предупреждений; сохранение/обновление/откат логируют WARN. Без каталога
вендоров в репо детектируется только структурный сбой формы имени.

parseGPUModel: принимает суффикс-букву в номере модели (6000D → RTX6000D),
раньше терял её и схлопывал до RTX_84GB.

ADL bible-local/decisions/2026-09-01-article-category-from-component-universe.md,
2026-09-01-article-degraded-token-visibility.md; раздел «Article generation» в
02-architecture.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAhfF4P1ySRZ67yyUUeVw2
2026-09-01 09:01:56 +03:00
Mikhail ChusavitinandClaude Sonnet 5 ecef030699 docs: release notes v3.39
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-31 18:20:29 +03:00
Mikhail ChusavitinandClaude Sonnet 5 681ec15e2b feat: режим индикаторов без цветовой кодировки + переустройство /setup
Новая настройка app_settings.indicator_mode (color | accessible), переключается
на /setup. В режиме accessible сигналы, которые раньше держались только на цвете,
переходят на форму/текст:
- качество цены (0-9) — 5-ступенчатый signal-meter вместо градиентной точки/числа;
  единый модуль web/static/price-quality.js, ветвление по window.QF_INDICATOR_MODE;
- вкладка «Ценообразование»: ведущая колонка-meter вместо заливки строк,
  пометка W вместо амбер-подсветки world-цен, ⚠ вместо красного «загрязнённого» итога.
- GET/PUT /api/settings/ui (без рестарта), регистрируется в обоих наборах роутов.

/setup переустроен: две колонки одной высоты, кнопка «Вернуться в приложение»,
исправлен <title>.

Документация: bible-local 02/03/04 + decisions/2026-08-31-accessible-indicator-mode.md

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbRvQgPZpM4SJTaX3iLrXk
2026-08-31 18:08:38 +03:00
mchusandClaude Sonnet 5 d97ac447ca chore: обновить субмодуль bible
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 15:03:18 +03:00
mchusandClaude Sonnet 5 a25df8a90f fix: показывать версию приложения только в шапке рядом с логотипом
Убрано дублирование "vv" (Version() уже включает "v") и хардкод
"v1.0" на странице настройки БД.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-25 15:01:06 +03:00
Mikhail ChusavitinandClaude Sonnet 5 3329ff12f3 fix: убрать дублирующее объявление escapeHtml
Две одноимённые функции в одном файле — только вторая (с явным
экранированием кавычек) реально выполнялась, первая была мёртвым
кодом и сбивала с толку при чтении.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 18:39:05 +03:00
Mikhail ChusavitinandClaude Sonnet 5 414c8a7de2 docs: release notes v3.37
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 18:36:53 +03:00
Mikhail Chusavitin 639fe98945 fix: количество исчезало при вводе в мультиселект-таблицах конфигуратора
Партиальный DOM-патч в updateMultiQuantity() писал сумму в ячейку
«Кол-во» (td:nth-child(5)) вместо «Стоимость» (td:nth-child(6)),
затирая input с количеством. Заодно ужесточили поиск строки: вместо
нестрогого querySelector по подстроке onchange-атрибута — точный
поиск по data-lot-name.
2026-08-24 18:36:14 +03:00
Mikhail ChusavitinandClaude Sonnet 5 1898a4cb09 docs: release notes v3.36
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-12 17:34:15 +03:00
Mikhail ChusavitinandClaude Sonnet 5 e201f36fe7 fix: убрать лишнее пояснение из подписи DDP в экспорте проекта
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-12 17:31:06 +03:00
Mikhail ChusavitinandClaude Sonnet 5 c83e4ec9f9 fix: DDP-экспорт с проекта игнорировал свой аплифт каждой конфигурации
ProjectPricingExportOptions.SaleMarkup всегда был 0 при экспорте с
проекта, поэтому применялся единый дефолт 1,3 вместо сохранённого в
Notes конфигурации значения. buildPricingExportBlock теперь берёт
аплифт из настроек каждой конфигурации — экспорт с проекта идёт по
тому же конвейеру, что и экспорт с конкретной конфигурации.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-12 17:29:25 +03:00
143 changed files with 7967 additions and 1137 deletions

No files matched your search

+27 -5
View File
@@ -1,4 +1,4 @@
.PHONY: build build-release clean test run version install-hooks
.PHONY: build build-release clean test run version install-hooks migrate migrate-status migrate-baseline security-check
# Get version from git
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
@@ -13,20 +13,20 @@ build:
go build -o bin/$(BINARY) ./cmd/qfs
# Build for release (optimized, with version)
build-release:
build-release: security-check
@echo "Building $(BINARY) version $(VERSION)..."
CGO_ENABLED=0 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY) ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)"
@./bin/$(BINARY) -version
# Build release for Linux (cross-compile)
build-linux:
build-linux: security-check
@echo "Building $(BINARY) for Linux..."
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-linux-amd64 ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)-linux-amd64"
# Build release for macOS (cross-compile)
build-macos:
build-macos: security-check
@echo "Building $(BINARY) for macOS..."
CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-darwin-amd64 ./cmd/qfs
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-darwin-arm64 ./cmd/qfs
@@ -34,7 +34,7 @@ build-macos:
@echo "✓ Built: bin/$(BINARY)-darwin-arm64"
# Build release for Windows (cross-compile)
build-windows:
build-windows: security-check
@echo "Building $(BINARY) for Windows..."
CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -ldflags="$(LDFLAGS)" -o bin/$(BINARY)-windows-amd64.exe ./cmd/qfs
@echo "✓ Built: bin/$(BINARY)-windows-amd64.exe"
@@ -55,10 +55,32 @@ clean:
rm -rf bin/
rm -f $(BINARY)
# Server (MariaDB) migrations — migrations/*.sql tracked in qt_schema_migrations.
# Always built from the current source (go run), never a possibly stale bin/ binary.
# Uses the connection saved in the local app settings.
# List migrations with applied/pending state (read-only)
migrate-status:
go run ./cmd/qfs -migrate-status
# Apply pending migrations
migrate:
go run ./cmd/qfs -migrate
# Record migrations up to UPTO as applied WITHOUT running them (one-time, for a server
# whose schema is already current): make migrate-baseline UPTO=033_add_price_quality_to_pricelist_items.sql
migrate-baseline:
@test -n "$(UPTO)" || (echo "usage: make migrate-baseline UPTO=<file.sql>"; exit 1)
go run ./cmd/qfs -migrate-baseline "$(UPTO)"
# Run tests
test:
go test -v ./...
# Pin the scanner so releases do not depend on a locally installed executable.
security-check:
go run golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...
# Run development server
run:
go run ./cmd/qfs
+2 -1
View File
@@ -24,7 +24,8 @@ go run ./cmd/qfs
Useful commands:
```bash
go run ./cmd/qfs -migrate
make migrate-status
make migrate
go test ./...
go vet ./...
make build-release
+11
View File
@@ -19,3 +19,14 @@ var StaticFiles embed.FS
func StaticFS() (fs.FS, error) {
return fs.Sub(StaticFiles, "web/static")
}
// MigrationFiles contains the MariaDB server migrations (migrations/*.sql), so
// `qfs -migrate` works from any directory. Applied via models.RunSQLMigrations.
//
//go:embed migrations/*.sql
var MigrationFiles embed.FS
// MigrationsFS returns a filesystem rooted at migrations/.
func MigrationsFS() (fs.FS, error) {
return fs.Sub(MigrationFiles, "migrations")
}
+16
View File
@@ -0,0 +1,16 @@
package quoteforge
import (
"io/fs"
"testing"
)
func TestMigrationsEmbedded(t *testing.T) {
migrations, err := MigrationsFS()
if err != nil {
t.Fatal(err)
}
if _, err := fs.Stat(migrations, "034_add_feature_flags_to_configurations.sql"); err != nil {
t.Fatalf("migration 034 not embedded: %v", err)
}
}
+1 -1
Submodule bible updated: 52444350c1...1977730d93.
+11
View File
@@ -68,3 +68,14 @@ bible/ shared engineering rules
bible-local/ project-specific architecture
releases/ release artifacts and notes
```
## Local HTTP trust boundary
Loopback binding does not make a browser client trusted. Both setup and normal
routers reject non-loopback Host, Origin and Referer values, opaque origins and
cross-site Fetch Metadata requests before any handler runs. CLI clients may omit
browser headers. API requests carrying a body require application/json, except
the project vendor-import endpoint, which also accepts multipart/form-data.
Setup retains HTML forms; an empty password reuses the saved credential only
when host, port and user are unchanged. Changing the destination requires an
explicit password.
+137 -5
View File
@@ -30,6 +30,8 @@ Pull-only:
- pricelists and pricelist items;
- partnumber books and partnumber book items.
Configuration feature flags — see [Configuration feature flags](#configuration-feature-flags).
Readiness guard:
- every sync push/pull runs a preflight check;
- blocked sync returns `423 Locked` with a machine-readable reason;
@@ -62,13 +64,52 @@ Rules:
## Pricing tab layout
The Pricing tab (Ценообразование) has two tables: Buy (Цена покупки) and Sale (Цена продажи).
Their column sets differ (see below); the shared per-LOT row expansion/grouping rules apply to both.
Column order (both tables):
Buy table column order:
```
PN вендора | Описание | LOT | Кол-во | Estimate | Склад | Конкуренты | Ручная цена
```
Buy table money cells (Estimate / Склад / Конкуренты / Ручная цена) show the row's line
total (unit × qty), the same as the Sale table's "Цена" — never a unit price; the footer
"Итого" is the sum of those cells. A "Ручная цена" share shown in a row is that row's share
of the manual total, not divided by qty.
Sale table column order:
```
PN вендора | Описание | LOT | Кол-во | Накидка, % | Цена
```
Sale per-row pricing (`_saleRowPrice`/`recomputeSalePrices` in `index.html`,
`applySaleRowPricing` in `internal/services/export.go`):
- each row's base is its raw Estimate unit price × qty, unless the table-wide "Ручная
цена" input is set — then the base is that row's proportional share of the manual
total (same distribution as before this existed, last row absorbs the rounding
remainder), overwriting the row's own base rather than filling a separate column;
- `Цена = base × (Аплифт к estimate + row's own Накидка% / 100)` — Накидка is a
per-row percentage, additive to the table-wide uplift, entered directly in the
row's own input; unset rows default to 0% (no change to older configs' totals);
negative values are discounts, and the factor is floored at 0 so no price goes
negative (`_saleRowPrice` / `applySaleRowPricing`);
Склад/Конкуренты are not shown or exported for the Sale table;
- per-row Накидка is keyed by `"<vendorPN>::<lot>"` (uppercased, "NONE" when there's
no vendor PN) — plain `lot_name` isn't a safe key since the same LOT can appear
under more than one vendor-PN group (see row expansion rules below) — and persisted
as an additive key, `Notes.pricing_ui.sale_row_markups`, alongside the existing
`sale_uplift`/`sale_custom_price`; unknown `Notes` / `pricing_ui` keys are carried
through verbatim on save (`loadedNotesExtra` / `loadedPricingUIExtra`) so a client
never erases fields added by a newer version (see
[decisions/2026-09-15-sale-row-markup-and-minimal-csv.md](decisions/2026-09-15-sale-row-markup-and-minimal-csv.md));
- the Sale table's own "Экспорт CSV" button additionally sends
`minimal_sale_columns: true` + `sale_row_markups`, which collapses
`POST /api/configs/:uuid/export/pricing` output to exactly `LOT;Описание;Кол-во;Цена`
(`ProjectPricingExportOptions.MinimalSaleColumns` in `internal/services/export.go`).
This is scoped to that one request: Buy export and the project-level bulk export
never set it and are unaffected.
Per-LOT row expansion rules:
- each `lot_mappings` entry in a BOM row becomes its own table row with its own quantity and prices;
- `baseLot` (resolved LOT without an explicit mapping) is treated as the first sub-row with `quantity_per_pn` from `_getRowLotQtyPerPN`;
@@ -85,6 +126,50 @@ Controls terminology:
CSV export reads PN вендора, Описание, and LOT from `data-vendor-pn`, `data-desc`, `data-lot` row attributes to bypass the rowspan cell offset problem.
In the colour-optional indicator mode (`app_settings.indicator_mode = "accessible"`, see
[03-database.md](03-database.md) and [decisions/2026-08-31-accessible-indicator-mode.md](decisions/2026-08-31-accessible-indicator-mode.md))
both tables gain a narrow leading price-quality meter column (per sub-row), the row background
tint is removed, `world`-fallback cells show a `W` text marker instead of amber, and the footer
total shows a `⚠` prefix instead of red. CSV output is unaffected.
## Configuration feature flags
A configuration records which data features it uses in `feature_flags`, so a client
that does not know one of them refuses to write it instead of silently dropping data.
Rationale: [decisions/2026-09-24-configuration-feature-flags.md](decisions/2026-09-24-configuration-feature-flags.md).
Storage:
- `qt_configurations.feature_flags` / `local_configurations.feature_flags` — JSON array
of strings, sorted, no duplicates; empty list is stored as `NULL`
(e.g. `["sale_row_markups"]`);
- carried in version snapshots; exposed on reads as `feature_flags`, plus
`unsupported_features` when this build doesn't know some of them.
Flag name format:
- `lower_snake_case`, ASCII `[a-z0-9_]`, starts with a letter, at most 64 chars;
- names the **data** the configuration holds, not the UI or the release:
`<area>_<data>` — e.g. `sale_row_markups`, not `new_pricing_tab` or `v3_42`;
- a name is permanent: never renamed, never reused for different data, never removed
from `configfeatures.known` while any configuration can still carry it;
- a change in the *meaning* of existing data that old clients would mis-handle gets a
new flag (e.g. `sale_row_markups_v2`), not a silent reinterpretation of the old one.
Lifecycle:
- flags are derived from content on every local write (`configfeatures.Detect`), never
sent by the UI; a flag disappears when its data does;
- a client refuses any write (local save or sync push) to a configuration carrying a
flag missing from its `configfeatures.known`, returns `409 unsupported_config_features`,
and the configurator locks itself with a banner listing the raw flag names;
- a new data feature that older clients would drop on save must add its flag — to
`known`, to `Detect`, and to the registry below — in the same change. Data that old
clients round-trip verbatim needs no flag.
Registry:
| Flag | Set when | Data | Since |
| --- | --- | --- | --- |
| `sale_row_markups` | `Notes.pricing_ui.sale_row_markups` has at least one non-zero value | per-row «Накидка, %» on the Sale table (negative = discount) | flag introduced 2026-09-24; data since v3.41 |
## Configuration versioning
Configuration revisions are append-only snapshots stored in `local_configuration_versions`.
@@ -123,12 +208,17 @@ Configurations have a `config_type` field: `"server"` (default) or `"storage"`.
Rules:
- `config_type` defaults to `"server"` for all existing and new configurations unless explicitly set;
- `config_type` is a free-form code (no enum on client or server); every write path — update, clone,
clone-from-version, rollback, sync push/pull — must carry it unchanged, including codes this client
does not know;
- the configurator page is shared for both types; the SW tab is always visible regardless of type;
- storage configurations use the same vendor_spec + PN→LOT + pricing flow as server configurations;
- storage component categories map to existing tabs: `ENC`/`DKC`/`CTL` → Base, `HIC` → PCI (HIC-карты СХД; `HBA`/`NIC` — серверные, не смешивать), `SSD`/`HDD` → Storage (используют существующие серверные LOT), `ACC` → Accessories (используют существующие серверные LOT), `SW` → SW.
- `DKC` = контроллерная полка (модель СХД + тип дисков + кол-во слотов + кол-во контроллеров); `CTL` = контроллер (кэш + встроенные порты); `ENC` = дисковая полка без контроллера.
- the available config types and their localized names flow from `qt_settings.config_types` on the server;
QF falls back to hardcoded "server/Сервер" and "storage/СХД" when the setting is absent.
QF falls back to hardcoded "server/Сервер" and "storage/СХД" when the setting is absent;
- every "new configuration" type picker (`/configs` and the project page) renders its buttons from
`config_types` — a new type added on the server appears without a client release.
## Server-driven configurator settings (`qt_settings`)
@@ -145,12 +235,43 @@ full contract and JSON schemas.
| `support_pricing` | Price data for the Base tab's support-level picker (see below) |
Rules:
- sync runs as part of the pricelist pull; failure is non-fatal (Warn log only);
- sync runs in the background sync worker (after the pricelist pull) and in the manual full sync
(`POST /api/sync/all`); failure is non-fatal (Warn log only);
- `local_qt_settings` is a read-only cache — never written by user actions;
- absent or unparseable settings: QF uses hardcoded fallbacks for that key only;
- absent or unparseable settings: `GET /api/configurator-settings` serves the Go-side default for that
key only (`internal/handlers/component.go`); this is the single fallback — the configurator page
holds no copy of tabs/categories and builds its tab buttons from the served `tab_config`;
- `config_types[].categories` is an allowlist: a category absent from all types is shown everywhere;
- `qt_categories.name` and `qt_categories.name_ru` are not used by QF runtime; do not depend on them.
## Article generation
`internal/article` builds the `article` string (`{MODEL}-{CPU}-{MEM}-{GPU}-{DISK}-{NET}-{PSU}-{SUPPORT}`)
from the configuration's `items` + `server_model`.
- which segment a cart LOT belongs to is decided by its `lot_category`, resolved through
`ResolveLotCategories` → `GetLocalComponentCategoriesByLotNames`, i.e. the **component
universe** (latest active `world` ∪ `estimate`), the same source the configurator/BOM/pricing
tab use. It must **not** be scoped to the configuration's pinned pricelist: a world-only LOT
(e.g. `GPU_NV_RTX_PRO_6000D_...`, priced by the world fallback, never added to the estimate
pricelist) is a legitimate cart member and still carries a real `lot_category` in `world`.
Scoping to one pricelist silently dropped such LOTs from the article. See
[decisions/2026-09-01-article-category-from-component-universe.md](decisions/2026-09-01-article-category-from-component-universe.md);
- `BuildOptions` no longer takes a pricelist; `POST /api/configs/preview-article` still accepts
`pricelist_id` but ignores it;
- category comes only from real synced pricelist columns — never inferred from the `lot_name`
prefix (the `SVC_` SUPPORT segment is the sole lot_name-pattern exception, see below);
- within a segment the model/capacity/speed **token** is still parsed from the `lot_name` —
that is the article text itself, not categorization;
- when a token can't be parsed (the `lot_name` doesn't fit `{GROUP}_{VENDOR}_{MODEL}[_{SPEC}…]`)
the segment carries the LOT's `lot_category` as the token — **never** a bare `UNK`. Such a
segment comes back with `Recognized = false` in `BuildResult.Segments`, plus a `Warnings`
entry naming the `lot_name`. The configurator highlights the segment (amber) and lists the
warnings; create/update/rollback log `WARN "article generation degraded"`. No catalog of real
lot_names / model names lives in the repo (`no-hardcoded-vendors`), so only *structural*
parse failure is detected, not a wrong-but-well-formed token. See
[decisions/2026-09-01-article-degraded-token-visibility.md](decisions/2026-09-01-article-degraded-token-visibility.md).
## Support as a BOM LOT
The Base tab's support-level picker adds/replaces a synthetic LOT in `cart` (e.g.
@@ -173,7 +294,7 @@ rental calc the same way any BOM line does, with no special-casing needed in tho
a QuoteForge release — see `bible-local/server-contract-qt-settings.md` for the schema; the
fixed list of offerable level×duration codes stays in the frontend, only the price is
server-driven;
- platform (x86 vs HGX-H100/H200 vs HGX-B200 vs HGX-B300) is auto-detected client-side from the
- platform (x86 vs HGX-H100/H200 vs HGX-B200 vs HGX-B300) is auto-detected client-side from the (HGX only for SXM/HGX-named GPUs or H100/H200/B200/B300 non-PCIe; PCIe GPUs like A30 stay x86)
cart's GPU components, the same chip-generation classification used by the rental
depreciation calc (`internal/services/rental.go`), so the picker only offers combinations
valid for the current configuration;
@@ -221,3 +342,14 @@ Rules:
- PN to LOT resolution uses the active local partnumber book;
- canonical persisted mapping is `lot_mappings[]`;
- QuoteForge does not use legacy BOM tables such as `qt_bom`, `qt_lot_bundles`, or `qt_lot_bundle_items`.
## Browser rendering
`web/static/html-escape.js` provides the shared `escapeHtml` for HTML text and
quoted attributes. Never interpolate data into JavaScript event attributes.
Cart and rental controls carry escaped LOT values in `data-lot` and dispatch
through delegated listeners. Sync errors are escaped as text; toast messages use
`textContent`. Run `node scripts/tests/html-security.cjs` for the fast regression check and
`node scripts/tests/html-security-browser.cjs` for real DOM parsing and delegated
events in headless Chrome (`CHROME_BIN` overrides the executable). The browser
check uses synthetic data, controlled API responses and a temporary profile.
+65 -5
View File
@@ -17,7 +17,7 @@ Main tables:
| `local_partnumber_book_items` | PN -> LOT catalog payload |
| `pending_changes` | sync queue |
| `connection_settings` | encrypted MariaDB connection settings |
| `app_settings` | local app state |
| `app_settings` | local app state (key→value); includes `indicator_mode` = `color` \| `accessible`, a per-client display preference set on `/setup` |
| `local_schema_migrations` | applied local migration markers |
| `local_qt_settings` | server-pushed configurator settings cache (from `qt_settings`) |
@@ -57,7 +57,7 @@ Server-side only (not queried by client runtime):
- `qt_lot_metadata` — component metadata / price settings; the Go server-side component/category management layer (`ComponentRepository`, `CategoryRepository`, `ComponentService`) was removed — no client code reads this table
- `qt_component_usage_stats` — aggregated component popularity stats (written by server jobs)
- `qt_pricing_alerts` — price anomaly alerts (models exist in Go; feature disabled in runtime)
- `qt_schema_migrations` — server migration history (applied via `go run ./cmd/qfs -migrate`)
- `qt_schema_migrations` — server migration history (applied via `make migrate`, see [Migrations](#migrations))
- `qt_scheduler_runs` — server background job tracking (no Go code references it in this repo)
### Competitor subsystem (server-side only, not used by QuoteForge Go code)
@@ -195,6 +195,8 @@ PK: lot_name
| vendor_spec | longtext JSON | |
| rental_items | JSON | per-lot New/БУ condition for the Аренда tab; see [02-architecture.md](02-architecture.md#rental--paid-testing-pricing-contract) |
| rental_uplift_percent | decimal(8,2) DEFAULT 0 | uplift applied to Estimate buy price for rental "Цена" |
| feature_flags | JSON NULL | added by migration 034. Data features the configuration uses (`["sale_row_markups"]`); derived on save, a client refuses to write a config with a flag it doesn't know. See [decisions/2026-09-24-configuration-feature-flags.md](decisions/2026-09-24-configuration-feature-flags.md) |
| server_updated_at | timestamp(6) NOT NULL | added by migration 035. Server-maintained change marker (`DEFAULT`/`ON UPDATE CURRENT_TIMESTAMP(6)`, index `(server_updated_at, id)`); never written by clients, used for incremental pulls. See [decisions/2026-10-26-incremental-config-project-pull.md](decisions/2026-10-26-incremental-config-project-pull.md) |
### qt_lot_metadata
PK: lot_name
@@ -261,7 +263,9 @@ PK: lot_name
| price | decimal(12,2) NOT NULL | |
| price_quality | tinyint unsigned, nullable | added by migration 033. Set by the external pricelist-building tool, 0-9, based on quote recency/count per its own per-lot pricing-period settings — QF only reads and displays it, never computes it. |
`price_quality` is synced through `LocalPricelistItem` (pricelist detail page) and, separately, through `LocalComponent`/`services.ComponentView` (`/api/components`, used by the configurator's search dropdown and item table) — both read paths go through `pricelistItemRow`/`toLocalComponent()` in `internal/localdb/components.go`. The `/api/components` path reads the component universe (`world` ∪ `estimate`, see [decisions/2026-07-24-component-universe-world-union.md](decisions/2026-07-24-component-universe-world-union.md)), where a world-only LOT is forced to `price_quality = 0` — the only place QF writes this field rather than reading it. The color scale (red 0 → yellow 5 → green 9, gradient) lives in **one shared JS module**, `web/static/price-quality.js` (loaded by `base.html` for every page) — `priceQualityColor`/`qualityDotHtml`/`qualityBadgeHtml`/`qualityRowStyle`. Do not reimplement the color scale locally in a template; add a new helper to that module instead. Used in: pricelist detail "Качество" column, configurator search dropdown (colored dot), configurator table (leftmost quality-dot column), pricing tab (row background tint).
`price_quality` is synced through `LocalPricelistItem` (pricelist detail page) and, separately, through `LocalComponent`/`services.ComponentView` (`/api/components`, used by the configurator's search dropdown and item table) — both read paths go through `pricelistItemRow`/`toLocalComponent()` in `internal/localdb/components.go`. The `/api/components` path reads the component universe (`world` ∪ `estimate`, see [decisions/2026-07-24-component-universe-world-union.md](decisions/2026-07-24-component-universe-world-union.md)), where a world-only LOT is forced to `price_quality = 0` — the only place QF writes this field rather than reading it. The visual scale lives in **one shared JS module**, `web/static/price-quality.js` (loaded by `base.html` for every page) — `priceQualityColor`/`priceQualityLevel`/`qualityMeterHtml`/`qualityDotHtml`/`qualityBadgeHtml`/`qualityRowStyle`. Do not reimplement the scale locally in a template; add a new helper to that module instead. Used in: pricelist detail "Качество" column, configurator search dropdown, configurator table (leftmost quality column), pricing tab.
The module has **two render modes**, chosen per client by `window.QF_INDICATOR_MODE` (injected by `base.html` from `app_settings.indicator_mode`, default `color`): `color` = hue scale red 0 → yellow 5 → green 9; `accessible` = colour-blind-safe 5-step signal-strength meter (`priceQualityLevel`: `0-1/2-3/4-5/6-7/8-9`), single neutral hue. In `accessible` mode the pricing tab drops the row background tint for a leading meter column, the `world`-fallback amber cell tint becomes a `W` text marker, and the "contaminated total" red becomes a `⚠` prefix — see [decisions/2026-08-31-accessible-indicator-mode.md](decisions/2026-08-31-accessible-indicator-mode.md). Any new colour-only signal must add a branch in this module keyed on `isAccessible()`, not a local one.
The real table also has `price_method`, `price_period_days`, `price_coefficient`, `manual_price`, `meta_prices` and `lead_time_weeks` columns, owned and written by the external pricing engine that maintains `qt_pricelist_items` — **keep them in the table for backward compatibility with that system; QF must never drop, rename, or write to them.** QF itself does not model, sync, or display any of them (tried once, removed as unused/dead in the client). Do not re-add them to `models.PricelistItem`/`LocalPricelistItem` without a concrete UI need.
@@ -303,6 +307,7 @@ PK: username
| rental_enabled | tinyint(1) DEFAULT 0 | shows the Аренда tab on this project's configurations |
| created_at | timestamp | |
| updated_at | timestamp ON UPDATE | |
| server_updated_at | timestamp(6) NOT NULL | added by migration 035. Server-maintained change marker (`DEFAULT`/`ON UPDATE CURRENT_TIMESTAMP(6)`, index `(server_updated_at, id)`); never written by clients, used for incremental pulls. See [decisions/2026-10-26-incremental-config-project-pull.md](decisions/2026-10-26-incremental-config-project-pull.md) |
### qt_schema_migrations
| Column | Type | Notes |
@@ -424,5 +429,60 @@ SQLite:
- local migration state is tracked in `local_schema_migrations`.
MariaDB:
- SQL files live in `migrations/`;
- they are applied by `go run ./cmd/qfs -migrate`.
- SQL files live in `migrations/` (embedded into the binary), one file per change,
`NNN_short_topic.sql`, with the recovery/verify header used by the existing files;
filenames containing `local_` are SQLite-only and skipped;
- applied state is tracked per filename in `qt_schema_migrations`;
- commands (always from current source — never run a `bin/` binary of unknown age):
- `make migrate-status` (`qfs -migrate-status`) — list applied/pending, read-only;
- `make migrate` (`qfs -migrate`) — apply pending files in filename order, then exit;
refuses when `qt_schema_migrations` has no applied rows at all (every file would re-run);
- `make migrate-baseline UPTO=<file.sql>` (`qfs -migrate-baseline`) — record files up to
`UPTO` as applied **without running them**, for a server whose schema is already current;
- migrations use their own connection without read/write timeouts (`openMigrationDB`),
not the app's 3s-timeout sync connection — a long backfill must not be cut mid-file;
- server migration commands finish before runtime connection workers start;
- `-migrate` requires a DBA maintenance window with every other writer stopped,
directly granted global ALL PRIVILEGES, and no other connection using the schema.
Do not expand runtime-user privileges; use the existing developer DBA workflow;
- the runner holds a schema advisory lock and one connection for the entire session.
It automatically writes a full consistent snapshot before changes and per-table
snapshots before every file; each archive is restored into a temporary schema
and compared before any corresponding original-schema change. Snapshot or
verification failure stops the run. See [06-backup.md](06-backup.md);
- on a statement/verify failure, per-step snapshots are restored in reverse order,
including migration history; a full-session snapshot is the fallback. A persistent
marker refuses retry after interruption until `qfs -migrate-restore <archive.zip>`
restores and verifies the original state. Connection/recovery failure retains the
marker and archives and emits an explicit restore diagnostic;
- recovery supports InnoDB base tables and their triggers. Views, routines,
events and cross-schema FKs block migrations before changes; new migrations must
stay within supported table SQL. Existing triggers are captured with their
definer, body, order and SQL/charset context, then restored after all data; Verification requires CREATE/DROP DATABASE.
The runner refuses NO_BACKSLASH_ESCAPES sessions;
- each file, its `-- verify: <description> | <SQL>` queries and history insertion run
on one database session, preserving session variables;
- every verify query and the history insert use a fresh GORM statement
(`Session{NewDB: true}`) on that session; chaining them on one statement leaked state
and broke files with two or more verify checks (fixed 2026-10-26 with migration 035);
- a file is recorded only after statements and all verify checks succeed. Verify
queries return rows only for broken invariants; a row or SQL error stops the run
without marking that file applied. Malformed verify headers fail before execution;
- only duplicate column/key names and already-existing objects are tolerated.
FK creation errors (including errno 150/121) remain fatal;
- each file must be safe to re-run after a partial failure (`ADD COLUMN IF NOT EXISTS`,
idempotent `UPDATE ... WHERE <not yet done>`). The statement parser preserves
semicolons in literals/quoted identifiers and strips ordinary comments. Malformed
quotes, executable comments and DELIMITER directives are rejected before execution;
- there is no GORM `AutoMigrate` against MariaDB: it retypes production columns to match the
running binary's models. On 2026-09-24 a stale v1.18 binary run with the old AutoMigrate
`-migrate` did exactly that, which is why it was removed.
## Connection transport
`connection_settings.tls_mode` defaults to `preferred`; `tls_ca_pem` stores an
optional public CA certificate bundle. SQLite startup adds these columns for
legacy settings. Every runtime/setup MariaDB DSN uses `internal/mariadb` to
apply the policy documented in [05-config.md](05-config.md). Private keys are
never part of the CA field. No mode silently falls back to plaintext except
`preferred`.
+16
View File
@@ -29,6 +29,17 @@
`POST /api/restart` exists only in `debug` mode.
## Settings
| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/api/settings/ui` | read per-client UI display preferences → `{ "indicator_mode": "color" \| "accessible" }` |
| `PUT` | `/api/settings/ui` | set `indicator_mode` (body `{ "indicator_mode": ... }`); `422` on an unknown value |
Stored in `app_settings` (SQLite); no restart. Registered in both the normal and the setup-mode
route sets. Consumed by `web/static/price-quality.js` and the pricing tab via a
`window.QF_INDICATOR_MODE` global that `base.html` renders from the same setting.
## Reference data
| Method | Path | Purpose |
@@ -83,6 +94,11 @@
| `PUT` | `/api/configs/:uuid/rental` | persist Аренда tab state (per-lot New/БУ, weeks, uplift %, annual support) |
| `POST` | `/api/configs/:uuid/rental/calculate` | compute rental/paid-testing pricing without persisting |
Configuration reads include `feature_flags` and, when this build doesn't know some of
them, `unsupported_features`. Any write to such a configuration returns `409`
`{"code":"unsupported_config_features","unsupported_features":[...],"error":"..."}`
(see [decisions/2026-09-24-configuration-feature-flags.md](decisions/2026-09-24-configuration-feature-flags.md)).
## Projects
| Method | Path | Purpose |
+24 -2
View File
@@ -62,8 +62,12 @@ They are stored in SQLite and encrypted with `local_encryption.key` unless `QUOT
| --- | --- |
| `-config <path>` | config file path |
| `-localdb <path>` | SQLite path |
| `-reset-localdb` | destructive local DB reset |
| `-migrate` | apply server migrations and exit |
| `-reset-localdb` | reset after mandatory verified backup; refuses unsynced work |
| `-force-reset-localdb` | with `-reset-localdb`, allow clearing unsynced work; backup remains mandatory |
| `-migrate` | apply pending server SQL migrations (`migrations/*.sql`) and exit |
| `-migrate-restore <archive.zip>` | restore and verify a server migration snapshot during DBA maintenance |
| `-migrate-status` | list server SQL migrations with applied/pending state and exit |
| `-migrate-baseline <file>` | record server migrations up to `<file>` as applied without running them, and exit |
| `-version` | print app version and exit |
## First run
@@ -72,3 +76,21 @@ They are stored in SQLite and encrypted with `local_encryption.key` unless `QUOT
2. runtime opens the local SQLite database;
3. if no stored MariaDB credentials exist, `/setup` is served;
4. after setup, runtime works locally and sync uses saved DB settings in the background.
`QFS_RESET_LOCAL_DB` is no longer supported. Reset requires an explicit CLI flag
on each launch. A forced reset still stops if the safety backup cannot be created
and verified. See [06-backup.md](06-backup.md).
## MariaDB TLS
`/setup` stores `tls_mode` and `tls_ca_pem` in SQLite `connection_settings`,
alongside encrypted credentials. Existing and new connections default to
`preferred`, which encrypts when advertised by MariaDB but can fall back to
plaintext and does not authenticate the server. `required` mandates encryption
without verifying the certificate. `verify-ca` requires a PEM CA and validates
the server certificate chain without checking its hostname. `verify-full`
validates both chain and hostname, using the supplied PEM or system roots.
Only the verification modes accept CA input; invalid modes/PEM are rejected.
Use `verify-full` for authenticated server connections. The setup form tests
the selected policy before saving. Sync, setup checks and migration tooling
share the same TLS configuration builder.
+82
View File
@@ -53,3 +53,85 @@ Archive name:
4. replace `config.yaml` if needed;
5. restart the app;
6. re-enter MariaDB credentials if the original encryption key is unavailable.
## Pre-reset safety backup
Before clearing any table, `-reset-localdb` creates a fresh ZIP under
`<backup root>/pre-reset/qfs-pre-reset-v1-<UTC timestamp>-<unique>.zip`, outside
normal retention and regardless of `QFS_BACKUP_DISABLE`. The archive contains
a consistent SQLite snapshot and the selected config when present. Every ZIP
entry is read back for CRC validation; the extracted database must pass SQLite
`integrity_check`. A backup or verification failure prevents the reset. The
backup directory must also resolve outside git after following symlinks.
Reset refuses queued changes and any project/configuration not marked `synced`,
including NULL/legacy states. A missing sync-status column also blocks reset.
Only `-reset-localdb -force-reset-localdb` bypasses this data check. Deletions run
in one transaction and preserve connection settings. Missing legacy tables are
skipped. Use the restore procedure above for the pre-reset archive; retain it
until the reset has been verified. The environment reset switch was removed to
prevent repeated destructive startup.
## Server migration snapshots
`qfs -migrate` writes mandatory archives under
`<backup root>/server-migrations/qfs-mariadb-v1-<label>-<UTC timestamp>-<unique>.zip`.
`QFS_BACKUP_DIR` overrides the root; `QFS_BACKUP_DISABLE` never disables safety
snapshots. Paths are checked outside git, including symlink targets, before
creation and restore. Archives have private filesystem permissions and contain
server table schemas and data, which may be sensitive; never put them in git.
Connection passwords and the local encryption key are not copied into metadata.
One full snapshot precedes the session. Before each file, a separate snapshot
contains a schema definition and row stream for **every** table: this includes
all affected tables even when a legacy Tables-affected header is incomplete.
This intentionally favors recovery over storage/time efficiency for the manual
DBA tool. Retain safety snapshots outside normal rotation until the migration
has been verified; remove verified-session artifacts as operational cleanup.
The versioned ZIP manifest records table schemas, writable column order, row
counts and order-independent row digests. Row streams preserve NULL, empty
strings, binary values and fractional timestamps. Restore temporarily uses
NO_AUTO_VALUE_ON_ZERO to preserve explicit zero IDs and legacy zero dates;
row digests detect coercion and the original session mode is restored. Generated columns are rebuilt
from schema, not inserted. Trigger bodies, definers, firing order and creation
SQL/charset context are preserved. Triggers are recreated after row loading so
restore does not duplicate audit data or alter restored values. CRC/format/digests are checked, then each archive is
restored into a unique temporary schema on the same server and compared for
schema and data equality. The temporary schema is dropped afterward. A failed
or interrupted check must be investigated before retry; abandoned `qfs_verify_`
schemas are DBA cleanup artifacts, never application data.
Stop every other writer for the entire migration/recovery window, including
clients that could reconnect. The runner detects other current schema connections
and uses an advisory lock to exclude a second runner; the lock does not stop
unrelated SQL clients. An active client appearing during failure recovery causes
recovery refusal rather than overwriting its work. InnoDB base tables and triggers are supported; views, routines, events and
cross-schema FKs require explicit recovery support before migration. Triggers
whose recorded database collation differs from the current database collation
also cause refusal until that legacy state has supported recovery. Trigger
routine calls and qualified references to other schemas are refused; source
SQL must have no effects outside the captured schema.
Unsupported schemas stop before changes. Runtime database-user grants stay as
listed in [03-database.md](03-database.md); the tool requires an existing DBA account.
Failure restores step snapshots in reverse order; the full-session archive is
used if step recovery is insufficient. Schema, writable data and migration
history must match the session start before recovery is reported successful.
The persistent `.pending` marker is keyed by configured endpoint and schema, so
server restarts do not hide an interrupted session. Backups survive connection
loss; a failed recovery retains its marker and reports the full snapshot path.
After an interruption, keep clients stopped and use the snapshot path reported
by the runner:
```bash
go run ./cmd/qfs -migrate-restore /outside/repository/server-migrations/qfs-mariadb-v1-session-<timestamp>-<unique>.zip
make migrate-status
make migrate
```
Restore uses the currently saved DBA connection, requires the same database name,
checks the archive by restoring it into a temporary schema first, restores the
original schema, verifies equality and clears the matching pending marker. It
does not require mysql/mysqldump or any operator-installed database utility.
+25 -1
View File
@@ -4,10 +4,18 @@
```bash
go run ./cmd/qfs
go run ./cmd/qfs -migrate
make migrate-status # server SQL migrations: applied/pending
make migrate # apply pending server SQL migrations
go run ./cmd/migrate_project_updated_at
go test ./...
go vet ./...
make security-check # pinned govulncheck, blocks reachable vulnerabilities
# Optional live transport checks; owns disposable Docker containers:
go test -tags integration ./internal/mariadb -run TestMariaDBTLSIntegration -count=1
# Snapshot/DDL recovery checks; owns a disposable MariaDB 11.8 container:
go test -tags integration ./internal/models -run TestSQLMigrationSafetyMariaDB -count=1
# Production-dump rehearsal; source path via QFS_TEST_PRODUCTION_DUMP:
go test -tags integration ./internal/models -run TestProductionDumpMigrationMariaDB -count=1 -v
make build-release
make install-hooks
```
@@ -33,3 +41,19 @@ make install-hooks
Release history belongs under `releases/<version>/RELEASE_NOTES.md`.
Do not keep temporary change summaries in the repository root.
## Toolchain and release security
`go.mod` requires Go 1.27.3 or newer, including security fixes in the standard
library. This minimum also rejects older compilers with `GOTOOLCHAIN=local`;
normal Go toolchain selection downloads a suitable compiler when necessary.
`make security-check` runs `golang.org/x/vuln/cmd/govulncheck@v1.9.0 ./...`.
The scanner version is pinned in the Makefile. All release build targets and
`scripts/release.sh` require this check to succeed before producing artifacts.
A scanner/database/download failure also stops release.
`node scripts/tests/release-security.cjs` verifies failure stops all release
entry points before artifact creation using an isolated failing scanner.
When updating dependencies: run `go mod tidy`, `go test ./...`, `go vet ./...`,
`make security-check` and the JavaScript regression check documented in
[02-architecture.md](02-architecture.md).
+1
View File
@@ -16,6 +16,7 @@ Project-specific architecture and operational contracts.
| [09-vendor-spec.md](09-vendor-spec.md) | Vendor BOM and CFXML import contract |
| [10-agent-api-guide.md](10-agent-api-guide.md) | End-to-end API guide for agents pricing servers from a TZ |
| [11-lot-suggestions.md](11-lot-suggestions.md) | lot_suggestion column in qt_vendor_partnumber_seen — write/read contract for manual UI mappings |
| [issue-plan-2026-10-26.md](issue-plan-2026-10-26.md) | Open issue plan, implementation status and validation |
| [decisions/](decisions/README.md) | Architectural decision log, one file per decision |
## Rules
@@ -0,0 +1,51 @@
# Decision: project-level DDP export uses each configuration's own sale uplift
**Date:** 2026-08-12
**Status:** active
## Context
Two code paths produce the same "Ценообразование" (pricing) CSV: exporting a single
configuration from `index.html` (`exportPricingCSV`), and exporting an entire project
from `project_detail.html` (`exportProject` → `POST /api/projects/:uuid/export` →
`ExportProjectPricingCSV`). Both end up in `ExportService.buildPricingExportBlock`.
Each configuration's "Аплифт к estimate" (`#pricing-uplift-sale` in the pricing tab) is
saved per configuration, piggybacked into `Configuration.Notes` as
`{"pricing_ui":{"sale_uplift":...}}` (see `serializeConfigNotes`/
`restorePricingStateFromNotes` in `index.html`). The single-config export sends this value
explicitly as `sale_markup` in the request, so it always applies the right uplift.
The project-level export modal never had an uplift input and never sent `sale_markup`, so
`ProjectPricingExportOptions.SaleMarkup` was always `0`. `buildPricingExportBlock` used to
fall back to a single hardcoded `defaultSaleMarkup` (1.3) for every configuration in the
project, silently ignoring each configuration's own saved uplift — a bulk DDP export did
not match what exporting those same configurations individually would produce.
## Decision
`buildPricingExportBlock` resolves the DDP estimate factor per configuration via
`ProjectPricingExportOptions.effectiveSaleMarkupFactor(cfg)`:
- an explicit `opts.SaleMarkup` (only ever sent by the single-config pricing tab, which
knows the live-edited value that may not be saved yet) always wins;
- otherwise it reads that configuration's own saved uplift from `Notes`
(`configSavedSaleUplift`);
- otherwise it falls back to `defaultSaleMarkup` (1.3), same as before, for configurations
that never had an uplift saved.
Both `applyDDPMarkup` call sites in `buildPricingExportBlock` (the BOM-driven branch and
the plain-items fallback) use this per-configuration factor. Stock/Competitor keep using
the fixed `stockCompetitorMarkupFactor` (1.3) — only the Estimate uplift is configurable.
## Consequences
- Project-level DDP export now goes through the same "pipeline" as single-config export:
each row's Estimate is scaled by that configuration's own saved uplift, not a
project-wide constant.
- If a future caller needs to force one uplift across an entire project export regardless
of individual configs' saved settings, it must do so explicitly via `sale_markup` in the
request — do not reintroduce a silent global default that overrides saved per-config
values.
- `configSavedSaleUplift` only reads `Notes`; it does not fail loudly on malformed/foreign
JSON in that field — it just returns 0 and lets the 1.3 fallback apply.
@@ -0,0 +1,63 @@
# Decision: `indicator_mode` — a colour-optional rendering mode for UI signals
**Date:** 2026-08-31
**Status:** active
## Context
Several UI signals were encoded by hue alone and unreadable with a colour-vision deficiency:
- `price_quality` (0–9) as a red→yellow→green gradient — a dot in the configurator search
dropdown and inline before `lot_name`, a coloured number in the pricelist "Качество" column,
a full-row background tint on the "Ценообразование" tab;
- the amber `world`-fallback price cell tint
([2026-07-10-world-pricelist-fallback.md](2026-07-10-world-pricelist-fallback.md));
- the red "contaminated total" in the pricing footer
([2026-07-24-pricing-total-world-share.md](2026-07-24-pricing-total-world-share.md)).
The bible has no dedicated accessibility contract, but `table-management` §Icon Semantics,
`controls-selection` §Buttons, `go-code-style` §Business Logic Placement and `build-version-display`
say the same thing: meaning is never carried by one ambiguous channel — back it with
shape / number / explicit `title`+`aria-label`, and keep the indicator visually subordinate.
## Decision
One per-client preference, **`app_settings.indicator_mode`** = `color` (default) | `accessible`,
set on `/setup` ("Настройки") via `GET`/`PUT /api/settings/ui` (`internal/handlers/settings.go`)
— no restart. Deliberately named for the concern (how indicators are rendered), not for the
current widget, so future colour-only signals attach to the same switch.
`internal/handlers/web.go` `render()` and `internal/handlers/setup.go` inject the mode for every
page; `base.html` publishes it as `window.QF_INDICATOR_MODE`. `web/static/price-quality.js` is
still the single source of the price-quality scale and branches on the mode internally, so its
call sites do not change.
In `accessible` mode:
- `price_quality` renders as a **5-step signal-strength meter** (`priceQualityLevel` =
`min(4, floor(score/2))` → `0-1 / 2-3 / 4-5 / 6-7 / 8-9`), five bars in one neutral hue, each
instance carrying `title`/`aria-label` = `Качество цены: N/9`. `qualityDotHtml` and
`qualityBadgeHtml` return the meter;
- `qualityRowStyle` returns `''` — the pricing tab instead shows the meter in a narrow leading
column (conditional `<th>` / `colspan` in `index.html`, per-sub-row `<td>`);
- the `world`-fallback amber class is dropped for a `W` superscript text marker on the price
(text, not a colour class — `applyCustomPrice`'s colour-stripping regex is a non-issue);
- `_setPricingTotal` drops `text-red-600` for a leading `⚠` glyph, on the same trigger as the
red (`worldShare > 0`). The hover popup — including the "prices for N of M positions" coverage
line — is unchanged in both modes; per 2026-07-24 there is still no colour-coded coverage-only
signal, and `accessible` mode does not add one.
`color` mode is byte-for-byte the previous behaviour.
## Consequences
- Default unchanged; nothing moves until a user opts in on `/setup`.
- `web/static/price-quality.js` stays the only place that knows the 0–9 scale. Any new
colour-coded quality/price signal adds a branch there keyed on `isAccessible()`, never a local
reimplementation, and its `accessible` form must be shape/text, not another hue.
- The pricing-tab tables have a mode-dependent column count (8 / 9). `index.html` guards the
`<th>`, the empty-state `colspan`, the `tfoot` "Итого:" `colspan` and the JS row template on
`ACCESSIBLE_MODE` / `{{ eq .IndicatorMode "accessible" }}`. CSV export is untouched — it reads
`data-*` row attributes, not cell positions.
- New routes `GET`/`PUT /api/settings/ui`, registered in both the normal and setup-mode route
sets in `cmd/qfs/main.go`. `render()` nil-guards `localDB` for tests.
@@ -0,0 +1,62 @@
# Decision: article generation reads lot_category from the component universe, not one pricelist
**Date:** 2026-09-01
**Status:** active
## Context
`internal/article/generator.go` classifies each cart LOT into an article segment
(CPU / MEM / GPU / DISK / NET / PSU) by its `lot_category`. That category was
resolved by `ResolveLotCategories` → `GetLocalLotCategoriesByServerPricelistID`,
which queries exactly **one** pricelist — the configuration's pinned
`pricelist_id` (an `estimate` pricelist). A segment builder skips any LOT whose
category is unknown, so a LOT absent from that one pricelist vanished from the
article with no warning.
`GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE` exposed this: it exists only in `world`
pricelists (where it correctly carries `lot_category = GPU`) and in no `estimate`
pricelist. It was priced (world fallback), exported, and shown in the cart, but
dropped from the article.
This is the same class of bug as
[2026-07-24-component-universe-world-union.md](2026-07-24-component-universe-world-union.md):
the cart's LOT set is `world` ∪ `estimate`, and no read path may silently drop a
LOT that only lives in `world`. Article generation was the one component-reading
path still scoped to a single pricelist.
Deriving the category from the `lot_name` prefix was rejected — it is forbidden by
the pricelist-contract rules (`02-architecture.md`, `bible/rules/patterns/`):
category comes only from real synced pricelist columns.
## Decision
`ResolveLotCategories(local, lotNames)` now calls
`LocalDB.GetLocalComponentCategoriesByLotNames`, which reads the component universe
(`componentUniverse()` — latest active `world` ∪ `estimate`, estimate wins a
collision). The server-pricelist parameter is gone.
- `article.BuildOptions` no longer has `ServerPricelist`.
- `POST /api/configs/preview-article` and the create/update paths still accept
`pricelist_id` but ignore it for article purposes.
- The `lot_name`-token parsing inside a segment (CPU model, memory size, GPU model,
disk capacity, port speed, wattage) still parses the token from the `lot_name` —
that produces the article text, it does not categorize. One parser bug was fixed
alongside: `parseGPUModel` required the model-number token to be all digits
(`isNumeric`), so `RTX_PRO_6000D` lost its `D` and collapsed to `RTX_84GB` —
indistinguishable from a real `RTX` card and different from its sibling
`RTX_PRO_6000` → `RTX6000_96GB`. It now accepts a digit-first alphanumeric token
(`isModelNumber`), giving `RTX6000D_84GB`.
## Consequences
- A world-only LOT lands in the article with its real category. Article generation
no longer depends on which pricelist a LOT was added to.
- The article can shift if the latest active `world`/`estimate` pricelist
recategorizes a LOT — acceptable, and consistent with how the configurator's
category tabs already behave.
- `GetLocalLotCategoriesByServerPricelistID` still exists and is still used by
`services/rental.go` and `services/export.go`. Those paths have the same
single-pricelist blind spot for world-only LOTs; migrating them is out of scope
here but should follow the same direction.
- Covered by `TestResolveLotCategories_WorldOnlyLot` in
`internal/article/categories_test.go`.
@@ -0,0 +1,66 @@
# Decision: unparsed article tokens degrade to a visible category placeholder, never "UNK"
**Date:** 2026-09-01
**Status:** active
## Context
`internal/article` builds each segment by parsing a spec token out of the `lot_name`
(CPU model, memory size, GPU model, disk capacity, port speed, wattage). When a
`lot_name` did not match the expected shape the generator failed silently or
misleadingly:
- `parseCPUModel` / `parseGPUModel` fell back to `normalizeModelToken` or a literal
`"UNK"` with **no signal** — and a partial parse (`RTX_PRO_6000D` → `RTX_84GB`)
produced a plausible but wrong token indistinguishable from a real result;
- `NET` / `PSU` emitted `UNKNET` / `UNKPSU`;
- `MEM` / `DISK` dropped the segment or the capacity and emitted only a terse
`mem_unknown` / `disk_unknown` warning;
- `BuildResult.Warnings` existed but **nothing consumed it**: the `preview-article`
frontend ignored the field and the create/update paths discarded it.
So a new naming shape (a vendor renames a card family, moves memory ahead of the
model, glues an architecture onto the token) would silently produce a wrong or
truncated article and no one would notice.
A vendor/model catalog in the repo to validate tokens against was rejected — it
violates `bible/rules/patterns/no-hardcoded-vendors` and needs constant upkeep.
Detection is therefore limited to **structural** failure (the name doesn't fit
`{GROUP}_{VENDOR}_{MODEL}[_{SPEC}…]`); semantic drift where a wrong token still
parses cleanly cannot be caught automatically and is out of scope.
## Decision
1. `"UNK"` / `"UNKNET"` / `"UNKPSU"` are gone. When a spec token can't be parsed the
article carries the LOT's `lot_category` as the token (`4xGPU`, `2xCPU`,
`768G+MEM`, …) via `placeholderToken`.
2. Each `build*Segment` returns a `segmentResult{value, degraded, warnings}`. Every
degraded segment produces a warning that **names the offending `lot_name`** and
states which category was written instead.
3. `BuildResult` gains `Segments []ResultSegment{Group, Text, Recognized}`.
`Recognized == false` marks a segment whose token is a placeholder.
4. `parseCPUModel` / `parseGPUModel` return `(token, ok)`; `ok == false` on the
last-ditch fallback path.
5. Consumers surface it:
- `POST /api/configs/preview-article` returns `segments` and `warnings`; the
configurator's article line renders each not-`recognized` segment highlighted
(amber) and lists the warnings beneath it (`renderArticleDisplay` in
`web/templates/index.html`);
- create / update / rollback log `WARN "article generation degraded"` with the
`server_model`, article and warnings.
6. No list of real `lot_name`s or model names is committed to the repo — not as a
fixture, not as a golden table. Tests use synthetic names only.
## Consequences
- An unrecognised LOT is always visible in the UI (highlighted token + warning) and
in server logs — never a silent drop or a bare `UNK`.
- The stored `article` string still contains only the placeholder token; the
`recognized` flags are not persisted (regenerated on next save / preview).
- `compressArticle` now returns `[]namedSeg` (the caller re-joins) so the degraded
flags survive compression.
- Structural detection only: a wrong-but-well-formed token (e.g. a future `GTX`
where `RTX` was expected) still passes. Closing that needs a validation source,
deferred pending a server-side structured-attribute contract.
- Covered by `TestBuild_UnparseableModel_CategoryPlaceholder` and
`TestParseGPUModel_VendorLetterSuffix`.
@@ -0,0 +1,69 @@
# Decision: per-row "Накидка" on the Sale table + minimal Sale CSV columns
**Date:** 2026-09-15
**Status:** active
## Context
The Sale (Цена продажи) table only had one table-wide "Аплифт к estimate" multiplier
and one table-wide "Ручная цена" override. There was no way to push margin up on
individual rows (e.g. scarce GPUs) without changing the uplift for every row, and the
table showed Estimate/Склад/Конкуренты/Ручная-цена columns that a sales rep handing a
quote to a customer never needed — the actual deliverable is just LOT, Описание, Кол-во,
and a final price.
## Decision
- The Sale table gains a per-row "Накидка, %" input. Each row's final "Цена" is
`base × (Аплифт к estimate + Накидка% / 100)`, where `base` is the row's raw Estimate
unit price × qty, or — when the table-wide "Ручная цена" is set — that row's
proportional share of the manual total (same last-row-absorbs-remainder distribution
the table-wide manual price already used, now overwriting the row's base instead of
filling a separate column).
- The Sale table's visible/exported columns collapse to
`PN вендора | Описание | LOT | Кол-во | Накидка, % | Цена` on screen, and exactly
`LOT;Описание;Кол-во;Цена` in its own CSV export. Estimate, Склад, Конкуренты, and
the old Ручная-цена column are no longer shown or exported for this table. The Buy
table (Цена покупки) is untouched.
- Per-row markup is keyed by `"<vendorPN>::<lot>"` (uppercased, `NONE` when there is no
vendor PN) — not by `lot_name` alone, since the same LOT can appear under more than
one vendor-PN BOM group (see "Per-LOT row expansion rules" in `02-architecture.md`).
- Persistence is additive: `Configuration.Notes.pricing_ui` gains a new
`sale_row_markups` map alongside the existing `sale_uplift`/`sale_custom_price`, the
same JSON-blob-in-`Notes` mechanism established by
[2026-08-12-project-export-per-config-sale-uplift.md](2026-08-12-project-export-per-config-sale-uplift.md).
Zero/unset rows are omitted from the map entirely.
- CSV export: `POST /api/configs/:uuid/export/pricing` gained `minimal_sale_columns`
(bool) and `sale_row_markups` (map) request fields, read into
`ProjectPricingExportOptions.MinimalSaleColumns`/`SaleRowMarkups`. These are wired
**only** in `ExportConfigPricingCSV`, gated on `req.MinimalSaleColumns` being true —
the Sale table's own "Экспорт CSV" button is the only caller that sets it. Buy export
(basis=fob) and the project-level bulk export (`ExportProjectPricingCSV`) never send
it and keep their full existing column sets/output unchanged.
- `ProjectPricingExportOptions.ManualPrice`, previously parsed from the request but
never actually read into the options struct (a pre-existing dead field), is now wired
— but only inside the same `MinimalSaleColumns` gate, so this bug fix does not change
Buy-table CSV output for anyone already relying on the old (broken) behavior.
## Consequences
- Already-saved configurations are unaffected: `sale_row_markups` absent from `Notes`
means every row's markup defaults to 0%, reproducing the exact totals the table
produced before this change (uplift-only).
- **Correction (2026-09-24):** the original claim that old clients "simply ignore" the
new key was wrong. A pre-v3.41 client rebuilds `Notes` from scratch on save
(only `buy_custom_price`/`sale_uplift`/`sale_custom_price`), so opening and saving a
config in an old client — including the best-effort save on page exit — erases
`sale_row_markups` on the server, and the next pull overwrites every other client's
local copy. Observed in the field: a markup saved and pushed on 09-22 came back as
`null` on the 09-23 pull. From this fix on the client carries unknown `Notes` /
`pricing_ui` keys through verbatim, so future additive keys survive a save by the
current version; nothing can protect against clients older than this fix — all users
must upgrade.
- Negative Накидка is allowed (a per-row discount). The factor
`uplift + pct/100` is floored at 0 on both client and CSV export, so a large discount
gives a zero price instead of a negative one.
- If a future caller needs the full-column DDP CSV format with per-row markup applied,
it must do so explicitly — `MinimalSaleColumns` intentionally couples "reduced
columns" and "per-row markup applied" as one request-level switch rather than two
independent options, since the only current caller needs both together.
@@ -0,0 +1,58 @@
# Decision: per-configuration feature flags — clients refuse to write what they don't understand
**Date:** 2026-09-24
**Status:** active
## Context
Configurations are synced last-write-wins, and every client rebuilds a configuration's
content from what *it* knows before saving. When a newer client adds data (e.g. the
per-row "Накидка" in `Notes.pricing_ui.sale_row_markups`, see
[2026-09-15-sale-row-markup-and-minimal-csv.md](2026-09-15-sale-row-markup-and-minimal-csv.md)),
an older client that opens and saves the same configuration silently drops it; the drop
is pushed to the server and the next pull spreads it to every other client. This
happened in the field (a markup pushed on 2026-09-22 came back `null` on the 09-23 pull).
A global minimum-app-version gate was considered and rejected: it would block all sync
for an outdated client, even for configurations that use no new feature.
## Decision
- `qt_configurations.feature_flags` (JSON array of strings, NULL = none; server
migration `034`) and `local_configurations.feature_flags` (AutoMigrate) record which
data features a configuration uses. Synced like any other column; carried in version
snapshots.
- `internal/configfeatures` is the single source of truth: `known` (flags this build
understands), `Detect(notes)` (derive flags from content), `Unknown`, `Check`,
`UnsupportedError`. Current flags: `sale_row_markups` (at least one non-zero row markup).
- Flags are **derived** on every local write, never sent by the UI: `saveWithVersionAndPending`,
`createWithVersionTx`, `rollbackToVersion`, `UpdateServerCount` first `Check` the flags
*as stored* and refuse with `*configfeatures.UnsupportedError` if any is unknown, then set
`FeatureFlags = Detect(notes)`. This covers update, rename, move, refresh prices,
vendor-spec, rental, server count, deactivate/reactivate, rollback (current and target
snapshot) and clone (the clone inherits the original's flags). Project line reorder is
exempt: it only touches `line_no` and writes the row back with its flags and notes intact.
- Every HTTP handler reports errors via `handlers.RespondError`, which turns an
`UnsupportedError` into `409` `{"code":"unsupported_config_features","unsupported_features":[...],"error":"..."}`
regardless of the status the caller picked.
- Reads expose `unsupported_features` (computed in `LocalToConfiguration`). The
configurator shows a red banner listing them, disables "Сохранить", skips the
exit-save, and also locks itself when a save comes back with the 409.
- Sync push (`pushConfigurationUpdate`) reads the server row first and refuses if its
flags are unknown — a newer client may have written it after our last pull. Such a
refused change is kept in the queue (it is exempt from the max-attempts abandon) so the
local edit is pushed once the app is updated.
- Until migration `034` runs, the server write retries without `feature_flags`
(`withOptionalColumns` in `internal/repository/configuration.go`, same fallback as
`line_no`) and logs a warning — sync keeps working, but flags are not stored on the server.
## Consequences
- **Rule:** any new data feature that an older client would lose on save must add a
flag to `configfeatures.known` and to `Detect` in the same change. Features that old
clients round-trip safely (e.g. data they carry verbatim) need no flag.
- Flags only protect against clients that already contain this check. Clients older
than this change ignore `feature_flags` and can still erase newer data — rollout must
get every user onto a version with this check.
- Order of rollout: apply server migration `034` before (or together with) releasing
the client; otherwise flags silently stay local.
- The UI shows raw flag ids (an old client, by definition, has no human names for them).
@@ -0,0 +1,52 @@
# Decision: incremental pull of configurations and projects via server_updated_at
**Date:** 2026-10-26
**Status:** active
## Context
Opening the configurations or projects list (`GET /api/configs`, `/api/projects`,
`/api/projects/:uuid/configs`, on-demand with a 30s cooldown) and the manual sync ran
`ImportConfigurationsToLocal` / `ImportProjectsToLocal`, which read **every** server row
and rewrote it locally. Measured on production (2026-10-26): configurations 22.2s
(4158 rows, ~3.2 MB, 4078 rewritten for one actual change), projects 4.6s (1692 rows),
holding a pool connection the whole time. `qt_configurations` had no change column;
`qt_projects.updated_at` is set by clients (`SaveProjectPreservingUpdatedAt`), so a change
made offline and pushed later carries an old timestamp and cannot drive a filter.
## Decision
- Server migration `035_add_server_updated_at.sql` adds
`server_updated_at TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6)`
plus index `(server_updated_at, id)` to `qt_configurations` and `qt_projects`. MariaDB
maintains it; no client writes it and it is **not** in `models.Configuration` /
`models.Project`, so writes from any client version (GORM `Create`/`Save`, raw
`UPDATE` with column lists) keep working and bump the marker on every real change.
- Client (`internal/services/sync/pull_cursor.go`):
- if the column is missing (server not migrated yet) every pull is full, exactly as
before; presence is re-checked until found, then cached for the process;
- the first pull of each table per process is full (restores rows lost locally once per
app start), later pulls read `server_updated_at >= cursor - 120s` in one query;
- the cursor is the server's `NOW(6)` read *before* the pull, stored in `app_settings`
(`server_pull_cursor:<table>`) together with `host:port/database`; a cursor from a
different server is ignored;
- the per-row merge (skip `pending`, keep local deactivation sticky, keep `line`) is
unchanged; only the set of rows read differs.
- Old clients keep doing full pulls and are unaffected.
## Consequences
- Idle pulls go from ~27s / 3.2 MB to one indexed range query.
- Hard deletes on the server are not propagated — same as the full pull before.
- Changes applied by InnoDB `ON UPDATE CASCADE` (e.g. a `qt_projects.uuid` change
cascading to `qt_configurations.project_uuid`) do not bump `server_updated_at`; they are
picked up by the next full pull (app start).
- Cursor comparison happens in the server session time zone. Production is `SYSTEM`/MSK
(no DST); on a server with DST, rows changed around the autumn transition could be
missed until the next app start.
- Applying 035 adds the column and builds an index on both tables (~4k and ~2k rows on
production, seconds); `qfs -migrate` is run once by the developer as usual.
- Tests: `internal/services/sync/incremental_pull_mariadb_test.go` (needs
`QF_TEST_MARIADB_DSN`, a disposable MariaDB with the production table DDL; verified on
MariaDB 11.8, same as production), migrator fix in
`internal/models/sql_migration_verify_test.go` (`two checks pass`).
@@ -0,0 +1,54 @@
# Decision: one MariaDB pool per client — reuse, never orphan
**Date:** 2026-10-26
**Status:** active
## Context
A single client held ~30 MariaDB connections during sync. `ConnectionManager.GetDB`
treated a pool older than `pingInterval` (30s) as unusable and dialed a brand-new
`*gorm.DB` without closing the previous one. Sync calls `getDB()` per step, so every
~30s a new pool appeared while the old one kept its idle connections until
`ConnMaxLifetime` (5 min): ~10 orphaned pools × 2 idle + up to 10 active ≈ 30. Each
orphaned `*sql.DB` also leaked its opener goroutine. `TryConnect`, the background
ping-failure path and a failed ping inside `dial` dropped pools the same way.
## Decision
- `GetDB` pings the existing pool when it is merely stale and returns it; it dials only
when there is no pool or the ping fails.
- Whenever the manager replaces or drops a pool (`GetDB`, `TryConnect`,
`checkOnlineNow`, `MarkOffline`, `Disconnect`, failed ping in `dial`) the old
`*sql.DB` is closed.
- Pool limits: `MaxOpenConns=2`, `MaxIdleConns=2`, `ConnMaxLifetime=5m`,
`ConnMaxIdleTime=1m` — one client holds at most 2 connections, idle ones are released
within a minute. Sync is sequential (one query at a time); the second connection lets
UI server queries (active viewers, users status) run alongside a long sync step.
- Background sync interval (`backgroundSyncInterval` in `cmd/qfs/main.go`) is 1 minute
with ±30% random jitter (`sync.WorkerJitter`), so clients do not hit MariaDB in
lockstep. The next cycle is scheduled from the *start* of the previous one, so
heartbeats (`qt_client_schema_state.last_checked_at`) are at most 1.3 × interval apart
while a cycle is shorter than that.
- `/api/sync/users-status` online threshold = interval × (1 + `WorkerJitter`).
- `PullPartnumberBooks` reads only book headers (`id, version, created_at, is_active`)
each cycle; `partnumbers_json` (~5 MB for all books) is fetched per book only when the
book is new or its local items are incomplete. An empty book counts as complete. An
incomplete book is re-pulled at most once per process per version: a server book can
list partnumbers missing from `qt_partnumber_book_items` (seen: `BOT_AGIBOT_G2_PERF`
in `PNBOOK-2026-10-21-001`), which a re-pull cannot fix; this is logged as `WARN` with
the missing partnumbers.
- A worker cycle sends one heartbeat: the worker calls `EnsureReadinessForSync` once and
then the unexported `pushPendingChanges` (the exported `PushPendingChanges` still checks
readiness itself for the manual-sync handlers).
- `NeedSync` asks for the latest active pricelist of all three sources in one
`UNION ALL` query (`GetLatestActiveIDsBySources`): 0.71s → 0.24s.
- `SyncQtSettings` skips the local `DELETE` + `INSERT` when the server rows equal the cache.
- Measured against the production server: a cold sync took 171s (pricelists 105s,
partnumber books 38s, configurations 23s) and is bound by server reads, not pool
size. A warm cycle takes ~1.4s (was ~2.5s plus a 5 MB download).
## Consequences
- Code that keeps a `*gorm.DB` across a reconnect gets `sql: database is closed`; sync
steps must call `getDB()` per operation instead of caching the handle.
- Tests: `internal/db/connection_test.go`, `internal/services/sync/worker_jitter_test.go`.
@@ -0,0 +1,62 @@
# Decision: verified snapshots and whole-session recovery for server migrations
**Date:** 2026-10-26
**Status:** active
## Context
Issue #3 fixes FK error propagation, session variables and verify execution.
Its proposed manual-backup exception conflicts with the shared go-database
contract. MariaDB DDL cannot be made rollback-safe by wrapping the migration
files in an ordinary transaction: see [MariaDB implicit commits](https://mariadb.com/docs/server/reference/sql-statements/transactions/sql-statements-that-cause-an-implicit-commit).
## Decision
The migration engine, rather than the operator, owns full-session and pre-file
per-table backups. A Go ZIP/JSON snapshot format avoids external database-client
dependencies. Every archive is restored into a temporary schema and compared
before the corresponding original-schema changes. Per-file archives include all
tables to cover incomplete legacy affected-table headers and dynamic SQL.
On failure, restore each step snapshot in reverse order; use the full-session
snapshot as fallback. Include migration history in recovery. Persist a recovery
marker before applying any changes, refuse an interrupted session on retry and
ship an explicit `-migrate-restore` path. Restore failures retain archives and
produce diagnostics instead of claiming success.
The production dump contains 27 triggers, so the implementation preserves
InnoDB tables and trigger bodies, definers, order and SQL/charset context.
Triggers are recreated after row loading to avoid firing them during restore;
verification binds them to the temporary schema and performs no later DML.
Views, routines, events, cross-schema FKs and legacy trigger/database collation
mismatches are refused before changes. It requires an existing
DBA account with directly granted global ALL PRIVILEGES so object/client
visibility cannot silently hide unsupported state, plus a maintenance window
without other writers. Do not grant these privileges to the runtime user. The
advisory migration lock excludes other runners, not arbitrary SQL clients.
The parser supports SQL literals and quoted identifiers containing semicolons,
escaped/doubled quotes and ordinary comments. It rejects malformed input,
executable comments and DELIMITER rather than silently misinterpreting them.
Existing merged SQL files are unchanged.
## Consequences
Safety snapshots and restore verification add disk usage, temporary server schema
creation and runtime to the developer-only migration command. Normal local-first
CRUD and sync gain no MariaDB dependency. Migration commands start no background
connection worker. Before production use, rehearse on a current production dump
in MariaDB matching production version and collation; synthetic tests alone do
not satisfy that gate. Unsupported schema features need explicit recovery support
before such schemas may be migrated by this tool.
## Acceptance
Rehearsed on the supplied 2026-10-26 production dump (45 tables, 27 triggers),
using an isolated MariaDB 11.8.3 container and utf8mb4_uca1400_ai_ci. The test
verifies full restore, actual migration 035 on the reconstructed pre-035 clone,
reverse recovery after a later DDL/verify failure, successful retry and exact
restoration of the original dump state. Synthetic tests additionally exercise
FK errno 150, interrupted-session recovery, NULL/empty/binary/decimal/timestamp
values, zero auto-increment IDs and legacy zero dates. No production changes
are performed by these checks.
+130
View File
@@ -0,0 +1,130 @@
# Issue implementation plan - 2026-10-26
Repository: https://git.mchus.pro/mchus/QuoteForge
Scope: issues #2-#7. Initial fixes were merged into `main` in
[PR #8](https://git.mchus.pro/mchus/QuoteForge/pulls/8), with release notes in v3.50.
The follow-up adds completed acceptance checks and server migration recovery
required by the shared rules. Each issue has its own implementation/acceptance commit.
| Order | Issue | Implementation and acceptance | Status |
| --- | --- | --- | --- |
| 1 | [#2](https://git.mchus.pro/mchus/QuoteForge/issues/2) | Block foreign Host/Origin/Referer and cross-site browser requests in both routers; validate API body types; refuse saved-password reuse for a changed host/port/user. Middleware and real setup/runtime router tests reject attacks without changing saved credentials or user data. | Acceptance passed |
| 2 | [#4](https://git.mchus.pro/mchus/QuoteForge/issues/4) | Move LOT-bearing cart/rental events to delegated listeners and escaped data attributes; escape BOM and sync output; share HTML escaping; render toast as text. Malicious LOT regression checks and headless Chrome DOM checks pass. | Acceptance passed |
| 3 | [#7](https://git.mchus.pro/mchus/QuoteForge/issues/7) | Require a fresh verified archive outside rotation before reset, refuse unsynced/legacy states without explicit force, use transactional deletion, remove environment-triggered reset. Verified refusal, ZIP restore, backup failure, rollback and CLI startup after restore. | Acceptance passed |
| 4 | [#5](https://git.mchus.pro/mchus/QuoteForge/issues/5) | Persist TLS mode/CA in SQLite, expose setup controls, share policy across setup/sync/migration DSNs. Verified certificate trust, hostname checks, fallback policy, settings round-trip and live MariaDB 11.8 transport. | Acceptance passed |
| 5 | [#3](https://git.mchus.pro/mchus/QuoteForge/issues/3) | Keep each migration file in one session, execute verify headers before history insertion, propagate FK failures. Failed queries/checks restore the entire session. Verified snapshots, literal-aware SQL parsing, trigger preservation and crash recovery pass synthetic tests and production-dump rehearsal on MariaDB 11.8.3. | Acceptance passed |
| 6 | [#6](https://git.mchus.pro/mchus/QuoteForge/issues/6) | Require Go 1.27.3, update vulnerable dependencies, block release on pinned govulncheck. Tests, vet, security scan, cross-platform builds and release failure checks pass. | Acceptance passed |
## Decisions
- HTTP forms remain supported on setup and multipart vendor import; JSON APIs
reject non-JSON bodies. CLI clients may omit browser origin headers.
- CSP without inline scripts remains outside this patch; the issue lists it as optional.
- Forced reset still requires a verified backup. `QFS_RESET_LOCAL_DB` is ignored,
with a startup warning; only explicit CLI flags can initiate reset.
- TLS defaults to `preferred` for existing-server compatibility, as requested in
the issue. It permits plaintext fallback; use `verify-full` for authenticated
encryption. The setup form explains this distinction.
- `go 1.27.3` enforces the minimum compiler, including with `GOTOOLCHAIN=local`.
A redundant `toolchain go1.27.3` is removed by Go tooling and is unnecessary.
## Validation and handoff
The original implementation PR is merged. Current acceptance results are listed
below. Govulncheck reports zero reachable vulnerabilities and one module-level
advisory whose affected code is not called. Initial cross-platform builds and
all Go/JavaScript checks pass; final verification also covers the recovery change.
Production-dump rehearsal also passes on MariaDB 11.8.3 with the production
collation. No production migration or new release is performed by this work.
## Acceptance follow-up
### Issue #2
`go test ./cmd/qfs -run TestRequestBoundaryOnApplicationRouters -count=1`
exercises the production setup and runtime route constructors with temporary
SQLite state. Foreign setup forms, opaque origins and rebinding hosts are
rejected, text/plain JSON receives 415, and both setup handlers reject saved
password reuse for a changed destination. Local reads still succeed. Saved
credentials, project/configuration counts and the pending queue remain unchanged.
The setup router construction was extracted without changing its routes so
this regression exercises the same middleware registration as startup.
### Issue #4
`node scripts/tests/html-security-browser.cjs` passes in headless Chrome with
an isolated temporary profile. It executes the actual template rendering
functions in a browser DOM with controlled API responses. Both cart layouts,
rental, parsed BOM, toast, pending sync errors, error history and repair errors
preserve malicious strings as text. Real click/change/focus events preserve
LOT values through delegated handlers; no injected elements or payload execution
are observed. This is a rendering integration check, not a full application
navigation test. Set `CHROME_BIN` outside macOS to select a Chrome executable.
### Issue #7
`go test ./cmd/qfs ./internal/localdb -run TestReset -count=1` covers reset
safety and a subprocess executing the real `main` flag/startup path. With
rotating backups disabled, the environment switch does not reset pending work,
normal reset refuses it, and explicit forced reset creates a verified archive.
The test restores ZIP contents and reopens the database through normal startup:
connection settings and the pending queue survive. It deliberately selects
`-migrate-status` with an unavailable loopback database to stop startup before
HTTP/browser launch; no production database or user state is touched.
### Issue #5
`go test -tags integration ./internal/mariadb -run TestMariaDBTLSIntegration -count=1`
starts its own disposable MariaDB 11.8 containers on random loopback ports, with
TLS enabled and disabled, and removes both afterward. The test verifies the
actual session cipher: preferred encrypts with a TLS server and falls back only
with a plaintext server; required and verification modes reject plaintext.
Correct trust succeeds, unrelated roots and wrong hostname fail. Certificates,
private keys and database storage are temporary; no saved user DSN is used.
The matching image must already exist locally (`docker pull mariadb:11.8`).
Setup policy, local settings and synthetic TLS tests also pass.
### Issue #6
Go 1.27.3: `go test ./...`, `go vet ./...` and `make security-check` pass.
CGO-free builds produce macOS arm64 Mach-O and Windows amd64 PE executables
outside the repository. `node scripts/tests/release-security.cjs` substitutes a
failing Go scanner in an isolated directory and confirms all five release build
targets and `scripts/release.sh` stop before compilation or artifact creation.
The scanner reports zero reachable vulnerabilities and one uncalled module-level
advisory. No release is published by these acceptance checks.
### Issue #3
The parser, mandatory server snapshots, restore-readiness checks, reverse step
recovery and explicit crash restore are implemented; see
[server migration recovery decision](decisions/2026-10-26-server-migration-recovery.md).
`go test -tags integration ./internal/models -run TestSQLMigrationSafetyMariaDB -count=1`
passes on a disposable MariaDB 11.8: session variables and two verify checks,
semicolon literals, a later failed verify, a real FK errno 150 after successful
DDL, reverse recovery of prior files, interrupted-session refusal/restore/retry,
unsupported-schema refusal, active-client refusal and backup failure.
Restored schema, binary/NULL/empty/decimal/timestamp data and history are compared
to the original snapshot. Unit tests cover snapshot format/digests, symlink path
refusal and all embedded migration SQL syntax.
The supplied `2026-10-26_RFQ_LOG.sql` dump (45 tables, 27 triggers) is loaded
read-only into a new disposable MariaDB **11.8.3** container with
`utf8mb4_uca1400_ai_ci`, matching the production version and collation.
`TestProductionDumpMigrationMariaDB` verifies a full snapshot restore, rebuilds
the pre-035 schema only in that clone, applies the real embedded migration 035,
forces a subsequent DDL/verify failure, verifies reverse recovery against the
complete pre-session snapshot, then applies 035 successfully and restores the
exact original dump state. Trigger bodies, definers and order are preserved;
triggers never fire during row loading. No production schema or dump file is changed.
Run this optional, destructive-to-its-own-container rehearsal with an existing
`mariadb:11.8.3` image and `QFS_TEST_PRODUCTION_DUMP=/outside/repository/dump.sql`:
```bash
go test -tags integration ./internal/models -run TestProductionDumpMigrationMariaDB -count=1 -v
```
The environment variable supplies the source file only; no externally supplied
DSN is accepted. The test owns and removes its container and temporary archives.
+2
View File
@@ -75,6 +75,8 @@ Update this file in the same commit as any change to the flows below.
- `POST /api/sync/push` drains the queue by writing to MariaDB.
- If a push fails, `increment_attempts` and `last_error` are updated; row stays in queue.
- `RepairPendingChanges` reconciles orphaned changes (configuration/project deleted locally).
- Background cycle (`sync.Worker.runSync`) runs independent streams: push (gated by readiness guard), then pricelists, partnumber books, qt_settings. A failure in one stream is logged and never stops the others.
- Configurations left `sync_status=pending` with no `pending_changes` row (queue row dropped, e.g. a stale event skipped without marking synced) are re-enqueued as an `update` with the current snapshot at the start of each push (`requeueOrphanedPendingConfigurations`).
---
+37 -17
View File
@@ -43,7 +43,7 @@ category codes that are allowed for each type. QF uses this for:
"MB","CPU","MEM","RAID",
"SSD","HDD","M2","EDSFF","HHHL",
"GPU","NIC","HCA","DPU","HBA",
"PSU","PS","ACC","RISERS","CARD","BB"
"PSU","PS","ACC","RISERS","CARD","BB","SFP"
]
},
{
@@ -53,7 +53,7 @@ category codes that are allowed for each type. QF uses this for:
"categories": [
"DKC","CPU","MEM","PS",
"SSD","HDD","M2","EDSFF","HHHL",
"NIC","HBA","HCA","ACC","CARD"
"NIC","HBA","HCA","ACC","CARD","SFP"
]
}
]
@@ -109,7 +109,16 @@ single-select mode.
]
},
{ "key": "power", "label": "Power", "single_select": false, "categories": ["PS","PSU"] },
{ "key": "accessories", "label": "Accessories", "single_select": false, "categories": ["ACC","CARD"] },
{
"key": "accessories",
"label": "Accessories",
"single_select": false,
"categories": ["ACC","CARD","SFP"],
"sections": [
{ "title": "Аксессуары", "categories": ["ACC","CARD"] },
{ "title": "Трансиверы", "categories": ["SFP"] }
]
},
{ "key": "sw", "label": "SW", "single_select": false, "categories": ["SW"] }
]
```
@@ -126,9 +135,15 @@ contain any items. Other tabs are hidden when empty.
**Value format:** JSON string array.
```json
["base", "storage", "pci"]
["base"]
```
"Empty" is evaluated after the per-`config_type` category filter, so with only
`base` listed a tab such as Storage/PCI shows for a server configuration (its
categories have components) and hides for a type whose allowlist excludes all of
its categories (e.g. networking). Listing a tab here forces it visible for every
config type.
---
### `required_categories`
@@ -173,19 +188,19 @@ the regulation and the picker won't show it.
},
"hgx_price": {
"HGX-H200": {
"B": {"1": 20000, "3": 55000, "5": 80000},
"S": {"1": 28000, "3": 70000, "5": 100000},
"P": {"1": 63000, "3": 105000}
"B": {"1": 28000, "3": 77000, "5": 112000},
"S": {"1": 39500, "3": 98000, "5": 140000},
"P": {"1": 88500, "3": 147000}
},
"HGX-B200": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 101000, "3": 176000}
"B": {"1": 158500, "3": 230500, "5": 284500},
"S": {"1": 176500, "3": 266500, "5": 332500},
"P": {"1": 232000, "3": 322000}
},
"HGX-B300": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 112000, "3": 187000}
"B": {"1": 192000, "3": 270000, "5": 328500},
"S": {"1": 211500, "3": 309000, "5": 380500},
"P": {"1": 286000, "3": 383500}
}
}
}
@@ -200,10 +215,15 @@ current as the regulation evolves.
## Backward compatibility
- If `qt_settings` does not exist (old server): QF logs `Warn` during sync and
leaves `local_qt_settings` empty. The frontend falls back to hardcoded defaults
for all four settings. No crash, no data loss.
- If a specific key is absent from `qt_settings`: QF falls back to the hardcoded
default for that key only.
leaves `local_qt_settings` empty. `GET /api/configurator-settings` then serves
the Go-side defaults (`internal/handlers/component.go`, `default*()`) for every
setting. No crash, no data loss.
- If a specific key is absent from `qt_settings`: the same Go-side default is served
for that key only.
- The Go-side defaults are the **only** fallback and are not kept in step with the
server — change tabs/categories in `qt_settings`, not in code. The configurator page
has no JS copy: `TAB_CONFIG` (except "Other") and the tab buttons are built from the
served `tab_config`; a config type with no `config_types` allowlist sees all categories.
- Old QF clients that do not know about `local_qt_settings` continue to use their
hardcoded JS constants unchanged.
+58 -82
View File
@@ -46,7 +46,7 @@ import (
var Version = "dev"
var errVendorImportTooLarge = errors.New("vendor workspace file exceeds 1 GiB limit")
const backgroundSyncInterval = 5 * time.Minute
const backgroundSyncInterval = time.Minute
const onDemandPullCooldown = 30 * time.Second
const startupConsoleWarning = "Не закрывайте консоль иначе приложение не будет работать"
@@ -59,8 +59,12 @@ func main() {
configPath := flag.String("config", "", "path to config file (default: user state dir or QFS_CONFIG_PATH)")
localDBPath := flag.String("localdb", "", "path to local SQLite database (default: user state dir or QFS_DB_PATH)")
resetLocalDB := flag.Bool("reset-localdb", false, "reset local SQLite data on startup (keeps connection settings)")
migrate := flag.Bool("migrate", false, "run database migrations")
resetLocalDB := flag.Bool("reset-localdb", false, "reset local SQLite data after a verified safety backup (keeps connection settings)")
forceResetLocalDB := flag.Bool("force-reset-localdb", false, "allow -reset-localdb to clear unsynced work after a verified backup")
migrate := flag.Bool("migrate", false, "apply pending server (MariaDB) SQL migrations and exit")
migrateRestore := flag.String("migrate-restore", "", "restore a verified server migration snapshot during maintenance and exit")
migrateStatus := flag.Bool("migrate-status", false, "list server SQL migrations with applied/pending state and exit")
migrateBaseline := flag.String("migrate-baseline", "", "record server SQL migrations up to and including this file as applied, without running them, and exit")
version := flag.Bool("version", false, "show version information")
flag.Parse()
@@ -110,8 +114,11 @@ func main() {
}
}
if shouldResetLocalDB(*resetLocalDB) {
if err := localdb.ResetData(resolvedLocalDBPath); err != nil {
if os.Getenv("QFS_RESET_LOCAL_DB") != "" {
slog.Warn("QFS_RESET_LOCAL_DB is ignored; use -reset-localdb explicitly")
}
if *resetLocalDB {
if err := localdb.ResetDataWithOptions(resolvedLocalDBPath, resolvedConfigPath, *forceResetLocalDB); err != nil {
slog.Error("failed to reset local database", "error", err)
os.Exit(1)
}
@@ -126,6 +133,10 @@ func main() {
// Check if running in setup mode (no connection settings)
if !local.HasSettings() {
if *migrate || *migrateStatus || *migrateBaseline != "" || *migrateRestore != "" {
slog.Error("server migration command requires saved DBA connection settings")
os.Exit(1)
}
slog.Info("no database settings found, starting setup mode")
runSetupMode(local)
return
@@ -168,13 +179,6 @@ func main() {
// Create connection manager. Runtime stays local-first; MariaDB is used on demand by sync/setup only.
connMgr := db.NewConnectionManager(local)
// Keep the online-status cache fresh in the background so request-handling
// goroutines (health checks, middleware, etc.) never block on the MySQL
// dial/read timeout themselves.
connMgrCtx, connMgrCancel := context.WithCancel(context.Background())
defer connMgrCancel()
connMgr.Start(connMgrCtx)
dbUser := local.GetDBUser()
slog.Info("starting QuoteForge server",
@@ -185,28 +189,36 @@ func main() {
"online", false,
)
if *migrate {
mariaDB, err := connMgr.GetDB()
if *migrate || *migrateStatus || *migrateBaseline != "" || *migrateRestore != "" {
mariaDB, err := openMigrationDB(local)
if err != nil {
slog.Error("cannot run migrations: database not available", "error", err)
os.Exit(1)
}
if mariaDB == nil {
slog.Error("cannot run migrations: database not available")
switch {
case *migrateRestore != "":
err = models.RestoreSQLMigrationSession(mariaDB, *migrateRestore)
case *migrateStatus:
err = runMigrateStatus(mariaDB)
case *migrateBaseline != "":
err = runMigrateBaseline(mariaDB, *migrateBaseline)
default:
err = runMigrateApply(mariaDB, resolvedLocalDBPath)
}
if err != nil {
slog.Error("migration command failed", "error", err)
os.Exit(1)
}
slog.Info("running database migrations...")
if err := models.Migrate(mariaDB); err != nil {
slog.Error("migration failed", "error", err)
os.Exit(1)
}
if err := models.SeedCategories(mariaDB); err != nil {
slog.Error("seeding categories failed", "error", err)
os.Exit(1)
}
slog.Info("migrations completed")
os.Exit(0)
}
// Keep the online-status cache fresh in the background so request-handling
// goroutines (health checks, middleware, etc.) never block on the MySQL
// dial/read timeout themselves.
connMgrCtx, connMgrCancel := context.WithCancel(context.Background())
defer connMgrCancel()
connMgr.Start(connMgrCtx)
gin.SetMode(cfg.Server.Mode)
restartSig := make(chan struct{}, 1)
@@ -303,22 +315,6 @@ func showStartupConsoleWarning() {
fmt.Printf("\033]0;%s\007", startupConsoleWarning)
}
func shouldResetLocalDB(flagValue bool) bool {
if flagValue {
return true
}
value := strings.TrimSpace(os.Getenv("QFS_RESET_LOCAL_DB"))
if value == "" {
return false
}
switch strings.ToLower(value) {
case "1", "true", "yes", "y":
return true
default:
return false
}
}
func derefString(value *string) string {
if value == nil {
return ""
@@ -553,42 +549,12 @@ func nextBackupTime(now time.Time, hour, minute int) time.Time {
func runSetupMode(local *localdb.LocalDB) {
restartSig := make(chan struct{}, 1)
// In setup mode, we don't have a connection manager yet (will restart after setup)
templatesPath := filepath.Join("web", "templates")
setupHandler, err := handlers.NewSetupHandler(local, nil, templatesPath, restartSig)
router, err := setupSetupRouter(local, restartSig)
if err != nil {
slog.Error("failed to create setup handler", "error", err)
slog.Error("failed to create setup router", "error", err)
os.Exit(1)
}
gin.SetMode(gin.ReleaseMode)
router := gin.New()
router.Use(gin.Recovery())
if staticFS, err := qfassets.StaticFS(); err == nil {
router.StaticFS("/static", http.FS(staticFS))
} else {
slog.Error("failed to load embedded static assets", "error", err)
os.Exit(1)
}
// Setup routes only
router.GET("/", func(c *gin.Context) {
c.Redirect(http.StatusFound, "/setup")
})
router.GET("/setup", setupHandler.ShowSetup)
router.POST("/setup", setupHandler.SaveConnection)
router.POST("/setup/test", setupHandler.TestConnection)
router.GET("/setup/status", setupHandler.GetStatus)
// Health check
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "setup_required",
"time": time.Now().UTC().Format(time.RFC3339),
})
})
addr := "127.0.0.1:8080"
slog.Info("starting setup mode server", "address", addr, "version", Version)
@@ -892,6 +858,11 @@ func setupRouter(cfg *config.Config, local *localdb.LocalDB, connMgr *db.Connect
router.POST("/setup/test", setupHandler.TestConnection)
router.GET("/setup/status", setupHandler.GetStatus)
// Per-client UI display preferences
settingsHandler := handlers.NewSettingsHandler(local)
router.GET("/api/settings/ui", settingsHandler.GetUI)
router.PUT("/api/settings/ui", settingsHandler.PutUI)
// Web pages
router.GET("/", webHandler.Index)
router.GET("/configs", webHandler.Configs)
@@ -1077,6 +1048,7 @@ func setupRouter(cfg *config.Config, local *localdb.LocalDB, connMgr *db.Connect
}
c.JSON(http.StatusOK, gin.H{
"article": result.Article,
"segments": result.Segments,
"warnings": result.Warnings,
})
})
@@ -1603,6 +1575,14 @@ func setupRouter(cfg *config.Config, local *localdb.LocalDB, connMgr *db.Connect
c.JSON(http.StatusOK, simplified)
})
// ProjectWithWarning flattens a project's fields (via embedding) alongside an
// optional variant_warning, set when an invalid variant name (reserved "main" or
// disallowed characters) was auto-converted into a valid one instead of rejected.
type ProjectWithWarning struct {
*models.Project
VariantWarning string `json:"variant_warning,omitempty"`
}
projects.POST("", func(c *gin.Context) {
var req services.CreateProjectRequest
if err := c.ShouldBindJSON(&req); err != nil {
@@ -1613,17 +1593,15 @@ func setupRouter(cfg *config.Config, local *localdb.LocalDB, connMgr *db.Connect
c.JSON(http.StatusBadRequest, gin.H{"error": "project code is required"})
return
}
project, err := projectService.Create(dbUsername, &req)
project, variantWarning, err := projectService.Create(dbUsername, &req)
if err != nil {
respondByErrCase(c, err,
errCase{services.ErrReservedMainVariant, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectCodeInvalidChars, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectVariantInvalidChars, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectCodeExists, http.StatusConflict, "conflict detected"},
)
return
}
c.JSON(http.StatusCreated, project)
c.JSON(http.StatusCreated, ProjectWithWarning{Project: project, VariantWarning: variantWarning})
})
projects.GET("/:uuid", func(c *gin.Context) {
@@ -1644,20 +1622,18 @@ func setupRouter(cfg *config.Config, local *localdb.LocalDB, connMgr *db.Connect
respondError(c, http.StatusBadRequest, "invalid request", err)
return
}
project, err := projectService.Update(c.Param("uuid"), dbUsername, &req)
project, variantWarning, err := projectService.Update(c.Param("uuid"), dbUsername, &req)
if err != nil {
respondByErrCase(c, err,
errCase{services.ErrReservedMainVariant, http.StatusBadRequest, "invalid request"},
errCase{services.ErrCannotRenameMainVariant, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectCodeInvalidChars, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectVariantInvalidChars, http.StatusBadRequest, "invalid request"},
errCase{services.ErrProjectCodeExists, http.StatusConflict, "conflict detected"},
errCase{services.ErrProjectNotFound, http.StatusNotFound, "resource not found"},
errCase{services.ErrProjectForbidden, http.StatusForbidden, "access denied"},
)
return
}
c.JSON(http.StatusOK, project)
c.JSON(http.StatusOK, ProjectWithWarning{Project: project, VariantWarning: variantWarning})
})
projects.POST("/:uuid/archive", func(c *gin.Context) {
+134
View File
@@ -0,0 +1,134 @@
package main
import (
"fmt"
"io/fs"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/models"
mysqlDriver "github.com/go-sql-driver/mysql"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// Server (MariaDB) migration commands. Each runs only the embedded migrations/*.sql
// files tracked in qt_schema_migrations — never GORM AutoMigrate, which can silently
// retype production columns to match whatever models the running binary was built with.
// See bible-local/03-database.md "Migrations".
// openMigrationDB opens a dedicated MariaDB connection for migrations. The app's shared
// connection (localdb.GetDSN) has 3s read/write timeouts for offline-first sync; a
// backfill UPDATE over qt_pricelist_items needs minutes, and a client-side timeout
// mid-statement leaves the file half-applied and unrecorded.
func openMigrationDB(local *localdb.LocalDB) (*gorm.DB, error) {
dsn, err := local.GetDSN()
if err != nil {
return nil, fmt.Errorf("server connection settings: %w", err)
}
cfg, err := mysqlDriver.ParseDSN(dsn)
if err != nil {
return nil, fmt.Errorf("parse DSN: %w", err)
}
cfg.Timeout = 10 * time.Second
cfg.ReadTimeout = 0
cfg.WriteTimeout = 0
db, err := gorm.Open(mysql.Open(cfg.FormatDSN()), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return nil, fmt.Errorf("connect to server: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return nil, fmt.Errorf("migration pool: %w", err)
}
sqlDB.SetMaxOpenConns(1)
sqlDB.SetMaxIdleConns(1)
return db, nil
}
// runMigrateStatus prints every server migration and whether it is applied.
func runMigrateStatus(db *gorm.DB) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
states, err := models.SQLMigrationStatus(db, migrations)
if err != nil {
return err
}
pending := 0
for _, st := range states {
if st.AppliedAt != nil {
fmt.Printf(" applied %s (%s)\n", st.Filename, st.AppliedAt.Format("2006-01-02 15:04"))
} else {
fmt.Printf(" PENDING %s\n", st.Filename)
pending++
}
}
fmt.Printf("%d migrations, %d pending\n", len(states), pending)
return nil
}
// runMigrateApply applies pending server migrations. Refuses on a server with no
// migration history at all: its schema was built by other means, so every file would be
// re-run — record a baseline with -migrate-baseline first.
func runMigrateApply(db *gorm.DB, localDBPath string) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
if err := requireMigrationHistory(db, migrations); err != nil {
return err
}
backupRoot, err := appstate.ServerMigrationBackupDir(localDBPath)
if err != nil {
return err
}
applied, err := models.RunSQLMigrations(db, migrations, func(filename string) {
fmt.Printf(" applying %s ...\n", filename)
}, backupRoot)
if err != nil {
return err
}
if len(applied) == 0 {
fmt.Println("server schema is up to date — nothing to apply")
return nil
}
fmt.Printf("applied %d migration(s)\n", len(applied))
return nil
}
// runMigrateBaseline records migrations up to and including upto as applied without
// running them.
func runMigrateBaseline(db *gorm.DB, upto string) error {
migrations, err := qfassets.MigrationsFS()
if err != nil {
return err
}
recorded, err := models.BaselineSQLMigrations(db, migrations, upto)
for _, f := range recorded {
fmt.Printf(" recorded %s (not executed)\n", f)
}
if err != nil {
return err
}
fmt.Printf("baseline: %d migration(s) recorded as applied, up to %s\n", len(recorded), upto)
return nil
}
func requireMigrationHistory(db *gorm.DB, migrations fs.FS) error {
states, err := models.SQLMigrationStatus(db, migrations)
if err != nil {
return err
}
for _, st := range states {
if st.AppliedAt != nil {
return nil
}
}
return fmt.Errorf("qt_schema_migrations has no applied migrations, so all %d files would run against this server. "+
"Record what the server already has first: qfs -migrate-baseline <last-applied-file.sql> (make migrate-baseline UPTO=...)", len(states))
}
+72
View File
@@ -0,0 +1,72 @@
package main
import (
"net/http/httptest"
"reflect"
"strings"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/config"
"github.com/gin-gonic/gin"
)
func TestRequestBoundaryOnApplicationRouters(t *testing.T) {
moveToRepoRoot(t)
local, connMgr, _ := newAPITestStack(t)
if err := local.SaveSettings("db.example", 3306, "example", "example", "example"); err != nil {
t.Fatal(err)
}
cfg := &config.Config{}
setConfigDefaults(cfg)
runtimeRouter, _, err := setupRouter(cfg, local, connMgr, "example", nil)
if err != nil {
t.Fatal(err)
}
setupRouter, err := setupSetupRouter(local, nil)
if err != nil {
t.Fatal(err)
}
for name, router := range map[string]*gin.Engine{"setup": setupRouter, "runtime": runtimeRouter} {
t.Run(name, func(t *testing.T) {
before, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
projects, configs, pending := local.CountProjects(), local.CountConfigurations(), local.CountPendingChanges()
for _, tt := range []struct {
name, method, path, host, origin, contentType, body string
want int
}{
{"foreign setup save", "POST", "/setup", "", "https://attacker.example", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 403},
{"foreign setup test", "POST", "/setup/test", "", "https://attacker.example", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 403},
{"rebind settings read", "GET", "/setup/status", "attacker.example:8080", "", "", "", 403},
{"opaque origin", "PUT", "/api/settings/ui", "", "null", "application/json", `{"indicator_mode":"accessible"}`, 403},
{"plain JSON", "PUT", "/api/settings/ui", "", "", "text/plain", `{"indicator_mode":"accessible"}`, 415},
{"saved password destination save", "POST", "/setup", "", "", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 422},
{"saved password destination test", "POST", "/setup/test", "", "", "application/x-www-form-urlencoded", "host=attacker.example&user=example", 422},
{"local read", "GET", "/api/settings/ui", "", "http://127.0.0.1:8080", "", "", 200},
} {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, "http://127.0.0.1:8080"+tt.path, strings.NewReader(tt.body))
if tt.host != "" {
req.Host = tt.host
}
req.Header.Set("Origin", tt.origin)
req.Header.Set("Content-Type", tt.contentType)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != tt.want {
t.Fatalf("status=%d want=%d body=%s", w.Code, tt.want, w.Body.String())
}
})
}
after, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(before, after) || projects != local.CountProjects() || configs != local.CountConfigurations() || pending != local.CountPendingChanges() {
t.Fatal("rejected requests changed local state")
}
})
}
}
+130
View File
@@ -0,0 +1,130 @@
package main
import (
"archive/zip"
"context"
"flag"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
)
func TestResetCLIHelper(t *testing.T) {
if os.Getenv("QFS_RESET_CLI_HELPER") != "1" {
return
}
for i, arg := range os.Args {
if arg == "--" {
os.Args = append([]string{"qfs"}, os.Args[i+1:]...)
flag.CommandLine = flag.NewFlagSet("qfs", flag.ExitOnError)
main()
return
}
}
t.Fatal("missing CLI arguments")
}
func TestResetCLIAndRestore(t *testing.T) {
dir := t.TempDir()
t.Setenv("QFS_STATE_DIR", dir)
t.Setenv("QFS_BACKUP_DISABLE", "1")
t.Setenv("QFS_RESET_LOCAL_DB", "yes")
t.Setenv("QFS_BACKUP_DIR", filepath.Join(dir, "backups"))
dbPath := filepath.Join(dir, "qfs.db")
configPath := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(configPath, []byte("server:\n host: 127.0.0.1\n"), 0600); err != nil {
t.Fatal(err)
}
local, err := localdb.New(dbPath)
if err != nil {
t.Fatal(err)
}
if err := local.SaveSettings("127.0.0.1", 1, "example", "example", "example"); err != nil {
t.Fatal(err)
}
if err := local.AddPendingChange("configuration", "example", "update", "{}"); err != nil {
t.Fatal(err)
}
if err := local.Close(); err != nil {
t.Fatal(err)
}
run := func(args ...string) string {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
argv := append([]string{"-test.run=^TestResetCLIHelper$", "--", "-localdb", dbPath, "-config", configPath, "-migrate-status"}, args...)
cmd := exec.CommandContext(ctx, os.Args[0], argv...)
cmd.Env = append(os.Environ(), "QFS_RESET_CLI_HELPER=1")
output, err := cmd.CombinedOutput()
if ctx.Err() != nil {
t.Fatalf("CLI timed out: %s", output)
}
if err == nil {
t.Fatalf("expected reset refusal or offline migration-status exit: %s", output)
}
return string(output)
}
assertPending := func(want int64) {
t.Helper()
local, err := localdb.New(dbPath)
if err != nil {
t.Fatal(err)
}
defer local.Close()
if got := local.CountPendingChanges(); got != want {
t.Fatalf("pending=%d want=%d", got, want)
}
if !local.HasSettings() {
t.Fatal("connection settings lost")
}
}
if output := run(); !strings.Contains(output, "QFS_RESET_LOCAL_DB is ignored") {
t.Fatalf("environment reset not diagnosed: %s", output)
}
assertPending(1)
if output := run("-reset-localdb"); !strings.Contains(output, "reset refused") {
t.Fatalf("pending reset not refused: %s", output)
}
assertPending(1)
if output := run("-reset-localdb", "-force-reset-localdb"); !strings.Contains(output, "verified pre-reset backup") {
t.Fatalf("missing safety backup: %s", output)
}
assertPending(0)
archives, err := filepath.Glob(filepath.Join(dir, "backups", "pre-reset", "*.zip"))
if err != nil || len(archives) != 1 {
t.Fatalf("archives=%v err=%v", archives, err)
}
archive, err := zip.OpenReader(archives[0])
if err != nil {
t.Fatal(err)
}
defer archive.Close()
for _, entry := range archive.File {
if entry.Name != "qfs.db" && entry.Name != "config.yaml" {
t.Fatalf("unexpected archive entry: %s", entry.Name)
}
in, err := entry.Open()
if err != nil {
t.Fatal(err)
}
data, readErr := io.ReadAll(in)
closeErr := in.Close()
if readErr != nil || closeErr != nil {
t.Fatalf("restore entry: read=%v close=%v", readErr, closeErr)
}
if err := os.WriteFile(filepath.Join(dir, entry.Name), data, 0600); err != nil {
t.Fatal(err)
}
}
assertPending(1)
if output := run(); !strings.Contains(output, "cannot run migrations: database not available") {
t.Fatalf("restored startup did not reach migration status: %s", output)
}
assertPending(1)
}
+57
View File
@@ -0,0 +1,57 @@
package main
import (
"fmt"
"net/http"
"path/filepath"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/handlers"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/middleware"
"github.com/gin-gonic/gin"
)
func setupSetupRouter(local *localdb.LocalDB, restartSig chan struct{}) (*gin.Engine, error) {
// In setup mode, we don't have a connection manager yet (will restart after setup)
templatesPath := filepath.Join("web", "templates")
setupHandler, err := handlers.NewSetupHandler(local, nil, templatesPath, restartSig)
if err != nil {
return nil, fmt.Errorf("create setup handler: %w", err)
}
gin.SetMode(gin.ReleaseMode)
router := gin.New()
router.Use(gin.Recovery())
router.Use(middleware.CORS())
if staticFS, err := qfassets.StaticFS(); err == nil {
router.StaticFS("/static", http.FS(staticFS))
} else {
return nil, fmt.Errorf("load embedded static assets: %w", err)
}
// Setup routes only
router.GET("/", func(c *gin.Context) {
c.Redirect(http.StatusFound, "/setup")
})
router.GET("/setup", setupHandler.ShowSetup)
router.POST("/setup", setupHandler.SaveConnection)
router.POST("/setup/test", setupHandler.TestConnection)
router.GET("/setup/status", setupHandler.GetStatus)
settingsHandler := handlers.NewSettingsHandler(local)
router.GET("/api/settings/ui", settingsHandler.GetUI)
router.PUT("/api/settings/ui", settingsHandler.PutUI)
// Health check
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "setup_required",
"time": time.Now().UTC().Format(time.RFC3339),
})
})
return router, nil
}
+23 -23
View File
@@ -45,7 +45,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// list versions happy path
listReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions?limit=10&offset=0", nil)
listReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions?limit=10&offset=0", nil)
listRec := httptest.NewRecorder()
router.ServeHTTP(listRec, listReq)
if listRec.Code != http.StatusOK {
@@ -53,7 +53,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// get version happy path
getReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/1", nil)
getReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/1", nil)
getRec := httptest.NewRecorder()
router.ServeHTTP(getRec, getReq)
if getRec.Code != http.StatusOK {
@@ -62,7 +62,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
// rollback happy path
body := []byte(`{"target_version":1,"note":"api rollback"}`)
rbReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/rollback", bytes.NewReader(body))
rbReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/rollback", bytes.NewReader(body))
rbReq.Header.Set("Content-Type", "application/json")
rbRec := httptest.NewRecorder()
router.ServeHTTP(rbRec, rbReq)
@@ -84,7 +84,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 404: version missing
notFoundReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/999", nil)
notFoundReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/999", nil)
notFoundRec := httptest.NewRecorder()
router.ServeHTTP(notFoundRec, notFoundReq)
if notFoundRec.Code != http.StatusNotFound {
@@ -92,7 +92,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 400: invalid version number
invalidReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID+"/versions/abc", nil)
invalidReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/versions/abc", nil)
invalidRec := httptest.NewRecorder()
router.ServeHTTP(invalidRec, invalidReq)
if invalidRec.Code != http.StatusBadRequest {
@@ -100,7 +100,7 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// 400: rollback invalid target_version
badRollbackReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/rollback", bytes.NewReader([]byte(`{"target_version":0}`)))
badRollbackReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/rollback", bytes.NewReader([]byte(`{"target_version":0}`)))
badRollbackReq.Header.Set("Content-Type", "application/json")
badRollbackRec := httptest.NewRecorder()
router.ServeHTTP(badRollbackRec, badRollbackReq)
@@ -109,28 +109,28 @@ func TestConfigurationVersioningAPI(t *testing.T) {
}
// archive + reactivate flow
delReq := httptest.NewRequest(http.MethodDelete, "/api/configs/"+created.UUID, nil)
delReq := httptest.NewRequest(http.MethodDelete, "http://127.0.0.1:8080/api/configs/"+created.UUID, nil)
delRec := httptest.NewRecorder()
router.ServeHTTP(delRec, delReq)
if delRec.Code != http.StatusOK {
t.Fatalf("archive status=%d body=%s", delRec.Code, delRec.Body.String())
}
archivedListReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=archived&page=1&per_page=20", nil)
archivedListReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=archived&page=1&per_page=20", nil)
archivedListRec := httptest.NewRecorder()
router.ServeHTTP(archivedListRec, archivedListReq)
if archivedListRec.Code != http.StatusOK {
t.Fatalf("archived list status=%d body=%s", archivedListRec.Code, archivedListRec.Body.String())
}
reactivateReq := httptest.NewRequest(http.MethodPost, "/api/configs/"+created.UUID+"/reactivate", nil)
reactivateReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/reactivate", nil)
reactivateRec := httptest.NewRecorder()
router.ServeHTTP(reactivateRec, reactivateReq)
if reactivateRec.Code != http.StatusOK {
t.Fatalf("reactivate status=%d body=%s", reactivateRec.Code, reactivateRec.Body.String())
}
activeListReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=active&page=1&per_page=20", nil)
activeListReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=active&page=1&per_page=20", nil)
activeListRec := httptest.NewRecorder()
router.ServeHTTP(activeListRec, activeListReq)
if activeListRec.Code != http.StatusOK {
@@ -151,7 +151,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"P1","code":"P1"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"P1","code":"P1"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -164,7 +164,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
}
createCfgBody := []byte(`{"name":"Cfg A","items":[{"lot_name":"CPU","quantity":1,"unit_price":100}],"server_count":1}`)
createCfgReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/configs", bytes.NewReader(createCfgBody))
createCfgReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs", bytes.NewReader(createCfgBody))
createCfgReq.Header.Set("Content-Type", "application/json")
createCfgRec := httptest.NewRecorder()
router.ServeHTTP(createCfgRec, createCfgReq)
@@ -179,7 +179,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected config project_uuid=%s got=%v", project.UUID, createdCfg.ProjectUUID)
}
cloneReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/configs/"+createdCfg.UUID+"/clone", bytes.NewReader([]byte(`{"name":"Cfg A Clone"}`)))
cloneReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs/"+createdCfg.UUID+"/clone", bytes.NewReader([]byte(`{"name":"Cfg A Clone"}`)))
cloneReq.Header.Set("Content-Type", "application/json")
cloneRec := httptest.NewRecorder()
router.ServeHTTP(cloneRec, cloneReq)
@@ -194,7 +194,7 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected clone project_uuid=%s got=%v", project.UUID, cloneCfg.ProjectUUID)
}
projectConfigsReq := httptest.NewRequest(http.MethodGet, "/api/projects/"+project.UUID+"/configs", nil)
projectConfigsReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/configs", nil)
projectConfigsRec := httptest.NewRecorder()
router.ServeHTTP(projectConfigsRec, projectConfigsReq)
if projectConfigsRec.Code != http.StatusOK {
@@ -210,14 +210,14 @@ func TestProjectArchiveHidesConfigsAndCloneIntoProject(t *testing.T) {
t.Fatalf("expected 2 project configs after clone, got %d", len(projectConfigsResp.Configurations))
}
archiveReq := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/archive", nil)
archiveReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/archive", nil)
archiveRec := httptest.NewRecorder()
router.ServeHTTP(archiveRec, archiveReq)
if archiveRec.Code != http.StatusOK {
t.Fatalf("archive project status=%d body=%s", archiveRec.Code, archiveRec.Body.String())
}
activeReq := httptest.NewRequest(http.MethodGet, "/api/configs?status=active&page=1&per_page=20", nil)
activeReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs?status=active&page=1&per_page=20", nil)
activeRec := httptest.NewRecorder()
router.ServeHTTP(activeRec, activeReq)
if activeRec.Code != http.StatusOK {
@@ -245,7 +245,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"Move Project","code":"MOVE"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"Move Project","code":"MOVE"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -257,7 +257,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("unmarshal project: %v", err)
}
createConfigReq := httptest.NewRequest(http.MethodPost, "/api/configs", bytes.NewReader([]byte(`{"name":"Move Me","items":[],"notes":"","server_count":1}`)))
createConfigReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs", bytes.NewReader([]byte(`{"name":"Move Me","items":[],"notes":"","server_count":1}`)))
createConfigReq.Header.Set("Content-Type", "application/json")
createConfigRec := httptest.NewRecorder()
router.ServeHTTP(createConfigRec, createConfigReq)
@@ -269,7 +269,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("unmarshal config: %v", err)
}
moveReq := httptest.NewRequest(http.MethodPatch, "/api/configs/"+created.UUID+"/project", bytes.NewReader([]byte(`{"project_uuid":"`+project.UUID+`"}`)))
moveReq := httptest.NewRequest(http.MethodPatch, "http://127.0.0.1:8080/api/configs/"+created.UUID+"/project", bytes.NewReader([]byte(`{"project_uuid":"`+project.UUID+`"}`)))
moveReq.Header.Set("Content-Type", "application/json")
moveRec := httptest.NewRecorder()
router.ServeHTTP(moveRec, moveReq)
@@ -277,7 +277,7 @@ func TestConfigMoveToProjectEndpoint(t *testing.T) {
t.Fatalf("move config status=%d body=%s", moveRec.Code, moveRec.Body.String())
}
getReq := httptest.NewRequest(http.MethodGet, "/api/configs/"+created.UUID, nil)
getReq := httptest.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/configs/"+created.UUID, nil)
getRec := httptest.NewRecorder()
router.ServeHTTP(getRec, getReq)
if getRec.Code != http.StatusOK {
@@ -307,7 +307,7 @@ func TestVendorImportRejectsOversizedUpload(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
createProjectReq := httptest.NewRequest(http.MethodPost, "/api/projects", bytes.NewReader([]byte(`{"name":"Import Project","code":"IMP"}`)))
createProjectReq := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects", bytes.NewReader([]byte(`{"name":"Import Project","code":"IMP"}`)))
createProjectReq.Header.Set("Content-Type", "application/json")
createProjectRec := httptest.NewRecorder()
router.ServeHTTP(createProjectRec, createProjectReq)
@@ -334,7 +334,7 @@ func TestVendorImportRejectsOversizedUpload(t *testing.T) {
t.Fatalf("close multipart writer: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/projects/"+project.UUID+"/vendor-import", &body)
req := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/projects/"+project.UUID+"/vendor-import", &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
@@ -358,7 +358,7 @@ func TestCreateConfigMalformedJSONReturnsGenericError(t *testing.T) {
t.Fatalf("setup router: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/configs", bytes.NewReader([]byte(`{"name":`)))
req := httptest.NewRequest(http.MethodPost, "http://127.0.0.1:8080/api/configs", bytes.NewReader([]byte(`{"name":`)))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
+27 -21
View File
@@ -1,11 +1,11 @@
module git.mchus.pro/mchus/quoteforge
go 1.24.0
go 1.27.3
require (
github.com/gin-gonic/gin v1.9.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-sql-driver/mysql v1.7.1
github.com/go-sql-driver/mysql v1.10.1
github.com/google/uuid v1.6.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/mysql v1.5.2
@@ -13,35 +13,41 @@ require (
)
require (
github.com/bytedance/sonic v1.9.1 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic v1.15.0 // indirect
github.com/bytedance/sonic/loader v0.5.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.12 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/glebarez/go-sqlite v1.21.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/go-playground/validator/v10 v10.30.1 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.59.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/stretchr/testify v1.11.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/net v0.46.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/text v0.30.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
github.com/ugorji/go/codec v1.3.1 // indirect
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
golang.org/x/arch v0.22.0 // indirect
golang.org/x/crypto v0.58.0 // indirect
golang.org/x/net v0.61.0 // indirect
golang.org/x/sys v0.49.0 // indirect
golang.org/x/text v0.43.0 // indirect
google.golang.org/protobuf v1.36.10 // indirect
modernc.org/libc v1.22.5 // indirect
modernc.org/mathutil v1.5.0 // indirect
modernc.org/memory v1.5.0 // indirect
+67 -56
View File
@@ -1,20 +1,24 @@
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
github.com/bytedance/sonic v1.9.1 h1:6iJ6NqdoxCDr6mbY8h18oSO+cShGSMRGCEo7F2h0x8s=
github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZXU064P/U=
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 h1:qSGYFH7+jGhDF8vLC+iwCD4WpbV1EBDSzWkJODFLams=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k=
github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE=
github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg=
github.com/gin-gonic/gin v1.9.1/go.mod h1:hPrL7YrpYKXt5YId3A/Tnip5kqbEAP+KLuI3SUcPTeU=
github.com/gabriel-vasile/mimetype v1.4.12 h1:e9hWvmLYvtp846tLHam2o++qitpguFiYCKbn0w9jyqw=
github.com/gabriel-vasile/mimetype v1.4.12/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/glebarez/go-sqlite v1.21.2 h1:3a6LFC4sKahUunAmynQKLZceZCOzUthkRkEAl9gAXWo=
github.com/glebarez/go-sqlite v1.21.2/go.mod h1:sfxdZyhQjTM2Wry3gVYWaW072Ri1WMdWJi0k6+3382k=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
@@ -25,16 +29,17 @@ github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/o
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.14.0 h1:vgvQWe3XCz3gIeFDm/HnTIbj6UGmg/+t63MyGU2n5js=
github.com/go-playground/validator/v10 v10.14.0/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU=
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
github.com/go-sql-driver/mysql v1.7.0/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/go-sql-driver/mysql v1.7.1 h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=
github.com/go-sql-driver/mysql v1.7.1/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/go-sql-driver/mysql v1.10.1 h1:arlSnNLq6a5yxGxV7qg9lF4j0C+KwD6NbQyKr9QL6ME=
github.com/go-sql-driver/mysql v1.10.1/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
@@ -46,62 +51,69 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pelletier/go-toml/v2 v2.0.8 h1:0ctb6s9mE31h0/lhu+J6OPmVeDxJn+kYnJc2jZR9tGQ=
github.com/pelletier/go-toml/v2 v2.0.8/go.mod h1:vuYfssBdrU2XDZ9bYydBu6t+6a6PYNcZljzZR9VXg+4=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
golang.org/x/net v0.46.0 h1:giFlY12I07fugqwPuWJi68oOnpfqFnJIJzaIIm2JVV4=
golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210=
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.30.0 h1:yznKA/E9zq54KzlzBEAWn1NXSQ8DIp/NYMy88xJjl4k=
golang.org/x/text v0.30.0/go.mod h1:yDdHFIX9t+tORqspjENWgzaCVXgk0yYnYuSZ8UzzBVM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
@@ -118,4 +130,3 @@ modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds=
modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/sqlite v1.23.1 h1:nrSBg4aRQQwq59JpvGEQ15tNxoO5pX/kUjcRNwSAGQM=
modernc.org/sqlite v1.23.1/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
+8 -2
View File
@@ -273,7 +273,10 @@ func createBackupArchive(destPath, dbPath, configPath string) error {
}
if strings.TrimSpace(configPath) != "" {
_ = addZipOptionalFile(zipWriter, configPath)
if err := addZipOptionalFile(zipWriter, configPath); err != nil {
_ = zipWriter.Close()
return fmt.Errorf("archive config: %w", err)
}
}
if err := zipWriter.Close(); err != nil {
@@ -355,7 +358,10 @@ func isSQLiteBusyError(err error) bool {
func addZipOptionalFile(writer *zip.Writer, path string) error {
if _, err := os.Stat(path); err != nil {
return nil
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("stat optional backup file: %w", err)
}
return addZipFile(writer, path)
}
+111
View File
@@ -0,0 +1,111 @@
package appstate
import (
"archive/zip"
"fmt"
"io"
"os"
"path/filepath"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// CreatePreResetBackup is mandatory and independent of recurring backup settings.
func CreatePreResetBackup(dbPath, configPath string) (string, error) {
root := filepath.Join(resolveBackupRoot(dbPath), "pre-reset")
if err := validateBackupRoot(root); err != nil {
return "", err
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create pre-reset backup directory: %w", err)
}
resolved, err := filepath.EvalSymlinks(root)
if err != nil {
return "", fmt.Errorf("resolve pre-reset backup directory: %w", err)
}
if err := validateBackupRoot(resolved); err != nil {
return "", err
}
file, err := os.CreateTemp(resolved, "qfs-pre-reset-v1-"+time.Now().UTC().Format("20060102T150405Z")+"-*.zip")
if err != nil {
return "", fmt.Errorf("reserve pre-reset backup: %w", err)
}
path := file.Name()
if err := file.Close(); err != nil {
return "", fmt.Errorf("close pre-reset backup: %w", err)
}
if err := createBackupArchive(path, dbPath, configPath); err != nil {
return "", fmt.Errorf("write pre-reset archive: %w", err)
}
if err := verifyResetArchive(path, filepath.Base(dbPath)); err != nil {
return "", fmt.Errorf("verify pre-reset archive: %w", err)
}
return path, nil
}
func verifyResetArchive(path, dbName string) error {
archive, err := zip.OpenReader(path)
if err != nil {
return fmt.Errorf("open archive: %w", err)
}
defer archive.Close()
dir, err := os.MkdirTemp("", "qfs-verify-reset-*")
if err != nil {
return fmt.Errorf("create verification directory: %w", err)
}
defer os.RemoveAll(dir)
found := false
for _, entry := range archive.File {
in, err := entry.Open()
if err != nil {
return fmt.Errorf("open archive entry: %w", err)
}
var out io.Writer = io.Discard
var restored *os.File
if entry.Name == dbName {
restored, err = os.OpenFile(filepath.Join(dir, "verify.db"), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
in.Close()
return fmt.Errorf("create verification database: %w", err)
}
out = restored
found = true
}
_, copyErr := io.Copy(out, in)
closeErr := in.Close()
if restored != nil {
if err := restored.Close(); err != nil {
return fmt.Errorf("close verification database: %w", err)
}
}
if copyErr != nil {
return fmt.Errorf("read archive entry: %w", copyErr)
}
if closeErr != nil {
return fmt.Errorf("close archive entry: %w", closeErr)
}
}
if !found {
return fmt.Errorf("archive has no database snapshot")
}
db, err := gorm.Open(sqlite.Open(filepath.Join(dir, "verify.db")), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return fmt.Errorf("open restored snapshot: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return fmt.Errorf("get restored database: %w", err)
}
defer sqlDB.Close()
var result string
if err := db.Raw("PRAGMA integrity_check").Scan(&result).Error; err != nil {
return fmt.Errorf("check restored snapshot: %w", err)
}
if result != "ok" {
return fmt.Errorf("snapshot integrity check failed: %s", result)
}
return nil
}
+51
View File
@@ -0,0 +1,51 @@
package appstate
import (
"fmt"
"os"
"path/filepath"
)
// ServerMigrationBackupDir is separate from local backup retention.
func ServerMigrationBackupDir(dbPath string) (string, error) {
root := filepath.Join(resolveBackupRoot(dbPath), "server-migrations")
if err := ValidateBackupLocation(root); err != nil {
return "", err
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create server backup directory: %w", err)
}
resolved, err := filepath.EvalSymlinks(root)
if err != nil {
return "", fmt.Errorf("resolve server backup directory: %w", err)
}
if err := ValidateBackupLocation(resolved); err != nil {
return "", err
}
return resolved, nil
}
// ValidateBackupLocation checks the destination, including existing symlinks.
func ValidateBackupLocation(path string) error {
if err := validateBackupRoot(path); err != nil {
return err
}
existing := path
for {
if _, err := os.Lstat(existing); err == nil {
break
} else if !os.IsNotExist(err) {
return fmt.Errorf("inspect backup path: %w", err)
}
parent := filepath.Dir(existing)
if parent == existing {
return fmt.Errorf("backup path has no existing parent")
}
existing = parent
}
resolved, err := filepath.EvalSymlinks(existing)
if err != nil {
return fmt.Errorf("resolve backup path: %w", err)
}
return validateBackupRoot(resolved)
}
@@ -0,0 +1,29 @@
package appstate
import (
"os"
"path/filepath"
"testing"
)
func TestServerBackupRejectsGitAndSymlinks(t *testing.T) {
root := t.TempDir()
repo := filepath.Join(root, "repo")
if err := os.MkdirAll(filepath.Join(repo, ".git"), 0700); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "link")
if err := os.Symlink(repo, link); err != nil {
t.Fatal(err)
}
for _, path := range []string{filepath.Join(repo, "backups"), filepath.Join(link, "backups")} {
t.Setenv("QFS_BACKUP_DIR", path)
if _, err := ServerMigrationBackupDir(filepath.Join(root, "state", "qfs.db")); err == nil {
t.Fatalf("accepted repository backup: %s", path)
}
}
t.Setenv("QFS_BACKUP_DIR", filepath.Join(root, "backups"))
if _, err := ServerMigrationBackupDir(filepath.Join(root, "state", "qfs.db")); err != nil {
t.Fatal(err)
}
}
+8 -5
View File
@@ -42,14 +42,17 @@ func GroupForLotCategory(cat string) (group Group, ok bool) {
}
}
// ResolveLotCategories returns lot_category for each lotName found in local_pricelist_items
// for the given server pricelist. Lots not found in the pricelist are omitted from the result —
// callers must treat a missing key as "no category" and skip that lot.
func ResolveLotCategories(local *localdb.LocalDB, serverPricelistID uint, lotNames []string) (map[string]string, error) {
// ResolveLotCategories returns lot_category for each lotName, read from the component
// universe (latest active world ∪ estimate pricelists — the same source the configurator,
// BOM and pricing tab use, see bible-local/decisions/2026-07-24-component-universe-world-union.md).
// It must NOT be scoped to a single pricelist: a world-only LOT is a legitimate cart member
// and still carries a real lot_category in the world pricelist. LOTs present in neither
// pricelist are omitted — callers treat a missing key as "no category" and skip that lot.
func ResolveLotCategories(local *localdb.LocalDB, lotNames []string) (map[string]string, error) {
if local == nil {
return nil, fmt.Errorf("local db is nil")
}
cats, err := local.GetLocalLotCategoriesByServerPricelistID(serverPricelistID, lotNames)
cats, err := local.GetLocalComponentCategoriesByLotNames(lotNames)
if err != nil {
return nil, err
}
+51 -2
View File
@@ -20,6 +20,7 @@ func TestResolveLotCategories_MissingLotOmitted(t *testing.T) {
Source: "estimate",
Version: "S-2026-02-11-001",
Name: "test",
IsActive: true,
CreatedAt: time.Now(),
SyncedAt: time.Now(),
}); err != nil {
@@ -35,7 +36,7 @@ func TestResolveLotCategories_MissingLotOmitted(t *testing.T) {
t.Fatalf("save local items: %v", err)
}
cats, err := ResolveLotCategories(local, 1, []string{"CPU_A", "UNKNOWN"})
cats, err := ResolveLotCategories(local, []string{"CPU_A", "UNKNOWN"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -76,7 +77,7 @@ func TestResolveLotCategories_ReturnsKnownCategories(t *testing.T) {
t.Fatalf("save items: %v", err)
}
cats, err := ResolveLotCategories(local, 1, []string{"CPU_B", "MB_X", "NOT_IN_PL"})
cats, err := ResolveLotCategories(local, []string{"CPU_B", "MB_X", "NOT_IN_PL"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -91,6 +92,54 @@ func TestResolveLotCategories_ReturnsKnownCategories(t *testing.T) {
}
}
// TestResolveLotCategories_WorldOnlyLot covers a LOT that exists only in the world
// pricelist (never added to estimate) but is a legitimate cart member. Its real
// lot_category must be resolved from the component universe, not dropped.
func TestResolveLotCategories_WorldOnlyLot(t *testing.T) {
local, err := localdb.New(filepath.Join(t.TempDir(), "local.db"))
if err != nil {
t.Fatalf("init local db: %v", err)
}
t.Cleanup(func() { _ = local.Close() })
saved := func(serverID uint, source string, items []localdb.LocalPricelistItem) {
if err := local.SaveLocalPricelist(&localdb.LocalPricelist{
ServerID: serverID, Source: source, Version: source + "-v1", Name: source,
IsActive: true, CreatedAt: time.Now(), SyncedAt: time.Now(),
}); err != nil {
t.Fatalf("save %s pricelist: %v", source, err)
}
pl, err := local.GetLocalPricelistByServerID(serverID)
if err != nil {
t.Fatalf("get %s pricelist: %v", source, err)
}
for i := range items {
items[i].PricelistID = pl.ID
}
if err := local.SaveLocalPricelistItems(items); err != nil {
t.Fatalf("save %s items: %v", source, err)
}
}
saved(1, "estimate", []localdb.LocalPricelistItem{
{LotName: "CPU_INTEL_8358", LotCategory: "CPU", Price: 1},
})
saved(2, "world", []localdb.LocalPricelistItem{
{LotName: "CPU_INTEL_8358", LotCategory: "CPU", Price: 1},
{LotName: "GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE", LotCategory: "GPU", Price: 9900},
})
cats, err := ResolveLotCategories(local, []string{
"CPU_INTEL_8358", "GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE",
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if cats["GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE"] != "GPU" {
t.Fatalf("world-only LOT category not resolved: %q", cats["GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE"])
}
}
func TestGroupForLotCategory(t *testing.T) {
if g, ok := GroupForLotCategory("cpu"); !ok || g != GroupCPU {
t.Fatalf("expected cpu -> GroupCPU")
+192 -128
View File
@@ -11,15 +11,25 @@ import (
)
type BuildOptions struct {
ServerModel string
ServerPricelist *uint
ServerModel string
}
type BuildResult struct {
Article string
Segments []ResultSegment
Warnings []string
}
// ResultSegment is one dash-separated piece of the article, tagged so the UI can
// highlight pieces the generator could not fully parse. Recognized is false when
// the segment contains a category placeholder (e.g. "GPU" instead of a real model
// token) because a lot_name did not match the expected naming shape.
type ResultSegment struct {
Group string `json:"group"`
Text string `json:"text"`
Recognized bool `json:"recognized"`
}
var (
reMemGiB = regexp.MustCompile(`(?i)(\d+)\s*(GB|G)`)
reMemTiB = regexp.MustCompile(`(?i)(\d+)\s*(TB|T)`)
@@ -33,6 +43,17 @@ var (
type namedSeg struct {
group string // "MODEL","CPU","MEM","GPU","DISK","NET","PSU","SUPPORT"
value string
// degraded marks a segment whose value contains a category placeholder instead
// of a parsed spec token, because some lot_name did not match the expected shape.
degraded bool
}
// segmentResult is what each build*Segment helper returns: the rendered value, a
// degraded flag, and human-readable warnings that name the offending lot_names.
type segmentResult struct {
value string
degraded bool
warnings []string
}
func Build(local *localdb.LocalDB, items []models.ConfigItem, opts BuildOptions) (BuildResult, error) {
@@ -43,71 +64,59 @@ func Build(local *localdb.LocalDB, items []models.ConfigItem, opts BuildOptions)
if model == "" {
return BuildResult{}, fmt.Errorf("server_model required")
}
segs = append(segs, namedSeg{"MODEL", model})
segs = append(segs, namedSeg{group: "MODEL", value: model})
lotNames := make([]string, 0, len(items))
for _, it := range items {
lotNames = append(lotNames, it.LotName)
}
if opts.ServerPricelist == nil || *opts.ServerPricelist == 0 {
return BuildResult{}, fmt.Errorf("pricelist_id required for article")
}
cats, err := ResolveLotCategories(local, *opts.ServerPricelist, lotNames)
cats, err := ResolveLotCategories(local, lotNames)
if err != nil {
return BuildResult{}, err
}
if cpuSeg := buildCPUSegment(items, cats); cpuSeg != "" {
segs = append(segs, namedSeg{"CPU", cpuSeg})
}
memSeg, memWarn := buildMemSegment(items, cats)
if memWarn != "" {
warnings = append(warnings, memWarn)
}
if memSeg != "" {
segs = append(segs, namedSeg{"MEM", memSeg})
}
if gpuSeg := buildGPUSegment(items, cats); gpuSeg != "" {
segs = append(segs, namedSeg{"GPU", gpuSeg})
}
diskSeg, diskWarn := buildDiskSegment(items, cats)
if diskWarn != "" {
warnings = append(warnings, diskWarn)
}
if diskSeg != "" {
segs = append(segs, namedSeg{"DISK", diskSeg})
}
netSeg, netWarn := buildNetSegment(items, cats)
if netWarn != "" {
warnings = append(warnings, netWarn)
}
if netSeg != "" {
segs = append(segs, namedSeg{"NET", netSeg})
}
psuSeg, psuWarn := buildPSUSegment(items, cats)
if psuWarn != "" {
warnings = append(warnings, psuWarn)
}
if psuSeg != "" {
segs = append(segs, namedSeg{"PSU", psuSeg})
for _, sb := range []struct {
group string
build func([]models.ConfigItem, map[string]string) segmentResult
}{
{"CPU", buildCPUSegment},
{"MEM", buildMemSegment},
{"GPU", buildGPUSegment},
{"DISK", buildDiskSegment},
{"NET", buildNetSegment},
{"PSU", buildPSUSegment},
} {
res := sb.build(items, cats)
warnings = append(warnings, res.warnings...)
if res.value != "" {
segs = append(segs, namedSeg{group: sb.group, value: res.value, degraded: res.degraded})
}
}
if supportSeg := buildSupportSegment(items); supportSeg != "" {
segs = append(segs, namedSeg{"SUPPORT", supportSeg})
segs = append(segs, namedSeg{group: "SUPPORT", value: supportSeg})
}
article := strings.Join(namedSegsValues(segs), "-")
if len([]rune(article)) > 80 {
article = compressArticle(segs)
segs = compressArticle(segs)
article = strings.Join(namedSegsValues(segs), "-")
warnings = append(warnings, "compressed")
}
if len([]rune(article)) > 80 {
return BuildResult{}, fmt.Errorf("article_overflow")
}
return BuildResult{Article: article, Warnings: warnings}, nil
result := BuildResult{Article: article, Warnings: warnings}
for _, s := range segs {
result.Segments = append(result.Segments, ResultSegment{
Group: s.group,
Text: s.value,
Recognized: !s.degraded,
})
}
return result, nil
}
func namedSegsValues(segs []namedSeg) []string {
@@ -145,38 +154,43 @@ func buildSupportSegment(items []models.ConfigItem) string {
return ""
}
func buildCPUSegment(items []models.ConfigItem, cats map[string]string) string {
type agg struct {
qty int
// placeholderToken returns the token used in the article when a lot's spec can't be
// parsed: the lot_category itself (never a bare "UNK"), uppercased.
func placeholderToken(cat string) string {
t := strings.ToUpper(strings.TrimSpace(cat))
if t == "" {
return "X"
}
models := map[string]*agg{}
return t
}
func buildCPUSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
models := map[string]int{}
res := segmentResult{}
for _, it := range items {
group, ok := GroupForLotCategory(cats[it.LotName])
if !ok || group != GroupCPU {
continue
}
model := parseCPUModel(it.LotName)
if model == "" {
model = "UNK"
model, parsed := parseCPUModel(it.LotName)
if !parsed {
model = placeholderToken(cats[it.LotName])
res.degraded = true
res.warnings = append(res.warnings, fmt.Sprintf("CPU: не распознана модель LOT %q — в артикул записана категория %q", it.LotName, model))
}
if _, ok := models[model]; !ok {
models[model] = &agg{}
}
models[model].qty += it.Quantity
models[model] += it.Quantity
}
if len(models) == 0 {
return ""
return res
}
parts := make([]string, 0, len(models))
for model, a := range models {
parts = append(parts, fmt.Sprintf("%dx%s", a.qty, model))
}
sort.Strings(parts)
return strings.Join(parts, "+")
res.value = joinQtyTokens(models)
return res
}
func buildMemSegment(items []models.ConfigItem, cats map[string]string) (string, string) {
func buildMemSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
totalGiB := 0
res := segmentResult{}
unparsed := 0
for _, it := range items {
group, ok := GroupForLotCategory(cats[it.LotName])
if !ok || group != GroupMEM {
@@ -184,21 +198,31 @@ func buildMemSegment(items []models.ConfigItem, cats map[string]string) (string,
}
per := parseMemGiB(it.LotName)
if per <= 0 {
return "", "mem_unknown"
unparsed++
res.degraded = true
res.warnings = append(res.warnings, fmt.Sprintf("MEM: не распознан объём LOT %q", it.LotName))
continue
}
totalGiB += per * it.Quantity
}
if totalGiB == 0 {
return "", ""
parts := make([]string, 0, 2)
if totalGiB > 0 {
if totalGiB%1024 == 0 {
parts = append(parts, fmt.Sprintf("%dT", totalGiB/1024))
} else {
parts = append(parts, fmt.Sprintf("%dG", totalGiB))
}
}
if totalGiB%1024 == 0 {
return fmt.Sprintf("%dT", totalGiB/1024), ""
if unparsed > 0 {
parts = append(parts, placeholderToken("MEM"))
}
return fmt.Sprintf("%dG", totalGiB), ""
res.value = strings.Join(parts, "+")
return res
}
func buildGPUSegment(items []models.ConfigItem, cats map[string]string) string {
func buildGPUSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
models := map[string]int{}
res := segmentResult{}
for _, it := range items {
group, ok := GroupForLotCategory(cats[it.LotName])
if !ok || group != GroupGPU {
@@ -207,30 +231,28 @@ func buildGPUSegment(items []models.ConfigItem, cats map[string]string) string {
if strings.HasPrefix(strings.ToUpper(it.LotName), "MB_") {
continue
}
model := parseGPUModel(it.LotName)
if model == "" {
model = "UNK"
model, parsed := parseGPUModel(it.LotName)
if !parsed {
model = placeholderToken(cats[it.LotName])
res.degraded = true
res.warnings = append(res.warnings, fmt.Sprintf("GPU: не распознана модель LOT %q — в артикул записана категория %q", it.LotName, model))
}
models[model] += it.Quantity
}
if len(models) == 0 {
return ""
return res
}
parts := make([]string, 0, len(models))
for model, qty := range models {
parts = append(parts, fmt.Sprintf("%dx%s", qty, model))
}
sort.Strings(parts)
return strings.Join(parts, "+")
res.value = joinQtyTokens(models)
return res
}
func buildDiskSegment(items []models.ConfigItem, cats map[string]string) (string, string) {
func buildDiskSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
type key struct {
t string
c string
}
groupQty := map[key]int{}
warn := ""
res := segmentResult{}
for _, it := range items {
group, ok := GroupForLotCategory(cats[it.LotName])
if !ok || group != GroupDISK {
@@ -238,14 +260,14 @@ func buildDiskSegment(items []models.ConfigItem, cats map[string]string) (string
}
capToken := parseCapacity(it.LotName)
if capToken == "" {
warn = "disk_unknown"
res.degraded = true
res.warnings = append(res.warnings, fmt.Sprintf("DISK: не распознан объём LOT %q", it.LotName))
}
typeCode := diskTypeCode(cats[it.LotName], it.LotName)
k := key{t: typeCode, c: capToken}
groupQty[k] += it.Quantity
groupQty[key{t: typeCode, c: capToken}] += it.Quantity
}
if len(groupQty) == 0 {
return "", ""
return res
}
parts := make([]string, 0, len(groupQty))
for k, qty := range groupQty {
@@ -256,23 +278,25 @@ func buildDiskSegment(items []models.ConfigItem, cats map[string]string) (string
}
}
sort.Strings(parts)
return strings.Join(parts, "+"), warn
res.value = strings.Join(parts, "+")
return res
}
func buildNetSegment(items []models.ConfigItem, cats map[string]string) (string, string) {
return buildProfileSegment(items, cats, GroupNET, parsePortSpeed, "UNKNET", "net_unknown")
func buildNetSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
return buildProfileSegment(items, cats, GroupNET, parsePortSpeed, "NET")
}
func buildPSUSegment(items []models.ConfigItem, cats map[string]string) (string, string) {
return buildProfileSegment(items, cats, GroupPSU, parseWatts, "UNKPSU", "psu_unknown")
func buildPSUSegment(items []models.ConfigItem, cats map[string]string) segmentResult {
return buildProfileSegment(items, cats, GroupPSU, parseWatts, "PSU")
}
// buildProfileSegment groups items of the given category group by a profile token
// parsed from their lot name (e.g. port speed, wattage rating), falling back to
// unknownToken and warnCode when a lot's profile can't be determined.
func buildProfileSegment(items []models.ConfigItem, cats map[string]string, group Group, parseProfile func(string) string, unknownToken, warnCode string) (string, string) {
// parsed from their lot name (e.g. port speed, wattage rating). When a lot's profile
// can't be determined it falls back to the category placeholder and flags the segment
// as degraded, naming the lot in a warning.
func buildProfileSegment(items []models.ConfigItem, cats map[string]string, group Group, parseProfile func(string) string, groupLabel string) segmentResult {
groupQty := map[string]int{}
warn := ""
res := segmentResult{}
for _, it := range items {
g, ok := GroupForLotCategory(cats[it.LotName])
if !ok || g != group {
@@ -280,20 +304,27 @@ func buildProfileSegment(items []models.ConfigItem, cats map[string]string, grou
}
profile := parseProfile(it.LotName)
if profile == "" {
profile = unknownToken
warn = warnCode
profile = placeholderToken(groupLabel)
res.degraded = true
res.warnings = append(res.warnings, fmt.Sprintf("%s: не распознан профиль LOT %q — в артикул записана категория %q", groupLabel, it.LotName, profile))
}
groupQty[profile] += it.Quantity
}
if len(groupQty) == 0 {
return "", ""
return res
}
parts := make([]string, 0, len(groupQty))
for profile, qty := range groupQty {
parts = append(parts, fmt.Sprintf("%dx%s", qty, profile))
res.value = joinQtyTokens(groupQty)
return res
}
// joinQtyTokens renders {token: qty} as a sorted "NxTOKEN+MxTOKEN" string.
func joinQtyTokens(qty map[string]int) string {
parts := make([]string, 0, len(qty))
for token, n := range qty {
parts = append(parts, fmt.Sprintf("%dx%s", n, token))
}
sort.Strings(parts)
return strings.Join(parts, "+"), warn
return strings.Join(parts, "+")
}
func normalizeModelToken(lotName string) string {
@@ -305,18 +336,27 @@ func normalizeModelToken(lotName string) string {
return strings.ToUpper(strings.TrimSpace(token))
}
func parseCPUModel(lotName string) string {
// parseCPUModel extracts the model token from a CPU lot_name (shape
// CPU_{VENDOR}_{MODEL}, e.g. "CPU_INTEL_8592+"). The bool is false when the name
// has no parseable {VENDOR}_{MODEL} tail — a structural failure the caller surfaces.
// It cannot validate that the token is a real CPU model (no vendor catalog is kept
// in the repo by design), only that the name matched the expected shape.
func parseCPUModel(lotName string) (string, bool) {
parts := strings.Split(lotName, "_")
if len(parts) >= 2 {
last := strings.ToUpper(strings.TrimSpace(parts[len(parts)-1]))
if last != "" {
return last
return last, true
}
}
return normalizeModelToken(lotName)
return normalizeModelToken(lotName), false
}
func parseGPUModel(lotName string) string {
// parseGPUModel extracts a "MODEL[_MEM]" token from a GPU lot_name. The bool is
// false when no model token could be located and the result is only the last-ditch
// last-underscore-segment fallback — a structural failure the caller surfaces.
// Like parseCPUModel it does not validate the token against a catalog.
func parseGPUModel(lotName string) (string, bool) {
upper := strings.ToUpper(lotName)
if idx := strings.Index(upper, "GPU_"); idx >= 0 {
upper = upper[idx+4:]
@@ -347,7 +387,7 @@ func parseGPUModel(lotName string) string {
}
if model == "" && i > 0 {
model = p
} else if model != "" && numSuffix == "" && isNumeric(p) {
} else if model != "" && numSuffix == "" && isModelNumber(p) {
numSuffix = p
}
}
@@ -357,20 +397,24 @@ func parseGPUModel(lotName string) string {
full = model + numSuffix
}
if full != "" && mem != "" {
return full + "_" + mem
return full + "_" + mem, true
}
if full != "" {
return full
return full, true
}
return normalizeModelToken(lotName)
return normalizeModelToken(lotName), false
}
func isNumeric(s string) bool {
if s == "" {
// isModelNumber reports whether s looks like a GPU/accelerator model number token:
// it starts with a digit and contains only digits and uppercase letters. This keeps
// vendor-suffixed names like "6000D" (RTX PRO 6000D) intact instead of dropping the
// letter and collapsing two distinct models to the same token.
func isModelNumber(s string) bool {
if s == "" || s[0] < '0' || s[0] > '9' {
return false
}
for _, r := range s {
if r < '0' || r > '9' {
if (r < '0' || r > '9') && (r < 'A' || r > 'Z') {
return false
}
}
@@ -422,9 +466,28 @@ func parsePortSpeed(lotName string) string {
if m := rePortFC.FindStringSubmatch(lotName); len(m) == 3 {
return fmt.Sprintf("%spFC%s", m[1], m[2])
}
if model := parseDPUModel(lotName); model != "" {
return model
}
return ""
}
// parseDPUModel extracts the "{FAMILY}_{MODEL}" tail from a DPU lot_name (shape
// DPU_{FAMILY}_{MODEL}, e.g. "DPU_BF3_B3220-32G" -> "BF3_B3220-32G"). DPU lots have
// no port-speed suffix like NIC/HCA, so they fall through parsePortSpeed's regexes;
// this keeps the family prefix (e.g. BF2 vs BF3) so distinct DPU generations sharing
// a part-number suffix don't collapse into the same article token. Like parseCPUModel
// it does not validate against a vendor catalog, only matches the expected shape.
func parseDPUModel(lotName string) string {
upper := strings.ToUpper(lotName)
const prefix = "DPU_"
if !strings.HasPrefix(upper, prefix) {
return ""
}
rest := strings.TrimSpace(upper[len(prefix):])
return rest
}
func parseWatts(lotName string) string {
if m := reWatts.FindStringSubmatch(lotName); len(m) == 2 {
w := atoi(m[1])
@@ -478,42 +541,43 @@ func atoi(v string) int {
return n
}
func compressArticle(segs []namedSeg) string {
// compressArticle shortens an over-long article by progressively dropping/abbreviating
// segments. It returns the (possibly shortened) segment list; the caller re-joins it.
func compressArticle(segs []namedSeg) []namedSeg {
if len(segs) == 0 {
return ""
return segs
}
fits := func() bool {
return len([]rune(strings.Join(namedSegsValues(segs), "-"))) <= 80
}
for i, s := range segs {
segs[i].value = strings.ReplaceAll(s.value, "GbE", "G")
}
article := strings.Join(namedSegsValues(segs), "-")
if len([]rune(article)) <= 80 {
return article
if fits() {
return segs
}
// 1) remove PSU
if i := findSegGroup(segs, "PSU"); i >= 0 {
segs = append(segs[:i], segs[i+1:]...)
article = strings.Join(namedSegsValues(segs), "-")
if len([]rune(article)) <= 80 {
return article
if fits() {
return segs
}
}
// 2) compress NET/HBA/HCA
if i := findSegGroup(segs, "NET"); i >= 0 {
segs[i].value = compressNetSegment(segs[i].value)
article = strings.Join(namedSegsValues(segs), "-")
if len([]rune(article)) <= 80 {
return article
if fits() {
return segs
}
}
// 3) compress DISK
if i := findSegGroup(segs, "DISK"); i >= 0 {
segs[i].value = compressDiskSegment(segs[i].value)
article = strings.Join(namedSegsValues(segs), "-")
if len([]rune(article)) <= 80 {
return article
if fits() {
return segs
}
}
@@ -521,7 +585,7 @@ func compressArticle(segs []namedSeg) string {
if i := findSegGroup(segs, "GPU"); i >= 0 {
segs[i].value = compressGPUSegment(segs[i].value)
}
return strings.Join(namedSegsValues(segs), "-")
return segs
}
func compressNetSegment(seg string) string {
+121 -4
View File
@@ -47,8 +47,7 @@ func TestBuild_ParsesNetAndPSU(t *testing.T) {
{LotName: "SVC_1yW_x86", Quantity: 1},
}
result, err := Build(local, items, BuildOptions{
ServerModel: "DL380GEN11",
ServerPricelist: &localPL.ServerID,
ServerModel: "DL380GEN11",
})
if err != nil {
t.Fatalf("build article: %v", err)
@@ -118,8 +117,7 @@ func TestBuild_CompressArticle_NoGPU_PSUNotNIC(t *testing.T) {
{LotName: "PS_1500W_Platinum", Quantity: 2},
}
result, err := Build(local, items, BuildOptions{
ServerModel: "NF5280M6",
ServerPricelist: &localPL.ServerID,
ServerModel: "NF5280M6",
})
if err != nil {
t.Fatalf("build article: %v", err)
@@ -140,3 +138,122 @@ func TestBuild_CompressArticle_NoGPU_PSUNotNIC(t *testing.T) {
func contains(s, sub string) bool {
return strings.Contains(s, sub)
}
func TestParsePortSpeed_DPUModel(t *testing.T) {
cases := map[string]string{
"DPU_BF3_B3220-32G": "BF3_B3220-32G",
"DPU_BF2_B2220Q": "BF2_B2220Q",
}
for in, want := range cases {
if got := parsePortSpeed(in); got != want {
t.Errorf("parsePortSpeed(%q) = %q, want %q", in, got, want)
}
}
}
// TestBuild_ParsesDPU verifies a DPU lot (e.g. NVIDIA BlueField) resolves to its
// family+model token instead of falling back to the "NET" category placeholder —
// DPU lots don't carry a NIC/HCA-style port-speed suffix, so parsePortSpeed needs
// its own DPU branch (parseDPUModel).
func TestBuild_ParsesDPU(t *testing.T) {
local, err := localdb.New(filepath.Join(t.TempDir(), "local.db"))
if err != nil {
t.Fatalf("init local db: %v", err)
}
t.Cleanup(func() { _ = local.Close() })
if err := local.SaveLocalPricelist(&localdb.LocalPricelist{
ServerID: 10, Source: "estimate", Version: "v1", Name: "t",
IsActive: true, CreatedAt: time.Now(), SyncedAt: time.Now(),
}); err != nil {
t.Fatalf("save pricelist: %v", err)
}
pl, err := local.GetLocalPricelistByServerID(10)
if err != nil {
t.Fatalf("get pricelist: %v", err)
}
if err := local.SaveLocalPricelistItems([]localdb.LocalPricelistItem{
{PricelistID: pl.ID, LotName: "DPU_BF3_B3220-32G", LotCategory: "DPU", Price: 1},
}); err != nil {
t.Fatalf("save items: %v", err)
}
result, err := Build(local, models.ConfigItems{
{LotName: "DPU_BF3_B3220-32G", Quantity: 1},
}, BuildOptions{ServerModel: "X1"})
if err != nil {
t.Fatalf("build: %v", err)
}
if len(result.Warnings) != 0 {
t.Fatalf("expected no warnings, got %v", result.Warnings)
}
if !contains(result.Article, "BF3_B3220-32G") {
t.Fatalf("expected DPU model token in article: %s", result.Article)
}
}
func TestParseGPUModel_VendorLetterSuffix(t *testing.T) {
cases := map[string]string{
"GPU_NV_RTX_PRO_6000D_SERVER_84GB_PCIE": "RTX6000D_84GB",
"GPU_NV_RTX_PRO_6000_SERVER_96GB_PCIE": "RTX6000_96GB",
}
for in, want := range cases {
got, ok := parseGPUModel(in)
if !ok || got != want {
t.Errorf("parseGPUModel(%q) = %q, %v, want %q, true", in, got, ok, want)
}
}
}
// TestBuild_UnparseableModel_CategoryPlaceholder: a GPU LOT whose name does not
// match the expected shape must not silently vanish or emit "UNK" — the article
// carries the category token and the segment is flagged not-recognized with a
// warning naming the lot.
func TestBuild_UnparseableModel_CategoryPlaceholder(t *testing.T) {
local, err := localdb.New(filepath.Join(t.TempDir(), "local.db"))
if err != nil {
t.Fatalf("init local db: %v", err)
}
t.Cleanup(func() { _ = local.Close() })
if err := local.SaveLocalPricelist(&localdb.LocalPricelist{
ServerID: 9, Source: "estimate", Version: "v1", Name: "t",
IsActive: true, CreatedAt: time.Now(), SyncedAt: time.Now(),
}); err != nil {
t.Fatalf("save pricelist: %v", err)
}
pl, err := local.GetLocalPricelistByServerID(9)
if err != nil {
t.Fatalf("get pricelist: %v", err)
}
if err := local.SaveLocalPricelistItems([]localdb.LocalPricelistItem{
{PricelistID: pl.ID, LotName: "WEIRDGPUNAME", LotCategory: "GPU", Price: 1},
}); err != nil {
t.Fatalf("save items: %v", err)
}
result, err := Build(local, models.ConfigItems{
{LotName: "WEIRDGPUNAME", Quantity: 4},
}, BuildOptions{ServerModel: "X1"})
if err != nil {
t.Fatalf("build: %v", err)
}
if contains(result.Article, "UNK") {
t.Fatalf("article must not contain UNK: %s", result.Article)
}
if !contains(result.Article, "4xGPU") {
t.Fatalf("expected category placeholder 4xGPU in article: %s", result.Article)
}
var gpu *ResultSegment
for i := range result.Segments {
if result.Segments[i].Group == "GPU" {
gpu = &result.Segments[i]
}
}
if gpu == nil || gpu.Recognized {
t.Fatalf("GPU segment must be present and not recognized: %+v", result.Segments)
}
if len(result.Warnings) == 0 || !contains(strings.Join(result.Warnings, "|"), "WEIRDGPUNAME") {
t.Fatalf("expected a warning naming WEIRDGPUNAME, got %v", result.Warnings)
}
}
+111
View File
@@ -0,0 +1,111 @@
// Package configfeatures holds the per-configuration feature flags: markers recording
// which data features a configuration uses, so a client that does not know one of them
// refuses to rewrite the configuration instead of silently dropping data it cannot see.
//
// Flags are derived from content on every save (Detect) and synced as
// qt_configurations.feature_flags. A client checks a config's stored flags against the
// set it knows (Known) before any write. See
// bible-local/decisions/2026-09-24-configuration-feature-flags.md.
package configfeatures
import (
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
)
// SaleRowMarkups: Notes.pricing_ui.sale_row_markups has at least one row markup.
const SaleRowMarkups = "sale_row_markups"
// known lists every flag this build understands. Adding a flag here is the contract for
// "this client can round-trip the data behind it"; a new data feature that an older
// client would lose on save must add a flag here and to Detect in the same change.
var known = map[string]struct{}{
SaleRowMarkups: {},
}
// Known reports whether this build understands flag.
func Known(flag string) bool {
_, ok := known[flag]
return ok
}
// Unknown returns the flags this build does not understand, sorted and de-duplicated.
func Unknown(flags []string) []string {
seen := make(map[string]struct{}, len(flags))
var out []string
for _, f := range flags {
f = strings.TrimSpace(f)
if f == "" || Known(f) {
continue
}
if _, dup := seen[f]; dup {
continue
}
seen[f] = struct{}{}
out = append(out, f)
}
sort.Strings(out)
return out
}
// Detect derives the flags a configuration's content requires. Sorted; nil when none.
func Detect(notes string) []string {
var flags []string
if notesHaveSaleRowMarkups(notes) {
flags = append(flags, SaleRowMarkups)
}
sort.Strings(flags)
return flags
}
func notesHaveSaleRowMarkups(notes string) bool {
if strings.TrimSpace(notes) == "" {
return false
}
var parsed struct {
PricingUI struct {
SaleRowMarkups map[string]float64 `json:"sale_row_markups"`
} `json:"pricing_ui"`
}
if err := json.Unmarshal([]byte(notes), &parsed); err != nil {
return false
}
for _, pct := range parsed.PricingUI.SaleRowMarkups {
if pct != 0 {
return true
}
}
return false
}
// UnsupportedError is returned when a write is refused because the configuration
// carries flags this build does not know.
type UnsupportedError struct {
Flags []string
}
func (e *UnsupportedError) Error() string {
return fmt.Sprintf(
"конфигурация использует функции, неизвестные этой версии QuoteForge (%s) — обновите приложение, чтобы её изменять",
strings.Join(e.Flags, ", "))
}
// Check returns an *UnsupportedError when any of flags is unknown to this build.
func Check(flags []string) error {
if unknown := Unknown(flags); len(unknown) > 0 {
return &UnsupportedError{Flags: unknown}
}
return nil
}
// AsUnsupported unwraps err to an *UnsupportedError, if it is one.
func AsUnsupported(err error) (*UnsupportedError, bool) {
var ue *UnsupportedError
if errors.As(err, &ue) {
return ue, true
}
return nil, false
}
@@ -0,0 +1,50 @@
package configfeatures
import (
"reflect"
"regexp"
"testing"
)
func TestDetectSaleRowMarkups(t *testing.T) {
cases := []struct {
notes string
want []string
}{
{"", nil},
{"not json", nil},
{`{"pricing_ui":{"sale_uplift":1.5,"sale_row_markups":null}}`, nil},
{`{"pricing_ui":{"sale_row_markups":{"NONE::A":0}}}`, nil},
{`{"pricing_ui":{"sale_row_markups":{"NONE::A":-50}}}`, []string{SaleRowMarkups}},
}
for _, c := range cases {
if got := Detect(c.notes); !reflect.DeepEqual(got, c.want) {
t.Errorf("Detect(%q) = %v, want %v", c.notes, got, c.want)
}
}
}
func TestUnknownAndCheck(t *testing.T) {
got := Unknown([]string{"zeta", SaleRowMarkups, "alpha", "zeta", " "})
if want := []string{"alpha", "zeta"}; !reflect.DeepEqual(got, want) {
t.Fatalf("Unknown = %v, want %v", got, want)
}
if err := Check([]string{SaleRowMarkups}); err != nil {
t.Fatalf("Check(known) = %v", err)
}
ue, ok := AsUnsupported(Check([]string{"future_x"}))
if !ok || !reflect.DeepEqual(ue.Flags, []string{"future_x"}) {
t.Fatalf("Check(unknown) = %v, %v", ue, ok)
}
}
// Flag names are permanent and follow bible-local/02-architecture.md
// "Configuration feature flags": lower_snake_case, starts with a letter, ≤ 64 chars.
func TestKnownFlagNamesFormat(t *testing.T) {
valid := regexp.MustCompile(`^[a-z][a-z0-9_]{0,63}$`)
for flag := range known {
if !valid.MatchString(flag) {
t.Errorf("flag %q violates the naming format", flag)
}
}
}
+67 -18
View File
@@ -22,9 +22,10 @@ const (
// goroutines must never pay the MySQL dial/read timeout themselves.
defaultStatusCheckInterval = 15 * time.Second
maxOpenConns = 10
maxOpenConns = 2
maxIdleConns = 2
connMaxLifetime = 5 * time.Minute
connMaxIdleTime = time.Minute
)
// ConnectionStatus represents the current status of the database connection
@@ -49,11 +50,13 @@ type ConnectionManager struct {
statusCheckInterval time.Duration // background prober cadence (default: 15s)
stopStatusLoop chan struct{} // closed by Stop() to end the background loop
dialFn func() (*gorm.DB, error) // opens a new pool; cm.dial unless overridden in tests
}
// NewConnectionManager creates a new ConnectionManager instance
func NewConnectionManager(localDB *localdb.LocalDB) *ConnectionManager {
return &ConnectionManager{
cm := &ConnectionManager{
localDB: localDB,
connectTimeout: defaultConnectTimeout,
pingInterval: defaultPingInterval,
@@ -63,6 +66,8 @@ func NewConnectionManager(localDB *localdb.LocalDB) *ConnectionManager {
lastError: nil,
lastCheck: time.Time{},
}
cm.dialFn = cm.dial
return cm
}
// Start launches a background goroutine that keeps the online-status cache
@@ -137,25 +142,66 @@ func (cm *ConnectionManager) GetDB() (*gorm.DB, error) {
return db, err
}
newDB, err := cm.dial()
// The cached pool is only stale, not broken: ping it instead of dialing a
// new one. Re-dialing here used to orphan the old *sql.DB together with its
// idle connections, leaking a pool every pingInterval during sync.
cm.mu.RLock()
existing := cm.db
cm.mu.RUnlock()
if existing != nil {
if err := cm.ping(existing); err == nil {
cm.mu.Lock()
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
return existing, nil
}
}
newDB, err := cm.dialFn()
cm.mu.Lock()
old := cm.db
if err != nil {
// Drop stale handle so callers don't treat it as an active connection.
cm.db = nil
cm.lastError = err
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return nil, err
}
cm.db = newDB
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return newDB, nil
}
// ping checks an existing pool with the connect timeout. No locks held.
func (cm *ConnectionManager) ping(db *gorm.DB) error {
ctx, cancel := context.WithTimeout(context.Background(), cm.connectTimeout)
defer cancel()
sqlDB, err := db.DB()
if err != nil {
return err
}
return sqlDB.PingContext(ctx)
}
// closeDB releases a pool that is no longer referenced by the manager.
func closeDB(db *gorm.DB) {
if db == nil {
return
}
if sqlDB, err := db.DB(); err == nil {
sqlDB.Close()
}
}
// cachedResult returns (db, err, true) if the cached state is still fresh
// enough to answer without a new network round-trip: either a live
// connection within pingInterval, or a recent failure still within
@@ -200,15 +246,17 @@ func (cm *ConnectionManager) dial() (*gorm.DB, error) {
return nil, fmt.Errorf("getting sql.DB: %w", err)
}
// Ping with timeout
if err = sqlDB.PingContext(ctx); err != nil {
return nil, fmt.Errorf("pinging database: %w", err)
}
// Set connection pool settings
sqlDB.SetMaxOpenConns(maxOpenConns)
sqlDB.SetMaxIdleConns(maxIdleConns)
sqlDB.SetConnMaxLifetime(connMaxLifetime)
sqlDB.SetConnMaxIdleTime(connMaxIdleTime)
// Ping with timeout
if err = sqlDB.PingContext(ctx); err != nil {
sqlDB.Close()
return nil, fmt.Errorf("pinging database: %w", err)
}
return db, nil
}
@@ -264,20 +312,17 @@ func (cm *ConnectionManager) checkOnlineNow() bool {
}
// Perform ping with timeout — no locks held here.
ctx, cancel := context.WithTimeout(context.Background(), cm.connectTimeout)
defer cancel()
sqlDB, err := db.DB()
if err == nil {
err = sqlDB.PingContext(ctx)
}
err := cm.ping(db)
cm.mu.Lock()
defer cm.mu.Unlock()
if err != nil {
cm.lastError = err
cm.lastCheck = time.Now()
cm.db = nil
if cm.db == db {
cm.db = nil
}
closeDB(db)
return false
}
cm.lastCheck = time.Now()
@@ -292,19 +337,23 @@ func (cm *ConnectionManager) TryConnect() error {
cm.connMu.Lock()
defer cm.connMu.Unlock()
newDB, err := cm.dial()
newDB, err := cm.dialFn()
cm.mu.Lock()
defer cm.mu.Unlock()
old := cm.db
if err != nil {
cm.db = nil
cm.lastError = err
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return err
}
cm.db = newDB
cm.lastError = nil
cm.lastCheck = time.Now()
cm.mu.Unlock()
closeDB(old)
return nil
}
+74
View File
@@ -0,0 +1,74 @@
package db
import (
"path/filepath"
"testing"
"time"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func newTestConnectionManager(t *testing.T) (*ConnectionManager, *int) {
t.Helper()
local, err := localdb.New(filepath.Join(t.TempDir(), "qfs.db"))
if err != nil {
t.Fatalf("open local db: %v", err)
}
cm := NewConnectionManager(local)
dials := 0
cm.dialFn = func() (*gorm.DB, error) {
dials++
return gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "remote.db")), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
}
return cm, &dials
}
func TestGetDBReusesStalePoolInsteadOfRedialing(t *testing.T) {
cm, dials := newTestConnectionManager(t)
first, err := cm.GetDB()
if err != nil {
t.Fatalf("first GetDB: %v", err)
}
// Simulate pingInterval elapsing between sync steps.
cm.mu.Lock()
cm.lastCheck = time.Now().Add(-2 * cm.pingInterval)
cm.mu.Unlock()
second, err := cm.GetDB()
if err != nil {
t.Fatalf("second GetDB: %v", err)
}
if second != first {
t.Fatalf("expected stale but healthy pool to be reused")
}
if *dials != 1 {
t.Fatalf("expected 1 dial, got %d", *dials)
}
}
func TestTryConnectClosesReplacedPool(t *testing.T) {
cm, _ := newTestConnectionManager(t)
first, err := cm.GetDB()
if err != nil {
t.Fatalf("GetDB: %v", err)
}
if err := cm.TryConnect(); err != nil {
t.Fatalf("TryConnect: %v", err)
}
sqlDB, err := first.DB()
if err != nil {
t.Fatalf("sql.DB: %v", err)
}
if err := sqlDB.Ping(); err == nil {
t.Fatalf("expected replaced pool to be closed")
}
}
+15 -10
View File
@@ -118,6 +118,11 @@ func (h *ComponentHandler) GetCategories(c *gin.Context) {
c.JSON(http.StatusOK, models.DefaultCategories)
}
// GetConfiguratorSettings serves the synced qt_settings. The default*() values
// below are the only client-side fallback, used per key when the local cache has
// no such setting (e.g. before the first sync); they are not kept in step with
// the server — edit qt_settings in MariaDB, not these. The configurator page has
// no fallback of its own.
func (h *ComponentHandler) GetConfiguratorSettings(c *gin.Context) {
s, _ := h.localDB.GetConfiguratorSettings()
if s == nil {
@@ -131,7 +136,7 @@ func (h *ComponentHandler) GetConfiguratorSettings(c *gin.Context) {
s.TabConfig = defaultTabConfig()
}
if len(s.AlwaysVisibleTabs) == 0 {
s.AlwaysVisibleTabs = []string{"base", "storage", "pci"}
s.AlwaysVisibleTabs = []string{"base"}
}
if len(s.RequiredCategories) == 0 {
s.RequiredCategories = map[string][]string{"server": {"CPU", "MEM", "BB"}}
@@ -158,19 +163,19 @@ func defaultSupportPricing() *localdb.SupportPricingTable {
},
HGXPrice: map[string]map[string]map[string]float64{
"HGX-H200": {
"B": {"1": 20000, "3": 55000, "5": 80000},
"S": {"1": 28000, "3": 70000, "5": 100000},
"P": {"1": 63000, "3": 105000},
"B": {"1": 28000, "3": 77000, "5": 112000},
"S": {"1": 39500, "3": 98000, "5": 140000},
"P": {"1": 88500, "3": 147000},
},
"HGX-B200": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 101000, "3": 176000},
"B": {"1": 158500, "3": 230500, "5": 284500},
"S": {"1": 176500, "3": 266500, "5": 332500},
"P": {"1": 232000, "3": 322000},
},
"HGX-B300": {
"B": {"1": 40000, "3": 100000, "5": 145000},
"S": {"1": 55000, "3": 130000, "5": 185000},
"P": {"1": 112000, "3": 187000},
"B": {"1": 192000, "3": 270000, "5": 328500},
"S": {"1": 211500, "3": 309000, "5": 380500},
"P": {"1": 286000, "3": 383500},
},
},
}
+18
View File
@@ -55,6 +55,15 @@ type ProjectExportOptionsRequest struct {
IncludeCompetitor bool `json:"include_competitor"`
Basis string `json:"basis"` // "fob" or "ddp"
SaleMarkup float64 `json:"sale_markup"` // DDP multiplier; 0 defaults to 1.3
ManualPrice *float64 `json:"manual_price"` // user-defined total price; distributed proportionally across rows
// MinimalSaleColumns/SaleRowMarkups are sent only by the single-config Sale-table
// "Экспорт CSV" button (index.html exportPricingCSV('sale')) — never by the Buy
// export or the project-level bulk export, which always leave these zero-valued so
// their output stays unchanged. See services.ProjectPricingExportOptions.
MinimalSaleColumns bool `json:"minimal_sale_columns"`
SaleRowMarkups map[string]float64 `json:"sale_row_markups"` // "<vendorPN>::<lot>" -> markup percent
}
func (h *ExportHandler) ExportCSV(c *gin.Context) {
@@ -254,6 +263,15 @@ func (h *ExportHandler) ExportConfigPricingCSV(c *gin.Context) {
Basis: req.Basis,
SaleMarkup: req.SaleMarkup,
}
// MinimalSaleColumns/SaleRowMarkups/ManualPrice are only ever sent by the Sale
// table's own "Экспорт CSV" button (index.html exportPricingCSV('sale')) — gating
// on the request's own flag keeps every other caller (Buy export, any other basis=
// ddp caller) byte-identical to before this field existed.
if req.MinimalSaleColumns {
opts.MinimalSaleColumns = true
opts.SaleRowMarkups = req.SaleRowMarkups
opts.ManualPrice = req.ManualPrice
}
data, err := h.exportService.ConfigToPricingExportData(config, opts)
if err != nil {
+11
View File
@@ -7,6 +7,7 @@ import (
"net/http"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/configfeatures"
"github.com/gin-gonic/gin"
)
@@ -14,6 +15,16 @@ func RespondError(c *gin.Context, status int, fallback string, err error) {
if err != nil {
_ = c.Error(err)
}
// A write refused over unknown configuration feature flags is a 409 with the flag list,
// whatever status the caller picked — every config write path funnels through here.
if ue, ok := configfeatures.AsUnsupported(err); ok {
c.JSON(http.StatusConflict, gin.H{
"error": ue.Error(),
"code": "unsupported_config_features",
"unsupported_features": ue.Flags,
})
return
}
c.JSON(status, gin.H{"error": clientFacingErrorMessage(status, fallback, err)})
}
+29
View File
@@ -2,7 +2,13 @@ package handlers
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/configfeatures"
"github.com/gin-gonic/gin"
)
func TestClientFacingErrorMessageKeepsDomain4xx(t *testing.T) {
@@ -39,3 +45,26 @@ type errString string
func (e errString) Error() string {
return string(e)
}
func TestRespondErrorUnsupportedFeaturesIs409(t *testing.T) {
gin.SetMode(gin.TestMode)
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
err := fmt.Errorf("wrapped: %w", configfeatures.Check([]string{"future_x"}))
RespondError(c, http.StatusInternalServerError, "internal server error", err)
if w.Code != http.StatusConflict {
t.Fatalf("status = %d, want 409", w.Code)
}
var body struct {
Code string `json:"code"`
Features []string `json:"unsupported_features"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("decode body: %v", err)
}
if body.Code != "unsupported_config_features" || len(body.Features) != 1 || body.Features[0] != "future_x" {
t.Fatalf("unexpected body %s", w.Body.String())
}
}
+47
View File
@@ -0,0 +1,47 @@
package handlers
import (
"net/http"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"github.com/gin-gonic/gin"
)
// SettingsHandler serves per-client UI display preferences stored in app_settings.
type SettingsHandler struct {
localDB *localdb.LocalDB
}
func NewSettingsHandler(localDB *localdb.LocalDB) *SettingsHandler {
return &SettingsHandler{localDB: localDB}
}
type uiSettingsResponse struct {
IndicatorMode string `json:"indicator_mode"`
}
// GetUI returns the current UI display preferences.
func (h *SettingsHandler) GetUI(c *gin.Context) {
c.JSON(http.StatusOK, uiSettingsResponse{
IndicatorMode: h.localDB.GetIndicatorMode(),
})
}
// PutUI updates UI display preferences. Currently only indicator_mode.
func (h *SettingsHandler) PutUI(c *gin.Context) {
req, ok := BindJSON[struct {
IndicatorMode string `json:"indicator_mode"`
}](c)
if !ok {
return
}
if err := h.localDB.SetIndicatorMode(req.IndicatorMode); err != nil {
RespondError(c, http.StatusUnprocessableEntity, "invalid indicator_mode", err)
return
}
c.JSON(http.StatusOK, uiSettingsResponse{
IndicatorMode: h.localDB.GetIndicatorMode(),
})
}
+57 -18
View File
@@ -1,6 +1,7 @@
package handlers
import (
"errors"
"fmt"
"html/template"
"log/slog"
@@ -10,10 +11,13 @@ import (
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/appmeta"
"git.mchus.pro/mchus/quoteforge/internal/db"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"github.com/gin-gonic/gin"
mysqlDriver "github.com/go-sql-driver/mysql"
"gorm.io/gorm"
)
type SetupHandler struct {
@@ -56,7 +60,9 @@ func (h *SetupHandler) ShowSetup(c *gin.Context) {
settings, _ := h.localDB.GetSettings()
data := gin.H{
"Settings": settings,
"Settings": settings,
"IndicatorMode": h.localDB.GetIndicatorMode(),
"AppVersion": appmeta.Version(),
}
tmpl := h.templates["setup.html"]
@@ -79,14 +85,24 @@ func (h *SetupHandler) TestConnection(c *gin.Context) {
port = p
}
// If password is empty, try to use saved password
if password == "" {
if settings, err := h.localDB.GetSettings(); err == nil && settings != nil {
password = settings.PasswordEncrypted // GetSettings returns decrypted password in this field
}
settings, err := h.localDB.GetSettings()
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
_ = c.Error(err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read settings"})
return
}
password, err = setupPassword(settings, host, port, user, password)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
dsn := buildMySQLDSN(host, port, database, user, password, 5*time.Second)
tlsMode, caPEM := c.DefaultPostForm("tls_mode", "preferred"), c.PostForm("tls_ca_pem")
dsn, err := buildMySQLDSNWithTLS(host, port, database, user, password, 5*time.Second, tlsMode, caPEM)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
lotCount, canWrite, err := db.ValidateMariaDBConnection(dsn)
if err != nil {
_ = c.Error(err)
@@ -107,7 +123,12 @@ func (h *SetupHandler) TestConnection(c *gin.Context) {
// SaveConnection saves the connection settings and signals restart
func (h *SetupHandler) SaveConnection(c *gin.Context) {
existingSettings, _ := h.localDB.GetSettings()
existingSettings, settingsErr := h.localDB.GetSettings()
if settingsErr != nil && !errors.Is(settingsErr, gorm.ErrRecordNotFound) {
_ = c.Error(settingsErr)
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to read settings"})
return
}
host := c.PostForm("host")
portStr := c.PostForm("port")
@@ -120,15 +141,19 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
port = p
}
// If password is empty, use saved password
if password == "" {
if settings, err := h.localDB.GetSettings(); err == nil && settings != nil {
password = settings.PasswordEncrypted // GetSettings returns decrypted password in this field
}
password, err := setupPassword(existingSettings, host, port, user, password)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
// Test connection first
dsn := buildMySQLDSN(host, port, database, user, password, 5*time.Second)
tlsMode, caPEM := c.DefaultPostForm("tls_mode", "preferred"), c.PostForm("tls_ca_pem")
dsn, err := buildMySQLDSNWithTLS(host, port, database, user, password, 5*time.Second, tlsMode, caPEM)
if err != nil {
c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()})
return
}
if _, _, err := db.ValidateMariaDBConnection(dsn); err != nil {
_ = c.Error(err)
c.JSON(http.StatusBadRequest, gin.H{
@@ -139,7 +164,7 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
}
// Save settings
if err := h.localDB.SaveSettings(host, port, database, user, password); err != nil {
if err := h.localDB.SaveSettingsWithTLS(host, port, database, user, password, tlsMode, caPEM); err != nil {
_ = c.Error(err)
c.JSON(http.StatusInternalServerError, gin.H{
"success": false,
@@ -162,7 +187,8 @@ func (h *SetupHandler) SaveConnection(c *gin.Context) {
existingSettings.Port != port ||
existingSettings.Database != database ||
existingSettings.User != user ||
existingSettings.PasswordEncrypted != password
existingSettings.PasswordEncrypted != password ||
existingSettings.TLSMode != tlsMode || existingSettings.TLSCAPEM != caPEM
restartQueued := settingsChanged && h.restartSig != nil
c.JSON(http.StatusOK, gin.H{
@@ -192,7 +218,7 @@ func (h *SetupHandler) GetStatus(c *gin.Context) {
})
}
func buildMySQLDSN(host string, port int, database, user, password string, timeout time.Duration) string {
func buildMySQLDSNWithTLS(host string, port int, database, user, password string, timeout time.Duration, tlsMode, caPEM string) (string, error) {
cfg := mysqlDriver.NewConfig()
cfg.User = user
cfg.Passwd = password
@@ -205,6 +231,19 @@ func buildMySQLDSN(host string, port int, database, user, password string, timeo
cfg.Params = map[string]string{
"charset": "utf8mb4",
}
return cfg.FormatDSN()
if err := mariadb.ConfigureTLS(cfg, tlsMode, caPEM); err != nil {
return "", fmt.Errorf("configure TLS: %w", err)
}
return cfg.FormatDSN(), nil
}
// setupPassword reuses a credential only for its saved destination and account.
func setupPassword(saved *localdb.ConnectionSettings, host string, port int, user, password string) (string, error) {
if password != "" || saved == nil {
return password, nil
}
if saved.Host != host || saved.Port != port || saved.User != user {
return "", fmt.Errorf("Password is required when changing host, port or user")
}
return saved.PasswordEncrypted, nil
}
+31
View File
@@ -0,0 +1,31 @@
package handlers
import (
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"testing"
)
func TestSetupPassword(t *testing.T) {
saved := &localdb.ConnectionSettings{Host: "db.example", Port: 3306, User: "example", PasswordEncrypted: "example"}
for _, tt := range []struct {
name, host, user, password string
port int
wantError bool
}{
{"same destination", "db.example", "example", "", 3306, false},
{"changed host", "other.example", "example", "", 3306, true},
{"changed port", "db.example", "example", "", 3307, true},
{"changed user", "db.example", "other", "", 3306, true},
{"explicit password", "other.example", "other", "example", 3307, false},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := setupPassword(saved, tt.host, tt.port, tt.user, tt.password)
if (err != nil) != tt.wantError {
t.Fatalf("unexpected error: %v", err)
}
if !tt.wantError && got != "example" {
t.Fatal("credential not preserved")
}
})
}
}
+37
View File
@@ -0,0 +1,37 @@
package handlers
import (
"net/http"
"net/http/httptest"
"net/url"
"path/filepath"
"strings"
"testing"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"github.com/gin-gonic/gin"
)
func TestSetupFreshSettingsValidateTLS(t *testing.T) {
local, err := localdb.New(filepath.Join(t.TempDir(), "qfs.db"))
if err != nil {
t.Fatal(err)
}
defer local.Close()
handler := &SetupHandler{localDB: local}
for _, test := range []struct {
path string
handler gin.HandlerFunc
}{{"/setup/test", handler.TestConnection}, {"/setup", handler.SaveConnection}} {
router := gin.New()
router.POST(test.path, test.handler)
body := url.Values{"host": {"db.example"}, "user": {"example"}, "tls_mode": {"invalid"}}.Encode()
req := httptest.NewRequest(http.MethodPost, test.path, strings.NewReader(body))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusUnprocessableEntity {
t.Fatalf("%s: status=%d body=%s", test.path, w.Code, w.Body.String())
}
}
}
+8 -2
View File
@@ -43,7 +43,7 @@ func NewSyncHandler(localDB *localdb.LocalDB, syncService *sync.Service, connMgr
syncService: syncService,
connMgr: connMgr,
autoSyncInterval: autoSyncInterval,
onlineGraceFactor: 1.10,
onlineGraceFactor: 1 + sync.WorkerJitter, // longest gap between jittered sync cycles
tmpl: tmpl,
}, nil
}
@@ -240,7 +240,7 @@ type SyncAllResponse struct {
// SyncAll performs full bidirectional sync:
// - push pending local changes (projects/configurations) to server
// - pull components, pricelists, projects, and configurations from server
// - pull components, pricelists, qt_settings, projects, and configurations from server
// POST /api/sync/all
func (h *SyncHandler) SyncAll(c *gin.Context) {
if !h.ensureSyncReadiness(c) {
@@ -282,6 +282,12 @@ func (h *SyncHandler) SyncAll(c *gin.Context) {
slog.Warn("partnumber books pull failed during full sync", "error", err)
}
// Configurator settings (config_types, tab_config, ...): non-fatal, the
// local cache is kept on failure.
if err := h.syncService.SyncQtSettings(); err != nil {
slog.Warn("qt_settings sync failed during full sync", "error", err)
}
projectsResult, err := h.syncService.ImportProjectsToLocal()
if err != nil {
slog.Error("project import failed during full sync", "error", err)
+4
View File
@@ -114,6 +114,10 @@ func NewWebHandler(_ string, localDB *localdb.LocalDB) (*WebHandler, error) {
func (h *WebHandler) render(c *gin.Context, name string, data gin.H) {
data["AppVersion"] = appmeta.Version()
data["IndicatorMode"] = localdb.IndicatorModeColor
if h.localDB != nil {
data["IndicatorMode"] = h.localDB.GetIndicatorMode()
}
c.Header("Content-Type", "text/html; charset=utf-8")
tmpl, ok := h.templates[name]
if !ok {
+4
View File
@@ -3,6 +3,7 @@ package localdb
import (
"time"
"git.mchus.pro/mchus/quoteforge/internal/configfeatures"
"git.mchus.pro/mchus/quoteforge/internal/models"
)
@@ -62,6 +63,7 @@ func ConfigurationToLocal(cfg *models.Configuration) *LocalConfiguration {
TotalPrice: cfg.TotalPrice,
CustomPrice: cfg.CustomPrice,
Notes: cfg.Notes,
FeatureFlags: cfg.FeatureFlags,
IsTemplate: cfg.IsTemplate,
ServerCount: cfg.ServerCount,
ServerModel: cfg.ServerModel,
@@ -112,6 +114,8 @@ func LocalToConfiguration(local *LocalConfiguration) *models.Configuration {
TotalPrice: local.TotalPrice,
CustomPrice: local.CustomPrice,
Notes: local.Notes,
FeatureFlags: local.FeatureFlags,
UnsupportedFeatures: configfeatures.Unknown(local.FeatureFlags),
IsTemplate: local.IsTemplate,
ServerCount: local.ServerCount,
ServerModel: local.ServerModel,
+45
View File
@@ -0,0 +1,45 @@
package localdb
import (
"path/filepath"
"testing"
)
func newModeDB(t *testing.T) *LocalDB {
t.Helper()
local, err := New(filepath.Join(t.TempDir(), "mode.db"))
if err != nil {
t.Fatalf("open localdb: %v", err)
}
t.Cleanup(func() { _ = local.Close() })
return local
}
func TestIndicatorMode(t *testing.T) {
local := newModeDB(t)
if got := local.GetIndicatorMode(); got != IndicatorModeColor {
t.Fatalf("unset: got %q, want %q", got, IndicatorModeColor)
}
if err := local.SetIndicatorMode(IndicatorModeAccessible); err != nil {
t.Fatalf("set accessible: %v", err)
}
if got := local.GetIndicatorMode(); got != IndicatorModeAccessible {
t.Fatalf("after set accessible: got %q, want %q", got, IndicatorModeAccessible)
}
if err := local.SetIndicatorMode(IndicatorModeColor); err != nil {
t.Fatalf("set color: %v", err)
}
if got := local.GetIndicatorMode(); got != IndicatorModeColor {
t.Fatalf("after set color: got %q, want %q", got, IndicatorModeColor)
}
if err := local.SetIndicatorMode("rainbow"); err == nil {
t.Fatal("expected error for invalid mode, got nil")
}
if got := local.GetIndicatorMode(); got != IndicatorModeColor {
t.Fatalf("after rejected set: got %q, want %q", got, IndicatorModeColor)
}
}
+64 -44
View File
@@ -14,6 +14,7 @@ import (
"git.mchus.pro/mchus/quoteforge/internal/appmeta"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"github.com/glebarez/sqlite"
mysqlDriver "github.com/go-sql-driver/mysql"
uuidpkg "github.com/google/uuid"
@@ -24,6 +25,8 @@ import (
// ConnectionSettings stores MariaDB connection credentials
type ConnectionSettings struct {
TLSMode string `gorm:"default:preferred"`
TLSCAPEM string
ID uint `gorm:"primaryKey"`
Host string `gorm:"not null"`
Port int `gorm:"not null;default:3306"`
@@ -50,47 +53,6 @@ var localReadOnlyCacheTables = []string{
"local_partnumber_books",
}
// ResetData clears local data tables while keeping connection settings.
// It does not drop schema or connection_settings.
func ResetData(dbPath string) error {
if strings.TrimSpace(dbPath) == "" {
return nil
}
if _, err := os.Stat(dbPath); err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil
}
return fmt.Errorf("stat local db: %w", err)
}
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
return fmt.Errorf("opening sqlite database: %w", err)
}
// Order does not matter because we use DELETEs without FK constraints in SQLite.
tables := []string{
"local_projects",
"local_configurations",
"local_configuration_versions",
"local_pricelists",
"local_pricelist_items",
"local_sync_guard_state",
"pending_changes",
"app_settings",
}
for _, table := range tables {
if err := db.Exec("DELETE FROM " + table).Error; err != nil {
return fmt.Errorf("clear %s: %w", table, err)
}
}
slog.Info("local database data reset", "path", dbPath)
return nil
}
// New creates a new LocalDB instance
func New(dbPath string) (*LocalDB, error) {
// Ensure directory exists
@@ -548,6 +510,14 @@ func (l *LocalDB) GetSettings() (*ConnectionSettings, error) {
// SaveSettings saves connection settings with encrypted password
func (l *LocalDB) SaveSettings(host string, port int, database, user, password string) error {
return l.SaveSettingsWithTLS(host, port, database, user, password, "preferred", "")
}
func (l *LocalDB) SaveSettingsWithTLS(host string, port int, database, user, password, tlsMode, caPEM string) error {
if err := mariadb.ConfigureTLS(mysqlDriver.NewConfig(), tlsMode, caPEM); err != nil {
return fmt.Errorf("validate connection TLS: %w", err)
}
// Encrypt password
encrypted, err := Encrypt(password)
if err != nil {
@@ -556,6 +526,8 @@ func (l *LocalDB) SaveSettings(host string, port int, database, user, password s
settings := ConnectionSettings{
ID: 1, // Always use ID=1 for single settings row
TLSMode: tlsMode,
TLSCAPEM: caPEM,
Host: host,
Port: port,
Database: database,
@@ -601,6 +573,9 @@ func (l *LocalDB) GetDSN() (string, error) {
"charset": "utf8mb4",
}
if err := mariadb.ConfigureTLS(cfg, settings.TLSMode, settings.TLSCAPEM); err != nil {
return "", fmt.Errorf("configure connection TLS: %w", err)
}
return cfg.FormatDSN(), nil
}
@@ -1130,6 +1105,43 @@ func (l *LocalDB) upsertAppSetting(tx *gorm.DB, key, value string, updatedAt tim
`, key, value, updatedAt.Format(time.RFC3339)).Error
}
// Indicator display mode: how the UI renders signals that are otherwise carried
// by colour alone (price-quality scale, world-fallback price cells, incomplete
// pricing totals). "color" (default) keeps the hue coding; "accessible" swaps it
// for shape/text so it stays readable with a colour-vision deficiency. Per-client
// preference stored in app_settings; consumed by web/static/price-quality.js and
// the pricing tab via base.html's window.QF_INDICATOR_MODE.
const (
IndicatorModeColor = "color"
IndicatorModeAccessible = "accessible"
)
// GetIndicatorMode returns the stored indicator display mode, falling back to
// "color" when unset or invalid.
func (l *LocalDB) GetIndicatorMode() string {
value, ok := l.getAppSettingValue("indicator_mode")
if !ok {
return IndicatorModeColor
}
switch strings.TrimSpace(value) {
case IndicatorModeAccessible:
return IndicatorModeAccessible
default:
return IndicatorModeColor
}
}
// SetIndicatorMode stores the indicator display mode. Only the two known
// literals are accepted.
func (l *LocalDB) SetIndicatorMode(mode string) error {
mode = strings.TrimSpace(mode)
if mode != IndicatorModeColor && mode != IndicatorModeAccessible {
return fmt.Errorf("invalid indicator mode %q", mode)
}
now := time.Now()
return l.upsertAppSetting(l.db, "indicator_mode", mode, now)
}
// SetLastSyncTime sets the last sync timestamp
func (l *LocalDB) SetLastSyncTime(t time.Time) error {
// Using raw SQL for upsert since SQLite doesn't have native UPSERT in all versions
@@ -1248,7 +1260,6 @@ func (l *LocalDB) GetLastComponentSyncError() string {
return strings.TrimSpace(value)
}
const openConfigUUIDsKey = "open_config_uuids"
// GetOpenConfigUUIDs returns UUIDs of all configurations currently open in the configurator.
@@ -1324,7 +1335,6 @@ func (l *LocalDB) CountAllPricelistItems() int64 {
return count
}
// DBFilePath returns the path to the SQLite database file.
func (l *LocalDB) DBFilePath() string {
return l.path
@@ -1825,6 +1835,17 @@ func (l *LocalDB) MarkChangesSynced(ids []int64) error {
return l.db.Where("id IN ?", ids).Delete(&PendingChange{}).Error
}
// ListConfigurationsPendingWithoutQueue returns configurations still marked pending
// whose pending_changes rows are gone, so nothing will ever push them.
func (l *LocalDB) ListConfigurationsPendingWithoutQueue() ([]LocalConfiguration, error) {
var configs []LocalConfiguration
err := l.db.Where(
"sync_status = ? AND uuid NOT IN (SELECT entity_uuid FROM pending_changes WHERE entity_type = ?)",
"pending", "configuration",
).Find(&configs).Error
return configs, err
}
// PurgeOrphanConfigurationPendingChanges removes configuration pending changes
// whose entity_uuid no longer exists in local_configurations.
func (l *LocalDB) PurgeOrphanConfigurationPendingChanges() (int64, error) {
@@ -2047,7 +2068,6 @@ func (l *LocalDB) GetLocalPricelistItemsPage(pricelistID uint, search string, pa
return items, total, nil
}
// GetSchemaMigrations returns all applied local schema migrations ordered by applied_at.
func (l *LocalDB) GetSchemaMigrations() ([]LocalSchemaMigration, error) {
var migrations []LocalSchemaMigration
+1
View File
@@ -102,6 +102,7 @@ type LocalConfiguration struct {
TotalPrice *float64 `json:"total_price"`
CustomPrice *float64 `json:"custom_price"`
Notes string `json:"notes"`
FeatureFlags models.FeatureFlags `gorm:"type:text" json:"feature_flags,omitempty"`
IsTemplate bool `gorm:"default:false" json:"is_template"`
ServerCount int `gorm:"default:1" json:"server_count"`
ServerModel string `gorm:"size:100" json:"server_model,omitempty"`
+51
View File
@@ -0,0 +1,51 @@
package localdb
import (
"encoding/json"
"fmt"
"time"
)
// serverPullCursor is the server-side NOW(6) taken before the last successful
// incremental pull of a table, bound to the server it was taken from.
type serverPullCursor struct {
Scope string `json:"scope"`
ServerTime string `json:"server_time"`
}
func serverPullCursorKey(table string) string {
return "server_pull_cursor:" + table
}
// ServerPullScope identifies the MariaDB server+database a pull cursor belongs to,
// so a cursor never survives a switch to another server.
func (l *LocalDB) ServerPullScope() (string, error) {
settings, err := l.GetSettings()
if err != nil {
return "", err
}
return fmt.Sprintf("%s:%d/%s", settings.Host, settings.Port, settings.Database), nil
}
// GetServerPullCursor returns the stored server time for table if it was taken
// on the server identified by scope.
func (l *LocalDB) GetServerPullCursor(table, scope string) (string, bool) {
raw, ok := l.getAppSettingValue(serverPullCursorKey(table))
if !ok {
return "", false
}
var cursor serverPullCursor
if err := json.Unmarshal([]byte(raw), &cursor); err != nil || cursor.Scope != scope || cursor.ServerTime == "" {
return "", false
}
return cursor.ServerTime, true
}
// SetServerPullCursor stores the server time a pull of table started at.
func (l *LocalDB) SetServerPullCursor(table, scope, serverTime string) error {
raw, err := json.Marshal(serverPullCursor{Scope: scope, ServerTime: serverTime})
if err != nil {
return err
}
return l.upsertAppSetting(l.db, serverPullCursorKey(table), string(raw), time.Now())
}
+21
View File
@@ -72,6 +72,11 @@ func (l *LocalDB) SyncQtSettings(mariaDB *gorm.DB) error {
return nil
}
var current []LocalQtSetting
if err := l.db.Select("name, value").Find(&current).Error; err == nil && sameQtSettings(current, rows) {
return nil
}
return l.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Exec("DELETE FROM local_qt_settings").Error; err != nil {
return fmt.Errorf("clearing local_qt_settings: %w", err)
@@ -142,3 +147,19 @@ func (l *LocalDB) GetConfiguratorSettings() (*ConfiguratorSettings, error) {
return out, nil
}
func sameQtSettings(a, b []LocalQtSetting) bool {
if len(a) != len(b) {
return false
}
values := make(map[string]string, len(a))
for _, row := range a {
values[row.Name] = row.Value
}
for _, row := range b {
if v, ok := values[row.Name]; !ok || v != row.Value {
return false
}
}
return true
}
+85
View File
@@ -0,0 +1,85 @@
package localdb
import (
"fmt"
"log/slog"
"os"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func ResetData(dbPath string) error {
configPath, err := appstate.ResolveConfigPathNearDB("", dbPath)
if err != nil {
return fmt.Errorf("resolve reset config: %w", err)
}
return ResetDataWithOptions(dbPath, configPath, false)
}
// ResetDataWithOptions requires a fresh verified backup even for forced resets.
func ResetDataWithOptions(dbPath, configPath string, force bool) error {
if strings.TrimSpace(dbPath) == "" {
return nil
}
if _, err := os.Stat(dbPath); err != nil {
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("stat reset database: %w", err)
}
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return fmt.Errorf("open reset database: %w", err)
}
sqlDB, err := db.DB()
if err != nil {
return fmt.Errorf("get reset database: %w", err)
}
defer sqlDB.Close()
return db.Transaction(func(tx *gorm.DB) error {
for _, table := range []string{"pending_changes", "local_projects", "local_configurations"} {
if !tx.Migrator().HasTable(table) {
continue
}
query := tx.Table(table)
if table != "pending_changes" {
if !tx.Migrator().HasColumn(table, "sync_status") {
if !force {
return fmt.Errorf("reset refused: %s has no sync_status; use -reset-localdb -force-reset-localdb after reviewing local work", table)
}
continue
}
query = query.Where("sync_status IS NULL OR sync_status <> ?", "synced")
}
var count int64
if err := query.Count(&count).Error; err != nil {
return fmt.Errorf("check reset data in %s: %w", table, err)
}
if count > 0 && !force {
return fmt.Errorf("reset refused: %s contains %d unsynced rows; sync first or use -reset-localdb -force-reset-localdb", table, count)
}
}
archivePath, err := appstate.CreatePreResetBackup(dbPath, configPath)
if err != nil {
return fmt.Errorf("reset safety backup: %w", err)
}
slog.Info("verified pre-reset backup", "path", archivePath)
for _, table := range []string{
"local_projects", "local_configurations", "local_configuration_versions",
"local_pricelists", "local_pricelist_items", "local_sync_guard_state", "pending_changes", "app_settings",
} {
if !tx.Migrator().HasTable(table) {
continue
}
if err := tx.Exec("DELETE FROM " + table).Error; err != nil {
return fmt.Errorf("clear %s: %w", table, err)
}
}
slog.Info("local database data reset", "path", dbPath, "forced", force)
return nil
})
}
+154
View File
@@ -0,0 +1,154 @@
package localdb
import (
"archive/zip"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestResetProtectsUnsyncedData(t *testing.T) {
for _, table := range []string{"pending_changes", "local_projects", "local_configurations"} {
t.Run(table, func(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("INSERT INTO " + table + " (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
err := ResetDataWithOptions(path, "", false)
if err == nil || !strings.Contains(err.Error(), table) {
t.Fatalf("expected refusal naming %s, got %v", table, err)
}
assertResetRows(t, db, table, 1)
})
}
}
func TestResetBackupAndRollback(t *testing.T) {
for _, failDelete := range []bool{false, true} {
t.Run(map[bool]string{false: "reset", true: "rollback"}[failDelete], func(t *testing.T) {
path, db := resetFixture(t)
t.Setenv("QFS_BACKUP_DISABLE", "1")
if err := db.Exec("INSERT INTO local_projects (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
if failDelete {
if err := db.Exec("CREATE TRIGGER block_reset BEFORE DELETE ON local_configurations BEGIN SELECT RAISE(ABORT, 'blocked'); END").Error; err != nil {
t.Fatal(err)
}
if err := db.Exec("INSERT INTO local_configurations (sync_status) VALUES ('pending')").Error; err != nil {
t.Fatal(err)
}
}
cfg := filepath.Join(filepath.Dir(path), "config.yaml")
if err := os.WriteFile(cfg, []byte("server: {}"), 0600); err != nil {
t.Fatal(err)
}
err := ResetDataWithOptions(path, cfg, true)
if (err != nil) != failDelete {
t.Fatalf("reset: %v", err)
}
want := int64(0)
if failDelete {
want = 1
}
assertResetRows(t, db, "local_projects", want)
archives, err := filepath.Glob(filepath.Join(filepath.Dir(path), "backups", "pre-reset", "*.zip"))
if err != nil || len(archives) != 1 {
t.Fatalf("archives=%v err=%v", archives, err)
}
archive, err := zip.OpenReader(archives[0])
if err != nil {
t.Fatal(err)
}
defer archive.Close()
for _, entry := range archive.File {
if entry.Name != filepath.Base(path) {
continue
}
in, err := entry.Open()
if err != nil {
t.Fatal(err)
}
data, err := io.ReadAll(in)
in.Close()
if err != nil {
t.Fatal(err)
}
restoredPath := filepath.Join(t.TempDir(), "restored.db")
if err := os.WriteFile(restoredPath, data, 0600); err != nil {
t.Fatal(err)
}
restored, err := gorm.Open(sqlite.Open(restoredPath), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := restored.DB()
defer sqlDB.Close()
assertResetRows(t, restored, "local_projects", 1)
}
assertResetRows(t, db, "connection_settings", 1)
})
}
}
func TestResetRefusesFailedBackup(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("INSERT INTO local_projects (sync_status) VALUES ('synced')").Error; err != nil {
t.Fatal(err)
}
t.Setenv("QFS_BACKUP_DIR", path) // A database file cannot be a backup directory.
if err := ResetDataWithOptions(path, "", true); err == nil {
t.Fatal("expected backup refusal")
}
assertResetRows(t, db, "local_projects", 1)
}
func TestResetLegacySchema(t *testing.T) {
path, db := resetFixture(t)
if err := db.Exec("ALTER TABLE local_projects DROP COLUMN sync_status").Error; err != nil {
t.Fatal(err)
}
if err := ResetDataWithOptions(path, "", false); err == nil {
t.Fatal("expected legacy-state refusal")
}
if err := ResetDataWithOptions(path, "", true); err != nil {
t.Fatal(err)
}
}
func resetFixture(t *testing.T) (string, *gorm.DB) {
t.Helper()
t.Setenv("QFS_BACKUP_DIR", "")
path := filepath.Join(t.TempDir(), "qfs.db")
db, err := gorm.Open(sqlite.Open(path), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := db.DB()
t.Cleanup(func() { sqlDB.Close() })
for _, table := range []string{"pending_changes", "local_projects", "local_configurations", "connection_settings"} {
if err := db.Exec("CREATE TABLE " + table + " (id INTEGER PRIMARY KEY, sync_status TEXT)").Error; err != nil {
t.Fatal(err)
}
}
if err := db.Exec("INSERT INTO connection_settings (sync_status) VALUES ('synced')").Error; err != nil {
t.Fatal(err)
}
return path, db
}
func assertResetRows(t *testing.T, db *gorm.DB, table string, want int64) {
t.Helper()
var count int64
if err := db.Table(table).Count(&count).Error; err != nil {
t.Fatal(err)
}
if count != want {
t.Fatalf("%s has %d rows, want %d", table, count, want)
}
}
+34
View File
@@ -0,0 +1,34 @@
package localdb
import (
mysql "github.com/go-sql-driver/mysql"
"testing"
)
func TestSettingsTLSRoundTrip(t *testing.T) {
t.Setenv("QFS_STATE_DIR", t.TempDir())
local := newModeDB(t)
for _, mode := range []string{"preferred", "required", "verify-full"} {
if err := local.SaveSettingsWithTLS("db.example", 3306, "example", "example", "example", mode, ""); err != nil {
t.Fatal(err)
}
settings, err := local.GetSettings()
if err != nil {
t.Fatal(err)
}
if settings.TLSMode != mode {
t.Fatalf("mode=%s, want %s", settings.TLSMode, mode)
}
dsn, err := local.GetDSN()
if err != nil {
t.Fatal(err)
}
cfg, err := mysql.ParseDSN(dsn)
if err != nil {
t.Fatal(err)
}
if cfg.TLS == nil || cfg.AllowFallbackToPlaintext != (mode == "preferred") {
t.Fatalf("wrong TLS policy for %s", mode)
}
}
}
+5
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"sort"
"time"
"git.mchus.pro/mchus/quoteforge/internal/models"
)
// BuildConfigurationSnapshot serializes the full local configuration state.
@@ -21,6 +23,7 @@ func BuildConfigurationSnapshot(localCfg *LocalConfiguration) (string, error) {
"total_price": localCfg.TotalPrice,
"custom_price": localCfg.CustomPrice,
"notes": localCfg.Notes,
"feature_flags": localCfg.FeatureFlags,
"is_template": localCfg.IsTemplate,
"server_count": localCfg.ServerCount,
"server_model": localCfg.ServerModel,
@@ -60,6 +63,7 @@ func DecodeConfigurationSnapshot(data string) (*LocalConfiguration, error) {
TotalPrice *float64 `json:"total_price"`
CustomPrice *float64 `json:"custom_price"`
Notes string `json:"notes"`
FeatureFlags models.FeatureFlags `json:"feature_flags"`
IsTemplate bool `json:"is_template"`
ServerCount int `json:"server_count"`
ServerModel string `json:"server_model"`
@@ -95,6 +99,7 @@ func DecodeConfigurationSnapshot(data string) (*LocalConfiguration, error) {
TotalPrice: snapshot.TotalPrice,
CustomPrice: snapshot.CustomPrice,
Notes: snapshot.Notes,
FeatureFlags: snapshot.FeatureFlags,
IsTemplate: snapshot.IsTemplate,
ServerCount: snapshot.ServerCount,
ServerModel: snapshot.ServerModel,
+185
View File
@@ -0,0 +1,185 @@
package mariadb
import (
"archive/zip"
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"gorm.io/gorm"
)
type snapshotTable struct {
Name, Create, Entry, Digest string
Columns []string
Rows int64
}
type snapshotManifest struct {
Format int
Schema, Collation string
Tables []snapshotTable
Triggers []snapshotTrigger
}
func freshDB(db *gorm.DB) *gorm.DB { return db.Session(&gorm.Session{NewDB: true}) }
func quoteIdentifier(s string) string { return "`" + strings.ReplaceAll(s, "`", "``") + "`" }
// CreateSnapshot writes a consistent InnoDB snapshot, without external DB tools.
// The caller supplies a dedicated connection and excludes concurrent writers
// for the entire migration session, including any recovery.
func CreateSnapshot(db *gorm.DB, root, label string) (string, error) {
if err := appstate.ValidateBackupLocation(root); err != nil {
return "", err
}
if filepath.Base(label) != label {
return "", fmt.Errorf("invalid snapshot label")
}
if err := os.MkdirAll(root, 0700); err != nil {
return "", fmt.Errorf("create snapshot directory: %w", err)
}
if err := appstate.ValidateBackupLocation(root); err != nil {
return "", err
}
file, err := os.CreateTemp(root, "qfs-mariadb-v1-"+label+"-"+time.Now().UTC().Format("20060102T150405Z")+"-*.zip")
if err != nil {
return "", fmt.Errorf("create snapshot: %w", err)
}
path := file.Name()
archive := zip.NewWriter(file)
err = freshDB(db).Transaction(func(tx *gorm.DB) error {
manifest, err := snapshotInventory(tx)
if err != nil {
return err
}
for i := range manifest.Tables {
table := &manifest.Tables[i]
table.Entry = fmt.Sprintf("tables/%06d.jsonl", i)
out, err := archive.Create(table.Entry)
if err != nil {
return fmt.Errorf("create table entry %s: %w", table.Name, err)
}
if err := snapshotRows(tx, table, out); err != nil {
return err
}
}
out, err := archive.Create("manifest.json")
if err != nil {
return fmt.Errorf("create snapshot manifest: %w", err)
}
if err := json.NewEncoder(out).Encode(manifest); err != nil {
return fmt.Errorf("write snapshot manifest: %w", err)
}
return nil
}, &sql.TxOptions{Isolation: sql.LevelRepeatableRead, ReadOnly: true})
zipErr := archive.Close()
syncErr := file.Sync()
closeErr := file.Close()
for _, candidate := range []error{err, zipErr, syncErr, closeErr} {
if candidate != nil {
return "", fmt.Errorf("write snapshot %s: %w", path, candidate)
}
}
if _, err := readSnapshot(path, nil); err != nil {
return "", err
}
return path, nil
}
func snapshotInventory(db *gorm.DB) (*snapshotManifest, error) {
m := &snapshotManifest{Format: 1}
if err := freshDB(db).Raw("SELECT DATABASE(), @@collation_database").Row().Scan(&m.Schema, &m.Collation); err != nil {
return nil, fmt.Errorf("read snapshot schema: %w", err)
}
var unsupported int64
query := `SELECT
(SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA=? AND (TABLE_TYPE<>'BASE TABLE' OR ENGINE<>'InnoDB')) +
(SELECT COUNT(*) FROM information_schema.ROUTINES WHERE ROUTINE_SCHEMA=?) +
(SELECT COUNT(*) FROM information_schema.EVENTS WHERE EVENT_SCHEMA=?) +
(SELECT COUNT(*) FROM information_schema.KEY_COLUMN_USAGE WHERE
(TABLE_SCHEMA=? AND REFERENCED_TABLE_SCHEMA<>?) OR (REFERENCED_TABLE_SCHEMA=? AND TABLE_SCHEMA<>?))`
if err := freshDB(db).Raw(query, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema, m.Schema).Scan(&unsupported).Error; err != nil {
return nil, fmt.Errorf("check snapshot support: %w", err)
}
if unsupported != 0 {
return nil, fmt.Errorf("snapshot refused: views, non-InnoDB tables, routines, events or cross-schema foreign keys require supported recovery before migration")
}
var names []string
if err := freshDB(db).Raw("SELECT TABLE_NAME FROM information_schema.TABLES WHERE TABLE_SCHEMA=? ORDER BY TABLE_NAME", m.Schema).Scan(&names).Error; err != nil {
return nil, fmt.Errorf("list snapshot tables: %w", err)
}
for _, name := range names {
table := snapshotTable{Name: name}
var ignored string
if err := freshDB(db).Raw("SHOW CREATE TABLE "+quoteIdentifier(name)).Row().Scan(&ignored, &table.Create); err != nil {
return nil, fmt.Errorf("read schema for %s: %w", name, err)
}
if err := freshDB(db).Raw(`SELECT COLUMN_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=? AND TABLE_NAME=? AND EXTRA NOT LIKE '%VIRTUAL GENERATED%' AND EXTRA NOT LIKE '%STORED GENERATED%' ORDER BY ORDINAL_POSITION`, m.Schema, name).Scan(&table.Columns).Error; err != nil {
return nil, fmt.Errorf("read columns for %s: %w", name, err)
}
if len(table.Columns) == 0 {
return nil, fmt.Errorf("snapshot table %s has no writable columns", name)
}
m.Tables = append(m.Tables, table)
}
if err := readSnapshotTriggers(db, m); err != nil {
return nil, err
}
return m, nil
}
func snapshotRows(db *gorm.DB, table *snapshotTable, out io.Writer) error {
columns := make([]string, len(table.Columns))
for i, col := range table.Columns {
columns[i] = "CAST(" + quoteIdentifier(col) + " AS BINARY)"
}
rows, err := freshDB(db).Raw("SELECT " + strings.Join(columns, ",") + " FROM " + quoteIdentifier(table.Name)).Rows()
if err != nil {
return fmt.Errorf("read snapshot table %s: %w", table.Name, err)
}
defer rows.Close()
var digest [32]byte
encoder := json.NewEncoder(out)
table.Rows = 0
for rows.Next() {
values := make([]sql.RawBytes, len(columns))
args := make([]any, len(columns))
for i := range args {
args[i] = &values[i]
}
if err := rows.Scan(args...); err != nil {
return fmt.Errorf("read row in %s: %w", table.Name, err)
}
if err := encoder.Encode(values); err != nil {
return fmt.Errorf("write row in %s: %w", table.Name, err)
}
addRowDigest(&digest, values)
table.Rows++
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterate %s: %w", table.Name, err)
}
if err := rows.Close(); err != nil {
return fmt.Errorf("close rows in %s: %w", table.Name, err)
}
table.Digest = hex.EncodeToString(digest[:])
return nil
}
func addRowDigest(sum *[32]byte, row []sql.RawBytes) {
encoded, _ := json.Marshal(row) // Raw bytes and nil always marshal successfully.
hash := sha256.Sum256(encoded)
carry := uint16(0)
for i := len(sum) - 1; i >= 0; i-- {
value := uint16(sum[i]) + uint16(hash[i]) + carry
sum[i], carry = byte(value), value>>8
}
}
+101
View File
@@ -0,0 +1,101 @@
package mariadb
import (
"archive/zip"
"database/sql"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
)
func readSnapshot(path string, visit func(snapshotTable, []sql.RawBytes) error) (*snapshotManifest, error) {
if err := appstate.ValidateBackupLocation(path); err != nil {
return nil, err
}
archive, err := zip.OpenReader(path)
if err != nil {
return nil, fmt.Errorf("open server snapshot: %w", err)
}
defer archive.Close()
files := map[string]*zip.File{}
for _, f := range archive.File {
if files[f.Name] != nil {
return nil, fmt.Errorf("duplicate snapshot entry")
}
files[f.Name] = f
}
manifestFile := files["manifest.json"]
if manifestFile == nil || manifestFile.UncompressedSize64 > 16<<20 {
return nil, fmt.Errorf("missing or oversized snapshot manifest")
}
in, err := manifestFile.Open()
if err != nil {
return nil, fmt.Errorf("open server manifest: %w", err)
}
data, readErr := io.ReadAll(in)
closeErr := in.Close()
if readErr != nil || closeErr != nil {
return nil, fmt.Errorf("read server manifest: %w", errors.Join(readErr, closeErr))
}
var m snapshotManifest
if err := json.Unmarshal(data, &m); err != nil {
return nil, fmt.Errorf("parse server manifest: %w", err)
}
if m.Format != 1 || m.Schema == "" || len(files) != len(m.Tables)+1 {
return nil, fmt.Errorf("invalid server snapshot manifest")
}
names, entries := map[string]bool{}, map[string]bool{}
for _, table := range m.Tables {
if table.Name == "" || names[table.Name] || entries[table.Entry] || files[table.Entry] == nil || len(table.Columns) == 0 || table.Rows < 0 || !strings.HasPrefix(table.Create, "CREATE TABLE "+quoteIdentifier(table.Name)+" (") {
return nil, fmt.Errorf("invalid snapshot table metadata")
}
names[table.Name], entries[table.Entry] = true, true
if err := readSnapshotTable(files[table.Entry], table, visit); err != nil {
return nil, err
}
}
return &m, nil
}
func readSnapshotTable(file *zip.File, table snapshotTable, visit func(snapshotTable, []sql.RawBytes) error) error {
in, err := file.Open()
if err != nil {
return fmt.Errorf("open snapshot table %s: %w", table.Name, err)
}
defer in.Close()
decoder := json.NewDecoder(in)
var rows int64
var digest [32]byte
for {
var values []sql.RawBytes
err := decoder.Decode(&values)
if err == io.EOF {
break
}
if err != nil {
return fmt.Errorf("read snapshot row in %s: %w", table.Name, err)
}
if len(values) != len(table.Columns) {
return fmt.Errorf("snapshot column count mismatch in %s", table.Name)
}
addRowDigest(&digest, values)
rows++
if visit != nil {
if err := visit(table, values); err != nil {
return err
}
}
}
if err := in.Close(); err != nil {
return fmt.Errorf("close snapshot table %s: %w", table.Name, err)
}
if rows != table.Rows || hex.EncodeToString(digest[:]) != table.Digest {
return fmt.Errorf("snapshot checksum mismatch in %s", table.Name)
}
return nil
}
+118
View File
@@ -0,0 +1,118 @@
package mariadb
import (
"database/sql"
"errors"
"fmt"
"strings"
"gorm.io/gorm"
)
// RestoreSnapshot restores and checks a snapshot on a dedicated connection.
// The same maintenance exclusion as migration is required throughout recovery.
func RestoreSnapshot(db *gorm.DB, path string) error {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
var schema string
if err := freshDB(db).Raw("SELECT DATABASE()").Scan(&schema).Error; err != nil {
return fmt.Errorf("read restore destination: %w", err)
}
if schema != m.Schema {
return fmt.Errorf("snapshot schema does not match restore destination")
}
if err := restoreSnapshotTables(db, path, m); err != nil {
return err
}
return compareSnapshot(db, m)
}
func restoreSnapshotTables(db *gorm.DB, path string, m *snapshotManifest) (err error) {
current, err := snapshotInventory(db)
if err != nil {
return err
}
var foreignKeys int
var sqlMode string
if err := freshDB(db).Raw("SELECT @@foreign_key_checks, @@sql_mode").Row().Scan(&foreignKeys, &sqlMode); err != nil {
return fmt.Errorf("read restore session policy: %w", err)
}
if err := freshDB(db).Exec("SET FOREIGN_KEY_CHECKS=0").Error; err != nil {
return fmt.Errorf("disable FK checks for restore: %w", err)
}
defer func() {
if restoreErr := freshDB(db).Exec("SET sql_mode=?", sqlMode).Error; restoreErr != nil {
err = errors.Join(err, fmt.Errorf("restore SQL mode: %w", restoreErr))
}
if restoreErr := freshDB(db).Exec("SET FOREIGN_KEY_CHECKS=?", foreignKeys).Error; restoreErr != nil {
err = errors.Join(err, fmt.Errorf("restore FK policy: %w", restoreErr))
}
}()
// Preserve explicit zero auto-increment IDs and legacy zero dates. Exact
// schema/data comparison detects any coercion before the source is changed.
if err := freshDB(db).Exec("SET sql_mode='NO_AUTO_VALUE_ON_ZERO'").Error; err != nil {
return fmt.Errorf("set snapshot insert mode: %w", err)
}
for i := len(current.Tables) - 1; i >= 0; i-- {
if err := freshDB(db).Exec("DROP TABLE " + quoteIdentifier(current.Tables[i].Name)).Error; err != nil {
return fmt.Errorf("drop restore table %s: %w", current.Tables[i].Name, err)
}
}
for _, table := range m.Tables {
if err := freshDB(db).Exec(table.Create).Error; err != nil {
return fmt.Errorf("restore schema for %s: %w", table.Name, err)
}
}
var prefix, rowSQL, tableName string
var args []any
var groups []string
var bytes int
flush := func() error {
if len(groups) == 0 {
return nil
}
// Bind through database/sql: GORM expands a byte slice after '('.
// Direct execution also keeps restored values out of SQL logs.
if _, err := db.Statement.ConnPool.ExecContext(db.Statement.Context, prefix+strings.Join(groups, ","), args...); err != nil {
return fmt.Errorf("restore rows in %s: %w", tableName, err)
}
args, groups, bytes = nil, nil, 0
return nil
}
_, err = readSnapshot(path, func(table snapshotTable, values []sql.RawBytes) error {
if tableName != table.Name {
if err := flush(); err != nil {
return err
}
tableName = table.Name
columns, placeholders := make([]string, len(values)), make([]string, len(values))
for i := range values {
columns[i], placeholders[i] = quoteIdentifier(table.Columns[i]), "?"
}
prefix = "INSERT INTO " + quoteIdentifier(tableName) + " (" + strings.Join(columns, ",") + ") VALUES "
rowSQL = "(" + strings.Join(placeholders, ",") + ")"
}
for _, value := range values {
if value == nil {
args = append(args, nil)
} else {
args = append(args, []byte(value))
bytes += len(value)
}
}
groups = append(groups, rowSQL)
if len(groups) >= 250 || len(args)+len(values) > 60000 || bytes >= 1<<20 {
return flush()
}
return nil
})
if err == nil {
err = flush()
}
if err != nil {
return err
}
return restoreSnapshotTriggers(db, m)
}
+87
View File
@@ -0,0 +1,87 @@
package mariadb
import (
"archive/zip"
"database/sql"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"testing"
)
func TestSnapshotRowsAndValidation(t *testing.T) {
row := []sql.RawBytes{[]byte{0, 1, 255}, {}, nil}
var sum [32]byte
addRowDigest(&sum, row)
for _, tt := range []struct {
name string
format int
rows int64
digest string
width int
valid bool
}{
{"valid", 1, 1, hex.EncodeToString(sum[:]), 3, true},
{"bad format", 2, 1, hex.EncodeToString(sum[:]), 3, false},
{"bad count", 1, 2, hex.EncodeToString(sum[:]), 3, false},
{"bad digest", 1, 1, "invalid", 3, false},
{"bad width", 1, 1, hex.EncodeToString(sum[:]), 2, false},
} {
t.Run(tt.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "backup.zip")
file, err := os.Create(path)
if err != nil {
t.Fatal(err)
}
archive := zip.NewWriter(file)
entry, err := archive.Create("table.jsonl")
if err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(entry).Encode(row); err != nil {
t.Fatal(err)
}
entry, err = archive.Create("manifest.json")
if err != nil {
t.Fatal(err)
}
table := snapshotTable{Name: "example", Create: "CREATE TABLE `example` (id INT)", Entry: "table.jsonl", Columns: []string{"a", "b", "c"}[:tt.width], Rows: tt.rows, Digest: tt.digest}
if err := json.NewEncoder(entry).Encode(snapshotManifest{Format: tt.format, Schema: "example", Tables: []snapshotTable{table}}); err != nil {
t.Fatal(err)
}
if err := archive.Close(); err != nil {
t.Fatal(err)
}
if err := file.Close(); err != nil {
t.Fatal(err)
}
_, err = readSnapshot(path, nil)
if (err == nil) != tt.valid {
t.Fatalf("valid=%v err=%v", tt.valid, err)
}
if !tt.valid {
if err := RestoreSnapshot(nil, path); err == nil {
t.Fatal("invalid archive reached restore without preflight refusal")
}
}
})
}
}
func TestSnapshotDigestPreservesValues(t *testing.T) {
a, b := []sql.RawBytes{[]byte("a")}, []sql.RawBytes{[]byte("b")}
var forward, reverse, null, empty [32]byte
addRowDigest(&forward, a)
addRowDigest(&forward, b)
addRowDigest(&reverse, b)
addRowDigest(&reverse, a)
if forward != reverse {
t.Fatal("digest depends on row order")
}
addRowDigest(&null, []sql.RawBytes{nil})
addRowDigest(&empty, []sql.RawBytes{{}})
if null == empty {
t.Fatal("digest conflates NULL and empty bytes")
}
}
@@ -0,0 +1,43 @@
package mariadb
import (
"fmt"
"regexp"
"strings"
)
// Recovery covers the selected schema, so external trigger effects are refused.
func validateSnapshotTriggerReferences(trigger snapshotTrigger, schema string) error {
body := trigger.Body
quoted := `'(?:''|\\.|[^'\\])*'`
if strings.Contains(trigger.SQLMode, "NO_BACKSLASH_ESCAPES") {
quoted = `'(?:''|[^'])*'`
}
body = regexp.MustCompile(quoted).ReplaceAllString(body, " ")
if !strings.Contains(trigger.SQLMode, "ANSI_QUOTES") {
double := `"(?:""|\\.|[^"\\])*"`
if strings.Contains(trigger.SQLMode, "NO_BACKSLASH_ESCAPES") {
double = `"(?:""|[^"])*"`
}
body = regexp.MustCompile(double).ReplaceAllString(body, " ")
}
if strings.Contains(body, "/*!") || strings.Contains(body, "/*M!") {
return fmt.Errorf("trigger %s has unsupported executable comments", trigger.Name)
}
body = regexp.MustCompile(`(?s)/\*.*?\*/|--[\t ][^\n]*|#[^\n]*`).ReplaceAllString(body, " ")
if regexp.MustCompile(`(?i)\bCALL\s+`).MatchString(body) {
return fmt.Errorf("trigger %s calls a routine outside supported recovery", trigger.Name)
}
identifier := "(?:`(?:``|[^`])+`|\"(?:\"\"|[^\"])+\"|[\\pL_$][\\pL\\pN_$]*)"
for _, pattern := range []string{
"(?i)\\b(?:FROM|JOIN|UPDATE|INTO)\\s+(?:(?:LOW_PRIORITY|HIGH_PRIORITY|DELAYED|IGNORE)\\s+)*(" + identifier + ")\\s*\\.",
"(?i)(" + identifier + ")\\s*\\.\\s*" + identifier + "\\s*\\(",
} {
for _, match := range regexp.MustCompile(pattern).FindAllStringSubmatch(body, -1) {
if !strings.EqualFold(strings.Trim(match[1], "`\""), schema) {
return fmt.Errorf("trigger %s references another schema outside supported recovery", trigger.Name)
}
}
}
return nil
}
@@ -0,0 +1,29 @@
package mariadb
import "testing"
func TestSnapshotTriggerReferences(t *testing.T) {
for _, tt := range []struct {
name, body, mode string
fail bool
}{
{"local", "INSERT INTO audit VALUES (NEW.id, OLD.value)", "", false},
{"local qualified", "UPDATE `example`.`audit` SET id=NEW.id", "", false},
{"alias", "SELECT 1 FROM audit a WHERE a.id=OLD.id", "", false},
{"literal", "INSERT INTO audit VALUES ('example.com', '$.field', 'UPDATE other.audit')", "", false},
{"comment", "/* UPDATE other.audit */ INSERT INTO audit VALUES (NEW.id)", "", false},
{"external write", "UPDATE other.audit SET id=NEW.id", "", true},
{"external insert", "INSERT INTO `other`.`audit` VALUES (NEW.id)", "", true},
{"ANSI external", `UPDATE "other"."audit" SET id=NEW.id`, "ANSI_QUOTES", true},
{"external function", "SET NEW.value=other.example(NEW.value)", "", true},
{"routine", "CALL example.change_rows()", "", true},
{"executable", "/*!50000 UPDATE other.audit SET id=1 */", "", true},
} {
t.Run(tt.name, func(t *testing.T) {
err := validateSnapshotTriggerReferences(snapshotTrigger{Name: "example", Body: tt.body, SQLMode: tt.mode}, "example")
if (err != nil) != tt.fail {
t.Fatalf("error=%v", err)
}
})
}
}
+83
View File
@@ -0,0 +1,83 @@
package mariadb
import (
"errors"
"fmt"
"strings"
"gorm.io/gorm"
)
type snapshotTrigger struct {
Name string `gorm:"column:TRIGGER_NAME"`
Table string `gorm:"column:EVENT_OBJECT_TABLE"`
Event string `gorm:"column:EVENT_MANIPULATION"`
Timing string `gorm:"column:ACTION_TIMING"`
Order int `gorm:"column:ACTION_ORDER"`
Body string `gorm:"column:ACTION_STATEMENT"`
SQLMode string `gorm:"column:SQL_MODE"`
Definer string `gorm:"column:DEFINER"`
Charset string `gorm:"column:CHARACTER_SET_CLIENT"`
Collation string `gorm:"column:COLLATION_CONNECTION"`
DatabaseCollation string `gorm:"column:DATABASE_COLLATION"`
}
func readSnapshotTriggers(db *gorm.DB, m *snapshotManifest) error {
query := `SELECT TRIGGER_NAME, EVENT_OBJECT_TABLE, EVENT_MANIPULATION, ACTION_TIMING,
ACTION_ORDER, ACTION_STATEMENT, SQL_MODE, DEFINER, CHARACTER_SET_CLIENT,
COLLATION_CONNECTION, DATABASE_COLLATION FROM information_schema.TRIGGERS
WHERE TRIGGER_SCHEMA=? ORDER BY EVENT_OBJECT_TABLE, ACTION_TIMING, EVENT_MANIPULATION, ACTION_ORDER`
if err := freshDB(db).Raw(query, m.Schema).Scan(&m.Triggers).Error; err != nil {
return fmt.Errorf("read snapshot triggers: %w", err)
}
for _, trigger := range m.Triggers {
if err := validateSnapshotTriggerReferences(trigger, m.Schema); err != nil {
return err
}
if trigger.DatabaseCollation != m.Collation {
return fmt.Errorf("trigger %s has a legacy database collation; restore support is required before migration", trigger.Name)
}
}
return nil
}
func restoreSnapshotTriggers(db *gorm.DB, m *snapshotManifest) (err error) {
if len(m.Triggers) == 0 {
return nil
}
var mode, charset, collation, schema string
if err := freshDB(db).Raw("SELECT @@sql_mode, @@character_set_client, @@collation_connection, DATABASE()").Row().Scan(&mode, &charset, &collation, &schema); err != nil {
return fmt.Errorf("read trigger restore session: %w", err)
}
defer func() {
for _, reset := range []struct{ sql, value string }{{"SET sql_mode=?", mode}, {"SET character_set_client=?", charset}, {"SET collation_connection=?", collation}} {
if e := freshDB(db).Exec(reset.sql, reset.value).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset trigger restore session: %w", e))
}
}
}()
for _, trigger := range m.Triggers {
if trigger.Timing != "BEFORE" && trigger.Timing != "AFTER" {
return fmt.Errorf("invalid trigger timing")
}
if trigger.Event != "INSERT" && trigger.Event != "UPDATE" && trigger.Event != "DELETE" {
return fmt.Errorf("invalid trigger event")
}
at := strings.LastIndex(trigger.Definer, "@")
if at < 1 || at == len(trigger.Definer)-1 {
return fmt.Errorf("invalid trigger definer")
}
for _, setting := range []struct{ sql, value string }{{"SET sql_mode=?", trigger.SQLMode}, {"SET character_set_client=?", trigger.Charset}, {"SET collation_connection=?", trigger.Collation}} {
if err := freshDB(db).Exec(setting.sql, setting.value).Error; err != nil {
return fmt.Errorf("set trigger restore session: %w", err)
}
}
// Explicitly bind name and target table to the destination. Trigger bodies
// retain their original SQL; verification performs no DML after creation.
statement := "CREATE DEFINER=" + quoteIdentifier(trigger.Definer[:at]) + "@" + quoteIdentifier(trigger.Definer[at+1:]) + " TRIGGER " + quoteIdentifier(schema) + "." + quoteIdentifier(trigger.Name) + " " + trigger.Timing + " " + trigger.Event + " ON " + quoteIdentifier(schema) + "." + quoteIdentifier(trigger.Table) + " FOR EACH ROW " + trigger.Body
if _, err := db.Statement.ConnPool.ExecContext(db.Statement.Context, statement); err != nil {
return fmt.Errorf("restore trigger %s: %w", trigger.Name, err)
}
}
return nil
}
+79
View File
@@ -0,0 +1,79 @@
package mariadb
import (
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"io"
"reflect"
"gorm.io/gorm"
)
// CheckSnapshot compares restored schema and all writable row values.
func CheckSnapshot(db *gorm.DB, path string) error {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
return compareSnapshot(db, m)
}
// VerifySnapshot proves restore readiness in an isolated temporary database.
func VerifySnapshot(db *gorm.DB, path string) (err error) {
m, err := readSnapshot(path, nil)
if err != nil {
return err
}
var random [12]byte
if _, err := rand.Read(random[:]); err != nil {
return fmt.Errorf("name verification schema: %w", err)
}
scratch := "qfs_verify_" + hex.EncodeToString(random[:])
var original string
if err := freshDB(db).Raw("SELECT DATABASE()").Scan(&original).Error; err != nil {
return fmt.Errorf("read original schema: %w", err)
}
if err := freshDB(db).Exec("CREATE DATABASE " + quoteIdentifier(scratch) + " COLLATE " + quoteIdentifier(m.Collation)).Error; err != nil {
return fmt.Errorf("create restore verification schema: %w", err)
}
defer func() {
if e := freshDB(db).Exec("USE " + quoteIdentifier(original)).Error; e != nil {
err = errors.Join(err, fmt.Errorf("return to original schema: %w", e))
}
if e := freshDB(db).Exec("DROP DATABASE " + quoteIdentifier(scratch)).Error; e != nil {
err = errors.Join(err, fmt.Errorf("remove verification schema %s: %w", scratch, e))
}
}()
if err := freshDB(db).Exec("USE " + quoteIdentifier(scratch)).Error; err != nil {
return fmt.Errorf("select verification schema: %w", err)
}
if err := restoreSnapshotTables(db, path, m); err != nil {
return err
}
return compareSnapshot(db, m)
}
func compareSnapshot(db *gorm.DB, m *snapshotManifest) error {
current, err := snapshotInventory(db)
if err != nil {
return err
}
if !reflect.DeepEqual(current.Triggers, m.Triggers) {
return fmt.Errorf("restored trigger metadata mismatch")
}
if len(current.Tables) != len(m.Tables) {
return fmt.Errorf("restored table count mismatch")
}
for i := range m.Tables {
want, got := m.Tables[i], current.Tables[i]
if err := snapshotRows(db, &got, io.Discard); err != nil {
return err
}
if want.Name != got.Name || want.Create != got.Create || want.Rows != got.Rows || want.Digest != got.Digest {
return fmt.Errorf("restored snapshot mismatch in %s (name=%t schema=%t rows=%t digest=%t)", want.Name, want.Name == got.Name, want.Create == got.Create, want.Rows == got.Rows, want.Digest == got.Digest)
}
}
return nil
}
+71
View File
@@ -0,0 +1,71 @@
package mariadb
import (
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"fmt"
"strings"
mysql "github.com/go-sql-driver/mysql"
)
// ConfigureTLS configures the DSN so setup and runtime use the same TLS policy.
func ConfigureTLS(cfg *mysql.Config, mode, caPEM string) error {
mode = strings.TrimSpace(mode)
if mode == "" {
mode = "preferred"
}
caPEM = strings.TrimSpace(caPEM)
if caPEM != "" && mode != "verify-ca" && mode != "verify-full" {
return fmt.Errorf("CA certificate requires verify-ca or verify-full")
}
switch mode {
case "preferred":
cfg.TLSConfig = "preferred"
return nil
case "required":
cfg.TLSConfig = "skip-verify"
return nil
case "verify-full":
if caPEM == "" {
cfg.TLSConfig = "true"
return nil
}
case "verify-ca":
if caPEM == "" {
return fmt.Errorf("verify-ca requires a CA certificate")
}
default:
return fmt.Errorf("unsupported MariaDB TLS mode")
}
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM([]byte(caPEM)) {
return fmt.Errorf("invalid CA certificate PEM")
}
tlsConfig := &tls.Config{MinVersion: tls.VersionTLS12, RootCAs: roots}
if mode == "verify-ca" {
// Verify the certificate chain explicitly; this mode omits hostname checks.
tlsConfig.InsecureSkipVerify = true
tlsConfig.VerifyConnection = func(state tls.ConnectionState) error {
if len(state.PeerCertificates) == 0 {
return fmt.Errorf("MariaDB did not provide a certificate")
}
intermediates := x509.NewCertPool()
for _, cert := range state.PeerCertificates[1:] {
intermediates.AddCert(cert)
}
_, err := state.PeerCertificates[0].Verify(x509.VerifyOptions{Roots: roots, Intermediates: intermediates, KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}})
if err != nil {
return fmt.Errorf("verify MariaDB certificate: %w", err)
}
return nil
}
}
name := fmt.Sprintf("qfs-%x", sha256.Sum256([]byte(mode+"\n"+caPEM)))
if err := mysql.RegisterTLSConfig(name, tlsConfig); err != nil {
return fmt.Errorf("register MariaDB TLS: %w", err)
}
cfg.TLSConfig = name
return nil
}
+128
View File
@@ -0,0 +1,128 @@
//go:build integration
package mariadb
import (
"context"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"database/sql"
"encoding/pem"
"math/big"
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
mysql "github.com/go-sql-driver/mysql"
)
func TestMariaDBTLSIntegration(t *testing.T) {
dir := t.TempDir()
if err := os.Chmod(dir, 0755); err != nil {
t.Fatal(err)
}
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatal(err)
}
cert := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "example"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}}
der, err := x509.CreateCertificate(rand.Reader, cert, cert, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
ca := string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}))
for name, data := range map[string][]byte{"server.crt": []byte(ca), "server.key": pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)})} {
// Docker's mysql user must read these disposable test files.
if err := os.WriteFile(filepath.Join(dir, name), data, 0644); err != nil {
t.Fatal(err)
}
}
otherCA, _ := testCertificate(t)
for _, tlsEnabled := range []bool{true, false} {
t.Run(map[bool]string{true: "TLS", false: "plaintext"}[tlsEnabled], func(t *testing.T) {
args := []string{"run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306",
"-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "-v", dir + ":/qfs-tls:ro", "mariadb:11.8"}
if tlsEnabled {
args = append(args, "--ssl-cert=/qfs-tls/server.crt", "--ssl-key=/qfs-tls/server.key")
} else {
args = append(args, "--skip-ssl")
}
out, err := exec.Command("docker", args...).CombinedOutput()
if err != nil {
t.Fatalf("start disposable MariaDB: %v: %s", err, out)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if out, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop test MariaDB: %v: %s", err, out)
}
})
out, err = exec.Command("docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatalf("test MariaDB port: %v: %s", err, out)
}
addr := strings.TrimSpace(string(out))
connect := func(mode, roots, serverName string) (string, error) {
cfg := mysql.NewConfig()
cfg.User, cfg.Net, cfg.Addr, cfg.Timeout = "root", "tcp", addr, time.Second
if err := ConfigureTLS(cfg, mode, roots); err != nil {
return "", err
}
cfg, err = mysql.ParseDSN(cfg.FormatDSN())
if err != nil {
return "", err
}
if serverName != "" {
cfg.TLS.ServerName = serverName
}
connector, err := mysql.NewConnector(cfg)
if err != nil {
return "", err
}
db := sql.OpenDB(connector)
defer db.Close()
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
var name, cipher string
err = db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&name, &cipher)
return cipher, err
}
deadline := time.Now().Add(45 * time.Second)
for {
if _, err := connect("preferred", "", ""); err == nil {
break
}
if time.Now().After(deadline) {
t.Fatal("test MariaDB did not become ready")
}
time.Sleep(100 * time.Millisecond)
}
for _, tt := range []struct {
mode, roots, name string
success bool
}{
{"preferred", "", "", true}, {"required", "", "", tlsEnabled},
{"verify-ca", ca, "", tlsEnabled}, {"verify-full", ca, "", tlsEnabled},
{"verify-ca", otherCA, "", false}, {"verify-full", otherCA, "", false},
{"verify-full", ca, "wrong.example", false},
} {
cipher, err := connect(tt.mode, tt.roots, tt.name)
if (err == nil) != tt.success {
t.Errorf("mode=%s name=%s success=%v error=%v", tt.mode, tt.name, tt.success, err)
}
if err == nil && (cipher != "") != tlsEnabled {
t.Errorf("mode=%s cipher=%q TLS=%v", tt.mode, cipher, tlsEnabled)
}
}
})
}
}
+97
View File
@@ -0,0 +1,97 @@
package mariadb
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"testing"
"time"
mysql "github.com/go-sql-driver/mysql"
)
func TestTLSPolicy(t *testing.T) {
ca, cert := testCertificate(t)
_, other := testCertificate(t)
for _, tt := range []struct {
mode, ca string
wantError, fallback, insecure bool
}{
{"", "", false, true, true},
{"preferred", "", false, true, true},
{"required", "", false, false, true},
{"verify-full", "", false, false, false},
{"verify-full", ca, false, false, false},
{"verify-ca", ca, false, false, true},
{"verify-ca", "", true, false, false},
{"verify-full", "invalid", true, false, false},
{"preferred", ca, true, false, false},
{"false", "", true, false, false},
} {
t.Run(tt.mode+tt.ca[:min(len(tt.ca), 5)], func(t *testing.T) {
cfg := mysql.NewConfig()
cfg.Net, cfg.Addr = "tcp", "db.example:3306"
err := ConfigureTLS(cfg, tt.mode, tt.ca)
if (err != nil) != tt.wantError {
t.Fatalf("configure: %v", err)
}
if tt.wantError {
return
}
parsed, err := mysql.ParseDSN(cfg.FormatDSN())
if err != nil {
t.Fatal(err)
}
if parsed.TLS == nil || parsed.AllowFallbackToPlaintext != tt.fallback || parsed.TLS.InsecureSkipVerify != tt.insecure {
t.Fatalf("unexpected TLS policy for %s", tt.mode)
}
if tt.mode == "verify-ca" {
if err := parsed.TLS.VerifyConnection(tls.ConnectionState{PeerCertificates: []*x509.Certificate{cert}}); err != nil {
t.Fatal(err)
}
for _, chain := range [][]*x509.Certificate{nil, {other}} {
if err := parsed.TLS.VerifyConnection(tls.ConnectionState{PeerCertificates: chain}); err == nil {
t.Fatal("untrusted certificate accepted")
}
}
}
if tt.mode == "verify-full" && tt.ca != "" {
if parsed.TLS.ServerName != "db.example" {
t.Fatalf("missing hostname verification: %s", parsed.TLS.ServerName)
}
if _, err := cert.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: parsed.TLS.ServerName}); err != nil {
t.Fatal(err)
}
if _, err := other.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: parsed.TLS.ServerName}); err == nil {
t.Fatal("untrusted certificate accepted")
}
if _, err := cert.Verify(x509.VerifyOptions{Roots: parsed.TLS.RootCAs, DNSName: "other.example"}); err == nil {
t.Fatal("wrong hostname accepted")
}
}
})
}
}
func testCertificate(t *testing.T) (string, *x509.Certificate) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "example"}, DNSNames: []string{"db.example"}, NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
cert, err := x509.ParseCertificate(der)
if err != nil {
t.Fatal(err)
}
return string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})), cert
}
+27 -1
View File
@@ -1,6 +1,7 @@
package middleware
import (
"mime"
"net"
"net/http"
"net/url"
@@ -11,6 +12,19 @@ import (
func CORS() gin.HandlerFunc {
return func(c *gin.Context) {
hostURL, hostErr := url.Parse("http://" + c.Request.Host)
if hostErr != nil || hostURL.Host != c.Request.Host || hostURL.Path != "" || hostURL.RawQuery != "" || !isLoopbackOrigin("http://"+c.Request.Host) {
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "Invalid request host"})
return
}
if c.GetHeader("Sec-Fetch-Site") == "cross-site" {
c.AbortWithStatus(http.StatusForbidden)
return
}
if referer := c.GetHeader("Referer"); referer != "" && !isLoopbackOrigin(referer) {
c.AbortWithStatus(http.StatusForbidden)
return
}
origin := strings.TrimSpace(c.GetHeader("Origin"))
if origin != "" {
if isLoopbackOrigin(origin) {
@@ -20,7 +34,7 @@ func CORS() gin.HandlerFunc {
c.Header("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization")
c.Header("Access-Control-Expose-Headers", "Content-Length, Content-Disposition")
c.Header("Access-Control-Max-Age", "86400")
} else if c.Request.Method == http.MethodOptions {
} else {
c.AbortWithStatus(http.StatusForbidden)
return
}
@@ -30,6 +44,15 @@ func CORS() gin.HandlerFunc {
c.AbortWithStatus(http.StatusNoContent)
return
}
if c.Request.Method != http.MethodGet && c.Request.Method != http.MethodHead &&
strings.HasPrefix(c.Request.URL.Path, "/api/") && c.Request.ContentLength != 0 {
mediaType, _, err := mime.ParseMediaType(c.GetHeader("Content-Type"))
isUpload := strings.HasPrefix(c.FullPath(), "/api/projects/") && strings.HasSuffix(c.FullPath(), "/vendor-import")
if err != nil || (mediaType != "application/json" && !(isUpload && mediaType == "multipart/form-data")) {
c.AbortWithStatusJSON(http.StatusUnsupportedMediaType, gin.H{"error": "Unsupported content type"})
return
}
}
c.Next()
}
@@ -43,6 +66,9 @@ func isLoopbackOrigin(origin string) bool {
if u.Scheme != "http" && u.Scheme != "https" {
return false
}
if u.User != nil || u.Opaque != "" {
return false
}
host := strings.TrimSpace(u.Hostname())
if host == "" {
return false
+66
View File
@@ -0,0 +1,66 @@
package middleware
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
)
func TestLocalRequestBoundary(t *testing.T) {
gin.SetMode(gin.TestMode)
tests := []struct {
name, method, path, host, origin, referer, fetchSite, contentType, body string
want int
}{
{name: "CLI JSON", method: "POST", contentType: "application/json", body: "{}", want: 200},
{name: "local browser", method: "POST", origin: "http://localhost:8080", contentType: "application/json; charset=utf-8", body: "{}", want: 200},
{name: "foreign form", method: "POST", path: "/setup", origin: "https://attacker.example", contentType: "application/x-www-form-urlencoded", body: "host=example", want: 403},
{name: "foreign JSON", method: "POST", origin: "https://attacker.example", contentType: "text/plain", body: "{}", want: 403},
{name: "rebind read", method: "GET", host: "attacker.example:8080", want: 403},
{name: "null origin", method: "POST", origin: "null", want: 403},
{name: "foreign referer", method: "POST", referer: "https://attacker.example/page", want: 403},
{name: "cross site", method: "POST", fetchSite: "cross-site", want: 403},
{name: "plain JSON", method: "POST", contentType: "text/plain", body: "{}", want: 415},
{name: "API form", method: "POST", contentType: "application/x-www-form-urlencoded", body: "x=y", want: 415},
{name: "setup form", method: "POST", path: "/setup", origin: "http://127.0.0.1:8080", contentType: "application/x-www-form-urlencoded", body: "host=example", want: 200},
{name: "upload", method: "POST", path: "/api/projects/id/vendor-import", contentType: "multipart/form-data; boundary=example", body: "example", want: 200},
{name: "multipart JSON route", method: "POST", contentType: "multipart/form-data; boundary=example", body: "example", want: 415},
{name: "bodyless action", method: "POST", want: 200},
{name: "local preflight", method: "OPTIONS", origin: "http://localhost:8080", want: 204},
{name: "foreign preflight", method: "OPTIONS", origin: "https://attacker.example", want: 403},
{name: "IPv6", method: "GET", host: "[::1]:8080", want: 200},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
router := gin.New()
router.Use(CORS())
called := false
handler := func(c *gin.Context) { called = true; c.Status(200) }
router.Any("/api/configs", handler)
router.Any("/setup", handler)
router.POST("/api/projects/:uuid/vendor-import", handler)
path := tt.path
if path == "" {
path = "/api/configs"
}
req := httptest.NewRequest(tt.method, path, strings.NewReader(tt.body))
req.Host = tt.host
if req.Host == "" {
req.Host = "127.0.0.1:8080"
}
for key, value := range map[string]string{"Origin": tt.origin, "Referer": tt.referer, "Sec-Fetch-Site": tt.fetchSite, "Content-Type": tt.contentType} {
if value != "" {
req.Header.Set(key, value)
}
}
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != tt.want || called != (tt.want == http.StatusOK) {
t.Fatalf("status=%d called=%v, want %d", w.Code, called, tt.want)
}
})
}
}
+36
View File
@@ -31,6 +31,40 @@ func (c *ConfigItems) Scan(value interface{}) error {
return json.Unmarshal(bytes, c)
}
// FeatureFlags lists the data features a configuration uses (see internal/configfeatures).
// Stored as a JSON array; an empty list is stored as NULL.
type FeatureFlags []string
func (f FeatureFlags) Value() (driver.Value, error) {
if len(f) == 0 {
return nil, nil
}
data, err := json.Marshal([]string(f))
if err != nil {
return nil, err
}
return string(data), nil
}
func (f *FeatureFlags) Scan(value interface{}) error {
*f = nil
var data []byte
switch v := value.(type) {
case nil:
return nil
case []byte:
data = v
case string:
data = []byte(v)
default:
return errors.New("type assertion failed for FeatureFlags")
}
if len(data) == 0 {
return nil
}
return json.Unmarshal(data, f)
}
func (c ConfigItems) Total() float64 {
var total float64
for _, item := range c {
@@ -137,6 +171,8 @@ type Configuration struct {
TotalPrice *float64 `gorm:"type:decimal(12,2)" json:"total_price"`
CustomPrice *float64 `gorm:"type:decimal(12,2)" json:"custom_price"`
Notes string `gorm:"type:text" json:"notes"`
FeatureFlags FeatureFlags `gorm:"type:json" json:"feature_flags,omitempty"`
UnsupportedFeatures []string `gorm:"-" json:"unsupported_features,omitempty"` // flags this build doesn't know; computed on read
IsTemplate bool `gorm:"default:false" json:"is_template"`
ServerCount int `gorm:"default:1" json:"server_count"`
ServerModel string `gorm:"size:100" json:"server_model,omitempty"`
+4 -55
View File
@@ -1,57 +1,6 @@
package models
import (
"log/slog"
"strings"
"gorm.io/gorm"
)
// AllModels returns all models for auto-migration
func AllModels() []interface{} {
return []interface{}{
&Category{},
&Project{},
&Configuration{},
&Pricelist{},
&PricelistItem{},
}
}
// Migrate runs auto-migration for all QuoteForge tables
// Handles MySQL constraint errors gracefully for existing tables
func Migrate(db *gorm.DB) error {
for _, model := range AllModels() {
if err := db.AutoMigrate(model); err != nil {
// Skip known MySQL constraint errors for existing tables
errStr := err.Error()
if strings.Contains(errStr, "Can't DROP") ||
strings.Contains(errStr, "Duplicate key name") ||
strings.Contains(errStr, "check that it exists") ||
strings.Contains(errStr, "Cannot change column") ||
strings.Contains(errStr, "used in a foreign key constraint") {
slog.Warn("migration warning (skipped)", "model", model, "error", errStr)
continue
}
return err
}
}
return nil
}
// SeedCategories upserts default categories, updating display_order on existing rows.
func SeedCategories(db *gorm.DB) error {
for _, cat := range DefaultCategories {
var existing Category
if err := db.Where("code = ?", cat.Code).First(&existing).Error; err != nil {
if err := db.Create(&cat).Error; err != nil {
return err
}
} else {
if err := db.Model(&existing).Update("display_order", cat.DisplayOrder).Error; err != nil {
return err
}
}
}
return nil
}
// Server schema changes are SQL files in migrations/, applied by `qfs -migrate`
// (models.RunSQLMigrations). There is deliberately no GORM AutoMigrate against
// MariaDB: it retypes production columns to match whatever models the running binary
// was built with — an old binary run with -migrate once did exactly that.
@@ -0,0 +1,162 @@
//go:build integration
package models
import (
"context"
"os"
"os/exec"
"regexp"
"strings"
"testing"
"testing/fstest"
"time"
qfassets "git.mchus.pro/mchus/quoteforge"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
// Input is read-only. All SQL runs in a new, disposable local container.
func TestProductionDumpMigrationMariaDB(t *testing.T) {
dump := os.Getenv("QFS_TEST_PRODUCTION_DUMP")
if dump == "" {
t.Skip("QFS_TEST_PRODUCTION_DUMP is not set")
}
input, err := os.Open(dump)
if err != nil {
t.Fatal(err)
}
defer input.Close()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
image := os.Getenv("QFS_TEST_MARIADB_IMAGE")
if image == "" {
image = "mariadb:11.8.3"
}
out, err := exec.CommandContext(ctx, "docker", "run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306",
"-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "-e", "MARIADB_DATABASE=RFQ_LOG", image,
"--skip-ssl", "--character-set-server=utf8mb4", "--collation-server=utf8mb4_uca1400_ai_ci", "--max-allowed-packet=256M").CombinedOutput()
if err != nil {
t.Fatalf("start dump rehearsal MariaDB: %v", err)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if _, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop dump rehearsal MariaDB: %v", err)
}
})
out, err = exec.CommandContext(ctx, "docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatal(err)
}
dsn := "root@tcp(" + strings.TrimSpace(string(out)) + ")/RFQ_LOG?parseTime=true&timeout=1s&tls=false"
var db *gorm.DB
deadline := time.Now().Add(45 * time.Second)
for {
db, err = gorm.Open(mysql.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err == nil {
break
}
if time.Now().After(deadline) {
t.Fatal("dump rehearsal database did not become ready")
}
time.Sleep(150 * time.Millisecond)
}
pool, err := db.DB()
if err != nil {
t.Fatal(err)
}
defer pool.Close()
pool.SetMaxOpenConns(1)
pool.SetMaxIdleConns(1)
importer := exec.CommandContext(ctx, "docker", "exec", "-i", id, "mariadb", "--skip-ssl", "--binary-mode", "--default-character-set=utf8mb4", "RFQ_LOG")
importer.Stdin = input
if output, err := importer.CombinedOutput(); err != nil {
// Import errors can include entire data rows; report only error code/line.
location := regexp.MustCompile(`(?m)^ERROR[^\n]*`).Find(output)
location = regexp.MustCompile(`'[^']*'`).ReplaceAll(location, []byte("'<redacted>'"))
if len(location) > 250 {
location = location[:250]
}
t.Fatalf("load dump into disposable database: %v (%s)", err, location)
}
var version, collation string
if err := db.Raw("SELECT VERSION(), @@collation_database").Row().Scan(&version, &collation); err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(version, "11.8.3-") || collation != "utf8mb4_uca1400_ai_ci" {
t.Fatalf("unexpected rehearsal server: version=%s collation=%s", version, collation)
}
var tables, triggers int64
if err := db.Raw("SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA=DATABASE()").Scan(&tables).Error; err != nil {
t.Fatal(err)
}
if err := db.Raw("SELECT COUNT(*) FROM information_schema.TRIGGERS WHERE TRIGGER_SCHEMA=DATABASE()").Scan(&triggers).Error; err != nil {
t.Fatal(err)
}
t.Logf("production dump imported: MariaDB=%s collation=%s tables=%d triggers=%d", version, collation, tables, triggers)
if err := db.Exec("SET time_zone='+00:00'").Error; err != nil {
t.Fatal(err)
}
root := t.TempDir()
var original string
if err := db.Connection(func(conn *gorm.DB) error {
var err error
original, err = mariadb.CreateSnapshot(conn, root, "production-dump")
if err != nil {
return err
}
return mariadb.VerifySnapshot(conn, original)
}); err != nil {
t.Fatal(err)
}
t.Log("full dump snapshot restore verified")
const filename = "035_add_server_updated_at.sql"
if err := db.Where("filename=?", filename).Delete(&SQLSchemaMigration{}).Error; err != nil {
t.Fatal(err)
}
// Reconstruct the real pre-035 schema only in this disposable clone.
for _, table := range []string{"qt_configurations", "qt_projects"} {
if err := db.Exec("ALTER TABLE " + table + " DROP INDEX IF EXISTS idx_" + table + "_server_updated_at, DROP COLUMN IF EXISTS server_updated_at").Error; err != nil {
t.Fatal(err)
}
}
var before string
if err := db.Connection(func(conn *gorm.DB) error {
var err error
before, err = mariadb.CreateSnapshot(conn, root, "before-rehearsal")
return err
}); err != nil {
t.Fatal(err)
}
content, err := qfassets.MigrationFiles.ReadFile("migrations/" + filename)
if err != nil {
t.Fatal(err)
}
files := fstest.MapFS{filename: &fstest.MapFile{Data: content}, "036_recovery_probe.sql": &fstest.MapFile{Data: []byte("-- Tables affected: qt_configurations\n-- verify: deliberate rehearsal failure | SELECT 1\nALTER TABLE qt_configurations ADD COLUMN qfs_recovery_probe INT;")}}
if _, err := RunSQLMigrations(db, files, nil, root); err == nil || !strings.Contains(err.Error(), "all session changes restored") {
t.Fatalf("dump rollback rehearsal: %v", err)
}
if err := mariadb.CheckSnapshot(db, before); err != nil {
t.Fatalf("dump changed after rollback: %v", err)
}
t.Log("real migration plus later failed DDL/verify restored the complete dump")
delete(files, "036_recovery_probe.sql")
if applied, err := RunSQLMigrations(db, files, nil, root); err != nil || len(applied) != 1 {
t.Fatalf("real migration rehearsal: applied=%v error=%v", applied, err)
}
states, err := SQLMigrationStatus(db, files)
if err != nil || len(states) != 1 || states[0].AppliedAt == nil {
t.Fatalf("rehearsal history invalid: %v", err)
}
if err := RestoreSQLMigrationSession(db, original); err != nil {
t.Fatal(err)
}
if err := mariadb.CheckSnapshot(db, original); err != nil {
t.Fatal(err)
}
t.Logf("production dump rehearsal complete: %s, exact original restored", filename)
}
@@ -0,0 +1,44 @@
package models
import (
"crypto/sha256"
"fmt"
"strings"
"gorm.io/driver/mysql"
"gorm.io/gorm"
)
// migrationJournalName uses the configured endpoint, which survives server restarts.
func migrationJournalName(db *gorm.DB, schema string) (string, error) {
dialect, ok := db.Dialector.(*mysql.Dialector)
if !ok || dialect.DSNConfig == nil {
return "", fmt.Errorf("migration requires a MariaDB DSN")
}
identity := dialect.DSNConfig.Net + "/" + dialect.DSNConfig.Addr + "/" + schema
return fmt.Sprintf("qfs-migration-v1-%x.pending", sha256.Sum256([]byte(identity))), nil
}
func requireExclusiveMigrationSession(db *gorm.DB) error {
var grants []string
if err := db.Session(&gorm.Session{NewDB: true}).Raw("SHOW GRANTS FOR CURRENT_USER").Scan(&grants).Error; err != nil {
return fmt.Errorf("read maintenance privileges: %w", err)
}
allowed := false
for _, grant := range grants {
if strings.Contains(grant, " ON *.* ") && strings.Contains(grant, "ALL PRIVILEGES") {
allowed = true
}
}
if !allowed {
return fmt.Errorf("migration recovery requires a DBA with directly granted global ALL PRIVILEGES to inspect all objects and clients")
}
var clients int64
if err := db.Session(&gorm.Session{NewDB: true}).Raw("SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE ID<>CONNECTION_ID() AND DB=DATABASE()").Scan(&clients).Error; err != nil {
return fmt.Errorf("check maintenance clients: %w", err)
}
if clients != 0 {
return fmt.Errorf("migration requires maintenance window: %d other connections use this schema", clients)
}
return nil
}
+45
View File
@@ -0,0 +1,45 @@
package models
import (
"fmt"
"io/fs"
"regexp"
"gorm.io/gorm"
)
func planSQLMigrations(db *gorm.DB, migrations fs.FS) ([]plannedSQLMigration, error) {
states, err := SQLMigrationStatus(db, migrations)
if err != nil {
return nil, err
}
var plans []plannedSQLMigration
unsupported := regexp.MustCompile(`(?i)\b(?:CREATE|DROP|ALTER)\s+(?:DATABASE|SCHEMA|VIEW|TRIGGER|PROCEDURE|FUNCTION|EVENT)\b|\b(?:USE|DELIMITER|GRANT|REVOKE)\s|\bSET\s+(?:GLOBAL|SQL_MODE)\b|\bENGINE\s*=\s*(?:MyISAM|Aria|MEMORY|BLACKHOLE)\b`)
for _, state := range states {
if state.AppliedAt != nil {
continue
}
content, err := fs.ReadFile(migrations, state.Filename)
if err != nil {
return nil, fmt.Errorf("read migration %s: %w", state.Filename, err)
}
statements, err := splitSQLStatements(string(content))
if err != nil {
return nil, fmt.Errorf("parse migration %s: %w", state.Filename, err)
}
checks, err := parseMigrationVerifications(string(content))
if err != nil {
return nil, fmt.Errorf("migration %s verify headers: %w", state.Filename, err)
}
if len(checks) == 0 {
return nil, fmt.Errorf("migration %s needs verify headers", state.Filename)
}
for _, stmt := range statements {
if unsupported.MatchString(stmt) {
return nil, fmt.Errorf("migration %s contains SQL outside supported table recovery", state.Filename)
}
}
plans = append(plans, plannedSQLMigration{state.Filename, statements, checks})
}
return plans, nil
}
+67
View File
@@ -0,0 +1,67 @@
package models
import (
"crypto/sha256"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/gorm"
)
// RestoreSQLMigrationSession is the shipped recovery path after interruption.
func RestoreSQLMigrationSession(db *gorm.DB, snapshot string) error {
return db.Connection(func(conn *gorm.DB) (err error) {
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
var schema, zone string
var acquired int
if err := fresh().Raw("SELECT DATABASE(), @@time_zone").Row().Scan(&schema, &zone); err != nil {
return fmt.Errorf("read recovery session: %w", err)
}
lock := fmt.Sprintf("qfs-migrate-%x", sha256.Sum256([]byte(schema)))
if err := fresh().Raw("SELECT GET_LOCK(?,0)", lock).Scan(&acquired).Error; err != nil || acquired != 1 {
return fmt.Errorf("migration lock unavailable for recovery")
}
defer func() {
if e := fresh().Exec("SET time_zone=?", zone).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset recovery time zone: %w", e))
}
if e := fresh().Raw("SELECT RELEASE_LOCK(?)", lock).Scan(&acquired).Error; e != nil {
err = errors.Join(err, fmt.Errorf("release recovery lock: %w", e))
}
}()
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return err
}
journalName, err := migrationJournalName(conn, schema)
if err != nil {
return err
}
journal := filepath.Join(filepath.Dir(snapshot), journalName)
if content, e := os.ReadFile(journal); e == nil {
if strings.TrimSpace(string(content)) != snapshot {
return fmt.Errorf("pending recovery requires its original snapshot: %s", strings.TrimSpace(string(content)))
}
} else if !os.IsNotExist(e) {
return fmt.Errorf("read recovery marker: %w", e)
}
if err := fresh().Exec("SET time_zone='+00:00'").Error; err != nil {
return fmt.Errorf("set restore time zone: %w", err)
}
if err := mariadb.VerifySnapshot(fresh(), snapshot); err != nil {
return fmt.Errorf("verify recovery snapshot: %w", err)
}
if err := mariadb.RestoreSnapshot(fresh(), snapshot); err != nil {
return fmt.Errorf("restore migration session: %w", err)
}
if err := os.Remove(journal); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("clear recovery marker: %w", err)
}
slog.Info("server migration snapshot restored", "schema", schema, "backup", snapshot, "state", "verified")
return nil
})
}
+165
View File
@@ -0,0 +1,165 @@
package models
import (
"crypto/sha256"
"errors"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/appstate"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/gorm"
)
type plannedSQLMigration struct {
filename string
statements []string
checks []migrationVerification
}
// RunSQLMigrations requires a DBA maintenance window: stop all other writers.
// Backups are mandatory; each file gets per-table snapshots before any DDL.
func RunSQLMigrations(db *gorm.DB, migrations fs.FS, onApply func(string), backupRoot string) ([]string, error) {
var applied []string
err := db.Connection(func(conn *gorm.DB) (err error) {
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
var schema, mode, zone, engine string
var foreignKeys int
if err := fresh().Raw("SELECT DATABASE(), @@sql_mode, @@time_zone, @@foreign_key_checks, @@default_storage_engine").Row().Scan(&schema, &mode, &zone, &foreignKeys, &engine); err != nil {
return fmt.Errorf("read migration session: %w", err)
}
if schema == "" || strings.Contains(mode, "NO_BACKSLASH_ESCAPES") || engine != "InnoDB" {
return fmt.Errorf("migration requires a selected InnoDB schema and backslash escape SQL mode")
}
lock := fmt.Sprintf("qfs-migrate-%x", sha256.Sum256([]byte(schema)))
var acquired int
if err := fresh().Raw("SELECT GET_LOCK(?,0)", lock).Scan(&acquired).Error; err != nil || acquired != 1 {
return fmt.Errorf("migration lock unavailable for %s", schema)
}
defer func() {
for _, reset := range []struct {
sql string
value any
}{{"SET time_zone=?", zone}, {"SET sql_mode=?", mode}, {"SET foreign_key_checks=?", foreignKeys}} {
if e := fresh().Exec(reset.sql, reset.value).Error; e != nil {
err = errors.Join(err, fmt.Errorf("reset migration session: %w", e))
}
}
if e := fresh().Raw("SELECT RELEASE_LOCK(?)", lock).Scan(&acquired).Error; e != nil {
err = errors.Join(err, fmt.Errorf("release migration lock: %w", e))
}
}()
plans, err := planSQLMigrations(fresh(), migrations)
if err != nil {
return err
}
if err := appstate.ValidateBackupLocation(backupRoot); err != nil {
return err
}
if err := os.MkdirAll(backupRoot, 0700); err != nil {
return fmt.Errorf("create migration backup root: %w", err)
}
journalName, err := migrationJournalName(conn, schema)
if err != nil {
return err
}
journal := filepath.Join(backupRoot, journalName)
if content, e := os.ReadFile(journal); e == nil {
return fmt.Errorf("interrupted migration requires restore before retry; safety snapshot: %s", strings.TrimSpace(string(content)))
} else if !os.IsNotExist(e) {
return fmt.Errorf("read migration recovery marker: %w", e)
}
if len(plans) == 0 {
return nil
}
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return err
}
if err := fresh().Exec("SET time_zone='+00:00'").Error; err != nil {
return fmt.Errorf("set snapshot time zone: %w", err)
}
full, err := mariadb.CreateSnapshot(fresh(), backupRoot, "session")
if err != nil {
return fmt.Errorf("pre-migration snapshot: %w", err)
}
if err := mariadb.VerifySnapshot(fresh(), full); err != nil {
return fmt.Errorf("pre-migration restore verification: %w", err)
}
marker, err := os.OpenFile(journal, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
return fmt.Errorf("create migration recovery marker: %w", err)
}
_, writeErr := fmt.Fprintln(marker, full)
syncErr, closeErr := marker.Sync(), marker.Close()
if e := errors.Join(writeErr, syncErr, closeErr); e != nil {
return fmt.Errorf("persist migration recovery marker: %w", e)
}
slog.Info("verified server migration backup", "schema", schema, "path", full)
var steps []string
for _, plan := range plans {
if err := requireExclusiveMigrationSession(fresh()); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
step, err := mariadb.CreateSnapshot(fresh(), backupRoot, plan.filename)
if err == nil {
err = mariadb.VerifySnapshot(fresh(), step)
}
if err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
steps = append(steps, step)
if onApply != nil {
onApply(plan.filename)
}
if err := ensureSQLMigrationsTable(fresh()); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
if err := applySQLMigrationOnSession(fresh(), plan.filename, plan.statements, plan.checks); err != nil {
return recoverSQLMigration(fresh(), journal, full, steps, plan.filename, err)
}
applied = append(applied, plan.filename)
}
if err := os.Remove(journal); err != nil {
return fmt.Errorf("remove completed migration marker: %w", err)
}
return nil
})
if err != nil {
return nil, err
}
return applied, nil
}
func recoverSQLMigration(db *gorm.DB, journal, full string, steps []string, filename string, cause error) error {
slog.Error("server migration failed", "step", filename, "error", cause, "backup", full)
if err := requireExclusiveMigrationSession(db); err != nil {
return errors.Join(cause, fmt.Errorf("recovery refused with active clients; keep maintenance window and restore %s: %w", full, err))
}
var recovery error
for i := len(steps) - 1; i >= 0; i-- {
slog.Warn("restoring migration step", "step", filename, "backup", steps[i])
if err := mariadb.RestoreSnapshot(db, steps[i]); err != nil {
recovery = err
break
}
}
if recovery == nil {
recovery = mariadb.CheckSnapshot(db, full)
}
if recovery != nil {
slog.Warn("restoring full migration session", "backup", full, "error", recovery)
recovery = mariadb.RestoreSnapshot(db, full)
}
if recovery != nil {
return errors.Join(cause, fmt.Errorf("recovery failed; database requires restore from %s: %w", full, recovery))
}
if err := os.Remove(journal); err != nil {
return errors.Join(cause, fmt.Errorf("database restored but recovery marker remains: %w", err))
}
slog.Warn("server migration session restored", "step", filename, "state", "before-session", "backup", full)
return fmt.Errorf("migration %s failed; all session changes restored: %w", filename, cause)
}
@@ -0,0 +1,227 @@
//go:build integration
package models
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"testing/fstest"
"time"
"git.mchus.pro/mchus/quoteforge/internal/mariadb"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func TestSQLMigrationSafetyMariaDB(t *testing.T) {
out, err := exec.Command("docker", "run", "--detach", "--rm", "--pull=never", "-p", "127.0.0.1::3306", "-e", "MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1", "-e", "MARIADB_ROOT_HOST=%", "mariadb:11.8", "--skip-ssl").CombinedOutput()
if err != nil {
t.Fatalf("start test MariaDB: %v: %s", err, out)
}
id := strings.TrimSpace(string(out))
t.Cleanup(func() {
if out, err := exec.Command("docker", "stop", "-t", "2", id).CombinedOutput(); err != nil {
t.Errorf("stop test MariaDB: %v: %s", err, out)
}
})
out, err = exec.Command("docker", "port", id, "3306/tcp").CombinedOutput()
if err != nil {
t.Fatalf("get test MariaDB port: %v: %s", err, out)
}
addr := strings.TrimSpace(string(out))
dsn := "root@tcp(" + addr + ")/?parseTime=true&timeout=1s&tls=false"
var admin *gorm.DB
deadline := time.Now().Add(45 * time.Second)
for {
admin, err = gorm.Open(mysql.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err == nil {
break
}
if time.Now().After(deadline) {
t.Fatalf("test MariaDB unavailable: %v", err)
}
time.Sleep(150 * time.Millisecond)
}
pool, err := admin.DB()
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pool.Close() })
newDB := func(t *testing.T) *gorm.DB {
t.Helper()
schema := fmt.Sprintf("qfs_test_%d", time.Now().UnixNano())
if err := admin.Exec("CREATE DATABASE " + schema + " COLLATE utf8mb4_unicode_ci").Error; err != nil {
t.Fatal(err)
}
db, err := gorm.Open(mysql.Open(strings.Replace(dsn, ")/?", ")/"+schema+"?", 1)), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatal(err)
}
p, err := db.DB()
if err != nil {
t.Fatal(err)
}
p.SetMaxOpenConns(1)
p.SetMaxIdleConns(1)
t.Cleanup(func() {
p.Close()
if err := admin.Exec("DROP DATABASE " + schema).Error; err != nil {
t.Error(err)
}
})
for _, sql := range []string{
"SET time_zone='+00:00'",
"CREATE TABLE parent (id INT PRIMARY KEY AUTO_INCREMENT, value TEXT, empty_value TEXT, nullable TEXT, raw VARBINARY(32), amount DECIMAL(20,6), stamp TIMESTAMP(6), generated INT AS (id+1) VIRTUAL) ENGINE=InnoDB",
"INSERT INTO parent (id,value,empty_value,nullable,raw,amount,stamp) VALUES (1,'quote; and ''text''','',NULL,X'0001FF',123.456789,'2026-10-26 12:34:56.123456')",
"SET @saved_test_mode=@@sql_mode",
"SET sql_mode='NO_AUTO_VALUE_ON_ZERO'",
"INSERT INTO parent (id,value,stamp) VALUES (0,'legacy','0000-00-00 00:00:00')",
"SET sql_mode=@saved_test_mode",
"CREATE TABLE child (id INT PRIMARY KEY, parent_id INT, CONSTRAINT child_parent FOREIGN KEY (parent_id) REFERENCES parent(id)) ENGINE=InnoDB",
"INSERT INTO child VALUES (1,1)",
"CREATE TABLE audit (id INT PRIMARY KEY AUTO_INCREMENT, note TEXT) ENGINE=InnoDB",
"CREATE TRIGGER insert_value BEFORE INSERT ON parent FOR EACH ROW SET NEW.value=CONCAT(NEW.value,'!')",
"CREATE TRIGGER audit_z AFTER UPDATE ON parent FOR EACH ROW INSERT INTO audit (note) VALUES ('first')",
"CREATE TRIGGER audit_a AFTER UPDATE ON parent FOR EACH ROW INSERT INTO audit (note) VALUES ('second')",
} {
if err := db.Exec(sql).Error; err != nil {
t.Fatal(err)
}
}
return db
}
migration := func(sql, checks string) *fstest.MapFile {
return &fstest.MapFile{Data: []byte("-- Tables affected: parent, child\n" + checks + "\n" + sql)}
}
for _, failure := range []string{"none", "verify", "FK"} {
t.Run(failure, func(t *testing.T) {
db, root := newDB(t), t.TempDir()
var original string
err := db.Connection(func(conn *gorm.DB) error {
var e error
original, e = mariadb.CreateSnapshot(conn, root, "original")
return e
})
if err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("SET @example='value; quoted'; PREPARE q FROM 'SELECT 1'; EXECUTE q; DEALLOCATE PREPARE q; ALTER TABLE parent ADD COLUMN added INT; UPDATE parent SET value=@example;", "-- verify: added missing | SELECT 1 FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='parent' AND COLUMN_NAME='added' HAVING COUNT(*)=0\n-- verify: wrong value | SELECT 1 FROM parent WHERE value<>'value; quoted'")}
if failure == "verify" {
files["002_b.sql"] = migration("CREATE TABLE new_table (id INT); UPDATE parent SET value='changed';", "-- verify: deliberate failure | SELECT 1")
}
if failure == "FK" {
files["002_b.sql"] = migration("ALTER TABLE child ADD COLUMN added INT; ALTER TABLE child ADD CONSTRAINT broken_fk FOREIGN KEY (added) REFERENCES parent(missing);", "-- verify: should never run | SELECT 1 WHERE 0")
}
applied, err := RunSQLMigrations(db, files, nil, root)
if failure == "none" {
if err != nil || len(applied) != 1 {
t.Fatalf("apply=%v error=%v", applied, err)
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Count(&count).Error; err != nil || count != 1 {
t.Fatalf("history=%d err=%v", count, err)
}
} else {
if err == nil || !strings.Contains(err.Error(), "all session changes restored") || len(applied) != 0 {
t.Fatalf("apply=%v error=%v", applied, err)
}
if err := mariadb.CheckSnapshot(db, original); err != nil {
t.Fatalf("recovery changed original state: %v", err)
}
}
markers, err := filepath.Glob(filepath.Join(root, "*.pending"))
if err != nil || len(markers) != 0 {
t.Fatalf("markers=%v error=%v", markers, err)
}
})
}
t.Run("interrupted session", func(t *testing.T) {
db, root := newDB(t), t.TempDir()
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
func() {
defer func() {
if recover() == nil {
t.Error("expected interruption")
}
}()
RunSQLMigrations(db, files, func(string) { panic("interruption") }, root)
}()
if err := db.Exec("ALTER TABLE parent ADD COLUMN added INT").Error; err != nil {
t.Fatal(err)
}
if _, err := RunSQLMigrations(db, files, nil, root); err == nil || !strings.Contains(err.Error(), "interrupted migration") {
t.Fatalf("retry not refused: %v", err)
}
markers, err := filepath.Glob(filepath.Join(root, "*.pending"))
if err != nil || len(markers) != 1 {
t.Fatalf("markers=%v err=%v", markers, err)
}
data, err := os.ReadFile(markers[0])
if err != nil {
t.Fatal(err)
}
if err := RestoreSQLMigrationSession(db, strings.TrimSpace(string(data))); err != nil {
t.Fatal(err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("interrupted DDL not restored")
}
if _, err := RunSQLMigrations(db, files, nil, root); err != nil {
t.Fatalf("retry after restore: %v", err)
}
})
t.Run("unsupported object", func(t *testing.T) {
db := newDB(t)
if err := db.Exec("CREATE VIEW example_view AS SELECT id FROM parent").Error; err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, t.TempDir()); err == nil || !strings.Contains(err.Error(), "snapshot refused") {
t.Fatalf("unsupported schema accepted: %v", err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("unsupported schema was changed")
}
})
t.Run("other client", func(t *testing.T) {
db := newDB(t)
p, err := db.DB()
if err != nil {
t.Fatal(err)
}
p.SetMaxOpenConns(2)
other, err := p.Conn(context.Background())
if err != nil {
t.Fatal(err)
}
defer other.Close()
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, t.TempDir()); err == nil || !strings.Contains(err.Error(), "other connections") {
t.Fatalf("active client accepted: %v", err)
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("active schema changed")
}
})
t.Run("backup failure", func(t *testing.T) {
db := newDB(t)
path := filepath.Join(t.TempDir(), "not-directory")
if err := os.WriteFile(path, []byte("example"), 0600); err != nil {
t.Fatal(err)
}
files := fstest.MapFS{"001_a.sql": migration("ALTER TABLE parent ADD COLUMN added INT", "-- verify: ok | SELECT 1 WHERE 0")}
if _, err := RunSQLMigrations(db, files, nil, path); err == nil {
t.Fatal("backup failure ignored")
}
if db.Table("parent").Migrator().HasColumn(&struct{ Added int }{}, "added") {
t.Fatal("schema changed without backup")
}
})
}
+76
View File
@@ -0,0 +1,76 @@
package models
import (
"fmt"
"strings"
"gorm.io/gorm"
)
type migrationVerification struct {
description string
query string
}
func parseMigrationVerifications(script string) ([]migrationVerification, error) {
var checks []migrationVerification
for _, line := range strings.Split(script, "\n") {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "-- verify:") {
continue
}
description, query, ok := strings.Cut(strings.TrimPrefix(line, "-- verify:"), "|")
description, query = strings.TrimSpace(description), strings.TrimSpace(query)
if !ok || description == "" || query == "" {
return nil, fmt.Errorf("invalid migration verify header")
}
checks = append(checks, migrationVerification{description: description, query: query})
}
return checks, nil
}
func applySQLMigrationFile(db *gorm.DB, filename, content string) error {
checks, err := parseMigrationVerifications(content)
if err != nil {
return fmt.Errorf("parse migration %s: %w", filename, err)
}
statements, err := splitSQLStatements(content)
if err != nil {
return fmt.Errorf("parse migration %s SQL: %w", filename, err)
}
return db.Connection(func(conn *gorm.DB) error {
return applySQLMigrationOnSession(conn, filename, statements, checks)
})
}
func applySQLMigrationOnSession(conn *gorm.DB, filename string, statements []string, checks []migrationVerification) error {
if err := executeMigrationStatements(conn, filename, statements); err != nil {
return err
}
// Each query gets a fresh statement on the pinned connection: chaining
// Raw/Create on conn itself leaks the previous query's state (with two
// verify checks the history insert targeted a table named "rows").
fresh := func() *gorm.DB { return conn.Session(&gorm.Session{NewDB: true}) }
for _, check := range checks {
rows, err := fresh().Raw(check.query).Rows()
if err != nil {
return fmt.Errorf("verify migration %s (%s): %w", filename, check.description, err)
}
failed := rows.Next()
rowErr := rows.Err()
closeErr := rows.Close()
if rowErr != nil {
return fmt.Errorf("read migration verification %s (%s): %w", filename, check.description, rowErr)
}
if closeErr != nil {
return fmt.Errorf("close migration verification %s (%s): %w", filename, check.description, closeErr)
}
if failed {
return fmt.Errorf("verify migration %s failed: %s", filename, check.description)
}
}
if err := fresh().Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record migration %s: %w", filename, err)
}
return nil
}
@@ -0,0 +1,70 @@
package models
import (
"path/filepath"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestMigrationVerification(t *testing.T) {
for _, tt := range []struct{ name, content, wantError string }{
{"valid", "-- verify: bad value | SELECT 1 FROM example WHERE value <> 1\nCREATE TEMP TABLE example (value INT); INSERT INTO example VALUES (1);", ""},
{"failed", "-- verify: bad value | SELECT 1 FROM example WHERE value <> 1\nCREATE TEMP TABLE example (value INT); INSERT INTO example VALUES (2);", "bad value"},
{"query error", "-- verify: missing table | SELECT 1 FROM missing\nCREATE TEMP TABLE example (value INT);", "missing table"},
{"malformed", "-- verify: no query\nCREATE TABLE example (value INT);", "invalid migration verify header"},
{"two checks pass", "-- verify: first | SELECT 1 WHERE 0\n-- verify: second | SELECT 1 WHERE 0\nCREATE TEMP TABLE example (value INT);", ""},
{"second check fails", "-- verify: first | SELECT 1 WHERE 0\n-- verify: second | SELECT 1\nCREATE TEMP TABLE example (value INT);", "second"},
{"execution failure", "-- verify: none | SELECT 1 WHERE 0\nINVALID SQL;", "exec migration"},
} {
t.Run(tt.name, func(t *testing.T) {
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "migration.db")), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, _ := db.DB()
defer sqlDB.Close()
if err := db.AutoMigrate(&SQLSchemaMigration{}); err != nil {
t.Fatal(err)
}
err = applySQLMigrationFile(db, "001_example.sql", tt.content)
if tt.wantError != "" && (err == nil || !strings.Contains(err.Error(), tt.wantError)) {
t.Fatalf("want %q, got %v", tt.wantError, err)
}
if tt.wantError == "" && err != nil {
t.Fatal(err)
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Count(&count).Error; err != nil {
t.Fatal(err)
}
if (count == 1) != (tt.wantError == "") {
t.Fatalf("incorrect history count=%d", count)
}
if tt.name == "malformed" && db.Migrator().HasTable("example") {
t.Fatal("malformed header mutated database")
}
})
}
}
func TestMigrationErrorAllowlist(t *testing.T) {
for _, tt := range []struct {
message string
want bool
}{
{"Duplicate column name 'example'", true},
{"Duplicate key name 'example'", true},
{"Table 'example' already exists", true},
{"Can't create table example (errno: 150 Foreign key constraint is incorrectly formed)", false},
{"Can't create table example (errno: 121 Duplicate key)", false},
{"Duplicate foreign key constraint name 'example'", false},
{"Access denied", false},
} {
if got := isIgnorableMigrationError(tt.message); got != tt.want {
t.Errorf("%s: got %v", tt.message, got)
}
}
}
+85 -88
View File
@@ -1,11 +1,9 @@
package models
import (
"bufio"
"errors"
"fmt"
"os"
"path/filepath"
"io/fs"
"sort"
"strings"
"time"
@@ -24,10 +22,87 @@ func (SQLSchemaMigration) TableName() string {
return "qt_schema_migrations"
}
// NeedsSQLMigrations reports whether at least one SQL migration from migrationsDir
// SQLMigrationState is one server migration file and when it was recorded as applied
// (nil = pending).
type SQLMigrationState struct {
Filename string
AppliedAt *time.Time
}
// SQLMigrationStatus lists every server migration in migrations with its applied state.
// Read-only: does not create qt_schema_migrations.
func SQLMigrationStatus(db *gorm.DB, migrations fs.FS) ([]SQLMigrationState, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return nil, err
}
applied := map[string]time.Time{}
if db.Migrator().HasTable(&SQLSchemaMigration{}) {
var rows []SQLSchemaMigration
if err := db.Find(&rows).Error; err != nil {
return nil, fmt.Errorf("read qt_schema_migrations: %w", err)
}
for _, r := range rows {
applied[r.Filename] = r.AppliedAt
}
}
states := make([]SQLMigrationState, 0, len(files))
for _, f := range files {
st := SQLMigrationState{Filename: f}
if at, ok := applied[f]; ok {
at := at
st.AppliedAt = &at
}
states = append(states, st)
}
return states, nil
}
// BaselineSQLMigrations records every migration up to and including upto as applied
// WITHOUT executing it — for a server whose schema was brought up to date by other
// means before qt_schema_migrations was used. Returns the filenames newly recorded.
func BaselineSQLMigrations(db *gorm.DB, migrations fs.FS, upto string) ([]string, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return nil, err
}
found := false
for _, f := range files {
if f == upto {
found = true
break
}
}
if !found {
return nil, fmt.Errorf("baseline migration %q not found", upto)
}
if err := ensureSQLMigrationsTable(db); err != nil {
return nil, fmt.Errorf("migrate qt_schema_migrations table: %w", err)
}
var recorded []string
for _, f := range files {
if f > upto {
break
}
var count int64
if err := db.Model(&SQLSchemaMigration{}).Where("filename = ?", f).Count(&count).Error; err != nil {
return recorded, fmt.Errorf("check migration %s: %w", f, err)
}
if count > 0 {
continue
}
if err := db.Create(&SQLSchemaMigration{Filename: f}).Error; err != nil {
return recorded, fmt.Errorf("record baseline migration %s: %w", f, err)
}
recorded = append(recorded, f)
}
return recorded, nil
}
// NeedsSQLMigrations reports whether at least one SQL migration from migrations
// is not yet recorded in qt_schema_migrations.
func NeedsSQLMigrations(db *gorm.DB, migrationsDir string) (bool, error) {
files, err := listSQLMigrationFiles(migrationsDir)
func NeedsSQLMigrations(db *gorm.DB, migrations fs.FS) (bool, error) {
files, err := listSQLMigrationFiles(migrations)
if err != nil {
return false, err
}
@@ -48,52 +123,6 @@ func NeedsSQLMigrations(db *gorm.DB, migrationsDir string) (bool, error) {
return count < int64(len(files)), nil
}
// RunSQLMigrations applies SQL files from migrationsDir once and records them in qt_schema_migrations.
// Local SQLite-only scripts are skipped automatically.
func RunSQLMigrations(db *gorm.DB, migrationsDir string) error {
if err := ensureSQLMigrationsTable(db); err != nil {
return fmt.Errorf("migrate qt_schema_migrations table: %w", err)
}
files, err := listSQLMigrationFiles(migrationsDir)
if err != nil {
return err
}
for _, filename := range files {
var count int64
if err := db.Model(&SQLSchemaMigration{}).Where("filename = ?", filename).Count(&count).Error; err != nil {
return fmt.Errorf("check migration %s: %w", filename, err)
}
if count > 0 {
continue
}
path := filepath.Join(migrationsDir, filename)
content, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("read migration %s: %w", filename, err)
}
statements := splitSQLStatements(string(content))
if len(statements) == 0 {
if err := db.Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record empty migration %s: %w", filename, err)
}
continue
}
if err := executeMigrationStatements(db, filename, statements); err != nil {
return err
}
if err := db.Create(&SQLSchemaMigration{Filename: filename}).Error; err != nil {
return fmt.Errorf("record migration %s: %w", filename, err)
}
}
return nil
}
// IsMigrationPermissionError returns true if err indicates insufficient privileges
// to create/alter/read migration metadata or target schema objects.
func IsMigrationPermissionError(err error) bool {
@@ -158,9 +187,6 @@ func isIgnorableMigrationError(message string) bool {
"duplicate column name",
"duplicate key name",
"already exists",
"can't create table",
"duplicate foreign key constraint name",
"errno 121",
}
for _, pattern := range ignorable {
if strings.Contains(lower, pattern) {
@@ -170,42 +196,13 @@ func isIgnorableMigrationError(message string) bool {
return false
}
func splitSQLStatements(script string) []string {
scanner := bufio.NewScanner(strings.NewReader(script))
scanner.Buffer(make([]byte, 1024), 1024*1024)
lines := make([]string, 0, 128)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
if strings.HasPrefix(line, "--") {
continue
}
lines = append(lines, scanner.Text())
}
combined := strings.Join(lines, "\n")
raw := strings.Split(combined, ";")
stmts := make([]string, 0, len(raw))
for _, stmt := range raw {
trimmed := strings.TrimSpace(stmt)
if trimmed == "" {
continue
}
stmts = append(stmts, trimmed)
}
return stmts
}
func listSQLMigrationFiles(migrationsDir string) ([]string, error) {
entries, err := os.ReadDir(migrationsDir)
func listSQLMigrationFiles(migrations fs.FS) ([]string, error) {
entries, err := fs.ReadDir(migrations, ".")
if err != nil {
if errors.Is(err, os.ErrNotExist) {
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return nil, fmt.Errorf("read migrations dir %s: %w", migrationsDir, err)
return nil, fmt.Errorf("read migrations dir: %w", err)
}
files := make([]string, 0, len(entries))
+56
View File
@@ -0,0 +1,56 @@
package models
import (
"path/filepath"
"testing"
"testing/fstest"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
)
func TestSQLMigrationStatusAndListing(t *testing.T) {
migrations := fstest.MapFS{
"002_b.sql": {Data: []byte("ALTER TABLE x ADD COLUMN b INT;")},
"001_a.sql": {Data: []byte("ALTER TABLE x ADD COLUMN a INT;")},
"006_local_only_versions.sql": {Data: []byte("-- sqlite only")},
"README.md": {Data: []byte("not a migration")},
}
db, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "m.db")), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
// No history table yet: everything pending, nothing created.
states, err := SQLMigrationStatus(db, migrations)
if err != nil {
t.Fatal(err)
}
if len(states) != 2 || states[0].Filename != "001_a.sql" || states[1].Filename != "002_b.sql" {
t.Fatalf("unexpected listing %+v", states)
}
if states[0].AppliedAt != nil || db.Migrator().HasTable(&SQLSchemaMigration{}) {
t.Fatal("status must be read-only and report pending")
}
if err := db.AutoMigrate(&SQLSchemaMigration{}); err != nil {
t.Fatal(err)
}
if err := db.Create(&SQLSchemaMigration{Filename: "001_a.sql"}).Error; err != nil {
t.Fatal(err)
}
states, _ = SQLMigrationStatus(db, migrations)
if states[0].AppliedAt == nil || states[1].AppliedAt != nil {
t.Fatalf("want 001 applied, 002 pending: %+v", states)
}
}
func TestSplitSQLStatementsSkipsComments(t *testing.T) {
got, err := splitSQLStatements("-- header; with semicolon\nALTER TABLE a ADD COLUMN x INT;\n\nALTER TABLE b\n ADD COLUMN y INT;\n")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[1] != "ALTER TABLE b\n ADD COLUMN y INT" {
t.Fatalf("unexpected statements %q", got)
}
}
+70
View File
@@ -0,0 +1,70 @@
package models
import (
"fmt"
"strings"
)
// splitSQLStatements supports MariaDB literals, quoted identifiers and comments.
func splitSQLStatements(script string) ([]string, error) {
var statements []string
var current strings.Builder
var quote byte
for i := 0; i < len(script); i++ {
ch := script[i]
if quote != 0 {
current.WriteByte(ch)
if ch == '\\' && quote != '`' && i+1 < len(script) {
i++
current.WriteByte(script[i])
} else if ch == quote {
if i+1 < len(script) && script[i+1] == quote {
i++
current.WriteByte(script[i])
} else {
quote = 0
}
}
continue
}
switch {
case ch == '\'' || ch == '"' || ch == '`':
quote = ch
current.WriteByte(ch)
case ch == '#' || (ch == '-' && i+2 < len(script) && script[i+1] == '-' && script[i+2] <= ' '):
for i < len(script) && script[i] != '\n' {
i++
}
current.WriteByte('\n')
case ch == '/' && i+1 < len(script) && script[i+1] == '*':
if strings.HasPrefix(script[i:], "/*!") || strings.HasPrefix(script[i:], "/*M!") {
return nil, fmt.Errorf("executable SQL comments are unsupported")
}
end := strings.Index(script[i+2:], "*/")
if end < 0 {
return nil, fmt.Errorf("unterminated SQL comment")
}
i += end + 3
current.WriteByte(' ')
case ch == ';':
if stmt := strings.TrimSpace(current.String()); stmt != "" {
statements = append(statements, stmt)
}
current.Reset()
default:
current.WriteByte(ch)
}
}
if quote != 0 {
return nil, fmt.Errorf("unterminated SQL quote")
}
if stmt := strings.TrimSpace(current.String()); stmt != "" {
statements = append(statements, stmt)
}
for _, stmt := range statements {
if strings.HasPrefix(strings.ToUpper(stmt), "DELIMITER ") {
return nil, fmt.Errorf("DELIMITER directives are unsupported")
}
}
return statements, nil
}
@@ -0,0 +1,70 @@
package models
import (
qfassets "git.mchus.pro/mchus/quoteforge"
"io/fs"
"reflect"
"strings"
"testing"
)
func TestSQLStatementParser(t *testing.T) {
for _, tt := range []struct {
name, sql string
want []string
fail bool
}{
{"empty", "-- comment;\n# more;", nil, false},
{"literal separators", "SELECT 'a;b', \"c;d\", `e;f`; SELECT 2;", []string{"SELECT 'a;b', \"c;d\", `e;f`", "SELECT 2"}, false},
{"doubled quotes", "SELECT 'a'';b', `a``;b`;", []string{"SELECT 'a'';b', `a``;b`"}, false},
{"backslash quote", "SELECT 'a\\';b'; SELECT 2", []string{"SELECT 'a\\';b'", "SELECT 2"}, false},
{"inline comments", "SELECT/* ; */ 1; -- end;\nSELECT 2# end;\n;", []string{"SELECT 1", "SELECT 2"}, false},
{"not a comment", "SELECT 4--2;", []string{"SELECT 4--2"}, false},
{"unclosed quote", "SELECT 'a;", nil, true},
{"unclosed block", "SELECT 1; /*", nil, true},
{"executable comment", "/*! SELECT 1 */", nil, true},
{"MariaDB comment", "/*M! SELECT 1 */", nil, true},
{"delimiter", "DELIMITER $$\nSELECT 1", nil, true},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := splitSQLStatements(tt.sql)
if (err != nil) != tt.fail {
t.Fatalf("error=%v", err)
}
if !tt.fail && !reflect.DeepEqual(got, tt.want) {
t.Fatalf("got=%q want=%q", got, tt.want)
}
})
}
long := "SELECT '" + strings.Repeat("x", 2<<20) + ";y';"
if got, err := splitSQLStatements(long); err != nil || len(got) != 1 {
t.Fatalf("large statement: count=%d err=%v", len(got), err)
}
}
func TestEmbeddedServerMigrationSQL(t *testing.T) {
migrations, err := qfassets.MigrationsFS()
if err != nil {
t.Fatal(err)
}
files, err := listSQLMigrationFiles(migrations)
if err != nil {
t.Fatal(err)
}
for _, name := range files {
t.Run(name, func(t *testing.T) {
content, err := fs.ReadFile(migrations, name)
if err != nil {
t.Fatal(err)
}
statements, err := splitSQLStatements(string(content))
if err != nil || len(statements) == 0 {
t.Fatalf("count=%d err=%v", len(statements), err)
}
checks, err := parseMigrationVerifications(string(content))
if err != nil || len(checks) == 0 {
t.Fatalf("checks=%d err=%v", len(checks), err)
}
})
}
}
+34 -17
View File
@@ -1,6 +1,8 @@
package repository
import (
"log/slog"
"slices"
"strings"
"git.mchus.pro/mchus/quoteforge/internal/models"
@@ -16,13 +18,9 @@ func NewConfigurationRepository(db *gorm.DB) *ConfigurationRepository {
}
func (r *ConfigurationRepository) Create(config *models.Configuration) error {
if err := r.db.Create(config).Error; err != nil {
if isUnknownLineNoColumnError(err) {
return r.db.Omit("line_no").Create(config).Error
}
return err
}
return nil
return withOptionalColumns(func(omit []string) error {
return r.db.Omit(omit...).Create(config).Error
})
}
func (r *ConfigurationRepository) GetByID(id uint) (*models.Configuration, error) {
@@ -44,21 +42,40 @@ func (r *ConfigurationRepository) GetByUUID(uuid string) (*models.Configuration,
}
func (r *ConfigurationRepository) Update(config *models.Configuration) error {
if err := r.db.Save(config).Error; err != nil {
if isUnknownLineNoColumnError(err) {
return r.db.Omit("line_no").Save(config).Error
}
return err
}
return nil
return withOptionalColumns(func(omit []string) error {
return r.db.Omit(omit...).Save(config).Error
})
}
func isUnknownLineNoColumnError(err error) bool {
// optionalConfigColumns are qt_configurations columns added by server migrations that a
// not-yet-migrated server may lack (line_no: 028, feature_flags: 034). A write that fails
// on one of them is retried without it, so clients keep syncing until the migration runs.
var optionalConfigColumns = []string{"line_no", "feature_flags"}
func withOptionalColumns(write func(omit []string) error) error {
var omit []string
for {
err := write(omit)
col := unknownOptionalColumn(err)
if col == "" || slices.Contains(omit, col) {
return err
}
slog.Warn("qt_configurations column missing on server — writing without it; run server migrations", "column", col)
omit = append(omit, col)
}
}
func unknownOptionalColumn(err error) string {
if err == nil {
return false
return ""
}
msg := strings.ToLower(err.Error())
return strings.Contains(msg, "unknown column 'line_no'") || strings.Contains(msg, "no column named line_no")
for _, col := range optionalConfigColumns {
if strings.Contains(msg, "unknown column '"+col+"'") || strings.Contains(msg, "no column named "+col) {
return col
}
}
return ""
}
func (r *ConfigurationRepository) Delete(id uint) error {
+25
View File
@@ -141,6 +141,31 @@ func (r *PartnumberBookRepository) HasAllBookItems(bookID uint) bool {
return count == int64(len(book.PartnumbersJSON))
}
// MissingBookItems returns the book's partnumbers that have no local catalog row.
func (r *PartnumberBookRepository) MissingBookItems(bookID uint) []string {
book, err := r.getBook(bookID)
if err != nil || len(book.PartnumbersJSON) == 0 {
return nil
}
var present []string
if err := r.db.Model(&localdb.LocalPartnumberBookItem{}).
Where("partnumber IN ?", []string(book.PartnumbersJSON)).
Pluck("partnumber", &present).Error; err != nil {
return nil
}
have := make(map[string]struct{}, len(present))
for _, pn := range present {
have[pn] = struct{}{}
}
var missing []string
for _, pn := range book.PartnumbersJSON {
if _, ok := have[pn]; !ok {
missing = append(missing, pn)
}
}
return missing
}
func (r *PartnumberBookRepository) getBook(bookID uint) (*localdb.LocalPartnumberBook, error) {
var book localdb.LocalPartnumberBook
if err := r.db.First(&book, bookID).Error; err != nil {
+29
View File
@@ -156,6 +156,35 @@ func (r *PricelistRepository) GetLatestActiveBySource(source string) (*models.Pr
return &pricelist, nil
}
// GetLatestActiveIDsBySources returns, per source, the ID GetLatestActiveBySource
// would return, in one round trip. Sources with no such pricelist are absent.
func (r *PricelistRepository) GetLatestActiveIDsBySources(sources []string) (map[string]uint, error) {
result := make(map[string]uint, len(sources))
if len(sources) == 0 {
return result, nil
}
parts := make([]string, 0, len(sources))
args := make([]any, 0, len(sources))
for _, source := range sources {
parts = append(parts, `(SELECT id, source FROM qt_pricelists
WHERE is_active = 1 AND source = ?
AND EXISTS (SELECT 1 FROM qt_pricelist_items WHERE qt_pricelist_items.pricelist_id = qt_pricelists.id)
ORDER BY created_at DESC, id DESC LIMIT 1)`)
args = append(args, source)
}
var rows []struct {
ID uint
Source string
}
if err := r.db.Raw(strings.Join(parts, " UNION ALL "), args...).Scan(&rows).Error; err != nil {
return nil, fmt.Errorf("getting latest pricelists: %w", err)
}
for _, row := range rows {
result[row.Source] = row.ID
}
return result, nil
}
// Create creates a new pricelist
func (r *PricelistRepository) Create(pricelist *models.Pricelist) error {
if err := r.db.Create(pricelist).Error; err != nil {
+3 -1
View File
@@ -40,5 +40,7 @@ type ArticlePreviewRequest struct {
Items models.ConfigItems `json:"items"`
ServerModel string `json:"server_model"`
SupportCode string `json:"support_code,omitempty"`
PricelistID *uint `json:"pricelist_id,omitempty"`
// PricelistID is accepted for backward compatibility but ignored: article
// categories come from the component universe (world ∪ estimate), not one pricelist.
PricelistID *uint `json:"pricelist_id,omitempty"`
}
+132 -21
View File
@@ -3,6 +3,7 @@ package services
import (
"bytes"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"math"
@@ -58,9 +59,16 @@ type ProjectPricingExportOptions struct {
IncludeEstimate bool `json:"include_estimate"`
IncludeStock bool `json:"include_stock"`
IncludeCompetitor bool `json:"include_competitor"`
Basis string `json:"basis"` // "fob" or "ddp"; empty defaults to "fob"
SaleMarkup float64 `json:"sale_markup"` // DDP uplift applied to Estimate only; 0 defaults to 1.3
ManualPrice *float64 `json:"manual_price"` // user-defined total price; distributed proportionally across rows
Basis string `json:"basis"` // "fob" or "ddp"; empty defaults to "fob"
SaleMarkup float64 `json:"sale_markup"` // DDP uplift applied to Estimate only; 0 defaults to 1.3
ManualPrice *float64 `json:"manual_price"` // user-defined total price; distributed proportionally across rows
// MinimalSaleColumns/SaleRowMarkups back the Sale table's own per-row "Накидка"
// feature (index.html renderPricingTab/_saleRowPrice). Only ever set by
// ExportConfigPricingCSV when the request explicitly asks for it — the Buy export
// and project-level bulk export never set these, so their output is unaffected.
MinimalSaleColumns bool `json:"minimal_sale_columns"`
SaleRowMarkups map[string]float64 `json:"sale_row_markups"` // "<vendorPN>::<lot>" -> markup percent
}
// defaultSaleMarkup mirrors PricingMarkup.DEFAULT_SALE_UPLIFT in web/templates/index.html.
@@ -68,13 +76,41 @@ type ProjectPricingExportOptions struct {
// other API callers that omit sale_markup.
const defaultSaleMarkup = 1.3
func (o ProjectPricingExportOptions) saleMarkupFactor() float64 {
// effectiveSaleMarkupFactor resolves the DDP estimate uplift for one configuration.
// An explicit opts.SaleMarkup (sent by the single-config pricing tab, which knows the
// live-edited value) always wins. Otherwise each configuration's own saved uplift
// (persisted in Notes by the "Ценообразование" tab, see index.html buildPricingState/
// restorePricingStateFromNotes) is used, so bulk project export matches what exporting
// that same configuration individually would produce — one pipeline, not a single
// project-wide constant.
func (o ProjectPricingExportOptions) effectiveSaleMarkupFactor(cfg *models.Configuration) float64 {
if o.SaleMarkup > 0 {
return o.SaleMarkup
}
if v := configSavedSaleUplift(cfg); v > 0 {
return v
}
return defaultSaleMarkup
}
// configSavedSaleUplift reads the per-configuration DDP estimate uplift persisted by the
// pricing tab. Notes stores {"pricing_ui":{"sale_uplift":...}} as JSON — see
// serializeConfigNotes/restorePricingStateFromNotes in index.html.
func configSavedSaleUplift(cfg *models.Configuration) float64 {
if cfg == nil || strings.TrimSpace(cfg.Notes) == "" {
return 0
}
var parsed struct {
PricingUI struct {
SaleUplift float64 `json:"sale_uplift"`
} `json:"pricing_ui"`
}
if err := json.Unmarshal([]byte(cfg.Notes), &parsed); err != nil {
return 0
}
return parsed.PricingUI.SaleUplift
}
func (o ProjectPricingExportOptions) isDDP() bool {
return strings.EqualFold(strings.TrimSpace(o.Basis), "ddp")
}
@@ -265,10 +301,14 @@ func (s *ExportService) ToPricingCSV(w io.Writer, data *ProjectPricingExportData
return fmt.Errorf("failed to write pricing header: %w", err)
}
writeRows := opts.IncludeLOT || opts.IncludeBOM
writeRows := opts.IncludeLOT || opts.IncludeBOM || opts.MinimalSaleColumns
for _, cfg := range data.Configs {
if err := csvWriter.Write(pricingConfigSummaryRow(cfg, opts)); err != nil {
return fmt.Errorf("failed to write config summary row: %w", err)
// Minimal mode (single-config Sale export) skips the per-config summary line —
// it's just LOT/Описание/Кол-во/Цена item rows, see pricingCSVHeaders.
if !opts.MinimalSaleColumns {
if err := csvWriter.Write(pricingConfigSummaryRow(cfg, opts)); err != nil {
return fmt.Errorf("failed to write config summary row: %w", err)
}
}
if writeRows {
for _, row := range cfg.Rows {
@@ -456,12 +496,7 @@ func (s *ExportService) buildPricingExportBlock(cfg *models.Configuration, opts
CompetitorWorld: priceMap[lot].CompetitorWorld,
})
}
if opts.isDDP() {
applyDDPMarkup(block.Rows, opts.saleMarkupFactor())
}
if opts.ManualPrice != nil && *opts.ManualPrice > 0 {
distributeManualPrice(block.Rows, *opts.ManualPrice)
}
applySalePricing(block.Rows, cfg, opts)
return block, nil
}
@@ -493,16 +528,29 @@ func (s *ExportService) buildPricingExportBlock(cfg *models.Configuration, opts
})
}
if opts.isDDP() {
applyDDPMarkup(block.Rows, opts.saleMarkupFactor())
}
if opts.ManualPrice != nil && *opts.ManualPrice > 0 {
distributeManualPrice(block.Rows, *opts.ManualPrice)
}
applySalePricing(block.Rows, cfg, opts)
return block, nil
}
// applySalePricing applies the DDP estimate uplift/manual-price/markup pipeline to a
// pricing export block. MinimalSaleColumns (only ever set by the single-config Sale
// export button) switches to applySaleRowPricing, which additionally folds in each
// row's own "Накидка" percent; every other caller (Buy export, project bulk export)
// keeps the original applyDDPMarkup + distributeManualPrice behavior unchanged.
func applySalePricing(rows []ProjectPricingExportRow, cfg *models.Configuration, opts ProjectPricingExportOptions) {
if opts.MinimalSaleColumns {
applySaleRowPricing(rows, opts.effectiveSaleMarkupFactor(cfg), opts.ManualPrice, opts.SaleRowMarkups)
return
}
if opts.isDDP() {
applyDDPMarkup(rows, opts.effectiveSaleMarkupFactor(cfg))
}
if opts.ManualPrice != nil && *opts.ManualPrice > 0 {
distributeManualPrice(rows, *opts.ManualPrice)
}
}
// sortConfigItemsByCategoryMap returns a copy of items sorted by category display order.
// categories maps lot_name → category code; catOrder maps category code → display order.
func sortConfigItemsByCategoryMap(items models.ConfigItems, catOrder map[string]int, categories map[string]string) models.ConfigItems {
@@ -521,7 +569,8 @@ func sortConfigItemsByCategoryMap(items models.ConfigItems, catOrder map[string]
// stockCompetitorMarkupFactor is the fixed DDP multiplier applied to Stock and
// Competitor columns, independent of the user-configurable estimate uplift.
// Mirrors PricingMarkup.STOCK_COMPETITOR_FIXED in web/templates/index.html.
// Only used by the non-minimal (full-column) DDP export path — the Sale table itself
// no longer displays/exports Stock/Competitor, see applySaleRowPricing.
const stockCompetitorMarkupFactor = 1.3
func applyDDPMarkup(rows []ProjectPricingExportRow, estimateFactor float64) {
@@ -532,6 +581,58 @@ func applyDDPMarkup(rows []ProjectPricingExportRow, estimateFactor float64) {
}
}
// saleRowKey mirrors _saleRowKey in web/templates/index.html: a LOT can appear more
// than once across different vendor-PN BOM groups, so lot_name alone isn't a safe key.
// "—" is the row-builder's own placeholder for "no vendor PN" (see the VendorPN: "—"
// literals in buildPricingExportBlock), matching the frontend's empty-string case.
func saleRowKey(vendorPN, lot string) string {
v := strings.ToUpper(strings.TrimSpace(vendorPN))
if v == "" || v == "—" {
v = "NONE"
}
return v + "::" + strings.ToUpper(strings.TrimSpace(lot))
}
// applySaleRowPricing implements the per-row "Накидка" pricing pipeline for the
// minimal Sale CSV export: each row's base (its raw Estimate, or a proportional share
// of manualPrice when set — same last-row-absorbs-remainder distribution as
// distributeManualPrice, but overwriting the base instead of a side field) is
// multiplied by (saleUplift + that row's own markup percent / 100). Mirrors
// _saleRowPrice/recomputeSalePrices in web/templates/index.html exactly.
func applySaleRowPricing(rows []ProjectPricingExportRow, saleUplift float64, manualPrice *float64, rowMarkups map[string]float64) {
rawTotal := 0.0
lastPricedIdx := -1
for i, row := range rows {
if row.Estimate != nil && *row.Estimate > 0 {
rawTotal += *row.Estimate
lastPricedIdx = i
}
}
useManual := manualPrice != nil && *manualPrice > 0 && rawTotal > 0
assigned := 0.0
for i := range rows {
if rows[i].Estimate == nil || *rows[i].Estimate <= 0 {
continue
}
base := *rows[i].Estimate
if useManual {
if i == lastPricedIdx {
base = math.Round((*manualPrice-assigned)*100) / 100
} else {
share := (*rows[i].Estimate / rawTotal) * (*manualPrice)
base = math.Round(share*100) / 100
assigned += base
}
}
key := saleRowKey(rows[i].VendorPN, rows[i].LotDisplay)
// Negative markup is a discount; floored at 0 like _saleRowPrice so a price never goes negative.
factor := math.Max(0, saleUplift+(rowMarkups[key]/100))
price := math.Round(base*factor*100) / 100
rows[i].Estimate = floatPtr(price)
}
}
func scaleFloatPtr(v *float64, factor float64) *float64 {
if v == nil {
return nil
@@ -832,6 +933,9 @@ func estimateOnlyTotal(estimatePrice *float64, fallbackUnitPrice float64, quanti
}
func pricingCSVHeaders(opts ProjectPricingExportOptions) []string {
if opts.MinimalSaleColumns {
return []string{"LOT", "Описание", "Кол-во", "Цена"}
}
headers := make([]string, 0, 9)
headers = append(headers, "Line Item")
if opts.IncludeLOT {
@@ -858,6 +962,14 @@ func pricingCSVHeaders(opts ProjectPricingExportOptions) []string {
}
func pricingCSVRow(row ProjectPricingExportRow, opts ProjectPricingExportOptions) []string {
if opts.MinimalSaleColumns {
return []string{
emptyDash(row.LotDisplay),
emptyDash(row.Description),
fmt.Sprintf("%d", exportPositiveInt(row.Quantity, 1)),
formatMoneyValue(row.Estimate),
}
}
record := make([]string, 0, 9)
record = append(record, "")
if opts.IncludeLOT {
@@ -931,7 +1043,6 @@ func pricingConfigSummaryRow(cfg ProjectPricingExportConfig, opts ProjectPricing
return record
}
func formatMoneyValue(value *float64) string {
if value == nil {
return "—"
+19
View File
@@ -561,3 +561,22 @@ type failingWriter struct{}
func (fw *failingWriter) Write(p []byte) (int, error) {
return 0, io.EOF
}
func TestApplySaleRowPricingNegativeMarkup(t *testing.T) {
rows := []ProjectPricingExportRow{
{LotDisplay: "GPU_A", Estimate: floatPtr(1000)},
{LotDisplay: "GPU_B", Estimate: floatPtr(1000)},
{LotDisplay: "GPU_C", Estimate: floatPtr(1000)},
}
applySaleRowPricing(rows, 1.5, nil, map[string]float64{
"NONE::GPU_A": -50, // discount: 1.5 - 0.5 = 1.0
"NONE::GPU_B": 25, // 1.5 + 0.25 = 1.75
"NONE::GPU_C": -300, // would be -1.5 — floored at 0
})
want := []float64{1000, 1750, 0}
for i, w := range want {
if rows[i].Estimate == nil || *rows[i].Estimate != w {
t.Fatalf("row %d: got %v, want %v", i, rows[i].Estimate, w)
}
}
}
+56 -12
View File
@@ -10,6 +10,7 @@ import (
"git.mchus.pro/mchus/quoteforge/internal/appmeta"
"git.mchus.pro/mchus/quoteforge/internal/article"
"git.mchus.pro/mchus/quoteforge/internal/configfeatures"
"git.mchus.pro/mchus/quoteforge/internal/localdb"
"git.mchus.pro/mchus/quoteforge/internal/models"
"git.mchus.pro/mchus/quoteforge/internal/services/sync"
@@ -70,13 +71,18 @@ func (s *LocalConfigurationService) Create(ownerUsername string, req *CreateConf
if strings.TrimSpace(req.ServerModel) != "" {
articleResult, articleErr := article.Build(s.localDB, req.Items, article.BuildOptions{
ServerModel: req.ServerModel,
ServerPricelist: pricelistID,
ServerModel: req.ServerModel,
})
if articleErr != nil {
return nil, articleErr
}
req.Article = articleResult.Article
if len(articleResult.Warnings) > 0 {
slog.Warn("article generation degraded",
"server_model", req.ServerModel,
"article", articleResult.Article,
"warnings", articleResult.Warnings)
}
}
total := req.Items.Total()
@@ -166,13 +172,18 @@ func (s *LocalConfigurationService) Update(uuid string, ownerUsername string, re
if strings.TrimSpace(req.ServerModel) != "" {
articleResult, articleErr := article.Build(s.localDB, req.Items, article.BuildOptions{
ServerModel: req.ServerModel,
ServerPricelist: pricelistID,
ServerModel: req.ServerModel,
})
if articleErr != nil {
return nil, articleErr
}
req.Article = articleResult.Article
if len(articleResult.Warnings) > 0 {
slog.Warn("article generation degraded",
"server_model", req.ServerModel,
"article", articleResult.Article,
"warnings", articleResult.Warnings)
}
}
total := req.Items.Total()
@@ -216,13 +227,10 @@ func (s *LocalConfigurationService) Update(uuid string, ownerUsername string, re
// BuildArticlePreview generates server article based on current items and server_model/support_code.
func (s *LocalConfigurationService) BuildArticlePreview(req *ArticlePreviewRequest) (article.BuildResult, error) {
pricelistID, err := s.resolvePricelistID(req.PricelistID)
if err != nil {
return article.BuildResult{}, err
}
// Categories for the article come from the component universe (world ∪ estimate),
// not from a specific pricelist, so req.PricelistID is no longer needed here.
return article.Build(s.localDB, req.Items, article.BuildOptions{
ServerModel: req.ServerModel,
ServerPricelist: pricelistID,
ServerModel: req.ServerModel,
})
}
@@ -317,6 +325,7 @@ func (s *LocalConfigurationService) CloneToProject(configUUID string, ownerUsern
TotalPrice: &total,
CustomPrice: original.CustomPrice,
Notes: original.Notes,
FeatureFlags: original.FeatureFlags,
IsTemplate: false,
ServerCount: original.ServerCount,
ServerModel: original.ServerModel,
@@ -325,6 +334,7 @@ func (s *LocalConfigurationService) CloneToProject(configUUID string, ownerUsern
PricelistID: original.PricelistID,
WarehousePricelistID: original.WarehousePricelistID,
CompetitorPricelistID: original.CompetitorPricelistID,
ConfigType: original.ConfigType,
DisablePriceRefresh: original.DisablePriceRefresh,
CreatedAt: time.Now(),
}
@@ -527,13 +537,18 @@ func (s *LocalConfigurationService) UpdateNoAuth(uuid string, req *CreateConfigR
if strings.TrimSpace(req.ServerModel) != "" {
articleResult, articleErr := article.Build(s.localDB, req.Items, article.BuildOptions{
ServerModel: req.ServerModel,
ServerPricelist: pricelistID,
ServerModel: req.ServerModel,
})
if articleErr != nil {
return nil, articleErr
}
req.Article = articleResult.Article
if len(articleResult.Warnings) > 0 {
slog.Warn("article generation degraded",
"server_model", req.ServerModel,
"article", articleResult.Article,
"warnings", articleResult.Warnings)
}
}
total := req.Items.Total()
@@ -646,6 +661,7 @@ func (s *LocalConfigurationService) CloneNoAuthToProjectFromVersion(configUUID s
return nil, fmt.Errorf("decode version snapshot for clone: %w", decErr)
}
snapshotCfg := localdb.LocalToConfiguration(snapshot)
snapshotCfg.ConfigType = original.ConfigType // not part of version snapshots
original = snapshotCfg
original.UUID = configUUID // preserve original UUID for project resolution
}
@@ -672,6 +688,7 @@ func (s *LocalConfigurationService) CloneNoAuthToProjectFromVersion(configUUID s
TotalPrice: &total,
CustomPrice: original.CustomPrice,
Notes: original.Notes,
FeatureFlags: original.FeatureFlags,
IsTemplate: false,
ServerCount: original.ServerCount,
ServerModel: original.ServerModel,
@@ -680,6 +697,7 @@ func (s *LocalConfigurationService) CloneNoAuthToProjectFromVersion(configUUID s
PricelistID: original.PricelistID,
WarehousePricelistID: original.WarehousePricelistID,
CompetitorPricelistID: original.CompetitorPricelistID,
ConfigType: original.ConfigType,
DisablePriceRefresh: original.DisablePriceRefresh,
CreatedAt: time.Now(),
}
@@ -916,6 +934,9 @@ func (s *LocalConfigurationService) UpdateServerCount(configUUID string, serverC
return nil, ErrConfigNotFound
}
if err := applyFeatureFlags(localCfg.FeatureFlags, localCfg); err != nil {
return nil, err
}
localCfg.ServerCount = serverCount
total := localCfg.Items.Total()
if serverCount > 1 {
@@ -1160,6 +1181,9 @@ func (s *LocalConfigurationService) createWithVersion(localCfg *localdb.LocalCon
}
func (s *LocalConfigurationService) createWithVersionTx(tx *gorm.DB, localCfg *localdb.LocalConfiguration, createdBy string) error {
if err := applyFeatureFlags(localCfg.FeatureFlags, localCfg); err != nil {
return err
}
if localCfg.IsActive {
if err := s.ensureConfigurationLineTx(tx, localCfg); err != nil {
return err
@@ -1205,6 +1229,9 @@ func (s *LocalConfigurationService) saveWithVersionAndPending(localCfg *localdb.
}
return fmt.Errorf("lock configuration row: %w", err)
}
if err := applyFeatureFlags(locked.FeatureFlags, localCfg); err != nil {
return err
}
if operation == "update" {
currentVersion, err := s.loadCurrentVersionTx(tx, &locked)
@@ -1285,6 +1312,7 @@ func hasNonRevisionConfigurationChanges(current *localdb.LocalConfiguration, nex
}
if current.Name != next.Name ||
current.Notes != next.Notes ||
strings.Join(current.FeatureFlags, ",") != strings.Join(next.FeatureFlags, ",") ||
current.IsTemplate != next.IsTemplate ||
current.ServerModel != next.ServerModel ||
current.SupportCode != next.SupportCode ||
@@ -1628,12 +1656,17 @@ func (s *LocalConfigurationService) rollbackToVersion(configurationUUID string,
return fmt.Errorf("decode target rollback snapshot: %w", err)
}
if err := configfeatures.Check(append(append([]string{}, current.FeatureFlags...), rollbackData.FeatureFlags...)); err != nil {
return err
}
// Keep stable identity/sync linkage; restore editable config content from target snapshot.
current.Name = rollbackData.Name
current.Items = rollbackData.Items
current.TotalPrice = rollbackData.TotalPrice
current.CustomPrice = rollbackData.CustomPrice
current.Notes = rollbackData.Notes
current.FeatureFlags = configfeatures.Detect(rollbackData.Notes)
current.IsTemplate = rollbackData.IsTemplate
current.ServerCount = rollbackData.ServerCount
current.ServerModel = rollbackData.ServerModel
@@ -1847,3 +1880,14 @@ func (s *LocalConfigurationService) resolvePricelistID(pricelistID *uint) (*uint
}
return &latest.ServerID, nil
}
// applyFeatureFlags refuses a write when the configuration as stored carries feature
// flags this build doesn't know (it could not round-trip the data behind them), and
// otherwise re-derives next's flags from its content.
func applyFeatureFlags(stored models.FeatureFlags, next *localdb.LocalConfiguration) error {
if err := configfeatures.Check(stored); err != nil {
return err
}
next.FeatureFlags = configfeatures.Detect(next.Notes)
return nil
}
@@ -0,0 +1,51 @@
package services
import (
"testing"
"git.mchus.pro/mchus/quoteforge/internal/models"
)
// config_type is free-form (codes come from qt_settings.config_types), so an
// unknown code like "networking" must survive every write path unchanged.
func TestConfigTypeSurvivesUpdateAndClone(t *testing.T) {
service, local := newLocalConfigServiceForTest(t)
created, err := service.Create("tester", &CreateConfigRequest{
Name: "net",
Items: models.ConfigItems{{LotName: "CPU_A", Quantity: 1, UnitPrice: 1000}},
ConfigType: "networking",
})
if err != nil {
t.Fatalf("create: %v", err)
}
// The configurator's save payload carries no config_type.
if _, err := service.UpdateNoAuth(created.UUID, &CreateConfigRequest{
Name: "net",
Items: models.ConfigItems{{LotName: "CPU_A", Quantity: 2, UnitPrice: 1000}},
}); err != nil {
t.Fatalf("update: %v", err)
}
if stored, _ := local.GetConfigurationByUUID(created.UUID); stored.ConfigType != "networking" {
t.Fatalf("config_type after update = %q, want networking", stored.ConfigType)
}
for _, fromVersion := range []int{0, 1} {
clone, err := service.CloneNoAuthToProjectFromVersion(created.UUID, "copy", "tester", nil, fromVersion)
if err != nil {
t.Fatalf("clone from version %d: %v", fromVersion, err)
}
if stored, _ := local.GetConfigurationByUUID(clone.UUID); stored.ConfigType != "networking" {
t.Fatalf("clone from version %d: config_type = %q, want networking", fromVersion, stored.ConfigType)
}
}
clone, err := service.CloneToProject(created.UUID, "tester", "copy2", nil)
if err != nil {
t.Fatalf("clone to project: %v", err)
}
if stored, _ := local.GetConfigurationByUUID(clone.UUID); stored.ConfigType != "networking" {
t.Fatalf("CloneToProject: config_type = %q, want networking", stored.ConfigType)
}
}
Loaded 100 of 143 files, more files were not shown because too many files have changed in this diff. Show more