PriceForge/bible-local/history.md

Change History

Architectural decisions and significant refactoring are recorded here. Every architectural decision MUST be documented in this file.

---

2026-03-07: Embedded Scheduler with MariaDB Advisory Locks

Decision

Moved periodic cron-style maintenance from an external cmd/cron runtime requirement into the main pfs application. Every app instance may run the scheduler loop, but each job is serialized across the shared MariaDB using advisory locks.

What changed

• Added embedded scheduler in internal/scheduler/scheduler.go.
• Added persisted scheduler state table qt_scheduler_runs via migration 030_add_scheduler_runs.sql.
• pfs now starts the scheduler on boot when scheduler.enabled = true.
• Added GET /api/admin/pricing/scheduler-runs and scheduler status table on the Settings page (/setup).
• Consolidated DB settings into /setup; removed the obsolete connection-settings modal flow and duplicate /api/connection-settings* routes.
• Added config section scheduler with per-job intervals:
  • alerts_interval
  • update_prices_interval
  • update_popularity_interval
  • reset_weekly_counters_interval
  • reset_monthly_counters_interval
• Coordination between multiple app instances uses MariaDB GET_LOCK/RELEASE_LOCK with one lock per job.
• cmd/cron remains available as a manual one-shot utility, but is no longer required for normal operation.

Rationale

PriceForge is deployed as local applications without a dedicated always-on server process that could own cron scheduling centrally. Embedding the scheduler into each app keeps the system self-contained while DB advisory locks prevent simultaneous job execution against the same database.

Constraints

• Scheduler safety depends on MariaDB advisory locks; without DB connectivity, jobs do not run.
• Jobs are interval-based from last_finished_at stored in qt_scheduler_runs.
• Duplicate scheduler loops across multiple app instances are acceptable because only one instance acquires the per-job lock at a time.

Files

• Scheduler: internal/scheduler/scheduler.go
• Scheduler tests: internal/scheduler/scheduler_test.go
• Config: internal/config/config.go, config.example.yaml
• App startup: cmd/pfs/main.go
• Migration: migrations/030_add_scheduler_runs.sql

2026-03-07: Remove Unused BOM Storage

Decision

Removed qt_bom and qt_configurations.bom_id because the schema was never adopted by runtime code and created dead database surface area.

What changed

• Added migration 029_drop_unused_qt_bom.sql to drop:
  • foreign key fk_qt_configurations_bom
  • index idx_qt_configurations_bom_id if present
  • column qt_configurations.bom_id
  • table qt_bom
• Removed BOM model from internal/models/lot.go.
• Removed Configuration.BOMID from internal/models/configuration.go.
• Removed BOM from internal/models/models.go auto-migration registry.

Rationale

The repository had schema and model stubs for BOM persistence, but no handler, service, repository, sync path, or UI used them. Keeping unused schema increases maintenance cost and confuses future changes.

Constraints

• This removal is safe only because no runtime code reads or writes qt_bom or qt_configurations.bom_id.
• Historical BOM payloads in qt_bom are not migrated elsewhere; the table is treated as disposable unused state.

Files

• Migration: migrations/029_drop_unused_qt_bom.sql
• Models: internal/models/lot.go, internal/models/configuration.go, internal/models/models.go

2026-03-07: Purge LOT Names Polluting Seen Registry

Decision

Rows in qt_vendor_partnumber_seen where partnumber is actually equal to lot.lot_name are treated as polluted data from external systems and must not appear in Global Vendor Mappings UI.

What changed

• Added purgeSeenLotNames in internal/services/seen_cleanup.go.
• VendorMappingService.List now deletes polluted seen rows before building the Vendor Mappings list.
• GetUnmappedPartnumbers now explicitly excludes qt_vendor_partnumber_seen.partnumber values that match an existing lot.lot_name.
• Added regression test coverage in internal/services/vendor_mapping_test.go.

Rationale

Another application writes non-partnumber LOT identifiers into qt_vendor_partnumber_seen. Those rows are noise for operators and should not be shown as unmapped vendor partnumbers.

Constraints

• Only polluted seen rows are removed: if a value equal to lot.lot_name also has an explicit mapping in lot_partnumbers, it is preserved.
• The cleanup targets the seen registry only; canonical PN mappings in lot_partnumbers are not touched.

Files

• Service: internal/services/seen_cleanup.go
• Service: internal/services/vendor_mapping.go
• Handler: internal/handlers/pricing.go
• Tests: internal/services/vendor_mapping_test.go
• Docs: bible-local/vendor-mapping.md

2026-02-21: Partnumber Book Snapshots for QuoteForge

Decision

Implemented versioned snapshots of the lot_partnumbers → LOT mapping in qt_partnumber_books / qt_partnumber_book_items. PriceForge writes; QuoteForge reads (SELECT only).

What changed

• Migration 026: added is_primary_pn TINYINT(1) DEFAULT 1 to lot_partnumbers; created qt_partnumber_books and qt_partnumber_book_items tables (version VARCHAR(20), later corrected).
• Migration 027: corrected version VARCHAR(20) → VARCHAR(30) — PNBOOK-YYYY-MM-DD-NNN is 21 chars and overflowed the original column.
• Migration 028: added description VARCHAR(10000) NULL to qt_partnumber_book_items — required by QuoteForge sync (SELECT partnumber, lot_name, is_primary_pn, description).
• Models PartnumberBook, PartnumberBookItem (with Description *string) added to internal/models/lot.go; IsPrimaryPN bool added to LotPartnumber.
• Service internal/services/partnumber_book.go:
  • CreateSnapshot: expands bundles (QuoteForge is bundle-unaware), copies description from lot_partnumbers to every expanded row, generates version PNBOOK-YYYY-MM-DD-NNN, deactivates previous books and activates new one atomically, then runs GFS retention cleanup.
  • expandMappings: filters out rows where lot_name is empty/whitespace; filters out partnumbers marked is_ignored = true in qt_vendor_partnumber_seen. Only valid PN→LOT pairs enter the snapshot.
  • applyRetention: deletes items explicitly (DELETE … WHERE book_id IN (…)) before deleting books — does not rely on FK CASCADE which GORM does not trigger on batch deletes.
  • ListBooks: returns all snapshots ordered newest-first with item counts.
• GFS retention policy: 7 daily · 5 weekly · 12 monthly · 10 yearly; applied automatically after each snapshot; active book is never deleted.
• Task type TaskTypePartnumberBookCreate added to internal/tasks/task.go.
• Handlers ListPartnumberBooks and CreatePartnumberBook added to internal/handlers/pricing.go; PartnumberBookService injected via constructor.
• Routes GET /api/admin/pricing/partnumber-books and POST /api/admin/pricing/partnumber-books registered in cmd/pfs/main.go.
• UI: "Снимки сопоставлений (Partnumber Books)" section with snapshot table, progress bar, and "Создать снапшот сопоставлений" button added to web/templates/vendor_mappings.html.

Rationale

QuoteForge needs a stable, versioned copy of the PN→LOT mapping to resolve BOM line items without live dependency on PriceForge. Snapshots decouple the two systems.

Constraints

• Bundles MUST be expanded: QuoteForge does not know about qt_lot_bundles.
• Snapshot rows with empty lot_name or is_ignored = true partnumbers MUST be excluded.
• description in book items comes from lot_partnumbers.description; for expanded bundle rows the description of the parent partnumber mapping is used.
• is_primary_pn is copied from lot_partnumbers into each book item; drives qty logic in QuoteForge.
• New snapshot is immediately is_active=1; all previous books are deactivated in the same transaction.
• Version format: PNBOOK-YYYY-MM-DD-NNN (VARCHAR(30)), sequential within the same day.
• Item deletion during retention MUST be explicit (items first, then books) — FK CASCADE is unreliable with GORM batch deletes.
• QuoteForge has SELECT permission only on qt_partnumber_books and qt_partnumber_book_items.

Files

• Migrations: 026_add_partnumber_books.sql, 027_fix_partnumber_books_version_length.sql, 028_add_description_to_partnumber_book_items.sql
• Models: internal/models/lot.go
• Service: internal/services/partnumber_book.go
• Handler: internal/handlers/pricing.go
• Tasks: internal/tasks/task.go
• Routes: cmd/pfs/main.go
• UI: web/templates/vendor_mappings.html

---

2026-02-20: Pricelist Formation Hardening (Estimate/Warehouse/Meta)

Decision

Hardened pricelist formation rules to remove ambiguity and prevent silent data loss in UI/API.

What changed

• estimate snapshot now explicitly excludes hidden components (qt_lot_metadata.is_hidden = 1).
• Category snapshot in qt_pricelist_items.lot_category now always has a deterministic fallback: PART_ is assigned even when a LOT row is missing in lot.
• PricelistItem JSON contract was normalized to a single category field (LotCategory -> json:"category"), removing duplicate JSON-tag ambiguity.
• Meta-price source expansion now always includes the base LOT, then merges meta_prices sources with deduplication (exact + wildcard overlaps).

Rationale

• Prevent hidden/ignored components from leaking into estimate pricelists.
• Keep frontend category rendering stable for all items.
• Avoid non-deterministic JSON serialization when duplicate tags are present.
• Ensure meta-article pricing includes self-history and does not overcount duplicate sources.

Constraints

• estimate pricelist invariant: current_price > 0 AND is_hidden = 0.
• category in API must map from persisted qt_pricelist_items.lot_category.
• Missing category source must default to PART_.
• Meta source list must contain each LOT at most once.

Files

• Model: internal/models/pricelist.go
• Estimate/Warehouse snapshot service: internal/services/pricelist/service.go
• Pricing handler/meta expansion: internal/handlers/pricing.go
• Pricing service/meta expansion: internal/services/pricing/service.go
• Tests:
  • internal/services/pricelist/service_estimate_test.go
  • internal/services/pricelist/service_warehouse_test.go
  • internal/handlers/pricing_meta_expand_test.go
  • internal/services/pricing/service_meta_test.go
  • internal/services/stock_import_test.go

---

2026-02-20: Seen Registry Deduplication by Partnumber

Decision

Changed qt_vendor_partnumber_seen semantics to one row per partnumber (vendor/source are no longer part of uniqueness).

Rationale

• Eliminates duplicate seen rows when the same partnumber appears both with vendor and without vendor.
• Keeps ignore behavior consistent regardless of vendor presence.
• Simplifies operational cleanup and prevents re-creation of vendor/no-vendor duplicates.

Constraints

• partnumber is now the unique key in seen registry.
• Ignore checks are resolved by partnumber only.
• Stock provenance must be preserved (source_type='stock') when stock data exists for the partnumber.

Files

• Migration: migrations/025_dedup_vendor_seen_by_partnumber.sql
• Service: internal/services/stock_import.go
• Service: internal/services/vendor_mapping.go
• Model: internal/models/lot.go

---

2026-02-18: Global Vendor Partnumber Mapping

Decision

Implemented global vendor partnumber → LOT mapping with bundle support and seen-based ignore logic.

Key rules

• lot_partnumbers is the canonical mapping contract (1:1 per key).
• Composite mappings use internal bundle tables (qt_lot_bundles, qt_lot_bundle_items).
• Ignore logic moved from stock_ignore_rules to qt_vendor_partnumber_seen.is_ignored.
• Resolver order: exact (vendor, partnumber) → fallback (vendor='', partnumber).

Rationale

• Preserves external/client contracts (lot_partnumbers, LOT-based pricelists).
• Avoids multi-row ambiguity in lot_partnumbers.
• Supports complex assembled vendor SKUs without client-side changes.
• Centralizes ignore behavior across all sources via seen-registry.

Update (2026-02-25)

• DELETE /api/admin/pricing/vendor-mappings now removes both:
• mapping rows from lot_partnumbers
• and matching entries from qt_vendor_partnumber_seen (so "seen/unmapped" rows disappear from the global UI immediately after delete).

Constraints

• Bundle LOT is internal and must stay hidden in regular LOT list by default.
• Resolver order is mandatory and fixed.
• Bundle allocation for missing estimate: fallback from previous active warehouse pricelist; if absent → 0.

Files

• Migration: migrations/023_vendor_partnumber_global_mapping.sql
• Backfill: migrations/024_backfill_vendor_seen_from_stock_and_ignore.sql
• Resolver: internal/lotmatch/matcher.go
• Service: internal/services/vendor_mapping.go
• Warehouse calc: internal/warehouse/snapshot.go
• Stock import: internal/services/stock_import.go
• API: internal/handlers/pricing.go, cmd/pfs/main.go

---

2026-02-10: LOT Page Refactoring

Decision

Moved LOT management to a dedicated /lot page. Removed LOT tab from Pricing Admin.

What changed

• Created web/templates/lot.html — two tabs: LOT (component management) and Mappings (partnumber ↔ LOT).
• Removed LOT tab from admin_pricing.html; default tab changed to estimate.
• Removed "Сопоставление partnumber → LOT" section from Warehouse tab.
• Updated navigation: LOT → /lot, Pricing Admin → /admin/pricing.
• Added Lot() handler in internal/handlers/web.go.
• Added /lot route in cmd/pfs/main.go.

Rationale

Separation of concerns: LOT/component management is distinct from pricing administration.

---

Warehouse Pricing: Weighted Average

Decision

Warehouse pricelist uses weighted_avg (quantity-weighted average) as the sole price calculation method. Commit edff712 switched from weighted_median to weighted_avg.

• price_method field always contains "weighted_avg".
• No price_period_days, price_coefficient, manual_price, meta_prices in warehouse pricelists.

---

Architecture Conventions

All future architectural decisions must be documented here with:

• Date
• Decision summary
• Rationale
• Constraints / invariants
• Affected files