- ConnectionManager reuses a stale-but-healthy pool instead of dialing a new one every 30s, and closes every pool it replaces (one client held ~30 connections). Pool capped at 2 connections, idle released after 1 min. - Background sync every 1 min with ±30% jitter, scheduled from cycle start; users-status online threshold = interval × 1.3. One heartbeat per cycle. - Partnumber books: read headers only, fetch partnumbers_json per book when new/incomplete; empty book counts as complete; an incomplete book is re-pulled once per process (server book lists BOT_AGIBOT_G2_PERF missing from qt_partnumber_book_items). - NeedSync in one UNION ALL query; SyncQtSettings skips unchanged rewrite. - Migration 035: server-maintained server_updated_at on qt_configurations and qt_projects. Clients pull only changed rows after one full pull per process; servers without 035 and old clients keep full pulls. - Migrator: fresh GORM statement per verify query and history insert; files with two verify checks failed to record. ADL: bible-local/decisions/2026-10-26-mariadb-single-pool.md, bible-local/decisions/2026-10-26-incremental-config-project-pull.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.3 KiB
3.3 KiB
Decision: one MariaDB pool per client — reuse, never orphan
Date: 2026-10-26 Status: active
Context
A single client held ~30 MariaDB connections during sync. ConnectionManager.GetDB
treated a pool older than pingInterval (30s) as unusable and dialed a brand-new
*gorm.DB without closing the previous one. Sync calls getDB() per step, so every
~30s a new pool appeared while the old one kept its idle connections until
ConnMaxLifetime (5 min): ~10 orphaned pools × 2 idle + up to 10 active ≈ 30. Each
orphaned *sql.DB also leaked its opener goroutine. TryConnect, the background
ping-failure path and a failed ping inside dial dropped pools the same way.
Decision
GetDBpings the existing pool when it is merely stale and returns it; it dials only when there is no pool or the ping fails.- Whenever the manager replaces or drops a pool (
GetDB,TryConnect,checkOnlineNow,MarkOffline,Disconnect, failed ping indial) the old*sql.DBis closed. - Pool limits:
MaxOpenConns=2,MaxIdleConns=2,ConnMaxLifetime=5m,ConnMaxIdleTime=1m— one client holds at most 2 connections, idle ones are released within a minute. Sync is sequential (one query at a time); the second connection lets UI server queries (active viewers, users status) run alongside a long sync step. - Background sync interval (
backgroundSyncIntervalincmd/qfs/main.go) is 1 minute with ±30% random jitter (sync.WorkerJitter), so clients do not hit MariaDB in lockstep. The next cycle is scheduled from the start of the previous one, so heartbeats (qt_client_schema_state.last_checked_at) are at most 1.3 × interval apart while a cycle is shorter than that. /api/sync/users-statusonline threshold = interval × (1 +WorkerJitter).PullPartnumberBooksreads only book headers (id, version, created_at, is_active) each cycle;partnumbers_json(~5 MB for all books) is fetched per book only when the book is new or its local items are incomplete. An empty book counts as complete. An incomplete book is re-pulled at most once per process per version: a server book can list partnumbers missing fromqt_partnumber_book_items(seen:BOT_AGIBOT_G2_PERFinPNBOOK-2026-10-21-001), which a re-pull cannot fix; this is logged asWARNwith the missing partnumbers.- A worker cycle sends one heartbeat: the worker calls
EnsureReadinessForSynconce and then the unexportedpushPendingChanges(the exportedPushPendingChangesstill checks readiness itself for the manual-sync handlers). NeedSyncasks for the latest active pricelist of all three sources in oneUNION ALLquery (GetLatestActiveIDsBySources): 0.71s → 0.24s.SyncQtSettingsskips the localDELETE+INSERTwhen the server rows equal the cache.- Measured against the production server: a cold sync took 171s (pricelists 105s, partnumber books 38s, configurations 23s) and is bound by server reads, not pool size. A warm cycle takes ~1.4s (was ~2.5s plus a 5 MB download).
Consequences
- Code that keeps a
*gorm.DBacross a reconnect getssql: database is closed; sync steps must callgetDB()per operation instead of caching the handle. - Tests:
internal/db/connection_test.go,internal/services/sync/worker_jitter_test.go.