Gogentic Finance · internal engineering record

Sheet 3 of 6

The Close Cycle

There is no month-end here. The ledger closes every five minutes: pull, reconcile against what is already stored, classify what is new. This sheet follows one full pass — and stops at the point where two files with the same import specifier quietly disagree about what happens next.

Sync interval
*/5
Classify interval
*/10
Page size
500
Batch size
50
Dedupe key
mercury_id
Retries
3
1

One pass of the cycle

A pass begins with a scheduled GET and ends with a JSON receipt. Between those two points the application resolves a bank key, upserts accounts, walks every transaction page, and writes what it has not seen before.

Exhibit 4
The incremental synchronisation pipeline as executed A scheduled GET to the cron sync endpoint is authorised against a bearer secret, then runs runIncrementalSync from src/server/sync.ts. The key is decrypted from settings or read from the environment, accounts are upserted, transactions are fetched in offset pages of five hundred, filtered by status, checked individually for existence, and upserted. The response returns. No categorisation is triggered. CONTROL PATH — src/server/sync.ts DETAIL GET /api/cron/sync Authorization: Bearer <CRON_SECRET> strict bearer comparison read from env at request time 401 { error: 'Unauthorized' } runIncrementalSync() runFullSync() simply calls this resolve the Mercury key settings.mercury_api_key_encrypted isEncrypted() ? decrypt() : legacy plaintext falls back to process.env.MERCURY_API_KEY; throws 'Mercury API key not configured' when neither is present GET /accounts → upsert onConflictDoUpdate on accounts.mercury_id balance × 100 into cents then re-select to build a mercury_id → uuid map page the transactions limit 500, offset += 500 start is always 2024-01-01 stops on a short page, or when offset reaches the 50,000 hard cap next page status filter keep only 'sent' and 'received' per row: SELECT id, then upsert amount = round(abs × 100) direction = amount ≥ 0 ? inflow : outflow one existence query per transaction — an N+1 over the whole page on conflict it refreshes amount, description, counterparty and synced_at — but never category_id, date or direction 200 { success, accountsSynced, transactionsSynced, newTransactions, timestamp } Nothing here calls the classifier. See section 2.

← Scroll to see the full diagram →

One incremental pass, as the deployed code runs it. Two properties are worth noting. The window start is the hard-coded string 2024-01-01 on every run, so “incremental” describes the write behaviour rather than the read: each pass re-reads the entire history and relies on the upsert to make that cheap. And runFullSync() is a stub whose whole body calls runIncrementalSync(), with a comment acknowledging it.
2

Two files, one specifier

The repository contains both src/server/sync.ts and src/server/sync/index.ts. Both export runIncrementalSync, runFullSync and getSyncStatus. Only one of them ever runs.

Under the module resolution this project uses — tsconfig.json sets "moduleResolution": "bundler" — the bare specifier @/server/sync resolves to the file sync.ts in preference to the directory's index.ts. Every production import uses the bare specifier. Every test that exercises the categorisation trigger uses the deep path @/server/sync/index.

Exhibit 5
Module resolution collision between sync.ts and sync/index.ts Three production importers use the bare specifier at-slash-server-slash-sync, which resolves to the file sync.ts. Two test files use the deep specifiers at-slash-server-slash-sync-slash-index and slash-transactions, which reach the directory. Only the directory version triggers categorisation after a sync, so that behaviour is tested but never executed. IMPORTERS SPECIFIER RESOLVES TO PRODUCTION app/api/cron/sync/route.ts app/api/sync/route.ts server/actions/settings.ts all three use the bare specifier TESTS __tests__/server/sync/index __tests__/server/sync/transactions both use explicit deep paths '@/server/sync' '@/server/sync/index' '@/server/sync/transactions' src/server/sync.ts LIVE 296 lines · inline fetch, no client upserts existing rows balance converted to cents runFullSync is a stub no reference to lib/ai/categorize src/server/sync/index.ts DEAD delegates to accounts.ts + transactions.ts uses the getMercuryClient() abstraction skips existing rows rather than updating balance stored without × 100 fires categorisation when created > 0 getSyncStatus returns a different shape CONSEQUENCE tested behaviour 10 assertions in __tests__/server/sync/index prove sync triggers categorisation, async executed behaviour the live module never calls the classifier; it runs only from the */10 cron or the action The tests pass. They are exercising a module that no production import path can reach.

← Scroll to see the full diagram →

A shadowed module. The evidence is not merely that resolution prefers the file: it is visible in the field names each caller uses. src/server/actions/settings.ts reads result.transactionsSynced and result.newTransactions, which exist only on sync.ts's result type; and /api/sync serves the getSyncStatus() shape that sync.ts returns, not the { lastSyncTime, totalTransactions, uncategorizedCount, accountBalances } shape the directory version returns.
Note — what this costs

Nothing is broken in a way a user would see. Transactions still sync, and they are still classified — by the independent */10 categorisation cron, which was added in TASKS.md Phase 8 as a “backup categorization pass”. The practical effect is that the backup became the only pass, and the worst-case delay between a transaction arriving and being categorised is ten minutes rather than the few seconds the sync-triggered path was designed to give. The more interesting cost is to assurance: ten tests certify a behaviour that never executes.

3

The Mercury client

src/lib/mercury/ holds a typed client: a class, a singleton accessor, a reset function for tests, and a structured error. It is a careful piece of work — and the live sync path does not use it. src/server/sync.ts calls fetch directly against the same base URL. The client is reached only through the shadowed sync/accounts.ts and sync/transactions.ts.

new MercuryClient(key, opts?)
Throws 'Mercury API key is required' on an empty or whitespace-only key. Options accept maxRetries, retryBaseDelay and timeout.
getBaseUrl()
Returns https://api.mercury.com/api/v1, exposed so a test can assert it without reaching into a private field.
getAccounts() / getAccount(id)
/accounts and /account/{id}. The single-account call rejects with 'Account ID is required' for a blank id, before any network call.
getTransactions(id, opts?)
/account/{id}/transactions. Query parameters are start, end, limit, offset and status.
getAllTransactionsForAccount(id, since?)
Pages at pageSize = 500, incrementing offset, until a short page arrives or the accumulated count reaches the reported total.
getMercuryClient()
Lazy singleton over MERCURY_API_KEY; throws 'MERCURY_API_KEY environment variable is not set' when unset.
Note — the pagination is offset, not cursor

PRD.md specifies “Cursor-based pagination for all transactions”, and src/lib/mercury/types.ts still carries a cursor field on the params interface and a MercuryPaginationCursor type to match. Neither is referenced anywhere. Both the client and the live sync walk a numeric offset. The cursor types are dead surface left behind by the specification.

Transport settings

DEFAULT_CONFIG and the retry policy
SettingValueBehaviour
maxRetries3Four attempts in total for a retryable failure
retryBaseDelay1,000 msDoubled per attempt: base × 2attempt
Jitter0–30%Math.random() * 0.3 * baseDelay added to each wait
Retry-AfterhonouredWhen present it overrides the computed backoff
timeout30,000 msEnforced with an AbortController; an abort becomes MercuryError('Request timeout', 408, 'TIMEOUT')
Concurrency1Accounts are walked serially

The synthesised 408 is worth noting: because isRetryable() covers only 429 and 5xx, a client-side timeout is not retried, while a server-side 503 is.

Error taxonomy

MercuryError predicates by HTTP status
Status isAuthError isRateLimited isRetryable
400falsefalsefalse
401truefalsefalse
403truefalsefalse
404falsefalsefalse
429falsetruetrue
500falsefalsetrue
503falsefalsetrue

Every cell is a separate assertion in __tests__/lib/mercury/client.test.ts. An authentication failure is explicitly not retryable, so a revoked key fails fast rather than consuming the retry budget.

Running the client suite

The Mercury tests are split. Twenty-seven unit tests always run; six integration tests make real API calls and are gated behind three conditions — a key must be present, native fetch must exist, and SKIP_MERCURY_INTEGRATION must not be set. When they are skipped the suite prints the reason rather than staying silent.

Terminal — unit run, no credentials present
$ npm test -- __tests__/lib/mercury/client.test.ts

> gogenticcfo@0.1.0 test
> jest __tests__/lib/mercury/client.test.ts

  console.log
    Integration tests: SKIPPED (MERCURY_API_KEY not set)

      at Object.<anonymous> (__tests__/lib/mercury/client.test.ts:352:7)

PASS __tests__/lib/mercury/client.test.ts
  MercuryClient
    constructor
       should create a client with a valid API key
       should throw an error when API key is empty
       should throw an error when API key is whitespace
       should accept custom configuration
    getBaseUrl
       should return the correct base URL
  MercuryError
    isRetryable
       should return true for 429 status
       should return false for 401 status
  MercuryClient Integration Tests
    ○ skipped 6 tests

Test Suites: 1 passed, 1 total
Tests:       6 skipped, 27 passed, 33 total

The console.log line is emitted by the suite itself — an “Integration Tests Status” block at the foot of the file exists purely to report why the integration describe was skipped. The totals reconcile: 33 it blocks in the file, six of them inside the conditionally skipped describe, and TASKS.md 2.2 independently records “27 tests”. The test script is plain jest, so npm test -- <path> forwards the path straight through.

4

Reconciliation & idempotence

Running the cycle twice must not double the ledger. The guarantee is structural — transactions.mercury_id carries a UNIQUE constraint — and the live path leans on it directly with onConflictDoUpdate.

Conflict target and update set src/server/sync.ts
// accounts
.onConflictDoUpdate({ target: accounts.mercuryId, /* name, type, balance, lastSyncedAt */ })

// transactions — note what is NOT in the update set
.onConflictDoUpdate({ target: transactions.mercuryId, set: {
    amount, description, counterpartyName, syncedAt
    // categoryId    — preserved, so a classification is never undone
    // date          — preserved
    // direction     — preserved
}})

Excluding categoryId from the update set is what makes a re-sync safe: a manual override, or a category the model assigned, survives every subsequent pass. It also means a correction to date or direction upstream will never propagate.

What a freshly synced row looks like

Post-insert state __tests__/server/sync/transactions.test.ts
tx.aiCategorized       === false
tx.categoryId          === null
tx.manuallyOverridden  === false

// signed-to-unsigned conversion, asserted on the same rows:
upstream  50000  →  amount 50000, direction 'inflow'
upstream -15000  →  amount 15000, direction 'outflow'
upstream      0  →  amount     0, direction 'inflow'

The zero case carries its own test, titled “should return ‘inflow’ for zero (edge case)”. A fee reversal of exactly zero therefore lands on the inflow side of every aggregate.

Two conventions for the same thing. The live sync.ts writes direction with an inline ternary, amount ≥ 0 ? 'inflow' : 'outflow'. The shadowed sync/transactions.ts exports a named calculateDirection() that does the same thing and has thirteen tests. Both agree on the zero case. Only the second is covered.

5

Vendor normalisation

Bank counterparty strings are noisy: the same vendor arrives with a run number, a reference, an order id or a processor prefix appended. Before anything is matched or learned, the string passes through normalizeVendorName in src/server/patterns/index.ts.

normalizeVendorName — asserted input/output pairs
Input Output Rule exercised
Case and whitespace
amazon web servicesAMAZON WEB SERVICESUppercase
  Stripe  STRIPETrim
\t AWS \nAWSTrim, including tabs and newlines
Google   Cloud   PlatformGOOGLE CLOUD PLATFORMCollapse internal runs of whitespace
Trailing reference stripping
GUSTO 121524GUSTOTrailing digits
AWS 1234567890AWSTrailing digits, any length
ORDER #12345ORDERTrailing # then digits
UBER *12345UBERTrailing * then digits
LYFT **5678LYFTRepeated * accepted
STRIPE TRANSFER 12345STRIPE TRANSFERMulti-word names keep every word
Boundaries
uber *trip #1234UBER *TRIPOnly the trailing reference goes; a mid-string * survives
McDonald's RestaurantMCDONALD'S RESTAURANTApostrophes are not punctuation to be stripped
AMAZON WEB SERVICESAMAZON WEB SERVICESIdempotent on already-clean input
'   '''Whitespace-only collapses to empty

Vendor strings shown here are the test fixtures' own examples — well-known software and transport brands used to illustrate a string-handling rule, not counterparties drawn from the ledger.

The rest of the pattern module

findPatternMatch(name)
Exact ilike first, then a substring scan. A hit increments match_count as a side effect — a read that writes.
createPattern(pattern, categoryId, source)
Insert or update a row in category_patterns. source defaults to 'manual_rule'.
applyPatternToTransactions(pattern, categoryId)
Bulk update: sets category_id and ai_categorized on every transaction whose counterparty matches, excluding rows already flagged manually_overridden. Returns the affected row count.

Matching is exact, then substring. A test asserts that the pattern AMAZON WEB SERVICES does not match the vendor Amazon — normalisation never shortens a name to its first word. But findPatternMatch matches in the other direction: a stored pattern appearing as a substring of a longer incoming name is a hit. The asymmetry lets one learned rule absorb an entire family of suffixed variants.

6

Classification

The classifier is built to call the model as rarely as possible. Learned patterns are consulted first, and only vendors that no rule covers are batched off to Claude. Every accepted answer becomes a rule, so the population needing inference shrinks with use.

Exhibit 6
Categorisation decision flow Uncategorised transactions yield vendor strings, preferring counterparty name and falling back to description. Each is normalised and checked against learned patterns. Matches are applied directly and skip the model. The remainder are batched fifty at a time to Claude Sonnet, whose JSON reply is parsed, mapped against the twelve category names with a fallback to Other, and written back to the transactions. SELECT ROUTE RESOLVE & WRITE BACK getUncategorizedVendors() WHERE ai_categorized = false AND category_id IS NULL AND manually_overridden = false vendor identity counterpartyName   || description || '' blank strings are dropped entirely normalizeVendorName(v) uppercase, trim, strip trailing refs findPatternMatch(vendor) exact ilike, then substring, against category_patterns.pattern on a hit: match_count += 1 hit apply programmatically counts toward skippedKnownPatterns; the model is never called miss batch, 50 vendors 150 unknown vendors → 3 requests categorizeVendors([]) returns [] without contacting the API at all claude-sonnet-4-5-20250929 max_tokens 4096; the prompt is sent as a user turn, not a system turn { categorizations: [   { vendor, category, confidence } ] } map name → category id a ```json fence is stripped first unrecognised names fall back to Other and increment result.errors applyCategoriesToTransactions LOWER(counterparty_name) = vendor OR (counterparty_name IS NULL    AND LOWER(description) = vendor) write back as a learned pattern RESULT OBJECT processed vendors examined categorized assigned a category skippedKnownPatterns resolved without the model errors count, not an array Spread into the /api/categorize response body alongside success and an ISO timestamp.

← Scroll to see the full diagram →

Two routes, one destination. The left branch out of findPatternMatch is free; the right branch costs a model call. Note the description fallback in the select stage: a wire transfer with no counterparty name is still classifiable, and the write-back SQL mirrors the same fallback so the match actually lands. Note also that an unrecognised category name both falls back to Other and increments the error count — the row is categorised, but the pass reports it as a failure.

The closed vocabulary

The model is not free to invent a category. The prompt lists twelve labels with parenthetical examples and demands a bare JSON object; the reply is then looked up case-insensitively against the twelve seeded names, and a miss resolves to Other. The vocabulary is fixed at the data layer, not merely requested in prose.

Who calls it

Entry points to categorizeUncategorizedTransactions()
CallerLive?Note
POST /api/categorizeyesThe */10 Vercel cron, and any manual trigger
runCategorizationAction()yesServer action in src/server/actions/categorize.ts; revalidates /, /transactions, /expenses
src/server/sync/index.tsnoFire-and-forget trigger after a successful sync — in a module nothing imports. See section 2
7

Schedule & authorisation

Two scheduled endpoints on interlocking intervals, declared in vercel.json, with deliberately different authorisation postures.

The whole file vercel.json
{
  "crons": [
    { "path": "/api/cron/sync",  "schedule": "*/5 * * * *"  },
    { "path": "/api/categorize", "schedule": "*/10 * * * *" }
  ]
}
Exhibit 7
Cron schedule and authorisation matrix A thirty-minute timeline showing the sync endpoint firing every five minutes and the categorisation endpoint every ten, coinciding at minutes zero, ten, twenty and thirty. Below, a matrix compares the authorisation behaviour of the two endpoints for a missing header, a wrong secret, a wrong scheme and a correct bearer token. THIRTY MINUTES OF SCHEDULED ACTIVITY */5 /api/cron/sync */10 /api/categorize 0 5 10 15 20 25 30 minutes past the hour Because sync no longer triggers classification, these two schedules are the only things that drive the pipeline. AUTHORISATION MATRIX REQUEST CARRIES /api/cron/sync /api/categorize no Authorization header 401 200 — runs Bearer <wrong secret> 401 401 Basic <correct secret> 401 not asserted Bearer <CRON_SECRET> 200 200

← Scroll to see the full diagram →

Schedules and gates. The asymmetric cell is the top right: an unauthenticated POST /api/categorize runs. The route's own comment explains it — “Allow either cron secret or no auth for manual triggers”. A wrong secret is still rejected, so the rule is “absent or correct”. Both paths are reachable from the open internet: middleware.ts lists /api/cron(.*) and /api/categorize(.*) as public, so the bearer check inside /api/cron/sync is the only thing protecting it.

The shared cron helpers

src/lib/cron.ts
ExportStatusBody
verifyCronAuth(request)Synchronous. Returns true only when the header equals `Bearer ${process.env.CRON_SECRET}` under a plain === — a byte-wise comparison, not a constant-time one
unauthorizedResponse()401{ error: 'Unauthorized' }
cronSuccessResponse(data)200{ success: true, ...data, timestamp } — the payload is spread onto the top level, not nested
cronErrorResponse(error)500{ error: 'Operation failed', message }; a non-Error throw yields 'Unknown error'

Both routes decline the generic error label. cronErrorResponse produces error: 'Operation failed', but the sync route returns 'Sync failed' and the categorise route 'Categorization failed'. Each builds its own 500 rather than delegating, so that branch of the shared helper is never taken.

Endpoint contracts

Scheduled endpoints, full contract
  /api/cron/sync /api/categorize
MethodGETPOST
Other verbsNo GET export; Next.js answers 405
Interval*/5 * * * **/10 * * * *
CallsrunIncrementalSync()categorizeUncategorizedTransactions()
Argumentsnonenone
200 bodysuccess, accountsSynced, transactionsSynced, newTransactions, timestampsuccess, processed, categorized, errors, skippedKnownPatterns, timestamp
401 body{ error: 'Unauthorized' }{ error: 'Unauthorized' }
500 body{ error: 'Sync failed', message }{ error: 'Categorization failed', message }

The timestamp field is asserted to round-trip: new Date(json.timestamp).toISOString() === json.timestamp. A third cron-adjacent route, GET /api/cron/health, is public and unauthenticated by design and returns { status: 'healthy', timestamp }.