Payments Engine
Architecture

Eleven contexts that cannot reach into each other

Each owns its own schema, its own invariants and its own failure modes. They talk through generated contracts, and a lint rule fails the build if one imports another's internals. The result is that a change to sweep economics cannot break payout idempotency, because there is no path between them.

YOUR SIDE — no keys, no chain code Customer pays in / takes out Your app balances, limits SDK signs each call THE ENGINE — you run this Operator API HMAC signature, replay guard, idempotency Deposit detection credit exactly once Payouts one broadcast, ever Key custody the only signer Treasury hot / cold float OUTSIDE Chains EVM networks Postgres every guarantee lives here signed instruction signed webhook back to your app Everything above the line runs on infrastructure you control The chain is the only thing that is not yours
The contexts

What each one owns, and what holds it

Key custody

Sealed key material, derivation, signing policy, velocity caps and the two-actor path for cold-to-hot movement.

Guarantee
A 24-hour signing cap cannot be reset by restarting the process, and is one cap across every instance.
Held by
exact sliding window + pg_advisory_xact_lock

The window is over individual signing amounts, not a bucket that resets on the hour — a tumbling bucket allows nearly double the cap across its boundary. This is the only context permitted to import a cryptographic library, enforced by a lint rule rather than by review.

Deposit detection

Watched addresses, watcher cursors, transfers, confirmations, re-org events and unknown transfers.

Guarantee
A confirmed deposit is credited exactly once, however many times its block is re-scanned.
Held by
UNIQUE (chain_id, tx_hash, output_index)

Idempotence lives in the schema rather than in a memory of what has been processed, so a watcher that crashes and re-scans a range costs time and nothing else. A re-orged deposit becomes a terminal state; the row is never deleted, because reconciliation has to be able to see that it existed.

Payouts

The payout state machine, idempotency records, assemblies, fee estimates, broadcast reservations and attempts.

Guarantee
One payout is broadcast at most once, even if the host dies mid-broadcast.
Held by
PRIMARY KEY on broadcast_reservations

The reservation is taken by an insert that either succeeds or violates the key — there is no window between checking and acting, because the check is the act. The signed payload digest is recorded with it, so a retry re-sends that exact payload rather than signing a new one that could also confirm.

Treasury

Positions per asset and tier, sweep candidates, batches, float thresholds and skip records.

Guarantee
A sweep that would cost more in fees than it consolidates is skipped, with the reason recorded.
Held by
fee ceiling as a proportion of the swept amount

Positions are updated from confirmed on-chain totals rather than from intent, so a broadcast that never confirms does not quietly move the books. Skips are rows, not silence — "nothing happened" is never an unexplained state.

Reconciliation

Position snapshots, operator-reported positions, findings and their resolutions.

Guarantee
A discrepancy stays open until a person closes it, with an actor and a reason on the record.
Held by
NOT NULL actor and reason on the resolution row

Three-way rather than two: chain against engine catches detection bugs, engine against your books catches integration bugs, and only having all three catches the case where two of them agree and are both wrong.

Webhooks

Subscriptions, the outbox, deliveries, attempts and dead letters.

Guarantee
An event is never lost because the process died between committing and sending.
Held by
outbox row written in the same transaction as the state change

Commit-then-send has a window in which the deposit is confirmed and the notification was never sent, and nothing in the system can later notice. A relay loop claims work with FOR UPDATE SKIP LOCKED so several workers share the queue without duplicating or blocking.

Operator API

The canonical string, timestamp validation, the replay guard and authentication outcomes.

Guarantee
A signature is accepted once, within sixty seconds, and every rejection looks identical.
Held by
replay guard + constant-time verify against a placeholder

An unknown operator is verified against a placeholder of the same shape rather than returning early, so response timing does not enumerate the operator list. The context is given the ability to check a signature and never the key that checks it — credentials stay in the host process.

Chain gateway

Chain configuration, RPC endpoints, block headers, fee observations and the broadcast log.

Guarantee
Swapping an RPC provider never touches payout logic.
Held by
endpoints are rows, not code paths

Fee observations are recorded over time, which is what makes low-fee scheduling and the sweep fee ceiling something other than a guess.

Enforcement

The rules are in CI, not in a document

Architectural rules that live in a wiki decay within a quarter. These fail the build.

RuleWhat it stops
Context boundary lintOne context importing another's internals. Only generated contracts cross, and a host may import only src/module.
Crypto import restrictionAny context except key custody importing node:crypto signing primitives, ethers, viem, bitcoinjs-lib or similar.
Key-bearing declaration ruleA variable named for a secret, seed, mnemonic or private key existing outside custody — or existing inside it without the KeyMaterial type.
Money safety lintA float literal or numeric operation on a money-named identifier, anywhere in a context or in the money package.
Source layout ruleLogic in a constants file, types in an application file, or anything at all at the source root except a re-export barrel.
Migration lintA destructive statement outside a contract migration. Expand-then-contract, or the build fails.
Planted violation suiteThe lint rules themselves silently breaking — deliberate violations are committed and the suite asserts each is still caught.
The last one matters more than it sounds. A lint rule that stops working is worse than no rule, because everyone keeps believing it is enforced. The planted-violation suite is the test that the tests still work.
A payout, end to end

Every hop, and what could go wrong at it

1
Authenticated
Signature verified, timestamp inside the window, signature admitted once.
2
Deduplicated
Idempotency key and instruction fingerprint written in the same transaction as the payout.
3
Assembled
Inputs pinned, fee estimated. Pinning is what makes the payload deterministic.
4
Policy-checked
Ceiling, sliding-window velocity, destination rules. A refusal is a recorded decision.
5
Signed
Inside custody. The digest is recorded before anything is sent.
6
Reserved
One insert. Success means this process owns the broadcast; conflict means it does not.
7
Broadcast
Attempt logged with its outcome and transaction hash.
8
Settled
Confirmed to depth. Positions move from the confirmed total, never from intent.

The failure everyone worries about

The host dies between signing and broadcasting. On restart, the payout is SIGNED and a reservation exists with a payload digest. The retry re-sends that payload rather than signing a new one — because two validly signed transactions for the same payout can both confirm, and the chain has no opinion about which one you meant.

The failure that actually happens

A client times out and retries with a fresh idempotency key. There is nothing the engine can do about that — the two requests are, as far as anything can tell, two different payments. This is why the key is a required argument in the SDK rather than an option, and why the documentation keeps saying it.

Concurrency

Where the interesting bugs live

Every guarantee above holds under concurrent writers, and the tests that prove it are the ones that found the real bugs.

what the tests actually do
// 100 concurrent submissions of one idempotency key.
// Caught a real bug: ON CONFLICT (payout_id) passed serially and failed here,
// because the table carries two unique constraints and the conflict arrived
// on the other one — leaving the transaction aborted mid-request.
await Promise.all(Array.from({ length: 100 }, () =>
  client.payouts.submit(sameRequest, 'withdrawal-0001')));
assert.equal(await countPayouts(), 1);

// The process is killed between signing and broadcasting, then restarted.
await killAfterSigning();
await restartAndDrain();
assert.equal(await countBroadcasts(), 1);

// Two instances race the same signing cap from opposite ends.
// Without the advisory lock this passes locally and fails in production.
await Promise.all([instanceA.signUpToCap(), instanceB.signUpToCap()]);
assert.ok(await totalSigned() <= cap);

Read the mechanism, then read the code

Every claim on this page is a constraint you can find in a migration and a test you can run.

Run it locally API reference