Data pipeline & refreshers

How data gets into creddit, and how it stays fresh.

The read-only-app principle

The Next.js app never writes data and never reads on-chain at request time for time-series. Pages are Server Components that read Postgres (schema onchain_credit) through a typed query() wrapper, plus a small amount of just-in-time RPC for "now"-only values. Everything time-series is produced ahead of time by off-chain jobs in scripts/ that read on-chain state (and a few external APIs) and upsert into onchain_credit. So:

• All ingestion lives in scripts/refreshers/*.ts, driven by scripts/refresh-*.ts cron entrypoints.
• The app is a pure reader of the DB. If a number looks stale, the question is always "did the refresher run, and did the page re-prerender", never "is the app computing it wrong live".
• Every venue is measured with the same APY convention: the realised ratio of an on-chain compounding index between two blocks, annualised by the actual elapsed time (annualizeRatio in src/lib/data/apy.ts). Never average per-snapshot annualised rates. The canonical write-up is in Metrics: how & why.

Where the data comes from

Source	Helper	Used for
Ethereum RPC (current state)	src/lib/data/rpc.ts, ETHEREUM_RPC_URL (default ethereum-rpc.publicnode.com)	live eth_call, eth_getStorageAt
Ethereum RPC (archive)	src/lib/data/rpc.ts, ETHEREUM_ARCHIVE_RPC_URL (default eth.drpc.org)	historical-block reads for trailing-ratio APYs; block-by-timestamp FALLBACK
Batched RPC	src/lib/data/rpc-batch.ts	refresher-scale Multicall3 + chunked eth_getLogs with retry
DefiLlama Coins API	src/lib/data/prices.ts, src/lib/data/llama-prices.ts, rpc.ts	USD prices + token decimals (historical endpoint also serves "now"; the collateral-exposure writers additionally accept a current quote only when it is under 6h old and above the confidence floor, else it counts as absent — other consumers require only a finite positive price); block-by-timestamp
NY Fed reference-rates API	inline in refreshers/sofr-rates.ts	SOFR + compounded averages + the SOFR index
Morpho Blue GraphQL	inline (blue-api.morpho.org/graphql)	curator-vault fee / allocation, market collateral-USD
Euler Goldsky subgraph	inline in refreshers/curator-vault-state.ts	Euler vault state (partial)
Fluid public API	adapters under src/lib/data/adapters/	Token symbols, decimals and prices for the Fluid collateral-exposure slices (the vault universe itself is the on-chain census)
creddit_indexer_fluid_dex_pool (rindexer)	SQL join in refreshers/fluid-dex.ts	DEX swap volume aggregation

In production both RPC vars point at Alchemy; the publicnode / dRPC URLs in code are the unset-env dev fallbacks.

Block-by-timestamp is the pipeline's most load-bearing dependency, and it is not the RPC. Every 6h refresher must resolve its snapshot block before it can read anything, and that lookup goes to DeFiLlama's Coins API (free, unauthenticated). On 2026-06-25 00:00 UTC it returned HTTP 500; five of the six refreshers that call it (fluid-ll, token-yields, aave-v3, sparklend, morpho) threw and the tick wrote nothing, while the paid archive RPC was healthy throughout. blockByTimestamp now falls back to a binary search over the archive RPC, with a process-level circuit breaker (re-probing Llama periodically so a long backfill self-heals) so one failure does not make every later lookup re-pay the ~15s retry ladder.

The fallback resolves the same block the primary does: both answer with the first block at or after the queried second, and the bisect matches it exactly, verified against the live chain. Parity between the two sources is the point: every row already in the database was anchored by Llama, so a fallback that rounded the other way would anchor ~12s earlier and read subtly different on-chain state than a live tick, invisibly.

At-or-after is not a function of the second alone, though. It is only well defined once the chain — and the source's index of it — is past the second, and for a boundary a live cron is snapshotting right now, neither may be. Measured at the real 2026-08-11 12:00:00Z boundary: the source's index sat 37s behind the chain and answered the last block it had indexed, a block before the second, and that answer was edge-cached for minutes so every refresher in the same tick inherited it. Which block a live tick landed on was therefore decided by the source's indexing lag at the instant of the first lookup, and a replay of the same boundary days later, with the index caught up, resolved a different one.

So both regimes now apply one rule. blockByTimestamp takes an explicit at-or-before anchor — the last block at or before the second, every answer verified against the chain — which is a function of the second alone, and every 6h snapshot END asks for it, live cron and backfill alike: the five refresh*ForSnapshot helpers take it as an argument and every caller passes it, the cron entry points included, as does every backfill that walks the 6h grid (the reserve, Fluid LL, token-yield and Morpho refills, plus the per-token token_yield_apy history scripts, which resolve their boundary block directly). Live and replay land on the same block because they apply the same rule to the same second, not because the timing lines up. The live tick pays for it: an aged boundary verifies in ~2 archive reads, but at the head, where the index has not passed the second, its answer cannot locate the boundary and the resolution falls through to an exact bisect (~30 reads, ~2.5s) once per lookup. And if the source answers a block the archive node has not imported, the resolution throws: that refresher writes nothing for the tick and the cron alert fires naming it, leaving a hole a later replay fills, rather than publishing a row on an unverified block.

Two deliberate exceptions. fluid-dex takes no anchor: it resolves both window endpoints five minutes behind their nominal seconds (a swap-indexer safety lag), so the second alone already decides the block, and asking for at-or-before would move a refill OFF the live row. backfill-fluid-core is the second: it stamps its fluid_dex_apy rows with the same anchored block as its Liquidity-Layer pass, because its default window predates the DEX resolvers (no live row exists in it) and that block is where the reserves it publishes were actually read. Separately, only the window END takes the anchor, where the window is annualised: the 6h-earlier endpoint is long past whenever it is resolved, live or replay, so it is already regime-independent. Adjacent windows meet at the boundary rather than overlapping it — window T ends on the last block at or before T, window T+6h starts on the first block at or after T, the same block when one lands exactly on the second and the next one when it falls in a gap — and each window annualises off its own two block timestamps, so nothing is double-counted. A window that is SUMMED over its blocks anchors both ends. That gap case is a one-block hole between adjacent windows (the normal case on a 12s chain), which a ratio of two endpoint reads never notices but a sum does: the block in the hole is counted by neither window. backfill-fluid-core's swap aggregation is the one such consumer — volume, fees and fee APY summed over (start, end] — so it resolves its window START by the same at-or-before rule and consecutive windows tile exactly; any new summed window does the same. That costs it one more aged-boundary verification per snapshot (~2 archive reads, against the ~100k a default run already makes). Paths that resolve a block from a timestamp for something other than a snapshot boundary — the event ledger's coverage floor, the portfolio replay grid (whose live tick reads at the chain head, not from a timestamp) — are unaffected.

Re-running an anchored backfill moves the rows it touches, once. Rows written before this change sat on whichever block the index happened to answer at the time; the first re-run of any anchored backfill re-lays its whole range on the deterministic block, an offset of one block (~12s of interest accrual on a 6h window). That is a one-time operational consequence of the rule, not a repair job: nothing re-lays a row until someone runs the backfill that owns it, and every run after the first rewrites the same values on the same blocks. It applies to every for (t = START_UTC; t <= alignedNow; …) walker — backfill-sparklend-stables, backfill-usds-reserves, backfill-morpho, backfill-emode-reserve-history, backfill-susdai and each ERC-4626 share-rate script through scripts/lib/backfill-erc4626-rate.ts — all of which are documented as idempotent and are run whenever a reserve or token is added. backfill-fluid-core moves on re-run too, on both endpoints: its stamped block shifts like the others', its Liquidity-Layer APYs are re-measured over a window that opens on the previous boundary's block, and its DEX rows additionally pick up the boundary block their swap window used to drop.

Scheduled refreshers

All crons run on the Hetzner box (root@dexhq.io), in UTC, via the scripts/run-cron.sh wrapper. The verified server crontab:

Cron (UTC)	Entrypoint	Writes (onchain_credit.*)	Upstream	How it works
0 */6 * * *	refresh-assets.ts	see sub-refreshers below	mixed	Orchestrator: runs the fourteen per-asset refreshers in order, catching per-asset errors so one bad asset never blocks the rest.
15 */6 * * *	refresh-vault-capacity.ts	vault_capacity	on-chain RPC + DefiLlama + Morpho Blue API	Per-strategy borrow/supply-cap headroom + total supplied/borrowed USD. Aave/Spark static governance caps (getReserveCaps); Fluid dynamic Liquidity-Layer caps (auto-expanding to maxBorrowLimit); Morpho Blue markets have no caps, so market(id) deposits/borrows are read on-chain and the deposits are stored as the cap (headroom = available liquidity), with posted-collateral USD from the Blue API (NULL on failure); registry-driven vault/carry set. Upserts only when values change (bumps changed_at), else touches last_checked_at.
30 */6 * * *	refresh-collateral-exposure.ts	market_collateral_exposure (basis fluid_borrow), market_risk_current	on-chain RPC (vault census + tick storage) + Fluid public API (token labels / prices)	Fluid "underwritten capital": per collateral asset, how much of the supplied stablecoin is borrowed against it. The vault universe is the on-chain census, one resolver call returning every vault's state at the anchor block, not the public list endpoint (which filters). Normal debt is the vault's own Liquidity-Layer borrow; a smart-debt leg is the shared DEX pool's Liquidity-Layer borrow split by the vault's share of the pool's borrow shares, so pool participants that are not vaults are never allocated away. The market's book is the Liquidity Layer's own total borrow for that stablecoin at the same block, and everything the vaults do not account for — unlisted pool participants, non-vault borrowers, a vault whose collateral cannot be named or priced — is published as an explicit Unattributed slice, with coverage_pct the attributed share of that book. Vaults wound down (borrow limit collapsed below the vault's own minimum borrow) keep their existing debt in the slices but contribute no capacity. Plus a tick-based risk layer read straight from the vaults' own tick/branch storage (readFromStorage via Multicall3), measured against the vaults' CURRENT exchange prices, giving distance-to-liquidation buckets. Markets are isolated: one whose stable price or book read fails keeps BOTH its exposure rows and its risk row, and a stablecoin whose book reads zero is treated as no market only when no vault reports debt in it.
45 */6 * * *	refresh-lending-positions.ts	lending_reserves, lending_borrowers, chain_scan_cursors, lending_positions_current, market_collateral_exposure (basis position_collateral), market_risk_current	on-chain RPC + DefiLlama	Aave v3 / SparkLend account-level positions. Cursor-scans variableDebt mint/burn logs to maintain a borrower registry, multicalls current debt for the candidate set, reads the largest borrowers' collateral until 99% of the borrowed book is covered, or 1,000 accounts, whichever binds first (getUserAccountData + config bitmask + per-reserve balances, all pinned to one anchor block), and attributes each account's stablecoin debt to its collateral pro-rata. Debt the scan cannot attribute truthfully — the tail below the coverage target, an account with an unreadable leg, an account holding a reserve discovery could not resolve — is published as an explicit Unattributed slice, and coverage_pct is the attributed share of the book. First run does the full log backfill (to 2023) and can take tens of minutes; later runs scan only since the stored cursor. A failed inner read is skipped, never written as zero, and a reserve that cannot be read moves every account holding it into the remainder (logged, naming the reserve and the account count) rather than passing as an empty leg.
50 */6 * * *	refresh-portfolio.ts	portfolio_position_snapshots, portfolio_flow_events_v2, portfolio_held_pts, portfolio_wallet_index, chain_scan_cursors	on-chain RPC + Dune mirror (token_price_bars) + DefiLlama (block-by-ts only)	Read-only portfolio spine + flow ledger (WS4). MARKET marks come from the Dune price mirror (same-bar division); DeFiLlama is no longer a mark source (Milestone C), only the block-by-timestamp lookup. Snapshots every eligible registered wallet's legs across all six venue readers at one anchor block, valued in both marks, keyed on the aligned 6h window (upsert -> a same-window re-run is idempotent); cursor-derives the tick's block window from the ingested event store for the derived wallet set (wider than the snapshot set; see below), and merges the receipts. Registration backfills moved to the minutely drain cron below. See the portfolio subsection below.
5 * * * *	check-ingester-freshness.ts	nothing (read-only)	pm2 process list + the checkout's .next/BUILD_ID + chain_scan_cursors + the live RPC head	Hourly alarm, two arms. Freshness: pages when the event ingester started before the checkout's current build (15-minute grace for the deploy's own restart), is not online, or is missing. Feed lag: behind an ingester of this checkout that is online and NOT already paging, pages when any followed stream's ledger:<stream> cursor is more than 6h behind the chain head (INGEST_FEED_LAG_MAX_HOURS), or when a rolled-out stream has no cursor at all. See deployment.
* * * * *	drain-portfolio-backfills.ts	portfolio_position_snapshots, portfolio_flow_events_v2, portfolio_backfill_state	archive RPC + DefiLlama	WS5 registration-backfill queue drain: FIFO, one wallet at a time (claim -> archive-routed child -> next), until the queue is empty or the per-invocation cap (PORTFOLIO_BACKFILL_MAX, default 10). Quiet no-op when the queue is empty. Stale-running reaper + attempts budget run every invocation. A failed child is retried while its budget holds and then parked (MAX_BACKFILL_ATTEMPTS = 4), and EVERY failure exits 2 to page the WS8 cron alert, naming the wallet; a row parked by WRITER CONTENTION is re-queued by the drain itself after a rest (bounded, see the self-heal below). A signup's portfolio therefore appears ~a minute after sign-in rather than at the next 6h tick. The child stamps a live reconstruction cursor (progress_done/total/started_at, migration 060, best-effort) after each daily grid point; /api/portfolio/summary derives the /portfolio building state's progress bar + measured-rate estimate from it while the row is running. See Registration backfill.
20 3 * * *	refresh-morpho-universe.ts	morpho_market_registry (status='universe' rows), chain_scan_cursors	archive RPC	Daily. Exhaustive Morpho Blue market-universe ingestion (T4 §3.5): a cursor-driven CreateMarket log scan that keeps EVERY mainnet market in the registry as a universe row (loan/collateral token + on-chain decimals), so the portfolio loader can qualify any market. First run walks from the Morpho Blue deploy (MORPHO_UNIVERSE_FROM_BLOCK overrides for the deep backfill); steady-state scans only new blocks. Inert to the curated machinery (see database.md). Bounded per-tick by the T4 §3.6 discovery intersection, so the exhaustive universe never costs a per-tick read of thousands of markets.
40 3 * * *	refresh-metamorpho-factory.ts	metamorpho_vault_registry (status='universe' rows), chain_scan_cursors	archive RPC	Daily. MetaMorpho factory vault-universe ingestion (T5 §3.5): a cursor-driven CreateMetaMorpho log scan over BOTH factories (v1.0 0xa9c3…1101 + v1.1 0x1897…5c24, identical event) that keeps EVERY MetaMorpho vault in the registry as a universe row (on-chain-verified asset + share/asset decimals), so the portfolio erc4626 loader values a wallet's deposit in ANY MetaMorpho vault as a Money market fund (role curator), not just the hand-curated Repo-lending subset. First run walks from the v1.0 deploy (METAMORPHO_FACTORY_FROM_BLOCK overrides); steady-state scans only new blocks. This is the PORTFOLIO universe ONLY — curator-vaults.ts / the Repo lending page are NOT widened. Bounded per-tick by the same T4 §3.6 discovery intersection.
*/10 * * * *	refresh-portfolio-discovery.ts	portfolio_wallet_index, chain_scan_cursors	on-chain RPC	Every 10 min. T4 §3.6 discovery enrollment drain: enrolls eligible wallets that have no FRESH completeness certificate (a one-time FULL-universe current read per wallet that indexes its current Morpho/erc4626 holdings + stamps accounts.discovery_scanned_*), capped PORTFOLIO_ENROLL_MAX (default 10). Selection is freshness-based, not presence-based (coverage hardening): missing, older than the accounts row, or past the 18h staleness horizon. That is the repair half of the freshness gate — a demoted wallet is re-certified within one run instead of paying the full-universe read tax indefinitely, and it is what would have repaired the incident wallet, whose stale stamp existed and so was skipped by the old "no watermark at all" rule forever. Un-scanned wallets force the 6h batch to a full-universe read (the fail-open), so draining them is what makes the discovery intersection ENGAGE for new signups; quiet no-op once every eligible wallet is watermarked (the migration-050 seed watermarks all historically-active wallets, so this drains only genuinely new signups). Best-effort per wallet: a failed read never sets a watermark (retried next run).
30 4 * * 0	refresh-portfolio-reconcile.ts	portfolio_wallet_index, accounts (discovery certificate)	on-chain RPC	Weekly (Sun). T5 §3.6 discovery reconciliation SAFETY NET: reads each SCANNED wallet against the COMPLETE universe (not the discovery-bounded read) and adds any currently-held Morpho/erc4626 membership the index MISSED, raising a WS8 drift alert. Bounds a discovery bug to "found within a week + alert", never "silently wrong PnL". Wallets processed least-recently-reconciled first (accounts.discovery_reconciled_at LRU, which only this sweep writes; ported off the portfolio:discover:% scope rows, which migration 106 deletes — selecting from them would have silently emptied this sweep and killed the backstop with no error anywhere), capped PORTFOLIO_RECONCILE_MAX (default 50), so coverage rotates across all scanned wallets over successive weeks. Carries a starvation tripwire: eligible wallets present but an empty batch prints a [fail] STARVED line and exits non-zero, so "the backstop checked nothing" can never read like a healthy no-op. It also prints one [fail] DIVERGENCE wallet=… venue=… key=… (repaired) line per missed membership and repairs dangling links (an account_wallets row whose wallet has no accounts row, so since 056 the pipeline can write nothing for it at all: re-mints the shadow account via trackedAccountUpsert and requeues its backfill). Any of the three ends the run non-zero, because a run-cron alert needs both a non-zero exit and a grep-matching line. Steady-state finds ZERO AGED drift; a position the byproduct has not reached yet is repaired quietly rather than paged, which is what keeps a brand-new position from becoming a weekly false alarm. Its cohort is the widened DERIVED population, so a briefly-unlinked wallet — which keeps a fresh certificate across the gap and is therefore bounded on re-add — is included.
50 3 * * *	sync-money-market-funds.ts	money_market_manager, money_market_fund_registry	Morpho Blue API (discovery) + on-chain RPC (everything the decision rests on) + DefiLlama (asset prices, strict bounds: a refused quote keeps the last good pair for up to a day and flags the row)	Daily. Money Market Funds coverage (processes.md). Walks every Morpho mainnet vault of both generations from the API, shortlists anything at or above a review floor set BELOW the listing floor, confirms size / fee / roles / notice period / asset decimals on chain at one pinned block, and scores each against the nine gates. Persists every discovered vault, not only the listed ones, so a fund that grows past the floor is picked up without a code change. PROPOSE only for a new manager: a curator nobody has approved lands proposed and its funds do not render, however well they score. A human applies one with --approve-manager <key>. A manager read off the fund's own NAME also only proposes, even when that house is approved, because a vault name is a string its deployer chose; an administrator confirms it with --bind-manager <slug> <key>. The gone sweep runs only on a run that proved its own discovery: both walks end on an empty page by design, so a degraded resolver returns a short list with no error, and NOT (address = ANY(ARRAY[])) is true for every row. Each generation's walked count is checked against the endpoint's own reported total AND against how many rows the registry already carried; short on either, nothing is marked gone and the run says so loudly. --dry-run is read-only including beside an admin flag. Manual crontab add — apply migration 086 FIRST.
30 3 * * 1	refresh-vault-risk.ts	vault_risk_params	on-chain RPC	Weekly. Per-strategy max-LTV + liquidation threshold (Fluid getVaultVariables2Raw bit-unpack; Aave/Spark self-discover the live e-mode category then read its collateral config). LTV/LT move on a governance timescale, so weekly is plenty. Upserts only on change with a changed_at audit.
30 4 * * 1	sync-portfolio-tokens.ts	none in propose mode (--approve inserts portfolio_tokens)	DefiLlama stablecoins API + on-chain RPC	Weekly. Portfolio token-registry sync (T6). PROPOSE + ALERT only, never mutates on the cron: ranks the DefiLlama top-20 mainnet stablecoins (by Ethereum circulating), diffs them against portfolio_tokens, and asserts (a) every assets profile ticker has a registry row and (b) every wallet-tracked variable_rate row with a base resolves a redemption rate (a rate_getter on its registry row OR a token_yield_apy.share_rate row; a no-base row has no redemption line to resolve, see the Registry sync subsection). Ranking drift, a profile gap, or an unresolved rate source each fire a WS8 alert. A human applies an addition with sync-portfolio-tokens.ts --approve <SYMBOL> (resolves the mainnet address from the DefiLlama detail endpoint + reads decimals on-chain, inserts an idle par row); it never retires a row. See the Registry sync subsection below.
0 13 * * 1-5	refresh-sofr.ts	sofr_rates	NY Fed	Weekdays at 13:00 UTC (NY Fed publishes ~08:00 ET). Pulls the latest ~30 business days of SOFR + compounded 30d/90d/180d averages + the SOFR index and upserts by date (generous overlap captures late revisions).
10 4 * * 1	chat-retention.ts	chat_conversations, chat_messages (cascade), chat_usage	— (DB only)	Weekly (Mon 04:10). Prunes the app-state chat tail so it does not append forever (audit B6): deletes conversations with no activity in 365 days (their messages cascade via the migration 038 FK) and chat_usage rows older than 400 days, both in one transaction. Child/leaf deletes only — the migration 056 uid FKs point at accounts, so pruning a conversation or usage row never cascades into an account or its portfolio history. Portfolio history is RETAINED IN FULL by design (see database.md → Retention). Manual crontab add, prod only (staging carries no crons).
15 2 * * *	ops/backup-creddit.sh	(DB dump, not a table)	—	Daily Postgres backup of the creddit DB at 02:15 UTC. Server-only script (not committed in this repo).
hourly	(disk-usage alert)	—	—	Operational alert, not a data job.

Sub-refreshers inside refresh-assets.ts (the 6h "assets" job)

refresh-assets.ts is an orchestrator (scripts/refresh-assets.ts) that runs these modules in order. Each is independently importable and several have a *ForSnapshot export the backfills reuse.

Module	Writes (onchain_credit.*)	Upstream	How it works
refreshers/fluid-ll.ts	fluid_ll_apy	on-chain RPC	Fluid Liquidity Layer supply/borrow exchange prices; APY = annualised realised index ratio over the 6h window (and a 24h-trailing variant for smoother long charts).
refreshers/fluid-vault-rates.ts	fluid_vault_rates	archive RPC	The per-VAULT counterpart of the row above: Fluid prices borrowing per TOKEN at one shared Liquidity Layer, and each vault then applies its own term on top, which the token series cannot see. One row per covered Fluid vault per window, from the vault's OWN supply/borrow exchange prices read through FluidVaultResolver.getVaultEntireData at the SAME two blocks the Liquidity-Layer snapshot used — sharing the block-resolution rule is what makes the two series comparable at all, and the 6h boundary itself is resolved ONCE per tick by refresh-assets.ts and handed to every Fluid series a carry's history joins — the layer's, this one, and fluid_dex_apy, which thirteen of the twenty-one Fluid carries anchor on — so a run that straddles a boundary cannot stamp them differently and drop the window from a Fluid carry. On a normal-debt vault the index grows at the layer's growth times borrowRateMagnifier / 10000, so at 1x the two series are identical and the published figures do not move; Fluid routes borrow incentives through that magnifier, so a vault can sit below 1x for as long as a programme runs. On a smart-debt vault the tokens' interest lives inside the DEX borrow share and the vault index carries the vault's own fixed overlay ALONE (0 today on all seven, negative when the vault pays its borrowers). Its universe is the registry (status IN ('active','proposed')), so an approval changes what it covers with no deploy. A failed read and a block at which the vault did not exist both write NO ROW — the first counts as a failed item for the partial-failure floor, the second does not, and neither is ever a coerced zero. --backfill [--from] [--to] [--vault] walks the same 6h grid oldest-first (each window's 24h column is measured against the stored row 24h behind it) and is idempotent; the block pair is resolved once per window for every vault due at it.
refreshers/fluid-dex.ts	fluid_dex_apy	rindexer swaps + DefiLlama	Takes its 6h boundary from the tick refresh-assets.ts resolves once per run, like the two Fluid rate series: thirteen of the twenty-one Fluid carries anchor their history on this table and join those two to ITS snapshot. Sums swap volume from the indexer over the 6h window, reads fee rate + smart-col/smart-debt reserves via DexResolver, prices in USD, computes fee_apy_usd = fee_window_usd x 1460 / (tvl_col + tvl_debt), LP-net of the protocol revenue cut. Also reads FluidDexResolver.getDexState(pool) at block_at_snapshot for the per-1e18-share pot content + the pool price/centre pair (6 raw columns, migration 047) — the inputs to the REALISED cum-return series. That read is the only NON-FATAL one in the pool's Pass-1: it is caught individually so a resolver miss degrades to NULL columns instead of costing the pool its fee/TVL row, and the upsert COALESCEs the six columns so a later failed re-run cannot wipe values already landed.
refreshers/token-yields.ts	token_yield_apy	archive RPC	Share-to-asset rate per yield wrapper at the snapshot block (convertToAssets(1e18) for ERC-4626, getStETHByWstETH for wstETH, getRate() for weETH/ezETH, exchangeRate() for cbETH (the Coinbase-published rate), NAVConsumer for reUSD, external rateSource for osETH, etc). Stores share_rate, 24h supply_apy, 30d apy_30d, and total_supply. After each write it repaints a silent publish gap, if the snapshot just closed one: a NAV-style wrapper that publishes late leaves a 24h window with no mark in it, so that window reads 0% and the catch-up window reads double — same income, wrong measurement window. When the previous material rate move is between 24h and 7 days back and at least one reading strictly inside the span printed near zero (the sawtooth's zero tooth — without it a daily-rebasing LST would be marked down every time one of our own snapshot rows went missing), every snapshot from it through the new one is rewritten to the realised growth across the span, annualised by the true elapsed time (gapRepaintForLatest / repaintSilentGap, shared with the one-time repair; the full rule is in metrics.md). A continuously accruing wrapper never reaches it — its last material move is one snapshot back — a flat run longer than the 7-day cap is a parked asset and keeps its truthful 0% readings, and a NULL supply_apy is never written over. The repaint is deliberately OUTSIDE the per-token failure tally: the snapshot itself is already written and correct, so a repaint that cannot run logs [token-yields/gap-repaint-skip] rather than counting a lost token and tripping the partial-failure alert. A second, RUNTIME union comes from money_market_fund_registry, read per snapshot rather than at module load: a Money Market Fund listed by a manager approval starts being recorded on the next tick instead of waiting for a release, which is what stops its return series beginning with a hole exactly where a reader looks first. The union is by address with the static entry winning, and it is wrapped so a pre-migration tick is a no-op rather than a crash that takes every other tracked token down with it. Registry-driven besides: every curator vault in src/data/curator-vaults.ts and every Fluid fToken in src/data/fluid-ftokens.ts is auto-included (an fToken needs a row here so its /portfolio leg has a quoted APY). A wrapper that is NOT one of those two registries is a hand-curated BASE_YIELD_TOKENS entry, and the August 2026 USD yield batch added five (USD3, srUSDe, sUSDD, stUSDS, wsrUSD; see the batch's section under Backfill scripts). The multi-strategy funds in src/data/erc4626-universe.ts are auto-included on the same registry rule, which is how the 20-decimal IPOR Fusion share arrives here with shareDecimals: 20 / rateDivisorPow10: 16 (its section under Backfill scripts). scripts/refreshers/token-yields.test.ts decodes those five and the IPOR entry against recorded mainnet raw values, because a wrong shareDecimals or rateDivisorPow10 does not throw: it stores a rate off by a power of ten for as long as nobody looks. The mellow-oracle kind is wired PER VAULT, not off shared constants: a Mellow oracle answers for one vault and one asset, so each Lido Earn fund carries its own rateSource (the oracle), oracleAsset (the quote asset word) and oracleAssetDecimals, and the stored rate is 10^(18 + shareDecimals − assetDecimals) / priceD18 — 1e18 for earnETH (18-dec share over 18-dec ETH) and 1e30 for earnUSD (18-dec over 6-dec USDC). The same branch also rejects a report the vault has not accepted: word 2 of Mellow's DetailedReport is isSuspicious, and a flagged report is stored WITHOUT being handed to the vault, so it is a price no deposit or redemption converts at — reading it throws, which writes no row. See Lido Earn USD under Backfill scripts. A second per-token override, supplySelector, names the getter the total_supply column is read with when a vault's share count is not its ERC-20 totalSupply(). A failed read writes no row, never a zero — so a snapshot can be MISSING for a token that is otherwise covered, and every reader must treat that as a gap rather than a 0% yield (wrapperRowPresent in carries-table.ts; see metrics.md, "A missing wrapper row is a gap, never a zero"). This is a real failure mode, not a theoretical one: the 2026-06-25 00:00 UTC run wrote only 4 of 78 tokens when the DefiLlama block-by-timestamp lookup returned HTTP 500 (see the block-by-timestamp note under "Where the data comes from"). Its partial-failure floor is tuned separately (partial: { knownPermanent: 5, ratio: 0.03 } in refresh-assets.ts): five curator vaults fail this job every single run, and leaving them inside a bare 10%-of-everything ratio meant the money market funds roughly doubling the item count also doubled how many NEW silent failures the alert would tolerate, from about three to about eight. The known count now sits outside the ratio, so the tolerance stops moving with the item count.
refreshers/token-basis.ts	token_basis, token_price_bars	Dune mirror + DB	basis = market_price_usd / redemption_value_usd - 1, where redemption value = (latest share_rate from token_yield_apy, or 1 for par tokens) x numeraire (USD / ETH / BTC). Market + ETH/BTC numeraire quotes come from the Dune price MIRROR (getBarSeriesAt(tsSec) over token_price_bars), NOT DeFiLlama. The same-bar rule, via newest-common-bar pairing: for an ETH/BTC-numeraire token the reader returns each leg's whole covering-bar window (all bars in [align(ts) − 48h, align(ts)], BAR_STALE_HARD) and computeBasisFromBars divides the token and its numeraire at their NEWEST COMMON bar_ts. It never divides a token bar by an independently walked-back numeraire bar — that leaks the cross-bar ETH/BTC move straight into the basis (the ±30-50bps of cross-vintage noise this refactor deletes). No common bar in the window counts the token failed (never fabricated, never a DeFiLlama fallback); the run logs how many tokens paired at a >1h-old common bar. A snapshot where ALL market-class tokens fail — meaning no bar within the 48h ceiling, a genuinely DEAD mirror rather than one merely lagging past the 6h soft threshold (M20) — prints a literal [fail] line and exits non-zero. No token skips the mirror bar any more. The pinned basis class was retired by #810 Y2 (M19), so sUSDS is measured off its own Dune tape like everything else and can publish a non-zero basis; there is no branch that writes market_price_usd := redemption_value_usd. Every token in this loop therefore reads a real market bar, which is why the mirror-health counters and the overall tally now count the same events: the run freezes only when marketOk === 0 && marketFailed > 0, and a token with no bar is a failed item rather than a silent bypass. Runs after token-yields so the current tick's share_rate is available. Prepends the best-effort HOURLY mirror sync (syncBars → token_price_bars, ~0.25 credits) over a window opening at the OLDEST PER-TOKEN cursor in token_price_sync_state — never the table-wide MAX(bar_ts), which used to step over a lagging token's window permanently — plus the auto-backfill of any token missing its history at the 2026-01-01 floor, the accept-then-retract spike gate, and the dark-feed [fail] alert for any standing feed with no accepted bar in 12h (a >48h gap is still clamped to 48h and the older gap is caught up by the bulk tool) that supplies the very bars this run then consumes. The computation is the shared computeBasisFromBars (src/lib/data/basis-compute.ts), reused by Milestone C's priceInBookFromMirror via the exported pairAtNewestCommonBar, so the paths cannot drift. Also runs the AGGREGATE leg for every llama_aggregate token (BTC.b, USDai): DefiLlama's own hourly observation, stamped at the hour, for every hour since that token's own newest bar — or from the 2026-01-01 floor when its history does not reach it, which is this feed's auto-backfill on add. It needs no ordering against the Dune legs (nothing it reads was written by them) and is best-effort in the same way. It records its own token_price_sync_state row, so the spike gate has a cursor for it and the dark-feed exemption is not silently applied. And the two legs the pricing-categories release added, both after the three writers above so they only ever act on what those legs did not manage: the HOLE FILL, which buys any hour older than 6h with no accepted bar from CoinGecko and then DefiLlama and cross-checks every accepted Dune bar against the two vendors (Filling the tape's holes); and the LIVE-PRICE CHECK, which runs R6's band over every market-priced asset and prints a [fail] price disagreement line (plus a non-zero exit) for any asset whose level the page would be withholding (Live prices). See Token price bars (Dune mirror), The aggregate feed and Rebuilding token_basis from the mirror.
refreshers/token-pricing.ts	token_pricing_measurements, portfolio_tokens.loopable	archive RPC + DB	The capacity series and the pricing-category verdicts. Every registry row that declares an ATOMIC route is read at ONE block pinned for the whole pass — how much can be minted and how much can be redeemed right now — and each reading is appended in USD, converted through the underlying's newest stored bar so a capacity and a position valued in the same tick agree about what a dollar of the underlying is. A read that FAILED stores nothing and is counted; "the vault can pay nothing" and "we could not ask" are opposite statements, and a zero for the second would move a category on an RPC timeout. It then recomputes every yield-bearing row's category CANDIDATE and its loopable flag from those readings and the weekly market measurement. loopable is written back to the row when it is known and left alone when it is not; the candidate is never written. A candidate that has disagreed with the row's DECLARED category for fourteen consecutive days is reported on a [fail]-prefixed line and, on the tick the fortnight completes, posted to the alert bot directly — the wrapper's own alert fires only on a non-zero exit, and this leg must never produce one for a verdict. The run still exits zero: applying a flip is a registry edit in a pull request, because a category change restates how an asset is valued. Runs straight after token-basis.ts, which is what has just written the bars it converts through. Zero partial-failure floor, like funding-shock: the work list is the handful of rows that declare a route, so a route that could not be read is never routine noise.
refreshers/aave-v3.ts	aave_v3_reserve_apy	on-chain RPC	Aave v3's supply / variable-borrow accumulators (RAY-scaled) annualised over 6h. The pair is read BROUGHT CURRENT to each anchor block, from the Pool's getReserveNormalizedIncome / getReserveNormalizedVariableDebt, never from getReserveData's stored liquidityIndex / variableBorrowIndex: the stored pair only advances when a transaction touches the reserve, so on a quiet reserve the ratio spans the gap between the last two touches while the window annualises 6h (see refreshers/aave-pool.ts). getReserveData is still read once, at the window end, for the spot rates and the deposited total. Also stores deposited plus the liquidity the reserve will actually release at the snapshot block — the pool-enforced virtual balance, bounded by what the aToken holds — NULL when that read fails, never a derived aToken-supply-minus-debt figure and never a zero. Reports a per-item tally; a reserve written without its liquidity leg counts as a failed item, and each one prints [aave-v3/liquidity/fail].
refreshers/sparklend.ts	sparklend_reserve_apy	on-chain RPC	Same method as Aave, brought-current index pair included, against SparkLend's pool (separate table to avoid (snapshot_ts, token_address) collisions on shared tokens). SparkLend is a v3.0 fork with no virtual accounting, so the liquidity a borrower can actually draw IS the aToken's own token balance; same NULL-on-failure rule, same per-item tally, marker [sparklend/liquidity/fail].
refreshers/morpho.ts	morpho_market_apy, market_collateral_exposure (basis isolated_market)	on-chain RPC + Morpho Blue API + DefiLlama	Active Morpho Blue markets from morpho_market_registry (both tracks). Supply + borrow share rates from Morpho.market(id) (borrow_share_rate added migration 041), realised trailing-24h APYs, spot borrow APY from the AdaptiveCurve IRM. The market's four totals are first advanced to the anchor block's timestamp by replaying Morpho's own expectedMarketBalances (accrueMorphoMarket), because market(id) returns them as last written; the book-size columns come from the same accrued totals. Without it a market quiet enough to go untouched across a window would report the span between its last two touches as the window's yield. Market params read via on-chain idToMarketParams (exact 1e18 lltv). Exposure rows for repo-track markets only, and each is a marked USD figure: the borrowed loan-token amount times the loan token's live USD price, the convention the other two writers use on that column. Rows are written per market, so one whose read fails or whose loan token has no usable price keeps its previous snapshot while its siblings advance, each skip printing a [morpho/fail] line naming the market and the reason. Two admitted markets on the same deposit asset collateralised by the same TOKEN are indistinguishable downstream (both readers resolve an isolated market's book by collateral address): NEITHER publishes, both keep their previous snapshot and both are named in the log (registry admission dedups by collateral/loan address pair, so this is a backstop rather than a live case). Two markets whose collateral merely shares a TICKER are a different case and BOTH publish, because the exposure key carries the collateral address as of migration 079 (database.md). The delete that precedes a write is scoped to that market's own collateral token and deliberately not to its ticker, so a sibling sharing the ticker keeps the row it just wrote, and re-running a window after a market left the registry leaves that market's row at that stamp to age out like any other stale row instead of removing it. The mark has two sources, and the window a run may value follows from which one it uses. The cron marks at the CURRENT quote, so it can only value a window at most ~2 windows behind wall clock; past that it fills APY / index history and writes no exposure row rather than stamping today's price on an old book. A backfill can instead mark at the VINTAGE quote, the loan token's price at the window itself, and then any window is valuable (see Backfill scripts). Nothing else differs between the two: same per-market isolation, same collision guard, same no-price skip, and the vintage quote is held to a bound of its own (it has to have been observed within 3h of the window it marks, else that market's row is skipped). A registry that reads fine but admits no active market writes nothing and counts as a failed item; only a registry read that THROWS falls back to the seed set. Live runs fetch posted-collateral USD from the Blue API best-effort.
refreshers/funding-shock.ts	funding_shock	archive RPC + DefiLlama (prices)	Per shown carry: what five more percentage points of utilization would do to its funding cost, and how much additional borrowing that is. Every input for one strategy is read at ONE archive block. Aave v3 and SparkLend simulate the reserve's own deployed rate strategy twice (current and post-borrow) rather than reimplementing its curve, with the debt the Pool prices rebuilt from the variable debt token's scaled supply and the reserve's STORED index — the pair the Pool itself used at its last write, since interest accrued since then has not been priced yet. Morpho brings the market forward the way Morpho does, then FREEZES the adaptive target rate and moves only utilization (borrowRateView returns an interval average and would let a stressed call move the target rate retroactively). Fluid ports calcRateV1/calcRateV2 literally, truncations and all, over the Liquidity Layer's accrued totals, and maps the layer rate onto a vault's borrowers with its live borrowRateMagnifier. The run also SAMPLES each market's curve across the whole utilization axis and stores it beside the two figures, through the same model code path and the same parameters, so the detail panel's chart is drawn from what produced the numbers rather than from a second implementation of four venues' arithmetic; both scenario points are pinned onto the sampled curve, so a marker cannot land off the drawn line. It is a field on an existing jsonb payload — no migration, and modelVersion is unmoved, since the calculators did not change. A row written before the field existed is still served and renders no chart, and a curve that could not be sampled is dropped with a diagnostics.curve_unavailable note rather than withholding the market: sampling runs after the venue's parity gate has accepted the figures, so a fault there is a drawing fault, and withholding would both blank a real number and trip this refresher's zero-tolerance alert. A market publishes only when its own state reproduces its own published borrow rate EXACTLY; anything else is stored with a health state (unsupported_model, parity_failed, stale_inputs, configuration_changed_revalidating) and shows as no data. Every rate-driving parameter is re-read and fingerprinted each run: a change is recomputed and revalidated in the SAME run and published if it passes parity, with a structured configuration_changed diagnostic and a [funding-shock/config-change] line — holding a passing answer back for a cycle would blank SparkLend's WETH row a quarter of the time, since its kink rate follows an external feed that moves daily. Registered with partial: { ratio: 0 }, unlike every other sub-refresher: a withheld market here is never routine, so one trips the cron alert, and each withheld market prints a [funding-shock/fail] <key> (<health>): <reason> line — the /fail] marker and the front-loaded key are what put the market's NAME in the alert's why: slice, which greps for that marker and cuts each line at 220 characters. Each row is stored inside its OWN try: a row that was computed but could not be written prints [funding-shock/fail] <key> (not stored): <error> and counts as a failure, so one unstorable row costs one row rather than every strategy after it in the run. Every read is retried on transport failures only (three attempts, backing off), so a rate limit on the ~300 sequential archive calls a run makes does not withhold a market; a revert or malformed returndata is answered by the node and is not retried. Also runnable read-only as npx tsx scripts/refreshers/funding-shock.ts --dry-run [--strategy <key>], which computes exactly what the cron would write and prints it as a table without touching the database. The table's first column is as wide as the longest strategy key in the run, so a key can be copied out of it and fed straight back to --strategy, which matches exactly; a key that matches nothing is an error rather than an empty run, since an empty table under 0 healthy, 0 withheld reads exactly like a clean one.
refreshers/yield-token-assets.ts	assets	DB rollup	Rolls token_yield_apy history into the Asset-profiles table: current_apy (latest apy_30d), 1M/YTD/1Y cumulative returns, productive vs underlying market cap.
refreshers/curator-vault-state.ts	curator_vault_state	Morpho Blue API + Euler subgraph	Per money-market curator vault: fee, net APY, total assets, and per-market allocation (collateral / supplyUsd / maxLltv). Markets sharing a collateral symbol are merged into one entry, and maxLltv is the HIGHEST liquidation LTV among them — a max, not a supply-weighted average, so it can belong to a market holding only part of the summed supply. Euler allocation is partial; those vaults get description + return chart but no allocation block yet. The vault registry also declares what each fund's yield is denominated in: a symbol that is neither a known par stablecoin nor a known yield-bearing wrapper fails the build rather than being assumed to be USD, so a sync that introduces a new denomination has to classify it first.
refreshers/money-market-funds.ts	money_market_fund_state, money_market_fund_allocation	on-chain RPC (everything published) + Morpho Blue API (discovery, veto, labels) + DefiLlama (asset prices, strict bounds: a refused quote keeps the last good pair for up to a day and flags the row)	Per listed Money Market Fund: fee, roles, notice periods, caps, per-market allocation, exit liquidity, pending changes, deposit and exit gates, and depositor concentration, all read at ONE pinned block. The API is asked only which pending calls and which holder addresses to go look at, for its own warnings, and for labels; every published number is then confirmed on chain, and a failed read writes NULL and records the divergence rather than being filled in from the API. A Vaults V2 exit is not a MetaMorpho exit: maxDeposit / maxWithdraw are hardcoded to zero on V2, and a plain withdrawal only reaches the vault's own cash plus its ONE liquidity route, so everything else is quoted separately as force-deallocatable with its penalty. A fund that holds another fund is walked through to the child's own exit path, bounded to two levels. Each tick also cross-checks its own exit figure against Morpho's published one and writes anything more than 2% apart into chain_api_delta (a log line for a human, not a gate). Bad debt is the one figure here the chain cannot confirm and is read once per tick BY MARKET ID (so it covers both generations), then prorated to each fund by its share of that market's supply; a market the response did not carry is NULL, never zero. A reading that FAILED, and a 200 that answered for fewer than half the markets it asked about or for none of a given fund's, both leave every stored figure and every chip exactly where they were, marked with the date they were taken: one degraded response must never read as a clean bill of health on a tab whose subject is risk. Reports a per-item tally, so a run that loses an anomalous share of its funds counts as a partial failure rather than a quiet success. Its work-list is the registry, so an approval changes what it covers with no deploy.
refreshers/pendle-markets.ts	pendle_markets, pendle_market_state	Pendle hosted API + on-chain RPC	Registry sync (new markets from the API, verified on-chain via readTokens()/expiry() before insert; deterministic active → matured flip from maturity_ts, mirrored onto term carries in carry_registry — but there on the 30-day runway floor rather than at maturity, and this pass is the single owner of that flip since sync-carries is ad-hoc; see processes A.0 and src/lib/data/carry-runway.ts) plus a matured catch-up: the active list is a snapshot of now, so a market that had already matured when this deployment first synced can never appear in it, and its PT would resolve to no market, no underlying and no decimals forever — a position holding it books an unpriced close. The catch-up reads the same API's paginated ?is_expired=true catalogue, keeps only markets whose PT the portfolio has actually recorded (the four indexed held-by-anyone probes of migration 059 — a direct pendle leg, or a PT in any venue's accounting/asset slot, on the snapshot spine and the flow ledger), and puts the survivors through the identical on-chain verification and insert path. It is a catch-up, not a mirror: the whole expired catalogue is deliberately not admitted, so the registry stays the set of markets creddit can be asked to value. Insert status is always active; the deterministic flip below it, running in the same pass off the on-chain expiry(), remains the single owner of the status column (is_expired=true is Pendle's inactive flag rather than "expiry has passed", so an inactive-but-unexpired market is deliberately admitted as active and keeps reporting its live fixed rate). Then an accounting-asset reconcile: underlying_address must hold the unit getPtToAssetRate is quoted in — the SY's assetInfo() asset, not its yieldToken(), which differ on most markets by the SY exchange rate — so every registry row's assetInfo() is re-read each pass and any row that disagrees with the chain is corrected. That is both the migration for rows written before the distinction was drawn and the standing guard that keeps the column true; an unreadable read leaves the row alone rather than nulling a unit. A correction is applied only when it leaves the market in the same book it resolves into today, because this column is what a PT leg's book resolves through as well as what its mark is priced in: a right unit that no registry can book would take the market out of the yield book (and, with no price bar of its own, out of valuation too) in one pass, with nothing said. Such a row keeps its stored unit and is named on a standing operator line every tick instead, and the correction applies by itself once the unit is registered. The book is resolved through the runtime token registry first and the static seed only as a fallback — the same chain the app books a leg with, so the check cannot pass something production would fail. Then one multicall snapshot per run pinned to a single block: readState().lastLnImpliedRate → implied_apy, PendlePYLpOracle 900s TWAP pt_to_asset_rate/pt_to_sy_rate (the oracle reverts rather than serve a degraded window; reverted reads store NULL), SY.exchangeRate(), YT.pyIndexStored(), pool sizes. The last two are the redemption-index pair (migration 101): min(1, sy/py) is what one PT actually settles for once the asset behind it has been written down, and it is stored on every row — recorded in both regimes, applied in neither by this pass — because a write-down is only visible as a STEP between two moments, which no read of "now" can reconstruct. liquidity_usd is a display aid from the same API response — the RPC snapshot never depends on the API being up; underlying_apy is backfill-only (the active-list endpoint does not serve it), the realised figure derives from sy_exchange_rate. One-shot history: scripts/backfill-pendle-history.ts (per-market daily series from the API, basis='pendle_api'; --market / --since YYYY-MM-DD / --dry-run / --no-factors). That script also fills the redemption-index pair on the rows it visits, which the API does not serve: it resolves the last block at or before each daily point and reads the two legs there, one block lookup plus two archive calls per row — which is why it is run per market on a market a tracked wallet holds, never in bulk. A caught-up matured market has no snapshot rows and cannot get any — the loop above visits only status='active' markets, so the market whose gap the catch-up exists to close is precisely the one the 6h pass never marks. The v2 history endpoint does serve an expired market in full, so that one-shot backfill is its only source of history. It is no longer a step anyone has to remember: the pass now runs that per-market backfill itself, on every market either discovery arm inserted in the tick, after the snapshot loop. That closes a gap that was invisible until somebody reported a dash — a principal token records the yield it locked from the redemption factor at its own purchase moment, and both arms only begin recording that factor from the tick that found the market, so a token bought between a market's listing and its first visit here (and, for a caught-up matured market, bought at any point in its life) had nothing to read and the whole position's locked-in rate and accrued yield were withheld for as long as it stayed open. The cost is bounded and stated: at most two markets a tick — roughly 2,500 chain reads and five minutes in the worst case, and nothing at all in the normal one. Markets found in the tick come first, then a standing drain of markets that still hold no history from the hosted feed, so a deferred or failed fill is retried without an operator. A newly listed market is hours old and costs a handful of reads; a matured catch-up admission can be a full market-year and still runs inline, because it is the market whose factor series would otherwise never exist. Two things keep the drain from becoming the bulk pass the runbook forbids. It is narrowed, in the query itself, to principal tokens a tracked wallet holds — through the same indexed held-token registry the portfolio's own valuation universe is decided from, so the two can never disagree about which markets matter. And a drain candidate's history is read only back to the start of the accounting window (2026-01-01): no holding can predate it, and the factor is only ever read at or before a purchase, so everything earlier is work no screen or statement could consult. The drain also rotates rather than always starting at the oldest market, because a market can legitimately finish with nothing new written — its daily points land on the same timestamps the 6h pass already recorded — and would otherwise hold the front of the queue for good. Anything past the cap is named on a log line with the manual command, a backfill failure never costs the tick its snapshot or its exit code, and the fill is idempotent, so a retry only ever adds. The manual --market invocation stays the tool for markets discovered before this wiring. It is display history for the term surfaces, not the portfolio mark: a PT position is valued from the registry row plus an archive getPtToAssetRate read at the position's own block (the redemption index at/after maturity, never par), which is why the registry row alone is what unblocks a previously unpriced PT leg.

refresh-portfolio.ts (the 6h portfolio job, WS4)

Two jobs run per tick (entrypoint scripts/refresh-portfolio.ts, logic scripts/refreshers/portfolio.ts, shared read/valuation modules in src/lib/portfolio/); both share ONE anchor block. (Registration backfills are NOT part of this tick: the minutely drain cron owns the WS5 queue, see below.)

1. Valuation spine -> portfolio_position_snapshots. Selects eligible wallets (accounts LEFT JOIN portfolio_backfill_state, snapshot when status IS NULL OR status NOT IN ('queued','running') -- a chat-seeded account has no state row and must not be skipped; running is excluded so the WS5 backfill and this cron never write the same wallet's window at once). Since migration 048 an accounts row alone no longer earns a snapshot: a row may be a SHADOW account, minted only so a wallet somebody TRACKS on /portfolio has a FK target for portfolio_backfill_state. An account is eligible when it is a real signed-in user (last_seen_at IS NOT NULL, stamped only by the SIWE verify path) OR some account's account_wallets list still references it. A shadow is therefore snapshotted while at least one account tracks it and drops out of the pass the moment nobody does — which is what makes un-tracking a wallet free at delete time (the link is removed, no history is pruned; re-adding requeues a gap PATCH that preserves the pre-gap history, for a gap longer than 12h; a SHORTER gap is instead covered by the widened DERIVED population, since the requeue guard deliberately no-ops there — see "The derived population is wider than the snapshot population" below). The 048 self-row seed is load-bearing here: the 042 chat-seeded accounts carry last_seen_at = NULL (only a sign-in stamps it), so without their seeded self-row they would have silently dropped out of this pass. Runs all six venue readers (readAllPositions, venue-isolated) at the anchor block, values each leg in both marks (buildSnapshotRows), and upserts keyed on the aligned 6h window. snapshot_ts is ALWAYS the aligned window; block_number is the anchor. The upsert makes a same-window re-run idempotent. Rate resolution uses mode now (the wrapper's own on-chain getter at the anchor block, same-block as the venue index); the WS5 archive backfill uses mode history, which tries the SAME on-chain getter at the historical grid block first (read via the archive RPC, so it is exact per-block and matches a coinciding live snapshot) and falls back to the block-anchored token_yield_apy.share_rate series when a wrapper has no getter. A now ERC-4626 convertToAssets(1e18) getter scales the raw read to the human assets-per-share rate by 10^(assetDecimals + 18 − shareDecimals) (= 18 for every covered vault, including the 6-dec-share syrupUSDC/syrupUSDT: 6 + 18 − 6 = 18); it is NOT the assetDecimals unless the share is 18-dec, so it agrees with the history share_rate (token-yields' default divisor 18). index_raw stores the BARE venue index; the value columns already fold in the accounting-asset->book rate, and WS6 recomposes the index for exact ratio math.

2. Flow ledger -> portfolio_flow_events_v2. Derives every wallet's receipts for the tick's own window straight from the ingested event store (raw_events) and merges them, one wallet per transaction, after the spine has committed. The window's floor is the tick's own cursor (portfolio:v2:tick) plus one; its ceiling is min(anchor − CURSOR_SAFETY_BLOCKS, the ingested tip), where the tip is the MINIMUM ledger:<stream> cursor over the rolled-out streams — the slowest stream the derivation may rely on. Deriving past that tip would read a lagging stream as EMPTY and then delete the stored rows it could not reproduce, so the tick HOLDS at the tip and says so ([portfolio] ledger lag …) rather than skipping a range. The derivation itself, the venue adapters behind it and the mark resolver are src/lib/portfolio/derive/ (below); a fault is a returned value, never a throw, counted and printed under [v2-partial], and failures or pageable partial merges exit the tick 2.

   The cursor stops at the settle line, and so does a row's basis. The cursor advances to max(range.from − 1, min(range.to, min(anchor − 64, the chain's finalised head))): the band this tick derived above finality stays ABOVE the cursor and is re-derived by the next tick, whose floor is cursor + 1. The SAME line is handed to the derivation, so a row is provisional exactly when it sits above that cursor (issue #777) — a label the Activity feed serves as its Confirming tag, and one that would otherwise stick to rows nothing re-derives. An unreadable finality line settles nothing and moves no cursor: the next tick re-derives the whole window.

   Two registries ride on the merge's own rows. portfolio_held_pts (so loadPendleMarkets keeps a matured PT in the universe) and portfolio_wallet_index (so the discovery intersection below keeps bounding the reader) are fed from the legs the merge actually inserted, which is the only place a market or a principal token the wallet moved through — but never held at a snapshot instant — can be seen. The memberships are persisted BEFORE the cursor advances, so a failed upsert holds the window open for the next tick.

   The old chain-scanning flow pass, its two-tier provisional sweep, its Fluid decoded-event cache and the mode switch between the two were deleted with the old engine; the relations they wrote were dropped by migration 095 (see the retirement runbook).

Writer serialisation (the advisory lock): with backfills running on their own minutely schedule, a backfill can be mid-flight DURING this tick. The eligible-wallet exclusion covers the common case, but two residual races remain: an account (chat-seeded, no state row) selected as eligible at the start of a tick can be enqueued+claimed mid-tick, and a JIT page-load can persist ledger rows mid-backfill. All writers of the two history tables therefore take ONE transaction-scoped advisory lock (src/lib/portfolio/write-lock.ts, pg_advisory_xact_lock(hashtextextended('onchain_credit.portfolio_history_writers', 0))): the tick's writer and the backfill's windowed delete+insert take it blocking (every lock-holding transaction is pure DB work — reads and valuation happen before BEGIN — so worst-case blocking is sub-second), and the JIT ledger persist takes the TRY variant and SKIPS when busy (a page load never queues; the next load re-derives the same window and the 6h tick catches up regardless). Without the lock, an upsert landing between a backfill's DELETE and INSERT aborts that backfill on a PK collision.

One commit per tick. The tick takes that lock ONCE, for its snapshot rows. It used to commit them separately — the snapshot pass first, then one transaction per flow scope as each chain scan finished — which left a window, as wide as the whole scanning pass, in which a reader saw the tick's new position values with none of the receipts that explain them: a deposit inside that window read as yield until the receipts landed, on every tracked wallet, four times a day. The tick therefore stages its rows in memory and issues them in one transaction after the last read and the last valuation of the pass, so the two halves of an interval become visible at the same instant. Nothing is held longer: the work under the lock is the same set of inserts and deletes it always was, and no network call happens inside it. Cursors advance only after that commit, so a failure anywhere in the pass leaves every cursor where it was and the next tick re-derives the same range (the merge restates a range rather than appending to it, so a re-run writes the same rows on the same keys). The just-in-time page-load persist is deliberately unchanged: it still skips rather than queueing, and a skipped range is recorded so the next tick absorbs it. What a repeatedly failing tick now costs. Because nothing lands until the end, a tick that throws part-way through publishes nothing and holds every cursor, including the one that marks how far the dirty set has been read. Before, the snapshot half had already committed and that cursor with it, so a scope that failed every tick still let the spine advance. Now the dirty candidate set widens by one tick's blocks on each failed tick and every retry re-reads more wallets than the last, saturating at "every eligible wallet is dirty" (the cost the ledger-off full read already pays). Freezing is the correct direction, since a spine that advances without its receipts is exactly the phantom this commit closes, but a tick that keeps failing gets more expensive to retry and should be fixed rather than left to heal. Where a statement sits inside a long hold is a rule too, not a formatting choice. The registration replay is the one writer whose transaction stays open across work that takes minutes, so anything issued at the top of it holds its row locks for that whole span. That is free for the two history tables, because every writer of them takes the same advisory lock first and so can never be queued behind those rows. It is not free for any other table: the replay updates its own progress row (what the "building history" bar reads) once per day of history, on a separate connection and deliberately without the lock, so that a progress update can never queue. A write to that row taken early in the replay would therefore block the replay's own progress updates until each one timed out, adding tens of minutes to the global write lock and freezing the bar at its first step. So the replay issues its history deletes at the start of its transaction and its one write to the progress row at the end, immediately before the commit, where the same atomicity costs milliseconds. The acceptance suite pins that placement for all three replay writers.

The six venue readers (src/lib/portfolio/readers/*): aave, sparklend, morpho-blue, erc4626, pendle, and fluid (FWS2). The erc4626 universe is portfolioErc4626Vaults() (src/data/erc4626-universe.ts) = the Morpho/Euler curator funds (src/data/curator-vaults.ts, generated) PLUS the Fluid Liquidity Layer fTokens (src/data/fluid-ftokens.ts). The fToken set is the FULL FluidLendingFactory.allTokens() enumeration (T4 §3.5): fUSDC / fUSDT / fGHO / fUSDtb (USD book), fsUSDS (USD book, sUSDS wrapper), fWETH (ETH book), fwstETH (ETH book, wstETH wrapper) — every based underlying, not only the USD ones, now that all three PnL books chart. Deduped by address (the vault:<addr> positionKey is part of the snapshot PK, so a duplicate entry would fail the write outright). Note the two Fluid shapes split across TWO readers: a Fluid vault position is an NFT (fluid), while a plain Fluid lending deposit has no NFT and is just an ERC-4626 share token, so it reads, flows, values and books through the generic erc4626 path (mint from 0x0 = deposit, burn to 0x0 = withdraw; accounting asset = asset() = the underlying, which buckets.ts books into its USD/ETH book — and every fToken underlying is already book-mapped WITH a redemption-rate path, par or a JIT wrapper getter, so none is skipped M9). The merged universe is used at all five portfolio call sites (reader binding, JIT mini-scan, backfill probe/replay, 6h cursor scan, leg name map); the assistant's curator-funds tool reads bare CURATOR_VAULTS, so Fluid never appears there. Every reader is DB-free (its universe is injected) EXCEPT fluid, which needs no universe at all: FluidVaultResolver.positionsNftIdOfUser(wallet) lists a wallet's NFT ids and positionByNftId(id) returns ONE self-describing 109-word payload (its vault, type, both token pairs, exchange prices, accrued supply/borrow/dustBorrow), decoded with the published resolver ABI (fluid-abi.ts, the SAME tuple the capacity refresher reuses). Per NFT the reader emits: a NORMAL leg per side (index = the vault exchange price, M14, qty = normalAmount x 1e12 / exPrice, debt is the NET borrow and EXCLUDES dustBorrow, the tick padding the vault extinguishes without payment; see M14's The tick padding in metrics.md for why, and for the one consumer that keeps borrow + dustBorrow: the health / liquidation-distance readout, which quotes the ratio the vault liquidates on); or, for a SMART leg (T2/T4 col, T3/T4 debt), TWO reads per side (one per pool token, qty = shares x tokenPerShare / 1e18 from one getDexState per DEX per anchor, read at the block being valued because pool composition drifts, index = null -> accrual value). It branches on isSmartCol/isSmartDebt from the payload, NEVER on the exchange price (smart => exPrice == 1e12 is one-directional). Closed positions and any read whose decimals / getDexState fail are skipped (M9); a position holding nothing but tick padding reads CLOSED. Fluid legs chart: FWS2 held every one "Outside the yield book" behind an interim gate until FWS3 landed the flow scanner, and the M1/M14 inclusion rules (same-book / cross-book, no e-mode gate) now decide.

Discovery intersection (T4 §3.6, src/lib/portfolio/discovery.ts). The Morpho and ERC-4626 universes are now EXHAUSTIVE (every mainnet market via morpho-universe.ts; the full fToken + curator + managed vault set, PLUS the exhaustive MetaMorpho factory universe via metamorpho-factory.ts — T5 §3.5 — assembled into reg.erc4626Universe by loadRegistries), so reading position()/balanceOf() over the whole universe every tick would grow without bound. Instead the cron INTERSECTS those two universes with portfolio_wallet_index (restrictToDiscovered, reusing the restrictRegistries/open-set override shape): the READER gets only the markets/vaults these wallets have touched, while the FULL exhaustive reg still feeds the flow scan (so it keeps discovering — once the registry is exhaustive, scanMorphoFlows's byId admits every market). The small universes (Aave/Spark, Pendle, Fluid) stay full-sweep. Discovery is a FREE byproduct of the merge: the rows it actually inserted become index rows (no extra read). A new wallet is ENROLLED by the /10min discovery drain (refresh-portfolio-discovery.ts → runEnrollmentDiscovery): a one-time FULL-universe strict current read (readDiscoveryPositionsOrThrow) finds exactly its current holdings and sets a watermark (the current read is COMPLETE for discovery — an event scan bounded to a coverage floor could miss a still-held position acquired before it, which the watermark would then mark scanned and the loader silently drop). Migration 050 seeds the index AND the watermark from existing snapshot/flow history. A wallet is SCANNED iff it has a watermark, not merely an index row: the incremental byproduct only adds the current window's markets, so an un-watermarked wallet (however many byproduct rows it has) is still read full-universe. Fail-open (M9-class): if the index is unreadable (pre-migration) OR any wallet in the batch is un-scanned, the read falls back to the FULL universe — a not-yet-discovered wallet is slow-but-correct, never bounded to empty; a scanned wallet that opens a brand-new position is picked up by the next tick's byproduct ("found a tick late", never mischarted). The bound is measured on read COUNT: a wallet's read scales with its touched markets (verified: an obscure non-curated WBTC/USDC position bounded a full universe to the wallet's 5 markets and still charted), so cron wall time stays flat as the universe grows. The watermark is never derived from a partial read. A watermark means "the index is COMPLETE for this wallet" and is what authorises BOUNDING every future read, so it is baked state — it must not come from readAllPositions, whose contract is to SWALLOW a venue's failure. Both enrollment and the weekly reconcile therefore read through readDiscoveryPositionsOrThrow (readAllPositionsSettled, throwing if ANY venue failed) BEFORE any index or watermark write: an incomplete read leaves no trace, the wallet stays un-scanned, keeps being read full-universe (the fail-open), and is retried next run. They also assertRegistriesComplete before reading at all, because the strict read closes only VENUE failures and a REGISTRY-level degradation is invisible to it: loadRegistries catches a loadFactoryVaults failure and silently falls back to the static erc4626 set (measured: 73 → 72 vaults, losing exactly the factory rows), so every venue then "succeeds" against a shrunken universe, a wallet's factory-vault membership is never seen, and the watermark is baked anyway — the same silent-bounding class, entered one level up. The reconcile is the worse of the two: it would report ZERO drift for a membership it never looked for AND bump the LRU stamp, rotating that still-blind wallet to the back of the weekly queue (weeks, at PORTFOLIO_RECONCILE_MAX=50). Both abort instead, write nothing, and exit non-zero so the cron-failure alert pages. This is why the taxonomy release step orders the migrations before these crons. Otherwise a single Morpho RPC hiccup during enrollment would index a wallet with none of its Morpho memberships, stamp the watermark anyway, and silently drop those legs from every future bounded read — and the reconcile, the designated backstop for exactly that, would have read an empty "found" set, reported zero drift, bumped the watermark and LRU-rotated the still-blind wallet to the back of the queue. It cannot detect drift in a venue it failed to read, so it must not claim to have checked it. Reconciliation safety net (T5 §3.6, scripts/refreshers/portfolio-reconcile.ts, weekly): because the index BOUNDS a read, a discovery bug that missed a membership would silently drop a leg — so a low-cadence sweep reads each SCANNED wallet against the COMPLETE universe, adds any held membership the index missed, and raises a WS8 drift alert. A discovery bug therefore degrades to "found within a week + alert", never "silently wrong PnL" (verified: a wallet whose index was seeded MISSING a real MetaMorpho holding had it detected, added, and alerted).

Completeness certificates (coverage hardening). The paragraph above describes the design as shipped in T4/T5, where the "scanned" test was the PRESENCE of a chain_scan_cursors scope row. That is no longer true, and the reason is the 2026-07-21 coverage incident: migration 056 gave portfolio_wallet_index.wallet an ON DELETE CASCADE FK to accounts, so deleting an account purged a wallet's memberships while its scope row survived (no FK on a text scope). The wallet was re-added, the next tick saw "certified complete" over an EMPTY index, bounded the read to nothing, and a value-accrual leg's whole principal booked as a permanent phantom loss. Three changes close it, and the hardening plan has the full causal chain.

1. The certificate moved onto the accounts row (discovery_scanned_block / discovery_scanned_at, migration 061) — the FK parent whose deletion cascades the memberships away, so certificate and memberships now die together and the poisoned state is not representable. See Discovery certificate lifecycle. The old scope rows are retired: the one-release dual-write and the pre-061 read fallback are gone, and migration 106 deleted the rows.
2. Freshness, with demotion. A wallet is SCANNED iff its certificate exists AND is at least as new as accounts.created_at AND is younger than STALE_CERTIFICATE_MAX_SECONDS (18h = 3 ticks). The staleness horizon is the load-bearing half: comparing against created_at alone can never demote a FROZEN certificate, because created_at does not move. Anything else demotes to the full-universe read, and the /10min drain re-certifies. Staleness is a performance state, never a correctness state.
3. Earned advancement, per scope. The 6h tick now persists each scan scope's memberships INSIDE that scope, ordered scan → stage the scope's flows → upsertWalletIndex → advance that scope's cursor, so an upsert failure holds that scope's cursor back and the next tick re-scans (flow writes are idempotent on the PK). The cursor advance itself waits for the tick's single commit (above), so a cursor can never sit ahead of rows that were never written. The previous single best-effort upsert at the end of the run lost every scope's memberships whenever a LATER scope threw, while the certificate stayed intact — the incident's state reached through a different door. After all scopes complete, certificates advance for the tick's wallets only if the enumerated veto signals are clean: a degraded registry load (which silently SHRINKS the target universe while every venue reports success) and a failed membership upsert. "No exception was thrown" is explicitly not sufficient.

A re-add PATCHES the gap; it no longer destroys the history (Phase 7a). Until now a re-add was a destructive full delete plus an open-set replay, which erased not only positions that closed inside the gap but positions that closed BEFORE it while the wallet was tracked and live-captured. Now, when a wallet with a durable coverage anchor is requeued: the pre-gap history stays exactly as recorded, the gap is reconstructed as if tracking never stopped, and the positions reconstructed through the gap are those OPEN AT GAP START plus those TOUCHED DURING THE GAP — including ones that closed inside it, not "open now". First-time adds keep today's behaviour.

Five details carry the design:

1. The trigger is the durable anchor (portfolio_backfill_state.covered_through_*, migration 063), written only on a terminal done (or a committed gap-patch segment) and advanced by the 6h tick for every un-vetoed SNAPSHOTTED wallet — including one demoted to the full-universe read, whose snapshots are complete even when its discovery certificate is stale (advancing only the discovery-certified subset would FREEZE a demoted wallet's anchor while live 6h rows kept landing above it, and a later patch would re-lay that span at daily granularity over them). The trigger reads the anchor alone, never the live status: a re-add requeues done→queued and the drain claims queued→running before the child runs, so a status = 'done' gate would always fall through to the destructive fresh replay, and an out-of-budget resume would replay over its own committed segments. The anchor's lifecycle carries the provenance instead — it is CLEARED on empty (the prune deletes the history it certified) and kept intact on error and across the requeue/claim, so a crashed or resumed patch resumes from the durable anchor rather than fresh-replaying. "Stored snapshots exist" is deliberately NOT the trigger: a crashed first-time replay also leaves rows behind but no anchor, so it correctly fresh-replays.
2. The anchor stores the BLOCK, not just the timestamp. A live snapshot keys an aligned ts (18:00) but READS at the tick's real head block (~18:50 in the incident). Deriving the sweep's lower bound with blockByTimestamp(ts) would land below the block the tip was actually read at, so the sweep would re-detect flows already baked into the tip's balances and book a phantom move at the seam.
3. A separate grid. computeGapWindow places every point strictly ABOVE the anchor (first UTC midnight after the tip, then daily, then the 6h seam). Reusing computeBackfillWindow would day-floor the start, and three of the four tick phases are non-midnight, so grid[0] would collide with a preserved row on the snapshot PK — a deterministic crash for nearly every patch.
4. The group set is re-read from chain at the tip block, not taken from the stored tip, because a stored tip can itself be a partial snapshot (the incident's held 2 of 5 legs); a group the re-read finds but the stored tip lacks is logged as a partial-tip tripwire. Every group must resolve to a reader-universe entry or the patch aborts loudly — a group that cannot be read would be reconstructed as "held nothing" for the whole gap.
5. Segmented atomic writes. Each ~30-day segment commits its flows, its snapshots and the advanced anchor in ONE transaction, so a gap of any length makes durable progress, a re-run resumes a strictly smaller gap, and the fresh path's two-transaction crash hole (snapshots committed, crash before the flow write, flows then lost forever at a moved tip) does not exist here. Native-ETH flows (no Transfer logs) are the deltas between consecutive balance anchors, seeded per segment from the PREVIOUS segment's final anchor — not the loop-invariant tip — so a balance move in an early segment is booked once, not re-booked by every later segment. A segment also OWNS its (startTs, anchorTs] window for those synthetic native-ETH rows: it window-deletes them (matched by the reserved nativeEthDiffTxHash prefix) before re-writing its own, symmetric with the snapshot windowed delete and in the same transaction, so a live-captured diff row that a still-snapshot-eligible wallet booked at its 6H WINDOW ts above a FROZEN anchor (a parked-error span, whose anchor is deliberately kept but done-guarded from advancing, or the concern-1 demotion lag) is REPLACED by the patch's DAILY-grid row rather than persisting as a second synthetic PK for the same move (native ETH is par, so the yield curve is shielded, but net-capital in/out and entered-basis P&L would otherwise silently double; B5). REAL flows keep their pure upsert, since their real (tx_hash, log_index, leg) PK is idempotent under re-derivation and a real hash never carries the reserved prefix. When the patch COMPLETES it runs the same residual (nowBlock, head] tail sweep the fresh path does (full-universe, non-native, UPSERTED as ledger rows above the last snapshot seam): the patch held running for minutes while the 6h tick advanced the global cursor past that span, so a flow landing there would otherwise be missed forever.

empty also changes meaning: a wallet is pruned only when the gap-mode group set is empty AND no stored history exists, so a wallet whose positions all closed during the gap gets the patch and ends done rather than losing six months of chart for being empty on re-add day. floor_ts is unchanged (the original "tracked since" anchor is part of the preserved history), and multiple gaps compose — each re-add patches from the then-current anchor. Cross-account inheritance is deliberate: account B first-adding a wallet account A previously tracked inherits the preserved history and the patched gap; history is wallet-keyed by design, tracking is watch-only, and everything reconstructed is public chain data.

The derived population is wider than the snapshot population (Phase 5). Snapshots are written for ELIGIBLE wallets; the tick's ledger merge runs over eligible ∪ tracked ∪ snapshotted-in-48h. The hole this closes: a wallet unlinked and re-added within 12h skips the backfill requeue (correctly — the STALE_HISTORY_INTERVAL guard exists so the shared-wallet add case does not burn a 90-day archive replay), but during the gap it was not eligible, so nothing derived it either, and the cursors advanced past those blocks for everyone else. The gap's movements and memberships were therefore missed permanently by the incremental path.

This is load-bearing, not belt-and-braces. Un-tracking deletes only the account_wallets row, and trackedAccountUpsert is ON CONFLICT DO NOTHING, so accounts.created_at does not move on re-add. Across a ≤12h gap the wallet's certificate was last advanced ≤12h ago against an 18h horizon, so it is still FRESH: the tick on re-add BOUNDS that wallet's read against its index. Nothing demotes it and nothing replays it. These gap memberships being present is the only thing keeping that bounded read truthful.

The TRACKED arm joins accounts (a dangling account_wallets link whose shadow account was deleted must not enter, or the merge trips 082's pfe2_wallet_fk to accounts and aborts the tick) and excludes queued/running — without that exclusion its only marginal contribution over the eligible set would be exactly the mid-backfill wallets the eligible predicate deliberately holds out, so it would silently reverse a concurrency guarantee. The 48h arm is what actually bridges an unlink gap, since an unlinked wallet has no account_wallets row at all; migration 062 indexes (chain_id, snapshot_ts DESC) for it, because that predicate has no wallet term and the spine is retained in full. The union always contains the eligible set, so the scan can only widen, and it degrades to exactly the eligible set if the query fails.

Memberships written for a temporarily untracked wallet are harmless (FK-cascaded with the account, ignored while ineligible) and its certificate is untouched (only the eligible-and-fresh set advances). Known limit: native-ETH flows are derived from snapshot balance diffs rather than scanned, so they cannot widen — a native-ETH move inside an unlink gap is never booked (the ≤12h re-add skips the replay that would recover it). Native ETH is 'none' accrual so no yield is fabricated, but the wallet keeps a permanent yield-invariant residual for that amount.

The registration backfill now certifies what it learned. It is the one process that provably walks a wallet's whole history, and it used to write nothing to the index and never touch the certificate — which is why the incident survived a complete, correct re-derivation of the wallet's history. On a terminal done (or empty; never error) it derives memberships from the UNION of the probe's strict full-universe current read, every snapshot row the replay wrote, and every flow row the run wrote including the full-universe tail sweep (a position opened mid-replay appears only there), upserts them, and then stamps the certificate at the tail sweep's head block. Bake THEN certify: a crash between the two leaves the wallet un-certified, which is the safe side.

Divergence is detected by machine, not by reading charts (Phase 6). Two tripwires, deliberately on different paths. The weekly reconcile is the SLOW one: it reads each certified wallet against the complete universe, repairs what the index missed, prints one [fail] DIVERGENCE … line per membership and ends the run non-zero. Its honest acceptance is "within one reconcile CYCLE for its cohort", not "within a week" unconditionally, because the batch is capped and LRU-ordered; the dangling-link check is the exception, being a single un-capped query per run. The 6h tick carries the FAST one: after committing both its snapshots and its flows it reports any leg that was present at a wallet's previous snapshot, is absent now, and has no flow explaining it. Running it after the flow writes is what keeps an ordinary close silent. Together with M21's read-path coverageAnomalies that is same-tick detection on the write path, per-request detection on the read path, and a weekly full-universe backstop.

Staging. scrub-staging-pii.sql deletes the per-wallet cursor rows alongside the user-data TRUNCATE, and reseed-staging.sh fails closed if any survive. Two prefixes are deleted and watched: portfolio:derive:v2:%, the flow ledger's per-wallet re-derivation resume point (082), and portfolio:discover:%, the retired discovery watermark. Nothing reads or writes the second any more and migration 106 deletes its rows, but a prod dump taken before 106 ran on prod still carries prod's enrolled-wallet list under it, so it stays in the scrub, the leak check and the user wipe until a reseed from a post-106 dump has run (see release steps). Without the first, the nightly reseed would manufacture the incident's poisoned state on staging for every wallet in the prod dump: a surviving cursor certifies work against a table the TRUNCATE has just emptied. It is a wipe-on-reset family, which is an invariant on what may be STORED under it and not merely a note about the scrub. It holds per-wallet state only, so a programme-global record parked under it is destroyed by one nightly reseed or one portfolio-user wipe, and the destruction reads as success because the survivor count goes to zero. Campaign-global values live under portfolio:v2-campaign:%, which nothing in the scrub, the reseed's leak check or scripts/ops/reset-portfolio-users.ts deletes. scripts/ops/scrub-staging.test.ts extracts the prefix set from all three statements and checks this page and database.md name every prefix in it, so a third prefix cannot land in the code and leave a prose copy behind.

Fluid event decoding (FWS3, src/lib/portfolio/fluid-events.ts). Measured chain-wide volume: ~142 LogOperate/day, ~1 LogLiquidate/day, ~59 factory ERC-721 Transfer/day. LogOperate and LogLiquidate carry ZERO indexed params (user/liquidator is msg.sender or a DSA, never an owner), so neither can be wallet- or vault-filtered at the node; the factory ERC-721 Transfer stream CAN be, and it is what the registration probe sweeps to find a wallet's first Fluid activity, because a position opened before the window and closed inside it leaves no other wallet-topic trace. What survives here is the three pure decoders and that scan. Everything else moved to the rebuilt derivation (derive/fluid.ts, derive/fluid-state.ts): the leg semantics (a signed colAmt/debtAmt becoming deposit/withdraw or borrow/repay; a SMART leg's 1e18 DEX shares decomposed into their pool tokens at the flow block), the M16 hand-over rules, the M17 opening-transfer suppression and the M15 state-diff liquidation detector. The fluid_event_log / fluid_event_coverage cache those paths read is gone: the ingested event store is the cache now, one copy for every consumer, so there is no deploy seed, no coverage start to keep below a moving history floor, and no cache tip for a registration backfill to read past. dRPC pacing: a chain-wide (no-address) getLogs is expensive server-side even for few results, so the ingester's own scan uses ≤1000-block chunks; the dRPC free tier returns HTTP 408 on a wide range, retried alongside 429/5xx (rpcRequest), and getLogsChunked halves the span + retries on any residual timeout.

JIT / live path (src/lib/portfolio/live.ts, triggered by POST /api/portfolio/refresh AFTER first paint; the GET routes serve STORED rows only, so a page load never blocks on RPC and no unpersisted reading is ever merged into a response): a signed-in wallet's current legs read across all readers at anchor = min(head, the ingested tip), the same block this refresh's ledger merge is clamped to rather than the chain head above it, so the reading and the receipts that explain it are a statement about ONE block; plus a merge of that wallet's receipts into the rebuilt ledger. A complete reading is then STORED as the wallet's live tip (writeLiveReading in snapshot-write.ts, the checkpoint's own writer) and served from the database like any other spine row, which is what puts a position opened minutes ago on the page dated by the block it was read at. Two conditions gate that write, and both are about never storing a view nothing else agrees with: no venue read failed (a partial reading restates a wallet as having lost whatever could not be read), and this refresh's ledger merge committed and claimed its range, the same verdict that lets a tick advance its cursor, so a merge that was skipped for a busy writer, that failed, or that landed partial leaves the previous reading standing and the route answers refreshed:false with the reason. The tip is off the six-hourly grid by construction (its snapshot_ts is the anchor block's own timestamp, epoch % 21600 != 0, one predicate owning the rule in live-tip.ts) and there is at most one per wallet: writing a new one retires the older one in the same transaction, block-ordered so a concurrent newer sync cannot be overwritten by a slower older one, and the next 6h checkpoint retires it alongside its own insert. When the ingested tip has not passed the wallet's last reading there is nothing fresher to read, and the refresh says so (nothing-newer) instead of paying for a duplicate. Rate-limited to one live read per wallet per five minutes (LIVE_REFRESH_INTERVAL_MS, env override PORTFOLIO_LIVE_COOLDOWN_MS), server-side, and the same window for a page load and for the Synchronize button, which forces a FULL read past the recompose fast path but never past the window; concurrent cold-cache calls for the same wallet COALESCE onto one in-flight computation (coalesce), so a page-reload storm cannot fan out the per-wallet multicall. Fluid's positionsNftIdOfUser is one ~30k-gas call, so the JIT path needs no NFT cache. The merge's floor is the 6h TICK's own cursor, not the snapshot spine's. The wallet's last snapshot block sits ABOVE what the rebuilt ledger is derived to (the tick's cursor stops at the settle line while the snapshot is written at the anchor), so a page load deriving from it would skip the band in between — the band nothing else re-derives between two ticks. It derives from min(the wallet's last snapshot block + 1, the tick cursor + 1), which can only ever WIDEN the window; the writer clamps the top to how far ingestion has finished across every stream it relies on, and it never stamps the tick cursor (that cursor certifies a whole population). A refresh that cannot read the tick cursor (a read that fails, as opposed to a box with no cursor yet) still merges from the wallet's last snapshot block + 1, but stores no reading (merge-partial): after a tick that floor sits at least 64 blocks above the cursor, nothing records the stretch between, and a stored reading counts toward the wallet's derived-through as if its merge had started at the cursor. Once the worker owns derivation, such a refresh queues no sync job from that floor either (merge-failed), since a done sync counts the same way (PR #959 review round 5, SF-1). The settle line is min(anchor − 64, the chain's own finalized head), one eth_getBlockByNumber("finalized") per refresh (12s cache) rather than the block margin alone: head − finalized measured 64 to 93 blocks, so anchor − 64 sat above finality most of the time. That line is handed to the derivation, so rows above it are written provisional — the tier a later pass re-derives — and if the finalized head cannot be read at all, NOTHING is settled that pass. Only settled rows feed portfolio_held_pts (a registry nothing removes a PT from, so a row that may still vanish must not widen the PT universe), judged on the BLOCK against the line this refresh just read. A skipped merge (the writer lock is busy) records a jit-pending marker at the lowest unpersisted block, which the next 6h tick absorbs and clears. Freshness is stamped at COMPLETION, not at start: the pipeline routinely takes tens of seconds, and a start-stamp would spend that duration out of the cooldown window (under the 60s window this replaced, a 45s run landed with 15s of servable life and a >60s run was DEAD ON ARRIVAL). That stamp paces the NEXT read; what dates the page is the reading's own block time, served as readAt and rendered as the freshness stamp, so a refresh that took forty seconds never claims the data is forty seconds younger than the block it came from.

Quoted rates — earned vs advertised (FWS4, quoted-rate-tables.loadQuotedRates, read by GET /api/portfolio/positions): the advertised comparator per current leg, feeding the dashboard's "24h APY" column. Each venue-rate loader prefers the de-noised *_24h trailing column and falls back per row to the latest 6h-window value where the 24h figure is not yet populated (a pool younger than 24h). Aave/Spark read the latest {aave,sparklend}_reserve_apy (supply_apy_24h/borrow_apy_24h, 6h fallback); Morpho reads morpho_market_apy (same 24h columns); ERC-4626 and wrapper terms read token_yield_apy (already a trailing-24h rate, unchanged). Fluid reads fluid_ll_apy (latest per token_address, the ETH pseudo aliased to WETH; supply_apy_24h/borrow_apy_24h with a 6h fallback) for a normal leg's LL rate, and fluid_dex_apy (fee_apy_usd_24h per pool_address, falling back to the 6h fee_apy_usd until the pool has 3 prior rows) for a smart leg's pool fee, marked APPROXIMATE. The fee's SIGN follows the leg side (carries-table.ts smartColApy/smartDebtApy): a smart-collateral leg EARNS the fee (fee + LL supply + wrapper); a smart-debt leg supplies debt-side liquidity and ALSO earns it, so the fee REDUCES its funding cost (LL borrow + wrapper − fee). Because a smart-leg key carries no pool, loadQuotedRates resolves each held vault's per-side DEX pool on-chain ONCE via readFluidVaultDexes (getVaultEntireData.constantVariables.supply/.borrow, the Liquidity Layer mapped to null); best-effort. The pool fee is the DEFINING component of a smart leg, so an unresolved fee — an untracked/off-list pool OR a failed readFluidVaultDexes read — nulls the WHOLE smart-leg quoted rate to a dash (M9), never a fee-less number. No new cron or table.

RPC routing: current-state reads use ETHEREUM_RPC_URL (publicnode default); the flow scanner's eth_getLogs and all block-pinned reads use ETHEREUM_ARCHIVE_RPC_URL (dRPC default), because publicnode rejects archive eth_getLogs over wide ranges. getLogsChunked gained an optional rpcUrl for this. The FWS4 quoted-rate getVaultEntireData reads run at latest on publicnode (the DEX addresses are immutable per vault).

Crontab (manual server step, listed in the deployment runbook and the PR body): 50 */6 * * * /opt/onchain-credit/scripts/run-cron.sh refresh-portfolio.ts (one minute after refresh-lending-positions.ts (45), reusing the same reserve registry that job maintains) and * * * * * /opt/onchain-credit/scripts/run-cron.sh drain-portfolio-backfills.ts (the WS5 queue drain; run-cron's per-script flock keeps invocations from stacking when a backfill outlives its minute).

Approving a completed historical event import. A completed sweep is accepted only on four conditions: its row count against an expected size, unbroken scan receipts across the whole range, a re-scan of a sampled window against a second provider, and a named on-chain anchor together with a certificate for every address the stream declares. All four have to pass before the history is marked complete; the fourth refuses outright if it would have nothing to evaluate.

Where the expected size comes from, and where it may never come from. The expected row count is either the measured logs-per-block density recorded for that venue in the programme's data contract, compared with a 70%–200% allowance, or a full enumeration of the range read back from the chain through a provider other than the one that collected the sweep, compared exactly. It is never a count of the rows already imported: a size taken from the import it is meant to judge passes every import, complete or not, and two production sweeps that had missed their expected size were accepted that way. record-ledger-backfill-band.ts performs the independent enumeration and records it as production data, so a newly measured range does not need an application release; each recorded number carries the derivation that produced it, and one recorded without an independent derivation is refused by name rather than quietly ignored. Recording a size approves nothing on its own.

The wallet-movement stream is sized differently, and only in one direction. Its volume follows the tracked-wallet set rather than chain activity, but wallet activity is extremely uneven: across the wallets enrolled in production one contributes over twelve hundred movements and another contributes none. So the enrolled wallet count widens the upper bound and never the lower one. The lower bound is the enumerated size of the history already imported, held fixed. That is safe because enrolment only ever adds wallets, and the one process that removes imported rows — the reorg repair, which clears a block whose stored identity the chain no longer serves — re-imports the range it cleared, so a shortfall between the two fails this check rather than passing it. The honest cost of holding the lower bound fixed: it is already satisfied by the wallets enrolled when it was measured, so a wallet enrolled later whose import collected nothing still clears this one condition. That is unavoidable given how uneven the distribution is (no positive per-wallet floor can be true of a wallet with no movements at all), and the other three conditions still apply to it.

Concurrency: run-cron.sh takes a per-script flock, keyed by checkout ($LOG_DIR/$(basename "$DIR")-<script>.lock), so if a portfolio tick overruns its 6h slot on a slow archive RPC the next tick logs "already running ... skipping this tick" and exits 0 rather than stacking a second scan. The lock is per-checkout, so the prod and staging working copies on the shared box never serialize against each other. See the run-cron.sh section below.

Staging user data (reseed + scrub): the nightly reseed (scripts/ops/reseed-staging.sh) DROPs and fully restores the staging DB from a prod dump, with no table-exclusion mechanism, so a wallet's real positions would otherwise ride into staging. scripts/ops/scrub-staging-pii.sql therefore TRUNCATEs the whole user-data graph right after the restore: accounts, account_wallets, the chat tables (038/039/040), and every per-account portfolio relation (portfolio_position_snapshots, the flow ledger portfolio_flow_events_v2 since 082, portfolio_wallet_index since 050, portfolio_backfill_state), plus the repartition aside tables described below; the reseed's fail-closed leak check RAISEs if any watched table still holds rows. Read the file for the membership, not a count in this sentence: the set has grown with 048, 050, 067/072 and 082 and shrank with 095, and scripts/ops/scrub-staging.test.ts asserts the scrub covers everything the leak check watches, so the two move together whatever the number is. account_wallets (048) is in the list because the account↔tracked-wallet mapping is private user data (the positions themselves are public chain data; the fact that an account watches them is not). They all go in ONE TRUNCATE, because accounts is FK-referenced by account_wallets, portfolio_backfill_state and (since 056) the chat tables, portfolio_wallet_index, portfolio_position_snapshots and portfolio_flow_events_v2, and Postgres refuses to truncate an FK-referenced table unless every referrer is in the same statement, so the list is built dynamically from whichever tables exist (to_regclass), tolerating a dump that predates any of 038 / 042 / 043 / 048 / 050 / 082. Adding 048 to this scrub is not optional hygiene: without it the nightly scrub starts failing outright (cannot truncate a table referenced in a foreign key constraint). To give staging something to render, scripts/ops/seed-portfolio-fixtures.ts re-registers 2-3 public whale wallets as accounts; it must be re-run after each nightly reseed (or touch /root/.reseed-paused during a multi-day validation window so the reseed is skipped and the fixtures persist). The same trap reappears with the DESTRUCTIVE snapshot repartition (067): it RENAMEs the live table to portfolio_position_snapshots_preswap and keeps it for a manual DROP, and a rename carries the 056 FK to accounts along with a full copy of the user data — so an aside table still standing at the 02:15 backup rides into the 03:00 reseed and breaks the same TRUNCATE, aborting the script before every leak check below it. portfolio_position_snapshots_preswap is therefore in both the scrub list and the leak-check list (to_regclass-guarded, so a no-op whenever it does not exist), portfolio_flow_events_preswap was pinned beside it pre-emptively and migration 072 (the flow ledger's hash repartition) then produced exactly that table, so the fix was already in place rather than rediscovered by a failed reseed — since 095 dropped both that copy and the ledger it copied it stays only as a standing guard for the shape, because a repartition of the live ledger would rename aside to portfolio_flow_events_v2_preswap, which means portfolio_position_snapshots_preswap is the aside entry that can still fire — scripts/ops/scrub-staging.test.ts keeps the two lists in lockstep, and the deployment runbook additionally pauses the reseed across the verification window. Migration 082 is the third occurrence of the same class, closed in the PR that created it: the flow ledger carries the identical 056-style FK to accounts, so it is named in the single TRUNCATE (the partitioned parent only, since truncating it empties all 16 partitions) and watched by the leak check. scripts/ops/reset-portfolio-users.ts carries the same statement against the same tables, so any change to one of the two files must be made in both.

Alerting (WS8): several fail-soft Telegram paths, sharing one bot (env ALERT_TG_BOT_TOKEN + ALERT_TG_CHAT_ID; both unset = silent no-op, so a dev box or an un-provisioned env never posts). The shared, unit-tested formatters + fail-soft sender live in scripts/ops/alert.ts. (1) A cron-failure alert lives in scripts/run-cron.sh (see its section below): on a non-zero exit it POSTs a message naming the checkout, script, exit code and the reason — it greps this run's slice of the logfile for the failure markers the jobs already print ([fail], [partial], .../fail], fatals) and includes them, so an alert is self-explanatory instead of costing an SSH round trip. This covers ALL cron jobs, not just the portfolio one. A job that wants to page therefore only has to exit non-zero and print a matching line — which is exactly how the backfill drain's per-wallet failure alert works (it exits 2 and prints one ERROR uid=… attempt n/4 … line per failure). Two constraints bind, and a job that pages must respect both: the 220-char cut per line (so each line leads with the load-bearing facts and trails the unbounded provider message), and — the tighter one — tail -n 8 lines. The drain's per-wallet lines do NOT own those 8 slots: the backfill child's stdio is inherited into the same logfile, so its own [backfill] fatal: … matches the grep too and competes ~1:1. In a broad outage the per-wallet lines are therefore truncated to an arbitrary few, which is why the drain's summary line (printed last, so it always survives the tail) carries the aggregate count AND names the wallets that actually parked. That line has to contain a literal ERROR: to match at all: the grep's failure token is [fail] with brackets, so a line saying only "FAILED" matches nothing and never reaches Telegram. No second bot, no second code path. (2) An unmapped-asset alert fires from refresh-portfolio.ts AFTER it commits: any leg the book map sent to EXCLUDED while a wallet holds a nonzero position at/above the $1 dust floor, aggregated to one message per tick, now tagged with the venue that surfaced it (T6 extended it past the original wallet path to the exhaustive Morpho loan assets too). Legs the registry KNOWS are excluded from it: an asset with a portfolio_tokens row is by definition not an unknown asset, whatever book it resolves to. That matters for the deliberate book NULL carve-outs (a non-USD fiat like EURC; gold, XAUt; a governance token, AAVE; a dollar-denominated token with no par claim, apxUSD and the Pareto FalconX tranche AA_FalconXUSDC with its wrapper wFalconX; an accruing wrapper over one, apyUSD and sUSDat — declared in migrations 071, 073 and 074), which are valued in USD and never charted BY DESIGN and so land EXCLUDED-with-value on every tick — it would otherwise page every EURC holder every 6h with a remediation ("map it in buckets.ts") that is wrong for a token the registry already covers, and muting a real signal under known noise is the failure this alert can least afford. A token absent from the registry AND unbucketed still fires. (3) Three registry alerts fire from the weekly sync-portfolio-tokens.ts propose run: top-20 stablecoin ranking drift, an asset profile with no portfolio_tokens row, and a wallet-tracked variable_rate token that resolves no rate source. A KNOWN, decided skip is filtered upstream so it never pages — the same "known + quiet" discipline as the partial-failure floor. (4) A discovery reconciliation drift alert TYPE/formatter is defined here for T5's weekly full-universe sweep to emit when it finds a position the per-wallet discovery index missed (found a week late, paged, never silently mis-charted). (5) A coverage-anomaly WARNING (M21) is the one tripwire that is deliberately NOT on a Telegram path: the engine reports every leg transition the flow ledger cannot explain, and the READ path prints [portfolio] WARNING unexplained leg <kind> wallet=… key=…. It therefore lands in the app's pm2 log, not a cron log, and is invisible to run-cron.sh's grep (which cannot see it anyway, and whose failure tokens it does not match, so it can never trip a false cron alert). It is throttled to one line per (wallet, kind, key, mark) per UTC day, and transitions at the newest point of the series are recorded on the curve but never printed (they are dominated by two benign races: the 6h refresher's snapshot/flow split commit, and a JIT tip read whose flow persist was lock-skipped). The WRITE-path counterpart is alert path (6) below. (6) An unexplained leg disappearance alert (alertLegDisappearance) is that tripwire's WRITE-path counterpart: after the 6h tick commits its snapshots AND its flows, any leg present at a wallet's previous snapshot, absent from the one just written, and with NO flow row in the scanned window is reported and POSTed as one aggregated message. It runs after the flow writes on purpose, so an ordinary close and its withdrawal are judged together and a real exit stays silent; it fires on the tick that CAUSED the gap rather than when somebody next loads a page, and unlike the read-path WARNING it is in a cron, which is the only place a Telegram alert can originate. Fully fail-soft: it never affects a tick that has already committed. (7) The weekly reconcile now also exits non-zero on a DIVERGENCE (a membership a bounded read had been dropping, repaired) or a DANGLING LINK, each printing a [fail]-tagged line, so those pair with the cron-failure alert instead of only appearing in the drift message.

Registry sync — sync-portfolio-tokens.ts (T6)

Mirrors the sync-carries.ts --approve shape but, unlike the ad-hoc carry/curator syncs, it runs as a weekly cron because its checks power the WS8 registry alerts. The default run (the cron) is PROPOSE + ALERT and never mutates — the "never auto-mutates" rule from the taxonomy plan §3.3. Its three checks (pure logic in scripts/portfolio-tokens-diff.ts, so they unit-test without a DB/RPC/DefiLlama):

1. Top-20 stablecoin diff. Ranks the DefiLlama stablecoins list by Ethereum circulating, takes the top 20, resolves each entrant's mainnet address from the per-stablecoin detail endpoint, and diffs against portfolio_tokens by address: a new entrant with no ACTIVE registry row at that address is PROPOSED as an idle par, wallet-tracked add (a non-USD fiat peg proposes book NULL); a known accruer is never a par proposal; a source='stablecoin-top20' row no longer ranked is reported as drifted OUT for a human to retire (never auto-retired); and a ranked coin whose ADDRESS is a RETIRED row is reported as ranked but retired and never re-proposed — a retirement is a coverage decision and a ranking does not overturn it (2026-09-16), which is what the eleven rows that rule retired were retired FOR. Address, not symbol — DefiLlama symbols are not unique across pegged assets, and a symbol key silently counts the WRONG token as covered, permanently, since the registry row never leaves. Live: DefiLlama lists two "GUSD" assets — Gate USD (~$320M Ethereum circulating, 0xaf6186b3…) and Gemini Dollar (~$38M, 0x056fd409…, what 049 seeds). Both are real mainnet contracts whose symbol() returns "GUSD", so the seeded Gemini row absorbed Gate USD's top-20 slot and the $320M member could never be proposed. The map is keyed by address and admits every row, so a retired one can only ever suppress ITSELF — which is the point — while a namesake at a different contract is still proposed. An entrant whose mainnet address cannot be resolved is reported as unresolved (console only, no alert): coverage is undecidable, so it is neither counted as covered nor proposed, and it cannot drift its own symbol's row out. That is a real state — USDD's detail address is tron:TXDk8mbt…, not an EVM address at all.

2. Asset-profile completeness. Every onchain_credit.assets profile ticker must have a portfolio_tokens row, or its bare balance can never enter the wallet venue as a Variable rate asset.

3. Variable-rate rate-source resolution. Every wallet-tracked variable_rate row that has a base must resolve a redemption rate via a block-pinned rate getter on its own registry row (rate_kind = 'getter') OR token_yield_apy.share_rate; an unresolved one would be silently skipped by the wallet venue (M9). The check is scoped two ways, and both scopes are the SKIP's own rather than a preference. It skips UNTRACKED rows because an untracked vault share is valued through the erc4626 venue, not the bare-token path. And since the 2026-09-16 coverage rule it skips rows with book NULL, because such a leg never reaches the branch that skips: redemptionRateKind short-circuits EXCLUDED to unknown before a rate is asked for, and the leg is stored at quantity × its market price in dollars with a NULL redemption mark. An unresolved rate source on a no-base row is therefore not a gap — it is the row saying it has no redemption line, which is true of a bitcoin claim and of a PT's accounting asset. Without that scope the five the rule starts sweeping (eBTC, apyUSD, sUSDat, wFalconX, AA_FalconXUSDC) would page this check every week forever, which is the muted-alert failure alert.ts warns about. The ACKNOWLEDGED set is empty and has been since migration 079 untracked eBTC, its only member.

--approve <SYMBOL> applies a named new-entrant proposal: it takes the mainnet address the diff already keyed on (DefiLlama chain-prefixes some — bsc:0x8d0D… for USD1 — which a bare-0x parse rejected outright, so the prefix is stripped and the address treated as a CANDIDATE only), reads decimals()/symbol() on-chain to verify it ON MAINNET and refuses on mismatch or an absent contract (which is what makes accepting a prefixed address safe: a token that genuinely lives only on BSC fails this check), and inserts an idle par, wallet-tracked row (ON CONFLICT DO NOTHING). It refuses a known accruer, a symbol not currently a top-20 entrant, or one it cannot resolve, and it NEVER retires a drift-out row (retirement stays a manual, deliberate UPDATE/DELETE). Coverage extensions are registry rows or a flipped wallet_tracked flag from here, never a code change.

Two gaps closed after the 2026-06-25 incident. (a) The alert used to say only "script X exited 1", on the stated theory that exit-code alerting cannot see log lines — but the wrapper OWNS the logfile and can simply read it back. (b) More seriously, a partial failure alerted nothing at all: refresh-assets.ts exited non-zero only when a refresher THREW, while a refresher that catches errors per item (token-yields catches per token, writing no row for a token that fails) could lose most of its rows and still return normally — exit 0, silence, and a hole in token_yield_apy that only surfaced weeks later as a false 0% APY on the carry charts. A reported per-item tally (RefresherStats) now makes an anomalous partial a failure. Anomalous, not any: five curator vaults fail every single run today (Cannot convert 0x to a BigInt), so a "failed > 0" rule would page every 6h until muted. The floor is a failure RATE (PARTIAL_FAILURE_RATIO, 10%; today's token-yields steady state is ~6% and stays quiet), unit-tested in scripts/ops/alert.test.ts. A per-item refresher can only be wired to the rate floor if its steady state sits BELOW it: token-yields qualifies, but curator-vault-state currently fails ~40% of its 60 vaults every run (a pre-existing Morpho-V2 / Euler indexing gap), so under any single global floor it would page continuously. Until those underlying failures are fixed it deliberately stays all-or-nothing (void), rather than reporting a tally that would either spam or force the floor so high it hides real regressions elsewhere. So: token-yields reports; the chronically-degraded per-item refreshers stay void by design; genuinely all-or-nothing refreshers return void because they have nothing partial to report.

Three more refreshers report a tally. aave-v3 and sparklend count a reserve as failed when it is written without its drawable-liquidity leg, so a run that loses more than the floor's share of its reserves pages instead of exiting 0 with quietly missing liquidity. morpho reports TWO item classes in one tally — market snapshots and exposure rows — measured against the same 10% floor combined, which is the denominator to read a [partial] morpho alert against: a price outage that unprices every mark surfaces as skipped exposure rows and pages, rather than passing as a clean tick. A Morpho registry that reads fine but admits no active market reports 0 ok / 1 failed, which pages while deliberately writing nothing.

(2) An unknown-asset alert lives in the refresher itself (scripts/ops/alert.ts, alertUnknownAssets): exit-code alerting can't see a data event where the run SUCCEEDS, so after committing its snapshot writes the job POSTs directly when the bucket map (buckets.ts) sent an accounting asset to EXCLUDED while a wallet holds a nonzero MARKET value in it (a new/unmapped asset that would otherwise silently drop out of the books). One aggregated message per tick (all such assets, distinct-wallet + summed-value), so the same unknown asset does not fan out; the pure parts are unit-tested in scripts/ops/alert.test.ts. Both paths are strictly fail-soft: a failed POST is logged and swallowed and never changes the job's exit code. Job failures also remain visible in /tmp/onchain-credit-cron/onchain-credit-refresh-portfolio.log and the cron mailer. Since #811 a Pendle PT leg is named by its PAYOUT asset, never by the PT address: a PT posted as collateral used to be filtered out of this alert entirely (the per-maturity PT address churns every roll, so paging on it would be noise), which meant the one gap worth fixing — a PT whose settlement asset this product does not track — was also the one gap that never paged. The alert now resolves the leg's market to its payout asset, states that address with the ticker from pendle_markets.underlying_symbol (the only place it lives, since an untracked payout asset has no portfolio_tokens row), and admits the row without a value: such a leg has no book unit to be priced in, so its MARKET value is null however large the position is and the $1 dust floor could never let it through. A payout asset the registry already carries — including a declared exclusion such as apxUSD — stays suppressed, because the alert asks whether the decision was MADE, not whether the asset is booked.

(2a) An uncovered-asset census (uncoveredAssetCensus, same refresher) is the read-only counterpart, added with #811 (closing #703 item 3). It pages nobody and suppresses nothing: one [portfolio/census] line per tick naming every asset the tick read and did not book — declared exclusions included — with the leg and wallet counts, the venues, and the summed MARKET value where the legs carried one. PTs are counted under their payout asset for the same reason the alert names them that way, so the census does not report motion every time a market rolls. grepping the cron log answers "what was outside coverage on this date" without touching a database.

(3) A discovery-drift alert (scripts/ops/alert.ts, alertDiscoveryDrift) fires from the weekly reconciliation sweep (refresh-portfolio-reconcile.ts, T5 §3.6) when it finds a currently-held Morpho/erc4626 membership that the discovery index MISSED — i.e. a bounded 6h read had been silently dropping that leg. Unlike the unknown-asset alert there is no value floor: any missed membership is a real bug (a gap in the merge's index byproduct or the enrollment scan), and the sweep both ADDS it to portfolio_wallet_index (so the next tick reads it) and posts one aggregated message (distinct wallets + memberships, up to DISCOVERY_DRIFT_ALERT_MAX_ROWS enumerated). Steady-state fires ZERO rows; a non-empty alert means "investigate discovery.ts; the positions are now indexed". Same fail-soft POST + alert.test.ts unit coverage as the other paths.

(4) Two unmodelled rate model alerts (scripts/ops/alert.ts, alertForeignIrmMarkets + alertFluidMagnifiers) cover the case where a borrow market prices on a curve the platform has not read. Every rate-model figure published for a venue is a property of a SPECIFIC contract, and in both venues an instance can point somewhere else — on a run that SUCCEEDS, so, like the unknown-asset alert, the job POSTs directly after its writes.

• Morpho, a foreign IRM. The 90% target utilization belongs to the AdaptiveCurve IRM contract, and a market's irm is one of the five immutable parameters that ARE the market. sync-carries.ts compares the CHAIN's idToMarketParams.irm against the canonical address (the Blue API's field is veto-only, and a market whose params would not decode reports no IRM at all rather than falling back to it — a read failure must not be dressed up as a finding): a mismatch already hard-fails admission (rule H2) and now also names the market, its pair and the foreign IRM in one message. Deduped on morpho_market_registry.irm_flagged_at (migration 081), stamped straight after the alert. That marker exists rather than reusing irm_address because the two record different facts: the sync has written irm_address for every discovered candidate, hard-failed ones included, since 041, so treating "we recorded its IRM" as "we told somebody" would have silenced today's foreign-IRM markets permanently without announcing one. As written, the first run after 081 reports the standing set once and every later run is quiet.
• Fluid, a rate magnifier off neutral — on EITHER side. Fluid sets its rate curves per TOKEN at the shared Liquidity Layer, and a vault then scales what its own suppliers and borrowers get by configs.supplyRateMagnifier / configs.borrowRateMagnifier (1e2 scale, 10000 = 1x; a packed sign+magnitude overlay on a smart side, where a raw 1 means none), which Fluid's rebalancers move to route rewards. refresh-vault-capacity.ts already decodes the struct both fields live in; it persists them (vault_capacity.*_rate_magnifier, migration 080) and alerts when either side of an active vault leaves neutral. Deduped per side against the value the PREVIOUS run stored, so a vault is announced when it moves and stays quiet in between. The two sides mean different things and the message says which: the BORROW side is notice of a change, because every published Fluid funding figure is already the vault's own rate (metrics); the SUPPLY side is notice of a GAP, because a carry's target leg still reads the collateral wrapper's appreciation plus the layer's supply rate, so a vault-level supply term sits outside the published carry for as long as it runs.

Both are pure-formatted and unit-tested in scripts/ops/alert.test.ts, dedup decisions included (shouldFlagForeignIrm / newForeignIrmMarkets in scripts/morpho-rule.ts, shouldFlagFluidMagnifier / fluidMagnifierAction / alertWasReported in alert.ts), and both are strictly fail-soft.

Both dedups fail OPEN, and that is the deliberate choice. When the marker cannot be read — the migration is not applied yet, or the query simply failed — the run judges NOTHING rather than treating an empty answer as "nothing has ever been seen". An empty map would re-announce every standing finding on any transient blip, which is how an alert earns the reflex to ignore it. Each also keeps its alerting state off the maps that drive real decisions: the Morpho marker is read by its own query, never bolted onto the prior-status map that governs the admission state machine and the hysteresis floors, so a missing column can never empty that map and demote every active market to proposed.

Failing open means deferring a finding, never dropping one, and that takes two rules. The first is obvious once stated and was got wrong in review: never advance a memory you could not read. A run whose prior read failed does not write the new value either — writing it would overwrite the evidence, and the next run would compare the value with itself, so a magnifier that moved 1x → 0.7x across exactly that run would be announced on neither run and lost permanently. Skipping the write costs one run of staleness in a column nothing but the dedup reads. The second: only a REPORTED finding is recorded. sendTelegramAlert returns false on a failed POST rather than throwing, and because these two markers are PERSISTED — the first alerts in this file that write down "we said this" rather than recomputing findings from data each run — stamping regardless would let one transient Telegram failure silence a standing finding forever instead of costing one message. So the Morpho stamp and the Fluid write of a finding both wait on alertWasReported; an unprovisioned env counts as reported (there is no destination, the finding is in the job's own report, and re-deciding it every run would log the same batch forever on a dev box), a provisioned env that failed to send does not. Ordinary (1x) magnifier values are still written inline — they are not findings and nothing waits on them.

Neither write can take its job down. borrow_rate_magnifier is written by a Fluid-only guarded UPDATE, not as two more columns on the capacity upsert every venue shares (that upsert runs for Aave/Spark outside any try, so an unknown column there would abort the whole refresher before Fluid and Morpho and leave every venue's capacity stale). irm_flagged_at is likewise stamped in its own guarded statement. Both migrations carry the ORDERING: … apply BEFORE the deploy line anyway; the guards are what make getting it wrong cost one field instead of one table.

Registration backfill (WS5)

So an account is not born with an empty chart, each newly-registered wallet gets its history replayed once at signup. Three decisions shape the replay: a rolling history window — a wallet's replay reaches back to the UTC midnight HISTORY_WINDOW_DAYS = 30 days before that wallet was FIRST ADDED, recorded once on its account row as history_floor_ts / history_floor_block (migration 103) and read by everything that needs it through src/lib/portfolio/history-floor.ts. It replaced a FIXED 2026-01-01 date, whose two costs both grew by a day every day: the archive replay walks one grid point per day of window, and the wallet's own bare-token ingestion walked the whole span from the event store's 2025-05-21 floor (~3.4M blocks, ~17 minutes) before anything about that wallet could be certified. A rolling window is a constant amount of work per new wallet instead. What it costs, stated plainly: a position opened before the window is not charted from where it was opened — it enters the first snapshot as an opening balance at its value on the window start, and its yield and return count from there, which is the same rule a position opened before 2026-01-01 already followed at a further boundary. It is the only rule, and that is the change (#873): the replay opens at the wallet's stored floor and at nothing else. A fixed 2026-01-01 fallback, a ninety-day span bound and an operator --days used to sit on top of it, and every one of them was a second opinion about a boundary the completeness certificate treats as literal — the derivation walks from the stored floor and stamps at that same block, so a replay opening anywhere else left a wallet either certified over a hole or uncertified for ever. One thing still moves the start, and it moves it UP for one kind of wallet only: the floor is a FLOOR for a wallet that held nothing yield-bearing at it, so that wallet's range start is raised to its own first curve-starting activity. A wallet that already held a position at the floor opens exactly there, with the position as an opening balance (#852) — raising its start would drop history it genuinely has, which is the opposite of what the raise is for. Every account row that predates the rolling window was given the 2026-01-01 pair explicitly by migration 104, so those wallets keep every point they have and the code carries no fallback rule; a row with no pair at all (a boundary block that could not be resolved at add time) reads as the bottom of the indexed event history, which withholds rather than over-claims, and its next replay resolves and stores the real pair. "First added" is the WALLET's own accounts.created_at, shadow accounts included: the history belongs to the wallet, so a wallet another account added earlier keeps and shares everything since then. The replayed set — the position groups the wallet holds when the backfill runs, UNIONed with the groups the event ledger shows it held earlier IN THAT WINDOW, so a position closed inside the window is replayed over its own lifespan and its exit nets against the cash it paid out instead of drawing that cash as a deposit from outside (a position closed BEFORE the window still has no history here, so unrecognisable history cannot surface); and a daily pre-signup grid (UTC midnights; 6h resolution starts at signup). The grid used to be visible on the 1W timeframe, which drew the raw 6h cadence (?bucket=6h, added 2026-07-17): across the pre-signup part of a recently-registered wallet's week each midnight point sat between three empty 6h windows, so it rendered as a lone dot rather than a line (the chart drew a dot for a reading with a gap on both sides — without it the point would have drawn nothing at all, since a segment needs two consecutive non-null points). That scatter is half of why the 7-day range was removed (2026-08-10); every remaining timeframe reduces to one point per UTC day, where the distinction does not arise. The lone-dot rendering itself went on 2026-08-12, when interior gaps started drawing as a flat line carried forward from the last reading (bridgeGaps in chart-series.ts): a reading between two missing days now joins the line rather than standing alone. scripts/backfill-portfolio-wallet.ts (logic in src/lib/portfolio/backfill.ts) is BOTH the queued job the drain cron runs and the manual repair tool — note a FRESH repair run (no coverage anchor, or --fresh) re-derives the replayed set AT RUN TIME, so re-running a wallet that closed a position BEFORE the window also prunes that position's history, by design.

Open-group granularity (src/lib/portfolio/open-set.ts): "open" is decided per position GROUP, never per leg — dropping one closed leg of a still-open group would misstate historical NET value (an Aave debt repaid pre-signup under still-held collateral would inflate every past equity point). Groups: aave/sparklend = the VENUE ACCOUNT (cross-margined account-wide, so any open leg keeps the wallet's whole venue history and its full reserve universe); morpho-blue = the MARKET id; pendle = the PT instrument; erc4626 = the vault; fluid = the NFT. One derivation from the probe's strict current read feeds three surfaces that cannot drift: the replay's restricted reader universes (restrictRegistries, incl. the reg.curatorVaults erc4626 override), the grid-read post-filter (filterReadsToOpenSet, which also reins in the self-describing Fluid reader whose per-block enumeration would resurface closed NFTs), and the flow filters.

Enqueue (built in src/app/api/auth/verify, not WS1): on a successful SIWE verify, after the account is upserted, enqueueBackfill(uid) inserts a portfolio_backfill_state row status='queued' — INSERT ... ON CONFLICT (uid) DO NOTHING, so it fires exactly once (on account creation, or a chat-seeded account's first login with no state row) and a re-login of an already-processed account is a no-op (a completed backfill is never re-queued). The FK to accounts(uid) requires the account to exist first.

Probe: ONE STRICT multicall round of the wallet's CURRENT positions across all readers — strict via readGridPointOrThrow, because this read ALONE decides terminal empty (never re-queued on re-login), so a swallowed venue failure must abort the run, not silently drop a venue. empty = no current positions, full stop; the wallet ends status='empty' after ONE read pass with NO discovery and NO log sweep at all (past activity alone still does not earn a replay — the empty verdict prunes history and is terminal, so it stays a question about today). A non-empty wallet's reads define the groups it holds now; historical discovery (below) then adds the groups it held earlier in the window, and only then does the candidate window get swept — RESTRICTED to the union's tokens/events, wallet as the topic filter, post-filtered by the open set (pool-level Aave/Spark liquidation events cannot be restricted at the RPC) — in ascending block segments (BACKFILL_FLOW_SEGMENT_BLOCKS, default 200k ≈ 27.8 days) whose detected flows are FOLDED into the first-activity answer and released, never retained (probe memory is O(one segment) regardless of wallet activity), with an early exit once a curve-starting block is found (no later, higher-block segment can lower the minimum — an active wallet's probe scans ~one segment; the par-only fallback is exactly the case that still needs the whole window). First activity block = the first swept log of a REPLAYED group that can start a yield curve: a par wallet-token (idle) flow is excluded unless it is the only activity (firstCurveActivityBlock), so an old stablecoin top-up cannot pin the chart's start months before the first yield-bearing position. The floor read then asks the one question that decides whether that block matters at all: one STRICT read of the wallet AT THE FLOOR BLOCK, over the same restricted universe and through the same open-set filter the replay's first grid point passes through, so the answer and grid[0] are the same read rather than two that agree most of the time (heldCurveStartingAt). A wallet counts as holding at its floor when any leg there has a non-zero balance on a venue, or a non-zero bare-wallet balance of a token that is not par — no valuation and no dollar threshold, so a dust leftover counts and an idle stablecoin or ETH balance does not (the same judgement the flow side makes, read off a balance instead of a movement). Strict for the same reason the current read is: a silently-dropped venue here reads as "held nothing at the floor" and cuts the window exactly as the defect did, so a failed venue aborts the run and the drain retries it. It costs one archive multicall round per registration, over a universe smaller than the current read's. What it cannot see, so the rule below is not read as unconditional: the floor read runs over the REPLAYED universe and through that same open-set filter, so a position the wallet genuinely held at its floor is invisible to it whenever the group is absent from that universe — not held at the tip, and not admitted by historical discovery, which admits a row only where a live coverage certificate already reaches down to that row's own block. Such a wallet reads as "held nothing at the floor" and still has its start raised to its first later move. That is the deliberate trade (§4.3 of the plan): the window's answer and the universe the replay actually walks are one read, consistent by construction, instead of two that agree most of the time — widening the universe from what the floor read finds is a separate question and is not answered here. Fluid probe (FWS3): the wallet-filtered sweep structurally CANNOT see LogOperate (zero indexed params), so the fluid probe uses FACTORY ERC-721 Transfers only, filtered to the REPLAYED NFT ids. The fluid FLOW rows themselves come from the cache during the replay, not from the probe.

Replay: from the wallet's own history floor, raised to its first curve-starting activity ONLY when it held nothing curve-starting at that floor, to now. Three lines, and they are the whole rule: held a position at the floor opens AT the floor, that position entering as an opening balance; held nothing at the floor is raised to the first curve-starting activity, so the chart does not open on a flat-zero lead-in; holds nothing now is empty, and that verdict is terminal. The conditional half is new (#852, 2026-09-22): the raise used to be unconditional, so a wallet holding positions opened BEFORE its window lost every day before its first in-window move — up to 25 days on the wallet that surfaced it — and where that first move was the EXIT of one of those positions (an exit starts a curve too) its chart opened on the day it closed something. The wallet's own history floor is the UTC midnight 30 days before it was FIRST added (migration 103, stored on its account row; migration 104 wrote the 2026-01-01 pair for every row that predates the rolling window, so those wallets keep the start they already serve). There is no other term. On a DAILY grid — UTC midnights (floor_ts = the midnight at/below the range start) plus ONE final 6h-aligned seam point when "now" lies past the last midnight, so the live cron's next 6h tick continues the series without a same-day gap (MAX_GRID_POINTS=500 safety clamp, which the daily grid from the floor stays inside into mid-2027). Midnight is on the 6h grid, so every replay ts is a valid aligned window and nothing downstream can tell a daily point from a 6h one. Each grid point is read via archive multicalls at blockByTimestamp(gridTs) over the RESTRICTED replay universe and valued in both marks by the SAME buildSnapshotRows path the live cron uses (verified byte-identical at a shared block), but written with basis='backfill'. floor_ts records the range start = the account's "tracked since" anchor (per-account, shown in the methodology footer). It is SET on done (and cleared on empty), never LEAST-merged: under full-history ownership (below) every run deletes everything at/below its own end before re-laying rows, so the anchor always equals the run's grid[0] (= min(snapshot_ts) up to an at-most-24h empty lead-in day, which writes no rows) — the old monotone-down LEAST merge belonged to the windowed-delete world and would leave the anchor stranded below the earliest row after a raised-start re-run. A movement landing DURING the replay — which takes minutes — needs no sweep of its own any more: the replay's whole-history merge runs after the spine commits and derives up to the ingested tip, and the 6h tick's own cursor then covers everything above that, so a receipt in that span is reached by one pass or the next rather than needing a separate tail scan to rescue it.

Where the chart actually starts (M28): the replay lays down a GRID, and the engine used to open a book at its first grid point, so a position acquired between the grid start and the first snapshot that saw it had its opening flows absorbed into an opening balance and everything it did in those hours (up to 24h on the daily pre-signup grid, up to 6h live) charted nowhere. Since the birth anchor the curve opens at the book's OPENING FLOW itself, valued from the flow rows in both marks, whenever the legs' own flows explain the value they hold at that first snapshot; a book that predates the window is untouched. Where a book was opened over several transactions (a wallet funded, then the position deployed; a supply tx then a borrow tx) the opening point sits at the LAST of them, so it carries the equity the first interval earns on — nothing is lost, because a leg born earlier opens at the capital that went into it and its whole span is still charted in that interval. This is a READ-time rule over the same stored rows — nothing about the write path changes, and no wallet needs re-deriving for it — and it covers all three ways a position can be born inside the window: the registration replay above, the live-signup path (a first position opened between two 6h crons, where the JIT tip is the only snapshot), and a group discovered by the ledger that opened AND closed in-window, which now starts its segment at its own deposit. It also moves the account's tracked since date to the day the first position was opened rather than the first grid point after it. Full rule, the shapes it deliberately declines to anchor (three of which withhold the whole book's anchor) and the rate-bounded band that decides a birth, in Metrics → M28.

One pass, and no depth owed afterwards (#873). A signup used to replay its history in TWO tiers — a bounded recent window first, then the rest of the depth as background queue work (deep_extend_ts, migration 076) — because the fixed 2026-01-01 window was ~2.4x the archive reads of a rolling ninety-day one and grew by a point a day. The rolling 30-day window removed the reason for it: a new wallet's whole window is ~31 grid points, so the fresh replay covers it in one pass. The tiering, its ninety-day span bound, its backward-extension path and the operator's --days are gone with it; backfillPath is two shapes again (--fresh wins outright, no anchor means fresh, anything anchored patches the tip). The deep_extend_ts column is left in place and unread — dropping it is destructive and belongs in its own contract migration. What a long window costs now that nothing segments it: a wallet tracked for months and then replayed FROM SCRATCH (a repair, or a lost coverage anchor) spans one grid point per day since its floor in one transaction, against the drain's 30-minute child SIGKILL. MAX_GRID_POINTS = 500 is the safety clamp that bounds it, and the ordinary paths do not reach it — a re-add is a gap patch, which is already segmented, and a fresh replay of a recently added wallet is thirty-one points. A wallet's window cannot be deepened after the fact, by any tool today (#861). It would take the wallet's bare-token coverage being swept and lowered FIRST and its stored floor moved after, in that order — the same rule catchUpPassFloor applies to the ingester — and no tool does the first half: record-ledger-backfill-band.ts writes acceptance bands and never touches coverage, and backfill-event-ledger.ts --stream wallet-token from a deeper floor stamps an up-front [floor, floor] ownership marker for every enrolled wallet, which mergeCoverage refuses for any wallet whose row already starts above it, before a single window is swept. The same per-address floor is why --verify's condition 4 reports every rolling wallet as "not live from the floor": it measures against the stream's raw floor rather than each wallet's own. Both halves are #861.

Strict grid read (M9): unlike the live path, which reads through readAllPositions and silently swallows a venue's RPC failure (the next tick recovers), the backfill BAKES each grid point into history, so it reads through readAllPositionsSettled (which reports the venues that threw) via readGridPointOrThrow, with strictNative set (below). A grid point whose read reports ANY failed venue is retried (GRID_READ_MAX_ATTEMPTS=4, backing off); if it still fails the whole backfill THROWS (the account is left non-done for a clean re-run) rather than persisting a phantom-empty window (a total failure) or a leg silently MISSING from an otherwise-present ts (a partial failure, which would read as a phantom loss then gain). An all-venues-succeed empty result is a genuine "held nothing" and is written as an empty window. Two things the venue-level strictness alone did not cover. (a) The UNIVERSE the venues are read against: loadRegistries CATCHES a loadPortfolioTokens/loadFactoryVaults failure and degrades to an empty wallet universe / the static erc4626 set, which is right for the live paths but indistinguishable from "the registry is empty" — so for a wallet whose only holdings are wallet-venue (a pure sUSDe/reUSD holder) the probe would read "holds nothing", and the empty branch PRUNES the account's history and sets a terminal status='empty' that is never re-queued. loadRegistries now REPORTS every degradation and the backfill entry calls assertRegistriesComplete before the probe, aborting the run instead (the account is left running for the caller to mark error, so it is retried, not resolved wrongly). (b) The native-ETH leg: the wallet reader skips a FAILED eth_getBalance exactly as it skips a true zero, INSIDE an otherwise-successful venue read, so readGridPointOrThrow never saw it — and the anchor path recorded the absence as a 0 balance, which the balance-diff pass of the day turned into a fabricated full-balance transfer_out + next-day transfer_in plus a one-day equity dip. Both backfill reads (probe and grid) now pass strictNative, so a failed native read fails the wallet venue, retries, and then aborts the point. It is scoped to the native leg deliberately: eth_getBalance never reverts, so a null there is unambiguously a failure, whereas a null ERC-20 balanceOf can be a permanent revert and a strict mode there would abort every backfill forever. The throw fires BEFORE the windowed delete+insert, so prior good rows are never touched. Positions that predate the range enter as the OPENING BALANCE of the first snapshot (their qty at grid[0]), NOT as a flow — that is what makes the chart honest. First activity RAISES the start so a chart never opens on a flat-zero lead-in — for a wallet that held nothing curve-starting at its floor, the only wallet that has such a lead-in to avoid — and only CURVE-STARTING activity counts (par/idle wallet-token flows are excluded unless they are all there is), so the lead-in cannot re-enter through an idle top-up months before the first yield-bearing position; a par flow the raised start leaves behind is absorbed into grid[0]'s opening balance by the flow clip below. Note this start is per-WALLET, so it belongs to whichever BOOK opened first, and it is the start EVERY book is charted from: the read-time per-book re-anchor (trimIdleLeadIn) went with the retired reader, so a book that opened later is drawn with a flat lead-in rather than trimmed (see Portfolio). That is a read-time decision either way and changes no stored row. The receipts are not scanned here at all any more. The replay lays the position spine; the movements that explain it come from one whole-history derivation of the ledger (persistReplayV2), run AFTER the spine transaction has committed and in its own transaction — never folded into it, because the replay already holds the blocking writer lock for minutes on a whale grid and the derivation reads the event store. It derives [derivation floor, ingested tip] in one bounded merge, so there is no per-segment write phase, no second delete window and no tail residue to reconcile: the merge owns the wallet's whole history or it writes nothing and says why (the enrolment gate, below). Its outcome is independent of the spine's — a spine that committed with no derivation leaves the wallet with history and no derive cursor, which the served path withholds and the 6h reconciliation alarm carries as excluded. The anchored-wallet case is explicit (review findings): the only way an ANCHORED wallet reaches this destructive path is the operator's --fresh, and that dispatch REVOKES the coverage anchor up front — and again atomically inside the wipe transaction — precisely so the crash-intermediate state stays repairable: with the anchor left standing, a run that died between the wipe and the last segment would dispatch its retry to the GAP path, which by design never re-derives at/below the anchor, permanently stranding the wiped span (every lost deposit/withdrawal would read as yield/loss beside complete snapshots). Revoked, the retry re-enters the FRESH path and fully re-derives; a completed run re-earns the anchor at done. A healthy destructive re-run of a done wallet has a reader-visible window (accepted, operator-only): between the spine transaction's commit and the derivation's — the minutes the whole-history merge takes — a concurrent /portfolio read of that wallet sees complete snapshots beside receipts the merge has not restated yet, so historical deposits can transiently read as yield and entry bases vanish; fresh signups have no prior rows and first-time wallets sit behind the building gate, so only the manual repair of a live done wallet exposes it. Valuation is strict about block timestamps: the per-distinct-block eth_getBlockByNumber reads run at bounded concurrency (MARK_READ_CONCURRENCY, default 24) with a 4-attempt retry ladder, and a block that still cannot be resolved THROWS — failing the merge like any failed read — instead of a silent per-flow skip, which could complete a run done with movements missing from the ledger (their principal booking as yield). PT entries older than the derived range get the synthetic opening lot per M34 downstream (the read-time accrual curve reconstructs the entry fill from the first snapshot's own implied rate when no in-range acquisition exists).

Wallet-venue history (T3): the wallet venue's FLOW ledger is replayed over history too. Before T3 the backfill read wallet BALANCES only, so a variable_rate wallet token (sUSDe / reUSD / …) whose balance changed mid-window booked the whole balance step as phantom yield — a bare variable-rate leg is a value-accrual leg, attributed as Δvalue − netLegFlow, so with no flow the acquired principal reads as profit. Two derivations, mirroring the forward cron (both idempotent, both flowing through the SAME open-set filter, valuation and delete+insert as the other venues): (1) ERC-20 replay — the OPEN wallet tokens' Transfer streams are swept alongside the position-token streams (a SEPARATE getLogs pass, so a bare token can never collide with a position-token address in scanTransferFlows' by-token map; the native-ETH sentinel is dropped — it has no Transfer logs), so every historical acquisition/disposal books as a transfer_in/transfer_out at its real block and the replayed variable-rate curve then attributes only share-rate yield (the flow nets the acquired principal). A variable_rate wallet-token acquisition can also be the wallet's first-activity block (a par token's cannot — firstCurveActivityBlock). (2) Native-ETH balance diffs — the eth_getBalance the wallet reader already issues at each grid point (≈1 read/wallet/day) is diffed against the previous anchor into a transfer_in/transfer_out (native ETH is par, so Δvalue = net flow EXACTLY, gas spend lands as a small transfer_out; grid[0] is the opening balance, never a flow), keyed by the grid point's aligned window (nativeEthDiffTxHash) so a re-run upserts the same row and the 6h seam anchor hands the series to the live cron (which continues by diffing its next snapshot against the seam — a snapshot-diff flow self-heals, so native ETH needs no tail sweep). No double-count at the live/backfill boundary: an ERC-20 flow carries the real (tx_hash, log_index) PK the forward cron re-produces, a native-ETH flow the window-keyed synthetic tx, so a forward re-scan of the seam overlap UPSERTS the same row rather than duplicating it (verified live: the backfilled curve of a real held-through sUSDe holder with a mid-window top-up attributes only the share-rate yield, not the top-up principal, with a clean per-interval yield-invariant residual).

Archive RPC: readAllPositions reads through multicall3, which targets ETHEREUM_RPC_URL — and publicnode REJECTS archive eth_call. So the backfill runs in its OWN process with ETHEREUM_RPC_URL defaulted to the archive endpoint (the CLI entry sets it before importing anything that snapshots the URL at module load; the cron spawns the CLI as a child with the archive env). This keeps the read/value code path identical to live while routing every read to an archive-capable node.

Processing (the minutely drain cron): * * * * * run-cron.sh drain-portfolio-backfills.ts drains the queue with a WORKER POOL (BACKFILL_WORKERS, default 4; Phase C D3, above) — the workers RACE on the same atomic UPDATE ... WHERE uid IN (SELECT ... ORDER BY updated_at LIMIT 1 FOR UPDATE SKIP LOCKED) claim (flipping the oldest available row running), each spawns its archive-routed child and awaits it before claiming again — until the queue is empty or the SHARED per-invocation cap (PORTFOLIO_BACKFILL_MAX, default 10; 0 disables) is reached; leftovers ride the next minute. SKIP LOCKED locks disjoint rows, so no two workers ever claim the same wallet and approximate FIFO holds. run-cron's per-script flock guarantees invocations never overlap, so the reclaim's live-child safety holds even when a run outlives its minute, and an empty-queue invocation prints nothing. The child owns the done/empty transition; the drain owns retry, parking and alerting (next paragraph). Because the live-snapshot wallet selection excludes queued and running, a wallet is live-snapshotted only after its backfill finishes; the residual concurrent-writer races are closed by the advisory writer lock (above).

Child failure: retry while the budget holds, then park, and page either way (2026-07-16; budget raised 2026-08-06). A backfill child that exits non-zero has its attempts burned and is either RE-QUEUED (budget left) or PARKED as error (attempts >= MAX_BACKFILL_ATTEMPTS = 4, i.e. three retries). Every failure — the retryable first one AND the final one — is returned to the drain, which exits 2, which is what fires run-cron's Telegram alert; the per-failure log line leads with the WALLET, the attempt number and whether it was FINAL, so the alert answers "who lost history, and is it over" without an SSH round trip. The retried wallet is skipped for the REST of that invocation (the drain would otherwise re-claim it milliseconds later and burn the whole budget in one tick against a provider that just failed); the next minutely tick is the backoff.

This replaced a silent park. The child records its own status='error' + message before exiting non-zero, so the drain's old UPDATE ... WHERE uid = $1 AND status = 'running' fallback matched no row on the common path, and only the reaper's parks ever exited non-zero. A transient rpc http 503 therefore parked a wallet with no alert at all while the cron reported success.

How long a park lasted depended on the wallet, and this is the part worth understanding. The drain never re-claims an error row (the claim query takes only queued), but enqueueOrRequeueBackfill does: it requeues status IN ('error','empty') (resetting attempts=0, error=NULL), and the SIWE verify route calls it on every successful sign-in. So for an account's OWN wallet a park self-healed at the owner's next login — stale history until then, and nobody told, but self-limiting. For a watch-only tracked wallet (v0.9.0 multi-wallet) it did not: its accounts row is a SHADOW that nobody ever signs in as (last_seen_at stays NULL, which is what makes it a shadow), so the sign-in path can never fire and the only recovery is a remove + re-add or an operator re-queue. That is the live case: 0xef08c6a4…, a tracked wallet, parked 2026-07-15 14:28 and sat until a manual re-queue a day later. Tracking made the silent park open-ended for exactly the wallets whose owner is not the one signing in.

MAX_BACKFILL_ATTEMPTS shares the attempts column with the reclaim budget deliberately (that column already means "failures burned by the current run" and resets to 0 on done/empty, which is exactly the retry counter's semantics, and reusing it keeps the fix migration-free); the budgets can interact, so a wallet that orphaned once then fails a child parks on that first failure instead of retrying — earlier and still alerting, never later or silently. A timed-out child (SIGKILL at CHILD_TIMEOUT_MS) is the one failure never retried: the replay would still need >30 min, and run-cron's exclusive flock means a second 30-minute child blocks every other drain tick behind it.

The writer lock's wait has its own budget (2026-08-06). Every history writer takes one advisory key (above), and a whale replay legitimately holds it from its first grid flush to COMMIT — minutes. The prod role carries statement_timeout = 30s, and a blocking pg_advisory_xact_lock is ONE statement that sits still until the holder commits, so the role's ceiling was cancelling writers for QUEUEING rather than for working. Measured on the 2026-08-06 validation drain: BACKFILL_WORKERS=4, four children cancelled with canceling statement due to statement timeout, three parked as error and lost their history until a human re-queued them — after which each completed in 28-33s. PORTFOLIO_WRITE_LOCK_SQL therefore raises statement_timeout to PORTFOLIO_WRITE_LOCK_WAIT_MS (default 10 min) for the acquire alone and puts the previous ceiling straight back before any write runs under it. It STASHES that ceiling in a transaction-local custom setting rather than resetting to DEFAULT, because DEFAULT is the ROLE's value: a caller that had deliberately raised its own (ops/reset-portfolio-users.ts takes 10 minutes for its destructive delete) would otherwise be silently cut back to 30s for the rest of its transaction. A DO block would be the obvious alternative and is wrong — the timeout is armed when a statement STARTS, so changing it inside one does not affect the statement that is running, and separate statements are what make the budget real. SET LOCAL scopes both to the transaction, so a COMMIT or a ROLLBACK restores the session value even when the acquire itself throws. The budget is deliberately BELOW CHILD_TIMEOUT_MS (30 min): a queue that long is a capacity problem the operator should hear about as a reported failure, not as a SIGKILL with no message. The JIT path's TRY-acquire is untouched — it never waits, so it has nothing to wait out.

Contention self-heal (2026-08-06). error is effectively terminal for a watch-only tracked wallet (the sign-in escape below never fires for a shadow account), so a wallet parked by the operator's own contention stayed parked until a person noticed. On every invocation, before claiming, the drain now re-queues error rows that (a) carry a CONTENTION/CONNECTION message (isContentionParkError: a lock timeout, a deadlock, an exhausted connection pool, a dropped connection, or a cancellation of the lock ACQUIRE itself). A BARE statement timeout is NOT in the class: Postgres words "cancelled while queueing for the writer lock" and "this wallet's own write ran past the role's 30s ceiling" identically, and the acquire hands that ceiling back before any write runs under it, so healing on the message alone would give a wallet whose write is genuinely too slow four extra DESTRUCTIVE full re-derivations that fail exactly the same way. The acquire therefore tags its own failure (WRITE_LOCK_WAIT_MARKER, carried into the row's error), and only that shape heals, (b) have rested SELF_HEAL_COOLOFF_MS (15 min), and (c) are still under MAX_SELF_HEAL_ATTEMPTS. The class test lives in TS, not as a SQL regex, so "our fault, not the wallet's" has ONE definition, and the UPDATE re-asserts status and budget so a row a sign-in already re-queued cannot be flipped under a live child. It terminates: attempts is NOT reset by a heal, a park arrives at MAX_BACKFILL_ATTEMPTS and each healed run that fails again adds one, so the ceiling arrives after a bounded number of rests and the row then waits for a human. A PROVIDER fault (an rpc http 503, a registry gap) is deliberately outside the class: it has already spent the raised attempt budget, and a backfill run is a DESTRUCTIVE full re-derivation, so re-running one every quarter of an hour against a broken upstream forever is worse than a parked row and an alert. Pre-045 (no attempts column) the heal is SKIPPED rather than run unbounded.

Crash recovery (stale running reclaim): the queued→running flip commits BEFORE the child is spawned/awaited, so if the whole cron process tree is torn down mid-run (server reboot, OOM-kill of the cgroup) neither the parent's crash-fallback nor the child's terminal write runs, and the row is stuck running forever — permanently excluded from BOTH this queue and the live snapshot cron with no other recovery. So on every drain invocation, before claiming, processBackfillQueue reclaims any running row older than STALE_RUNNING_MS (60 min) back to queued, incrementing its attempts (migration 045); a row whose budget is spent (attempts >= MAX_RECLAIM_ATTEMPTS = 3) parks as error instead — a wallet whose backfill reliably tears the process down must not reclaim-loop forever under a minutely schedule (repair stays available via the manual CLI). The staleness threshold MUST exceed CHILD_TIMEOUT_MS (30 min): a legitimately in-flight child's updated_at is stamped at claim time and not heartbeated during its run, and run-cron.sh's flock forbids overlapping invocations, so a row older than the reclaim cutoff cannot be a live child. A reclaimed row re-enters the queue and re-runs (the windowed delete+insert makes that idempotent).

Repair / idempotency of the FRESH replay path (M9, the append-only carve-out; a re-add with a coverage anchor takes the gap-patch path instead, which preserves everything at or below the anchor — see above): a fresh run OWNS the wallet's ENTIRE history — it deletes everything at/below its own end for BOTH tables (snapshots by ts, flows by block) and re-lays the daily replay. A lower-bounded ("windowed") delete would orphan old rows below a raised range start (the re-derived replay set can raise first activity when the earliest-activity group has dropped out of the window) and splice stale full-universe history onto the new — the orphaned rows would chart a pruned leg's value and then book it as a phantom realized loss at the seam. A re-run is therefore a full re-derivation, not a patch: it REPLACES post-signup 6h live rows with daily rows and PRUNES groups that have dropped out of the window since the last run (by design); a wallet re-run with NO open positions prunes its whole history and clears floor_ts (status='empty'). That prune reaches the position spine and the retired flow ledger, and deliberately not the rebuilt ledger or the wallet's derive cursor, because a verdict may delete an OBSERVATION and never a DERIVATION: nothing re-reads the spine for a wallet with no window, whereas the rebuilt ledger is a pure function of raw_events that the same call's own whole-history merge, every 6h tick (an empty wallet stays in the tick population) and the re-derivation trigger all reproduce. Deleting it can only ever achieve a PARTIAL history — the tick's own window and nothing below it — which reads as a wallet that appeared from nowhere. So a wallet that closed everything keeps an Activity feed of the movements that closed it, which is the true answer to "what happened to my position", and loses the curve and the positions, which are the claims about holdings the verdict is actually about. It is NOT the recovery for a missed 6h snapshot tick — a missed tick is just a chart gap the next tick moves past. Two runs with no intervening wallet activity produce identical rows (blockByTimestamp is deterministic for a second the chain passed long ago, which every grid point of a replay is; archive reads at a fixed block are deterministic; DeFiLlama historical prices are deterministic).

Repair runbook (manual; ONLY for a broken/incomplete backfill — a missed 6h tick needs no repair, it is just a chart gap):

# DESTRUCTIVE RE-DERIVATION (the FRESH path; a wallet with a coverage anchor takes the gap patch instead, or pass `--fresh` to force this one): deletes the wallet's ENTIRE stored history (both
# tables) and re-lays the daily replay from the recomputed range start.
# Post-signup 6h resolution is thinned to daily and groups closed since the last
# run are pruned. The run replays the wallet's OWN window and takes no bound of
# its own: there is no --days, precisely because a bound plus a full-range delete
# amputated everything older than it.
# `--fresh` on an anchored wallet REVOKES the coverage anchor up front, so an
# ABORTED repair re-runs FRESH (never gap-patches over its own wipe); the anchor
# is re-earned at 'done'. While the repair runs, a concurrent /portfolio read of
# that wallet transiently sees snapshots without charted-range flows (minutes).
ETHEREUM_ARCHIVE_RPC_URL=<archive> DATABASE_URL=<...> \
  npx tsx scripts/backfill-portfolio-wallet.ts --uid 0x<wallet>
# To re-queue instead of running inline, INSERT ... ON CONFLICT DO UPDATE SET
# status='queued' and the minutely drain picks it up within ~a minute.
#
# EXIT CODES. 0 = the wallet was replayed. 1 = the run did NOT complete, and the
# wallet's backfill state carries why. One case of that 1 is not an outage: a run
# whose universe would have shrunk the ledger REFUSES (see "A narrowing can never
# silently delete" below) before touching anything, so both the flow rows and the
# position history are left exactly as they were, takes no coverage anchor, and
# exits non-zero so the failure reaches the alert instead of a log file. Read the
# [fail] line before re-running: it says whether re-running after the wallet-keyed
# backfill is the remedy, or whether it never can be.

A held PT's real purchases from before the window are recorded last (migration 113, src/lib/portfolio/pt-prewindow-fills.ts). After both writers have committed, every wallet build (backfillWallet, unconditionally, like the ledger merge) looks at each bare Pendle leg the wallet held at its FIRST reading with no ledger receipt at or before it — a PT older than the window, which the lot book would otherwise open on a stand-in struck at that reading's price. It reads the market's router fills below the opening block through the two topic indexes (topic2 = market for the swaps, topic3 = YT for mint and redeem), keeps the wallet's own (attributed on receiver, never caller), and stores them in portfolio_pt_prewindow_fills only when their net PT quantity equals the opening reading's EXACTLY with no negative running balance, and every acquisition priced (through the same valuation resolver as a ledger receipt, stamped the same way) and read its factor. Anything short of that stores nothing new. A stored set is never lost to a transient failure: a leg whose stored set still explains the same opening (block and raw quantity) is kept without re-valuation (its fills are history), a set is replaced only by a complete recompute, and it is removed only when it no longer explains anything (the opening moved, or the leg is no longer a holding older than the window). Writes are per leg, in one transaction; a failure logs one line and never fails the build; a box without the table is detected by a probe before any valuation read and stores nothing. One line per build names what it did: pre-window PT fills: N of M held PT(s) explained by their own router fills (K fill(s) stored), with the reason for each stand-in kept.

Event ledger ingester (portfolio 100k scale)

The event ledger is the wallet-count-independent spine that lets the portfolio serve up to 100k registered wallets without the wallet-topic eth_getLogs OR-arrays that hard-fail at ~1-2k wallets. It is not a cron: it is an always-on PM2 process (creddit-event-ingester, scripts/ingester/ingest-events.ts), added as a manual server step (see Deployment → event ledger ingester). The ledger accumulates events; the venue readers stay the balance-of-record.

• What it scans. The scan surface is derived once per cycle from the SAME DB registries the venue readers use, by the pure trackedContracts(reg, vaults, dexPools, wallets) (src/lib/portfolio/tracked-contracts.ts) — the single source, so the reader universe and the event universe cannot drift. Twelve of the thirteen log streams are filtered by contract address and/or event-signature topic0, never by wallet; the thirteenth, wallet-token, is the deliberate exception and is described under its own heading below. A fourteenth ledger stream, native-eth, is not a log scan at all: native ether emits no log, so the ingester reads it from the blocks themselves (the native-ether block feed):

  stream	addresses	events	coverage
  transfers	aTokens + variableDebtTokens + every ERC-4626 share token + ALL Pendle PT addresses	Transfer	per-address
  morpho	the Morpho Blue singleton	the 7 supply/withdraw/borrow/repay/collateral/liquidation events	singleton
  aave-pool / spark-pool	the Aave v3 Pool / SparkLend Pool	Supply, Withdraw, Borrow, Repay, LiquidationCall, DeficitCreated, DeficitCovered, UserEModeSet	singleton
  aave-token / spark-token	the aToken + variableDebtToken of every reserve on that venue (170 addresses across the two: 134 Aave, 36 SparkLend, zero overlap, measured on prod's registry)	Mint, Burn, BalanceTransfer	per-address
  fluid-operate	chain-wide, no address filter (the events carry zero indexed params)	LogOperate, LogLiquidate	singleton
  fluid-nft	the Fluid VaultFactory	ERC-721 Transfer	singleton
  fluid-dex	every Fluid DEX pool, read from Fluid's DexResolver each time the surface refreshes (49 on 2026-08-20, up from 48 three days earlier)	the 10 position-changing pool events (collateral in/out and debt in/out, each in its exact-amount and its perfect-share form)	per-address (one row per pool)
  erc4626	every vault in the runtime universe and every wallet-tracked token (546 addresses measured on prod: 502 vaults, 45 wallet-tracked, deduped, less the native-ETH sentinel)	the canonical ERC-4626 Deposit, Withdraw	per-address
  pendle-router	Pendle Router V4	every event it emits (no topic0 filter)	singleton
  wallet-token	none — filtered on the wallet (topic1 for sends, topic2 for receipts), minus every address another stream already claims on Transfer	Transfer, TransferShares, and WETH9's Deposit / Withdrawal claimed from WETH9's own address only (issue #966)	per-address, one row per WALLET
  escrow	the Lido withdrawal queue, the EtherFi withdrawal NFT and the two Maple/Syrup pool + withdrawal managers — pinned in code and re-read from the chain, four declared filters rather than one address × topic product	per filter: Lido's request/claim + the ERC-721 Transfer; EtherFi's request/claim/seizure/invalidation/validation + its Transfer; Syrup's RequestCreated/Processed/Removed/Decreased; the pool managers' SharesRemoved	address-set — one row per escrow contract, written only by the one-time backfill
  native-eth	none: every block the ingester scans, eth_getBlockByNumber(block, true) + the receipts the tracked wallets need (none, one transaction's, or eth_getBlockReceipts(block)), filtered in memory against the tracked wallets	synthetic rows on the native-ether sentinel: a transaction's value, its fee, a beacon withdrawal, a fee recipient's priority fees, and (landed by the reading audit) a contract's internal payment	per-address, one row per WALLET, from the first block the feed read with the wallet in its set; the cursor is the live bound

  Why the Pool streams carry more than the liquidation. The Pool events are the only logs that state a true amount AND name the position's own owner, so they are what a movement is attributed from; the token events beside them carry the quantities the Pool events do not — BalanceTransfer is the only Aave log with a scaled amount, and Mint/Burn carry both balanceIncrease (which is how a pure-interest "phantom" mint is told apart from a real deposit) and the operation's own index. Transfer is deliberately NOT in the token streams: every aToken and variableDebtToken is already in the transfers address universe, so scanning it twice would write one chain fact under two stream names.

  Which means the Aave derivation reads transfers too, and that is a requirement rather than a convenience. A bare aToken Transfer is stored on transfers and can be stored nowhere else, and the derivation routes a log to a venue adapter by its STREAM before the adapter ever looks at its address. Aave emits that bare Transfer beside a richer log for the same movement every time — Mint/Burn on the mint and burn sides, BalanceTransfer wallet to wallet — and the Aave adapter's job is to drop the bare one as a duplicate. An adapter that did not read transfers would never see it, and the duplicate would reach the generic wallet-book classifier unclaimed and be booked a second time. The two halves are pinned to each other by a test, so removing either fails the build.

• The wallet-token stream, and why one stream is keyed on the wallet. The streams above can only reach tokens we can enumerate. The bare tokens a wallet actually holds — USDC, USDT — cannot be address-scanned at any price, so a wallet's ordinary token history is invisible to an address-scanned ledger: today none of the 45 wallet-tracked registry tokens appears among the 741 covered transfers addresses. This stream filters on the wallet instead (the padded wallet in topic1 for sends and topic2 for receipts, no address filter), which makes the volume a function of wallet activity rather than token activity — measured over the whole coverage window, two served wallets produced 501 and 192 logs respectively.

  • Its coverage rows are keyed on the WALLET, not the token, and that is the whole design. A wallet-filtered scan cannot honestly stamp a token-level certificate: the coverage key is (chain, stream, address), so a row keyed on USDT would claim USDT is covered while only the wallets we happened to scan were — a false certificate by construction. Keyed on the wallet, the certificate answers the question it is actually asked: does the ledger reach back far enough for this wallet's bare-token history?
  • It claims only what no other stream can reach, and that is enforced rather than intended. Filtering on the wallet returns every token that wallet touched, including the ~742 in transfers and the Fluid VaultFactory in fluid-nft. One chain log is one stored row carrying one stream name, so two streams claiming it means the loser's reads silently miss it. The stream therefore declares an exclusion set — computed from the other streams' declarations, never hand-listed — and the scanners drop those logs before writing. Nothing is lost: an excluded address has its own per-address certificate reaching the same floor. What follows is that a wallet's whole transfer history is the union of wallet-token, transfers and fluid-nft, each with its own certificate, and the replay universe reads all three. A token listed after a wallet moved it is the one case an exclusion computed today cannot cover, so the ledger writer re-labels on re-scan in one direction only: a residual wallet-token label yields to a declared stream's, never the reverse.
  • Enrolment happens on registration, derived rather than written: a wallet joins the declared set as soon as the product serves it, and the ingester re-reads that set every cycle (uncached — the registries are cached ~5 min, but a signup is somebody waiting). The set is the same population the refreshers serve — tracked wallets, signed-in accounts, anything with backfill state or a snapshot in the last 48h — rather than the tracked list alone, because a wallet outside it would never enrol, never gain a coverage row, and therefore never certify, with nothing anywhere to say so. Registration starts a second, independent clock as well, the registration replay, and the replay normally wins — so there is an ordinary window in which a new wallet's derived rows exist and its own bare-token coverage does not. That window is correct behaviour: the coverage certificate withholds until the enrolment backfill lands.
  • The declared set is monotone. It is that served population unioned with every wallet that already holds a coverage row, so a wallet leaving the population does not drop out of the live filter. Without that, a wallet removed and re-added would carry a live certificate over blocks nobody scanned — the per-address model freezes to_block at catch-up completion and leans on the shared cursor for freshness, which is only true while every covered address stays in the scan. Once enrolled, always scanned; the only thing that removes a wallet is the deliberate wipe of the user graph, which empties both arms together.
  • Cost, stated rather than hidden. Twelve of the thirteen streams are wallet-count independent. This one is not, and cannot be: the padded wallets ride an eth_getLogs topic OR-array chunked at WALLET_TOPIC_CHUNK (500), so no single request approaches the ~1-2k topic-array ceiling, and the per-cycle request count grows as ceil(N / 500) × 2 topic positions. The log volume stays a function of wallet activity.

  A stream whose address set resolves EMPTY is withheld, never scanned. An empty address list reaches eth_getLogs as no address filter, i.e. the whole chain on those topics — which is the only way fluid-operate can be scanned and a catastrophe for any other stream, because the registry loaders return [] for a registry that could not be READ just as readily as for one that is genuinely empty — and the same is true of the one address set that comes from an on-chain read rather than a registry, fluid-dex's pools. So a chain-wide scan has to be declared, and a declared address set that comes back empty drops its stream for that cycle: the ingester names it in the run log and picks it up on the next successful read, while the one-time backfill refuses to run at all rather than certify a partial surface.

  One filter covers the whole history, and that was measured rather than assumed. Five Aave Pool implementations and four aToken implementations live inside the replay window. A 2,000-block window read at the first block of every implementation era returns the same topic0 set in all five, so no per-era filter branch exists. SparkLend needs none either: it has emitted zero Upgraded events since block 22,000,000.

  SparkLend stable-rate debt is deliberately unscanned, because all 18 stable-debt tokens have emitted exactly one log each in their entire existence (their deployment Initialized) — historical exposure is zero for every address on mainnet, not just for tracked wallets. The forward risk is that governance re-enables it and the gap goes unnoticed, so the 6h SparkLend refresher carries a tripwire: it enumerates the stable-debt tokens from the PoolDataProvider and asserts Σ totalSupply() == 0. A readable non-zero supply fails the tick and pages; an unreadable read is logged and does not, and never counts as a pass. The breach is raised as an outright tick failure rather than as a failed item in the refresher's tally: the tally is escalated on a failure RATE, whose denominator is the venue's tracked-token count, so a tallied breach would go quiet the moment SparkLend tracks a ninth token. An invariant that is either intact or broken cannot be alerted through a ratio.

• erc4626 — why the semantic events, and not just the share transfer. The share Transfer says units moved; the semantic event says what they were worth and whose position they were. Three fields only it carries: assets, the true amount that crossed the vault boundary (shares × the redemption index is the mark, and on a vault with an exit fee the two differ — measured +5.00 bps on fLiteUSD); owner, the position's owner as the vault itself states it, which under a zap or a flash-loan proxy is not the caller; and Withdraw.receiver, the only field that separates an escrowed exit (an sUSDe cooldown, a Syrup queue) from a cash-out. In one measured 5,000-block window at the tip, 20% of Withdraws had a receiver that was not the owner. The address set is registry-derived on both halves — the vault universe and the wallet-tracked tokens — so a newly listed vault or a newly tracked wrapper is covered without a code change; the native-ETH sentinel is the one exclusion, since it is not a contract and native ETH is never ledger-covered by design. Morpho Vaults V2 needs no stream of its own: its events keccak to the same two topics, onBehalf occupies the owner slot, and neither write path takes a fee.

• pendle-router — why the whole stream. A PT purchase is detected today from the PT's ERC-20 Transfer and valued at the oracle mid, which is not what the buyer paid; the wedge is one-directional and was measured at 4–22 bps. The router states the actual consideration. It is scanned with no topic0 filter for two reasons: receiver is a data word (not a topic) on three of the six net-fill events, so the wallet cannot be filtered at the node and the filtering happens in SQL afterwards; and a topic0 list would drop 57% of the router's logs at the coverage floor — the YT swaps, the liquidity events, the SY mint/redeem pair — for a saving of a fraction of ~0.49M rows, while making any later scope a fresh archive backfill instead of a query. Router V4 went live 2,768,286 blocks below the 2025-05-21 coverage floor, so no PT flow this pipeline can ingest predates it and the fill-price correction is fully backfillable.

• fluid-dex — the true amounts behind a smart leg. A Fluid vault with a smart collateral or smart debt side does not state that side in tokens: its LogOperate carries a 1e18 DEX share delta, and the token pair behind it is emitted by the DEX pool in the same transaction. Reconstructing the pair by splitting the shares at the pool's current composition is a pool-average split of an amount that was frequently not pool-average, and it is wrong by the whole position on a single-sided move (measured on our own history: a real withdraw of 0 weETH + 319.9 ETH reconstructs as 38.982 weETH + 277.106 ETH, and a real borrow of 0 USDC + 300.000000 USDT as 221.750993 USDC + 78.091342 USDT). This stream ingests the ten position-changing pool events so the amounts come from the chain instead. The pool-internal Swap and LogArbitrage are deliberately not ingested (they change no position), and neither are the pool's admin/config writes — which is why the ten are enumerated by topic0 rather than recognised by payload shape: at least one config event carries the same three-uint256 payload as a real position leg, so the topic is the only thing that tells them apart. Volume is small: 88 logs per 5,000 blocks at the 2025-05-21 floor and 84 at the tip, about 56k rows over the whole covered window — small, and second only to escrow among the streams this rebuild adds.

  The universe is read, not frozen. getAllDexAddresses() on the DexResolver is the authoritative pool set and it grows — 48 on 2026-08-17, 49 on 2026-08-20 — so a list in code would stop covering the newest pool silently, and a pool missing from the scan surface is a pool with no certificate, which resolves to failed and vetoes the wallet for a reason nobody can trace back to a stale constant. A failed read degrades to an empty set, which the emptiness rule above turns into a withheld stream for that cycle rather than the chain-wide scan an empty address list would otherwise mean.

  Coverage is per pool, and the live scan does not write it. One row per pool, so a vault's smart side can require the certificate of its own pool rather than a blanket one. Like transfers and erc4626, those rows belong to the historical backfill and the new-address catch-up: the live scan starts at the tip, not at the coverage floor, so stamping from there would certify a range nobody scanned and strand the stored range 3.2M blocks above the floor. The backfill also stamps this stream's '*' rollout marker, which it can because the declared set is the resolver's own enumeration; until it runs, the scope is not yet required of any wallet.

  It attributes a vault's smart side. It does not discover direct DEX positions, and it cannot. Every user-facing pool event is address-free, and the pool's user functions take no owner/on-behalf parameter — the position is keyed on msg.sender. A pool event is attributable only by joining it to a vault LogOperate in the same transaction, which names the position NFT and therefore its owner. A direct pool deposit, with no vault in the transaction, has no owner anywhere in the log stream and is un-ingestible from logs at any price, so a direct Fluid DEX position is out of scope and stays out of scope. Fluid smartLending (fSL) is the opposite case and is not this stream's business: it is a transferable ERC-20 with an ordinary Transfer stream, so covering it would be a registry line rather than code.

• escrow — the exits that do not pay out in the same transaction. Some venue exits hand the wallet a queue ticket instead of money: the shares leave at request and the payout arrives hours or weeks later. In between the value belongs to neither the position nor the wallet, and a portfolio that only sees the two ends reads it as a position closed and, days later, a brand-new one opened. This stream ingests the events those queues emit, so a later phase can carry the value across the gap.

  Six classes in four families, all mainnet: the Ethena-style cooldowns (sUSDe, sUSDf), Lido's withdrawal queue, EtherFi's withdrawal NFT, and the two Maple/Syrup queues (syrupUSDC, syrupUSDT). rETH is deliberately not one: its burn pays out of the deposit pool synchronously and reverts if the pool is short, so it is a native-ETH case, not a queue.

  The classes live in code, not in a table (src/lib/portfolio/escrow-registry.ts). Each needs its own decoder — its own topics, its own field map, its own ownership stream — so a row cannot describe one, and a table would imply that adding a row is enough. What can drift, the per-pool addresses, is re-read from the chain instead: by npm run verify:escrow-registry as a release step, and by the ingester itself in one multicall round before it scans, which holds the escrow cursor on a disagreement it can read and leaves every other stream running.

  Not every leg of a class is on this stream, and that is a correctness rule rather than an optimisation. One chain log is one stored row carrying one stream name, so two streams asking for the same (address, topic0) means the loser's reads silently miss it. The Ethena request leg is an ERC-4626 Withdraw on sUSDe / sUSDf — and both wrappers are wallet-tracked tokens, so erc4626 already declares exactly that pair. The escrow registry therefore records, per leg, which stream stores it; the Ethena request names erc4626 and this stream does not re-scan it. Nothing is lost: receiver, the field that tells a cooldown from a cash-out, is on the stored erc4626 row, which is what that stream exists to preserve.

  Several filters, not one address × topic product. The emitters do not share a topic set (Lido's queue emits three events, EtherFi's NFT six, the Syrup withdrawal managers four, the Syrup pool managers one), so one product would claim pairs the stream never asks for. That matters because the disjointness check between streams reads each stream's claim, and an over-wide claim is a false collision waiting for the next stream to land on an invented pair. All four filters are scanned before a range is committed — committing on a partial read would certify blocks the scan never looked at, and afterwards a short scan and a quiet range are indistinguishable.

  Coverage is keyed on the contract that HOLDS the value, not on the contracts that emit. For Syrup those differ: the wallet's shares move to the pool manager, which forwards to the withdrawal manager the events come off, so a counterparty rule keyed on the emitter never fires. Those rows belong to the one-time backfill, never to the live scan (see the loop above). The two Ethena classes earn no row here at all — this stream reads nothing they emit — so their in-transit leg requires the erc4626 certificate on the wrapper instead.

  The Ethena close is a state read, and the reason is measured. unstake(address receiver) pays receiver, and the only receipt is a bare ERC-20 transfer out of the silo whose to is that receiver — the cooldown's owner is not in the log at all. Measured over mainnet blocks 25,749,118–25,799,117: of 127 payouts that amount-match a prior cooldown request, 33 (26%) paid an address other than the request's owner. A leg keyed on the payout receiver would therefore open for one address and close for another about a quarter of the time. The chain does publish a key it can produce on both sides — cooldowns(owner) on the wrapper returns the open cooldown's end time and amount, keyed on the same owner the request states — so the registry declares the close as that read rather than as a log. The cost is recorded rather than assumed away: where a payout went is not observable for this class. When it lands in a tracked wallet it is that wallet's ordinary wallet-token credit; when the owner routed it elsewhere, the chain recorded an instruction the wallet's own book cannot see.

  The in-transit position leg now exists, and the scope is live. {#the-in-transit-leg} The snapshot pass reads a wallet's open queue tickets at the same anchor block as the rest of its positions and records each one as its own leg (venue = 'escrow', position_key = escrow:<class>:<ticket>, in the payout asset's book), so value in a queue is a position the portfolio holds rather than a gap between two ends. escrow is therefore no longer withheld from a wallet's required scopes, and the backfill runner stamps its rollout marker when the sweep completes — the two halves of "this scope is real" landed together, deliberately.

  The mark is the CLAIMABLE amount, re-read every snapshot, never frozen at the requested one. Queues can pay less than they were asked for: 1 of 1,739 matched Lido claims settled 50.5 bps short. A leg frozen at the requested figure turns that shortfall into a phantom movement at the claim and hides the real loss; a re-read leg books it as the negative yield it is, on the position that took it. The reader asks each family the view that answers it — Lido's finalized claimable ether, EtherFi's claimable amount, what a Syrup queue's shares would pay out now, Ethena's open cooldown — and skips a leg it cannot read rather than marking it at zero.

  The leg follows the claim ticket where the ticket is transferable. Lido and EtherFi mint an ERC-721 that can be sold; 6 real Lido claim rights changed hands in ~21 days. Ownership is resolved from chain state at the anchor block for every class, so selling a ticket moves the position rather than leaving a phantom claim behind on the seller.

  A new wallet's reconstructed history carries the leg too. The registration backfill replays each of its grid points against the chain at that block, and the in-transit leg is replayed with everything else rather than filtered out: the queue itself decides, at that block, whether a ticket was open and who owned it, so a ticket requested and claimed before the wallet was ever tracked contributes nothing while one that was live inside the window contributes exactly the days it was live. Without that, a wallet enrolled mid-cooldown would get a reconstructed history with the gap put back in — the source position closing, days of nothing, and the value reappearing on claim.

  The leg is SERVED, and the exclusion that used to hide it is gone. It was written and then withheld from every read behind the portfolio's curves for one release, because the attribution shipped then had no vocabulary for a movement that stays inside the account: it would have read the new leg as a gain for the whole cooldown and an equal loss at the claim. The engine that serves the page does have that vocabulary — the request and the payout are an internal_out and an internal_in sharing one group — so the leg is read like any other and the value in the queue is on the page for the days it is there. The venue guard that enforced the exclusion (v1-read-guard.ts) was deleted with the reader it protected, and no read carries a venue predicate any more.

  The disappearing-position alarm judges it too, which is a change an operator meets on the first tick after that release. The alarm explains a closed leg from the ledger's receipts, and the ledger now holds an escrow: leg's receipts, so hiding the leg would mean an in-transit position that vanished raised no alarm at all. The cost is the other direction: a tracked leg that was mid-transfer at the previous snapshot and whose closing receipt is not in the ledger pages once, and that page is not an incident about the retirement — see the step-1 note in the retirement runbook.

  One class's coverage is not complete, and it is stated rather than assumed away. EtherFi's withdrawal NFT publishes no way to list the tickets one address owns, so the reader takes its candidate ticket ids from the stream's own stored transfer logs and re-checks each one on chain. A wallet whose EtherFi ticket predates this stream's history therefore has no candidate and no leg until the stream's backfill reaches it.

  The leg exists before its two movement records do, and that ordering was the one thing to watch. A queued redemption is a position that moves value within an account: the source position gives it up and the payout gives it back, and both are derived records. A leg written before its pair is a holding with no explanation of where it came from, which is exactly the shape the attribution treats as a gain when it has no receipt — harmless while the leg was withheld from every served read, and not harmless once it is served. The pair therefore had to land before the reader did, and the reconciliation gate that requires every position change to be explained is what held the order. It is what still catches a leg whose pair goes missing.

• The loop (~60s). head = eth_blockNumber, finalized = eth_getBlockByNumber("finalized"), scanTo = head − 64 (a read-consistency margin against a lagging replica, not a finality claim — finality is the line the chain itself reports). Then the reorg sweep, then per stream: cursor-scan (chain_scan_cursors scope ledger:<stream>) from cursor+1 to scanTo via getLogsChunked, decode → resolve block headers (exact per-block, an in-process LRU with BOUNDED fan-out so a wide window cannot burst the node; every log-producing block plus the scan tip, which is what makes the reorg sweep's detection surface a covering one) → write chain_blocks → idempotent write into raw_events (a partition is auto-created before a batch crosses into a missing one) → (singleton streams only) stamp the '*' coverage certificate → write the chain_scan_ranges receipt → advance the cursor. A per-address stream is stamped nowhere in this loop — not transfers, not erc4626, not fluid-dex; their certificates belong to the catch-up and the one-time backfill, for the reason under Coverage certificates below. That order is load-bearing: the cursor is committed LAST, never ahead of either the certified range or the receipt, so a failed step self-heals on the next cycle's re-scan instead of wedging the certificate at a gap or leaving a range certified by nothing. Per-stream fail-soft: one stream's failure logs [ingester/<stream>] ... error and never blocks the others — which is also why a single stalled stream is invisible from the outside, and why the hourly ingester alarm measures every followed stream's cursor against the chain head as well as the process's own age.

  A stop finishes the cycle it is in. SIGTERM/SIGINT set a flag; the loop completes the current cycle, closes the pool and exits 0. pm2 only waits kill_timeout for that, so the process is started with --kill-timeout 120000 rather than pm2's 1.6s default — see the deploy's ingester restart.

  Last in every cycle: continuous derivation. Once every stream and catch-up has committed, the range ingestion newly finished — (the producer's cursor, the ingested tip], the tip being the slowest rolled-out stream's cursor, which is exactly the ceiling the ledger writer clamps to — becomes one continuous derivation job per wallet of the 6h tick's own derivation population that the range can have moved, in one transaction with the producer's own cursor (portfolio:worker:continuous). A wallet whose previous job is still queued gets no second one: that job is widened to the new range instead, so the queue holds one open continuous job per wallet however long the worker was away. "Can have moved" is the derivation's own candidate read (src/lib/portfolio/derive-candidates.ts, intersected with the population inside the statement), a deliberate superset:

  • every address the range's logs name, on every stream the derivation reads — each topic that is a padded address and each ABI-aligned data word that is one. That covers a bare USDC/USDT transfer, an Aave/Spark/Morpho/ERC-4626 movement, an escrow request or claim, a Pendle router fill (whose receiver is a data word) and a Fluid operate made by the wallet itself;
  • the wallets holding a position an event moves without naming them, found through their own stored ledger rows or their latest reading: the position NFT a Fluid LogOperate names (routed operates), every position in a vault a LogLiquidate / LogAbsorb hits (a liquidation names no position), and the request an escrow cancel or partial fill names.

  Its cost does not grow with the wallet count, and one run is time-boxed (PR #949 review round 3), because it runs on the ingester's own critical path — a slow producer delays the next cycle of every stream. The population, and what it holds by id (its Fluid and escrow position keys, from the same two sources), are read at most once every five minutes (INGEST_CONTINUOUS_POPULATION_TTL_MS) and held in the ingester; each cycle then reads only its own range's events and matches the positions they touched against that index in memory, rather than probing every wallet's ledger and latest reading. Every statement of a run shares one budget (INGEST_CONTINUOUS_BUDGET_MS, 20 s: each is sent with statement_timeout set to what is left), and a run takes at most ~6 hours of blocks (CONTINUOUS_MAX_STEP_BLOCKS), so a catch-up after an outage goes a step per cycle instead of reading the whole backlog in one statement. A run that runs out of budget fails like any read: rolled back, the cursor unmoved, the same range next cycle. The price of the held population is freshness at its edges only: a wallet enrolled, or an id-keyed position opened, within the last five minutes gets no continuous job until the next read, and reaches the ledger at the next page load or 6h tick as before.

  Every run that completes stamps the producer's cursor row, an idle one in place, so the row's age is the producer's heartbeat: the ingester alarm's derive-lag arm pages when it has not moved for an hour while the ingester has been up that long. A producer that fails every cycle is otherwise one stderr line a cycle, behind a fresh worker heartbeat and an empty queue.

  The ledger worker runs the jobs, so those movements reach the ledger within about one cycle instead of at the next page load or 6h tick. A job's range is where the producer's cursor happened to be, but it does not start there: it derives from its wallet's tick floor (the tick cursor + 1, or the wallet's pending marker when that is lower), because a derivation anchors every position's running balance on the ledger below its range, and above the tick floor that ledger is not the tick's (the ledger worker). What it still cannot see reaches the ledger on that older schedule, as every movement did before: a close that leaves no log (the Ethena cooldown's claim is a state read), native ETH (no event), an id-keyed position neither the wallet's ledger nor its latest reading named when the population was last read, a stream ingesting behind the rolled-out tip, and a wallet whose job failed until the next job for it succeeds. So a per-wallet "derived through" stamp (R11) cannot be the top of the wallet's last done continuous job alone — every such job starts at the wallet's tick floor, so a done one does leave the wallet's ledger complete to its top, but that top does not move for a quiet wallet. It takes the producer's cursor, lowered below the wallet's job still owed (open, failed, or stopped short of its top), and that is what the stamp does (S4b, PR #959 review SF-2), with the follow record below. What the candidate read cannot see is not lowered for: the stamp can read later than such a movement until the next tick or page load derives it, and for that reason the reading audit never takes the producer's term (it asks for the derivation of the wallet's own range instead). The first cycle on a box starts following at the tip and enqueues nothing; a producer more than ~1 day behind skips ahead to the newest day and names the skipped blocks, which the 6h sweep derives anyway (and which a dirty wallet's next job covers too, starting at its floor); the reorg sweep rewinds the producer's cursor with the stream cursors, so a repaired range is re-enqueued. Fail-soft like a stream: a failure is one [ingester/derive-jobs] line and the next cycle enqueues the same range.

  The follow record (S4b, PR #959 review SF-2). The producer's cursor speaks for a wallet only over the ranges it scanned with that wallet in its population, and two things break that: the population is held for five minutes, so a wallet that joins it (an enrolment finishing) is scanned only from the first run that holds it; and a producer more than a day behind skips a stretch no one's scan covers. So the producer records, per wallet, the block after which it has scanned every range with the wallet in its population: a chain_scan_cursors row, portfolio:derive:v2:<chain_id>:<wallet>:followed-after, in the per-wallet wipe family. The first run over each newly loaded population reads the chain's records once and, in the run's own transaction, inserts one for each wallet that has none (at the block its range starts after) and deletes those of wallets it no longer holds; a run that skips a stretch raises every record above it. Every other run writes only its own cursor, as before, so the cost per cycle is unchanged (the first run after this release writes one record per wallet of the population, once). The stamp takes the producer's cursor for a wallet only where its record is at or below the block the wallet's own terms already reach (no stretch in between went unscanned for it) and the wallet's registration replay is not queued or running, and it caps the term below the wallet's owed tick-range job and its pending marker. A job is owed while it is queued, running or failed, and a partial one until the wallet's own terms reach its top. A job whose merge stopped short of its top (the ingested tip fell under it: a stream rolled out while its cursor was behind, or a stream cursor rewound by hand) records the rest in the wallet's pending marker, from the merged top + 1 (the tick's own rule for a clamp; S4b, PR #959 review round 2, B1), which the next tick widens down to. The term stops below the job itself until the wallet's own terms reach the job's top (round 3, B2), whatever becomes of the marker: a marker records only the bottom of what is owed, and a tick whose own top stops below the job's (the stall that clamped the job clamps the tick too) clears it as its merge covers it, which on the marker alone let the stamp pass a movement between the two tops.

• New-address catch-up (the "USDS" runbook). An address on a per-address stream not yet stamped live (a newly added curator vault, Pendle roll, or synced reserve — or any address after a coverage reseed) has its history backfilled from the 2025-05-21 floor to that stream's live cursor via the archive RPC, using that stream's own topic set, then stamped live. It is drained INCREMENTALLY: scan + write per ~50k-block window (bounded memory, never the whole history in one in-memory array), interpolated block timestamps (2 boundary reads/window), advancing a backfilling resume marker after each committed window and capped to a few windows per address per cycle so the catch-up never starves the live scans. Ownership is live-ONLY, so an address whose catch-up failed or is mid-drain RESUMES from its confirmed tip next cycle (never stranded, never restarted). It stays in the live scan throughout (idempotency makes the overlap harmless). For wallet-token the covered address is a WALLET, so the same loop narrows its filter in the topic slots rather than in the address key — one parameterised catch-up, not two, because the resume marker and the completion stamp are exactly what a second copy would drift on.

  Two budgets, and the split is deliberate. The per-cycle address budget is SHARED across the per-address CONTRACT streams, so a fresh deploy or reseed cannot stampede and adding streams does not multiply the loop's archive traffic. wallet-token gets its own (INGEST_CATCHUP_MAX_WALLETS, default 3): a contract enters the queue when a vault is listed, on nobody's clock, while a wallet enters it when a person signs up and watches a building chart until it drains. Sharing would queue that signup behind however many addresses a registry happened to add that week, each of which walks the store's whole ~3.25M-block depth. A wallet's own catch-up walks only that wallet's history window (~1-2 minutes for the rolling 30 days), which is the other half of why the two queues are separate: the cheap one must not wait behind the expensive one. The two run sequentially inside one cycle, so the archive load is additive rather than doubled in a burst. See Deployment → a new wallet's enrolment backfill for the wait and when it is a symptom.

  Who owns a stream's history, and how the loop knows. For transfers the tip loop owns it unconditionally — that stream's declared address set is knowingly incomplete, so it has no '*' row and never will. wallet-token is the same case for a different reason: its declared set grows on every signup, so "the set is complete" could never be true of it, and enrolment on registration is exactly this loop doing its job. For a stream whose address set IS fully declared by a registry (aave-token, spark-token), the one-time backfill runner owns the history; the tip loop stays out of it until that runner has finished, then treats a newly listed reserve as an ordinary new-address catch-up. Without the split, restarting the ingester would set a 60-second loop racing the backfill runner over 3.2M blocks of archive reads for 170 token addresses — duplicated work with a duplicated bill, since both writes are idempotent. "Finished" is read two ways, either of which opens the gate: the '*'rollout marker the runner stamps when its last declared address commits (which a human can also force with one row), or simply any address on the stream already live — a runner pass writes backfilling rows on the way (a [floor, floor] ownership mark up front if it starts at the coverage floor, then the blocks it has actually committed) and live only at completion, so one live address means it completed and the pending ones are new. Only a floor-starting pass writes the up-front mark, because a floor-bottomed marker written by a run resuming a million blocks above the floor describes nothing that happened. The mark is an anti-race claim and nothing more; what the completion rules below read is the recorded blocks.

• Coverage certificates. event_coverage holds a contiguous [from_block, to_block] range per (stream, address) ('*' for the singleton / chain-wide streams, one row per pool for fluid-dex). The rule the writers enforce (mergeCoverage): never stamp a range you did not scan — a gap between the stored range and an update throws. Only a live row is a completeness certificate; backfilling is a progress/ownership marker. The stamp is safe under the two writers the cutover runs at once (ingester + one-time backfill): the durable merge is atomic in SQL (LEAST/GREATEST/sticky-live), the JS gap check is the honesty gate. For a per-address stream, an address row's to_block is the historical catch-up completeness mark; the live freshness bound is that stream's shared ledger:<stream> cursor, which the live scan (not the per-address row) advances.

  On a per-address stream the '*' row means something the per-address rows cannot: the declared address set is complete. It is written only for a stream whose set is fully derived from a registry, and never for transfers, where it would be a false certificate — that stream is knowingly missing the bare wallet-held tokens, which is the gap the wallet-token stream exists to close.

  A per-address certificate is never written by the live scan, on any per-address stream — bounded address set or not. The live scan starts at the tip (its cursor, or head − 64 on a stream that has none yet), never at the coverage floor, so a row it stamped would read [scanTo, scanTo]. That range is literally truthful and still ruinous: its from_block then sits ~3.2M blocks ABOVE the floor, where every later backfill window that would extend the range downward is a gap mergeCoverage refuses — forever, on every re-run — while the catch-up reads the row as done and skips the address for good. It would look exactly like a certificate doing its job while making the floor permanently unreachable. So transfers, erc4626 and fluid-dex are all owned the same way, by the one-time backfill and the new-address catch-up, both of which start AT the floor. A bounded set buys exactly one thing here, and it is not this: it earns the '*' rollout marker above, which the backfill runner stamps when the last address of the declared set commits. Whoever adds the next per-address stream inherits the rule unchanged.

  What that means for an operator, because the healthy state looks alarming: a new per-address stream whose set is declared complete (erc4626, fluid-dex) holds no event_coverage rows at all between the release that names it and the one-time backfill that certifies it — not a partial set, none, however many live cycles run. The catch-up is gated out of it as well ("who owns a stream's history", above), precisely so it cannot race the runner. That is the correct reading of not yet rolled out (see the rollout marker), not a stalled ingester.

• Adding a stream is served-neutral, and there is exactly one thing to keep that way. Every read of raw_events on the served path is stream-scoped, so new rows in a new stream are invisible to it. The exception is the two places that iterate the stream name list rather than issuing a scoped query — the 6h refresher's ledger flow bound and the page-load gate's ledger tip — where a stream with no cursor row reads as block 0 and would drag the bound or the tip to the floor, which books nothing for the whole window between a release naming a new stream and the ingester restart that starts following it. Both iterate V1_LEDGER_STREAM_NAMES (the streams the served derivation actually reads), not the full declared set; a stream joins that list only when the served derivation gains a read of it, and a unit test pins the membership against that file. The split shipped with the stream framework itself (#631), ahead of the streams that rely on it — this page is simply where it gets written down.

The native-ether block feed

Native ether emits no log, so none of the thirteen eth_getLogs streams can see a plain ether transfer, the ether a transaction carries into a contract, or the fee a transaction pays. Until issue #966 the ledger had no record of any of it: the ether leg was a derived-evidence leg (coverage note CN-3, booked 0 by construction) and every reading of a wallet that paid gas disagreed with the ledger, up to four balance adjustments a day on an active wallet. Now native ether is an ordinary idle wallet-token leg with receipts, from three sources, each with a cost that does not grow with the number of tracked wallets.

• The block feed (native-eth stream; src/lib/portfolio/streams/native-eth.ts, src/lib/portfolio/native-feed.ts). One step of every ingester cycle, after the log streams and before the continuous producer: for each block between the stream's cursor (ledger:native-eth) and the cycle's scan tip, at most two provider calls per block: eth_getBlockByNumber(block, true) (every transaction's from, to, value), and the receipts (status, gasUsed, effectiveGasPrice, blob gas) of the transactions a tracked wallet needs (one it sent, a value transfer touching it, or every one of a block a tracked wallet produced), asked for by what they cost: none needed, no call; one needed, eth_getTransactionReceipt; two or more, eth_getBlockReceipts(block), one call for all of them. The filter is IN MEMORY against the cycle's tracked-wallet set (the wallet-token stream's declared set, read once per cycle; 50,000 addresses is a small Set). About 7,200 blocks a day, so at most about 14,400 calls a day (and one a block none of the tracked wallets took part in), whatever the population: a unit test holds a cycle's provider calls (and its database statements) equal at 5 tracked wallets and at 50,000. It writes synthetic raw_events rows, address the native-ether sentinel (0xeeee…, which no contract can be, so no real log collides with one), topics hashed from stated signatures, the parties padded at topics 1 and 2 exactly as a Transfer pads them (so the continuous producer's candidate read and the reorg sweep treat them like any log):

  row	written for	data
  NativeEtherTransfer(from, to, value)	a SUCCESSFUL transaction with value > 0 whose from or to is tracked (a creation's recipient is the contract it creates; a reverted one moved nothing)	the value
  NativeEtherFee(payer)	EVERY transaction a tracked wallet sent, approvals, zero-value calls and reverted ones included	the fee (gasUsed × effectiveGasPrice, plus blobGasUsed × blobGasPrice for a blob transaction), gas used, price, status, blob fee
  BeaconWithdrawal(address)	a consensus-layer withdrawal credited to a tracked wallet (the block's withdrawals, gwei stored as wei), under a deterministic synthetic hash	amount, validator index, withdrawal index
  NativeEtherPriorityFees(address)	a tracked wallet that is the block's fee recipient (a solo staker's vanilla block): the part of every fee not burned, Σ gasUsed × (effectiveGasPrice − baseFeePerGas), under a synthetic hash	the amount
  NativeEtherInternalTransfer(from, to, value)	NOT the feed's: ether a contract paid inside a transaction, landed on demand by the reading audit (below)	the value

  Where a row sits in its block, and why. A synthetic row has no log index of its own, so each gets one above every real log's (a block holds a few thousand): priority fees at 890,000,000, withdrawals at 900,000,000 + their position, a listed internal transfer at 950,000,000 + its ordinal in its transaction, the feed's rows at 1,000,000,000 + 2 × the transaction's index (its fee one above), and a wrap's ether leg at 1,600,000,000 + the Deposit's own index (an unwrap's at its Withdrawal's own index), whether or not a record paid it: the key is the event's, never the record's the event consumed, because the listing lands a contract wallet's record only at the next reading, after the wrap was derived, and a landing that MOVED a row which had closed the leg at zero left that closing row behind as a terminal the range merge refuses to delete while it wrote the new one, so the wrap stood twice for good (PR #967 review round 2, B1). A row's key is a fact about the chain, never about the wallet that read it: a stored row is shared by every tracked wallet it names, and raw_events dedupes on (transaction, log index) alone, so an internal transfer takes one index, its ordinal's, whichever of its two parties' audits lands it; keyed by its direction for the audited wallet, one payment between two tracked wallets (a Safe paying its owner) was landed twice and booked twice on both (PR #967 review, B1). Two internal transfers of different transactions in one block can share an index (each is ordinal 0 in its own); the ledger orders them by transaction hash, as the page's loader does, and the derivation closes the leg's running balance in that same order, so the block's last row always states the block's balance. The feed's rows keep true transaction order among themselves; the rows whose position is unknown go where they harm nothing for the case each exists for (withdrawals, priority fees and the listing's internal transfers early, which can only raise the running sums before their true position; a wrap's ether leg late, and an unwrap's at its real log's index, early), and the block's end is exact either way. The one ordering this cannot make safe is a tracked contract wallet paying ether out by an internal call that a feed row funded earlier in the same block: its running balance dips below zero between the two rows and is back at the block's end. Occupancy, the served row and the engine all read a leg's quantity at the end of a block, and the ether leg is outside the negative-balance terminal (the wallet book), so the dip is a quantity-drift line in the derivation's anomalies and nothing a user sees.

  What a block read refuses. The body and the receipts are separate answers about one block, and a reorg between them or a provider serving two views would pair one block's transactions with another's receipts, so every receipt must name the block's own hash and sit at its transaction's index, and (block receipts) the counts must agree. A disagreement throws, the cursor holds, and the next cycle reads the block again. A block whose block receipts fail that check three times in a row is read with its transactions' own receipts instead, which the check covers one by one: a provider answering one block inconsistently would otherwise hold the cursor, and after the rollout marker the ingested tip of every wallet, for good (PR #967 review, S3; the step's log line names such a block). An endpoint without eth_getBlockReceipts (the method refused, never a transient failure) switches the process, once, to eth_getTransactionReceipt for every block: its cost follows the tracked wallets' own activity in the block, still never the population.

  Coverage, per wallet. The feed can speak for a wallet only over the blocks it read with the wallet in its set, so each wallet gets ONE event_coverage row (stream native-eth, address the wallet), stamped the first time a step reads with it, at that step's first block, BEFORE the read (a failed step re-reads from the same block, so the row never claims a block nobody read with the wallet): [from, from] live, with the stream's cursor as the live bound, exactly as wallet-token's per-wallet rows lean on its shared cursor. Fifty thousand new wallets are one INSERT … SELECT unnest(…), not fifty thousand round trips. The enrolment read keeps every wallet holding such a row in the set, so the set never shrinks under a certificate. With nobody tracked a step reads nothing and still advances the cursor (an empty scan and its receipt, no call): the rows are relative to the tracked set, so no row is the complete answer, and the first wallet to enrol is covered from the next step's first block rather than behind a catch-up of the whole empty stretch. A step is bounded (INGEST_NATIVE_MAX_BLOCKS, 64 blocks), so an ingester that was down catches up over a few cycles instead of stalling one; INGEST_NATIVE_CONCURRENCY (4) bounds its parallel block reads, INGEST_NATIVE_CU_PER_SEC (2,000) is the throughput budget they spend (a token bucket of the provider's throughput compute units, where eth_getBlockReceipts weighs 500 against 20 for the other two methods, so a catch-up never bursts past the plan's per-second cap; the sizing), and INGEST_NATIVE_FEED=0 turns the step off. The hourly ingester alarm watches its cursor like every stream's (its feed-lag arm walks the full stream list and keeps a stream that has a cursor), so a feed that stops advancing pages. The reorg sweep protects it like any stream: the step writes the headers of the blocks that produced a row plus its tip, a scan receipt over its range and its rows' block hashes, so a replaced block's rows are deleted and the cursor rewound by the same sweep.

  Its history is a one-time run, not a live scan (scripts/ops/backfill-native-eth.ts, the release's gated step): the same reads per block over the enrolled wallets' window, ONCE for every enrolled wallet at the same time, lowering each wallet's coverage start over the blocks it read, window by window (resumable, idempotent), spending its own throughput budget (--cu-per-sec, 3,000 by default). O(blocks): about 216,000 blocks for a 30-day window, whatever the number of wallets.

  Gated like every rebuild stream. The derivation reads the stream (DERIVE_STREAMS) and its rows move the served ledger only once the operator's '*' rollout marker exists, which is stamped after the stream's cursor does (the release entry); before that, a missing cursor never reads as caught up.

• WETH9's wrap and unwrap (wallet-token stream). deposit() emits only Deposit(dst, wad) and withdraw() only Withdrawal(src, wad): a WETH balance moves with no Transfer, and an unwrap's ether arrives by an internal call no block states. Both index the wallet at topic 1, so they ride wallet-token's topic-1 pass at no request of their own, and the stream claims them from WETH9's own address only (topicAddresses: a Curve gauge's Deposit(address,uint256) has the same topic0 and is dropped at the write, at every scan site). Widening the topic set resets wallet-token's certificates (the release entry's gated step), since every per-wallet row below the change was earned without these two topics.

• Ether a contract pays inside a transaction: on demand, from the reading audit (the explanation ladder). A swap into ether, a withdrawal paid in ether, a bridge release: no block states these without traces, and traces per block are the one cost that does not scale. The six-hourly reading finds the gap the feed leaves, and the audit's explanation asks the provider ONCE per break and side for that wallet and that block range (alchemy_getAssetTransfers, categories external and internal, the wallet as sender and as receiver), lands what it names as NativeEtherInternalTransfer rows, re-derives and re-audits. Each row's key is the transaction and the transfer's ordinal there, as the provider states it (<hash>:internal:<n>), so the payer's audit and the payee's land the same row; a transfer listed without an ordinal has no such key and is not landed (a source that could not speak for it). Its block hash comes from the ledger's stored header (chain_blocks) where there is one, else from the endpoint, and a block neither states is a missing source too, never a failed audit job. Its cost follows the breaks, never the population: a wallet whose readings agree asks nothing. WETH9's Withdrawal is the one integrated venue event that states ether paid to the wallet, and the wallet book books it directly (below); an Aave or Spark ether withdrawal through the gateway and a router's ether payout name the recipient in no event (it is a call argument), so they are the listing's.

The requirement map, and what a coverage certificate now certifies

A coverage certificate used to be a statement about the pipeline: some scope scanned this window, so the tick counted. It is now a statement about one wallet: every scope that wallet's holdings depend on reported an outcome, and the wallet is certified only when all of them did. The three parts are deliberately in three different places, because they are changed by three different people at three different times.

1. What a venue needs (SCOPE_REQUIREMENTS, in code, versioned). A leg's venue maps to the ledger streams that must be certified for it: lending legs need the Pool and the position token, a vault leg needs the vault, a Pendle leg needs the router and the PT, a smart Fluid leg additionally needs the DEX pools, a bare-token leg needs the wallet-keyed stream. Editing this is a reviewed code change and ships with the adapter that needs it.

2. Whether a stream is ON (event_coverage, data). A stream enters a wallet's required set only once the rollout marker is stamped for it — and the marker governs only the streams this rebuild adds. The six that shipped before the convention are required unconditionally; dropping one of them for want of a marker row would un-require the certificate that carries every lending, vault and PT leg the product serves today. Enabling is therefore a data change with no deploy, which is what makes a rollout possible inside a backfill window.

Who stamps it depends on whether the stream's address set ever finishes. For the five whose set is derived from a registry, the backfill runner stamps it when the last address commits. The wallet-keyed stream's set is the tracked-wallet list and never finishes, so there is no completion event to hang a stamp on: an operator writes that one row, once the enrolled wallets' catch-ups are live. It is the same one-statement enable path, used as designed rather than as a workaround, and it must be done — the wallet-keyed stream is what puts a bare-token holding under a requirement at all, and a wallet with no requirement certifies vacuously. Marking it on early is safe in one direction only, and it is the safe one: every wallet whose own history is not yet indexed is held back until it is.

3. What is switched OFF (DEFERRED_SCOPES, in code). A named scope is removed from every wallet's required set regardless of coverage. It is empty: it held the escrow scope for one release, while the in-transit leg had no writer, and emptied when that writer landed. Turning a scope off is a reviewed code change; turning one on is not. The asymmetry is the point: withholding a number is recoverable, publishing a wrong one is not. Deferring is also the only safe way to withhold — a scope that is off this list but has no rollout marker is not withheld, its requirement is dropped, and the wallets that needed it certify over nothing.

The wallet's own bare-token requirement is unconditional in the formula, not added by whoever calls it. Everything else is derived from what the wallet holds, and three ordinary situations leave that empty — a wallet with no positions, a wallet whose holdings are read from something written later in the cycle, and a wallet all of whose scopes are deferred or not yet rolled out. An empty requirement set is not a small requirement, it is no requirement, and it certifies vacuously. So the conjunction reports "this wallet required nothing" separately from "this wallet is certified", and a consumer that must not act on a vacuous certificate can see the difference.

Per-scope outcomes, and the per-wallet veto. Each required scope resolves to scanned (the derivation covered this window), at-tip (nothing below the bound to derive — the cursor is already there), skipped (neither), or failed (no live certificate reaching the window start — never absent, because a requirement nobody registered is a requirement nobody checks). A skipped or failed scope vetoes that wallet, never the batch: one uncovered vault must not freeze every user's certificate. History is checked before freshness and its failure is terminal — a fresh tip does not fill a hole underneath it.

What a veto does today, stated so nothing is read as already wired. The wallet's scan certificate stops advancing, so it stops gaining ground; that is the whole effect. No page reads the conjunction, and no published number is withheld on it. Two consumers arrive later: the completeness stamp on the rebuilt index, and the whole-wallet withhold in the rebuilt engine — the second is what would otherwise draw a chart at zero over an unindexed source, and it lands with the release that switches the portfolio reader over. Each verdict carries the block its scope actually reached, and the certificate carries the minimum across them, because in the mode production runs the tick's own derivation stops at the ingester's tip rather than at the block the certificate is being asked about; a consumer must compare against that figure, never assume the two are equal.

Two certification sources, kept apart. History is certified by event_coverage (status = 'live' and from_block at or below the window's start block; to_block is never read, because on a per-address stream it freezes at catch-up completion). Freshness at the tip is certified by the scan itself — the refresher's own per-scope outcome for the streams it derives, and the ingester's ledger:<stream> cursor for the streams only the ingester follows.

Enabling a stream after the history build is a data change PLUS a re-derivation. The marker is global per stream, so stamping it makes the scope required for every tracked wallet in the same instant — including wallets already marked complete, whose index holds no rows from that stream and which nothing re-derives from below their cursor. Removing an entry from the deferred list has identical consequences. In both cases the affected wallets must be re-derived over the campaign's full range, with the range stated explicitly: a run that resumed from those wallets' own cursors would start above the range and derive nothing.

Reorg repair and the scan receipt

A coverage certificate says which blocks were scanned. It does not say which chain, and it cannot say whether the response was complete — a truncated eth_getLogs that returns HTTP 200 with half the range advances the cursor exactly as smoothly as a complete one. Two tables close that, both written by the ingester and neither read by the app.

• chain_blocks — which chain. One row per log-producing block the live scan read — plus the scan tip, every cycle, whether or not it produced a log: hash, parent hash, timestamp. It costs no extra RPC: the block timestamp every event row needs comes from a header read, and the header carries the hash. Blocks at or below the chain's finalized head are stamped finalized_at and never re-read.

  The tip anchor is what makes the surface covering rather than incidental. This table is the sweep's working set, so a height with no row here is a height whose replacement is never compared. With only the log-producing blocks stored, a reorg at a quiet height — one that emitted nothing we track, which is most heights — was undetectable, and if the block that replaced it carried a tracked log, the cursor had already passed that height and nothing re-scanned it: a log on the canonical chain and in no row of ours. A reorg at or below the scan tip necessarily replaces the tip too, so one stored hash per cycle makes every such reorg visible. Above the tip there is nothing to miss — the cursor has not reached those blocks, and the next cycle scans them on whichever chain won.

  One endpoint owns which chain the live path is on. The headers stored here are read from ETHEREUM_RPC_URL — the endpoint that served the logs beside them, and the one the sweep verifies against. Storing one provider's header next to another's rows and falsifying it against a third view would report a reorg at every height whenever the two configurations differ. The archive endpoint is used only where no hash is stored or compared (the per-address catch-up's interpolation timestamps).

• chain_scan_ranges — what came back. One receipt per scanned range per scope: log_count, block_count, a log_digest (sha256 over the logs' sorted (block_number, log_index, block_hash)), the head_block and finalized_at_scan at the time, and the provider host that served it. A receipt is checkable — re-scan the range, re-digest, compare — where a cursor position is not. Receipts accompany cursor advances: the live per-stream scans write them, and the per-address transfers catch-up does not, because its completeness certificate is its event_coverage row.

• The optional second-provider cross-check. With LEDGER_VERIFY_RPC_URL set, every live range is re-scanned against that second archive-capable provider and the two digests are compared before the receipt is written; a match stamps verified_by, and a mismatch holds the cursor (the range is never certified, because at least one of the two responses was incomplete and nothing here can tell which). A difference is escalated only after the mirror's own head is confirmed to be past the range: a provider that has not reached the range never served it, so a short answer there is an absence rather than a disagreement, and treating the two alike would let an ordinary replica lag hold every cursor indefinitely with no automatic recovery. Unset, receipts are written unverified and the digest degrades to single-provider self-consistency — the ingester logs that once per process so the state is visible rather than assumed.

• The repair, run before any scan each cycle, in two phases. Detect: re-read the headers of the stored blocks the chain has not finalized. Where a hash still matches, nothing happens; where it differs, the block at that height was replaced. Every chain read happens here. Repair: for all the replaced heights at once, and all predicates — delete every receipt whose range covers one of them, rewind the cursors those receipts named plus the two the caller names (the cron's ledger dirty tip and the market-impairment writer's own cursor, which earns no receipt because it scans no chain range), delete the raw_events rows at those heights whose block_hash is not the canonical one (rows with no hash — everything written before migration 082 — are never deleted on that evidence), and store the canonical headers last. The next cycle re-scans from there and the next cron tick re-derives the wallets in it.

  The repair is one transaction, and its statement order says the same thing twice. Every prod deploy restarts the ingester, so a cycle killed in the middle of a repair is a per-release event rather than an exotic one — and the header rewrite is what makes a repaired height look repaired to the next sweep, so anything still owed after it is owed for ever: cursors that were never rewound leave a certificate spanning a range whose orphaned logs were never replaced. Wrapping the writes in one transaction makes a kill leave either the old state, which the next sweep detects again, or the whole repair; the chain reads stay outside it so a provider's bad minute is never held across a connection. The order is the belt for that brace: every cursor rewind lands before any header is rewritten, so even a repair that somehow committed halfway leaves a rewound-but-unstamped block, which only re-scans a little more.

  The rewind carries a margin, and the margin is not caution — it is the shape of the detection surface. Below the tip, the heights we hold are the log-producing ones, which are sparse; a reorg replaces every block from its root upward, but the lowest replacement we can see is the lowest height we happen to hold, which can sit above that root. Rewinding to exactly one block below it would leave the replaced blocks in between un-re-scanned, including any transaction the reorg moved down into one of them. So the rewind goes 64 blocks further back, floored at the finalized head (a finalized block cannot have been replaced, so re-scanning under the line buys nothing). Re-scanning extra blocks is idempotent and costs one getLogs window; missing them is permanent.

backfill-event-ledger.ts (the historical stream backfill runner)

scripts/backfill-event-ledger.ts fills raw_events for a stream from the 2025-05-21 coverage floor (block 22,527,558, a literal — see below) up to where the ingester's live cursor has reached, so the ledger has depth below the point the ingester started following live. Run attended, one stream at a time, after the PM2 ingester is up. Its rules live in scripts/lib/ledger-backfill.ts and are unit-tested there.

Ownership split. The one-time backfill owns [floor, cursor-at-start]; the live ingester owns (cursor, head]. Neither writes the other's range, so where they meet is a contiguity property rather than a timing one — the tip loop may have been up for three weeks or three minutes and the check reads the same. Both writers are idempotent regardless, so an overlap costs archive reads and nothing else.

Resumable, and what an explicit range means. Each stream sweeps bottom-up in bounded windows. Every window commits in the order events → chain_scan_ranges receipt → resume cursor (chain_scan_cursors scope ledger:<stream>:backfill), so a crash re-scans that window rather than skipping it and the cursor is never ahead of the receipt that makes its range checkable. A run with no --from resumes from that cursor; a run with --from sweeps its whole range, cursor or no cursor, including a cursor at or above the range's top — the cursor is a resume hint for an interrupted sweep, never a floor on a deliberate re-run. --to alone only caps the top. --budget N bounds the run to N windows per stream and is a planned stop, not a failure: it stamps no completion and the next invocation continues.

The floor is a literal. Every from_block in the rebuild is 22,527,558, the rollout rule is written from_block <= 22527558, and the reader compares against that literal. Resolving a timestamp to a block here could only ever produce the same number or a certificate that does not line up with the rule that reads it, so the runner uses the constant and refuses a --from below it.

And it is not the floor a DERIVATION starts at. Two constants, deliberately different numbers, both literals for the same reason:

	block	owns
LEDGER_COVERAGE_FLOOR_BLOCK	22,527,558 (2025-05-21)	the raw event store: protocol-wide stream sweeps, their from_block, rollout markers, acceptance bands
LEDGER_DERIVATION_FLOOR_BLOCK	24,136,053 (2026-01-01)	the ABSOLUTE bottom of anything derived, and the floor a wallet gets when it has none of its own
accounts.history_floor_block	per wallet (migration 103)	THAT wallet's history: its replay window, its bare-token catch-up start, the enrolment gate, the re-derivation range, its completeness certificate, the reconciler's two figures

The portfolio view supports 2026 onward, so nothing below the derivation floor is ever derived, valued or served — and since the rolling window shipped, each wallet's own history starts at or above that floor rather than exactly on it. The per-wallet number is stored rather than recomputed precisely because six mechanisms in three processes read it and the certificate is an equality-sensitive comparison against it; the one reader is src/lib/portfolio/history-floor.ts. A wallet with no recorded floor (added before migration 103, or one whose boundary block could not be resolved) reads the derivation floor, which is the behaviour every wallet had before. The store keeps its deeper certified depth because it is already ingested and because widening the served window later is then a constant plus a rebuild rather than a re-ingestion. 24,136,053 is the first mainnet block at or after 2026-01-01T00:00:00Z (its own timestamp is 2026-01-01T00:00:11Z; the block below it is 2025-12-31T23:59:59Z).

Completion, and the rollout marker. A stream is stamped live only at completion, never mid-sweep, and from the bottom the receipts reach, not from the bottom this invocation happened to start at: a sweep broken across several runs — which is what --budget is for — has a last run starting wherever the resume cursor left off, and the certificate is a claim about the range, made by every run together. When the receipts have a hole the bottom falls back to what this run swept, so no claim ever reaches past a gap. For a stream whose scope is marker-governed the runner then stamps the (chain_id, stream, '*') row live from the floor, which is what turns the scope on. It refuses to stamp it when any declared address is still short of the floor or when chain_scan_ranges has a gap over the certified range, and a refusal fails the run: a marker ahead of its own coverage certifies every wallet's conjunction in one instant over an index with no rows in it.

That refusal is inside the same transaction as the stamps, and the set it checks is re-read. The order is: write the per-address coverage rows, re-read the stream's declared address set from the registry, check the stored certificate against that set, and stamp the '*' marker only if nothing is missing — then commit. Checking after the commit instead would be a query reading back the rows the same transaction had just written, which cannot fail; the registry read is the one input the transaction did not produce, so it is what makes this a check. An address the registry gained while the sweep was running is therefore caught (the Fluid DEX pool set moved 48 → 49 inside three days), and a registry read that comes back empty is refused rather than treated as "nothing to check". A refusal rolls the promotion back in full: no coverage row, no marker, no live handoff, non-zero exit.

And the same run refuses again on the next attempt, which is the half that makes the first one worth having. A rolled-back promotion leaves the imported events, the scan receipts and the resume cursor exactly where they were — deliberately, because they are evidence and re-collecting them costs the same archive bill twice. So the obvious next move after a refusal is to run the command again, and by then the address that caused the refusal is in the run's own declared set: nothing in the check above would notice, because it compares the registry with what the run just wrote and the two now agree. The second condition closes that: a run may only mark an address complete if that address's history over the certified range was actually imported. The evidence is the union of two things — the blocks this run committed, and the blocks the campaign already recorded for that address — and they have to meet without a hole. An address that appeared after the campaign swept the range has neither, so a re-run refuses for the same reason as the first one, by name, and keeps refusing until the history is imported.

Blocks, not intent, and that distinction is the whole gate. The record the check reads is written from the windows a run actually committed, as it commits them, including when the run dies partway. The up-front [floor, floor] ownership mark is deliberately not evidence: it is written before the first window, so it says a pass began at the floor with the address declared — a pass that then covers 1.5% of the range leaves exactly the same mark as one that covers all of it. (This is the same reading the tip loop's own catch-up planner applies to that row: a to_block at or below the floor means the floor block itself was not scanned by whoever wrote it.) The practical consequence for an operator: the repair has to finish. A re-sweep capped by a window budget, killed by an RPC error, or stopped with Ctrl-C records only the blocks it covered, and the promotion still refuses on the rest.

The repair itself is the one the design already has for a new address: the tip loop's per-address catch-up sweeps it from the floor and certifies it when it finishes (for a stream the runner still owns, an explicit --from <floor> re-sweep run to completion does the same thing), after which the promotion accepts it.

Which is why that record may only ever extend one, never create one. The gate is not its only reader: the tip loop's catch-up planner reads the same row to decide where a new address's sweep starts, and it reads the row's top alone. A record created for an address that has no row would be bottomed wherever the run resumed — on an ordinary resumed run, millions of blocks above the floor — and the catch-up would then start above it, reach the tip in a single cycle and certify the address complete from the floor over a range nothing ever scanned for it: the same false claim, arrived at through the repair this paragraph prescribes, and the one shape the gate cannot catch, because by then the row is live from the certified bottom (see the paragraph below on what it trusts). So a run records progress only for an address that already has a row — which on the one pass that may enrol, the floor-starting one, is every declared address, since the ownership marks are written before the first window. An address with no row is left with none, which is exactly the state the tip loop is built for: pending, swept from the floor, certified only once it has actually been scanned. The run names any address it skipped for this reason in its log, and the promotion keeps refusing it until the history is imported.

What the check trusts without re-deriving it, written down rather than left to be found. An address already certified live from the coverage floor passes this condition on its stored row alone. That is deliberate: such a row is a completeness claim an earlier accepted promotion already published and the product is already serving, so refusing to re-stamp it would not un-publish anything — it would only block the re-promotion that could correct it. A wrongly live row is the row-count and receipt-contiguity conditions' to catch, not this one's. The narrower carve-out this replaces — treating an in-flight catch-up as enrolment — is gone: an address whose catch-up is still running has recorded only the blocks it has reached, so the promotion refuses it until the catch-up completes and flips the row to live.

A run that does not claim completion is excused, not refused. An explicit --from A --to B is a hole repair and a --to below the stream's target is a slice: neither is evidence that the stream is complete, so neither evaluates the marker, both stay green, and both say so in the log. Such a run also leaves the live cursor where it was — walking it forward on a repair would leave the blocks between the old cursor and A owned by no writer at all. A refusal is reserved for evidence that contradicts a claim the run actually made, because inside a campaign whose rule is any condition missing = stop, a refusal that fires on a healthy run is how an operator learns to read the next one as noise.

Two streams are never stamped by the runner, both because their address set is open and so has no completion event to hang a marker on: transfers is required unconditionally and has no '*' row by design (a marker reading of it would un-require the one scope that is actually certified today); wallet-token's set is the signup list, and its row is the operator INSERT described under the rollout marker. Every other stream the rebuild adds — escrow included — is stamped here on completion. Withholding a marker is only the safe direction while the scope is also on the deferred list: a stream that is un-deferred and marker-governed with no marker has its requirement dropped from every wallet, not vetoed, so those wallets certify over a scope nothing ever certified. The two switches move together or not at all.

Enabling a stream AFTER the history build obliges a re-derivation

The marker is global per stream, so the instant it lands every tracked wallet's conjunction certifies — including wallets stamped complete long before, whose v2 index holds no rows from that stream below their derive cursor. Re-derive the affected tracked wallets over [24,136,053, arm_block] with the explicit range, not a resume: those wallets are complete by construction, so a resumed run starts above their cursors and derives nothing. The same sentence applies to the hand-UPDATE enable path and to removing a stream from the deferred list. The runner prints this obligation whenever it stamps a marker while a recorded arm block exists.

ETHEREUM_ARCHIVE_RPC_URL=<archive> DATABASE_URL=<...> \
  npx tsx scripts/backfill-event-ledger.ts --stream erc4626 \
    [--from N] [--to M] [--window 50000] [--budget 20] [--dry-run]

--dry-run prints the plan, the share of the window it covers, the number of certificate keys taken from the live registry read, and the row count §4.2's measurement expects for that slice. It writes nothing.

Acceptance: four numeric conditions, decided by the command. --verify renders the verdict so an operator does not eyeball row counts. Any condition missing = stop; do not start the next stream.

#	condition	what it catches
1	row count within 70%–200% of a preflight estimate, or within the exact endpoints of a completed production-measured band	a backfill that silently lost 60% of a stream (an order-of-magnitude band passes that); a measured history is not widened again
2	chain_scan_ranges contiguous over [floor, cursor], both scopes unioned	a truncated response usually shows as a missing window before it shows as a low count
3	digest re-check on a sampled receipt, re-scanned and compared	a response truncated within a window, which 1 and 2 both miss
4	the stream's named anchors — an exact receipt, or an independently counted number of rows per event type over a pinned window — plus every declared address certified from the floor	one exact row proves the decode and a counted window proves it was not truncated; the certificate proves nobody was left out of a 546-address sweep

ETHEREUM_ARCHIVE_RPC_URL=<archive> LEDGER_VERIFY_RPC_URL=<second provider> DATABASE_URL=<...> \
  npx tsx scripts/backfill-event-ledger.ts --verify --stream erc4626 [--sample <from_block>]

--verify is read-only. Condition 3 uses LEDGER_VERIFY_RPC_URL when it is set, which turns self-consistency into corroboration; without it the re-scan goes to the same archive endpoint and the verdict says so. It samples the sweep's own receipts, never the tip loop's: the ingester writes one receipt per 60-second cycle, so a day of uptime is roughly 1,440 of them against a stream sweep's ~65, and a uniform draw over both would almost never land on the history this condition exists to check. Condition 2 still reads both. --sample <from_block> re-checks one named sweep receipt rather than a random one, so a reported miss is repeatable. The second provider's free tier times out on a 5,000-block range across erc4626's 546 addresses (1,000 serves fine) — a server-side cost of the address-set width, not of log density — so pass a smaller --window for that stream's verify leg.

Which endpoint you pick as the second provider changes what condition 3 can do. Providers refuse an oversized log query in three unrelated ways: some cap the block span and say so, some return a bare HTTP error with no explanation, and some cap the number of results and report it as a generic "invalid params" whose only usable detail sits in the error's data field. The scan narrows its window and retries on all three, so a refusal is a slower verify rather than a failed one — but a provider that signalled a refusal in some fourth way would be retried at the same width and would fail the leg. A condition-3 miss that names an RPC error, rather than a digest mismatch, is a report about the endpoint and not about the ledger; re-run it against a different provider before treating it as a gap.

Condition 1's expectation is sized off the registry read, not off a frozen count: the erc4626 universe and the Fluid DEX pool set both grow, the wallet-token estimate is per enrolled wallet, and the bare-token transfer stream's ceiling widens with the number of token addresses its registries declare. The wallet-token one is a linear extrapolation from three wallets, so a miss there is more often population drift than a truncated scan — conditions 2 and 3 are the discriminators.

The four streams that predate this programme are judged too, and two of them are red.transfers, morpho, fluid-operate and fluid-nft were imported releases before the rebuild, so no expected size and no anchor had ever been written for them — and a stream with neither cannot pass conditions 1 and 4 at all. Since the bare-token transfer stream is required for every tracked wallet, that left the rebuild unable to reach its final gate for a reason that had nothing to do with whether the data was good. Each of the four now carries a full enumeration of its own declared filter over the exact range its original import covered, read from the chain before any stored row was consulted, plus anchors that assert an independently counted number of rows per event type over a pinned 50,000-block window at the bottom of that range and one exact receipt for each rare event.

Two of the four do not pass today, and that is the gate working rather than a defect in it. The Fluid position-birth stream and the Fluid vault-operation stream each had an event type added to what they collect after their history was imported; the live collector picks the new events up, but the history below the handoff does not contain them (12,712 position-birth records and 63 absorbed-position records respectively). Re-earning that history is the delete-and-re-import step the schema notes already call for, and conditions 1 and 4 now refuse those two streams until it is run instead of leaving it to be remembered.

Every failure prints a [backfill-ledger/fail] … line and exits non-zero, which is both halves the cron alert needs; the lines are front-loaded and printed last, because the alert keeps the final eight matches and truncates each at 220 characters.

Re-run a stream at the same --window you swept it with

Receipts are keyed on (scope, from_block), so a re-run at a narrower window replaces a wider receipt on the same key. Finishing the re-run closes the range; abandoning it midway leaves condition 2 reporting a gap over blocks that were in fact scanned.

A bounded live smoke (scripts/ingester/smoke-ledger.ts, read-only, not part of npm test) scans a few-hundred-block window for one stream and asserts the decoded rows equal the eth_getLogs ground truth.

The ledger-driven cron

The 6h cron CONSUMES the ingested event store, and that is the only thing it does. There is no mode flag: the variable that selected between this and the chain-scanning flow pass went with that pass, and with the parity harness that compared the two. What the tick does now, end to end:

1. The dirty set + recomposition (below) produce the window's snapshot rows, committed in ONE lock-held transaction with the window's own delete.
2. The merge derives every wallet's receipts for the tick's window from raw_events and merges them into portfolio_flow_events_v2, one wallet per transaction, after that commit.

The tick's own window. The floor is the tick's own cursor, portfolio:v2:tick, plus one; the ceiling is min(anchor − 64, the ingested tip), where the tip is the MINIMUM ledger:<stream> cursor over the rolled-out streams — the slowest stream the derivation may rely on. If the ingester lags, the tick HOLDS at the tip and logs [portfolio] ledger lag … loudly: never a silent skipped range, never a fall-back to a chain scan. The cursor then advances to the tick's own settle line (min(range.to, min(anchor − 64, the chain's finalised head))), so the band it derived above finality stays above the cursor and is re-derived by the next tick. The full bootstrap chain for a box that has never had that cursor is in deployment.

A row's basis answers that same line (issue #777). The derivation is handed the tick's own settle line rather than inferring one from the range it was given, so provisional means exactly "above the cursor this tick stamps" — the band the next tick re-derives — and nothing below the cursor can be left labelled unsettled for ever. The page load passes its own line the same way; the whole-history writers (the registration replay, the re-derivation trigger, the offline history build) pass none, because they stamp a cursor over their entire range and have no unsettled tier. And a settled row stays settled: the merge never writes a stored live or backfill row back as provisional when a later pass with a LOWER line re-derives it (keepSettledBasis, applied inside the merge's own transaction). Two writers over one range can hold two lines — a worker job queued before a tick, a page load that could not read finality — and without the rule the second would relabel what the first settled, below a cursor nothing re-derives. The rule is the merge's, so it holds for the inline writers too, and it changes what two populations of rows show: a page load whose finality read failed (it derives with no settled tier) no longer relabels the settled rows it re-derives as provisional; and the near-head rows the registration replay and the re-derivation trigger write live (they pass no line) are no longer relabelled provisional by the next page load or tick whose line sits below them — they stay live while a reorg could, in principle, still take them. The Activity feed's "confirming" tag and the group total it withholds therefore no longer appear for those rows. The direction is the safe one: finality is monotone, and the reorg repair re-derives by rewinding cursors, never by looking for the tag, so a relabel was only ever cosmetic and a row a reorg orphans is deleted by the merge whatever its label says. scripts/worker/sweep-parity.test.ts pins the replay-then-page-load case on the fixture.

The two registries the tick maintains are fed from the rows the merge actually inserted: portfolio_held_pts (so loadPendleMarkets keeps a matured PT in the universe) and portfolio_wallet_index (so the discovery intersection keeps bounding the reader). They used to be a byproduct of the chain flow scans. The pairing matters: the wallet index is what the 6h reader's universe is narrowed to, and a FRESH discovery certificate is what makes that narrowing engage — so the memberships are persisted BEFORE the tick cursor advances (a failed upsert holds the window open for the next tick to retry), and the certificate's own liveness signal comes from the same merge. An index that stopped learning while certificates kept advancing would read a newly-entered market at no venue at all and drop the leg from the spine.

Dirty set + recomposition (dirty-set.ts + recompose.ts). Instead of re-reading every eligible wallet, the tick re-reads only the DIRTY set — wallets appearing in raw_events since the previous tick's scan tip (Transfer/Morpho-owner/Fluid-NFT topics + UserEModeSet users), always-dirty Fluid holders, a rotating reconciliation shard (hash(wallet) mod K == tick mod K, K = RECON_DAYS × 4, so every wallet is re-read once per RECON_DAYS, default 7), and never-snapshotted wallets — WALLET-SHARDED at ≤500 per pass with a per-shard transaction. Every OTHER eligible wallet is RECOMPOSED: its latest stored legs × freshly-read shared sources (per-reserve normalized indexes, ERC-4626 share rates, Morpho accrued rates, Pendle PT rates, mirror prices — each read ONCE per tick, O(universe)), producing rows shape- and value-identical to a full re-read (recompose reconstructs a synthetic read and runs the SAME buildSnapshotRow). M9 honesty: if ANY leg of a wallet cannot be recomposed honestly (missing/failed shared source, unrecognized leg shape, a Fluid leg — Fluid recompose is deferred), the WHOLE wallet is PROMOTED into the dirty re-read set, never written partial/stale/zeroed. The shared universe reads are venue-isolated (one venue's transport failure degrades to an empty index → the affected wallets promote, never abort the tick). Native ETH is par and eventless, so a recomposed wallet carries its stored ETH qty forward (bounded by the reconciliation shard).

Three extra always-dirty conditions cover surfaces recompose cannot own honestly. (1) Bare wallet-venue holders — EVERY tracked wallet token, par included. A bare ERC-20 transfer is NOT a ledger stream (the ledger excludes high-volume bare tokens for scale), so a mover is never ledger-dirtied — but its Transfer flow IS scanned every tick, so a stale recomposed quantity netted against that flow fabricates return (Δvalue − netFlow, with Δvalue = 0 while a real flow lands). Their holders are re-read every tick, never recomposed. This condition used to be scoped to variable_rate wrappers (wstETH, weETH, …) on the premise that a PAR token fabricated nothing when stale, with the residual value lag accepted as bounded by RECON_DAYS. The premise was wrong and the carve-out is gone: the figure fabricated is the FLOW'S OWN MAGNITUDE, which no rate class bounds. A tracked wallet sold its entire USDC balance on 2026-08-21, the chain went to zero that block, and the stored spine repeated the dead balance for 25 consecutive windows — written fresh each tick — while the total-return line published the whole sale as profit. The carve-out's removal stops the next one; it does not unwrite the 25 rows already stored, and those rows are still the record the product serves. Restating them is the ghost adjudicator's job, and that job exists because a stored row of this shape is only half the class: the identical row is also what a derivation missing an inbound receipt produces, and there the stored snapshot is the record that is RIGHT. (2) Wallet-venue movers: a wallet with a venue='wallet' row in the ledger over the last derived block range. This covers the leg the snapshot condition cannot see — a wallet that held no bare token, acquired one, and would otherwise stay invisible until the reconciliation shard. It heals one tick after the movement by construction (the snapshot pass runs before the flow pass and the tick commits once), which is why condition (1) and not this one is what closes the same tick. (3) Incomplete-window wallets: a transient venue-read failure in the dirty pass persists a snapshot missing that venue's legs; any wallet whose latest window dropped a venue present in its prior window is re-read, healing the gap in ONE tick (as the legacy full-read path does) instead of recompose reproducing it until the reconciliation shard.

All three are PROMOTIONS unioned into the dirty set after it is built, so they can only ever move a wallet from the reconstructed path to the authoritative full read. The tick's tally line prints each one's count (bare-wallet=, wallet-flow=, incomplete=) so the cost of the widened re-read stays visible beside the wallets it covers.

What condition (1) costs, stated rather than discovered. Its bound is no longer the small variable_rate token list — it is the WALLET POPULATION. Native ETH is a tracked wallet token, so in practice every tracked wallet matches and the dirty/recompose split described above has no population left: the tick re-reads everyone, which is the pre-optimisation cost. That is paid knowingly — the alternative is a spine that can publish a dead balance for up to RECON_DAYS, which is what produced the 2026-08-21 phantom — and it is measured, not assumed: bare-wallet= on the tally line.

The scaling answer is to re-read the bare LEG rather than promote the whole wallet — a per-leg balance read the recompose path folds in, which keeps the split alive at 100k — and it is a change of its own rather than something smuggled into a correctness fix.

Mark coherence: one market context per tick. The cron builds exactly ONE loadMarketContext per tick, over the UNION of the read-pass accounting assets and the stored-leg assets recompose folds (both expanded via accountingAssetsOf, incl. PT underlyings), and every valuation in that tick — the legacy/dirty pass, the recompose pass (buildRecomposeSources takes the context INJECTED, never fetching its own) — prices from that single object. This is the intra-tick form of the vintage-coherence principle: the price mirror is written concurrently by other refreshers, so two context fetches minutes apart inside one tick can see different bars, and the 2026-07-25 prod soak showed exactly that — rows with identical qty_raw/index_raw/value_redemption drifting 0.003%–0.06% on value_market alone (plus one priced-vs-null split). With the shared context, identical qty_raw/index_raw yields a BIT-identical value_market on both paths, and a mid-tick mirror write can no longer split them. Coherence is per-ASSET: an asset surfacing ONLY in a promoted wallet's fresh re-read (requested by no dirty read and no stored leg tick-wide) is priced by one supplemental fetch merged ADD-ONLY into the tick context (mergeMarketContextAssets) — an already-requested asset, priced or honestly null, is never re-marked, so no asset can carry two marks in one tick while the promoted leg still gets a price instead of a one-tick null. The JIT positions fast path keeps its own per-request fetch — it has no tick to cohere with and is already coherent within its one refresh.

Held-PT set (portfolio_held_pts, migration 065). loadPendleMarkets used to keep a matured PT in the universe by seq-scanning the two biggest user tables on every cold refresh. It now consults portfolio_held_pts (active OR within grace OR pt ∈ held_pts); the snapshot pass and the ledger merge upsert a PT there whenever they write a pendle-venue / known-PT row, and migration 065 SEEDED it from the same four "held-by-anyone" arms, so the universe is behavior-identical (proven in registry.test.ts). The four-arm scan it replaced was kept as a pre-065 fallback until #909 removed it; every database carries 065.

Backfill from the ledger, JIT fast path, worker-pool drain (Phase C)

Phase C removed the last O(wallets) hot paths and the last archive-getLogs cost on the signup + request paths. The mode gates it shipped behind are gone with the pipeline they selected between; what is described here is simply what runs. The one gate that remains is the JIT positions fast path's freshness bound, which is OFF by default and ON in production (below).

• Registration-probe discovery from the ledger (D1). When event coverage is CERTIFIED back to the segment start for every stream a detection call reads (each replayed-group position token live in event_coverage, plus the morpho/aave-pool/spark-pool singletons — and fluid-nft when the factory transfers are swept, i.e. only the probe), detectFlowSegment derives the wallet's flows/activity from raw_events by SQL (ledgerTransferFlows / ledgerMorphoFlows / ledgerLiquidationFlows / ledgerFluidFactoryTransfers) instead of an archive getLogs sweep. The ingester's 64-block margin means the ledger tip trails head, so the ledger serves [from, ledger-tip] and a SMALL chain residual sweeps (ledger-tip, segment end] — so the OUTPUT is IDENTICAL to the chain path (the deep-history bulk moves to an indexed SQL query; only the last ~100 blocks touch chain). Bare wallet-venue tokens are never ledger-covered (plan §3.2), so they stay a wallet-filtered chain scan in BOTH paths. If ANY part of a segment is not certified, that segment falls back to the legacy chain sweep and logs [backfill] ledger-coverage fallback.

• Historical group discovery from the ledger (D2). The registration probe also asks the ledger WHICH position groups the wallet held at any point in the replay window, which is what lets a position closed inside the window be replayed at all (src/lib/portfolio/historical-groups.ts). Four wallet-topic-filtered reads of raw_events — position-token Transfers, Morpho events, Aave/Spark LiquidationCall, Fluid factory ERC-721 Transfers — name every group the wallet touched, with NO address filter and no archive traffic at all: the whole-universe alternative is a wallet-filtered getLogs over ~1,000 token addresses per registering wallet, which is the cost that stopped this fix being built at all. The reads are SEGMENTED on the same block-segment constant the ledger flow reads use (DISCOVERY_SEGMENT_BLOCKS, default 200k): the window can now span ~1.5M blocks (the 2026-01-01 floor) or ~3.5M (an account clamped to the coverage floor), and migration 064 indexes topic0/topic1/topic2 but NOT topic3 — so the Morpho read's topic2 OR topic3 arm cannot BitmapOr and degrades to a scan, which against the prod role's 30s statement_timeout would cancel the whole registration (this module deliberately propagates anything but "the ledger is not deployed"). The per-group fold is incremental, so N segments and one segment give the same answer. Each row is admitted only where a live certificate in event_coverage reaches DOWN TO THAT ROW'S OWN BLOCK, so a token still catching up contributes nothing rather than a half-truth. Groups no reader universe can serve (a delisted vault, a matured PT, an uncovered Morpho market) are DROPPED with a log line rather than replayed as "held nothing", and a deployment with no ledger at all (pre-064) discovers nothing and behaves exactly as it did before. FLUID IS THE ONE VENUE THE LEDGER CANNOT ANSWER FOR, and it is answered separately. A Fluid position NFT is NOT burned when the position closes: the close emits only a LogOperate, whose zero indexed params put it out of reach of every wallet-topic query, so a position opened BEFORE the window and closed INSIDE it leaves no factory Transfer to find and no legs for the current read to return — the one venue where the phantom deposit would have survived, and the venue creddit's carry users live on. The probe therefore also takes the wallet's owned NFT ids straight from the resolver (readFluidOwnedNftIds, the positionsNftIdOfUser call the position read already makes, which lists closed NFTs too) and admits them alongside the ledger's. Admitting an id closed BEFORE the window costs nothing: every grid point reads a zero balance, which is no row. It also closes a tier SEAM — both tiers ask the same resolver at their own head block and get the same ids, so the deep pass cannot discover a Fluid group the recent pass missed and leave its leg dying at the stored floor. Discovery only ever WIDENS the read universe, and a group the wallet never held reads a zero balance at every grid point, which is no row, no flow and no chart — so the coverage certificate is the only gate it needs.

• Grid replay parallelism (D2, memory-bounded). The per-grid-point archive reads run with bounded concurrency (BACKFILL_GRID_CONCURRENCY, default 6) through a SLIDING-WINDOW pipeline (streamWithConcurrency): at most the window is live at once, and each completed point is FLUSHED to the replay's open transaction in strict ascending grid order as soon as it is next in line, after which its rows and read intermediates are released. Memory is O(concurrency window), not O(grid × legs) — the predecessor buffered the whole grid before writing, which OOM-killed a 76-leg wallet's ~112-point replay even at a 6 GB heap. Write order stays DETERMINISTIC and the lock/transaction/delete+insert contract is identical (one atomic transaction per fresh replay / per gap segment; BOTH paths lazy-open at the first flush, so the global write lock is held from the first flush through commit and never across an unread grid or a failing first point's retry cycle — see write-lock.ts for the full hold-shape contract); a point's strict read throwing (M9) rejects the pipeline and rolls the transaction back — nothing written, exactly the buffered abort. Month-partition auto-create runs on the pool BEFORE the transaction (post-067 CREATE TABLE … PARTITION OF takes an exclusive parent lock, which must never ride the minutes-long streamed transaction where it would block every /portfolio read until commit). Per-point venue isolation is unchanged, and the replay logs one rss/heapUsed line per 10 flushed points so whale onboarding is observable.

• Worker-pool drain (D3). drain-portfolio-backfills.ts runs BACKFILL_WORKERS (default 4) concurrent wallet backfills. The workers race on the same atomic FOR UPDATE SKIP LOCKED claim, which hands each a DISTINCT oldest row, so no two workers ever run the same wallet (and approximate FIFO holds). The queued→running→done/empty/error lifecycle, reclaim budget, parking, and the shared writer lock are unchanged; a slow child now occupies one worker instead of blocking the whole minute. STALE_RUNNING_MS (60 min) still exceeds CHILD_TIMEOUT_MS (30 min), so a live child is never reclaimed.

• JIT positions fast path (D4). Gated by JIT_LEDGER_LAG_BLOCKS (default 40) measured against the anchor (head). Because the ingester's tip trails head by ≥ 64, the default 40 keeps the path OFF (the page load stays on the fresh full read — no regression); an operator raises it above ~64 to opt in. Production opts in (JIT_LEDGER_LAG_BLOCKS=200 in its .env.local) and staging is meant to match it; see Environment variables. Before recomposing, the gate chain-scans the un-ingested tail (the tip to the anchor) for any position change, so the one staleness it accepts is an e-mode switch inside that tail, which heals on the next refresh after the ingester passes it. When the wallet has NO ledger event since its last snapshot AND holds no fluid / bare-variable-rate leg AND every shared recompose source resolves, positions are served via the recompose (mode now, the same price tier a full read uses, anchor stamped) instead of a full read; ANY miss → full read; force:true always full-reads. Coalescing and the per-wallet cooldown are unchanged by the gate (the cooldown is five minutes now, and LiveResult carries the reading's readAt/readBlock, whether it persisted, and cooldownUntil). The gate covers positions only. The page load's own ledger merge is not gated by it and never was optional: it derives the wallet's tip window from raw_events on every refresh, which is the only way a movement made seconds ago reaches the page before the next 6h tick.

• Wallet-venue scan chunking (D5). The registration probe's chain scans (scanTransferFlows / scanMorphoFlows / scanLiquidationFlows, now the only callers) chunk the padded wallet OR-array at WALLET_TOPIC_CHUNK (default 500) per getLogs pass (disjoint chunks → dedup is inherent) — the last OR-array ceiling. Changes call COUNT, never results.

• held_pts from the page load + the replay (D6). The page load's merge and the registration replay's merge upsert portfolio_held_pts from the rows they actually inserted (the same ON CONFLICT DO NOTHING the cron uses), so a matured PT whose only presence is a page-load or replayed history stays in loadPendleMarkets' held set without waiting for the cron. The page load applies its own settle line to the BLOCK first: a registry nothing ever removes a PT from must not admit one on a row a reorg can take away.

• Coverage tripwire (D7). After the snapshot writes, the cron flags any WRITTEN leg whose ledger stream/token lacks a live event_coverage row (a new reserve/vault/PT the ingester has not caught up) and POSTs one aggregated, deduped Telegram alert (the alertUnknownAssets transport, prefix LEDGER COVERAGE GAP). Fail-soft. The set of requirements it checks is the requirement map below, so a scope becomes visible to this alert at the moment it is rolled out — before the conjunction below withholds anything on it.

• Partitioning (D8): both history tables are partitioned, by different keys, for the same reason (per-wallet read locality at the 100k target — never retention; neither table is ever pruned). portfolio_position_snapshots is repartitioned monthly by snapshot_ts (migration 067, -- DESTRUCTIVE, manual — see the 067 release step), and the writers auto-create future month partitions (ensureMonthPartitions, a safe no-op before the migration). portfolio_flow_events is repartitioned BY HASH (wallet), MODULUS 16 (migration 072, also -- DESTRUCTIVE/manual): wallet is already the second PK column, so the PK, all three ON CONFLICT targets and every writer are unchanged, and each hot flow statement prunes to one partition. Its 16 partitions are created once by the migration (a hash modulus is fixed at CREATE time), so the flow writers take no partition step at all and ensureMonthPartitions refuses any non-RANGE parent. Monthly range partitioning of the flow ledger was designed and rejected (its PK lacks a timestamp, adding one breaks the conflict targets, and a month key prunes no hot flow read) — see the migration headers and docs/database.md. portfolio_flow_events itself was dropped by migration 095; its partitioning is recorded here because the same reasoning produced the served ledger's, which is identical.

The flow ledger — DERIVE (src/lib/portfolio/derive/)

The movements behind every portfolio surface used to be produced by four per-venue scanners that each classified a log in isolation. DERIVE replaced them: one routine that reads a block range's logs out of raw_events and writes portfolio_flow_events_v2 (migration 082). Those scanners and the ledger they wrote are gone; this is the only writer of movement rows there is, and portfolio_flow_events_v2 is the only ledger any page is served from. The READ is not staged behind anything: the reader that chose between two ledgers, and the environment switch that decided, were deleted with the retirement, and the relation they chose against was dropped by migration 095 (the runbook is deployment.md).

What a row's three money columns are worth, and which of them the mirror's staleness reaches. Every receipt is priced at its OWN block, against the newest price-mirror bar at or below it — which can be up to an hour old, because nothing trues the mirror up to the exact minute any more (the pass that did was deleted with the engine that called it). Same-bar cancellation decides what that costs, and it protects a ratio rather than a level:

• value_market on a booked leg is the token's bar DIVIDED BY its book's numeraire bar at the same instant, so the bar's vintage cancels and only the basis drift inside the window survives — a few basis points. This is the column the Activity feed serves, so the served degradation is that figure and no more.
• value_usd on an ETH-book leg is that ratio multiplied by WETH's own dollar bar, which makes it a level: it carries the whole ETH/USD move inside the bar window, tens of basis points in an ordinary hour. Nothing reads it on the request path today — its intended consumer is the cross-book capital netting, which has no caller yet — so this is a property of stored rows rather than of a published number. It is permanent for those rows, because nothing re-derives below the tick's cursor, and it is the first thing a release that starts serving value_usd has to deal with (by truing those rows up, in derive/marks.ts, where the header states it).
• An EXCLUDED leg has no book unit, so both its columns are the same dollar level and carry the same exposure. The feed prints such a row's amount and dashes its figure.

Why one routine and not four. A token movement says that units moved; it does not say who owns the position. Attributing ownership from a transfer is what makes a router-mediated exit look like the router's, and what lets a liquidation's protocol fee be mistaken for the seizure it sits beside. So every log is first sorted into one of three classes and only the first class is ever allowed to name an owner:

class	what it is	example
SEMANTIC	a venue event that states a true amount AND names its own owner	Aave Supply, Morpho Withdraw, an ERC-4626 Deposit
QUANTITY	a token movement, with no owner claim	an ERC-20 Transfer, an aToken Mint / Burn / BalanceTransfer
CONTEXT	a fact that changes how the others are read, never a row of its own	AccrueInterest, ForceDeallocate, an ERC-4337 UserOperationEvent

The routine, in order. Decode into the three classes → three unconditional drops (an aToken/vToken mint whose whole value is accrued interest, a zero-amount movement, a self-transfer) → build the transaction's context index → guards claim first, binding a movement to a branch and removing it from the pool entirely → the semantic walk consumes the NEAREST unclaimed movement on the same token in a consistent direction (on the two adapters that key admissibility on the amount their own event states, ERC-4626 and Morpho Blue, an event stating a ZERO amount claims none: it says nothing crossed the boundary, so the movement beside it is somebody else's) → whatever is left goes down the classifier ladder → the settlement pass gives every consumed movement's OTHER leg its row → the post-pass closes the legs.

Two properties of that order are worth stating because both are load-bearing. Pairing is by consumption, never by key: a single transaction can carry two supplies on the same reserve for the same wallet, and matching on (wallet, reserve, side) attaches the wrong movement to the wrong event and reports a discrepancy that does not exist. And no venue's log-ordering convention decides anything: six conventions are in play and two of them are opposite inside a single brand, so the ordering table breaks an exact tie and nothing else. A tie it cannot break is consumed in log order and the row says so.

Two quantities per row, and no fee is ever stored. Each row carries what actually crossed the wallet's boundary (in the moved asset's own smallest units) and the signed change in the leg's own position quantity, in the same unit the position snapshots use. Any wedge between the two is a fee, a haircut or slippage, and it is deliberately not stored: it falls out as yield on that leg exactly once, because the leg's value has already moved by the mark while the receipt states the true amount. That one rule covers the ERC-4626 exit fee, the stake-token redeem haircut, a Fluid borrow fee, a Pendle fill-versus-mid gap and a withdrawal-queue shortfall, with no per-venue column and no version branching.

Units on Aave and SparkLend. The position quantity for these venues is the SCALED balance — the figure that stays put while the index and the underlying value both rise — but the Pool events state NOMINAL amounts, and BalanceTransfer is the only Aave log that carries a scaled one. So most rows need a nominal-to-scaled inversion, and Aave rounds differently per era AND per operation (v3.5, from block 23,088,584, rounds each of mint, burn and transfer in the direction that favours the protocol; earlier Aave and SparkLend at every block are plain half-up). The index for that inversion always comes off a log in the same transaction, so the derivation issues no RPC read for it; a transaction with no such log leaves the quantity NULL and raises an anomaly rather than inventing one.

One event can move two positions. Repaying an Aave loan out of the collateral already posted (repayWithATokens) emits a single Pool event and no withdrawal event at all, while the collateral is burned to settle the debt. It is booked as what it is: a repayment on the debt position AND a withdrawal on the supply position, from the same log, told apart by the position they belong to. Booking only the repayment would leave the collateral position's balance falling on chain with no receipt for it, and the self-audit below would then flag a position on which nothing is actually wrong.

What a liquidation records, and why the fee is its own line. A liquidation is booked on the BORROWER's side only, as two gross facts: the collateral that left involuntarily with nothing given in return, and the debt that was extinguished without the borrower paying for it. The lender who funded the market is deliberately absent, because that loss already reaches its owner through the market's own index, and a second entry for it would count it twice.

The protocol's cut of the seized collateral is a THIRD line rather than an adjustment to the first. It is a real part of what the borrower lost, it is charged at a rate that differs per reserve (0.86% on one measured Aave USDC liquidation, 0.48% on a SparkLend WETH one), and bundling it into the seizure means depending on finding it. The documented fallback when that lookup fails is to treat the fee as zero, which understates the borrower's loss and leaves no trace. As its own line the fee's absence is visible instead.

Getting those three lines attributed to the right movements is what the guard stage exists for. In the receipt this branch is written against, the collateral going to the liquidator and the collateral going to the treasury are both outbound from the borrower, and the liquidation event sits closer to the fee than to the seizure. Left to "the nearest movement wins", the routine treats the fee as the thing the liquidation explains: the fee line vanishes and the real seizure is booked a second time as an ordinary transfer out. So a liquidation claims its own movements up front, before any general matching runs, and the claim is narrow: the borrower's movements to the liquidator, to the protocol treasury and to the pool. A transfer to anyone else that merely shares the transaction stays visible as the ordinary transfer it is.

A liquidation claims a movement by the AMOUNT the movement states, not by how near it sits. That matters when one transaction liquidates the same position twice, which bots do: the chain emits each liquidation's debt write-down first in its block and the liquidation event last, so the next liquidation's write-down is nearer to an event than its own. Pairing by proximity there puts both records on one movement, leaves the position short the difference, and raises a data alarm on a receipt where nothing is wrong. Each line is therefore built from the movement its own event accounts for, decided once and carried, never looked up again.

The liquidation penalty is not stored. It falls out of the two gross lines: the collateral lost less the debt written off, counted once in total return and nothing at all in accrual. There is no penalty line and no penalty column, and the figure is never floored at zero. For a position that was already underwater, the liability extinguished is worth more than the asset seized, so the final interval is a gain. Flooring it at zero would leave the account's equity moving with nothing in the record to explain the move, which is exactly the unexplained residual this rebuild removes. Bad debt written off after a liquidation is recorded the same way, one line per market, so a sweep across several markets in one transaction needs no special handling.

Positions that move through a router. A supplied position can change hands without ever being redeemed: aggregators, settlement contracts and bundlers move the position itself, and nine distinct ones appear across the history. When that happens the amount is read from the wallet's OWN transfer, never from the pool event beside it. On a measured receipt the wallet moved 1.000000 WETH while the pool event stated 0.200000, so taking the pool event's figure understates the exit by 80%. A chain that passes the position through an intermediate wallet is recorded hop by hop rather than collapsed: the middle wallet gets both the arrival and the departure, which net to zero on that position because it held it for no time at all, and collapsing the chain would lose the two ends.

Suppression never removes the only record of a movement. Aave restates the same movement in more than one log, and two of those restatements are dropped here: the plain token transfer that sits beside every mint or burn, and the interest slice the protocol re-mints when accrued interest exceeds a withdrawal (that interest is already inside the position quantity, so the mint moves nothing). Each is dropped only when the event that states the movement truly is present in the same transaction, and each carries the reason it was dropped, so the claim is checkable rather than asserted.

Vault positions: the amount paid out, not the amount it was worth. For an ERC-4626 vault — a curator fund, a Fluid fToken, a multi-strategy vault, a Morpho Vaults V2 vault — the venue event states both the true asset amount that crossed the vault boundary and the share delta, so neither has to be reconstructed. That matters where the two disagree: on a vault with an exit fee, shares multiplied by the share rate is the redemption value and not the amount paid out (measured +5.00 bps on one fund), and the retired scanner booked that gap as capital returned. Storing the event's own amount instead lets the gap fall out as yield, once, under the same wedge rule as every other fee. The position's owner is read from the vault's own owner field, never from the caller, so a position opened through a zap or a bundler still belongs to the wallet.

The forced-deallocation fee is recorded as a withdrawal of nothing. Morpho Vaults V2 lets anyone pull liquidity out of an allocation for a small fee, and it charges that fee by burning the holder's shares and paying the proceeds to the vault itself. The holder receives nothing, and the line that records it says exactly that: the amount withdrawn is zero, because nothing reached the holder, and the shares are gone, because they were burned. The position's value falls by what those shares were worth and the line returns no capital against it, so the whole fee lands on the return line as a loss, once, on the day it was charged.

Both of the ways this can be got wrong are worth naming, because they pull in opposite directions. Record the fee as an amount withdrawn and it reads as the holder's own money coming back, which overstates their return by exactly the fee. Record the vault as somewhere the money went and the queued-payout rule fires — the recipient is not the holder — opening a position waiting on a payment that will never arrive, permanently. The line that is neither is the one above: recognised by the recipient being the vault itself, which is true of the fee and of nothing else because the protocol hands it back to the vault by construction, and confirmed by the forced-deallocation event the protocol emits immediately beside it, which is also the only other place the fee's size is written down.

A withdrawal paid to the holder is always recorded, whatever else its transaction contains. This is the correction that matters, because the two travel together. Pulling liquidity out of an allocation is what a holder does in order to exit a fund that has lent more than it holds in cash, so the app puts the pulls and the real withdrawal in one transaction, and a rule that suppressed the whole transaction deleted the holder's own payout with the fee. One tracked holder does this routinely: 106 such transactions on a single fund, five of which charged a fee at all (about $13 in total). The withdrawals that went missing beside them ran to tens of millions of dollars, and the position's return line carried the loss. Across the eighteen registry V2 vaults the surveyed window holds 476 of these fee events, 46 distinct holders, and 458 of them charge nothing at all. A pull that charges nothing burns no shares and moves nothing, so it gets no line — there is nothing for one to carry.

A line is also what lets the fee survive being rebuilt. A position's history is rebuilt in ranges rather than always from the beginning, and each range opens from the balance the previous one left behind. A fee that had no line of its own had nowhere to leave that balance: where it was the last thing to happen in a range, the next rebuild opened from a balance taken before it and the burn was simply gone — which is how a position can end up claiming shares it no longer holds. Now the fee is a line like any other, the balance after it is stored on that line, and a position's own lines add up to the balance it reports. Rebuilding a stretch of history in one pass or in two stores the same balances and the same closes either way.

A fee and the withdrawal it was charged for are one transaction, so the statement shows them as one entry, worth the withdrawal alone. A fee charged on its own would show as a withdrawal of nothing.

Fixed-rate (Pendle) purchases are valued at what was paid, and it depends on the route. A principal token bought on the market is recorded at the router's stated fill, not at the pool's mid price. The gap between them is a real, one-directional transaction cost that the retired scanner hid entirely (measured −21.5, −12.0 and +4.1 basis points on three purchases), and it belongs on the total-return line. A principal token minted is a different transaction and gets a different rule: minting costs par and hands the buyer a principal token and a yield token, and creddit tracks no yield tokens. Charging the whole payment to the principal token would therefore book a day-one loss the holder never took, so a mint is recorded at the principal token's own mark and the part of the payment the yield token absorbed is reported as a coverage note rather than booked. The route is decided by which router operation was used.

A redemption is that same rule run backwards. Redeeming through the router burns the principal token and the yield token and pays out for both, so the payout is not what the principal token alone was worth, and recording it as such credits the account with the yield token's share. It is the same condition the mint rule answers, in the other direction: part of the consideration is outside what creddit tracks. So a redemption is recorded at the principal token's own mark too, with the yield token's share reported as a coverage note. Only a market trade keeps the true-fill rule, where both sides are tracked and the gap against the pool's mid is a genuine cost.

This was invisible until the holdings themselves were correct. While a wallet's principal-token holding was overstated by everything it had posted as collateral, the app never looked at it, so a mint and a redemption minutes apart booked nothing. Once the holding is right the round trip is visible, and priced on two different bases it published plus $42,948 for a trade that made nothing, and plus $101,972 over that position's life. Recorded on one basis at both ends, the same round trip nets to a few dollars.

That distinction is sharper than it sounds, and creddit's own history is the reason. The one real principal-token purchase a tracked wallet has ever made does mint a yield token in the same transaction — to the router, which immediately sells it into the pool. It is a market purchase, and a rule that keyed on "a yield token was minted here" would have mispriced it. The test is who received it. That receipt is pinned as a fixture.

One purchase, several movements. The same receipt fills a single purchase from two sources at once, so the principal token arrives in two separate movements that sum to the router's stated net. Only one row is written, for the net, and the other movements are matched against that net before being set aside — if they do not add up, nothing is set aside and the leftover is reported as unexplained. The retired scanner wrote both movements and booked one purchase as a deposit and a transfer in.

Exits that leave through the protocol rather than the market. Redeeming a matured principal token sends it to the market's own yield-token contract to be burned. That is a position closing, not a transfer to a stranger, and it is booked as a withdrawal at the position's own mark. It has to be recognised from the movement itself: the wallet's real 2026-06-23 redemption rode a router operation whose layout is not established, so nothing in the modelled event set covers it, and without this rule the position's exit reads as value walking out to an unknown address.

Pendle market records must be retained, and this is a correctness dependency. Every principal-token position resolves its market, its yield token, its underlying and its accounting from the Pendle market registry. Raw logs are kept forever, but a principal token whose registry row is gone can never be re-derived — not even by a full replay from scratch — because nothing can say which market its movements belonged to. Today the market refresher only adds rows and flips matured ones to a matured status; nothing deletes them, so the property holds. If a retention or purge policy is ever added, matured Pendle markets must be excluded from it. A tracked wallet's fill naming a market the registry does not know is reported as an anomaly, which is what makes the day that changes visible.

How a position's exit is closed, and why it is not arithmetic. Every row also carries the leg's quantity after the event, and a terminal flag that the database ties to it: a row may not claim a closed leg while holding a non-zero balance. The balance is derived from the running total where the ledger anchors it and read ONCE, per leg, where it does not — one archive read per leg per replay (52 across the whole of today's tracked population, and none at all on the live path), never one per receipt. A FULL exit is closed at exactly zero by the venue's own evidence, not by the arithmetic: the venue computed the receipt's amount from the same balance and the same index, so equality is its statement that the whole position was consumed. One unit of rounding drift, closed the other way, would leave the position permanently open and the series would never end.

And the most passive holding pattern there is: a leg with no movement at all. A wallet that acquired a token before its history floor and never moved it since has no receipt in the window. Before ledger-first such a leg was receiptless and took its occupancy from the valuation spine; since ledger-first (plan R2, R4) the ledger speaks for it too: the registration replay writes an opening row at the wallet's floor reading for every leg that reading holds, stating the read quantity as the balance the leg opens at (qty_delta = to_balance, at the reading's block, after every log of it), and the reading audit writes the same row wherever a later reading first finds a leg the ledger has no history for (a coverage start). Every leg means every leg: a holding worth under a cent is opened too, and only kept off the page (R7; the comparator run on staging, PR #962, F1: the reading audit). So the leg is held from its opening, earns its yield normally on both lines, and raises nothing, exactly as the receiptless rule used to make it; and a leg with no row at all is simply not held. An "unanchored" leg (W6, production budget zero) is still one whose quantity the ledger CANNOT STATE: an anchor read that was ATTEMPTED AND FAILED (the whole replay), or a history that opens from a balance no row states — a first movement that leaves the leg held without creating it, with no opening before it (that one interval, below).

Where the boundary between the two sits. Before a leg's FIRST row the ledger says it held nothing, whatever that row's two columns are: the balance before a partial exit or a top-up is not back-derived from to_balance − qty_delta any more (R7 retired that arithmetic), because the opening row is where a holding that predates the first movement is stated. That is what lets the engine value a position at zero when it is acquired inside a window the spine could not read, so the stretch from the acquisition to the first successful read still books, and what lets a snapshot row sitting before a position's first row be recognised for what it is: a row the ledger cannot explain, named by the ghost cross-check and owed an opening or a correction by the reading audit (below). The zero is not assumed where the first row itself says otherwise, though: a first movement that leaves the leg held without creating it (a partial exit from a position no opening states) opens the leg's series from a balance nobody recorded, so the interval that holds it is not measured (W6 at that interval) rather than booked from zero — which would earn the whole earlier holding — and the leg books from the row's own to_balance on.

The free self-audit. For any leg whose stream is certified across the range, the sum of its quantity changes must equal the change in the position snapshots' own quantity over the same range — exact, integer, no tolerance. Any drift is a missed log or an uncovered stream, and it is reported as one.

Failures are values, never exceptions. DERIVE returns its rows together with a list of anomalies, and it throws only on a programming error. The live path calls it from a page load, so an adapter that threw on bad chain data would take the served page down with it. The anomaly list is the honest surface: an unexplained movement, a missing scaled quantity, an anchor read that failed, a mark that could not be resolved, a quantity that disagrees with the spine.

Morpho Blue: the quantity is in the event, and one event books two positions. Morpho states both figures on every supply, withdraw, borrow and repay — what moved, and the position's own share change — so this venue needs none of the inversion machinery Aave does, and neither figure is ever derived from the other (the round trip loses up to one raw unit, and one unit of drift on a full exit is exactly what stops a position closing). Collateral is simpler still: Morpho holds it in raw token units with no index and no interest. Two shapes are worth naming. A withdrawal denominated in shares can legitimately pay out nothing while burning shares — it is a total haircut on that slice, and it is booked as one rather than dropped as an empty movement. And a liquidation is one event that moves two positions: the collateral taken from the borrower, and the debt that disappeared. The debt side is written as a single row covering both the part the liquidator repaid and the part written off, because on the borrower's side they are the same thing — a liability gone, with the borrower paying nothing — and the split is kept on the row for anyone who needs it. Writing the liquidator's repayment as an ordinary repayment instead would treat somebody else's money as the borrower's own and overstate their loss by the whole repaid amount.

When a Morpho borrower is wiped out, the ledger says so. If the collateral runs out before the debt is covered, the protocol deletes what is left of the position. That is the venue stating both legs are at zero, so the ledger closes them there instead of reading a balance or inferring one, and the total loss books once. Without it a wipe-out books nothing and alarms forever.

A Morpho leg is anchored by the same single read every other venue gets. Morpho states the change on each event, never the balance, so the running total needs somewhere to start. That start is one read of the wallet's position in that market at the block before the leg's first event in the window — the market's own record of its supply, debt and collateral, in exactly the units the readings and the events already use. Everything above it is arithmetic: Morpho share quantities move only on the venue's own events, so the running total stays exact to the unit for the rest of the leg's life. One read per leg per replay, none at all on the live path.

The read is what makes both cases right. A position opened inside the window reads zero there, which is the same answer its opening event would give. A position opened earlier reads whatever it actually held, which no event in the window can say — its first event is a top-up or a partial repayment, and those describe a change, not a state. Assuming the zero would silently book the second case as a position that started empty. Getting this wrong is not cosmetic: a leg with no anchor carries no post-event balance on any of its rows, and a leg with no balances is withheld from both published lines for its whole history.

A Morpho market nobody registered is an alert, not a silence. The venue's stream carries every market on the chain, so a market outside the registry is normal — until a tracked wallet appears in one. At that point its loan token and decimals are unknown, no row can be written truthfully, and the run says so by name. The path this replaces returned nothing at all, which is how an entire market's history goes missing with nothing to read.

Market impairment — a dated event, never a number. When a Morpho liquidation leaves bad debt behind, the loss falls on everyone who supplied that market, and no event names any of them: the market's assets fall while its shares do not, so the redemption rate steps down in a single block (measured at −22.47% on the reference event). That loss already reaches the portfolio through the market's own rate, so it is not written as a flow — doing so would count it twice. Instead the ingester records the event at market scope (scripts/ingester/market-impairment.ts, writing morpho_market_impairment), and the derive tree reads it to do two things the magnitude alone cannot: move that slice of return out of "interest earned" and into "credit loss", and withhold the realised rate for any supply position whose window contains one. Annualising a one-block step of that size prints a number with no meaning, and the failure is not that it is large — it is that a credit loss is presented as an interest rate. The magnitude is untouched, so the return line still shows the loss; the rate line says nothing rather than something false, and the user gets a dated event that explains it.

The one number that record has to get right is how many shares the market had at the moment of the loss, since every supplier's slice is measured against it. A chain read answers as of the end of a block, which is the wrong instant whenever anything else touches that market later in the same block — and markets do take several of these hits inside a few thousand blocks. So the figure is rebuilt from the block's own supply and withdrawal events and then proved against the end-of-block read; if the two disagree, something moved shares that this stream cannot see, and the event is refused with the discrepancy named rather than recorded against a quietly wrong denominator. The ingester holds its own cursor just below a refusal, so progress below it is kept, a transient read failure clears itself on the next pass, and a real one keeps saying so. It never steps over one: an impairment that is missing does not read as missing, it reads as a market that had no bad debt, and nothing else re-derives this record. The operator's recourse, and the line to grep for, are in deployment.

Two consequences of that record being derived rather than scanned, both of which the pass has to answer for itself. It only walks blocks the venue's stream has certified it scanned, because to a reader of stored logs a block nobody scanned is indistinguishable from a block with no bad debt in it — so the history a one-time backfill fills in later is recorded by an explicit historical pass, not by the always-on one. And when the chain reorganises, the range it re-derives is reconciled against the stored logs rather than merely written over: a liquidation the chain later orphans is removed from the log store by predicate, but the record derived from it is keyed on the orphaned transaction, so a plain re-derivation would file the canonical liquidation beside the orphaned one — the same loss recorded twice, withheld twice and attributed twice.

Write ownership. The derive tree is application code (src/lib/portfolio/derive/) because the live path calls it, and it writes exactly one relation: the flow ledger, which is user-scoped. It READS the chain-identity tables, the market impairment record and the two registries, and writes none of them — those stay owned by the crons in scripts/. A test in the tree fails the build if an insert into one of them ever appears there.

Repeatability. The writer's upsert only rewrites a row whose derived content actually changed, so re-running a range over itself leaves the ledger — and every row's last-changed timestamp — untouched. Together with a deterministic tie-break on the one key that can collide, that is what makes "replay the same range twice, or in two halves, and get the same ledger" a property the build can assert rather than hope for.

Fluid: the seizure with no event, and the smart pair

Fluid needs three things no other venue in the tree needs, and each of them is a decision rather than a detail.

Nothing in a Fluid vault event names the owner. The position is an ERC-721 the vault factory mints, and the vault's own event carries the CALLER — a zapper, a router, a bundler — which in every real example is a contract rather than the wallet. So this venue is attributed by position NFT, from the factory, and the caller field is kept only as provenance. Attributing by it would book a whole levered position against whichever contract opened it.

A position on a smart vault states shares, not tokens. Such a vault's collateral (or debt) is a share of a two-token pool, and the vault event carries a share delta. The token amounts behind it are emitted by the POOL, in the same transaction. Splitting the shares by the pool's current composition instead is a pool-average split of an amount that was not pool-average: measured on our own history, a withdrawal of 0 weETH and 319.9 ETH was booked as 38.982 weETH and 277.106 ETH, and a borrow of 0 USDC and 300 USDT as 221.75 USDC and 78.09 USDT. So each pool token gets its own row with the amount the pool itself stated, and where the pool event is missing NOTHING is written and the gap is reported. A confident wrong split is worse than a visible hole.

A liquidation has no per-position event at all. The vault's liquidation event is vault-aggregate — one event for every position a cascade touched, naming none of them — and a full write-off through absorb() emits an event that carries only two totals and does not fire the liquidation event at all. The retired scanner triggered on the liquidation event, so an absorbed position's legs simply vanished from its ledger and the whole equity change landed in yield; 13 of Fluid's 179 live vaults carry absorbed bad debt today. Under DERIVE the position's own state before and after the block IS the receipt: it is read at both ends and produces two gross rows, the collateral seized and the debt cancelled.

Four consequences of that, all of which the retired path got differently:

• There is no realized-loss row and no floor. The loss is the difference between the two gross rows, and the engine's own arithmetic books it exactly once. A separately stored loss double-counts it, and flooring it at zero is numerically opposite to the truth for an underwater position, where wiping the position extinguishes a liability. The stored figure the retired pipeline computed became a cross-check against these two rows, never an input to them — the two gross amounts ride on the rows so the comparison is possible, and the comparison itself belongs to the later stage where both the old and the new curve are valued at once, since it is a comparison of money and this stage values nothing.
• A seizure in a block the owner also operated is no longer erased. The retired detector skipped the whole block, which silently recorded the seizure as a voluntary withdrawal. Both movements are real and both are stated: the vault event gives the operate exactly and the residual is the seizure exactly.
• The exit closes. The post-event balance is READ rather than derived, so a wiped position ends its series instead of leaving it open forever with an alarm.
• The rows are keyed on the triggering log. A state diff has no log of its own, so its rows inherit the transaction and log index of the event that triggered the read; where a block carries more than one such event on a vault, the first is used. That is what keeps a replay byte-identical and stops one diff being booked twice — a real transaction carries both an absorb and a liquidation event on the same vault.

Two rows are written only when the vault itself confirms the position was liquidated, which is what stops a diff produced by two reads landing either side of some unseen movement from being booked as a loss. The vault derives that verdict from the position's price tick, and a write-off sets the tick's own liquidated bit, so an absorbed position reads back as liquidated — which is why triggering on the write-off event works at all. A tracked position whose balances nevertheless moved beyond its owner's own actions, in a block its vault announced a seizure in, while that verdict says otherwise, is reported rather than passed over: nothing is booked either way, but the movement is countable instead of invisible.

The smart pair's drift is measured jointly, never attributed per leg. A smart side holds one share balance whose two token quantities are a function of the pool's mix at the block, and the pool re-mixes continuously — so each token leg's quantity moves between blocks with no receipt behind it, by construction. On our own July close the two legs read +1.323007 and −1.393289 ETH individually while the pair moved −0.070282. Attributing that per leg prints roughly five thousand dollars of fiction on each. The derivation therefore separates each leg's movement into the part the receipts explain (the share movement, converted at the block's mix) and the part they cannot (the mix's own change on the balance already held), measures the second at the two anchors it already reads, and stamps it on the rows. It is stamped per token, with the token it is denominated in: the pair's two figures are in two different currencies, so the single number the joint attribution needs is a sum of values, and this stage holds no prices. Every row of a side carries the whole pair's list, so the attribution happens once, at the position, in the engine.

And the drift never enters the amount the seizure states. A seizure receipt on a smart leg is the shares the liquidator took, converted at the block's mix — not the leg's whole stored fall, which also contains the pool re-mixing on the balance the position still holds. The distinction is not academic: the same ten shares out of the same hundred book 4.5 of a token when the mix holds and 19.5 when it moves against that leg, a four-fold overstatement of a realized loss, and the drift would then be counted a second time by the position-scope attribution above. The quantity that moved and the quantity that crossed are both kept: the receipt states what crossed, the position quantity states the stored move, and the drift stamped on the row is exactly the difference. For the same reason, whether a leg gets a receipt at all is decided by the seizure and never by the drift — a pool token whose mix rose faster than the liquidator took can end the block with more of it than it started, and it still lost what it lost, so it still books a receipt, a post-event balance and a re-anchor. Finally, if a side's balance at the venue fell and no leg of that side produced a receipt, the movement is reported rather than passed over, since a position quietly getting smaller with no receipt anywhere is the exact failure this whole stage exists to remove.

A quantity that moves without a receipt is exempted from the receipt cross-checks, not forgiven by them. The ledger checks every leg two ways: each receipt's quantity against the venue's own post-event balance, and the whole range's receipts against the position spine. Both define a mismatch as a missing event — which on a smart pair is a statement about nothing, since its quantity moves with no event to find. Left on, they fire on essentially every replay containing two smart receipts, and a false alarm on a routine shape is how an alarm surface stops being read. So a smart leg is marked as one whose quantity drifts: the per-receipt check is skipped (the venue's balance still overrides and re-anchors the leg, so no number changes), and the range check records the leg as unaudited, naming this as the reason, rather than reporting a pass it did not earn.

What it costs. The state read is per position per block-that-touched-it, not per receipt: one read answers for every leg of the position at that block. The pair around a liquidation is mandatory — without it there is no receipt for a Fluid seizure at all. The per-operate reads are what let a normal Fluid leg carry a quantity, because the vault's exchange price is in no log the chain emits (checked against a live receipt: an operate transaction contains the vault event and nothing else), and a caller may decline them and get honest nulls plus an anomaly per leg instead.

What it deliberately does not do. A factory transfer that hands a whole levered position to another wallet writes no row. Modelling it needs decisions this stage does not make — an intra-transaction round trip that must collapse to nothing, a zapper's opening transfer that must be suppressed, the pre-operate state a bundled hand-over must be valued at — and guessing would either double-book a position or delete one. Mints and burns are suppressed (the vault events in the same transaction carry the capital); a genuine hand-over is reported as an unexplained movement, so the gap is countable rather than invisible.

A Pendle receipt carries its market's facts at its own block

The flow valuation (derive/marks.ts) stamps three facts on every priced row whose own asset is a PT: the redemption-index factor at the row's block (one batched two-call read per market and block; at or after maturity the PT's rate IS the factor, so no second read), the PT's own rate by the one rate method, and the row valued at that rate in the leg's book. A market buy or sell is valued at its fill, so the last is priced through a SHADOW asset-basis plan over the same read fans rather than a second formula; a shadow that cannot be priced leaves the fact out and raises nothing. derive() merges the stated facts into meta (ptFactor, ptRate, ptMarkValue) and moves no money column. The lot book strikes each fill against the factor read at its own block, which is what retired the blank rates a fill between two distant stored readings used to leave, strikes each stamped acquisition at the payout coin's own bar, and gives each lot the market's own yield at its purchase, which splits the expanded PT row's mark to market (M34). Rows derived before the stamps existed carry none and read as before until re-derived.

The wallet book: what a bare token movement IS

A movement no venue event explained falls to a classifier ladder, and the ladder is what tells apart rows that are identical today. Of 117 inbound wallet rows on production, exactly one — $7,521.47 — is an unambiguous external capital contribution; the rest are DEX trades, venue-internal legs, poisoning dust, bridge arrivals, reward claims, self-custody moves and transfers between the user's own wallets. All 117 are stored as the same kind, so the engine books every one of them as new money.

The rungs, in order, first match wins:

#	test	result
1	the other party is the zero address	a mint or burn, booked as a transfer tagged zero
2	the other party is a wallet somebody tracks, on any account	still a transfer, tagged tracked_wallet, and both legs joined into one group keyed on the LOG
3	the transaction carries a claim event emitted by the other party	income, tagged distributor
4	the other party is a registered reward distributor	income, tagged distributor
5	the other party is a venue contract	a transfer tagged venue, always — the venue's own row states the venue's leg, this one states the wallet's
6	the transaction carries an ERC-4337 user operation and the movement pays its paymaster	cost, tagged venue
7	the other party is a 4+4 vanity look-alike of an address we know	still a transfer, tagged spam
8	anything else	a transfer, tagged external — the one class that is genuinely capital

Rules 3 and 4 are complementary on purpose. Rule 3 is self-maintaining and covers the reward family that emits an event; rule 4 is a registry table (reward_distributors) for the family that does not, and 75% of measured reward value comes from a distributor that emits nothing at all — two of its claims arrive wrapped in a Multicall3 batch, so even the transaction target is not the distributor. Any design that detects rewards by looking for a claim event misses three quarters of the value.

Rule 5 never suppresses, and the reason is the settlement pass below. A transfer between a wallet and a venue is the only on-chain record of two things at once: what the position did, and what the wallet's own token balance did. Deleting it in favour of the venue's row does not remove a duplicate, it removes the second holding's only evidence — so the movement always becomes a transfer whose class says the other party was a venue, and the venue's own row stands beside it. The two carry opposite signs on the same amount, so they cancel exactly in the transaction's net capital and neither holding is counted twice.

Rule 1 never suppresses either, and it is the same sentence one rung up. Its text read "a mint or burn; the venue path owns it", which holds only when a venue we actually model owns it, and by the time a movement reaches this ladder none does: the venues take what they explain first, and a venue's own receipt token never becomes a wallet holding at all. What is left is a token the wallet holds directly being created or destroyed by something outside our coverage — a real change in a real balance with nothing else to record it. On the largest tracked wallet that deleted five arrivals of a staked stablecoin, together about 15,400 units, and left the holding's running balance 15,318 units below zero. The mint is booked as an arrival tagged zero, and it is not unfunded money: in the transaction that produced it, the wallet paid for it in two other tokens on the way in, so the arrival and the payment cancel in the same net capital sum. Dropping the arrival did not make the transaction neutral, it made it read as a large outflow with the asset it bought invisible.

One residual is now closed rather than expected. The paragraph below (under the settlement pass) recorded ~15,400 units of a collateral token as a smaller second gap that would survive the R9 rebuild. That gap is exactly this rule, and it closes with it.

Native ether goes through the same ladder, and the wallet book states four things of its own (issue #966; src/lib/portfolio/derive/native-ether.ts). The block feed's value rows and the audit's listed internal payments are QUANTITY records on the native-ether sentinel, exactly as a bare Transfer is on its token: a tracked counterparty is a move between the reader's wallets, a venue contract a venue settlement, a matched trade a swap, anything else an external transfer (ether received from a person reads "Transfer received" like any token). Four records are the wallet book's own statements, emitted without the ladder:

record	row
a network fee (NativeEtherFee)	ONE cost row on the ether leg under the fee's own transaction, class zero: in both flow sets and never capital (M8's cost policy), held back on the statement by default like every fee
a beacon withdrawal, a fee recipient's priority fees	a transfer_in of the credited amount, class zero, as the ladder books any mint to a wallet (the explorer link opens the block)
WETH9's Deposit (a wrap)	ONE internal move: internal_out on the ether leg and internal_in on the WETH leg, both in one swap group (the statement reads it as one line, "Wrap"), counterparty WETH9, class venue. It CONSUMES the ether record that paid WETH9 (the feed's, for a transaction's own value; the listing's, for a contract wallet's internal call), so the ether is booked once and never again as a transfer to WETH9. The ether row sits after the block's feed rows on the Deposit's own slot whether or not a record was consumed (a record landed later restates it in place, never moves it: PR #967 review round 2, B1), and both rows name the consumed record (meta.etherLeg), which is how the audit's explanation reads that record as held. It consumes ONLY a record paid TO WETH9 (quantityCounterparty), never the nearest ether record of the same direction: a contract wallet's batch (pay 2 ETH to X and wrap 1, or an EntryPoint prefund before a wrap) carries others, and the nearest was measured to be one of those (PR #967 review, B2)
WETH9's Withdrawal (an unwrap)	the reverse pair, "Unwrap"; the ether row sits at the event's own log and consumes the listing's record of WETH9's payment (ONLY one paid BY WETH9, never a claim paid in the same transaction) where the audit landed one, naming it the same way

The settlement pass does not mirror these (a record a wallet-book semantic consumed is already booked on both of the wallet's legs), and the ether leg is outside the negative-balance terminal below: its receipts are complete only after its audit, since the feed cannot see a contract's payment until the listing lands it, and refusing the range would stop the very derivation the audit waits for. The dip is still reported (quantity-drift), and the audit that follows either books the listed payment as a receipt or pages the residue. Until then the page can leave the ether row out: a served row follows the leg's last movement, and a movement that spent ether a contract paid (not yet landed) takes the recorded balance to zero or below, which serves no row, where before issue #966 the stale reading was served. The next reading's audit lands the payment and the row returns; this is the latency class issue #966 accepts for contract payments.

A holding's balance can never be negative, and the pipeline now refuses rather than stores one

A holding's running balance starts from a reading of the chain and is carried forward by the movements the ledger recorded. A token balance cannot be below zero, so a derived balance that is below zero is proof that the run's own arithmetic is wrong — some arrival is missing — not a discrepancy to note and move on from. Storing it anyway is what turned three such gaps into 386 refused rows in the ledger audit: the reporting engine reads a non-positive balance as "this holding is gone", so it publishes nothing for a holding the wallet still has while the other side of the same trade books in full.

So a range whose derivation produces one is refused at write time: nothing is written and nothing is marked as covered, and the message names the holding, the balance it reached and the transaction it reached it in. A balance the run could not work out at all stays unknown and never refuses anything — that is what the existing withhold is for, and reading "unknown" as "impossible" would refuse ranges for the one condition the withhold exists to survive.

The check now covers borrowings and fixed-rate principal holdings as well as plain token holdings. It shipped scoped to plain holdings because a venue position's balance is kept in that venue's own accounting unit and it was not then settled that zero was the floor of all of them. It is settled for a borrowing: the snapshot contract keeps every leg quantity non-negative with the side carried in the leg's own name, and no venue lets an account owe less than nothing. The unit still differs per venue; the floor does not.

It is settled for a fixed-rate principal holding for a simpler reason: that is not a venue position at all, it is the wallet's own token balance wearing the fixed-rate venue's name because that venue owns its price. Both the app's reading of it and its opening anchor are the token's own balance, so the floor is the same floor a plain holding has. The two negative balances that stood in the ledger for three months were exactly this shape, and nothing objected to either.

Collateral positions stay outside the check, deliberately: nothing settles a floor for every venue's collateral unit the way it does for a borrowing, and a refusal is the wrong instrument for a question nobody has answered.

Three borrowings on production had gone below zero when the check was widened, all on the same venue and all in the same shape, and the fix for them is the section below rather than the refusal: the refusal is what catches the shape if it ever comes back.

A two-token venue position states every one of its legs, and the reading is now used for all of them

A position on the leveraged-vault venue can hold a share of a two-token pool on each side. The wallet's quantity in either token is then a function of the pool's own mix, which moves continuously and with no event anywhere to explain it — so a running total carried forward from movements alone is not that quantity, and drifts from it in whichever direction the pool re-mixed. The venue answers the question directly: one read per position per block returns the end-of-block quantity of every leg it holds.

That reading was applied to at most one movement per position per block, on the reasoning that an end-of-block answer can only describe the last movement in that block. That is right about a leg and wrong about a position, and the ordinary way to unwind a levered position is exactly what it gets wrong: repay the borrowing in one call and withdraw the collateral in the next, both in the same transaction. The reading went to the withdrawal, the repayment closed by the running total instead, and the borrowing was left holding a quantity the chain flatly contradicts — negative on one token of the pair, and too large by the mirror image on the other. On production that is three repaid borrowings reading −2,502.06 GHO, −1,373.00 GHO and −0.478 WBTC, each beside a sibling holding the equal and opposite surplus, while the venue's own reading at each of those blocks says the borrowing is repaid in full.

The reading is now matched per side: a movement takes it when nothing after it in that block moved its own side of the position. A liquidation moves both sides and so raises the bar on both. Nothing else changes — the movement keeps its own amount, and the reading settles only the balance — and a block where the venue did not answer is unchanged too: no reading, no claim, and the write-time refusal above is what stands between an impossible balance and the ledger.

What the rule does not claim is that the movement's own quantity stood still. On these vaults both sides can share one pool, so a movement on the other side re-mixes the pool and changes this side's token quantity with no event on this side at all. The venue's answer is an end-of-block composition either way, and that is precisely why taking it is right; matching per side only makes sure the answer goes to the last movement on that side rather than to one a later movement has already overtaken.

The correction is not confined to closings. Opening a levered position is the same shape upside down — deposit collateral, borrow against it, deposit the borrowed tokens, all in one transaction — so the collateral movements sat under the borrow and closed by the running total in exactly the same way. Nothing ever reported those, because a running total that is merely wrong by the pool's re-mix stays positive on the way in and breaks no floor. They are still quantities the venue contradicts, and they change too. Across the whole history the change touches twelve stored balances on two accounts: the six at the three closings (the three impossible borrowings and the three siblings holding the equal and opposite surplus) and six more at three openings. Measured over both accounts' own stored history, the six opening ones move nothing the product publishes: no interval's return, no leg's occupancy and no point on any served curve changes when they are applied. They are corrections to a stored balance and nothing else — which is why they are named here rather than left for whoever runs the rebuild to discover.

Both halves of a two-token position are recorded, including the half that was paid nothing

A settlement on one of these positions is usually paid out in one of the pool's two tokens. The venue's own event says so explicitly: it names both tokens and puts one of them at zero. The pipeline read that zero as "nothing to record here" and wrote a movement for the paid token only.

That is right about the token and wrong about the holding. Nothing of the second token left the position, but the wallet's holding of it changed all the same, because the pool re-mixed its two tokens to pay the settlement out in one of them. With no record of that, the holding never closes: it keeps whatever balance the last movement left it, the ledger goes on treating it as an open position for the rest of history, and the whole of the pool's re-mix is credited to the token the wallet was actually paid in.

What that costs is the largest single error the rebuilt ledger carried. One account's fixed-income book reported +$29,424.91 for a single day against a true +$187.42 (itself restated to −$14.91 once the Fluid tick padding came out of the debt marks, M14), on a position whose entire capital at risk was $21,270 — a day's return 38% larger than every dollar ever committed to it. Two more positions carry the same shape in the other direction: one account's whole reported dollar return was a −$29,927 day that was truly +$50, and another's whole reported ether return was +41.36 ETH on a day it truly lost a fraction of one, more than twice the position's equity booked as profit in an afternoon.

So both halves are now recorded. The half that was paid nothing is written with a movement of zero — which is what the venue said — and its balance is taken from the venue's own reading at that block, exactly as the paid half's is. Two consequences worth stating, because a reader meeting these numbers deserves them:

• The reading is what closes the position, and it has to be. Four of the four settlements in the history leave one unit of the pool's smallest denomination behind: the withdrawal asks for one less than the position holds. Closing by arithmetic would leave the holding a hair above zero, it would never be marked closed, and the phantom would come back in a smaller form. The venue reads exactly zero, so the record says exactly zero. Where the venue did not answer for that side at that block, nothing is written at all and the gap is reported by name: no reading, no claim.
• The two halves' returns are mirror images of each other, by construction. A pool that re-mixes to pay out in one token moves value from one half to the other; per half, one shows a large gain and the other an equal and opposite loss, and neither is a statement about the world. The honest figure is the pair's, which is how the reporting engine attributes it — one result for the position, never two offsetting fictions per token. A per-token line read on its own is not a return. And if either half cannot be valued on the day it settles — a missing price, an unread endpoint — the position is withheld whole rather than published half: one half alone is the re-mix with none of its offset, which is the same phantom in a smaller coat.

A leveraged position's opening day is on the books

The day a leveraged position opens is a real trading day: the money reaches the pool, the entry costs what it costs, interest starts running, and the pool's two tokens move against each other for the first time. All of it happened between the moment the position was opened and the first daily mark, and none of it was recorded. When the position later closed, the whole missing amount arrived on the closing day as a gain.

The error was one-sided, which is what made it expensive: the way in was never charged and the way out was always booked, so every leveraged position read better than it was. On one account's May position that is $220 of return the wallet never earned on a strategy that made $26; on another, a position that lost $21 was reported as a $38 gain.

Two separate things produced it and both are fixed.

• The venue's own record of a position's birth is now kept. A position's opening movement carries two quantities: the amount the venue's event says was moved, and the balance the venue itself reports at the end of that block. On a borrowing they are not the same number — the vault rounds the drawn debt up to land the position on one of its price ticks, and for a two-token position the balance is struck at the pool's composition rather than at the amounts drawn. The ledger used to work out whether a holding was brand new by comparing those two quantities, so a rounding of a billionth of the position was enough to make it conclude the holding had existed beforehand. It then invented a balance for every day before the position was created, marked every one of those days as a day it could not value, and reported nothing for the opening day itself. The venue publishes the position's creation explicitly, so that is what the record now carries, and the question is answered from it.

• A two-token position's first day is reported at the position's level. These positions are reported jointly rather than per token, for the reason the previous section gives: a pool that re-mixes moves value from one half to the other, so a per-token line is not a return. That joint reporting needs a value for the position at both ends of the day — and on the day the position is created there is nothing at the start of the day to read, because the position did not exist yet. Rather than report nothing, the record now says what it knows: a position created inside the day was worth nothing when the day began. That is the same rule already used at the other end, where a position that closed inside the day is worth nothing when the day ends.

What changes for the reader, beyond the day itself. A position's at-entry basis and its "since" date both come from the same question, so both move on the positions this fixes. Before, the app could not see the opening movement as an opening, so it reconstructed what the position had been worth at entry from the first daily reading it could find, and dated the holding to that reading. Now it reads the entry itself, and dates the holding to the day the money went to work. The new answer is the better one, and it is the one the secondary-market gain-or-loss on the position row is measured from.

Two things the joint reporting deliberately does not do. Being created inside a day is not on its own a reason to report a position jointly: a position that opened without its pool re-mixing is still reported per token, and so is one whose settlement happened to land in the same day. Their opening days do now carry a figure where they carried nothing, because the position is no longer treated as having existed before it did — but that is the first change above, not this one. And where a position was only partly created inside the day — one half already there, one half new — the conservative reading stands, exactly as before: the day is reported only when the whole position's figure can be stated.

One shape reports per token even though the whole position was created and closed inside the day. Joint reporting needs each half's share of the position, and a share is read off the daily values — of which there are none at either end of a day a position both opened and closed in. The day's total is right either way (it is what went in against what came out); what it cannot carry is the split between the two tokens, so the halves keep their own figures and the day is not reported jointly. That is the same rule the app already applies when a closing day's shares cannot be established.

The settlement pass: the other side of a venue movement

Supplying, withdrawing, borrowing and repaying all settle in a single token transfer between the wallet and the venue. The semantic walk binds that transfer to the venue's event, and for the whole of the history build's range that is where it stopped: the position leg got its row and the wallet's own token holding never heard about the movement. A wallet that borrows and spends looked, to the ledger, like a wallet that only spent.

What that costs compounds, because a holding's balance is a running total. On the largest tracked wallet — a levered position that borrows and re-supplies in a loop — the borrowed token's recorded balance went negative and stayed negative for the whole four-month history, which no token balance can be. The reporting engine reads a non-positive balance as "this holding is gone", declines to report anything for it, and books the other side of the loop in full. One six-hour window published −$1.93 million of return against a true +$136, and over the whole period −$5.77 million against a true +$9,321.

So a movement a venue event consumed is now handed to whoever owns the second leg, and the wallet book writes its row there — a transfer tagged venue, carrying a pointer back to the venue row it settles. Two rows, two holdings, one movement, opposite signs. A venue's own receipt token (an aToken, a vault share) gets no such row: there the position leg IS the wallet's holding, and a second row would count it twice.

A fixed-rate principal token is not one of those, and reading it as one cost a user a multi-million dollar day. A principal token is an ordinary token the wallet holds directly. It has its own holding, kept under the fixed-rate venue's name only because that venue owns its price, and the app reads that holding straight from the token's own balance. So when a principal token is posted as collateral it really does leave the wallet, and the wallet's holding has to be debited exactly as any other token's is. It was not. For three months the app recorded a wallet as holding its principal tokens AND the lending market as holding the same ones: the recorded wallet holding was, to the last unit, the true balance plus the collateral. While the true balance sat at zero the app never looked at the holding, so nothing showed. The day the user pulled a slice of collateral back into the wallet, the app compared its first real reading against 5.25 million tokens it believed were there and charged the whole gap to one day: minus $4.96 million on a book whose entire gross is $36.6 million. Three days later the reverse move handed back a fake $386 thousand. Those two days were 99.95% of that account's published lifetime result. The reverse direction had the same hole, and on a second position the sale side WAS recorded while the collateral side was not, so the recorded holding ran to minus 1,052,250 tokens: the only negative balances anywhere in the ledger.

Both directions are now recorded, and it takes two mechanisms because the venues differ in what their own events say they moved. A Morpho collateral event names the collateral token, so the wallet's transfer is the movement that event settles and the settlement pass hands it over. An Aave or SparkLend supply names the venue's own receipt token instead, so the principal token's transfer is explained by nothing and reaches the classifier ladder, where rule 5 recognises it as a movement to a venue the same transaction carries a row for. The test that pins this is the user's own transactions, decoded from the chain, plus the registry's real Aave principal-token reserve.

One holding per principal token, and nothing books it twice. A principal token never gets a plain wallet-token holding of its own: the pipeline declines every address another venue claims as its position token, and on production no principal token is in the tracked-token registry and no such row has ever been written. The fixed-rate venue's rungs also run before the generic classifier, so even a future registry edit could not produce two.

What is deliberately still loud, and the test is where the tokens WENT. A principal token moving between the wallet and something that is not a venue (a sale on an exchange, a transfer to another wallet, an over-the-counter hand-off) is still reported as a movement the ledger cannot explain, rather than guessed at. Those shapes need the classifier's own rules about groups, rewards and address poisoning, which is not a question a fixed-rate registry can answer.

The rule for telling the two apart is the one the classifier ladder already uses: the receiving address is one a venue this run declared settles through, or the tokens reach one of those addresses through the same transaction's own transfers. The second half matters because a bundled supply hands the tokens to the bundler and the bundler hands them to the venue, so the wallet's immediate counterparty is a contract in no registry and the movement is a settlement all the same. Asking instead whether the transaction merely carries a venue row naming that token would be far too loose: for every venue in this family the venue's own event names the principal token itself, so a bystander transfer riding beside a real supply would be recorded as if it had settled it, tagged as a movement to a venue, pointed at a position that received nothing, and the alarm that says the ledger cannot explain a movement would be silenced by the very movement it exists to report. The corpus has one such bystander, on the same account, and the holding's opening reading of the chain absorbs its quantity correctly.

The set of addresses rule 5 recognises is built from the venues a run declares — the aToken for the Aave family (a supply sends the underlying to the aToken, never to the Pool), the Morpho Blue singleton, the vault for ERC-4626, the market and the router for Pendle, and the liquidity layer, factory and vaults for Fluid. It had never been supplied at all, so rule 5 could not fire on any transaction ever derived and every one of those movements was stored as ordinary external capital.

Correcting a stored history means rebuilding it from the floor, not re-running a recent range. Each holding's balance is a running total from an anchor, and the anchor a re-run picks up is the balance the previous run stored. Re-derive only the last few weeks and the corrected movements are added on top of the old, wrong starting point: the balance stays impossible, the holding stays suppressed, and the job still reports success. Rebuilding from the history floor has no stored balance beneath it to inherit, so every holding is re-anchored against the chain itself. That is the only form of the repair that actually clears the defect above.

The residual this left is closed, and so is the same defect on the leveraged-vault venue. The investigation that found the settlement gap also measured, on the same wallet, about 15,400 units of the collateral token (~$19.1k) that the corrected pass still did not explain. That is rule 1 above, and it is fixed. Separately, the leveraged-vault venue never went through the settlement pass at all — its own events explain no token transfer, so its deposits and withdrawals fell to the classifier ladder, where the venue's adapter deleted them for the same mistaken reason rule 5 used to. That deletion is not symmetric in practice: deposits are usually routed through an aggregator and survived, while withdrawals arrive straight from the venue and were deleted. On one wallet a stablecoin holding went 21,307 units short and a second holding lost the only record of a 67.66-unit arrival, leaving it reading empty from that point on; a third holding on another wallet went 4,170 units short. The venue no longer deletes anything; the withdrawal becomes an arrival on the wallet's own holding, tagged venue, beside the position row that states the other side.

Rule 7's heuristic has a measured false positive and is guarded for it. Run read-only over the 809 addresses in our own registries and tracked-wallet set, the 4+4 rule matches one pair, and both halves are real MetaMorpho factory vaults sharing a 0x7777…7777 vanity. So an address that is itself in our corpus is never tagged, and every spam verdict carries the address it imitates so it is auditable per row. Until the heuristic is measured against a wider negative corpus, the tag may be carried but must not hide anything.

Rule 6 is dormant. No stream ingests the ERC-4337 EntryPoint's event today, so the nine measured paymaster-gas rows still classify at rule 8, exactly as they do now. The rung reads the transaction's own logs rather than a hard-coded address, so ingesting that event lights it up with no code change.

Rotations: a wrapper hop is not a capital event

A wallet that moves GHO into stkGHO, back out, into sGHO and back out again has not contributed or withdrawn a penny. Today each hop books as a capital event and each shows up as its own line in the activity feed. The rotation matcher groups them: token_lineage resolves every address to the claim it denotes, and one accumulator per (wallet, claim) opens on the first movement out, extends on every later movement, and closes once the books balance and no further movement occurs for about thirty days. Every row of the episode carries the same group id, and the id names the episode, not the transaction — the production anchor is six rows in six different transactions spanning 39 days.

Three consequences worth stating:

• The group can be one-sided, and stays that way. The intermediate wrapper is in no registry, so there is no destination position to be internal to. No synthetic one is invented: marking it would need a rate we do not have, and marking it at par would fabricate the yield the grouping exists not to invent.
• What it fixes, and what it does not. It removes the phantom capital events and collapses the feed lines. It does not recover the episode's +69.534487 GHO residual: while the wrapper is unmodelled that value is attributed to nothing and the return line is short by exactly it. That is reported as a coverage note (a dated statement that value left what we model — never a row, never an alarm) carrying both the exact quantity and the mark-weighted figure, which are different numbers. Registering the wrapper is the fix.
• A range replay rehydrates open groups before it starts. The group id embeds the block the episode opened at, and that is the one id in the ledger that is not a pure function of chain data. Replaying [a,b] in two halves would otherwise open a second group with a different id in the second half, and the two runs would store different bytes. So the matcher reads every group that is still unbalanced (or still inside its quiet window) out of the ledger first, and never opens a new group where a stored open one exists.
• The group now reaches the part of the reader that needs it. A rotation is where the entry of a holding travels: the money that comes back out of a wrapper is the money that went in, so the returning holding inherits the basis and the date the original was put to work rather than being read as a fresh purchase. That inheritance is keyed on the group id and nothing else — and the reader was building its movements without the two columns that carry it, even though it read them out of the database and used them for the coverage note in the same pass. The effect was silent and total: for any wallet with a rotation the entry column fell back to a dash for every holding, with only a log line to say why. The columns are now carried. One shape is still refused rather than answered — an episode that hops through two wrappers in a row, which is what the production anchor does, because the reader resolves a group's shares in one pass and an episode that feeds itself needs more. That refusal is named, it degrades to the same dash rather than to a wrong number, and neither published return line reads the entry basis at all.
• A late arrival after an episode has closed is still not new money. A residual redeem or a cooldown claim can arrive weeks after the books balanced. It opens an episode of its own rather than being read as a contribution, because the matcher also carries forward which claims have already been through a rotation. Read the other way, it would show up on the chart as money arriving that nobody sent.

The replay universe is not the delete scope

Which tokens a replay reconstructs used to be the tokens the wallet holds today. That is a measured capital defect: a transfer of USDT between two of one account's wallets at block 25,279,714 recorded only the outflow, because USDT was not in the receiving wallet's present holdings and so was never in its replay universe. 1,965 of prod's 1,981 flow rows are replay products, so essentially the whole ledger is subject to the narrowing, and the account's net capital reads $5,763.98 low — a withdrawal nobody made.

No classification rule can fix that: every rule above operates on a row that exists, and this row was never written. So the universe becomes every wallet-tracked token the wallet moved at any block inside the replay window, read out of the event ledger (the union of every stream that can hold a wallet's Transfer row), unioned with what it holds now, and intersected with the registry so nothing enters that has no reader and no mark.

Three properties hold it together:

1. Bounded by the window, not by the present, so two adjacent range replays compose and the "replay in two halves and get the same ledger" property survives the widening.

2. Sourced from the ledger, not from balances. USDC and USDT cannot be scanned by address at any price, so the only way to know a wallet once held USDT is the wallet-keyed scan. That is why this depends on the wallet-token stream and its backfill: until those exist the widening has no source and contributes nothing, which is its state on production today.

3. A narrowing can never silently delete. Before a range's stored rows are replaced, the run asserts that its universe is a superset of the assets those rows already name. If a token has dropped out, the run refuses, and it refuses before it has written or deleted anything: no stored row is touched, in either table.

   The check runs before the FIRST destructive write, not before the delete it names, and that placement is the rule rather than an implementation detail. A registration replay replaces the range in two tables: it wipes and re-lays the position history first, then the flow ledger. Both re-lay only what the run's own universe can read, so a refusal taken between them would keep the flow rows while the position legs behind them were already gone. That state is worse than either alternative: the engine reads stored flow rows with no join to a position, so with the legs missing the value change across the window is zero while the movements are not, and every kept movement is booked as return. The refusal therefore sits above the whole destructive sequence, where "keeps the rows" is true of the whole record. (The sibling gap-patch path has refused before writing anything, for the same reason, since it was written.)

   A refused wipe is a FAILED run, and it is failed all the way out to the exit code. A shrinking universe is a bug and a bug must be loud, and loud here means an exit status, not a log line: run-cron.sh gates its whole alert block on a non-zero exit, so a job that catches its own condition, logs it and returns 0 is invisible to alerting however loudly it logged. So the run does not reach done and earns no coverage anchor — an anchor is a claim that a run completed, and it routes the next run to the gap-patch path, which never re-derives at or below it, which would freeze a ledger that is a mix of rows the run derived and rows it could not. It records error on the wallet's backfill state with the verdict and the remedy, the single-wallet CLI exits non-zero, and the minutely drain counts the wallet failed and exits 2 — which is what carries the [fail] line into the alert.

   The assertion runs on the registration replay — the one path that deletes and re-lays today — and it is where the widening lands, so the guarantee is not deferred to the offline builder. Two production wallets currently hold rows naming a token they no longer hold; a forced repair of either now stops before its first write and pages, instead of replacing that wallet's history with a narrower version of it, until the wallet-keyed backfill covers them. That is the guard working, not a fault: the operator's move is to re-run once that backfill has landed. The repair is also cheap when it refuses, because it refuses on the probe rather than after a full replay it would then discard. It is not free: a forced repair revokes the wallet's coverage anchor before it starts (so that a crash mid-replay cannot leave a half-replaced range behind an anchor that says otherwise), and a refusal does not put it back, so that wallet's later runs replay-and-refuse instead of patching its tip until the backfill lands and one successful run re-earns the anchor.

   One case where re-running is not the remedy, and the line says so. Native ETH enters the universe only from a live non-zero balance read — it has no transfer log to scan for — so a wallet that has since spent its ETH to zero names it in its stored rows and can never name it in a universe. Its refusal is permanent, and telling that operator to wait for a backfill would be telling them to wait for something that cannot arrive. The refusal is still correct (the run genuinely cannot re-derive those rows, so wiping them would destroy the only record of that history), and the failure line says which of the two cases the wallet is in. Not observed on production today.

   What the check covers, stated so it is not read as wider than it is. The assets compared are the ones the wallet's stored bare-token movements name at or below the run's anchor, which is what this section is written about. A venue's own asset universe — a vault or a market that has left the registry — is that adapter's to guard, and the destructive replay has no equivalent refusal for it yet where it re-lays the position spine, though the sibling gap-patch path has refused in exactly that case since it was written. The asymmetry is closed on the ledger side by the bounded merge below, which owns every venue's universe rather than only this one.

What changes on the page, and when. Nothing changes when this ships: the wallet-keyed history does not exist on production yet, so the widened universe has no source and admits nothing. Once that history is backfilled, a wallet that is re-derived — a new registration, or a requeue after an error — gains flow rows for tokens it no longer holds, gains snapshot points for them across the window, and may start its curve at an earlier block, because the first thing it ever did can now be a token it has since spent to zero. Its past curve and its activity feed change shape. That is the fix working, and it is what wants eyes during the release validation.

The bounded merge, and the rule that protects a close

The ledger is written by a range-bounded merge, never by the delete-and-relay the retired chain scan used. The unit is one wallet over one block range, and it is bounded three ways at once:

• by range — nothing below the range's bottom is touched, ever;
• by venue — only the venues the derivation actually covered, taken from the adapter set it ran with and never inferred from the rows it produced. Inferring it looks like a simplification and is the bug: a venue that legitimately produced nothing would narrow the delete to zero exactly when a phantom row on that venue needed removing, and the phantom would then survive every replay forever;
• by derivation — only rows the re-derivation did not reproduce are candidates for removal.

The rule that protects a close. A replay may update a terminal receipt's derived fields. It may delete one only if its own derivation covered that row's venue, and that venue's stream is certified over the whole range, and the derivation produced a replacement terminal row for the same leg. "Replacement" is judged by the leg, not by the log: a re-derivation that finds the true exit at a different log is replacing the receipt, not losing it. Otherwise the delete is refused. Thirty-two closed position groups carrying $2,156,384.76 of notional are live in the production ledger today and every one of them carries a close; under the exit rule a lost close is not an approximation, it is a series that never ends and books 0 forever.

The venue half of the universe assertion lives here, and it is the sibling of the token half above: for every venue a run covered, the legs the run could resolve must be a superset of the legs its stored rows already name. A vault delisted from the registry, or a Pendle market rolled off it, narrows exactly the way a token does, and the answer is the same one — refuse the delete, keep the rows, record the anomaly. A covered venue whose universe the run cannot state at all refuses too: "nobody can say what this run covered" and "this run may delete" cannot both be true.

A transaction the derivation could not read is not an empty one. The derivation contains a per-transaction failure rather than propagating it, so that one bad decode cannot take down the live tick — but a transaction that produced nothing leaves its stored rows looking exactly like rows nothing re-derived, which is the definition of a phantom. So the merge reads the range's own failures, not only its rows: a contained adapter failure refuses the delete and the certificate for that range. A kept row costs a re-run; a deleted one is gone, and nothing re-derives beneath a certificate.

A refusal stops the delete and the certificate, never the upsert. Correcting the rows a run could see never loses a movement, so the upsert proceeds. Withholding the certificate is the other half and is not optional: a run that could not safely replace its range has not shown that it derived that range, and the cursor below is exactly that claim.

Idempotency, stated so it is achievable. Replaying [a,b] twice, and replaying [a,m] then [m+1,b], both land on the same ledger as one [a,b] replay — compared over every column except updated_at, by content hash. Excluding it is not a weakening: updated_at defaults to now() and the merge is an upsert, so including it makes the criterion unsatisfiable by construction. Two independent mechanisms keep the claim honest and each catches the other's failure — the writer preserves updated_at on a value-identical upsert, and the hash excludes it — plus a separately named assertion that a no-op replay leaves max(updated_at) unchanged.

The derive cursor, and what complete means

Each wallet's derivation carries a resume point at scope portfolio:derive:v2:<chain_id>:<wallet> in chain_scan_cursors. It advances to a range's top only inside the transaction that commits that range's merge — never before, never in a separate statement — and it advances with the house GREATEST upsert, so a re-run over a lower range can never walk it backwards. Without it, a build that dies mid-wallet leaves a ledger whose series stop at some block with nothing recording where, which under the exit rule is indistinguishable from a series that never ends.

It is a wipe-on-reset family (see database.md): the staging scrub and the user wipe clear it, which is what stops a cursor outliving the rows it claims. The campaign-global arm block deliberately sits under a different prefix that nothing deletes.

Two invocation modes, and each has its own callers.

mode	invoked with	start block	who uses it
sweep	a top only	the cursor + 1, or the floor with no cursor	the history build and its resumption
ranged	both bounds	the bottom it was given, cursor or no cursor	every re-derivation remedy

The ranged mode is required rather than convenient. Every wallet a re-derivation remedy targets is by construction already certified — that is the hazard it exists for — so under the resume rule alone such a run resolves its start above its own top, derives nothing, exits clean, and leaves an operator recording a repair that never happened. Re-deriving the whole range is safe because the merge is idempotent and the cursor write is monotone, so it costs archive reads and nothing else.

One property neither mode may break: a walk may only certify what it derived if its bottom is contiguous with what is already derived — at the floor, or at the cursor + 1. A ranged run starting above a hole derives a real range and would certify across the hole, so the claim is computed from the bounds rather than declared by the caller. The worst a mistyped bound can do is derive rows and decline to certify them.

complete is one definition with two halves, and both are checked in one place:

• (A) the wallet's derive cursor reaches the recorded arm block — never a live head. From the arm onward the live writers advance the ledger without touching the cursor, so a predicate against a head would make every wallet incomplete at flip time, permanently;
• (B) the wallet's required-scope conjunction reaches the coverage floor — the same per-wallet certificate that gates publishing anything about that wallet.

Neither half is the claim on its own. (A) alone certifies a wallet whose bare-token history is still being fetched: that stream is enrolled per wallet at registration and its catch-up is rate-capped, while the minutely replay finishes in minutes, so the replay usually wins and a cursor stamped on the caller's word alone would certify a wallet missing its plain-token deposits, its cross-wallet transfers and its reward receipts — permanently, because nothing re-derives below a cursor. (B) alone says the ledger reaches back far enough, not that anything derived it.

So the stamp requires both an intent and an evidence check: the caller declares whether it derived a history or a tick (a tick may never certify a history, whatever the coverage says), and the same transaction proves the coverage. When the intent is there and the evidence is not, the rows are written and the cursor is left alone, with a [v2-partial] line naming the scopes it is waiting on. That is an ordinary, self-clearing state for a newly registered wallet, not a fault. An empty requirement set never certifies — an empty conjunction is the absence of evidence, not evidence — and a zero-row derivation still stamps, because a wallet whose history is entirely above the arm block legitimately derives nothing and must still be able to become complete.

Four cases, exhaustive, and each has a different remedy: cursor at the arm block with a certified conjunction is complete; a short cursor resumes the build; no cursor waits on (or needs) a registration replay; a cursor past the arm block over an uncertified conjunction waits on that wallet's enrolment backfill. The fourth has to be named or an implementer reads the first as sufficient. What it cannot see is a global stream enabled after the build: its marker certifies every wallet at once, so an already-stamped wallet reads as complete over blocks that stream never covered. The guard for that is the rule at the enable site — enabling a stream after the history build requires re-deriving the affected wallets over the campaign's range, in the ranged mode — not this predicate.

The derivation itself: one composition, five callers

Every writer below — the offline builder, the three live writers and the re-derivation trigger — runs the same derivation over a (wallet, block range). It is one composition (src/lib/portfolio/derive/compose.ts), and the five callers pass it explicitly rather than inheriting it, so a caller that stops passing it is a failing test rather than a wallet that quietly stops being derived.

What the composition does, in order:

1. Loads the venue registries — the Aave and SparkLend reserves, the Morpho markets, the ERC-4626 vaults, the Pendle markets, the wallet-token registry — and builds one adapter per venue whose registry loaded. A registry read that failed refuses the range outright: a run that cannot say which venues it covered must not certify one. The ERC-4626 universe is the same two halves the rest of the app resolves a vault against — the curated set (the curator funds, the Fluid fTokens and the multi-strategy funds) unioned with the vaults the MetaMorpho factory scan found — because the database table holds only the second half, and every vault position the tracked wallets actually hold today sits in the first.
2. Reads the wallet's own positions and refuses the range if any of them sits at a venue this run did not cover. This is the composition's terminal, and it is above everything: deriving the covered venues anyway would write rows and let the range be certified as derived while a venue the wallet actually holds was never read. The terminal also runs one level down, because the venue is not always the unit of coverage: for the registry-backed venues the unit is the vault, the reserve, the market or the PT, and a position whose unit is missing from the registry is dropped by the decoder with no row and no alarm while its venue still reads as covered. So a position the run covered the venue for and still cannot name refuses the range too — the alternative is a wallet reported as fully derived over a position it holds.
3. Loads the range's raw events from the event ledger, on the streams the adapters read.
4. Resolves the Fluid position states the range needs, through the chain-read seam (compose-chain.ts). Fluid is the one venue whose universe is per wallet and whose liquidation receipt is a change in position across a block rather than an event, so a liquidation in range with no state read available refuses the range.
5. Runs the derivation: decode, the unconditional drops, the guards, the semantic walk, the classifier ladder, the post-pass. The wallet-book classifier runs last, after every venue's own rungs, because it is the least specific claim there is.
6. Reports what it covered: the adapter set it ran with (never what it happened to produce) and, per venue, the positions it could have resolved. The merge scopes its delete to the first and refuses on the second, so both are statements about the run, not about its output.
7. Puts a value on every row, at the row's own block. Each receipt is valued on the three stored lines: the market mark in the position's book, the redemption mark in the same book, and the dollar figure that lets capital net across books. Three rules decide what "its own block" means, and each exists because breaking it produces a plausible wrong number rather than a missing one:
   • The market price is the one that was quoted at that block's own minute, taken from the price mirror's covering bar; the wrapper compositions and the redemption rates are read at that block too, never at the newest one. Valuing a months-old movement at today's rate is a silent few-percent error that lands straight in attributed yield. The covering bar can be up to an hour old: the mirror's grid IS the hour (see Token price bars), and a composition that values at every receipt block could not afford a finer bar per block even if one existed — but both quotes still come from the same bar, so what is stale is the slow basis leg and never the exchange level.
   • An asset with no price of its own is composed, not skipped. A vault share and a par-rebasing base are marked from their underlying's price and their own share rate at that block, which is how the curator-fund holdings and the levered dollar book get a market value at all. The sources those compositions read are loaded in the same batched read as the assets that moved, so composing costs no extra round trip and no extra Dune credit.
   • The denomination is the position's, not the asset's. A receipt has to be expressed in the same book as the level it nets against, so a purchase paid for in another currency is converted into the position's book rather than stored in its own.
   • What a Pendle purchase cost is what the buyer actually paid, not the pool's mid-price. Minting is the exception: minting principal costs par, and part of that payment is the yield token creddit does not track, so the receipt is valued at the principal's own mark and the remainder is reported as value that left the perimeter rather than booked as a day-one loss.

A value that cannot be stated is left empty and named. It is never zero and never assumed: an unreadable token precision, a price bar the mirror does not hold, an oracle too young to answer — each leaves that row's market figure empty and raises a counted mark-unresolved anomaly naming the asset and the block, with one log line per asset carrying how many of its rows are affected and the lowest block, so a repair targets the asset rather than every wallet that touched the block. Everything downstream withholds off an empty mark instead of reading it as zero, so nothing is ever booked at a value nobody measured.

Completeness is measured on both money figures, and a half-valued row counts as unvalued. The redemption mark needs no price bar — for a par holding it is the identity — so a row whose bar is missing still carries a redemption figure and looks populated. It is not: the reconciliation is computed from the market mark, so that row is as unusable as an empty one and is counted as one. The dollar figure is counted the same way and can be empty on its own, because it is the market figure carried into dollars by the book's own unit: a holding of ether itself is worth a real number of ether with no price lookup at all, so where the ETH price is missing it has a book figure and no dollar one, and cross-book capital netting withholds the whole transaction on it. Only the redemption line has a by-design empty, which is why the count is never taken on it: a holding outside the yield books has no redemption claim, and a principal token's redemption mark is the pull-to-par curve the read path derives from the position's own entry.

So the price mirror's own history is a precondition of a valued historical build, not an optimisation — and it is a precondition per token, not in aggregate. A row is valued against its own token's bars, so the number that decides whether a range can be priced is the worst- covered token in it, never the mirror's earliest bar overall (those are more than a year apart on prod). The tokens counted are the ones a row can be denominated in — the tracked-token registry, the lending-reserve underlyings and the Pendle underlyings — not the ones seen emitting a log under their own address, which is the token only on a plain transfer and is the venue on every pooled or wrapped stream; scoping by the log stream drops exactly the assets a lending ledger is made of. A token the mirror holds no bar for at all is the worst case, not an absent one, and the check reports it as such rather than letting it fall out of a comparison over first bars. Filling a gap one window at a time from the pricing provider is bounded by a daily credit guard that stops silently once it trips. Loading the mirror back to the ledger's first event before the build is one long-window query and a couple of credits; the runbook states it as a numbered step, together with the exact load command (the check prints one naming the short tokens, because the loader's default set is its own and does not carry them) and the re-check that says whether the load reached far enough. A range derived before the resolver shipped, or below a token's own first bar, carries no market value and has to be re-derived; there is no re-mark repair for the rebuilt ledger.

Which rows are final. A row at or below the chain's settle line (the finalized head, with a 64-block floor under the range's own top) is written on the settled basis its writer declares; above it the row is written provisional, because a reorg can still take it. The historical build declares backfill and reads no finality at all — it derives to a recorded arm block, far below any settle line. The distinction is what the reorg repair finds its rows by. Finality does not un-happen, so a row once stored settled is never re-labelled provisional by a later pass that ran at a lower line (the merge keeps the stored basis); a settled log a reorg did orphan is not re-labelled either — its re-derivation no longer produces it, and the merge deletes it.

Where the post-event balance comes from, and why the live path costs nothing. Each position's running quantity is carried forward from receipt to receipt, so only the first receipt in a range needs a starting point. That starting point is read from the ledger itself — the same position's closing quantity on its newest earlier row — which means a range that follows another range on the same position anchors from Postgres and issues no chain read at all. The archive is reached only by the first range that ever touches a position, which is the historical backfill's cost and not the live path's. Where neither source can answer, the quantity is withheld and named rather than assumed: a fabricated zero would book the position as empty and every later receipt on it as a fresh entry.

build-shadow-ledger.ts (the historical ledger builder)

DATABASE_URL=<url> npx tsx scripts/build-shadow-ledger.ts \
  [--wallet 0x…,0x…] [--from <block>] [--to <block>] [--range <blocks>] \
  [--re-entry | --checkpoint-restored] [--dry-run]

Walks each tracked wallet's history through the derivation into the ledger, one bounded merge per range, each in its own transaction with the portfolio write lock held only for the merge itself — the derivation's archive reads happen outside it, so the 6h refresher does not queue behind a whale range. It writes flow rows and the derive cursor; the position history, the backfill queue and the chart's building state are untouched. It is not a no-op for the product, though: the rows it writes are the served ledger, so a wallet it re-derives changes what that wallet's page shows.

Omitting --from selects the sweep; passing it selects the ranged mode. --to defaults to the recorded arm block, and the run refuses if there is neither.

A sweep bottoms out at THAT WALLET'S history floor (accounts.history_floor_block, or the derivation floor 24,136,053 = 2026-01-01 when it has none), never at the raw ingestion floor. The event store is certified deeper than the portfolio view is served, and a walk that used the store's depth would spend hours deriving a year nobody is shown — and would stop on the first venue read no contract can answer, leaving that wallet permanently uncertified. Reaching below the wallet's OWN floor is the same waste plus a certificate its bare-token coverage row cannot back, so the stamp would be withheld and the wallet would never complete. The run prints each wallet's floor on its --dry-run line. --to beneath the derivation floor is refused for the same reason: no range exists down there.

The entry check runs before the first wallet: no tracked wallet may already carry a derive cursor, which is the mechanical evidence that the drain pause held. There are exactly two legitimate non-zero causes and they are different things, so the operator declares which: --re-entry (this step died part-way and is being resumed) and --checkpoint-restored (a restored ledger checkpoint brought the cursors back by design). The second is refused in the sweep mode on purpose — over restored cursors a sweep derives nothing, confirms completeness on the restored cursors and satisfies the idempotency triple vacuously, after which the reconciliation reports green over rows built by the previous release's code. Clear the restored cursors, or re-run ranged.

The check binds the sweep, which is the run the campaign is measured by. A run given an explicit range prints the count and proceeds: every re-derivation this programme prescribes targets a wallet whose cursor is already past the range it is handed — that is the hazard the ranged mode exists for — so a check applied there would refuse every repair, and the only way through would be declaring a restored checkpoint that did not happen.

The exit code is the contract. Non-zero on a refused delete, any unresolvable anomaly, a contained adapter failure, a wallet that derived rows and could not certify them, a failed entry check, an absent arm block, a --to beneath the derivation floor, or a flag typed without its value. A refusal that only logs is invisible to alerting, so every one of them is also a [shadow-build/fail] line with the wallet leading it, and the closing line reports how many wallets stamped against how many were in the run rather than asserting that all of them did. What to do when a wallet refuses is in the runbook.

The per-wallet wall-clock is its own log field rather than a step total, because it is the number the campaign's capacity planning and the re-derivation trigger's time budget are both set from.

Keeping the ledger current: the three live writers

The builder above is historical, offline and one-shot. Three writers produce ledger rows every day, and they are the only three there are:

writer	when	what it derives	how it is bounded
the 6h portfolio refresher	every 6h tick, after the window's snapshot write has committed	its own block window (below), one wallet at a time, up to the ingestion ceiling	its own transaction per wallet, the write lock held for the merge only
the registration replay	once per queued wallet, after the replay's own transaction commits — and only once that wallet's own bare-token history has been fetched (below)	the wallet's whole history from the derivation floor (2026-01-01), up to the ingestion ceiling	a second, separate lock hold — never an extension of the replay's minutes-long one; the queue's 30-minute per-wallet timeout bounds the pair
the JIT persist	on a /portfolio page load, after the positions read	from the lower of today's floor and the 6h tick's cursor, to the page's own anchor, up to the ingestion ceiling	the try lock: it skips rather than queueing

Every one of them stops at the same ceiling: the slowest stream the ingester has finished. The three writers pick their block windows from three unrelated places — the served tick's own scan window, the derivation floor, the block a page load anchored on — and none of those knows how far the event ingester has got. So each window's top is clamped to the minimum progress across the event streams a derivation is allowed to rely on, and a stream that is declared live but is not being followed pins that ceiling at the floor. Deriving past a stream that has stalled would read it as "nothing happened" rather than "nobody has fetched this yet", and the two are indistinguishable once written: the merge would commit an empty range as a derived one, over blocks its own cursor then passes and nothing re-derives. Clamping instead costs, at worst, a range the next pass picks up — and on the 6h tick that shortfall is recorded rather than dropped (below).

A fourth pass writes the same rows earlier: the ledger worker. It runs the ingester's continuous jobs through the same writer the 6h tick uses, with the tick's own rules and from the tick's own floor for the wallet (the tick cursor + 1, or the wallet's pending marker; it reads the marker and never clears it), so a movement the ingested range shows lands within about one ingester cycle. Starting where the tick would start is what makes "the same rows" true: a job that started at its producer's cursor instead anchored on whatever stretch above the tick cursor happened to be derived — nothing, on a first cycle — and wrote wrong running balances (PR #949 review round 2). The three writers above still run beside it for this release; the merge being idempotent over a range and both starting on the same derived prefix, whichever pass comes second writes nothing new. That holds while nothing rewrites the ledger under a job: when a whole-window merge lands between a job's derivation and its merge, the job merges nothing and derives again (the stale-merge fence). (The one input that can still differ between two derivations of the same stretch is a price bar that lands between them — the page load's residual too; the programme's values-at-read change retires it.)

Every one of these writes is unconditional, with no environment switch in front of it.

A merge failure is a run failure. Every call is wrapped so it can only return, and the failure is counted and summarised in one line per run rather than one per wallet. There is no second ledger to fall back on, so a tick that could not persist a wallet's range exits non-zero under [partial] and the cron alert fires. The one carve-out is a range partial ONLY because a wallet's own bare-token enrolment has not landed: nothing of that wallet is served, the drain reports the wait and the 6h alarm carries it, so a second page would be one alarm too many. The figure is printed either way; only the exit code narrows.

A write that does not happen is bounded, named and self-healing. Whenever a wallet has a block its own cursor has passed and the merge has not covered, that block is recorded as the wallet's lowest unpersisted block; the next 6h tick widens its own range down to it and clears the record inside the transaction that commits the merge. So the gap costs one tick of lag, never a permanent hole. Three cases produce one:

• a page-load persist that could not take the writer lock (it skips rather than queueing);
• a single wallet whose merge failed inside an otherwise healthy tick — including a tick whose range was clamped by ingestion and then failed on that narrower range. The clamp ALONE no longer produces a record: the tick's own cursor stops at what it actually derived, so the un-derived blocks are simply the next tick's floor. (Before the tick had a cursor of its own it borrowed one that advanced regardless, and the record was the only thing keeping those blocks reachable.) The other two writers need no record for a clamp either: a page load's window is re-derived by the next tick from below it, and a registration replay's window is the whole history, which the next replay or re-derivation starts from the floor again;
• a single wallet whose merge committed without covering its range. A derivation contains a per-transaction adapter failure on purpose, so that one bad read cannot take the served tick down — it reports it rather than aborting. The merge then keeps the affected stored rows and writes the rest, which is right, but one transaction inside that range produced nothing. The same holds when a registry could not name every position the wallet already holds. Such a range is recorded exactly like a failed one; the wallet is retried on a widening range every tick until its cause clears, and it is named in the log each time.

(The record keeps the LOWEST block, which is the opposite of the house idiom on that table; two gaps inside one window would otherwise collapse into the later one and lose the earlier. It has deliberately no attempt cap, unlike the re-derivation trigger below: giving up on it would leave a block a cursor has passed with nothing anywhere recording that it is owed.)

The 6h tick's own window

The tick's ledger pass used to borrow its block window from the old pipeline's flow scans, which is why the record above had to carry the tick's own shortfall: those scans advanced their cursors whatever the rebuilt half reached. Those scans are deleted, and the pass keeps a cursor of its own — portfolio:v2:tick, one row for the whole tracked population, in the same chain_scan_cursors table every other scan cursor lives in.

The floor is that cursor plus one. The cursor stops at the chain's finalised head, which is the same rule every old flow scan's cursor followed and it is load-bearing for the same reason: everything the tick derived above that line is reorg-exposed, so it sits above the cursor and the next tick re-derives it. Nothing re-derives below a cursor, so a cursor saved at the top of the derived range would strand that tail — a reorg that moved a receipt inside it would leave the moved row unreplaced for good. The re-derived overlap is tens of blocks against the ~1,800 a 6h tick covers, and the merge restates a range rather than appending to it, so the overlap re-writes the same rows on the same keys.

The ceiling is unchanged — the tick's own scan bound, clamped to the slowest ingestion progress across the streams the derivation reads. What the cursor adds is that a tick which cannot reach that ceiling now simply keeps its floor: the next tick starts where this one stopped, instead of having to be told about the gap by a per-wallet record.

The page-load write derives from the lower of that cursor and its own fixed lookback floor, up to its own anchor, so the two passes agree on where the ledger is current rather than each keeping a private idea of it — and taking the lower of the two means the window can only ever widen against what shipped, never narrow. That is not decoration: a wallet with no snapshot at all takes a fixed lookback below the anchor, which can sit beneath the population's cursor, and such a wallet was not in the population the cursor speaks for. It never advances the cursor: a page load derives one wallet.

A tick that finds no cursor bootstraps, in order: the lowest per-wallet derive cursor over the tick's own population (the highest block every wallet in it is already derived to, so this re-derives a little for the deeper wallets and skips nothing for any of them) → the recorded campaign arm block → nothing at all, said loudly. (A fourth rung above these took the old flow scans' window, so the release that introduced the cursor changed over with neither a gap nor a re-run of history; it went with those scans.) Each tick names the rule that produced its floor on one log line; the runbook has the exact line and what to do about the last case (deployment).

The ledger worker and the derivation queue

One always-on process derives the ledger from a durable queue (portfolio ledger-first plan, ruling R9): creddit-ledger-worker (scripts/worker/ledger-worker.ts, started as a pm2 process like the ingester, deployment) drains portfolio_derive_jobs one job at a time. Every job is one wallet over one block range, through persistV2WithPool — the writer the three live writers above call — so the merge takes the portfolio write lock exactly once, transaction-scoped, and nothing derives inside the hold.

kind	enqueued by	priority	runs as
sync	a page load's refresh (Synchronize)	0, first	the tick's step over its own range, with the page's own settle line and the page load's held-PT rule (settled legs only); refused while its producer is dormant (below)
enrol	a registration's ledger half, after its gate	10	the registration replay: the whole window from the wallet's own history floor, the stamp asked about that floor, the deferral marker cleared on a write; refused while its producer is dormant (below)
continuous	the ingester's cycle end (above)	20	the tick's step from the wallet's tick floor through the job's top, with the cycle's settle line and the page load's held-PT rule (settled legs only)
rederive	an operator (or, later, the trigger)	30	the re-derivation trigger's launch for one wallet, with its three launch guards: held while the wallet's registration replay is parked or pending; the whole window from the wallet's own history floor, the stamp asked about that floor; no stamp below the recorded arm block (held while the ingested tip is behind it); the marker cleared on a stamp
sweep	the 6h tick, one job per wallet of its population	40, last	the tick's own step, with the tick's settle line; the tick cursor moves when the WHOLE sweep is done (only once the worker owns derivation)

The table's pdj_priority_chk ties each kind to that priority, so a hand-typed row cannot jump the queue.

A job runs exactly as the call site it stands in for. A tick-range job (sweep, continuous, sync) is the 6h tick's per-wallet step, argument for argument: the same site (it queues for the lock rather than skipping), the same composition at the job's settle line, a pending marker for any range it leaves unpersisted, and the held-PT registry and the wallet index fed from the legs it inserted. That equivalence is a test, not a claim: scripts/worker/sweep-parity.test.ts runs a sweep through the worker and the tick's own step over the same range on the fixture database and requires the same ledger rows (their basis included, each side handing the composition its own settle line), pending markers, registry rows and tally — over a scenario whose legs feed both registries (a wallet's first ERC-4626 vault, a Pendle PT bought through the router), so a worker that fed neither fails it. The same file runs a worker soak: the ingester's own producer and the worker's own loop over three seeds of thirty simulated ingester cycles of random movements, with provider failures and retries, a parked job and its close, ticks whose wallet merges fail (markers), ticks fired between a job's derivation and its merge (the fence stops those merges whenever the tick moved the job's wallet's ledger), and jobs run after a tick that passed them, checked after every cycle against the true running balances and against a copy on which only the tick derives. The fence has fixture cases of its own: a whole-window merge landing between a job's derivation and its merge, before and after the tick passes the wallet; one already inside its transaction when the job began (committing while the job derives, or while it queues for the lock); and the tick's own straddle, inline and as the flip's sweep job.

A rederive job certifies nothing under conditions the re-derivation trigger would not (PR #949 review rounds 3 and 4). A whole-window job (enrol, rederive) derives a wallet's whole window and, when the evidence holds, stamps its derive cursor — the certificate that lets the served reader stop excluding the wallet. So the worker applies the trigger's three launch guards to a rederive, read when the job runs, and the first to its enrol too. The floor: the job opens at the wallet's own history floor, read through the one reader every whole-history writer uses, and the stamp is asked about that same block; a job whose from_block names any other block derives nothing and ends partial naming the floor. (Taken from the row instead, an operator's rederive over a recent stretch certified the wallet from a block ~264,000 above its floor: #756's stamp defect.) The hold: a wallet whose registration replay is parked (error) or still pending (queued, running) has a committed history that replay never built, so its evidence is a short leg set and a stamp would certify over a history nobody laid down; the job ends partial having derived nothing, and the wallet stays cursor-less (and excluded) until its replay completes and stamps it. The arm: completeness is defined against the recorded arm block, so a derive cursor stamped below it reads as a short build for good — and, being a cursor, stops the wallet matching the trigger, so nothing comes back for it. The job is therefore held while no arm block is recorded or the ingested tip is still below it (the trigger declines outright then too, and derives a cursor-less wallet by itself once the tip passes the arm), and refused when its own to_block is below it, since the stamp lands at the top the merge reached. An enrol job runs the floor rule only. It is the replay's own ledger half, enqueued after the replay's gate, which runs after the replay has written its terminal status — so the replay's row then reads done, or queued for a gap patch its wall-clock budget cut short. A hold on the status would stop that healthy gap patch, and the one fact that parks a replay, its own refusal, is the gate's already; the replay's inline call runs no arm guard either. Its from_block is always the floor, because the replay enqueues [floor, tip] from the same read.

A sync or enrol row is refused while its producer is dormant (PR #949 review round 4). The page load and the registration replay enqueue these kinds only once WORKER_OWNS_DERIVATION flips, so until then a row of either kind was typed by hand — and each trusts its producer for a guard a typed row does not carry. A sync derives exactly its own range, because its producer hands it the page load's floor, already at or below the tick cursor + 1: typed above a movement the tick has not derived, it anchored on a stale row and stored a balance 1,000 short. An enrol is not held on the wallet's registration status, because its producer runs after the replay's gate: typed for a parked, cursor-less wallet, it certified a history no replay laid down. So the worker ends either partial having derived nothing, the reason naming the runbook; the flip decides how a hand-made job is told apart from its producer's.

A continuous job starts where the tick would start for its wallet (PR #949 review round 2). A derivation of [from, to] anchors every position's running balance on the newest stored row below from — and rebuilds its rotation matching and its Fluid position-NFT set from the same rows — so it is only as right as the ledger below its range. The tick, the page load and the registration replay always start on a derived prefix (the tick cursor or the wallet's pending marker, min(last reading, tick cursor + 1), the history floor). A continuous job's from_block is the producer's cursor, up to one tick window ahead of the tick cursor, and the stretch between is derived only by whatever job happened to cover it: none on a first cycle or after a skip-ahead, a newer one first when a retried job waits out its backoff. Started there, a job anchored on a stale row and wrote wrong running balances — a +500 receipt that lowered the balance, a $52,000 book read as $0 until the next refresh — and in the rarer order where the job merged after the tick had passed it, it left them below the tick cursor for good. So a continuous job (and a sweep an operator inserts while the inline tick owns the window) derives from min(from_block, the wallet's tick floor, its pending marker), read when the job runs: the tick cursor + 1 in the steady state, and on a box that has no cursor yet the same bootstrap the tick itself uses (the wallet's own derive cursor + 1, then the recorded arm block; with none of them it derives nothing, and neither does the tick). Its anchors, rotations and NFT set are then the tick's, so in any order a job and the tick run they write the same rows (a third writer that rewrites those anchors while the job runs is the stale-merge fence's, below). The cost is re-deriving up to one tick window for one wallet per job — what a page load already does on every refresh — and more while the tick itself is behind. A sync job keeps the page load's own floor, which is already at or below the tick cursor + 1; the tick's own sweep starts at the cursor by construction.

A job never merges a derivation its wallet's ledger has moved under (PR #949 review round 5). The floor makes a job anchor where the tick anchors, which holds while nothing rewrites the wallet's ledger below the job's top during its run. A whole-window merge does exactly that: the registration replay's ledger half and the re-derivation trigger replace the wallet's whole window, and change its running balances when events below the tick floor were ingested after the tick derived them (the enrolment catch-up, a gap patch). A job whose derivation straddled one merged balances anchored on the ledger as it stood before — 1,000 short on the fixture, served at once because the whole-window merge had just certified the wallet, and below the tick cursor for good when the tick passed the wallet in between. So every tick-range job runs the writer's stale-merge fence: it fingerprints the wallet's rows at or below its top (how many, and the exact sum of their updated_at, which the upsert re-stamps only when a row's content changed) immediately before it derives, and again inside its merge's transaction once it holds the write lock. If the two differ it merges nothing, and the job goes back to the queue without spending an attempt; the retry starts from the wallet's floor on the ledger the other writer left. It compares fingerprints rather than asking whether any row changed since the job began because updated_at is a writer's transaction start: a whole-window merge already inside its transaction when the job began (upserting a whale's window under the lock, or queued for it) stamps its rows before that moment. The three inline writers do not run the fence, so the 6h tick's own step can still straddle a whole-window merge the same way: it derives, the replay or the trigger rewrites the wallet, the tick merges its stale rows and its cursor passes them. That is a defect of the inline tick that this release does not change; it closes when the tick's derivation runs as the worker's fenced sweep jobs, which is why the fence is a condition of that switch. The fence guards a merge against a ledger that moved under its own derivation; it cannot reach rows already committed, so the mirrored order stays open: a whole-window merge re-derives only through the tip it read when it began, and rows a tick-range merge committed above that tip in the meantime keep their old anchors. A continuous job's rows there sit above the tick cursor, and the next tick or the wallet's next job re-derives them; the inline tick's (and, after the switch, a sweep job's) sit below it for good, until every whole-window writer runs through the one worker. A whole-window job does not run the fence: it re-derives the wallet's window from its history floor instead of building on the ledger above it, as the replay's and the trigger's own calls do.

Only the tick's own sweep absorbs a pending marker. A marker records only the bottom of what a wallet is owed; the top is the tick cursor it was written beside, which the tick's own next range always reaches. A continuous or sync job's range stops wherever its producer's did — for a job queued before a tick and run after it, below that tick's cursor — so a job that absorbed the marker would merge [marker, its own top], clear the marker as covered, and leave the rest of the owed range un-derived and un-marked below a cursor nothing re-derives. A continuous job therefore READS the marker (its floor) and never clears it; it stays for the tick. A sweep absorbs it only once the worker owns derivation, when its range is the tick's.

The held-PT registry takes settled legs only from a job that derives the newest blocks.portfolio_held_pts never drops an entry, so a leg a reorg can still take away must not enter it: a continuous or sync job (and a hand-made sweep) admits only the legs at or below its settle line — the page load's rule, D5 — and the tick admits the rest within 6 h, once settled. The wallet index takes every inserted leg (a spare membership costs one read of a market the wallet left; a missing one would drop a live leg from the bounded read). The tick's own sweep keeps the tick's no-filter rule for both.

The settle line travels with the job (#777); a settled row is never relabelled. A job carries the finality line its producer read, raised to the tick cursor when the job runs (every block at or below that cursor is final, so a row the job derives there is written settled). That raise cannot see a tick that is still running — the cursor moves only at the end of a tick — so the guarantee is one level down, in the merge: a stored settled row is never written back provisional, whatever line the re-derivation ran at (the ledger-driven cron). A job that runs while a tick is mid-run therefore cannot leave the tick's rows showing as "confirming" below its cursor, in any order the two interleave.

A verdict per job. done when the merge covered the whole range (a whole-window job also stamped the derive cursor); partial when it committed short of that with what it owes recorded (a pending marker, or a withheld stamp; for a tick-range job whose merge the ingested tip clamped below its top, the marker starts at the merged top + 1, the tick's own clamp rule), or when a job derived nothing because it was refused or held (a hand-made sync or enrol before the flip; a whole-window job off its floor; a rederive held for its wallet's replay or for the arm block, or whose to_block stops short of the arm); a failure goes back to the queue with a backoff of one minute per attempt spent, counted from the failure, and the third failed attempt parks it failed with a [fail] ledger-worker: line in the worker's log. A write-lock wait spends no attempt: a writer that only queued for the portfolio write lock past its wait budget (behind a whale's registration replay, say) goes back to the queue with its attempt handed back and waits one backoff step, so contention never parks a job; a queue that stays busy pages through the queued job's age instead. Neither does a merge the stale-merge fence stopped (above): another writer moved the wallet's ledger, nothing of the job's failed, and a wallet whose ledger keeps moving under every attempt pages as an old queued job the same way. One standing failed continuous job per wallet: a continuous job that fails while its wallet already has one is folded into it (its range widened, the new row deleted), so a wallet refused deterministically (a registry gap) is one row naming everything owed rather than a count that grows every cycle. (One exception, and it is harmless: a job the worker finds still running at its own start with its attempts spent is parked without the fold, so it can stand beside the wallet's row until the close below takes both.) And a standing failure closes itself once the ledger has caught up over it: the worker turns it partial (the alarm stops paging; the row keeps what it failed on) when the tick cursor passes its top — the tick derived that range, or recorded it in the wallet's pending marker and pages for it through its own exit code — or when a later job for the wallet finishes done through its top, since every such job starts at the wallet's tick floor. The ingester keeps enqueueing a wallet with a standing failure (it no longer suppresses it), so a fault that has cleared is gone at the wallet's next movement, and a fault that persists keeps one row paging until it is fixed. Every verdict is fenced on the claim that produced it (claimed_by and attempts): a verdict for a job that was requeued and claimed again lands nowhere and settles no sweep.

The worker holds the one global write lock for every merge it runs, and a page load's merge only TRIES that lock. From the first deploy the worker is a frequent holder: every ingester cycle that saw a tracked wallet move is followed by that wallet's merge. A page load (Synchronize, or a mount's refresh) whose own merge meets the lock held skips: the reading it took is not stored (the refresh reports merge-skipped), and the wallet is marked pending from that range's bottom, which the next 6h tick derives. Before this release a skip happened only during a tick's commit or a whale's replay. The holds are sub-second at today's population, so skips should stay rare — but they correlate with the user's own activity (their transaction dirties their wallet, then they press Synchronize), which is why the page load's skip count is on the release watch (deployment). The contention goes away when the worker owns derivation and the page load becomes a sync job that queues for the lock instead of trying it.

One worker, enforced. The worker takes a session advisory lock on a connection of its own before it claims anything and holds it for its life. A second instance does not drain beside it: a bounded hand-run refuses, and a pm2 start waits for the holder to go. The lock is checked before every claim; if its connection fails, the worker stops claiming (and stamping its heartbeat) until it has taken the lock again, rather than draining unguarded or exiting. That is what makes the startup rule true — every job still running when the worker starts was left by a worker that stopped mid-job, and is requeued (or parked, once out of attempts, so a job that kills the worker cannot restart it forever). A job that HANGS inside a live worker is a different case: the in-loop reaper runs only between jobs, and the heartbeat is a timer that keeps beating, so the ingester alarm's derive-lag arm pages on a job running past its kind's bound (restart the worker; the job is requeued at the next start, having spent an attempt): 30 minutes for a tick-range job, and 60 for a whole-window job (enrol, rederive) — a registration replay's worth of work, which the registration drain itself lets run 30 minutes and reclaims only past 60, so a whale's legitimate job neither pages nor has an attempt spent on it. The reaper's own 30-minute requeue only ever meets a job whose verdict could not be written.

The sweep and the tick cursor. A sweep is the set of jobs one tick inserted in one statement. When its last job finishes, the worker advances portfolio:v2:tick by the tick's own rule — stopping at the sweep's settle line — if every job is done, and holds it (and says so) otherwise; the next tick then re-derives from the same floor. A tick that finds a sweep still open enqueues none. Only once the worker owns derivation: until then the cursor is the inline tick's, and a sweep job that exists anyway (an operator's INSERT) derives its range, is recorded, absorbs no marker and moves no cursor.

What is live in this release. The worker drains continuous jobs, and what an operator enqueues (a continuous or rederive job; see deployment; a hand-inserted sync or enrol is refused, above). The 6h tick, the registration replay and the page load keep deriving inline, exactly as before, while WORKER_OWNS_DERIVATION (src/lib/portfolio/derive-jobs.ts) is false — a constant, not an environment switch, pinned by a test at all three sites. Flipped (the next release), they enqueue a sweep, an enrol and a sync instead, and a page load waits for its sync job up to 20 s or what is left of the refresh route's own 30 s bound, whichever is less: past that, the page is served from the last derived state and the refresh reports refreshed: false with the reason derivation-pending. Three consequences of the flipped tick are known and deliberate for that release to settle: a tick that hands its window to the worker notes no scanned scope, so the coverage conjunction reads each stream's own cursor and the discovery certificates hold (a demotion to the full-universe read, slower, never wrong); a sweep holds the cursor on any partial job, where the inline tick marks the wallet and advances, so a chronically partial wallet would hold the population's cursor (a decision for the flip); and a failed sweep job adds a failed row per tick, which neither the one-standing-row rule nor the automatic close yet covers.

Two more lines of the queue's contract. The worker installs the token and fund registries once at start, like the ingester, so a registry edit reaches it on its next restart — which is why the alarm pages on a worker older than the checkout's build. It prunes done and partial jobs after 14 days and never prunes a failed one (a continuous one closes to partial as above, and is pruned 14 days after it closed). Its liveness is the portfolio:worker:heartbeat row, stamped every 30 s whatever it is doing, and the hourly ingester alarm's derive-lag arm pages when the worker runs old code, when the heartbeat goes stale, when a job hangs (by its kind's bound), when the continuous producer has stopped completing runs, when a job waits too long (a sweep is judged against one tick's period), or when one has failed.

A registration waits for its own history

Adding a wallet enqueues two things, and they run on different clocks:

	what it does	how long it takes
the enrolment	the ingester fetches that wallet's bare-token Transfer history into the event store, walking upward from that wallet's own history floor	~1–2 minutes for a wallet added under the rolling 30-day window; ~11 minutes for one with no recorded floor (and ~17 before the rolling window, from the raw 2025-05-21 floor), with three or fewer addresses in catch-up
the replay	derives the wallet's whole history and merges it into the ledger	the minutely drain picks it up ~1 minute after signup

The replay wins that race essentially always, and it used to derive anyway. With the wallet's own Transfer rows not yet fetched, its bare-token universe came out empty and every plain token movement in the range was skipped: not refused and not counted, because an empty scan and a scan with nothing to find are the same result set. The rows it could read were committed, the certification was correctly withheld, and the only pass that came back afterwards was capped at the campaign's arm block — which bounds the historical build's population, not any wallet's history. So for a wallet registered after that block, everything between the arm and its signup was derived for its venue positions and for none of its plain token movements, permanently.

The replay now waits instead. With the wallet's own coverage short of its own history floor it writes nothing, records a marker, and prints one [v2-deferred] line naming the scope it is waiting on. The re-derivation trigger below already runs every minute and already waits on exactly that certificate, so it is what derives the wallet — once, over [that wallet's history floor, ingested tip], which is what closes the gap the arm block left. The page a user sees is unchanged: a wallet with no certification is already shown the building state, so the wait was always a wait. What changed is only what is written during it.

Every one of those questions is asked at the SAME block, and that is a correctness rule rather than tidiness. The certificate is from_block <= <the floor asked about>, so the block the ingester stamped the wallet's coverage row from and the block the gate, the completeness stamp, the re-derivation range and the reconciler ask about must be one number. A floor asked TOO HIGH certifies the wallet over a hole; asked TOO LOW it holds the wallet uncertified over months nothing derives, which is the unclearable refusal. That is why the pair is stored on the account row and read through one module rather than recomputed from created_at at each site.

And the per-wallet floor applies to ONE stream, not to the whole conjunction. wallet-token is the only per-address stream whose addresses are wallets, so it is the only one with a per-wallet bottom; every other stream a wallet requires (aave-pool, morpho, transfers, erc4626, fluid-*, pendle-router, escrow) is protocol-wide, ingested once from the raw 2025-05-21 floor for everybody, and is still asked about the derivation floor. The reason is structural rather than stylistic: the reconciler's second figure reads event_coverage in bulk as a SET of certified scopes, and a protocol-wide scope is one row shared by every wallet, so that set cannot hold "certified for the wallet whose window opens last week, not for the wallet whose window opens on 2026-01-01". The depth test is from_block <= floor, so a higher floor is MORE permissive — asking a rolling wallet's protocol-wide requirements at its own floor would make the completeness predicate laxer than the bulk figure, invert uncertified ⊆ excluded, and page the on-call with a STAMP DEFECT for a wallet that is merely mid-backfill. The rule lives in one function (scopeFloorBlock, read by the completeness predicate, the stamp and the re-derivation trigger) and CERTIFIED_SCOPES_SQL mirrors it in its WHERE clause.

It waits for a second reason too, and that one is not self-clearing. A replay can also stop because it refused its own rebuild: the wallet already had rows naming an asset the replay could not put back, so the run declined to delete anything and parked the wallet as failed (that refusal is the safety rule that stops a rebuild shrinking a wallet's history, and it is working as intended). A parked run laid down no history — and the certification is a claim about the history that is laid down, read from those very rows, so a wallet with almost none of them certifies on almost no evidence. So the replay defers on its own refusal exactly as it defers on a short enrolment: nothing written, a marker, one named line, no certification. The difference is what clears it. An enrolment wait clears itself in about seventeen minutes; a refusal clears only when somebody re-queues the wallet, and the failure recorded on that wallet says whether fetching more history can ever resolve it (for native ETH it cannot: since issue #966 the replay reads the ether leg at every grid point whenever the token registry tracks native ether, so the sentinel is missing from a universe only where the registry no longer tracks it, which is a registry question and not a history one).

The wait is bounded, and past the bound the drain says which side is stuck. A deferral still standing an hour later is not a wait (PORTFOLIO_V2_REPLAY_DEFER_MAX_MS, default 1h; 0 turns the bound off, the way PORTFOLIO_BACKFILL_MAX=0 turns the drain off). The drain names it, and it names which of two mechanisms is blocked, because the two look identical in the marker and have opposite remedies: the wallet's own catch-up has not certified (look at the ingester), or it has certified and nothing has derived the wallet since (its replay is parked and needs a re-queue, or the re-derivation trigger has spent its attempts).

One alarm, and it is the 6h one. The drain reports a deferral and exits zero on it. An outstanding deferral is a standing condition read from a table rather than something that just happened, the cron alert has no throttle, and the drain runs every minute — so folding it into that exit would send roughly 1,440 messages a day for one wallet, about a state nobody can clear by reading it. Worse, the states it fires on most readily are a parked replay the trigger holds on purpose (nothing clears it) and a completely healthy burst of ten signups (four ingester rounds, about 68 minutes). Both are already carried by the reconciliation alarm: a wallet with no certification is excluded from its population with its case named, it pages on the second consecutive tick, and its page now carries the deferral's age and cause on the line. That is where every other "this wallet's history is not being built" finding already lives.

And the 6h portfolio tick does not page for it either. A tick landing inside a wallet's enrolment window derives that wallet over a range it cannot fully read, records it as a partial (so the next tick widens down to it, and so a certificate that later regresses is caught the same way) and prints it. It does not contribute to that job's exit code, even with the product served from the rebuilt ledger: nothing of that wallet is served — the reader fails closed with no certification and shows the building state. That carve-out reaches only a wallet with no certification at all; a wallet already being served whose coverage regresses pages exactly as before, as does a partial with any other cause. Both figures are printed; only the exit code narrows.

The re-derivation trigger

A wallet's history is certified only when the derivation ran over proven coverage. A wallet that registers today gets its replay within a minute while its own bare-token history is still being fetched, so its replay legitimately writes rows and declines to certify them. Something has to come back once that coverage lands, or the wallet stays uncertified forever.

The minutely drain does, after the registration queue has drained: one cheap query finds tracked wallets with no certification that either hold ledger rows or carry a deferred-replay marker, whose requirement set now reaches the floor, and re-derives each over [derivation floor, ingested tip].

The top of that range is the ingestion ceiling, not the campaign's arm block. The arm bounds what the one-shot history build covered; a wallet registered after it was never in that build, so capping the repair there left everything above the arm to the registration replay alone — the one pass that runs before a new wallet's own history has been fetched. That is the gap the enrolment gate describes, seen from the other side. If the ingestion is somehow behind the recorded arm block the tick declines outright rather than lowering the range: a certification stamped below the arm reads as a short build forever and stops matching this trigger, which is a wallet permanently incomplete with nothing coming back for it.

It declines a wallet whose registration replay owns it. A replay that is queued or running would have a second whole-history derivation racing it; a replay that is parked never laid the history down at all, so certifying that wallet would certify over evidence nobody wrote — which is exactly what happened on 2 September to a wallet the alarm then counted as reconciled and certified. Held wallets are named with their queue status on the heartbeat, because a park clears only when somebody re-queues it (Repair runbook), and stay excluded until it does. This trigger is only one of the two ways a parked wallet could have been certified; the other is the parked replay's own second half, which defers for the same reason a moment after the park (see above). Both doors are shut, because either alone leaves the wallet certified over a history nobody built.

Its bounds, because an unbounded version would be a minutely full-history loop on the archive quota:

• Inert unless the campaign's arm block is recorded, saying so on a half-hourly heartbeat rather than every minute — an inert state is level-triggered, and 1,440 lines a day would bury the one report below that matters. There used to be a second gate, on the write switch: disarming the write was the fastest rollback in this programme, and an un-gated trigger would have turned it into a full-history loop at exactly the moment someone reached for it. The switch is gone and so is that gate.
• At most PORTFOLIO_V2_REDERIVE_MAX wallets per invocation (default 2), run serially after the worker pool has drained, with their own budget rather than the queue's.
• PORTFOLIO_V2_REDERIVE_BUDGET_MS (default 3 min) is checked before each wallet is started. It bounds how many derivations start, not how long the invocation runs: one already under way runs to completion. A wallet that is not started matches again next minute.
• Five attempts, with a doubling backoff, counted in the campaign's own cursor family. A derivation that throws is swallowed by the isolation rule above, so without a cap the wallet would be retried every sixty seconds forever. At the cap it is reported and left alone, and the report distinguishes "waiting on coverage" (wait) from "the re-derivation keeps failing" (investigate). The 6h reconciliation alarm carries the same distinction on its page. It did not always: that alarm slept until the product was served from the rebuilt ledger, and in that window a wallet stuck at the cap was visible only in this drain's log. The alarm is unconditional now and carries it on the first tick after it happens.
• It never contributes to the drain's exit code, and it writes nothing to the registration queue, the retired ledger or the position history — so a wallet keeps rendering what it rendered. It does read the queue, for the parked case above; that is one statement per tick, and only on a tick that matched something.
• PORTFOLIO_BACKFILL_MAX=0 disables it along with the rest of the drain.

What the engine reads back out of the ledger

This is the served read path. Everything below describes src/lib/portfolio/v2/, which is what every portfolio request is answered from. There is no other reader.

The derivation's job is to record, per leg, what moved and what was left. The attribution side consumes exactly two of those columns and nothing else:

• to_balance — the leg's quantity after the receipt. Read at end-of-block granularity (within a block, the last receipt wins), it yields the leg's occupancy runs: the maximal stretches over which the leg held something. A run opens at the block the quantity rose from zero and closes at the block it returned to zero.
• terminal — the stored fact that a receipt drove the quantity to zero, which is what makes a close a record rather than an inference.

Three derived rules follow, and each is worth knowing when reading a derivation defect:

1. A run is anchored onto the wallet's snapshot grid at BOTH ends — back to the last reading at or before the acquisition, forward to the first reading at or after the exit — so a position's opening and closing windows fall inside the period the engine walks. A receipt left outside every period is a receipt that never nets against anything, which shows up as a whole opening balance booked as return.
2. Before its first row the leg held nothing (ledger-first R2, R7). A holding that predates the leg's first movement is stated by the opening row the registration replay writes at the wallet's floor reading (or the reading audit at a coverage start), so the ordinary case — a position opened before the wallet's history floor whose first in-window movement is a partial withdrawal or repayment — stays inside its chart because its opening says so, not because the balance before the withdrawal is back-derived from to_balance − qty_delta (retired). The engine and the chart read occupancy the same way, from the rows alone, so they cannot disagree about which windows exist.
3. A leg with no rows at all is not held, and never unanchored. Its readings, if any, are the ghost cross-check's to name (an alarm, not a decline, since W9 ghost-row was retired) and the reading audit's to answer with an opening (R3, R4). The one leg whose occupancy stayed the spine's was the derived-evidence leg (native ether, CN-3), which had no movement records by construction; since issue #966 native ether has receipts and is an ordinary leg here, so no stored leg is derived-evidence any more (the role and CN-3 stay as vocabulary nothing declares). A first row whose balance column is missing because the anchoring read failed is W6 from that row on, and not held before it. A first row that leaves the leg held without creating it, with no opening before it, is W6 in the interval that holds it alone (an unstated opening: its own columns say the leg held something before it, and nothing says what).

A derivation defect that misstates to_balance is therefore not a display bug. It changes how many holdings a position is cut into, which entry date each one carries, and which windows the chart bridges. The free self-audit the contract states over these rows — the recorded balance against the independently read balance at the same block — is the check that catches it, and it is the same comparison the engine uses to decide that a composite position's per-leg split is not a fact about that leg.

Explaining a disagreement: the explanation ladder, accepted causes and alert budgets

Wired by the reading audit. The library below is sub-PR S5 of the ledger-first programme (plan, rulings R6, R8, R10); the reading audit that calls it after every stored reading, books the adjustment rows and prints the pages is sub-PR S4a (the reading audit, below). The activity statement shows each adjustment as a line of its own, "Balance adjustment" (R5c, sub-PR S4b), never as a deposit, a withdrawal or anything the holder did, and never shows an opening.

A reading can disagree with the ledger: the balance read on chain at the reading's block is not the balance the ledger's movements add up to. That disagreement is a break, and before any of it is booked as a correction the ladder asks the chain what happened.

1. Ask the chain (R6). The venue adapter's explain(break, fetchLogs, registries) (src/lib/portfolio/adapters/explain.ts, one venue's evidence per file under adapters/evidence/) asks eth_getLogs for the leg's own movements over (lastAgreedBlock, readingBlock] and answers with candidate evidence: every log found, each labelled with the raw_events stream the ingester files a log of its shape under; what the venue's own decode makes of those logs alone (the rows its derivation adapter emits on the leg, and what it says it cannot book); and the range to re-derive, which is the break's own. Null means the chain has nothing on the leg in that range.
2. Land, re-derive, re-audit (R6, R3; the caller's, S4). A found log the ingester does not hold is landed under the stream it names (a log labelled with no stream is one no stream stores: shown, never written), the range is re-derived, and the leg is audited again against the same reading.
3. Book the residue (R5). Whatever still separates ledger and reading is one adjustment at the reading's block. Evidence never closes a break by itself: an unrelated 5-unit transfer in the range of a 1,000-unit break leaves a 995-unit adjustment, whatever the candidates say.
4. Classify it: accepted if its leg matches a registered cause (below), else unexplained.
5. Page it or not under the budgets (below).

What each venue asks. Each question is one eth_getLogs filter over the break's whole range (the production binding, adapters/log-fetcher.ts, asks the archive endpoint in windows of up to 50,000 blocks and halves a window only when the provider refuses its span, so a 6h reading interval is one request per question and a month about five), and all of a break's questions go out in one round. "The wallet on either side of a Transfer" is two requests, because the node's OR works inside a topic position and never across two, the same fact the ingester's wallet-keyed stream lives with; every other event that names the wallet in the same slot rides one of those two requests.

adapter	what is asked	requests
wallet (bare token)	the token's Transfer pair, plus TransferShares for a share-counted token (stETH, eETH), which is the log the decode books; on WETH, also WETH9's Deposit and Withdrawal (a wrap and an unwrap emit no Transfer), which wallet-token stores since issue #966: landed and decoded as the wrap or unwrap they are	2
wallet (native ether)	no log filter (native ether emits no log): its own explanation (adapters/native-explain.ts, issue #966). The block feed's rows the ledger does not hold yet (a record a WETH9 wrap or unwrap consumed counts as held: the event's row names it, meta.etherLeg); the provider's transfer listing (alchemy_getAssetTransfers, categories external and internal, the wallet as sender and as receiver, paged) over the break's range, whose INTERNAL transfers become NativeEtherInternalTransfer rows (one key per transfer, its transaction and ordinal, whichever party is audited); and, for the part of the range the feed did not read (history before the wallet was followed), one receipt per EXTERNAL transaction the listing names, turned into the rows the feed would have written on the same keys. It states what it could ask (sources): complete, or which source was missing	2 listings, plus a receipt per transaction outside the feed's range (the earliest 500; the rest named as missing), plus a header per block of a listed internal transfer where the ledger stores none
escrow (claim in transit)	nothing: its quantity is re-read at the payout rate and is not a count; its events are class-specific and the escrow stream's	0
aave-family (Aave, SparkLend)	the reserve's aToken (supply) or variable-debt token (debt), resolved from the pool's reserve registry: its Transfer pair with BalanceTransfer, Mint and Burn; the Pool's Supply / Withdraw / Borrow / Repay on the reserve naming the wallet; the LiquidationCall naming the reserve and the wallet; on a debt leg, the DeficitCreated naming both	4, or 5
morpho-blue	the singleton's own events on the market with the wallet as owner, one request per slot the owner is indexed in (no share of a Blue market is a token, so there is no Transfer)	2
fluid	the position NFT's Transfer and NewPositionMinted by its id (every direction in one request); the leg's vault's LogOperate for this NFT (kept by its data, since the event indexes nothing) and its vault-wide LogLiquidate / LogAbsorb; the Fluid wallet factory's Executed(owner, nft), the trace a position operated through its owner's Fluid wallet leaves (a vault migration, a deleverage, a roll)	3
erc4626 (curated funds, Morpho Vaults V1 and V2, fTokens)	the share Transfer pair, with the canonical Deposit and Vaults V2's ForceDeallocate on the inbound request (both index the wallet where Transfer puts the receiver); the canonical Withdraw naming the owner	3
pendle	the PT's Transfer pair (a matured PT is redeemed by sending it to its own YT, so the outbound request finds the redemption); the YT's Burn naming the wallet as its redeemer	2, or 3

What the candidate decode can and cannot say. It sees the logs the questions found, not the whole receipt, reads no anchor (so it states movements, never balances) and resolves no mark. A venue whose decode needs chain state the logs do not carry decodes what it can: a Fluid vault's operates need the vault's constants and the position's state at the block, which only the re-derivation reads, so they come back as found logs and count as undecoded. What the decode says about a leg other than the one asked about is left out, because that leg's own logs were not asked for. The re-derivation over the range, not the candidate, is the record.

Accepted causes (src/lib/portfolio/audit/accepted-causes.ts, R8b). A cause the programme has looked at and decided to book without paging, forever. The registry is code, edited by pull request (its test pins the list), and a residue matches the first entry whose rule matches its leg. The rule reads the leg's key and declaration, never the residue's size or sign.

cause	matches	why it is accepted
native-eth-transfer	the bare native-ether wallet leg (the whole native-ether key) WHERE ITS EXPLANATION COULD NOT ASK EVERY SOURCE for the break's range: the block feed had not read those blocks with the wallet in its set (history before the feed began, a wallet's window before it enrolled, a feed still behind the reading), or the transfer listing was unavailable or refused, or it named what the explanation could not land (an internal transfer without an ordinal to key it by, a block whose header neither the ledger nor the endpoint stated, transactions past the 500 receipts one explanation reads); or where no explanation ran at all	since issue #966 native ether's movements have sources (the block feed, WETH9's events, the listing), so a residue is accepted only where a source could not speak: a known gap in the sources, never a missed movement. A residue left after every source answered is unexplained and pages like any other. Not matched by a vault's own ether side, which uses the same address sentinel and is a receipted position

This is the one cause whose rule reads the explanation (needsExplanation): the audit runs the explanation first and matches the rule on what it could ask, where a cause about the leg alone would let the audit skip a search whose answer the registry already knew. An accepted adjustment is still booked, with its cause in cause, and the activity statement shows it as a balance adjustment whose source is not tracked. An unexplained one pages; for native ether its cause text says the block feed and the listing both answered for the whole range, or names what was missing.

The alert budgets (src/lib/portfolio/audit/budgets.ts, R8). Pure functions over what the audit booked and read:

• (a) a new unexplained adjustment pages, naming the wallet, the venue, the leg, the block range and the size;
• (b) an accepted one never pages, and never counts against its venue;
• (c) a leg that fails to read at two consecutive readings pages as a reader failure. Failing to read is the reader throwing, and also the reader answering "no position" for a leg the ledger holds: the reader cannot tell a genuine zero from a skipped inner revert, and treating that answer as zero would book an adjustment to nothing for a read that failed. The same answer is also exactly how a position reads when it was closed in a transaction the ledger never booked. Telling them apart takes a read of its own, which the audit takes (PR #961 final review, SF-3; the verdict absent): it reads such a leg directly and strictly, and a zero the chain returned is booked as the exit the ledger missed, which pages under (a). What reaches (c) is a leg the direct read found held (the reading missed it) or could not read (a revert, a leg outside the loaded registry, a degraded registry load, a Fluid position). The hourly arm judges the streak from the stored rows, where the direct read's answer is not, so its page still names both causes (a reader that is down, or an exit the ledger missed). A leg that is not a count (a smart pool's member, an escrow claim) is never read directly, and one whose newest reading was worth under a cent, and whose quantity in the ledger is still worth under a cent at that reading's marks, does not page when a reading omits it (PR #963): the Fluid reader drops a member that rounds to nothing at the pool's composition, and since every holding is opened, dust included, such a member is held. The job's log line for the reading names it. A side the reader failed to read drops both members, so its member worth a cent or more still pages;
• (d) a venue with no unexplained adjustment in the trailing 7 days pages on its first new one, named first on the page: a new gap on a venue that was clean. The budget escalates and never silences, so a venue that is already dirty still pages each new adjustment under (a).

A page fires on what is new (the run's own adjustments, or those booked since the alarm last looked), never again because a stored row still exists. An adjustment's identity is its wallet, its leg and the reading that booked it, so a stored copy of this run's own row in the trailing window is the row itself and never a record it broke. A reader failure is a condition, and it pages once per reading that extends a streak of two: every reading carries the instant it was recorded, and the hourly arm passes its cursor (the instant its previous look read the readings), so a look that finds no reading newer than the cursor stays quiet. The 6h checkpoint, run by run-cron.sh, prints one [partial] line last, after one alert-safe detail line per finding, and exits 2 with it: run-cron.sh pages only on a non-zero exit, so the line and its exit code come back together. The continuous path's page is the hourly ingester alarm's arm, one [fail] ledger-audit: <remedy>: … line, also with exit 2. Both open with the run's short counts and then name one finding by wallet, venue, compacted leg key and range before its size, so the page's 220-character cut loses precision and never identity:

[partial] ledger audit: 2 unexplained, 1 newly dirty venue; first=0x… aave aave:reserve:0xa0b8..eb48:supply [25000001,25001800] +1234.56 USDC (~$1235)
[fail] ledger-audit: explain or accept it: 1 unexplained; first=0x… fluid fluid:vault:0x0c8c..6cb3:nft:19309:supply [25999001,26001800] -0.5 WETH (~$1200) env=onchain-credit

To clear an unexplained adjustment: extend the venue's decoder and re-derive the range (a rederive job, or the trigger): the re-derivation queues the audit of the readings it covers, which restates their rows to the residue of the ledger as it now stands, so the adjustment is replaced by the proper record — smaller, or gone (the reading audit). Or, if its source is one no log states, accept its cause by a pull request that adds a registry entry.

The reading audit: every stored reading checks the ledger

What it is (portfolio ledger-first plan, rulings R3, R4, R5, R7, R8; sub-PR S4a). The ledger decides which positions exist and what moved; a READING (a row of the valuation spine, read from chain at a block) audits it. After a reading commits, its writer enqueues an audit job (kind = 'audit', migration 117) for the wallet over the reading's block range, and the ledger worker runs it (src/lib/portfolio/audit/audit-job.ts, the rule in audit/reading-audit.ts):

writer	the reading	the job
the 6h tick	every wallet its dirty pass READ at the anchor (a recomposed wallet's rows restate stored legs and are not a reading)	[anchor, anchor], after the tick's commit
the page-load refresh / Synchronize (live.ts)	the live tip it stored	[block, block], after the write
the registration replay	its grid, floor to newest	one job over the whole range, after the ledger merge and the floor openings

An identical job still open is not enqueued twice. The job runs only once the wallet's ledger is derived through the reading's block — settled and provisional alike: the wallet's derive cursor (the whole-history stamp, without which nothing is derived for the audit to compare), then the highest of that stamp, the 6h tick cursor (below the wallet's pending marker, when one exists), the top of the wallet's newest done derivation job and its live tip's block, a page load's own records (its live tip, and once the worker owns derivation its done sync job) also counted only below the marker: a page load merges from above the tick cursor and never widens down to a marker, so its reading proves nothing about a stretch the tick marked owed (S4b, PR #959 review round 4, B3). A read of the marker that fails leaves every term it caps out (the tick cursor, the live tip, a done sync job, and the stamp's producer term), rather than counting them as if no marker stood (round 5, SF-1). An audit of a page-load reading taken while the wallet owes such a stretch therefore waits for the derivation that starts at or below the marker (the wallet's continuous job it widens, or the next tick). Short of the reading's block it goes back to the queue without spending an attempt, having queued the continuous job that brings the ledger there (widening the wallet's queued one, once ingestion has reached the reading); six hours after it was queued it gives up (partial, "not audited"), which loses timing and never a correction, because the next reading's audit compares the same balances. And a wallet the native-ether block feed covers waits for the feed too (issue #966): where the wallet's native-eth coverage starts at or below the reading's block and the stream's cursor has not reached it, the ether leg's receipts for the last blocks are still to come, and an audit then would book them as a correction the feed then books again as movements. The job goes back to the queue the same way (no attempt spent) and gives up at the same horizon. A wallet the feed does not cover has nothing to wait for: its ether residue is the accepted cause's (the ladder).

In block order, from a watermark (PR #958 review round 2). A wallet's audits apply in block order: a job waits, its attempt handed back, while the audit of an earlier reading of the same wallet is still open, and gives up at the same six-hour horizon. Out of order, a tick's audit deferred behind its derivation and a live tip's audited meanwhile would each book the one difference — the tip's against a ledger without the checkpoint's correction — two corrections and two pages. Each wallet keeps an audited-through watermark: the block of the newest reading an audit compared, in chain_scan_cursors under portfolio:derive:v2:<chain_id>:<wallet>:audited-through (the per-wallet wipe family, beside the derive cursor), advanced, never lowered, once a job has compared its readings. It is the last block the chain and the ledger were compared at, so it — never merely the previous STORED reading — is where a break's search starts and where the orphans a job inherits start (below): a stored reading may be one no audit compared (a recomposed checkpoint carries its quantities forward from an earlier reading; a reading whose audit gave up), and a search from it skips the stretch the chain was last compared over. A reading can still be STORED below one already audited — the tick fixes its anchor when it starts and commits minutes later, and a page load's tip above that anchor may be stored and audited meanwhile — so every job at or below the watermark compares the readings above its range again, up to the watermark: a reading's correction is measured against the ledger below it. The difference is then booked once, at the first reading that shows it, and the tip's row goes; its page may repeat, never be lost. Every such job does it, whatever the run itself changed (PR #958 review round 3): the lower correction and the restatements above it are separate transactions, and a run stopped between them (a fence, a lock wait, a throw, a worker restart) left its retry a lower correction already in place, which changed nothing, so asked only when a run changed a row the retry skipped the pass and ended with the difference booked twice. The obligation is read from the wallet's watermark on every run; the pass writes nothing where a reading still agrees. The pass ends with the audit rows above the last reading it compared, up to its top (PR #958 review round 4): a row there has no reading standing above it to take it over. That is the tip's correction when an EMPTY reading retired the tip (a wallet that sold everything writes no row, so nothing replaces the tip and no audit is queued), and it stood beside the checkpoint's below it for as long as the wallet held nothing. Each such row is compared again at its own block from the reading it carries (below), on the ledger the pass left: removed where that ledger now holds what it stated, restated where the residue changed. A pass that never completes (its job gave up at the six-hour horizon, or was parked failed) leaves the tip's row to the next audit whose range reaches it: the audit of a reading stored above it, once the wallet is read again, or a re-audit below it (a re-derivation's). A wallet that holds nothing stores no reading, so for it the two statements of the one difference stand until it holds something again or is re-derived: the row does not page again, and its interval stays W11.

The comparison, per leg, exact and literal. The ledger's quantity at the reading's block is the leg's opening (or its newest adjustment, or a decoder's quantity statement) plus every movement's qty_delta after it, in integer arithmetic, and a leg with no row stating its quantity starts from zero: before its first row the ledger holds nothing, which is also what the engine books it at. Never to_balance − qty_delta of the first movement read back as the leg's opening: that is the back-derivation R7 retires ("pre-first-movement balances come from opening rows, not arithmetic"), and in exact integers it hides the same thing. A leg held from before its first movement with no opening stating it (a wallet enrolled before the opening backfill, a floor reading that left a held leg out, a re-derivation that added history for a leg no reading read) is a ledger MISSING ITS OPENING, and the audit must disagree with the reading rather than agree with it: its first reading after that movement is a break (a −400 withdrawal to 500 from a 900 holding reads 0 − 400 against 500), the whole missing opening is booked as the correction (+900) and paged, and the interval that holds the movement is W11's. A movement whose to_balance is null (its anchor read failed) still counts by its own qty_delta. Not the newest to_balance either: a running balance a derivation wrote before a correction existed carries the old quantity forward, and would book the correction twice. A leg whose stored quantity may move by an exact declared slack (a Fluid vault leg's one raw unit) reads its newest stated balance within it; a leg whose quantity is not a count at all (a smart pool member, an escrow claim) is never compared, because its drift is return. The verdicts:

• agree — nothing to do;
• coverage start — a leg the ledger has no history for at or before the block: an opening at the reading (the floor openings the registration replay writes are this rule at the wallet's first reading), whatever the holding is worth (the comparator run on staging, PR #962, finding F1). R7's dust rule governs what the surfaces show and what the audit books as a correction, never what anchors the ledger: a sub-cent holding left unopened is still on chain, so the first real movement on the leg states a to_balance that includes it and the literal ledger, counting from zero, misses it by exactly the dust (on staging: 100 raw USDT at 0x53e2's floor, then an address-poisoning transfer of 100 raw stating 200, booked as an unexplained +0.0001 USDT, paged, and its stretch "not measured"). Opened, the ledger holds what the chain holds and agrees at every reading after it; a dust leg that later grows with no movement the ledger records is then a break like any other (explained first, R6), not a second coverage start. The surfaces keep the row off the page while it is dust (below). Except native ether above the wallet's first reading (issue #966, absentMeansZero): from there on the ether leg's movements have sources, so ether a reading finds where the ledger has no row is an arrival the ledger missed (most often a contract's payment inside a transaction), a break against zero, never an opening that would absorb it unexplained. It is searched from the wallet's FIRST reading, the registration replay's strict read, and never from the reading before: the six-hourly read skips a failed eth_getBalance (M9, fail-open), so a reading without the leg may have held ether it failed to read, and an arrival before it would fall outside the search, every source would answer, and the residue would page for an outage alone (PR #967 review round 2, B2). It costs one listing pair over the longer range, for a leg with no row at all;
• dust (R7) — under one cent at the reading's own price where the ledger holds the leg at ZERO (its history emptied it): no correction, no alarm (and a reading of nothing where the ledger has no history opens nothing);
• break — explained first (the ladder: a native-ether break by its own sources, the block feed and the transfer listing, and every other leg by its venue's logs; no cause skips the explanation since issue #966; found logs and rows are landed under their streams and the break's range is re-derived through the worker's own job path, then the leg is audited again; a re-derivation that only waited, for the write lock past its budget or behind the stale-merge fence, sends the audit back to the queue with its attempt handed back, as it does any job), and the residue booked as one adjustment at the reading's block: qty_delta = read − ledger, to_balance = the read quantity, explain_status from the accepted-cause registry, cause the registry id or what the explanation found, reading_block, meta.audit the ledger's quantity and the search range (from the last block the chain was compared at for the leg: the watermark's reading, a reading of the same job before it, or an audit row at or below the watermark);
• absent — the ledger holds the leg and the reading has no row for it. A missing row is never a zero (M9): a reader answers "no leg" for a zero balance and for an inner read that failed alike, and also for a leg outside its loaded universe and, on Aave and SparkLend, for every leg of a debt-bearing account whose classification read was incomplete (R4, D4). So the audit READS THAT ONE LEG DIRECTLY at the reading's block (PR #961 final review, SF-3), and STRICTLY (round 2, B1; adapters/strict-read.ts): one eth_call for the count the leg's key names, never the venue reader, and only a count the chain returned can be a zero. Per venue: scaledBalanceOf(wallet) on the Aave or SparkLend reserve's aToken (supply) or variable debt token (debt); position(id, wallet) on the Morpho singleton (supply shares, collateral or borrow shares by the key's side); balanceOf(wallet) on an ERC-4626 vault or a PT; the wallet row's own call for a token (balanceOf, or a share-accounted token's share balance) and eth_getBalance for native ether. Each is the quantity the venue's reader stores, and each leg must be in the registry the worker loaded (reloaded every minute, AUDIT_REGISTRY_TTL_MS). The read is failed, never a zero, for a revert, empty or short returndata (a codeless target), a leg outside the loaded registry, a degraded registry load (assertRegistriesComplete), a registry load that throws, a Fluid position (no strict read yet) and a chain other than mainnet. It goes to the endpoint the venue readers read (ETHEREUM_RPC_URL), which the backfills' historical reads already require to serve past blocks; a node that cannot serve the block answers with an error, which is failed. Only a COUNT is read this way: a smart pool member or an escrow claim can read zero by drift with nobody touching it, and an adjustment to zero there would book the drift as a flow, so such a leg keeps the carry. (A leg whose literal quantity is below zero is not "held": it is a ledger missing its opening that no reading reads, and the engine names it, W6 at the unstated opening, below.) The answer decides:
  • zero (a count of zero the chain returned) is the exit the ledger missed, a position closed in a transaction no decoder booked. It is a break of minus the ledger's quantity, explained like any other (an exit event the ladder finds becomes its proper record, re-derived), and its residue is booked as an adjustment TO ZERO (to_balance 0, meta.audit.directRead = zero), unexplained unless its leg's cause is accepted, so it pages. The reading holds no row to value it from, so it is valued and sized from the newest stored reading at or below the block that read the leg, whose copy it carries (meta.audit.reading, naming that reading's own block_number; the read path and the statement value it from that reading). From its block the ledger holds none of the leg, so R2 serves it nowhere: before this, R2 carried it at its last reading for good, on the positions table and the All view's value, while the headline, built from the readings, said zero. A leg no reading ever read is booked unvalued. A re-audit of the same reading reads nothing again: the exit its block already holds is the chain's answer there, and is re-planned from the row;
  • held (a non-zero read) is a reader failure: no row, the leg stays served at the ledger's quantity, and two such readings in a row page (R8c);
  • unreadable (the strict read failed or threw, or nothing is wired to read it): today's carry, as held;
• unanchored — the ledger's quantity cannot be stated: a count leg's quantity statement that states none, or a slack or non-count leg whose newest to_balance is null (W6). Reported, never corrected; the engine's own W6 withholds the leg's whole replay and raises its alarm.

A reading's rows are restated, never stacked. The audit of a reading compares it with the ledger WITHOUT the audit's own rows at that reading's block, plans the rows the reading calls for, and makes the stored ones exactly those: inserted where missing, rewritten where the residue changed, removed where nothing is owed any more. A re-run of the same audit therefore writes nothing, and one run after the ledger changed under it (a decoder extended, the range re-derived) books the new residue rather than a second correction on top of the first (R6's replacement).

The rows. An audit row sits at the reading's block with log index 2,147,483,647 (after every log of that block: a reading is the state at the end of its block), under a transaction hash that is a SHA-256 of its kind, chain, wallet, venue, leg and reading block, so a re-run inserts nothing (ON CONFLICT DO NOTHING), under the portfolio write lock. Its marks are its reading's, pro rata to the quantity, and the read path computes them the same way from the reading it names. The merge never deletes an audit row (a re-derivation over its block keeps it), neither of the wallet-token universe's superset guards counts one (an opening names a token whether or not anything moved it in the range, so counting it refused the range's whole delete on every range holding a quiet leg's opening), and the derivation resumes a leg's running balance from it (closeLegs), so a movement derived after a correction is anchored on the corrected balance. A correction below rows the ledger already holds above it queues a continuous job over (its block, derived-through] that re-derives them on it, in the same transaction as the correction: queued after it, a job stopped before its end found the correction in place on its retry, owed nothing, and the rows above kept the balance chain from before it. Every row the audit writes also carries the reading row it was written from (meta.audit.reading: the columns the read path values a reading from), so it can be valued from that reading after the reading itself is gone (below).

The write is fenced. The audit compares a reading with ledger rows it read outside the lock, and while the inline writers still own derivation a tick, a page load, the registration replay or the re-derivation trigger can merge a movement at or below the reading between that read and the write (late ingestion, a gap patch, a whole-window rewrite). Written anyway, the correction would stand in for a movement the ledger now records. So the audit takes the ledger worker's stale-merge fence (readLedgerFingerprint: the wallet's row count and the exact sum of their updated_at at or below the range's top) before it reads the ledger and again under the write lock; a different one writes nothing, and the job goes back to the queue with its attempt handed back, to compare again on the ledger the other writer left. A write that only queued for the portfolio write lock past its wait budget goes back the same way, having written nothing: contention parks no job, the audit's own write included. And no wait of an audit outlives its horizon: six hours after the job was queued, a fence, a lock wait or an in-place re-derivation that only waited ends it partial ("not audited"), as the wait for its derivation and the wait behind an earlier reading's audit do, so the audits of the wallet's later readings stop waiting behind it; the next reading's audit compares the same balances.

An audit row moves with its reading where it can. A live tip's reading is deleted once a newer tip or the next checkpoint supersedes it, and a replay that re-lays the grid can move a reading. A row left behind can no longer be restated by its reading's audit, and its W11 would widen to the grid interval around it. So its correction MOVES to the next reading that stands above it, the reading that superseded it, where that reading can restate it: it read the row's leg, and no movement of the leg lies between the two blocks. A job takes over every such ORPHAN from the newest reading standing at or below the watermark up to its own first reading; the orphans that move leave the ledger every reading is compared with, are removed with the restatement of the reading that takes them, and what they stated is booked again there. Where that reading is one no audit compared (a recomposed checkpoint retires the tips below it like any checkpoint, while the tick audits only what it read), it is compared for the moved orphans' legs alone, against the ledger without them: nothing new is searched for there (it read nothing, so evidence found for it could contradict the chain), the moved residue keeps its explanation, and the reading never becomes the watermark or a break's floor. Moved further, to the next chain read, the interval up to that checkpoint would book the difference as earned: the checkpoint's quantity already carries it. A correction moved with its residue unchanged that the hourly arm has already paged is marked (meta.audit.rehomedFrom) and pages no more; one the arm never saw (a tip superseded within the hour) pages at its new home, so a move can repeat a page but never lose one.

An orphan the next reading cannot restate stays where it stands (PR #958 review round 3). A reading with no row for the leg books nothing for it (a missing row is never a zero, M9, and a leg the ledger alone holds at or below zero gets no verdict), so a move there lost the correction: a tip's +50 on a leg sold in full before the next reading left the ledger at 100 − 150 = −50 for a leg nothing reads again, and the re-derivation after it met the negative-balance terminal. And a correction moved above a movement it predates makes the recurrence between the two blocks state a balance nobody held (100 − 120 = −20 for a leg reduced after it). The kept row stays in the ledger every reading is compared with, and is valued from the reading it carries (meta.audit.reading): the read path values an audit row whose reading is not among the wallet's from that copy, exactly as it would have from the reading, so it is computed, never served its stored columns. It is also compared again at its own block whenever a job inherits it, for its leg alone, from the quantity that reading read: unchanged where the ledger below it still owes the residue it states, restated where that residue has changed (news, which pages), and removed where the ledger now holds what it stated (a decoder extended and the range re-derived, R6's replacement, or a lower correction booked since), with the rows above it re-derived. A row that carries no reading (written before rows carried theirs) is kept as it stands. The same holds for a row whose reading still stands at its block but no longer reads its leg (PR #958 review round 4): a live tip the 6h tick retired at its own anchor block and replaced there with a checkpoint, or a grid point a replay re-laid, whose read of that leg failed. The chain at a block does not change, so the leg missing there is a failed read (M9), never the leg gone; restated by that reading, the row would be removed, since nothing is planned for a leg a reading does not hold. So the reading restates only the legs it read, and the row is compared again from the reading it carries.

A re-derivation re-audits. The re-derivation trigger and a rederive job queue an audit job over the stored readings their range covers once they have written, so R6's replacement happens without an operator (a correction a new record now explains is restated, smaller or gone; a coverage-start opening a movement derived below it now duplicates is removed) and an audit that gave up at its six-hour horizon while the stamp was owed runs again.

What the engine does with them. Both kinds are in both flow sets, signed by the leg and the direction of their own quantity, so neither is ever earned. An opening is the opening anchor. An adjustment ends its stretch "not measured" on both lines: W11 corrected-reading, scope leg × the interval that ends at the correction's reading, magnitude the correction's equity-signed value, counted and alarm-free in the engine (the audit's own budgets page). A second opening on a leg the ledger already holds is a restatement of the holding, never an arrival (netted as a flow it would book the whole position as a loss): it is a W11 too, its magnitude the change it states. And a leg whose history OPENS from a balance no row states — its first block leaves it held without creating it, and no opening precedes it — is not booked from zero there (that would earn the whole earlier holding) and not back-derived either: the interval that holds that block is W6 unanchored-leg at leg-interval scope (not measured, both lines, a barrier, the unanchored alarm, W6's production budget of zero), unless the audit's correction at its close already makes it W11. The activity statement shows an adjustment as a line of its own and never an opening (below); the events list leaves both out.

Pages (R8). The job evaluates the budgets over what it booked and read, prints the detail lines and the page line to the worker's log, and ends partial with the page line as its reason when something pages. The pager is the ingester alarm's fourth arm, ledger-audit (deployment): hourly, it reads the adjustments booked since its last look, the trailing week's unexplained ones, and the reading histories of the wallets read since then (ordered by recording time, its cursor passed), prints exactly one line through the budgets' own verdict and only then stores its cursor (portfolio:worker:audit-arm, an instant in milliseconds). Its window stops five minutes short of the look (AUDIT_ARM_LAG_MS, PR #961 final review, SF-1), and the cursor it stores is that top. The audit's write waits for the portfolio write lock (up to ten minutes behind a tick's commit or an enrolment replay) inside its transaction, and a row is visible only once that transaction commits. Stamped with the transaction's start (the column's default), a row written after the wait carried a time from before it, below the cursor of a look that ran meanwhile, and no look paged it. So the audit stamps its rows when it writes them, after the lock (clock_timestamp() in its insert and its restatement), and a look reads only up to five minutes before it began, longer than the rest of that write can take before it commits: every row is committed before a window reaches its stamp. A row written in the last five minutes before a look is the next look's. The readings the arm picks its wallets by are stamped the same way (PR #961 final review round 2, SF-1): the 6h tick's and the page load's snapshot insert (insertSnapshotRows, its conflict update included) and the backfill's (insertBackfillSnapshotRows) set updated_at = clock_timestamp() after their own lock wait, so a checkpoint that queued behind a replay across a look is judged by the next one, and a reader-failure streak ending there still pages. A look judges each reading once: its streak read takes the newest two readings recorded by its window's top, never one recorded after it. Both state a correction's size in its asset's own units and in dollars where its book is the dollar (+50 USDC (~$50)): its count scaled by the ratio of its reading's accounting-unit amount to its stored count, the row's own value, and the asset's ticker; a stored count is often no token amount (a scaled aToken balance, a share count), and the raw one is the last resort. The reading is the one standing at the row's block with the row's leg, else the copy of it the row carries (PR #958 review round 4): a kept row's reading is gone, or no longer reads its leg, and joined to nothing the arm paged it by its raw count, with no dollars, while the job that restated it sized it from that copy.

What the surfaces do with them

Sub-PR S4b (rulings R2, R5b, R5c, R11). Each surface reads the audit's rows the way the rulings say, and none of them treats either kind as something the holder did.

• The activity statement (ledger-v2-activity.ts, v2/activity.ts). An adjustment is a line of its own: "Balance adjustment +X ASSET, cause under review" while its explain_status is unexplained, "…, source not tracked" once it is accepted, signed by the direction its quantity moved (qty_delta), sized in the asset's own units from its reading (a bare PT leg in PT, every other leg in its accounting asset through the reading's own quantity and value per stored count) and valued at the reading's marks. Where that reading is gone for good (a superseded live tip, a re-laid grid point) and the audit kept the correction where it stands, the copy of the reading the row carries (meta.audit.reading) sizes and values it instead, exactly as the loader values it; a reading on file always wins over the copy. It is never a deposit, a withdrawal or a user action: its action class is balance_adjustment (a filter option of its own), it raises no deposit or withdrawal flag on the chart, and it is outside every capital figure at every scope (M31). It links to no transaction, since none moved it. An accepted one is folded by default (R5c as amended 2026-09-26): gas moved native ether without a log at almost every checkpoint of an active wallet until issue #966 gave native ether receipts (an accepted one is now a stretch whose ether sources were unavailable, which is rare), and the statement holds every entry whose rows are all accepted adjustments out of its own lines (foldedAdjustmentSql, emitted from classify.tsadjustmentFoldedByDefault and tested against it), counts them through the page's own filters (foldedAdjustments, first page only, absent when zero), and serves them as their own lines again under adjustments=1, which the page's one summary line ("N balance adjustments, sources not tracked") asks for. An unexplained one is never folded. Folding is presentation only: nothing it hides is revalued or recounted, and a folded entry offers no filter pill, as a held-back fee offers none. An opening is never loaded into the statement at all (kind <> 'opening'), so it cannot appear on any page or in any count. The events list still leaves both kinds out.
• The entered basis (v2/entry-basis.ts, M32). An adjustment is a receipt like any other, signed by the leg and its direction and valued at its reading's two marks, so on a leg with a market-vs-redemption gap it blends in exactly as a top-up of the same quantity at that reading would, and on a par leg it moves nothing. The span's opening basis is its first opening row's two marks where the span opens with one, and the first stored reading only where no opening starts it yet. The opening's marks are read as its reading is SERVED (the comparator run on staging, PR #962, blocker B1): the financed-group rule (§C5, withFinancedGroupLineWithhold) blanks a group's line on the readings and not on the receipts, so an opening at a reading where a member of the group had no mark still carried both of its lit sibling's marks, and a basis was stated from the lit side alone where the reading states a dash (two Morpho debt legs on staging). ledger-v2-api.ts openingsAsServed blanks the opening's line wherever the rule blanked its reading's, for the entry basis only, so the span has no anchor there exactly as on the reading (M9's dash, nothing reconstructed). Whether a leg held anything before its first receipt is read off a record (an opening, or a reading below that receipt's block), never off to_balance − qty_delta: a first block that moves a holding no record states carries no basis (unstated-holding), so an exit from it can never net into the next holding's entry.
• The PT lot book (v2/pt-accrual.ts, M34). A correction on a bare PT leg, or on PT collateral a venue holds unscaled, sets the book to the quantity its reading read (to_balance): more PT opens a lot sourced adjusted, struck at the reading's own implied rate (the stored reading at the correction's block, else the copy the kept correction carries, meta.audit.reading, which the loader projects as the receipt's auditReading: PR #959 review round 2, SF-A); fewer reduces every lot pro rata. On index-scaled collateral (Aave, SparkLend) the correction states no PT count this book can read, so the book cannot be set to the holding and still holds the pre-correction quantity: the correction's flow AND the leg's level are withheld from that reading on (pt-quantity-unresolved, PR #959 review SF-5), so the accrual line never values the pre-correction holding, until the ledger states the leg empty (an exit in full, to_balance 0), where the book is emptied and a later re-entry is measured from its own fill. A correction that finds the holding empty states its count on every venue (zero) and is booked as a disposal.
• The positions, the as-of day and the All view (ledger-v2-api.ts). The row set is the ledger's occupancy (R2) everywhere: today, the positions a past day's newest reading dates (each at the newer of its last reading and its last movement at or before that reading's block, so a position moved after the day's last reading shows the reading's quantity, and its entered basis stops at that block), and each point of the All view's value history (servedRowsAtPoints), whose sum is over values computed at read. Less what R7 keeps off the page (the comparator run on staging, PR #962, F1; shownLegRows, hiddenAsDust): a row worth less than one cent (the audit's own floor per book and value rule) that has booked less than a cent on both lines through the moment is not listed: not in the positions table, the uncharted band, the divergence list, the "Not covered" count, nor the All view's total and the value line under it, which apply the rule at each point on what the leg had booked by then. The ledger still holds it (its opening), so the audit and the positions agree on the leg, and a book's own value line and headline keep what it is worth: by the rule's own condition less than a cent on each line per hidden row. A holding withdrawn to a residual keeps its row while its booked return is a cent or more, so the "Yield earned" column still foots to the headline; an unpriced row is never dust.
• The "Updated" stamp (wallet-stamp.ts, R11). The summary's readAt / readBlock is the wallet's derived-through block and the time of the newest block at or below it that a record states (the chain's header store, else the wallet's own reading at or below it), never a reading's own time and never the moment somebody asked. The block is the audit's own terms (readWalletDerivedThrough; a page load's own records, its live tip and a done sync job, count only below the wallet's pending marker, as the tick's cursor does, PR #959 review round 4, B3, and none of the terms the marker caps counts when the marker cannot be read, round 5, SF-1) plus the continuous producer's cursor for a wallet the producer follows (readDerivedThroughTerms, PR #959 review SF-2; its follow record): a quiet wallet's stamp therefore moves every ingester cycle, and one the scan found moving waits for its job. The audit does not take the producer's term: it compares a chain reading and must see every movement below it, including the ones the producer's candidate read cannot tie to the wallet, which only the derivation of the wallet's own range it asks for brings in. A wallet whose history is still being built states none, and an account containing a wallet whose rows are served with no stamp states none either (accountStamp, PR #959 review SF-1). Synchronize is "verify now": a fresh reading, the wallet's movements derived up to its block first (by the refresh itself in this release; once the worker owns derivation, a top-priority sync job awaited up to 20 s, derivation-pending when it has not finished) and that reading's audit job; the stamp it returns is the one after the call.
• The openings of wallets enrolled before this release are written by the opening backfill, below.

The opening backfill: a floor opening for every wallet enrolled before ledger-first

What it is for. The enrolment replay writes one opening per leg its floor reading holds (writeFloorOpenings, the last step of a wallet's build), so every wallet enrolled from this release on starts its ledger from a stated holding. A wallet enrolled BEFORE it has none, and under ledger-first that has two effects until one exists: a holding that never moved inside the history window has no ledger row and leaves the served row set (R2), and a holding whose first in-window record moves out of a balance no row states (a partial withdrawal from a position opened before the window) is not measured over the interval that holds that movement (W6 at the unstated opening). scripts/ops/backfill-opening-rows.ts writes the missing rows.

What it writes, per enrolled wallet (every wallet with a stored reading, or --wallets a,b): one opening per leg the wallet's floor reading holds, where the ledger has no history for that leg at or below the reading's block. The floor reading is the wallet's oldest stored reading at or after its history floor (the replay's own first grid point, chosen by the reading's label since the replay grids from the floor's instant). The rows are exactly the replay's: planned by the audit's own core (auditReading: a coverage start per such leg, a holding worth under a cent included since PR #963, the comparator run's F1 on PR #962), with the same deterministic key (auditTxHash), the same columns and the same marks (the reading's own, whole), each carrying the reading it was planned from (meta.audit.reading) as the replay's do. A sub-cent opening is written like any other and counted apart in the log (dust): the surfaces keep its row off the page while it is dust. On a wallet an earlier run of this backfill left its floor's dust unopened, a re-run writes those openings, and the re-audit it queues removes a correction an earlier audit booked for that dust (0x53e2's +0.0001 USDT on staging), which pages nothing again. A wallet with no derive cursor is skipped (its build is still running or was deferred, and it gets its openings from its own replay or from the audit of its readings), and so is one with no stored reading at or after its floor.

• Dry run by default. Every row it would write is logged and nothing is written; --apply writes.
• Idempotent and insert-only. The key is a function of the kind, chain, wallet, venue, leg and reading block, and the insert does nothing on a key that exists, so a second --apply writes nothing. An opening already stored is never rewritten, whatever it states (it is logged as present), and nothing is deleted.
• Fenced. A wallet's read, plan and write run in one transaction under the portfolio write lock every ledger writer takes, so no merge moves the ledger between the check and the insert.
• Logged. One line per row (wrote, would write, or present, left as is:), with the wallet, venue, leg, block, the quantity in its stored count and in the asset's own units, its value and its key; one line per skipped wallet; and one summary line to paste into the release log: opening-backfill: APPLIED; wallets N (skipped S); openings written W (of which dust, hidden on the surfaces, D), already present P; re-audits queued R. D counts the written openings of a holding worth under a cent (before PR #963 those were not written, and the line said dust (no row) D).

Order, and the re-audit that makes it not matter. The audit compares the ledger from zero at a leg's first row, so an audit of one of these wallets' readings that runs BEFORE the backfill books each missing opening as an unexplained correction at the leg's next reading (paged), and opens each unmoved leg at that later reading, which the floor opening then duplicates (the engine withholds the second as a W11 restatement). So the backfill runs before the worker's first audit where it can (on prod, at release, before the worker starts), and by default it also queues ONE audit job per wallet it wrote to, over the wallet's readings from the floor on: each reading's audit restates its own rows against the ledger that now opens at the floor, so a correction or a later opening an earlier audit left is removed, and nothing is booked where the ledger now agrees. Those jobs run when the ledger worker runs them. --no-reaudit writes the openings and queues nothing, and is only safe where no audit of those wallets has run yet. The procedure is the release's (release steps).

The two money columns are computed at read

A reading (portfolio_position_snapshots) and a movement (portfolio_flow_events_v2) each store two money columns, value_market and value_redemption, and every writer still fills them. A served path reads them only for a row it cannot compute (the loader rule, below). Every surface that states a value (the summary, the positions and the All view, the history at every range, the as-of day, the coverage notes, the activity statement, the events list, the PT exit cost, and both sides of the acceptance reconciler) takes the two values from src/lib/portfolio/read-values.ts, which works them out on every read from the row's stored facts, the RATES its writer read (the observed rate facts), and the price series at the row's own moment:

• A reading is its stored quantity (qty_underlying, the amount its writer formed at the block) priced by the writer's own branch for that leg (buildSnapshotRow, over the maps loadMarketContext builds for a stored row), at the reading's block time (block_ts), or at its window label where the row states no block time (a registration replay row). It is valued in the book the row was written in (its book column, which every consumer sums it under), whatever the registry says today, so re-booking an asset never re-denominates history.
• A movement is its stored amount and consideration run through the writer's own resolver (derive/marks.ts) at its block. It states no book of its own, so where the registry now prices it in another book than its leg's closing reading states, it is withheld rather than netted in the wrong unit. The events list and the activity statement apply the same rule over their page (pageLegBooks), and also withhold a movement whose value would sit beside a label of another unit.
• A PT movement's mark (meta.ptMarkValue: the movement valued at the PT's own rate at its block) is a money value too, so it is computed by the same resolver run, on the same bars as the movement's value. The PT lot book strikes a fill's payout coins as the value divided by the payout coin's price recovered from that mark, which is the exact coin count only when both were priced on one bar: a mark frozen on the bar its writer saw, divided into a value computed on the covering bar that landed later, would move the lot's locked yield (about 4bp of annual rate per 1bp of bar gap on a 100-day lot).
• A zero quantity (the ghost adjudicator's restatement) is zero on both lines, priced or not.
• A pre-window PT fill (portfolio_pt_prewindow_fills) is valued like an in-window movement, from its stored consideration, its stamps and its rate facts, in a valuation of its own, so the PT lot book never sets a frozen pre-window fill beside computed in-window ones.

The series are read in two statements per wallet in the steady state and answered in memory, and what they read grows with the moments the rows are valued at, not with the age of the wallet: each token's own bar at each hour it is priced at (WETH's included, and WETH's whole 48h walk-back at an hour a movement is valued at), with rejected bars skipped; and the share-rate observations inside the window and the newest one before it. Only where an hour's own bar cannot stand for its walk-back does a third or fourth statement read the walk-back itself: a token's, where it has no bar in that hour or none from the same price source as WETH's; WETH's, where it has no bar of its own there or a token beside it has an older bar to pair against. An hour whose own bar stands for its walk-back is priced exactly as from the whole walk-back: it is the newest bar there, and its pair with WETH is that same hour. A reading that asks while a read that will answer it is in flight waits for it, and an hour nobody foresaw is read on its own and counted (late-tokens on the reconciler's line). A page load makes no chain read. An answer is held for a minute against a digest of the facts it was computed from and of the registry it was computed under (the registry itself is held for a minute), so a new or corrected bar, or a registry edit, reaches the page within a minute.

What follows from it:

• A late or corrected price re-values history by itself. The 6h tick prices a reading with the bars the last sync had written, which are routinely hours old; read later, the same reading is valued at the bar covering its own moment. A movement derived before its covering bar landed moves the same way, and so does every row priced off a bar the spike gate rejects afterwards.
• Repairs that rewrite the stored columns no longer move a served number. The guarded re-mark, remark-fluid-dust-debt.ts and the value half of the ghost adjudicator's restatement rewrite columns that only the revert path (values: "stored" on every loader) and the parity census read. A wrong price is corrected in the series. A restated QUANTITY still moves served numbers, because the value is computed from it.
• Where the writer read the chain, the read takes the rate it read. A wrapper's redemption rate, a fund's NAV and a share-counted leg's per-share rate are read on chain at the row's block by the writer and kept as the row's fact, so the read reproduces them exactly instead of taking a series observation up to six hours away (or none: funds and share-counted legs have no series). A PT movement takes the rate the ledger stamped at its block. A live tip whose asset had no bar inside the walk-back stores no market value (decision c1: it used to store the vendor's live level, which no series holds), and the tip SERVED at page load keeps its live fetch.

The observed rate facts

A writer reads some rates on chain at the row's block, and the six-hourly series is only its fallback. So from migration 116 each writer keeps the rate it read as a fact of the row (the columns): the 6h checkpoint and the live tip (insertSnapshotRows), the enrolment replay (its own insert, through the same helper in snapshot-write.ts), the movement valuation (derive/marks.ts, into meta.rateFacts) and the pre-window PT fills (beside their consideration). A reading keeps the rate of its own accounting asset (a wrapper's share rate, a fund's NAV, the per-share rate of a stETH/eETH share count; a PT leg keeps the PT's own rate for reference, and its mark is never struck at it); a movement keeps every rate its value reads, a composed consideration's market-line rate included. Each fact says where it came from: chain (the getter at the block), series (the history-mode fallback, at or before the block) or live (the now-mode fallback, the newest observation when the row was written).

One row, two rates: the 6h tick. The tick composes its market context in history mode (the getter, else the series at or before the anchor block) and resolves each row's redemption in now mode (the getter, else the newest observation), and keeps the redemption's fact. Where the getter did not answer, that fact is live, and the row's market line was struck at the series at or before the block. So on a CHECKPOINT reading a live fact prices the redemption line only, and the read prices a composed market line from the series at or before the block (nothing where the series holds no observation there: the tick's market line was null too). A live tip composes both lines in now mode, so its live fact prices both.

The loader rule

A row's value is COMPUTED when it can be, and its STORED value is served when it cannot. It can be computed when every rate its value reads (its needs: walked with the composition function's own recursions, rate-facts.ts) is answered by a fact:

• the market line comes from the bars, with the writer's walk-back;
• a chain or live fact is the rate (a checkpoint's live fact on its redemption line only, above); a series fact re-reads the series at the row's block (so late or corrected observations re-value the row, R12), with the stored rate as its fallback where the series no longer answers;
• a reading's facts are shared by its READING (one wallet at one moment, which one writer valued in one context), so a second chain-read level another leg of that reading kept is answered; a rate two legs of one reading state two ways answers nothing (it was not one context, and neither fact is the other leg's), so the legs that read it are served stored;
• a movement is decided on its OWN facts (and, where a PT's rate prices it, the stamp for the regime its block is in), and valued at them: movements whose facts disagree at one block are valued apart, so the events list, the activity statement and the engine serve one movement alike whatever else is on their page;
• a pre-window PT fill needs its stored consideration.

A rate no fact answers is answered by nothing: never by a series its writer did not use, never by the chain, never by another row's fact. Such a row is served its stored columns and counted: LedgerV2Stats.values.factless*, printed by the reconciler as factless=readings/movements/fills.

What is served stored, and what gets it to zero.

• Rows written before 116: the rate-facts backfill fills a row's facts where they reproduce the value it stores, and, since a reading's facts are shared by its legs, fills a rate for a reading only where every leg the fact turns computed reproduces its own stored value too. It leaves the rest served stored: a row struck at a rate neither the chain nor the series answers any more, and the legs of its reading that read the same rate.

• A PT movement with no stamp (derived before v0.69.0): a re-derive stamps it.

• A pre-window fill stored without its consideration: the backfill recovers it from the fill's own router log.

• An opening or adjustment is valued from the reading it was written from (the reading audit), and where that reading is not among the wallet's (a superseded live tip whose correction the audit kept where it stands) from the copy of the reading the row carries (meta.audit.reading), exactly as from the reading itself. Only a row that carries none and whose reading is gone is served stored; every row the audit writes carries it.

• Two classes the writers keep producing, which no backfill can fill, because their value reads a rate the one fact column of a reading does not hold:

  • a reading over a TWO-LEVEL chain-rated composition (wFalconX over the FalconX tranche: its column holds wFalconX's own rate, and its value also reads the tranche's NAV), computed only where another leg of the same reading kept that NAV;
  • a Pendle PT leg, or a PT posted as collateral, whose payout asset is COMPOSED (its market line reads the payout asset's rate, while its column holds the PT's own rate, a reference no value reads), computed only where another leg of the same reading kept the payout asset's rate.

  So factless=0/0/0, which the contract release that drops the stored columns requires (the plan, §9), is reachable only after a decision owed before that release: give a reading a second fact (a rate_facts map beside rate_raw, as a movement and a fill have), or store on a PT leg the payout asset's rate its value reads instead of the PT's reference rate, or keep these rows' stored columns through the contract release. Until then the reconciler's factless= is expected to stay above zero wherever a wallet holds either shape.

The census

scripts/ops/value-parity.ts is the census of what the read computes against what the writers stored, read-only against any database (--db-url, under that database's own registry). For every stored reading, movement and pre-window fill it compares the computed pair with the stored one line by line (and a PT movement's mark as a line of its own), classes each unequal line by relation, venue, asset and moment (a checkpoint, a live tip, a movement's block), and values it again over the series its writer could have seen when it wrote the row (bars ingested by then, rate observations written by then): a line that recomputation reproduces is late data, one reproduced only when the bars the gate has rejected since are counted is a corrected bar, and the rest is the residue (a row that recompute would serve stored explains nothing: its stored pair would only have reproduced itself). A row served stored for want of a fact is not compared: it is its own outcome (served stored (no rate fact)). The census also lists the lines null on both sides (a moment neither the writer nor the read could price). The reconciler prints the same comparison per wallet on every run (its values= line).

scripts/run-cron.sh

The single cron wrapper (scripts/run-cron.sh <script.ts> [args...]). It:

1. cds to the repo root and sources .env.local (so cron jobs get the same env as the app).
2. Takes a non-blocking per-script flock on $LOG_DIR/$(basename "$DIR")-<script>.lock, keyed by the checkout (the repo-root basename) so the prod and staging working copies on the shared box never serialize against each other; two runs of the SAME script in the SAME checkout do. If the lock is held (a previous tick still running), it logs "already running ... skipping this tick" and exits 0 — a skipped tick is normal, not a failure. The lock releases automatically when the process exits (fd 9 closes), so a crash never wedges it.
3. Runs the script through the repo-local tsx (node_modules/.bin/tsx), capturing the real exit code (via rc=$?, kept off set -e) so cron still sees the script's true pass/fail.
4. Tees combined stdout/stderr to /tmp/onchain-credit-cron/<checkout>-<script>.log, prepended with a timestamp banner each run. The log name carries the checkout for the same reason the lock does: prod and staging share the box and the directory, and a job's log that mixed both was two environments' runs interleaved with nothing in the line saying which was which.
5. On a non-zero exit, best-effort alerts a Telegram bot (WS8; env ALERT_TG_BOT_TOKEN + ALERT_TG_CHAT_ID, both unset = silent skip). The message names the checkout (ALERT_ENV override, else the repo-root basename — prod and staging share the box, so it must say which failed), the script, the exit code, and a why: block: the wrapper greps this run's slice of its own logfile (everything after the last run banner) for the failure markers the jobs print — [fail], [partial], .../fail], fatals, 429/5xx — and includes the newest few, capped so a pathological run cannot exceed Telegram's message limit. (It long claimed exit-code alerting "cannot see log lines"; it writes that logfile, so it can.) The block runs with errexit off and a time-bounded curl, so alerting can never change $rc (re-exited unchanged) or wedge the wrapper; a failed POST is logged as "ignored". ALERT_TG_API_BASE overrides the endpoint for local testing (prod uses the Telegram default). This covers every cron job, not only the portfolio one.

Extra args are forwarded, e.g. a SOFR backfill:

/opt/onchain-credit/scripts/run-cron.sh refresh-sofr.ts --since=2018-04-03

To run any refresher by hand on the box, use the same wrapper so env + tsx resolution match cron exactly:

/opt/onchain-credit/scripts/run-cron.sh refresh-assets.ts
/opt/onchain-credit/scripts/run-cron.sh refresh-vault-capacity.ts

Deploy does NOT run refreshers

The production deploy (.github/workflows/deploy.yml, triggered on every push to main) is code only. The GitHub runner SSHes to root@dexhq.io, does git fetch + git reset --hard FETCH_HEAD, npm ci, npm run build, pm2 restart onchain-credit --update-env, then scripts/ops/restart-ingester.sh, which restarts the event ingester and then the ledger worker (rolling back to the previous commit if the build fails). It does not run DB migrations, backfills, or data refreshers.

So when a change needs new or backfilled data, that is a manual server step after the deploy:

# on the box, after the deploy lands
psql -d creddit -f /opt/onchain-credit/scripts/sql/0NN-whatever.sql   # migration, if any
/opt/onchain-credit/scripts/run-cron.sh refresh-<thing>.ts            # or a backfill

ISR re-prerender caveat

Pages are App Router with per-page ISR (revalidate 1800s on home / repo-lending / multi-strategy-funds, 3600s on asset-profiles / carries), prerendered at build time. A page only picks up freshly written DB rows on its next revalidate cycle. So if you run a refresher (or backfill) after a deploy's build has already prerendered the pages, the new data can be invisible for up to the revalidate window. To force it immediately, re-run the deploy (or otherwise rebuild) so the pages re-prerender against the now-current DB.

Backfill scripts

scripts/backfill-*.ts are one-off history-seeding jobs (not crons): seeding token_yield_apy for a newly added wrapper, reconstructing deep Fluid history from core-storage reads (backfill-fluid-core.ts, backfill-fluid-history.ts), repairing a gap (backfill-recent-gap.ts), etc. They reuse the same annualizeRatio math and (where possible) the same *ForSnapshot functions as the live refreshers, so backfilled history is methodologically identical to live data.

scripts/backfill-portfolio-wallet.ts is the exception that is BOTH a queued cron job and a manual tool: it replays ONE registered wallet's deep history back to the history floor (basis='backfill') and is the portfolio repair path (windowed delete+insert). See Registration backfill (WS5). It must run with ETHEREUM_ARCHIVE_RPC_URL set (publicnode rejects archive eth_call); the cron spawns it with that env automatically.

Run them through the same wrapper:

/opt/onchain-credit/scripts/run-cron.sh backfill-wsteth.ts
/opt/onchain-credit/scripts/run-cron.sh backfill-fluid-core.ts

The three rate backfills share one set of flags

backfill-aave-v3.ts, backfill-sparklend.ts and backfill-morpho.ts all walk the same aligned 6h grid and take the same range / resume / pacing flags from scripts/backfill-range.ts:

Flag	Meaning
--from / --to	The stretch to walk (YYYY-MM-DD or a full ISO instant). Default is the whole table, from the 2025-05-21 history floor to the current window. Both ends are FLOORED to the window containing them, so a bare --to 2026-07-11 ends at that day's 00:00 window, not at its last one; pass an explicit instant (--to 2026-07-11T18:00:00Z) when you mean a day's tail.
--extend-history	Lift the correction-only bound below. Inserts rows, and is not part of the index-correction program.
--days N	Shorthand for --from (now - N days). Ignored when --from is given, so the two cannot disagree.
--since <iso>	Resume. Skips any window whose rows were all written at or after that instant, so an interrupted multi-hour walk does not re-spend its archive reads. Each run prints the instant to resume it with in its header.
--sleep-ms N	Pause between windows when a public archive endpoint starts refusing.
--symbols A,B	Aave / Spark only: restrict to those reserves.
--market <id>	Morpho only: restrict to those bytes32 market ids (rate history only; refuses --exposure). Not needed to reach a delisted market: the correction-only default already does, because it is bounded by the stored rows rather than by registry status.
--drop-pre-creation-rows	Morpho only: delete the rows a market holds for windows before it existed on-chain. Dry run unless --apply.

Three rules hold for all three.

They are correction-only by default. Each key is walked across the span it already holds rows for, so a rewrite changes values and never row counts, and the verification step can compare counts before and after as a real check. This also means the walk reaches keys the live cron no longer selects, which is how a delisted Morpho market's history gets corrected at all (an unscoped live walk resolves membership through status = 'active' and cannot see it). Deepening coverage is the separate, opt-in --extend-history job: Aave cbETH has been a live reserve since before the app's history floor but only carries rows from 2026-06-08, and filling that gap is a different decision from correcting a method. A key that is present in the table but that the refresher no longer tracks at all (SparkLend weETH) can be corrected by neither, and each run names those keys in its header so nobody has to discover it from a row count.

They walk oldest first, because the *_apy_24h columns are computed against the stored index on the row 24h earlier, so a window can only be rewritten correctly once its predecessors have been.

A window that comes back with nothing written is retried (three attempts, backing off) and then counted: the refreshers catch per item, so a rate-limited window otherwise returns "0 ok, N failed" and the walk moves on leaving a hole that reads like a clean pass. All three exit non-zero on any failed item or lost window, which is what run-cron.sh alerts on.

After rewriting index history on any of the three, re-derive the 24h columns: backfill-apy-24h.ts for the Aave-family tables (it recomputes them from the stored indexes alone, which settles any window that straddled the rewrite), and a plain rate-only backfill-morpho.ts walk for Morpho, which has no bulk 24h script.

DefiLlama backfills must use batchHistorical. Per-timestamp historical price calls get rate-limited (HTTP 429) at backfill scale. The DefiLlama-backed backfills (e.g. backfill-token-basis.ts) encode {coin: [ts, ...]} and hit https://coins.llama.fi/batchHistorical so each request covers many timestamps at once. Do not loop per-timestamp.

Backfilling collateral-exposure history (backfill-morpho.ts --exposure)

backfill-morpho.ts fills rate history by default. --exposure additionally writes the market_collateral_exposure row for each window, marked at the loan token's price at that window rather than today's, which is what makes exposure history backfillable at all: a current mark says nothing about a book from 2025, so without a vintage price the only honest answer was to skip the window, which is what the writer did (and still does whenever it marks at current prices).

/opt/onchain-credit/scripts/run-cron.sh backfill-morpho.ts --exposure
/opt/onchain-credit/scripts/run-cron.sh backfill-morpho.ts --exposure --days 90

Three rules bound it, all about ownership of a table two writers touch:

• The present is never written here, and the present means the whole 24h freshness window (EXPOSURE_MAX_AGE_MS), not just the current stamp. Both readers serve the newest row per market while it is that young, so a backfilled row landing inside the window would be served as the current book. The run stops strictly short of that horizon: the trailing ~30h belongs to the 6h cron alone, which writes it complete, posted collateral included. Every row the backfill does write is already too old for the panel the moment it lands, so running it never changes what any page shows today.
• --exposure refuses a scoped (--market) run, loudly rather than silently writing nothing: the collateral-collision guard judges a contested key against the markets the registry ADMITTED, and a scoped read admits only the market asked for, so a sibling claiming the same collateral token is invisible to it; a one-market stamp would also run ahead of its siblings'.
• A window whose mark is refused keeps its rate rows and writes no exposure row. Never a zero, never par. Re-running fills it. The run's summary and progress lines report exposure as its own class (exposure N written, M skipped), and a run that filled none of the history it was asked for exits non-zero so the cron wrapper alerts on it instead of reading as a clean pass. A partial fill still exits 0: re-running is the documented repair.

What it does to the overcollateralization column. A vintage row the backfill creates carries collateral_usd null (the Blue API serves current state only, so posted collateral cannot be reconstructed for a past window) and the column hides for that vintage. It does not blank a figure the cron already stored: re-marking a stamp the cron had filled keeps that stamp's posted collateral and re-values only the borrowed side, because nothing could rebuild the collateral figure once it was gone. So --days 90 --exposure deepens history without eroding it, and the run is safe to repeat.

API budget. One keyless coins.llama.fi request per window, with every market's loan token batched into it, so a full run from the 2025-05-21 floor is ~1,800 price requests. That is deliberately a per-window loop rather than batchHistorical, and it is within budget for one specific reason: the job is archive-RPC-bound (roughly a hundred archive eth_calls per window), so the price calls arrive at a handful per minute rather than in a burst, and block-by-timestamp already resolves two blocks per window through the same host — the exposure marks are a ~50% increase on DefiLlama traffic the job has always made, not a new class of load. A 429 or 5xx is retried twice before the window gives up its exposure rows. If the walk is ever made faster, batchHistorical is the escape hatch, and the house rule above applies again.

Not wired for Fluid or Aave/SparkLend, deliberately. Their exposure rows are not a price problem. Fluid's slices come from an on-chain vault census plus per-vault tick/branch storage and the Liquidity Layer's book at the anchor block; Aave/SparkLend's come from a borrower-registry log scan and per-account collateral reads at the anchor block. Backfilling either means reconstructing historical on-chain STATE, of which a vintage price is a small part, so a historical price path buys them nothing on its own.

The August 2026 USD yield batch

The August 2026 USD yield-token batch (venue sizes and per-token dossiers in the verification survey) added five token_yield_apy series and one one-time backfill script each. All five are plain ERC-4626 vaults reading convertToAssets on their own address, so every script is a thin config over the shared scripts/lib/backfill-erc4626-rate.ts:

/opt/onchain-credit/scripts/run-cron.sh backfill-usd3.ts    # 3Jane USD3,       from 2025-10-23
/opt/onchain-credit/scripts/run-cron.sh backfill-srusde.ts  # Strata srUSDe,    from 2025-10-03
/opt/onchain-credit/scripts/run-cron.sh backfill-susdd.ts   # Savings USDD,     from 2025-09-29
/opt/onchain-credit/scripts/run-cron.sh backfill-stusds.ts  # Sky stUSDS,       from 2025-10-06
/opt/onchain-credit/scripts/run-cron.sh backfill-wsrusd.ts  # Reservoir wsrUSD, from 2025-04-29

Each START is a verified anchor, not a round date. Four of the five open at the first 6h boundary after the vault's first non-zero supply, and the reason is not tidiness: on an empty vault convertToAssets either reverts (srUSDe, which would exit the run non-zero) or returns a flat 1.0, which publishes weeks of manufactured 0% where nobody was earning anything. backfill-usd3.ts starts later still, and deliberately drops real history: one transaction on 2025-10-22 swapped the implementation behind the USD3 proxy and moved totalAssets with totalSupply unchanged, taking the share rate from 1.0023 to 1.1556 in a single block. That is an accounting migration, not a yield, and publishing it would put ~530% into the month's apy_30d while breaking the one invariant every series here holds to (a published yield is the realised ratio of ONE compounding index between two blocks, and the index was replaced wholesale between those two).

USD3's long 0% band is real. Do not "repair" it. From that start the share rate is frozen at exactly 1.155560891228316997 until 2026-06-03, then accrues at roughly 6.9%. Holders genuinely earned nothing for those seven and a half months, so every supply_apy and apy_30d in the stretch is a true 0.00%, not the missing-snapshot signature backfill-recent-gap.ts looks for. A gap repair run across it would fabricate yield that was never paid.

Three more things the batch pinned, each of which reads as a bug if you meet it cold:

• USD3 is 6-decimal on both sides, so convertToAssets(1e18) comes back 1e18-scaled and takes the DEFAULT divisor 18 with only shareDecimals: 6 set (for the total_supply column), exactly like yvUSD and the Euler eUSDC vaults. The MetaMorpho vaults in the same list carry rateDivisorPow10: 6 because their share is 18-decimal over 6-decimal USDC, and copying that line here stores a rate a trillion times too large. USD3 is also NOT Reserve's Web3 Dollar, an 18-decimal token shipping the same ticker.
• wsrUSD is the savings vault itself, not a wrapper over srUSD: its asset() is rUSD, the par stable, so the rate already comes back in the USD-book unit. It also holds no rUSD at all, because totalAssets there is internal accounting against Reservoir rather than custody, so no coverage or solvency check may be built on it. Only the share RATE is sound, and that is all this series reads.
• stUSDS may fall. It sits junior to sUSDS in the Sky waterfall, which is where its premium comes from, so a loss would arrive as a DECREASING share rate. Nothing reading this series may assume monotonicity.

Run order mattered for /portfolio, not just for the charts. When this batch shipped, USD3 and srUSDe were the two of the five with a block-pinned rate getter, so a wallet holding them was valued from an exact archive read at the flow block whether or not any history existed, while sUSDD, stUSDS and wsrUSD resolved ONLY through token_yield_apy.share_rate at or before the block: a leg older than the first refresher row had no rate, and a rate that cannot be read means the leg is SKIPPED (M9), never defaulted to 1. #810 Y6 closed that split: every wallet-tracked variable-rate row now carries rate_kind = 'getter' and its own rate_getter, read at the leg's own block in both the history and the "now" mode, with sUSDai the one declared stored-series exception (its ERC-4626 accounting lives on the Arbitrum hub). The history still matters for the charts, so run these five before announcing coverage, and expect the depth of a holder's stored history to be bounded by the START dates above.

The IPOR Fusion vault, and the first share token that is not 18-decimal

backfill-ipor-liquity-carry.ts seeds token_yield_apy for the IPOR Fusion vault rETHLPC (listed on /multi-strategy-funds as IPOR Liquity Carry), from the day it was first funded:

/opt/onchain-credit/scripts/run-cron.sh backfill-ipor-liquity-carry.ts          # from 2026-03-03
/opt/onchain-credit/scripts/run-cron.sh backfill-apy-30d.ts                     # fills apy_30d
/opt/onchain-credit/scripts/run-cron.sh backfill-ipor-liquity-carry.ts --dry    # print, write nothing
/opt/onchain-credit/scripts/run-cron.sh backfill-ipor-liquity-carry.ts --from=2026-08-01

The loop writes 7 of the 8 data columns the live refresher writes; apy_30d lands NULL, which is what the shared, idempotent backfill-apy-30d.ts is for. Same convention as Lido Earn USD below. backfill-token-supply.ts is not part of the sequence: this script writes total_supply itself.

It is a plain 6h convertToAssets(1e18) + totalSupply() replay, so it is not written against scripts/lib/backfill-erc4626-rate.ts: that helper writes the RATE column only, and this series wants total_supply on the same pass so the fund's size is right from its first row rather than after a second job.

⚠ The share token carries TWENTY decimals. IPOR Fusion mints vault shares at the underlying's decimals plus two, so this WETH vault's decimals() reads 20. It is the first token in the registry whose share has MORE decimals than its asset, and both scaling fields move off the house default at once:

	Value	What it descales
shareDecimals	20	total_supply = totalSupply() / 1e20, in WHOLE shares
rateDivisorPow10	16	share_rate = convertToAssets(1e18) / 1e16, in WETH per WHOLE share

rateDivisorPow10 is the same formula every other vault uses (assetDecimals + 18 − shareDecimals = 18 + 18 − 20); it only looks unfamiliar because every previous vault had a share no wider than its asset. Neither mistake throws. An 18 in the supply slot reports the fund as 158,650 ETH instead of 1,586 shares; an 18 in the rate slot reports the share at 0.0097 ETH instead of 0.967. And the trap a per-field check misses: 18 in BOTH leaves total_supply × share_rate closing on totalAssets() while every published figure on either side of it is out by 100x. scripts/refreshers/token-yields.test.ts therefore checks all three — each half against its own mis-wiring, and the closure — off raw values recorded at block 25,795,883.

Every row is written; the dust head is trimmed at the READ layer. The vault was deployed 2026-01-30 and first funded 2026-03-03, and its first nine snapshots (2026-03-03T12:00Z through 2026-03-05T12:00Z) sit pinned at 1.000000 on a total supply of 0.015 shares, about $35 of book. The tick where it is actually seeded, 2026-03-05T18:00Z, steps −2.767% in one window as supply jumps to 1.44 shares. That is the same seeding artefact backfill-yoeth.ts trims for, so getIporLiquityEthSeries starts the published series at 2026-03-05T18:00Z (startMs, the YOETH_LAUNCH_MS / EARNETH_LAUNCH_MS pattern) while the backfill keeps storing every row.

The early drawdown is real and is published. From that seeding tick the share value falls to a 0.8934 low on 2026-03-30 and recovers to 0.967, on a book that grows from 1.4 to 34 ETH through March, with single-tick moves of −5.03% (2026-03-28) and +5.43% (2026-03-31) as position setup costs land on very little capital. That is a loss whoever held the fund actually took, so it stays in the series and the fund's drawdown statistic reads it: 8.54% max drawdown and −0.50% YTD measured over the trimmed series to 2026-08-20 (10.66% and −3.25% with the dust head left in, which is what the trim removes). Trailing windows are unaffected either way; their anchors are months later.

The two snapshots on 2026-03-03 before the vault held any shares are SKIPPED by its empty-vault gate and the run still exits 0 — a backfill that exited non-zero on its own first two ticks would page the cron alert on every healthy re-run. That gate is the only no-row path that exits 0: a share rate outside the [0.5, 2] scaling band counts as a FAILURE, so a mis-scaled read (or a genuine loss past halving, which a levered Liquity position can take) exits 1 on a line the cron alert quotes rather than going missing.

Lido Earn USD

earnUSD (0x4ce1ac8f…) is the dollar sibling of Lido Earn ETH: the same Mellow meta-vault primitive, not an ERC-4626 (asset(), totalAssets() and convertToAssets() all revert), so its price is a curator-published report read off an oracle. Full evidence base in the plan.

/opt/onchain-credit/scripts/run-cron.sh backfill-lido-earn-usd.ts  # from 2026-03-07 06:00 UTC
/opt/onchain-credit/scripts/run-cron.sh backfill-apy-30d.ts        # fills apy_30d

START is the first clean 6h boundary that already carries a report. The vault, share token and oracle were all deployed in block 24,602,425 (2026-03-07 01:54:35 UTC), where getReport still returns priceD18 = 0; the first live report lands two blocks later (report ts 02:02:23 UTC, → 1.00004 USDC/share). Starting earlier reads blocks with no report at all, which the script skips rather than writing as zero. backfill-token-supply.ts is not part of the sequence here — this script writes total_supply itself, so that shared job is a no-op for the token.

Three things about this series read as bugs if you meet them cold:

• It has its own oracle. getReport(USDC) on earnETH's oracle reverts with 0xee84f40b, "unsupported asset". A Mellow oracle is per vault AND per asset, so a USD fund on the same primitive is never reachable through the ETH fund's constant.
• The scaling is 1e30, not 1e18. Mellow quotes priceD18 as raw shares per raw asset unit, so the decimals both sides dropped have to be reintroduced: 10^(18 + shareDecimals − assetDecimals) / priceD18. earnETH's 18-over-18 collapses to 1e18; earnUSD is an 18-dec share over 6-dec USDC. Reusing earnETH's constant stores 1.026e-12 instead of 1.026 and nothing throws.
• Size is totalShares() (0x3a98ef39), not totalSupply(). Shares priced at a report but not yet CLAIMED are already a claim on the assets and already inside the reported NAV, while unminted as ERC-20. At block 25,795,199 the split is exact and the unclaimed slice is 26% of the vault, so a TVL on totalSupply is a quarter short. earnETH has the same split, and it is neither small nor stable: sampled on chain it runs 14.7% at block 24,700,000, 2.9% at 25,000,000, 3.0% at 25,300,000, 1.4% at 25,600,000 and 1.2% at head, so its published TVL series is understated by a time-varying 1.2–15%, the same SHAPE error one order smaller. It deliberately stays on totalSupply here, because switching it restates a TVL series that has already been published and that re-derivation needs its own change and its own backfill — tracked in issue #626.

Expect a step function, and do not "repair" it. Reports land roughly once a day, so on the 6h grid three of every four snapshots repeat the previous value. That is the published price standing still, not a missing snapshot. The report clock WANDERS — read off the oracle at the stored blocks, the report standing at the 18:00 snapshot was stamped 12:50 UTC on 2026-03-20, 13:08 on 05-20, 07:41 on 07-20 and 05:55 on 08-20, and over the 163 steps in the backfilled series the snapshot at which the price moves is 12:00 UTC 102 times, 18:00 UTC 52 times and 06:00 UTC 8 times — so nothing should key off a fixed publication hour.

What that does to the 24h column, which is the part that reads as a bug in both directions: a 24h window on a daily-report series contains ZERO, ONE or TWO reports depending on where the report clock sits relative to the snapshot boundary, so the reading is 0%, about the daily rate, or about DOUBLE it. Measured on the 661 stored supply_apy values: 25 are exactly 0, 4 are negative (the markdown below) and 9 are above 15%, topping out at 22.77%. The pairs are adjacent — 2026-07-10 reads 19.2% and 07-11 reads 0%; 08-03 reads 16.0% and 08-04 reads 0% — and this persists for as long as the report clock sits on a boundary. The 7d and 30d windows always span several reports and carry none of it (7d has never read 0.00% for this fund). Nothing here is a defect: the window arithmetic is right and the inputs are right. It is the reason a lone 24h print off an oracle-reported fund means very little on its own.

Both writers of a Mellow fund's share_rate must produce the same DOUBLE. The refresher and the one-time backfill each write this column, and each backfill upserts share_rate = EXCLUDED.share_rate, so a repair re-run lands on top of live-written rows. 1 / (Number(priceD18) / 1e18) and Number(10^(pow+18) / priceD18) / 1e18 are the same number and not the same double — they differ in the last bit for a large fraction of the range — and flatPeriods groups a stall by EXACT float equality, so a 1-ULP seam inside a stall run splits it and can make the band, and its annotation, disappear. Both scripts therefore use the refresher's integer expression. backfill-earneth.ts was brought onto it here, which is why the earnETH history is re-run once after this ships (see the plan): the rows already stored carry the float form.

Nothing may assume this index only rises. The backfill reproduced all eleven independently sampled reference points exactly and surfaced one thing sampling missed: a −0.369 ppm markdown on 2026-06-19 12:00 UTC (block 25,351,484), verified against the oracle at the two adjoining blocks with isSuspicious false on both. A real, tiny published NAV decrease, well inside the oracle's 0.1% flag / 0.5% rejection rails.

Money market fund history

scripts/backfill-money-market-funds.ts seeds token_yield_apy for the funds on /money-market-funds. Its universe is money_market_fund_registry WHERE status = 'listed' AND kind = 'money_market', so approving a manager is all it takes to bring its funds into the next run; the portfolio's own ERC-4626 backfill (backfill-curator-vaults.ts) is a separate walk over a separate registry and is deliberately left alone. The kind clause matters because the multi-strategy funds share that table: this job assumes every row's stored rate IS its convertToAssets(1e18), which is false for the ones whose rate is quoted in another asset (Treehouse ETH's is ETH per share, not wstETH per share), and an unscoped run would write that second basis into the same series.

scripts/run-cron.sh backfill-money-market-funds.ts 365

• DAYS is positional (default 365) and validated to an integer in 1..3650, with the same "did you mean --only=" guard the other registry-driven backfill carries: Number() parses hex, so --only 0x9fb7… with a SPACE instead of an = would read the address as DAYS, build a tick array with more entries than anyone could enumerate, and hang the box while --only stayed empty and the run silently re-walked everything.
• --only=<addr>,<addr> seeds one newly listed fund without re-walking the registry.
• A point already on record is KEPT, never rewritten. That is the default, not a flag, and it is what makes a resumed run cheap: the job is long and interruptible by design. It matters beyond this tab, because token_yield_apy is the series the portfolio values every yield-token holding from, so a rerun over a window that is already covered would restate published history on nothing but a second archive read of the same blocks. The run reports how many points it left alone (kept=).
• --overwrite replaces them, for repairing a window known to be wrong. The refusal is in the statement as well as in the pre-check, so a concurrent refresher writing the live tick cannot slip past it.
• --missing-only is accepted and does nothing: it names what the default now is, and stays valid so an operator following an older runbook is not stopped.
• Block lookups are hoisted to once per tick and shared across every fund, the same economy the curator-vault backfill makes.
• Archive reads go to ETHEREUM_ARCHIVE_RPC_URL (dRPC free by default). A read that can never succeed at that block, because the vault did not exist yet or the call reverts, is classified permanent and not retried; retrying those would cost hours, since most funds predate only part of the window.

Why it does not use the API's own series. Morpho publishes a daily share price back to inception, and it would make this backfill minutes rather than hours. It is not used, and the reason is not convenience: a chart drawn from an API-sourced daily series spliced onto chain-sourced six-hourly rows is one column with two provenances and two grids, and every yield this product publishes is the realised ratio of an on-chain index between two blocks. The published series is used as a CROSS-CHECK instead: a point that differs from the archive read by more than ten basis points prints a [cross-check] line naming the fund, the block's reading and Morpho's.

Cost. Roughly two archive calls per fund per tick plus one block lookup per tick. A 365-day window is 1,461 ticks, so a cold run over ~50 funds is around 150,000 eth_calls: budget six to nine hours, run it under nohup, and expect to resume by re-running the same command: the points already written are kept.

Repainting silent publish gaps in supply_apy

scripts/backfill-yield-gap-repaint.ts is the one-time historical half of the publish-gap repair described in metrics.md, "Publish gaps in a NAV-style share rate". The 6h refresher applies the repaint going forward; this script applies the IDENTICAL function (findGapRepaints in src/lib/data/apy-gap.ts) to the history that predates it, so no row is repaired by one rule and written by another.

It walks every token's (snapshot_ts, share_rate, supply_apy) series in token_yield_apy, finds runs over which the share rate did not move materially (relative move ≤ 1e-6) that are bounded by publishes more than 24h and no more than 7 days apart and contain a near-zero reading, and rewrites each run's supply_apy to the realised growth across it, annualised by the actual elapsed time. A flat run longer than the cap is a parked asset rather than a late publisher and is left exactly as recorded, as is any span whose annualised rate overflows to infinity (a seeding-period share-rate read); see metrics.md for both.

It is dry-run by default. --apply is required to write.

# read-only: per-token gap count, row count, and the old → new APY range
/opt/onchain-credit/scripts/run-cron.sh backfill-yield-gap-repaint.ts --dry-run
# one token, with every affected row printed
/opt/onchain-credit/scripts/run-cron.sh backfill-yield-gap-repaint.ts --dry-run --token=reUSD --rows=40
# write
/opt/onchain-credit/scripts/run-cron.sh backfill-yield-gap-repaint.ts --apply

Flag	Meaning
--apply	Write. Without it (or with --dry-run) nothing is written.
--token=<symbol|0x…>	Restrict to one token.
--rows[=N]	Print the individual rows of each gap (default cap 40 per token).
--since=YYYY-MM-DD	Only report/write gaps closing on or after this date.

One transaction per token, one UPDATE per gap span, and a token that fails is logged, skipped and reported in the closing summary rather than aborting the run; the exit code is non-zero if any token failed. Idempotent: a repainted span no longer reads as a gap at all, so a second --apply writes nothing (the UPDATE also carries supply_apy IS DISTINCT FROM, so it does not even restamp updated_at). Rows whose supply_apy is NULL are never written, and share_rate, apy_30d and every other column are untouched. No other table is read or written — the lending-index tables carry continuous on-chain accumulators and have no publisher to wait on.

Two pages have to re-render, not one. /asset-profiles and /carries are both revalidate = 3600 and both prerendered at build, so neither shows the repaired series until its ISR window turns over (or the deploy is re-run — see ISR re-prerender caveat). /carries is affected because carries-table.ts reads supply_apy directly as the collateral leg's rate and turns it into the daily sample behind the carry's mean, its volatility statistics, and the screener's Carry range column. The sawtooth is currently inflating measured dispersion on every NAV-collateral carry, so after the repair expect those rows' displayed P10-P90 bands to narrow, and to move up both the Carry range sort and the assistant's vol-adjusted ranking. That is the intended direction, but it is a visible ranking change, not a silent one. The carry statistics are computed at read time from the raw snapshots and never persisted, so there is no stored sample that could disagree with the repainted rows.

The 6h cron may overlap this run, harmlessly. run-cron.sh's flock is keyed per script, so backfill-yield-gap-repaint.ts and refresh-assets.ts have different lock files and will happily run at the same time. Both compute the same value from the same function, so a contended row resolves the same way either way; still, prefer starting this just after a 6h tick rather than on the hour.

A later replay silently un-repaints a span.backfill-recent-gap.ts and backfill-fluid-history.ts re-run refreshTokenYieldsForSnapshot for historical timestamps, and that write is an unconditional upsert — so a replay covering ticks inside an already-repainted span puts the raw sawtooth values back. Nothing restores them on its own: those snapshots are not material rate changes, so the live repaint correctly finds nothing to do, and there is no log line or alert to notice. After any replay that writes token_yield_apy, re-run backfill-yield-gap-repaint.ts --apply (scope it with --since to the replayed window if you want a short run).

Adding a deposit asset to the Repo lending page

Three edits, in this order, then one backfill:

1. Rate history. Add the reserve to AAVE_V3_TOKENS / SPARKLEND_TOKENS (scripts/refreshers/{aave-v3,sparklend}.ts) with its real decimals. Fluid needs nothing: fluid-ll.ts enumerates every Liquidity Layer token already, so an asset there has history from 2025-05-21 on day one.
2. Underwritten capital. Add it to STABLES (scripts/refreshers/shared.ts) with the poolVenues it is actually a reserve on. That one constant drives both the Fluid and the Aave/Spark exposure writers.
3. The page. Add it to MoneyMarketAsset + MONEY_MARKET_ASSETS + ASSET_DECIMALS + ASSET_ADDRESS in the CLIENT-SAFE src/lib/data/money-market-assets.ts (NOT the reader: a value imported from money-market-rates.ts pulls Postgres into the client bundle), then add its venue rows to STATIC_MARKETS in src/lib/data/money-market-rates.ts. A venue is a repo market only if a third-party lender is compensated and the rate is cleared by utilization. A reserve that fails that test skips this step, and it may skip step 1 as well: Aave's GHO reserve takes step 2 only. It is deliberately absent from AAVE_V3_TOKENS, so it is snapshotted only while an active carry borrows it, which is why its rate history is not continuous and must not be assumed so. Register its coin in src/components/icons/token-marks.tsx, give each new STATIC market id a colour in MARKET_COLOR (MoneyMarketRatesChart.tsx), and widen REPO_LOAN_ASSETS in scripts/morpho-rule.ts if isolated markets on the asset should be admissible. Registry-driven Morpho markets need no entry: a market's colour is a function of its id alone, drawn from a separate palette that shares no value with the pinned ones and holds a perceptual distance from every one of them, assigned over a filter-independent anchor list. So a market keeps its colour as the selection changes, and past the palette's ten rungs it extends into deterministic shades rather than repeating. The tests in money-market-rates.test.ts, token-marks.test.ts and morpho-rule.test.ts fail on a missing decimals entry, a missing coin, or a loan-asset gate that has drifted from the tabs.

Then seed the reserve's history, which is the only manual step:

/opt/onchain-credit/scripts/run-cron.sh backfill-usds-reserves.ts

backfill-usds-reserves.ts walks every 6h snapshot from 2025-05-21 forward and calls the SAME refresh*ForSnapshot the live crons use, with a ["USDS"] allow-list so it re-reads one reserve instead of all 30+. The walk must run forward in time: each snapshot's supply_apy_24h anchors on the row at-or-before ts − 24h, so a backward walk would null the first day of every run. Both venues are attempted per timestamp and isolated from each other, so a transient failure on one does not punch a hole in both series.

Decimals are the trap here. fluid_ll_apy stores its size columns as RAW token amounts, so the divisor belongs to the token. USDS and GHO are 18-decimal against USDC/USDT's 6; a shared constant would report a $730M book as $730 trillion with no error anywhere. fluidScale() reads ASSET_DECIMALS, and the test asserts each entry against the token's real address and decimals().

Adding a token to basisTokens()

The basis class has ONE value now, market (#810 Y2; see metrics M19, retired). pinned is gone, and so is the whole pinned family in code: nothing writes a market_price_usd equal to its redemption_value_usd, and every tracked wrapper's real premium or discount draws. sUSDS was the only declared member, and it marks off its own bar like everything else.

Adding an entry is not only a charting change. What decides an asset's mark is its REGISTRY ROW (portfolio_tokens.valuation / feed), read through the composition function (src/lib/portfolio/unit-prices.ts); a basisTokens() entry buys the asset a stored basis SERIES and the Market Depth chart, not a valuation method. Before #810 the two were entangled — a par token in no mirror registry had its market mark pinned to par by construction, so its depeg could not draw anywhere, and adding it to a registry was how you dissolved the pin. Every idle asset declares a Dune feed now (R4), so that population is empty and the entanglement is gone. USDS (0xdc03…384f, listed on Aave v3 + SparkLend) and AUSD (0x0000…012a, an admitted Morpho funding leg) are par entries for the ordinary reason: both are covered assets and a basis series is worth having.

Every entry ALSO carries a required liquidity: "secondary" | "primary_buffer" and a liquidityVerified evidence date, on the same footing and enforced by the same test. A basisTokens() row is always secondary: this registry divides a market quote by a redemption reference, so an asset with no market has nothing to put in it. Before adding one, check that it is MARKET-priced at all — the two categories, the test behind them, the display states and the onboarding steps are in metrics and processes B.2a.

srUSDe, tETH and liquidETH joined the basis registry in a 2026-08 change (from MIRROR_EXTRA, a list #810 deleted). All three were already mirror-tracked (their marks floated off Dune bars) but nothing derived a basis for them, because the refresher iterates basisTokens() alone; all three trade and all three already had a share-rate source in BASE_YIELD_TOKENS, so both halves of market / redemption − 1 existed and only the registry row was missing.

Be precise about what that switches on: a SERIES, not a chart. The only reader of token_basis is the Market Depth panel, reached through collateralLegMap() / debtLegMap(), and no strategy names any of the three as a leg — so nothing renders them today, and MIN_HISTORY_DAYS is not the reason (there is no leg to hide). What the promotion buys is that the series ACCUMULATES from switch-on, so the day a lending market lists one as collateral it arrives with history instead of starting from zero, and the feed behind it is kept whole by the six-hourly hole fill in the meantime. If one is listed inside the first 30 days, the MIN_HISTORY_DAYS floor then does apply and its leg shows "no history tracked yet" rather than calling a one-week series a one-year drawdown.

srUSDe's redemption reference is USDe valued at $1. For a USD-numeraire wrapper the reference is share_rate × $1, and srUSDe redeems into USDe, so its published basis carries USDe's own deviation from the dollar on top of srUSDe's. This is the convention sUSDe already uses (migration 074 names it the synthetic-dollar caveat), not a new one, but it is worth knowing before reading the number: a USDe depeg moves srUSDe's basis without srUSDe having moved.

The 6h token-basis refresher iterates basisTokens(), so a new entry starts snapshotting on the next tick with no backfill. History is the manual part:

# START at/before the token's launch — the run only writes what DefiLlama serves.
/opt/onchain-credit/scripts/run-cron.sh backfill-token-basis.ts 2024-09-01T00:00:00.000Z --only=sUSDS

Always pass --only. A bare re-run re-fetches every price from DefiLlama and re-derives rows that already exist, so an upstream price revision would silently move published history; --only scopes the write to the new token and leaves other tokens' stored rows untouched. An unknown symbol is a hard error rather than a no-op, because a backfill that writes nothing looks exactly like one that succeeded.

--only does not protect the scoped token's own rows: the upsert is ON CONFLICT (snapshot_ts, token_address) DO UPDATE, so every timestamp in range is re-derived from freshly fetched prices. On a token whose history is already published, re-running from launch silently rewrites the series behind figures the docs and the panel already quote. That is fine on a token being backfilled for the first time (nothing to rewrite) and is why the sUSDS run below was safe: it had 3 refresher rows. On an established token, scope START to the range you actually mean to rebuild.

Run it in the same change that adds the token — the 30-day floor expires.MIN_HISTORY_DAYS = 30 in basis.ts hides any leg with under 30 days of history, so a freshly tracked token first renders /carries Market depth's "no secondary-market history is tracked for this pair yet". That is what hid sUSDS on morpho-susds-usdt for three days (tracked 2026-07-14, backfilled 2026-07-17).

The floor is not a standing guard, and this is the part that bites: historyDays is (Date.now() - sample_start) / 86_400_000 where sample_start is the oldest row in the last 365 days. The 6h refresher writes a row every tick, so historyDays grows on wall-clock alone. Skip the backfill and the leg does not stay hidden — at day 30 the gate simply opens, and the panel starts publishing worstAdverse1y and a chart labelled one year from a month of rows. An un-backfilled token fails loudly for 30 days and quietly thereafter, so "it's still hidden" is not evidence the backfill is outstanding, and the invisible window is the only period in which the omission is obvious.

Pick START from the token's launch, not from a recent date. The skipped (token, snapshot) count is expected and benign: the grid spans every basis token's timestamps, and a snapshot with no price (or no share_rate) for the selected token is skipped, so a run that writes 1,394 rows and skips 1,342 is healthy. A too-recent START silently costs history — it cannot be told apart from a source limit by looking at the result.

Do not audit coverage by dividing rows by 4. The grid is not a uniform 6h series: sUSDS is daily before ~2025-05 and 6h after, so its 1,394 rows are 431 distinct days, not 349. Count DISTINCT snapshot_ts::date and list the gaps (lag(snapshot_ts)) instead.

A superseded note here claimed DefiLlama "serves no historical price for sUSDS or sUSDe before roughly 2025-08-25" and that a re-run could not recover earlier months. Both were wrong, and it was the START (then 2025-06-01) that bounded the data: re-running sUSDS from 2024-09-01 recovered 2024-10-10 onward, and sUSDe has held rows back to 2024-03-06 all along. Verify a suspected source limit against the table before writing it down. What is real for sUSDS is a 202-day hole (2025-02-05 → 2025-08-25) where DefiLlama serves no price while share_rate stays continuous; that hole, not a coverage floor, is why its 1y stats read ~326 days.

Rebuilding token_basis from the mirror (done)

The 6h refresher sources token_basis market quotes from the Dune mirror (the same-bar rule above), but the served HISTORY was first written by the old DeFiLlama path. A one-time, review-gated procedure rebuilt it: a shadow backfill re-derived the series into a scratch table (token_basis_dune_shadow) with the refresher's exact computeBasisFromBars, a parity report compared the served series to it per token, and a single-transaction swap promoted it. It ran on prod on 2026-07-20 (24,820 rows over 17 tokens, 2025-07-20 to 2026-07-20; the served rows still match the shadow for 16 of them, and sUSDe was re-sourced from its DEX-ratio feed on 2026-08-11).

The tooling is gone: the two scripts and their stats helper were deleted in #909, and migration 108 drops the scratch table. A future rebuild would start from computeBasisFromBars and token_price_bars, both still standing. scripts/backfill-token-basis.ts remains for reconstructing PRE-mirror history only, behind its --i-know-deprecated refusal.

Repairing stored portfolio marks from the mirror (guarded re-mark)

This repair no longer moves a served number. The page computes both money columns at read (why); the stored value_market it rewrites is read only by the revert path and the parity census. It keeps that column consistent with the series.

The valuation switch (Token price bars (Dune mirror), Milestone C) makes NEW portfolio marks coherent, but the marks ALREADY stored in portfolio_position_snapshots.value_market were written by the old DeFiLlama path and carry the ±30-50bps of cross-vintage noise (the phantom-wallet −0.82 ETH entered basis, §1 of the plan). A guarded repair script re-marks that stored value_market column FROM THE STANDING MIRROR and nothing else. (Its counterpart for the retired flow ledger went with that ledger's writers; the rebuilt ledger values every row through the mirror as it derives it, so there is nothing of that vintage left to repair there.)

The script re-derives value_market = amount × priceInBook(asset, book, ts) where amount is the ALREADY-DESCALED stored column (qty_underlying), so the re-mark reproduces the exact composition buildSnapshotRow used, venue-uniformly (the Aave rayMul / ERC-4626 index / Fluid leg / Morpho descaling is already folded into the stored amount). The price is read from token_price_bars only (the hourly newest-common-bar ≤ the row's ts, same-bar division) — pure DB reads, ZERO Dune credits, no fetch-through (a scattered historical minute is uneconomical; the ~1-3bps of hourly-vs-minute staleness is within the phantom acceptance band). After the standing read, standingMirrorMarks runs the same COMPOSITION FUNCTION as the live path (unit-prices.ts: a derived base through its wrapper's bar, a composed wrapper or fund share through its own rate and its underlying's mark, chained down to a traded asset) so a repaired historical mark matches live by construction rather than by two implementations agreeing. Each repair row carries its stored block_number; rows are grouped by (timestamp, block_number), composition reads the newest share rate at or before that block, and no timestamp fallback is used when the block-anchored rate is absent. Derived stETH/eETH instead use the newest wrapper share-rate observation at or before the actual selected wstETH/weETH mirror bar_ts, matching the normal valuation path and preventing cross-time division. The pure re-derivation + diff logic is scripts/repair/remark-lib.ts (unit-tested in remark-lib.test.ts).

Locked rules (do not reopen):

• value_redemption is NEVER touched — it is block-precise and verified-correct (§1).
• A row with no bar / no common bar at its ts is LEFT UNTOUCHED, logged, and counted — never zeroed, never partially derived (the 2026-06-25 lesson: a partial repair must not manufacture values).
• Excluded from the re-mark: PT-family rows (plan §7 out of scope — venue pendle, or an Aave/Spark reserve whose underlying is a known PT); WETH / ETH-sentinel identity legs in the ETH book (market ≡ amount); book='EXCLUDED' (no book unit); rows with a null stored amount.
• Idempotent: re-deriving an already-repaired row yields the same value (same stored amount × the same standing bar), so a second run writes nothing.
• Writes run as batched UPDATEs (≤1000 rows/statement) inside ONE transaction that holds the portfolio writer advisory lock (write-lock.ts), serialising against the live cron / JIT / backfill writers. The script NEVER TRUNCATEs or DELETEs.

Ops flow (staging first, then prod with permission):

1. Dry-run (the default — no --execute). Prints per-wallet + per-asset repaired-row counts, the sum of |Δ value_market|, the 10 largest |Δ| rows (wallet, ts, asset, old, new), and any skipped-no-bar assets (a possible mirror coverage gap). Review these before writing.

   /opt/onchain-credit/scripts/run-cron.sh repair/remark-snapshot-marks.ts

2. Wallet-targeted mode (--wallet=<addr>) scopes both the dry-run and the execute to one wallet — used for the phantom-wallet acceptance check before a global run.

   /opt/onchain-credit/scripts/run-cron.sh repair/remark-snapshot-marks.ts --wallet=0xef08c6a47c04764b9c0964e00bd01c3e200707c5

3. Execute (--execute) performs the batched UPDATEs under the writer lock. The snapshot re-mark re-levels the stored basis / PnL chart series across the whole history — the charts will show less wiggle. That is the point (the DeFiLlama noise is deleted); communicate it. A [fail]-bracketed line + non-zero exit on any DB/derivation error pages the run-cron Telegram alert.

   /opt/onchain-credit/scripts/run-cron.sh repair/remark-snapshot-marks.ts --execute

Acceptance: the phantom wallet's entered basis reads +0.05 ETH ± 0.02 via the real /api/portfolio/* path (POST refresh before GET), a random sample of repaired snapshot rows re-verifies against on-chain pool reads at their blocks within a few bps, and a levered wrapper position's stored basis chart is visibly de-noised (before/after screenshot pair).

backfill-fluid-dex-pershare.ts (per-share pot content)

Fills the six per-share / pool-price columns (migration 047) on fluid_dex_apy rows written before the refresher read them. Every row already carries block_at_snapshot, so there is no timestamp-to-block resolution: it re-reads getDexState(pool) at that exact block and stores the raw words.

/opt/onchain-credit/scripts/run-cron.sh backfill-fluid-dex-pershare.ts
/opt/onchain-credit/scripts/run-cron.sh backfill-fluid-dex-pershare.ts --pool 0x3c04…cda7
/opt/onchain-credit/scripts/run-cron.sh backfill-fluid-dex-pershare.ts --force   # re-read non-NULL rows

• Per-pool floor. The DEX resolver (block 23,881,747, Dec 2025) is newer than most pools and returns EMPTY returndata below its deployment, so the script binary-searches each pool's first decodable block and skips every row beneath it. Those rows stay NULL forever, which is a property of the resolver, not a gap to repair; the per-pool report prints the floor and the below-floor count so the coverage is stated, never silently truncated.
• Idempotent + resumable. Rows are selected on token0_per_supply_share IS NULL, so a re-run only touches what is still missing.
• Retries are load-bearing. getDexPerShareState THROWS on an RPC failure and returns null only for genuinely empty returndata. Conflating the two would let a transient 429 push a pool's floor forward and permanently strand readable snapshots, so a failed probe retries with backoff and, if it still fails, the pool is skipped rather than floored.
• Order on prod: migration → deploy → run the backfill → check the per-pool report. Staging reseeds from prod nightly, so the durable run is the prod one.

Token price bars (Dune mirror)

onchain_credit.token_price_bars (migration 054) is a local mirror of Dune USD price bars — the ONLY historical market-price source the mark pipeline reads (the Dune price-mirror refactor built it; #810 hardened it in migration 098). Every bar is hourly (prices.hour, bars on the hour), there is no finer grid anywhere, and a DB CHECK enforces it. The point of one grid: a portfolio mark and a token_basis row divide two quotes from the same bar, so the fast ETH/BTC USD level cancels exactly and only the slow secondary-market basis survives — deleting the ±30-50bps of cross-vintage DeFiLlama noise that produced the phantom-wallet entered-basis error. A mark reads the newest common bar ≤ its ts, within ~1-3bps of basis drift in calm markets. The migration creates the table empty; it is filled by the bulk load + the 6h sync.

• Schema. (chain_id, token_address, bar_ts) PK; price_usd NUMERIC CHECK (> 0); source — the true query provenance stamped per row, dune:prices.hour (standing / window bars), dune:dex-ratio (a token routed away from the aggregated channel, below), llama:coins (a token priced from DefiLlama's aggregate), or coingecko:hourly / llama:chart (an hour the tape never printed, bought by the hole fill); ingested_at. chain_id 1 = ethereum (every ERC-20, incl. the WETH denominator), 0 = the bitcoin reference row (the BTC-book numeraire, normalized to a fixed sentinel address). bar_ts is the bar START (UTC), on the hour, with a DB CHECK on the alignment. Migration 098 adds rejected + reject_reason (the spike gate's verdict), volume_usd + dune_source (the two columns the saved queries emit), and carried (derived at write time by every writer — a bar that repeats the preceding hour exactly; wherever the writer could not have set it — a changed writer, a hole in the grid, an hour written before its own predecessor — it is false by construction rather than by measurement, which any reader of the column has to allow for, see Database & schema). Every reader names those five columns only after probing information_schema (src/lib/data/mirror-columns.ts), because the basis panel reads them during next build and a reference to an unapplied column would fail the deploy before the migration could land. See Database & schema.

• Client + the window query. src/lib/data/dune.ts — plain HTTP: execute a saved parameterized query → poll (reading execution_cost_credits free) → page (32k/page) → INSERT ... ON CONFLICT DO NOTHING. Every window goes through the HOURLY query (DUNE_QUERY_ID_HOURLY, query 8033816, prices.hour, bars on the hour, cost sublinear in span); Dune's finest-grained price table bills by the TIME PARTITION a query touches, ~110-170 credits for a month whatever slice is asked for, which is why nothing reads it. Reads take the bar covering a ts (bar_ts = floor(ts/3600)*3600), walking back to the newest bar within BAR_STALE_HARD (48h) and returning the bar's actual bar_ts so a caller can flag staleness. A served bar older than BAR_STALE_SOFT (6h) is still coherent (same-bar pairing holds at any age) and still served — a real, stored, coherent bar beats a null — but getBarSeriesAt logs one line per run (token count + max age) so a lagging mirror is visible in ops (M20). Beyond 48h the read misses and null takes over.

• Sync cadence, and the PER-TOKEN cursor. The token-basis refresher prepends an incremental HOURLY syncBars on its existing 6h grid (~0.25 credits/run, ~30/month); no crontab change. Until #810 R6 the window opened at the table-wide MAX(bar_ts), and that was a silent data-loss bug: the watermark is advanced by whichever token Dune ingested most recently, so a token that lagged had its window stepped over and, because bars are append-only and nothing else back-fills the standing feed, those hours were gone for good. Every token lost 19h in the week of 2026-08-24 that way, and PST was dark for 101h in July 2026 before anyone noticed, two months later. onchain_credit.token_price_sync_state now holds one cursor per token — its newest ACCEPTED bar at its last successful sync, plus the attempt / success stamps and a consecutive-failure count — and the window opens at the oldest cursor in the set. A failed execution records the attempt and leaves every cursor where it was, so the next tick asks for exactly the window that was lost. The tokens still ride ONE execution: the hourly query's cost is sublinear in span and dominated by a per-execution floor (~0.22 credits), so forty per-token executions would cost ~9 credits a tick against ~0.25 for one, and over-fetching a token that is already current returns bars ON CONFLICT DO NOTHING discards. What changed is where the window STARTS. A token whose own gap is WIDER than one execution's 48h window is not folded into that window and clamped. Clamping and then succeeding would advance its cursor to its newest bar, i.e. straight past the hole, and append-only would make that permanent — the same class of loss the global watermark caused, just narrower. It is carved out into a CATCH-UP leg instead and loaded in 90-day chunks from its own cursor, exactly as the floor load is, so nothing is jumped. A chunk that fails leaves the cursor where it was and prints a [partial] line leading with the bulk-tool remedy inside the alert's 220-character cut (the non-zero exit the alert also needs comes from the dark-feed check, which fires for those tokens by construction). A completed catch-up advances the cursor to the END of the span it asked for, even when nothing came back: a span Dune was asked for and had no rows in is answered, and without that a permanently quiet feed would re-buy the same chunks every six hours forever. The feed is still reported dark, because that reads the newest BAR, not the cursor.

• Auto-backfill on add, from the 2026-01-01 floor. A token in the standing set with no history at the floor — or whose floor load never completed — is loaded from 2026-01-01 by the next tick, in 90-day chunks, before the incremental leg. That closes the trap that left AUSD and USDS half empty for weeks after they joined the set on 2026-08-10: adding a token used to be two operations, and the second one was easy to forget and invisible for the 30 days MIN_HISTORY_DAYS keeps a panel hidden. The floor is the portfolio's own history floor and nothing is ever loaded below it: a bar older than it cannot be read by any mark, so buying it spends credits on rows nothing consumes. floor_loaded records that the load RAN over the whole span, which is not the same as "bars came back" — a token whose market did not exist in January must not re-buy its absent history every six hours forever.

• Fetch-through. A read miss beyond the 48h walk-back triggers ONE HOURLY syncBars for a ±1h window (cheap floor), then re-reads; a second miss returns null (M9 — an honest null, never a fabricated mark, never a thrown valuation).

• Burst control (the execution gate). Dune bills per execution but rate-limits per REQUEST, and the read path is naturally bursty: a valuation pass loads its marks 24 anchors at a time (each able to fetch through), and the long-lived server serves concurrent JIT reads. Unpaced, those fire near-simultaneous executions, trip the per-minute cap, and every 429 degrades a mark to an older bar. The gate makes that rare without changing an outcome — the fetch-once rule, the resume skip and the degradation ladder are exactly as above. It is per PROCESS: the 6h refresher awaits its valuation scope by scope, and the backfill drain's workers are separate child processes, each given a divided share of the budget before it is spawned (see External dependencies). At most DUNE_MAX_CONCURRENT (default 2) executions are in flight, with starts at least DUNE_EXECUTION_SPACING_MS (default 1000) apart, so a burst QUEUES instead of 429ing. The gate spends nothing — it only delays a start — and a call the credit guard has already refused never occupies a slot. Result reads are billed too. The guard accounts execution_cost_credits; Dune's pricing FAQ says the datapoints returned by every /results request also accrue against the quota. That axis is invisible to the guard, which is one more reason a window execution asks for the token list it actually needs.

• Append-only. A stored bar is FROZEN (ON CONFLICT DO NOTHING), so a Dune history restatement cannot move it — a stable audit trail. Migration 055 makes this structural by REVOKEing UPDATE, DELETE from the app role(s) (the schema-wide arwd default privilege had re-granted them despite 054 withholding UPDATE); the postgres owner keeps them for a deliberate operator purge. The spike gate below is the one exception, and it is scoped by the grant system rather than by convention: 098 adds a COLUMN-level GRANT UPDATE (rejected, reject_reason), so the gate can record its verdict while price_usd, bar_ts, source, volume_usd, dune_source and carried stay unwritable by the app role.

• Bulk load. scripts/backfill-token-price-bars.ts — the span from the history floor in ONE execution (allowLongWindow, ~16 credits for a year; --halves splits into two if it pages awkwardly), idempotent + resumable, hard-aborts a chunk that returns zero rows for all tokens or a credit-guard stop. The default START_ISO is the 2026-01-01 floor (it used to be one year back, which on any 2026 date reaches below it), an explicit START_ISO older than the floor is CLAMPED to it with a line saying so, and --floor=<ISO> moves the floor for a deliberate operator run. The resume yardstick is the trap when a token is ADDED to the registry: by default it counts on-the-hour WETH bars, which is the right proxy for "did this whole-universe load already run" and the wrong one for "does THIS token have bars" — the mirror already holds a year of WETH bars, so the chunk reads ≥90% complete, the operator sees already loaded … skip and exit 0, and the new token gets nothing (invisible for the 30 days MIN_HISTORY_DAYS keeps its panel hidden). So a registry addition must be loaded with --tokens=0x…,0x…, which judges the resume on the minimum hourly count across the NAMED tokens; --force skips the check entirely. Routed tokens (dexRatioSources()) are EXCLUDED from the default set — their query has a different cost model and their history is loaded by their own repair path, so dragging them along buys bars that ON CONFLICT DO NOTHING discards — and must be named explicitly to be loaded here. An aggregate-fed token is REFUSED by name with a [fail] line and exit 1: it rides no Dune CSV at all, so naming one would fetch nothing and read as a query bug. [fail]-tagged failure lines + a non-zero exit page the cron alert.

• Per-token routing (the dex-ratio seam). The aggregated-exchange channel behind prices.hour is not uniformly precise: for some assets it quotes to the whole cent. Measured on Dune (ethereum, 2026-08-05, 24 hourly bars): sUSDe had ONE distinct price all day (1.24), 24/24 exactly cent-aligned, while USDe, USDC, USDT and DAI each had 24 distinct prices and none cent-aligned. A cent is ~0.8% of sUSDe's price, and because its redemption value accrues daily while the quote sits still, market / redemption − 1 drifts down at the yield rate and snaps back on each cent tick — so the stored series was a quantisation sawtooth, and the published "largest 1-yr drawdown" for sUSDe was reading off it. A rounded quote is therefore worse than a missing one here. dexRatioSources() in dune.ts routes such an asset to its own saved query (sUSDe → DUNE_QUERY_ID_SUSDE), which prices it from actual DEX trades: per-trade ratio against a clean counter asset (USDe/USDC/USDT/DAI), hourly volume-weighted, times that counter's own full-precision prices.hour quote at the same hour. The two factors are MULTIPLIED, never divided, so this is not a cross-vendor ratio.

• Routing exclusivity is enforced at the writer, not by convention. Bars are frozen on write (ON CONFLICT DO NOTHING), so a single bar from the coarse channel landing in a routed asset's window is PERMANENT: readBarSeries is source-blind and newest-first, so the published basis would pair on the artifact, and the re-source repair would then refuse to re-run. Both writers therefore partition, and the shared choke point enforces it:

  • syncBars splits its token list, so a routed token is never in the standard CSV, and every window path inherits that (the 6h sync, the bulk load, fetch-through).
  • upsertBars REFUSES a bar for a routed token that does not carry that route's own source tag, counted and logged. That is what makes the invariant survive a future third writer, rather than depending on every writer remembering.

• Complete hours only. The routed query aggregates whole hours, and both production callers pass unaligned upper bounds (the refresher syncs [MAX(bar_ts), now]; fetch-through syncs ts ± 1h). A tick at 00:07 would otherwise freeze a seven-minute VWAP as hour 00:00's canonical bar, on exactly the instants the 6h token_basis grid pairs against, and append-only means the complete-hour value could never replace it. So the routed leg snaps its window inward to whole hours (completeHourWindow) AND the saved query only emits an hour that closed inside the window. A window containing no complete hour is skipped rather than approximated.

• The routed query's guards are shaped by its consumer, which is an EXTREMUM. The headline off this series is MIN(basis) over 365 days, so a single hour can move it in both directions: one bad hour plants a permanent fake trough, and one BLANKED hour during a real dislocation hides a true one (the reader walks back up to 48h, serving a pre-crash price). Hence:

  • Liquidity floor in TOKEN UNITS (≥ 8,000 sUSDe), not USD. A USD floor is computed from the very price being measured, so a 40% depeg cuts measured notional by 40% at identical turnover and the gate TIGHTENS exactly during the dislocation the panel exists to show — understating risk. A token-quantity floor is price-invariant. 8,000 sUSDe is ~$10k at the 2026-08 level and drops the same 2 of 24 hours on the sample day.
  • Venue allowlist (uniswap, curve, fluid, origin_arm, supernova). Measured: those five carried every sUSDe trade against a clean counter across all 24 sampled hours, so the list costs zero coverage while blocking the cheap attack — spin up a shallow pool on an obscure venue, dominate one quiet hour, plant the trough. A per-hour pool-count rule cannot do this job: 8 of those 24 hours had only ONE pool trading. The trade-off is that liquidity migrating to an unlisted venue makes the query go quiet there, which is the safe failure but a silent one — revisit the list when the venue mix changes.
  • Taker floor (≥ 2 distinct takers), plus the dust guard and the [0.5x, 2x] median band. Its purpose is narrow — remove the single-address wash case, where one party is both sides of every trade in a quiet hour — and 2 achieves that. An earlier cut used 3, which sat exactly on the minimum of a single sampled day: a threshold with zero headroom starts blanking real hours the first time the market is quieter than the sample, for no extra protection, since anyone who can fund two addresses can fund three. The venue allowlist is what does the real anti-manipulation work. Measured coverage with all guards: 564 of July 2026's 744 hours (76%); the walk-back covers the rest. The SQL is vendored at scripts/dune/susde-dex-ratio.sql so these guards are reviewable in a PR — Dune is still the source of truth, and the pairing is stated in both files.

• Routed cost is FLAT in the window, which changes the guard. Measured (small engine, 2026-08-10): 1 calendar month = 1.252 credits, 6 hours = 1.907 — the shorter window cost MORE. dex.trades is partitioned on (blockchain, project, block_month) and a block_time predicate does not prune inside a partition, so the bill is "one month partition scanned" whatever slice is asked for. That inverts the hourly query's economics, where a ±1h fetch-through is a ~0.25-credit rounding error: here every read miss that reaches Dune is a ~2-credit execution, and read misses come in floods (one prod drain logged 8,747 fetch-throughs in a single run).

• The cooldown compares against the ROUTED TOKEN'S OWN newest bar. This is the subtle part, and getting it wrong makes the dial a no-op. The 6h refresher syncs [MAX(bar_ts), now] where MAX is the table-wide high-water mark — a number every other token advances on every tick — so anything derived from the caller's from says nothing about whether this token is fresh. So the routed leg reads the token's own newest bar at-or-before the window end, skips (with a log) while that is inside minResyncSeconds, and when it does fire, starts the window at that own bar rather than at the caller's from: the ticks it skipped moved the table-wide watermark past hours this token never received, and bars are append-only, so a window starting there would leave them empty permanently. allowLongWindow callers (the bulk load, the re-source repair) bypass the cooldown entirely, and the 48h catch-up clamp is re-applied to the widened window so a long outage still goes through the bulk tool, not the cron.

• What the dial actually buys. The cron ticks every 6h and the leg fires on a tick only once the cooldown has elapsed, so settings quantise to multiples of that tick. DUNE_DEX_RATIO_MIN_RESYNC_SECONDS, default 12h:

  cadence	fires	~executions/month	credits/month	bar age at a tick	extra basis drift
  6h	every tick	~120	~230	0-1h	~0.06bps
  12h (default)	every 2nd	~60	~115	0-7h	~0.4bps
  24h	every 4th	~30	~57	0-19h	~1.1bps

  Be honest about the log line: BAR_STALE_SOFT is 6h, so ANY cadence above 6h means the ticks that skip serve a bar older than the soft threshold and getBarSeriesAt emits its one-line staleness notice for sUSDe — roughly half the ticks at 12h, three in four at 24h. That notice exists to reveal a lagging mirror, so choosing >6h is also choosing to see it routinely; only 6h keeps ops logs quiet. The mark itself stays coherent at any age (same-bar pairing is algebraic), and sUSDe accrues ~0.057bps/hour, so even the 24h case is ~1.1bps against the ~80bps artifact this route removes. Note the routed leg is bought on every qualifying tick whether or not anyone looks at sUSDe — it is a standing feed, not demand-driven.

• Tracked set. mirrorTrackedTokens() in dune.ts is the raw Dune sync set, and it is one column: every registry row with feed = 'dune_tape' (#810 Y3). The bitcoin reference rides the saved query unconditionally. A row whose mark is COMPOSED, DERIVED or IDENTITY declares no tape of its own and buys no slot — stETH/eETH consume wstETH/weETH bars, and every fund share consumes its underlying's — so empty raw slots are not requested. The composed set is a projection of valuation = 'composed', and the reason each row is in it is DERIVED from the route that row declares rather than stored: no_market where there is no secondary market to read (every curator and multi-strategy vault share, and every wrapper whose exit is a queue, a credential or a bounded buffer), instant_pinned where the route is uncapped in both directions and so holds the traded price at the rate. The reason decides what a reader is told, not how the row is valued: NO composed row consults its own bar, because the composition function takes the branch the row declares rather than filling holes the mirror left. No composed row declares a feed (migration 111). sUSDf and sGHO were the last two that did, from the days when a feed's quality was graded and the grade was the route back out of composition; the weekly DEX measurement is that route now and it reads trades and pool reserves rather than our own bars, so the series buys nothing and a declared feed nobody marks off is a dark-feed page waiting to happen. Both still change hands, so both keep liquidity = 'secondary': the class records what the asset IS. USD3 has its feed back and is market-priced again (migration 111): its own mint and redeem are shut while a Curve pool trades it at its rate, which is the exact shape the category test calls market-priced. Dune's hourly tape still prints nothing for it, so its bars arrive from the vendor fill until that changes. marketMarkTrackedTokens() is the wider logical coverage set used by mirror-coverage.test.ts: every market-class base must have an honest mark path, while explicitly composed wrappers are exempt from having their own bar.

• Spike gate: accept, then retract. Dune occasionally prints a value that is not a price. Measured on our own mirror: AUSD printed $17.6M for one hour, USR $0.116, earnETH $35,467, FDUSD wandered between $0.93 and $1.30, USDf hit $1.11. Each of those is a single hour that snapped straight back, and each one is permanently frozen into a series a chart reads an extremum off. The gate is deliberately NOT a write-time filter. The newest bar is written and served immediately, because a real depeg has to show at once — the live tip reads the newest bar for an idle asset, and a gate that held a print back would delay the one signal that cannot wait. On the NEXT sync, every bar that has since gained a neighbour on both sides is judged: it is rejected only when it departs from BOTH neighbours by more than 5% AND those two neighbours agree with each other to within 1%. Every clause earns its place. Departing from both is what makes it a round trip rather than a move. The neighbours agreeing is what distinguishes a bad print from a real dislocation: a price that moves and stays moved leaves its neighbours disagreeing, so the test cannot fire on it. There is no band around par and no notion of what the asset is worth: the gate knows only the shape of the series. The newest bar is never judged, and neither is the first bar of a token's history, because neither has two neighbours. A rejected bar KEEPS its row and its price, with rejected = true and a reason. Every read that prices skips it (readBar, readBarSeries, getBarSeriesAt, the shadow rebuild, the routed-feed cooldown), so a walk-back lands on the last ACCEPTED bar; the resume gauges deliberately still count it, because append-only means its grid slot can never be refilled and treating it as missing would re-buy a row every execution discards. The gate logs [spike-gate] with the token, the hour, the price and BOTH neighbours, and is idempotent: a bar it has already judged is excluded from the next pass. The judge window is bounded by the token's cursor ([cursor − 48h, now]), so a long-idle token does not rescan its whole history — except once, on the pass after its floor load, which is what catches a spike inside backfilled history. Scope: the standing sync set, which is 28 tokens today. The gate judges the tokens the six-hourly sync tracks (mirrorTrackedTokens()), so a token whose bars only ever arrived through fetch-through, and one that is composed or derived rather than tape-priced, is not judged. Of the bad prints on record that means AUSD is covered while earnETH, USR, FDUSD and USDf are not, because they are outside today's standing set. That set grows to every idle asset once the token registry drives it (#810 R4/Y1, piece B of this program): the standing sync becomes feed IS NOT NULL AND status = 'active', and every row it admits inherits the gate, the cursor, the auto-backfill and the dark-feed alert with no further change here. What the uniform 5% / 1% rule costs on a volatile feed. The thresholds are deliberately asset-blind and there is no per-asset band, which is right for the stablecoin shapes the rule was calibrated on. Eight of the standing feeds are ETH- or BTC-class (WETH, WBTC, cbBTC, wstETH, weETH, rETH, ezETH, osETH), and on those a GENUINE one-hour wick larger than 5% that fully reverts in the next hour is retracted by design: the mark series loses that hour and the total-return line reads through the drawdown as if it had not happened, with no user-visible signal, because the walk-back lands on the previous accepted bar. That is a known trade, not an oversight. The recourse is the audit trail: the bar keeps its row and its price, and the [spike-gate] line names the token, the hour, the price and both neighbours, so a retraction over an ETH- or BTC-class token can be inspected and, if it was a real print, taken up as a threshold question rather than discovered from a chart. Restatement is not automatic. A retraction changes what a mark should have been at that hour, and the existing re-mark library scopes by wallet and by accounting asset over a token's WHOLE stored history, with no timestamp scope and no flow-side entrypoint — so "this asset at this hour" is not a scope it can express. Each rejection therefore also logs one [remark-needed] line naming the asset and the hour, and the restatement is the operator's (a full re-derive, per the program's rules). The line is a claim about stored rows, so it is CHECKED before it is made: it is printed only for a retracted hour that some stored snapshot or flow row could have read, asked of every asset the bar prices and of the whole ETH book for WETH. The hole fill's vendor gate asks the same question through the same code (bar-consumers.ts); see Filling the tape's holes for the rule.

• Dark-feed alert. Any token in the standing set with no accepted bar newer than 12h makes the six-hourly refresher exit non-zero behind ONE aggregate [fail] line, printed LAST and naming the worst offenders with their ages. Before this the only signal was a log line at 6h that nobody was reading, which is how PST stayed dark for 101h in July 2026 and was found two months later. The line is shaped for the alert path: it has to carry [fail] in brackets AND the run has to exit non-zero (either alone is silent), and run-cron.sh keeps the last 8 matching lines cut at 220 characters, so the count and the worst feeds lead and the tail is what gets dropped. It changes nothing else the job does: every basis row the tick computed is written before the check runs.

• Per-hour volume and Dune's own source. The two saved queries emit volume_usd and dune_source alongside the price, and the sync stores both (rows are parsed by column name, so a query without them is not an error — the columns are simply stored NULL). They exist to answer one question exactly: was this hour a fresh print or a carried one? Dune forward-fills a price for up to 7 days, and the fill copies the WHOLE row. Measured 2026-09-07: PST printed price 1.1298310950021484 with volume 146527.69276955843 for six consecutive hours, dune_source dex.trades; and coinpaprika-sourced rows (BTC, USDC, the majors) carry volume_usd NULL in every hour whatever traded. So "null or zero volume means carried" is wrong in both directions. The rule that holds for both is the exact repeat: a bar is carried when its (price_usd, volume_usd) pair equals the immediately preceding hour's, with two NULLs counting as equal (which reduces the coinpaprika case to an exact price repeat). The flag is derived once at write time, because the comparison needs the neighbour and that is where the sync already is; only on-the-hour bars are compared, since a bar off the hourly grid has no preceding hour to repeat. The flag is what lets a reader tell a FORWARD FILL from a market that did not move: a stretch of carried bars is the vendor repeating itself, and a stretch of fresh prints that happen to be equal is a quiet market. It is computed over the HOURLY grid only: carried is defined against the preceding hour, so a bar off that grid would carry a false flag by construction. The grid is a DB CHECK, so the filter guards the flag's own arithmetic rather than a live series. The gate's report line names which instrument it used (… over 18h carried vs … over 31h estimated), because "measured" and "inferred" are different claims about the same feed. One caveat on carried: for a source that never reports a volume (coinpaprika — USDC, USDT, DAI, WBTC, cbBTC) the exact test degrades to "the price repeated", which is the same statistic the estimate computes, so the extra precision only exists where Dune reports a volume.

• Read-only consumers (no credits). GET /api/portfolio/prices serves the signed-in Portfolio's ETH ≈ $X display annotation from the newest WETH (chain 1) bar. It served a btc key off the chain-0 reference until the BTC book was retired in August 2026; the chain-0 bars are still mirrored, but token_basis is their only consumer now. It is hit on every signed-in page load, so it calls readBar ONLY — the pure, indexed LIMIT 1 DB read with no deps and no network path. It must never import a batched entry point (getBar, getBarsAt, getBarSeriesAt, or loadMirrorMarks / priceInBookFromMirror / loadMarketContext above them), all of which fetch through to Dune on a miss and would put page traffic on the credit budget. It therefore does not count against the ~20/day fetch-through alert below. A miss is served as null (M9); a bar past BAR_STALE_SOFT logs a warning (M20).

• Budget guard. The sync is 1 execution/6h regardless of traffic; a per-process credit guard (DUNE_MAX_CREDITS_PER_RUN, default 50) stops issuing once a run reaches it (returns partial), re-checked after the gate so a call that queued while a sibling spent the budget does not issue past the ceiling (a guard stop can still overshoot by the executions already in flight — the guard refuses STARTS, and a sibling that started still bills). In a multi-process cron (the backfill drain) the ceiling is divided across the child processes before they are spawned, since it is per-process. Alert if fetch-throughs exceed ~20/day (a coverage gap — fix the tracked set). Keyless (DUNE_API_KEY / the relevant DUNE_QUERY_ID_* unset) degrades to a logged skip + DB-only reads.

• Credits / ToS. ~30-60/month steady for the standard queries, plus ~115/month for sUSDe's routed dex-ratio leg at the default 12h cadence (flat-cost partition scans, see above), vs the ~2,500/month quota. The credit model retains query results in our DB, so confirm the Dune API terms permit that (one-time check) — see External dependencies.

The aggregate feed: DefiLlama, for an asset Dune does not cover

A fourth price source, added 2026-09-17 (#907, migration 109): feed = 'llama_aggregate', bars stamped llama:coins.

The problem it solves is COVERAGE, not precision. The dex-ratio route exists because the aggregated tape quotes some assets too coarsely. This one exists because Dune's tape can simply not cover an asset that is trading perfectly well somewhere else. Measured 2026-09-16 over prices.hour on the 30 days to that date: BTC.b printed 231 of 720 hours and its last print was 2026-08-29 — a tape that STOPPED — while Uniswap v4 alone took 242 trades worth $12.3M in the same window, the last of them 2026-09-17. USDai printed 81 of 720, all of them after 2026-09-12: a tape that had only just started. Neither gap was the market.

The bar is DefiLlama's own aggregate observation nearest the hour, stamped AT the hour:

price_usd(h) = the Coins API observation within 30 minutes of h, at confidence >= 0.9

Every published bar goes through exactly the accept/reject rule the rest of the tree uses (judgeQuote + vintageGate in src/lib/data/llama-prices.ts): the same confidence floor, the same vintage arithmetic, the same per-address log line naming what was refused and why. A refused hour is a MISSING bar, never a substituted one (M9).

Why batchHistorical and not /chart. Both return the same hourly prices. /chart states ONE confidence for the whole series and no per-point timestamp, so a single degraded hour could not be refused without refusing the series; batchHistorical carries both per point. It also answers about MANY moments for MANY addresses in one call — a week of hours for two tokens is 4.1 KB of URL and about 0.2s — which is what makes a floor-to-now backfill 38 requests per token rather than 6,200.

The returned moment is not the moment asked about, so a point is matched to the NEAREST requested hour and then judged against it. The bound is half an hour in both directions, for the reason the 3h vintage bound is half a 6h window: a quote accepted for one hour is by construction nearer to that hour than to either neighbour, so a bar can never be marked with its successor's price. The vendor's own searchWidth is sent AND the returned timestamp is checked — the width narrows what the API looks at, the timestamp it hands back is the only thing that proves what it found.

The observation runs about eight minutes early, consistently. Measured over 2 x 168 hours on 2026-09-17 with the exact call the leg makes (batchHistorical, searchWidth 1800): median offset 500s on both tokens, p90 510s, worst 800s (BTC.b) and 560s (USDai), minimum confidence 0.99. The sign barely varies — all 168 of USDai's points and 119 of BTC.b's sit about 500s BEFORE the hour they are stamped at, and BTC.b's other 49 within 140s after it — so this is a systematic bias, the vendor's hourly grid sitting a few hundred seconds behind ours, rather than jitter around the hour. It changes nothing functionally: the bar is stamped at the hour, the half-hour bound still holds and no bar can take a neighbour's price. What it sets is how much room the bound has — the worst observation clears 1800s 2.25x over. That is real headroom and it is the figure to quote; the one to re-measure when the window is questioned is the bias, not the spread.

It is for IDLE and no-base rows only. An aggregate blends venues, which is fine for a LEVEL and too noisy to carry a BASIS line — the finding docs/plans/dune-price-mirror-plan.md records and the reason the mirror was built on Dune's tape. A variable_rate row with a base publishes market / redemption - 1 as a series, and an aggregate's venue-mix wander would land in it as a basis nothing in the market did. So the registry admits it for a par row, mirror-coverage.test.ts asserts that over the rows rather than trusting the note, and the two rows that declare it are both idle.

Exclusivity, enforced at the writer, exactly as for the dex-ratio route. An aggregate token is partitioned out of every Dune CSV (partitionBySource), so the hourly sync, the bulk load and the read-miss fetch-through all drop it by construction, and upsertBars refuses a bar for one of them offered under any other source. Bars are frozen on write, so one tape bar landing in an hour this feed owns would win for ever.

The leg and its history. syncLlamaAggregateBars (src/lib/data/llama-aggregate-sync.ts) runs inside the six-hourly token-basis refresher after the Dune legs, best-effort in the same way and for the same reason (a vendor having a bad ten minutes must not cost the basis rows the tick would otherwise write). Its window is per token, opening at that token's own newest bar — or at the 2026-01-01 mirror floor when the token's history does not reach it, which is the auto-backfill on add for this feed and needs no separate tool. The floor test is the token's EARLIEST bar rather than "has it got one at all", and BTC.b is why: it holds a handful of bars from 2026-08-05 bought by a read-through months before it had a feed, and a planner that only looked at the newest bar would open in August and mark the row with seven months missing. An hour that already holds a bar is never re-fetched (ON CONFLICT DO NOTHING would discard it anyway), and hours inside the window that hold a bar from ANOTHER source are counted and named — they keep the old feed for ever, so a run must not report a clean load over a series it did not actually replace.

Live prices at the tip

The hourly tape is HISTORY. Its newest bar is routinely hours old, which is right for a series and wrong for the number a holder is looking at, so a page load and the Synchronize button value every market-priced asset from a current vendor quote instead (R6). A redemption-priced asset never fetches a quote at all: it composes off the level its chain reaches, exactly as it composes off a bar.

The vendor is CoinGecko, with DefiLlama behind it, and neither is an aggregator ROUTER. That distinction is what makes a live tip admissible at all: a router quotes the cheapest way to obtain a token, and where a token's own mint and redeem are atomic that route IS the redemption rate — so a routed quote reports the redemption rate back as a traded price, which is the one route a market test has to exclude. Both vendors report observed trades instead.

A level is served only when something corroborates it, because the failure that matters is not noise but a quote that is wrong and steady (the AUSD plateau: a vendor printing a plausible frozen level for weeks). Each level is judged against a reference from somewhere else entirely:

the asset	its reference
a yield-bearing token with a book	its redemption value — the composition's own second line, read on chain at the same block the caller values at
a USD idle claim	one dollar
ETH, WETH, the bitcoin wrappers, the volatile spot rows (AAVE, LINK, UNI, CRV, BAL, SNX, ENS, LDO, RPL, 1INCH, EURC, XAUt) and the no-base rows (eBTC, apyUSD, sUSDat)	its own last stored bar — they accrue against nothing this product accounts in, so there is no unit for a redemption value to be quoted in

Inside 3% of that reference the level is served. Outside it, the second vendor decides, and the outcomes are different events:

• the two vendors agree with each other within 1% — the move is real (a depeg has to draw), and the primary is served;
• the backup is itself inside the band — the primary is the outlier, and the backup is served;
• the two vendors are further apart than the band — they are not describing the same asset, nobody knows what it is worth, the level is withheld (M9: a dash, never a quiet fall-back to a settled bar), and the six-hourly job puts it on the alert line;
• they are further apart than the 1% that would have corroborated the move and closer than the band — that is not a contradiction, so the level is refused under its own name (uncorroborated) and the stored bar stands with its vintage stated. The 1% is the bar for AGREEMENT, never the definition of disagreement, and the difference decides the volatile rows in the table above: their reference is a stored bar that can be hours old, so on a moving day both vendors are far outside the band while differing by the little their venue mixes differ by. Treating that as a contradiction would dash a dozen ordinary holdings, drop them out of the tracked total and page the six-hourly job every tick through any volatile session;
• only ONE vendor answered at all and it is outside the band — that is an outage rather than a disagreement, and it is uncorroborated in the same way. Collapsing this into the withholding would dash every asset more than 3% from its newest bar for as long as one vendor is down — a spent monthly plan makes every CoinGecko answer absent — and page every six hours about a disagreement that never happened.

A reference that could not be READ is a weaker reference, not the absence of one. A redemption value is a chain read and can fail transiently, and dropping the reference on that failure would lift the asset out of the band for the tick — a vendor level at any distance from its own history, served unchecked, which is the plateau case with the guard switched off. The row's own last bar stands in instead, the level records which reference it was actually judged against, and the six-hourly job counts the degradation on its [live-price] line, because a rate reader that has quietly stopped working looks from every other line exactly like one that agreed.

And where there is no reference of any kind, nothing is convicted. A row with no redemption value, no par and no stored history has no band for a vendor to be OUTSIDE of, so R6's withholding — "both outside AND disagree" — cannot apply to it. A lone quote is served and said to be unchecked; two quotes that differ leave the stored answer standing under the same uncorroborated name. Withholding there would delete the asset's dollar level, render the holding as a dash, drop it out of the tracked total and page every six hours until two vendors happened to converge about a row nobody can referee.

ETH is judged first, and the ordering is load-bearing: an ETH-book wrapper's reference is its redemption value in ether carried into dollars, so building it from a stale bar would put every wrapper out of band on the numeraire's own move rather than on anything about the wrapper.

In the ETH book the vendor gives a LEVEL and the book needs a RATIO, and the two behave differently. ETH's own move dominates both quotes and cancels out of the ratio only when both saw it, so a wrapper's price in ether is accepted only from ONE vendor with the two observation stamps within 120 seconds of each other; otherwise the last stored bar stands for that tick, which is a coherent ratio of a known vintage rather than a fresh incoherent one. The dollar LEVEL is taken either way — the basis is the slow number and the level is the fast one.

Nothing live is ever stored. The served maps and the stored-row maps sit side by side in one MarketContext (priceInBook / priceInBookMirror, their per-share twins for the share-accounted rows, and usd / usdMirror for the "Other" holdings, which have no book unit and are valued straight off a dollar level), the snapshot row carries both values, and the total-return series values every point through the mirror map — a series with two methods in it draws the spread between them as return and reverses it at the next settled point (M23). Live prices are likewise never written into token_price_bars.

The bar stamp travels with the price, and only where the price went. A derived base divides its wrapper's book price by the wrapper's rate at that price's own moment (M5.1), so an asset that took the live level is stamped at the vendor's own observation time and an asset whose ratio the pair guard refused keeps its bar's stamp. Stamping the tick on both would divide a price struck up to 48h ago by a rate read this second; stamping the tick on the live half would pair a quote the vendor took fifteen minutes ago with a share rate read now, which is the same misalignment in miniature. The stamp is the SERVING vendor's own, backup included: that vendor's gate accepts an observation up to six hours old, and it is the vendor the band selects on exactly the day the difference costs something, so the tick stands only where a vendor states no observation time at all.

What the annotation beside a figure may CLAIM is the verdict, not the fact that a vendor answered. GET /api/portfolio/prices serves the ETH level with two facts attached and the disclosure prints both: the moment the price is of (the stamp above, never the moment of the page load, since a level dated "now" would put the freshest label on the stalest number the band can select), and what checked it. The four outcomes are four sentences, because they are four different claims and a reader can verify none of them from the screen: a second vendor corroborated it; the asset's own most recent stored price agreed with it, which is all the ordinary in-band verdict consults (the band returns before the backup's number is read at all); nothing could check it, which is the whole meaning of the unchecked verdict; or no level could be served and the stored price stands. A level whose vendor states no observation time is served as the stored bar instead, because the bar's hour is a fact and the clock is not.

A dash means "nobody knows", and only that. The live tier composes twice — once over the stored bars for the mirror maps, once over the overlaid levels for the served ones — and the second pass is contained, so one asset's failure cannot cost the others their live levels. A composed, derived or fund row carries no bar of its own, so an asset whose composition THREW there would be left holding a null that the page would render as a dash and drop out of the tracked total, on nothing more than a transient archive read. Those assets fall back to the answer the mirror pass computed a moment earlier from the same bars. A WITHHELD level is the opposite reading of the same null and keeps its dash, up the whole chain it composes into: serving the bar there would present a settled number as the live one.

With one exception, which is the ETH book's unit of account. A withheld level deletes the asset's dollar entry, and for WETH that entry is not one row's price: every composed and derived ETH-book row is carried into dollars through it. So a withheld ETH tip leaves usd[WETH] on its stored bar, for the same reason ETH's own withheld level still lets the wrappers be judged against a bar rather than against nothing — one refused level must not become a whole book with no dollar value. ETH's own tip is still a dash where it is shown: the annotation under the headline reads the verdict, not the map.

Budget. The Demo plan is about 10k calls a month, so the live batch is cached in process for five minutes — the same window as PORTFOLIO_LIVE_COOLDOWN_MS, so a signed-in page refreshing as often as it is allowed to costs one vendor call — every request is batched (all addresses in one call, all listing ids in a second), and a 429 opens a cool-off rather than being asked again by the next page load. A row the vendor does not list by its Ethereum contract carries a listing id on its registry row instead; BTC.b is the only one today.

Filling the tape's holes, and checking it against two vendors

The fourth writer of token_price_bars (R7), and the only one whose job is what the other three did not manage. Dune's ingestion is neither instantaneous nor complete: an hour can print late, print wrong, or never print at all. Each of those used to leave a permanent gap, because bars are append-only and the app roles cannot delete one, so an hour nobody wrote in time was an hour nobody could ever write. Reads covered it by walking back 48h, which is fine for a mark and useless for a series — a week's chart with twenty holes in it is a chart of when Dune was healthy.

• Six hours of grace, then a vendor. An hour older than BAR_STALE_SOFT with no accepted bar is bought from CoinGecko's hourly history and, where CoinGecko has none, from DefiLlama's. Every filled bar carries its own source. The grace period is what keeps the leg from racing the tape for an hour that is simply on its way: a vendor bar written into it would win permanently.
• A filled bar is held to the same standard as a tape bar. By the time a hole is filled BOTH vendors' points for that hour are already in hand — the free one was fetched for the cross-check below, the metered one for the fill — and the bar about to be written is permanent. So it is written only where the second vendor is inside the same 3% band the tape's own bars are judged by. Two vendors that contradict each other about an hour leave it a HOLE, which is what it already was and which every read walks back through, and the pass names the hour and both numbers on a [vendor-split] line. One vendor answering alone is not a contradiction and does fill the hour: CoinGecko has no listing for Ethereum PST, so most of that row's history is DefiLlama's word alone (R9), and refusing it would leave the row with no history at all. A persistent split at the tip needs no alert of its own — the hours stay holes, and the dark-feed line pages on its own 12-hour clock.
• And the spike gate judges a candidate BEFORE it is written. Where only one vendor has a point for an hour there is no second opinion to hold it to, so the check that is left is the one every bar on this tape passes: a candidate that departs from BOTH its neighbouring hours by more than 5% while those neighbours agree with each other to within 1% is not written, and the pass names it on a [spike-gate] line. It is judged now rather than on the next tick because a written bar cannot be taken back: the gate's own remedy after the fact is a retraction, which leaves the wrong price in every mark already taken off it. A level change disagrees with one neighbour and agrees with the other, so a real repricing is written. The neighbours it weighs are bars the pass stands behind, which is what keeps it from convicting the one price it exists to admit: where the tape is wrong and steady, the two neighbours agree BY DEFINITION at a level the vendors have just contradicted. So a bar the pass has RETRACTED this tick is out of the series (it is out of every read from now on, and leaving it in also lets a real spike through by putting a price nobody serves between the candidate and its true neighbours), and a bar the pass DOUBTS but could not convict is still served yet is not evidence to refuse anybody on. That second half is the case that never recovers: where the metered vendor does not list the row at all (Ethereum PST), nothing can convict the frozen bars, so the hour would be refused on identical evidence every tick until it ages out of the 48-hour window, after which no vendor can fill it at all. What is given up is narrow: where every neighbouring bar is under doubt, a lone vendor's spike is written rather than refused, and a wrong bar is retractable on any later tick while an hour that can never be filled is not.
• The window is a fixed 48-hour lookback, not the sync cursor. The cursor holds the newest ACCEPTED bar — which a hole does not move but every bar after it does — so a window opening there is only as wide as the tape's own ingestion lag while a tick advances six hours, and five hours in six would fall between two windows. Re-asking is self-limiting: an hour either gets a bar (and is then skipped by the stored-hours read) or ages out of the window after eight ticks.
• The tape is cross-checked, because a wrong bar is worse than a missing one. Every accepted Dune bar in the window is compared with the free vendor's point for the same hour. More than 3% apart only raises the question; it takes a SECOND vendor agreeing with the first, while both sit outside the band around the tape, to answer it. Then the bar is retracted with reject_reason vendor-disagreement, exactly as the spike gate retracts one, and every read skips it.
• A retracted hour is not refilled, and that is a property of the table rather than a choice: the row still occupies its key, ON CONFLICT DO NOTHING discards anything offered for it, and 055 revoked DELETE from the app roles. What the retraction buys is that the hour behaves as a HOLE rather than as a wrong price, which the 48h walk-back covers. Making such an hour genuinely refillable is an operator's DELETE, which is what the one-off re-judge does as the table owner (below).
• And a retraction says what it does NOT fix — when there is something it did not fix. Setting rejected changes every future read; it does not touch the snapshot and flow marks already written from that bar. So the leg prints a second [remark-needed] line naming the asset and the hour, word for word the spike gate's, because there is no scoped re-mark tool ("this asset at this hour" is not a scope remark-lib can express) and the restatement is the operator's full re-derive of that asset. That line is a claim about stored rows, and the leg now checks it before making it. For each retracted hour it asks whether any stored snapshot or flow mark falls between that bar and the next bar that was accepted when the marks were taken — the span in which a mark really would have read it, since a read takes the newest accepted bar at or before its moment. Nothing in that span, nothing said. The question is asked of every asset the bar PRICES, never of the retracted token alone. A composed, derived or fund-share row stores its own address as the accounting asset and a price taken from a SOURCE asset's bar — a stETH row is priced off wstETH's bar, a fund share off its deposit asset's, a PT off its payout asset's — so scoping the question to rows whose accounting asset IS the retracted token would go silent for every one of them. The set comes from the registry (marketConsumersOf, the inverse of the walk the valuation itself takes) plus the PTs that pay out into one of those assets, which live in pendle_markets rather than in a registry row. Two reads per token that retracted something and none at all on a tick that retracted nothing, so the six-hourly path pays for it only when a bar is actually convicted; a read that fails reports every retraction unchecked rather than going quiet. The snapshot side of the question is asked over a span one 6h window wider at the bottom, because the tick marks at the anchor block it read and stores the row at its aligned grid point: a late tick is stored EARLIER than the bar it consumed, and the flow side needs no such slack (a flow row carries the block its own mark was taken at). For the ETH book's numeraire the question is asked of the whole book rather than of WETH's own rows, because an ETH-book mark divides by WETH's bar whatever the asset is — an edge no registry column records. Without the check the line fired on every retraction, including on a box holding no marks at all — which is the state prod is in between a release that wipes wallets and the first sign-in. The spike gate asks the identical question through the identical code (bar-consumers.ts, shared by both gates), so the two lines mean the same thing.
• A filled bar never pairs with another vendor's. The same-bar rule exists so that the numeraire's own move cancels out of a ratio; what cancels is what the two quotes SHARE, so two vendors' levels at one hour leave their difference in the ratio instead. With the tape no longer having a single writer, pairAtNewestCommonBar therefore pairs only bars whose source names the same VENDOR (Dune's hourly tape and its dex-ratio query are one vendor; a CoinGecko fill and a Dune bar are not) and otherwise walks back to an hour one vendor wrote both legs of. A filled hour is still a real hour for every USD-book mark, the dollar series and the chart; what it cannot do is quietly become a few basis points of wrapper basis. No same-vendor common bar in the window is the same answer a hole was before the fill existed: the token fails rather than mixing two vendors.
• Which rows it touches. Every row a Dune leg writes: the tape rows and the ROUTED one (sUSDe's dex-ratio query), for both the fill and the cross-check. The aggregate rows are the one exclusion, from both halves: DefiLlama is already their standing writer, and comparing DefiLlama against DefiLlama says nothing.
• The routed row is filled as a BACKUP, not as a second writer. It was left out at first, and that left sUSDe with nothing behind its query: when the Dune plan lapsed on 2026-09-24 (00:00-08:00 UTC) its history simply stopped. It is backed up now, on three rules:
  • Only on a tick its own query FAILED. The six-hour grace keeps the fill from racing the tape because the tape is asked on every tick; the routed query is not. It re-executes only once its own newest bar is a cooldown old (twelve hours by default), because each execution costs a whole month partition of dex.trades, so an hour past the grace may simply not have been asked yet, and a vendor bar written there would win for ever over the route's own print. Age cannot tell "stalled" from "not due", and inferring whether the Dune legs ran is what breaks on the ticks that matter (a guard, a missing key or query id, a state read that fails before the legs). So the Dune client REPORTS what each routed query did on the tick (routeOutcomes: printed, failed, or not asked at all), and only a failed opens the backup (routedBackupWindow). A hard failure on the standard leg no longer stops the routed legs from being asked, so a Dune-wide outage does reach them and does open it.
  • Only after the route's own newest bar. Below it, an empty hour is one the route asked about and found no trade in: an answer, which every read covers by walking back to the last trade. Those quiet hours (about one in six over 2026, more in a quiet week) stay as they are. Filling them would put a vendor level into roughly one grid point in six of sUSDe's 6h basis line, and the vendors sit a median 2.2-2.7bp from the route's own DEX-traded bars (measured over ten days of September, p90 5-6bp) while the route's own 6h move is about 3bp: a step the size of the market's, for hourly continuity the 6h line does not use.
  • One writer per hour, and the route's freshness is its own. The writer's routing guard (admittedBarSources in dune.ts) admits coingecko:hourly and llama:chart for a routed row by name, beside the route's own dune:dex-ratio tag and nothing else, so the standard tape still cannot land there. And the routed leg's re-sync cooldown reads the ROUTE's newest bar, not the newest bar of any writer: counted as freshness, a backup bar would sit inside the cooldown on every tick of an outage and the route would never be asked again. When Dune comes back the route re-asks from its own newest bar (clamped to the 48-hour window like every leg); the hours the backup wrote keep their bars (ON CONFLICT DO NOTHING), and an outage longer than 48 hours leaves its older hours vendor-carried for good, because the per-token sync cursor counts the backup's bars and so never sends the row down the catch-up leg. Re-sourcing those hours is the re-source repair's job.
• What it costs, and the ceiling on it. One keyless DefiLlama call per token per tick, covering the whole window, because both halves need the same hours: most of those hours already hold a bar and are asked about as a bonus (the cross-check uses an answer where there is one and needs none), so the vendor client's own per-row miss line is turned off here and the pass reports ONE count instead. An hour that nobody could price is what matters, and it is already counted as dark. CoinGecko is asked only when there is something for it to answer — a hole to fill, or a suspect bar to corroborate — so a tick over a healthy tape spends nothing of the metered budget at all. Two rules bound the unhealthy tape, where every row has holes: a row the vendor has no entry for is never asked (COINGECKO_UNLISTED, PST today), and one pass spends at most 30 metered calls, rotating across the token list so the rows past the ceiling lead the next tick. A LISTED row past the ceiling does not fill at all this tick: its gecko map is empty because the call was unaffordable rather than because CoinGecko had nothing, and filling from the free vendor alone would stamp a permanent, uncorroborated bar in exactly the tape-wide outage where a thin row's vendor series is most likely to be frozen. The hour stays fillable for 42 more hours (seven ticks) and the rotation buys the second opinion within a tick or two, so the ceiling costs latency rather than coverage. An UNLISTED row is the opposite case and keeps filling from DefiLlama alone: it has one vendor by nature, not by affordability, so deferring it would defer it for ever. A deferred row can reach the dark-feed check below unfilled; the line it prints stays true as written — nothing has priced that hour yet — and it clears on the next tick, when the rotation fills the row's whole window. A multi-day tape outage announcing itself once per row is the better trade. See CoinGecko for the arithmetic.
• The dark-feed alert means more than it did. A standing feed with no accepted bar in 12 hours still prints the same [fail] dark feed line, but the fill has by then offered every hour in that token's window to both vendors — so the line now means "nobody can price this", not "Dune was late". For the routed row the sources tried are its own saved query, then CoinGecko, then DefiLlama. With the shipped 12-hour cooldown the route falls due on the same tick its newest bar reaches the 12-hour dark age, and the fill runs before the check, so the backup lands first; a cooldown raised past 12 hours (DUNE_DEX_RATIO_MIN_RESYNC_SECONDS) would leave the row dark between the two, which is the coupling DARK_FEED_MAX_AGE_SEC already warns about. The per-token provenance (how many hours each vendor answered, how many nobody could) is on the fill's own summary line rather than folded into a line that is cut at 220 characters, where every character of provenance is a character of NAMES the operator loses.
• A TAPE that has stopped is named separately, because the dark-feed line can no longer see it. The fill writes bars up to six hours behind wall clock and the dark-feed alert fires at twelve, so a token whose Dune query has stopped covering it entirely always holds an accepted bar under six hours old and can never reach that line again. The vendors carry the row, the chart stays continuous, and the asset's basis line quietly becomes a vendor level against a redemption rate rather than a traded price against one. So a row with no accepted DUNE bar in 24 hours (four ticks) while some other writer's bar does sit in that span is named on a [partial] line, built to the alert's own 220-character cut so the count and the remedy lead and the names are the tail: check the saved query's token coverage. [partial] and not [fail], deliberately — the alert grep matches it, so it travels with any run that fails, and it never pages on its own, because R9 accepts that some rows are vendor-carried for most of their history and a line that pages every six hours about an accepted state is how an alert channel stops being read. It is a LOG line, and who reads it is worth being exact about: the cron's alert quotes matching lines only on a run that already exited non-zero, so on a healthy tick this one is in the six-hourly job's log and nowhere else. The condition also does not clear by itself — this leg keeps a vendor bar inside every six-hour span, so a tape that has stopped for good leaves the row named on every tick — and telling a row that is vendor-carried by nature from a Dune query that has silently stopped covering one is a registry fact the release does not carry. The standing alerts for this leg stay the dark-feed line (nobody can price this row at all) and the price-disagreement line, both of which do end the run non-zero.
• And the live check runs on a schedule. The page withholds a level the vendors and the redemption rate cannot agree on, and a withheld level is a dash on a screen: nobody is paged. So the six-hourly job runs the same call on the same set through the same band and turns a withholding into a [fail] price disagreement line plus a non-zero exit within six hours. An asset merely served from the BACKUP vendor is logged and does not page — that is the band working, not an outage — and neither does an UNCORROBORATED one, where the stored bar is being served and the operational signal is the client's own 429 line. The whole set falling to the backup IS a page, and it is the one failure here that is otherwise perfect: a spent monthly plan refuses every primary call, every market-priced asset is served from DefiLlama, nothing on a screen is wrong, and what has stopped is the corroboration that makes two vendors worth having. Zero of about sixty rows served by the primary with the backup serving at least one prints [fail] the primary vendor answered for NONE of N asset(s) and ends the run non-zero, every tick it is true. The same line also counts the assets whose PRESCRIBED reference could not be read and were judged against their own last bar instead, which is the one signal that a redemption reader has started failing. A check that cannot RUN is itself a failure: the head-block read, the registry walk and the stored-bar read can all throw, and a box that lost this check to a refusing RPC would be exactly as blind as it was before the check existed, so a throw here prints [fail] and ends the run non-zero. It is safe to fail loudly because the check runs after every basis row is committed. Two properties make it a PROBE rather than a second reader: it asks the vendor for an uncached reading (a level repeated out of the five-minute live cache cannot have moved, so a check against one cannot notice a vendor that has started disagreeing), and it reads the last-bar references from stored bars only, with no fetch-through — a permanently dark row has no covering bar to find, and a read that fetched through would buy a Dune execution for it on every tick for ever.

Re-judging the stored history, once

The six-hourly leg above judges a bar as it arrives and fills an hour that goes missing, but only inside its own 48-hour window, and it has only run since v0.68.0 (2026-09-23). Every bar accepted before that was never judged, so known bad stretches were still stored and served: AUSD at $17.7M-$36.8M for 181 hours of September, apyUSD at $0.85-$1.09 against about $1.36 for three weeks, and a handful of LST and BTC-wrapper stretches 6-17% off both vendors. Any wallet whose 30-day window reaches one of those hours marks off it. scripts/repair/rejudge-price-bars.ts applies the leg's own rules to the stored history, once, as a gated operator step (runbook).

• The same rules, called rather than restated. For every active token with a standing feed, over [--from, --to] (default 2026-01-01 to now): every stored ACCEPTED Dune bar is judged with judgeDuneBar (retracted where both vendors agree with each other within 1% and both sit outside the 3% band around it), and every hour inside the token's series with no accepted bar is filled with decideHoleFill and spikedCandidates (CoinGecko's hourly point, else DefiLlama's, each under its own source, never where the two contradict each other, never a lone vendor's spike). The routed row is JUDGED and never filled: its gaps are hours its query answered with no trade, which the leg leaves to the walk-back too, and an outage of its query is the leg's to back up on the tick it happens. The aggregate rows are neither judged nor filled, as on the leg. A retraction carries reject_reasonvendor-disagreement-rejudge: …, so it is told apart from the leg's.
• The newest 48 hours are left to the leg. That span is the leg's own window, judged and filled on every tick in the order that keeps it from racing the tape (the Dune legs first, then the fill, on one clock); a repair run with the cron drained has no such ordering.
• An hour before a token's first bar is not a hole. The fill starts where the token's series starts: backfilling a token's pre-history from the vendors would be a coverage decision, not a repair.
• A retracted hour IS refilled, which the leg cannot do (the retracted row still holds its key and ON CONFLICT DO NOTHING discards anything offered for it). The repair deletes the retracted row inside the same transaction that writes the vendor's bar, so it runs as the table OWNER (055 revoked DELETE from the app roles) and writes every row it deletes to a JSON-lines backup first; a backup that cannot be written rolls the token back. Bars the spike gate or the leg retracted earlier are refilled the same way.
• One transaction per token, and it refuses a row that moved. A retraction names the hour, the source and the exact stored price; a deletion only ever takes a rejected row; the inserts must land one for one. Anything else (the cron was not drained, a second run, a hand edit) rolls that token back and the run exits non-zero. A token that throws for any other reason is reported and skipped the same way; the run carries on with the rest.
• A vendor that could not be ASKED defers the hour. The leg re-asks every tick, so a failed request costs it one tick; a repair over nine months that read a failed request as "no price" would fill hundreds of hours from one vendor. A span either vendor could not be asked about is left exactly as it is, counted, and the run exits non-zero so it is re-run.
• What follows the bars, and why it can always be finished. The token_basis rows those hours priced (every 6h grid point up to and including 48 hours after a changed hour, and every ETH-book token's when the hour is WETH's) are recomputed through the refresher's own write path over a database-only reader, so no Dune execution is bought; a grid point nothing can price any more loses its stale row. Then the wallets whose stored rows could have read a changed hour (asked of every asset the bar prices, as the gates ask it) are put through the re-derivation trigger's own gates (rederiveWouldLaunch: the arm block, its population, ledger rows or a deferral marker, certification to its own floor, the tip, the replay hold) and only the ones it would launch are queued, by removing their derive cursor; a cursor removed for a wallet the trigger will not come back for would leave that wallet's ledger withheld from its page, so the others keep theirs and are named with the manual remedy. Both steps take their scope from a change log, one JSON line per token written just before its transaction commits (a failure to write it rolls the token back, so the log can over-state a write, never miss one), not from the run's own plan: a token already repaired plans nothing, so a re-run after a basis error, a killed process or a failed wallet step would otherwise report itself clean with the follow-up undone. Each step appends a watermark once it completes without an error, so a re-run carries each step over exactly the writes it still owes: the basis recompute redoes what failed, and a wallet the trigger has already re-derived is not queued again. The log lives where nothing can move it: /var/tmp/rejudge-price-bars/rejudge-changes-<db>.jsonl, beside the deleted-row backup rejudge-deleted-<db>.jsonl, named by the database and by no date, variable or flag, so a re-run in a new shell or on another day finds the same log (/var/tmp because the owner the execution runs as can write it; the runbook copies both files to /root afterwards, since /var/tmp is swept after 30 days). The stored SNAPSHOT marks are the other half and are not restated by the re-derivation; the report prints the exact remark-snapshot-marks.ts --assets= scope.
• Cost. No Dune credits. DefiLlama (keyless) is asked about every hour of every judged token's series, 336 hours a request, about 19 requests a token for the nine months to September. CoinGecko (the metered plan) is asked only over spans holding a suspect bar or a gap, at most one request per 88 days of such span: the plan answers a range of 90 days or more at DAILY granularity (measured), so the spans stay under it. --vendor-cache=<dir> keeps every answer on disk: an execution run over the reviewed dry run's cache plans from the same answers, writes exactly what the dry run printed, and spends no second round of requests (--execute refuses to run without one). A cached CoinGecko answer serves any span inside what was cached, since a re-run after a partial write asks for fewer, differently cut spans.

Changing an asset's pricing CATEGORY: the stored history has to move with it

What the page shows no longer steps. Both money columns are computed at read with today's method (why), so the served history re-values every computed reading and movement on the new method at the next read, with no join (a row served stored for want of a rate fact, the loader rule, keeps the method it was stored with). What follows is now owed to the STORED columns, which the revert path and such rows serve and the parity census compares against.

portfolio_position_snapshots.value_market and the ledger rows' entered basis are stored columns. So the moment a registry edit moves an asset between the two pricing categories — off its own bar onto a composed mark, or back — every row written before the deploy carries one method and every row after carries the other, inside one series. Nothing reconciles them, and a NAV or total-return chart draws the join as a step at the deploy timestamp that a reader takes for return. For an asset whose own bar was NULL (no Dune row, M9 unpriced) the step has the other shape: the series gains value out of nowhere.

This is the outcome the sUSDe re-source repair calls worse than a gap, for the same reason, so a category change carries the same obligation: re-mark the stored rows, or state the discontinuity and take it deliberately.

WHEN THIS RUNS. A category is a measured question now, not an asserted one: the six-hourly job computes each token's candidate and PROPOSES a flip after fourteen consecutive days of disagreement, and applying it is a registry edit in a pull request (R4, Pricing categories). This section is what that pull request owes the history once it lands. It is not needed where there is no history to move — a release that deletes every tracked wallet leaves nothing to re-mark.

The tooling already exists for the snapshot spine and needs no new code path. remark-snapshot-marks.ts re-derives value_market through standingMirrorMarks → applyUnitPrices, which is the SAME composition function the live writers run (unit-prices.ts), so a re-run lands the stored rows on the new method by construction. And on the same RATE: the repair reads a share rate the way the cron does — the block-pinned getter first, the stored six-hourly series behind it, and never the series for a listed fund (R3), because a fund's whole value is its NAV per share and a rate six hours from the leg's block is a different number rather than an approximate one. The bar half of the repair stays a pure DB read and buys no Dune credits; the rate half costs one archive eth_call per fund per anchor, cached per asset and block. --assets= is what scopes the repair to the accounting assets that actually moved. The ledger's own rows have no equivalent script: re-deriving the affected wallets is what moves them, and the re-derivation trigger is the path for it (see The re-derivation trigger) — so a category flip owes a re-derivation of every wallet holding a moved asset, and the discontinuity stands in the ledger until it runs.

# 0. THE SCOPE IS THE ADDRESSES THE REGISTRY EDIT MOVED, comma-separated, read off the
#    pull request that flipped them — never a standing list, which goes stale against the
#    next flip. Anything that is not a 0x address is a hard error rather than an empty,
#    "clean" run.
ASSETS=0x<moved-asset>,0x<moved-asset>
# 1. DRAIN THE CRON so nothing writes a row on the old method behind the repair.
crontab -l > /tmp/cron.bak && crontab -r          # or comment the refresh lines
# 2. DRY-RUN first, always. It prints per-asset counts, sum |Δ| and the 10 largest rows.
/opt/onchain-credit/scripts/run-cron.sh repair/remark-snapshot-marks.ts --assets=$ASSETS
# 3. Read the magnitude. If it is not what the flip predicts, STOP.
/opt/onchain-credit/scripts/run-cron.sh repair/remark-snapshot-marks.ts --assets=$ASSETS --execute
# 4. Restore the cron.
crontab /tmp/cron.bak
# 5. Re-derive every wallet holding a moved asset — the ledger half, which this script
#    does not touch (see the re-derivation trigger above).

• The repair rewrites stored history, which is why it is dry-run by default, runs under the portfolio writer advisory lock, and is scoped. It never touches value_redemption (block-precise, and unchanged by a market-mark decision) and never deletes a row.
• Idempotent: re-deriving an already-repaired row yields the same value, so a second run reports repaired=0 | unchanged=N. That is also the check that it worked.
• Scope it, do not run it bare. A bare run re-derives EVERY asset's whole history, which is the mirror migration's job, not a targeted fix — and it makes the diff unreadable at exactly the moment you need to read it.
• A row the repair cannot mark is a SKIP, never a zero. An unavailable mark (no bar, no common bar at the row's timestamp) leaves the row untouched and counted, which is the 2026-06-25 lesson enforced structurally: a partial repair that writes zeroes manufactures a drawdown that never happened.

Re-sourcing an asset's market feed (the sUSDe repair)

A one-time, one-asset repair: replace sUSDe's cent-rounded bars over the past year with the trade-derived feed, and re-derive its token_basis history over the same window so the repaired past and the go-forward series are one method rather than two stitched at a boundary. Everything about it is scoped to sUSDe; no other token's bars or basis rows are read or written.

Why it needs a migration at all. token_price_bars is append-only by design — every write is ON CONFLICT DO NOTHING, and migration 055 REVOKEs UPDATE, DELETE from the app role so the app cannot mutate a bar even by mistake. That invariant is exactly what makes a purge necessary: with the old bars in place, every replacement bar is silently discarded, the sync reports rows fetched and zero inserted, and the series looks re-sourced while still being the old one. There is no in-place update path, and the DELETE has to run as the postgres owner, which is what migrate.sh does.

Order, and it is not optional:

# 0. PRECONDITIONS, both mandatory.
#  a. The backfill DRAIN must be idle for the whole window. Between the purge and the
#     repair sUSDe has NO bars, so every historical sUSDe read a drain makes becomes a
#     routed fetch-through at ~2 credits until DUNE_MAX_CREDITS_PER_RUN stops it — after
#     which the remaining marks come back null. Comment the minutely
#     `drain-portfolio-backfills.ts` line out of the crontab for the window (same shape as
#     the `touch /root/.reseed-paused` pause the 067/072 runbooks take), and confirm the
#     queue is actually empty first:
#       psql> SELECT status, count(*) FROM onchain_credit.portfolio_backfill_state GROUP BY 1;
#     Nothing may be `queued` or `running`.
#  b. Check what --allow-destructive would let through, because it is not scoped to 075:
#       psql> SELECT filename FROM onchain_credit.schema_migrations ORDER BY filename DESC LIMIT 5;
#     058 / 067 / 072 are also DESTRUCTIVE. If any is still pending on this DB, apply 075
#     BY HAND instead of running the whole ledger with the flag:
#       sudo -u postgres psql -v ON_ERROR_STOP=1 -X -q -1 -d <db> \
#         -f scripts/sql/075-susde-bar-resource-purge.sql
#       sudo -u postgres psql -q -d <db> -c "INSERT INTO onchain_credit.schema_migrations(filename) \
#         VALUES ('075-susde-bar-resource-purge.sql')"
# 1. purge the artifact bars (destructive, so it needs the flag)
scripts/ops/migrate.sh "$DATABASE_URL" --allow-destructive     # applies 075
# 2. refill + re-derive (needs DUNE_API_KEY and DUNE_QUERY_ID_SUSDE)
/opt/onchain-credit/scripts/run-cron.sh repair/resource-susde-bars.ts --dry-run
/opt/onchain-credit/scripts/run-cron.sh repair/resource-susde-bars.ts
# 3. restore the drain crontab line

• Between the two steps sUSDe has no bars in the window: the 6h refresher counts it failed (an honest miss, never a fabricated mark) and its Market Depth chart shows no history. Run them back to back. The repair REFUSES to start while any sUSDe bar at or after the boundary still carries a non-dune:dex-ratio source, so the two steps cannot be applied out of order.
• The purge boundary (2025-08-01) and the refill's start are the same instant, asserted by a unit test that parses the migration's own DELETE predicate. Bars before it survive: they are outside every published window, and deleting history nothing refills would be data loss rather than a repair.
• Resumable + idempotent. One Dune execution per calendar month (~1.25 credits each, ~15 for the full 12 months), skipping a month that already holds re-sourced bars. The resume floor is 0.5 (not the bulk loader's 0.9) because the dex-ratio query legitimately emits only ~79% of a month's hours. Re-running rewrites the same rows.
• The replay reads DB-only. refreshTokenBasisForSnapshot normally fetches through to Dune on a bar miss; over ~1,500 historical grid points that would buy up to 1,500 executions to re-price history already in hand. The repair injects a reader backed by a single prefetch instead — same shape, same newest-common-bar pairing, no network.
• A grid point the replacement cannot price has its OLD row DELETED, not left. The stale row is an artifact of the feed being removed, so keeping it would make the series a mixture of two methods; a gap is honest, a stale wrong number is not. A grid point that ERRORED (a statement timeout, a dropped connection) is a different thing and is left ALONE: that is a fact about our infrastructure, not about the market, and deleting on it would remove published history on a transient blip. Those are counted separately and the run exits non-zero, because the window is then part-repaired.
• The prefetch is source-filtered. It reaches 48h BEFORE the boundary so the first grid points have their full walk-back, but 075 only deletes at-or-after the boundary — so those 48h still hold surviving cent-rounded bars. Unfiltered, a grid point in the first two days with no dex-ratio bar in its window would be priced off one, planting an artifact trough exactly at the boundary. Only the asset being repaired is filtered; a numeraire leg is priced by the standard channel and is not what is being replaced.
• Verified against the fixture DB before release: on a seeded reproduction (8,977 bars, 7 distinct prices, a fabricated -0.42% worst drawdown) the repair produced 1,486 rows over 1,245 distinct market prices with zero cent-aligned quotes, cleared 13 grid points inside a >48h feed gap, and left the pre-boundary sUSDe rows and a control token's rows byte-identical.

Pricing categories: the capacity series, the market measurement, the verdict

Every tracked asset is valued either at what it trades for or at the rate it redeems into, and which one it is is a question about the route a holder actually has. The rule and its thresholds are in metrics; this is the machinery that measures it and what each piece refuses to do.

The registry row DECLARES the route (mint_terms, redeem_terms, capacity_reader, terms_verified, migration 111). A declaration is half an answer: "this vault redeems instantly" says nothing about whether it can redeem $5M today, which is what the test actually turns on. So the other half is measured, on two cadences, into the append-only token_pricing_measurements.

The capacity series (every 6 hours)

scripts/refreshers/token-pricing.ts, a leg of refresh-assets.ts, reads every row that declares an atomic route at one block pinned for the whole pass. Pinning it once is not tidiness: the two directions of a single route have to describe the same moment, or a redemption emptied at block N is compared with a mint cap read at block N+30.

Three readers, one per capacity_reader kind, each asking the question the protocol itself asks:

Kind	Mint bound	Redeem bound
erc4626	maxDeposit(probe) — the vault's own statement of what it will take in one call	the vault's own balance of its asset(), which is what a withdrawal is served from before it unwinds anything
sky_savings	none: a DSR-style module mints against the protocol	none, and the module's total assets ride along as its SIZE rather than as a bound
rocket_pool	the deposit pool's free space, with the pool resolved through RocketStorage	rETH's getTotalCollateral(), the figure burn itself checks

maxWithdraw is deliberately not used. It is PER OWNER: asked about a probe it answers that probe's balance (zero), and asked about a whale it answers about the whale. Neither is a statement about the route.

The ERC-4626 redeem reading is a FLOOR and says so in its stored detail. A vault that can pull back from its own strategies pays more than its idle balance, and nothing here knows how much more. Understating is the safe direction for a reading whose effect is to pin a price to a rate: it keeps the asset market-priced, which is the answer a chart can contradict. A vault holding NOTHING idle reads zero — a real measurement, and the honest one (srUSDe's buffer was empty at block 26,100,239).

Three answers, and they are not interchangeable. A number; an explicit "no bound", stored with no figure because inventing a ceiling would put a made-up number into the series every verdict is computed from; or NOTHING AT ALL when the read failed. A zero for the third would move a category on a timed-out request.

And the leg says WHY it read nothing, because that decides whether anybody is woken. This leg runs with a zero partial-failure floor, so one failed row exits the whole six-hourly cron non-zero — which is right for asset() answering a token the registry row does not declare (the vault has been re-pointed and every figure below it would have been priced with the wrong bar) or for a route declared uncapped that now answers a real cap. It is wrong for a pinned archive call that never came back, which is the most flake-prone read in the job and refuses BOTH directions of a route on its own. So the seam retries such a call once — and only a call that reached nobody: a node that ANSWERED, with a revert or an execution error, will answer the same way again — and a row still unread after that is LOGGED rather than failed for as long as its stored reading is inside the freshness bound. Two things end that tolerance, and both mean the verdict has actually lost something: nothing fresh left for that route, or a pass in which NOTHING could be read at all, which is the archive endpoint being down rather than a hiccup.

USD conversion goes through the underlying's newest STORED BAR — for ether, WETH's — so a capacity and a position valued in the same tick agree about what a dollar of the underlying is, and the series stays reproducible from the database alone. No bar, no reading — for a BOUNDED leg, whose whole content is the number. An UNCAPPED leg keeps its answer: a savings module's redeem leg carries its total assets as information while its answer is "there is no bound", and that was known before any dollar was involved. Dropping it would turn "no bound" into "unmeasured" and block the row's verdict on every tick while a newly tracked underlying's bars are still backfilling.

The market measurement (weekly)

scripts/refresh-token-market.ts (Monday 05:00) answers the other limb with three facts from three sources: trading days over 30, the median daily volume over those same 30 days, and the deepest single pool holding the token against an asset of its own book or a recognised counter.

• Trading days, and the on-chain median volume beside them. ONE Dune execution covers every token (DUNE_QUERY_ID_DEX_MARKET, vendored at scripts/dune/dex-market-measure.sql, handed the recognised dollar counters so a trade the vendor could not price is valued on its counter leg rather than thrown away).
• The median daily volume the bar is judged on: one CoinGecko market-chart call per token over the same window, resolved by listing id or contract exactly as the live price resolves it. The vendor's total_volumes series is a trailing 24-hour figure stamped hourly, so it folds to one value a day — the last point inside each UTC day — and the median of those days is stored under its own kind. A 404 is an ANSWER (the vendor carries no such coin; sGHO is the live case) and that row's bar reads the on-chain median instead, for good; a row the registry already records as unlisted is never asked at all. A 429, a timeout or an unrecognised body stores nothing and leaves the newest stored reading standing.
• Pool depth, a reserves read per token, paced to the endpoint in use — 30 a minute against CoinGecko's /onchain mirror when COINGECKO_API_KEY is set, four a minute against the keyless GeckoTerminal endpoint when it is not, because that is what the keyless door actually allows (it publishes 30 and delivers about five, measured against the live vendor). A 429 is waited out and retried twice before a token is given up for the week, and a key the vendor REFUSES (a 401 or a 403, not a 404, which is about the token) falls the rest of the pass back to the keyless door rather than costing the week's readings.

The volume call is paced through the same counter as the pool reads, because with a key the two endpoints are one vendor and one meter: 23 rows are about 46 requests at the door's own interval, which is inside the allowance, where pacing them separately would put two requests in every interval and spend the pass being throttled.

A pool's own share token is never one of its sides. Balancer's stable pools list their own share token as a member and the vendor reports the pool's balance of it as liquidity — ETHx reads a $25.8M "pool" that way, whose base token IS the pool's address. That is the pool's unminted supply rather than depth against anything, so the pool is skipped: not counted, and not recorded as a pool the counter rule excluded either, since the counter rule is not what left it out. The reading carries what it dropped (skipped, deepestSkipped) so the vendor's headline figure stays visible. Two tests decide it: the side's address being the pool's own (a composable stable pool's share token IS the pool), and the symbol naming a Balancer pool token (-BPT, the bb-a- family, a slash inside a ticker).

The deepest pool is WALKED, because the vendor will not sort by it. A page carries twenty pools and the only orderings on offer are volume ones — sort=reserve_in_usd_desc is refused with a 400 naming the three it accepts — so page 1 is the twenty busiest pools, not the twenty deepest: read live on 2026-09-22, sUSDe's page 1 ends on a $558 pool while a $443k pool sits on page 2. So the pass reads a second and a third page, and only when it has to: a pool that already clears the $1M bar ends the walk (nothing further down can unsee it), and so does a short page (the vendor's list has run out). A token whose list outlasts three pages stores what it counted and records that the list did not run out, which is what stops the verdict reading a floor as a superlative — see the withheld pool bar below.

Never an aggregator quote, and the exclusion is about ROUTES rather than about venues. An aggregator routes through a token's native mint and redeem wherever that route is atomic — exactly the route this limb has to exclude — so it would report a deep, tight market for an asset with no secondary market at all, and every route-pinned wrapper would read as traded. Traded volume and pool reserves are the opposite kind of evidence: a trade happened, and a balance is held. Where the trade happened does not change that, which is why exchange volume counts — while the DAY COUNT stays on-chain, because a reported figure cannot say which days had trades in them.

Weekly rather than six-hourly because a 30-day count moves by at most one day a day. A token the Dune execution did not answer for stores NOTHING rather than a zero: the query returns a row per requested token including the ones that never traded, so a missing row means the execution did not cover it. The same discipline holds on the pool half: a well-formed list of pools holding none that qualifies is a MEASURED zero, while a body that is not a pool list at all (a renamed envelope, an error object served with a 200) is no reading — stored as a zero it would read as "nobody trades this" for every token in the pass.

The measured set is exactly the set the test applies to. A fund share is redemption-priced by definition; an idle claim, a NO-BASE token and a DERIVED row are market-priced by rule. The last two are not merely saved requests. The pool bar names an asset of the token's own book or a recognised counter of that book, and both limbs are measured relative to a book a no-base token does not have, so it cannot clear the bar at any depth; and a derived row (stETH, eETH) is priced through its wrapper's bar and has no valuation of its own for the test to move, so a proposal to make one redemption-priced asks for something the pipeline cannot carry out. Measuring either would store a permanent "no market" for an asset that may trade perfectly well, and then propose flipping it against a ruling that fixes its category.

A LIMB IS THE UNIT OF FAILURE, not the row. A row counts as measured on any ONE of its readings, so losing a whole limb moves neither the ok nor the failed count: an unset query id, a thrown Dune execution, or a pool pass throttled end to end each leave every category verdict null for the week behind a run that exits zero. Each dark limb gets its own [fail] line AND a non-zero exit, which are the two halves the cron alert needs.

A PARTIAL volume outage gets its own line, because its symptom is a silence. A row with no reported reading is not left unmeasured: it is judged on the on-chain median instead, which is a real change of answer for a token that trades on exchanges and which no row count would show. The client opens a 60-second cool-off on a 429 and answers empty without asking again, so one throttled call can cost most of a paced pass this way — hence a [fail]-prefixed count of the readings that did not arrive, beside the pool half's own rate-limit line. A row the vendor does not LIST is not a missing reading and draws no such line.

The volume limb is dark when the price vendor answered for NOBODY — a spent plan, a refused key, a vendor-wide outage — and a 404 is not nothing, so a pass where every row asked came back unlisted is healthy. It is worth a [fail] of its own although no row fails and no verdict breaks: every bar silently falls back to the on-chain median, which counts DEX trades alone and understates a token that trades on exchanges, in the direction that pins a price to the issuer's rate. A category machine quietly changing its mind is exactly the failure this job's limb rule exists for.

The verdict (every 6 hours, and it proposes)

The same six-hourly leg recomputes each row's category CANDIDATE and its loopable flag from the newest readings that are still FRESH (src/lib/data/pricing-verdict.ts, pure and table-tested). loopable is written back to the registry row when it is KNOWN and left alone when it is not; the candidate is never written anywhere.

The volume bar reads the REPORTED median where there is a fresh one, and the on-chain median otherwise — which is the whole of the fallback for a token the price vendor does not list, and also what a week with no reported reading degrades to. Freshness is applied BEFORE the source is chosen, so the fallback is self-healing rather than sticky: a reported reading past its bound is simply absent and the on-chain one answers. The verdict's reason line names which of the two it read, because the fallback can only tighten the bar.

The pool bar itself is WITHHELD when the reading could not establish it. The bar is a statement about the DEEPEST pool, and a walk that ran out of pages with pools still unseen has a floor under the depth rather than a maximum. A number at or over $1M is an answer whatever else went unread — a pool that deep was seen — and under the bar it is an answer only when the vendor's list ran out inside the walk. Otherwise the whole market limb reads UNMEASURED for that row: no candidate, no loopable, and the reason line says the list outran the pages rather than leaving an operator to read "market unmeasured" as a job that has stopped. Storing a false there would write "no real market" — and loopable = false with it — for a token whose deepest pool may be one page down: a fact about our page budget written as a fact about the token.

loopable IS A PLAIN BOOLEAN (R5): the token clears the market bar, or $5M mints and redeems atomically both ways, or the flag is false. The only nulls left are rows the test never measures — a fund share, an idle claim, a no-base token, a derived row — and rows whose readings have not arrived, which the job leaves alone rather than writing.

Where the COUNTER rule left a deep pool out, the PROPOSAL says so. The pool bar counts only pools held against an asset of the token's own book or a recognised counter, so a deep pool against a governance token, an exotic wrapper or another yield-bearing share is excluded however large it is. When the pool bar is the ONLY one of the three that failed, the excluded pool is at least ten times the counted one and would itself have cleared the bar, the reason line names it: "no real market" resting on a pool nobody counted and "no real market" resting on a pool that is not there are different statements, and the person deciding the category has to tell them apart. That first condition is what keeps the note honest — a row that also fails on trading days or on volume is failing about the MARKET, counting the excluded pool would not have changed the limb's answer, and naming it would send a reader to the one remedy that changes nothing. A pool reading that ALREADY clears its bar excludes nothing that matters either: sUSDe holds $3,729,080 against sDAI, which clears on its own, beside $62,534,776 against DOLA that the counter rule does not recognise.

A candidate that has disagreed with the row's DECLARED category for fourteen consecutive days is a PROPOSAL. It is reported on a [fail]-prefixed log line — grep-visible, so a run failing for some other reason quotes it — and the run still exits zero. A category flip restates how an asset is valued; that is a decision to schedule, not an outage to wake someone for, and applying it is a registry edit in a pull request.

A proposal posts itself to Telegram, on the shared alert bot, the way the unknown-asset and discovery-drift alerts do. The cron wrapper's own alert fires only on a non-zero exit, so a [fail] line on a healthy run reaches the logfile and nobody: an exit code cannot say "the run was fine" and "there is a decision waiting" at once. Only the TRANSITION is sent — the first tick at which the fortnight completes — so a disagreement left in place deliberately costs one message rather than four a day until somebody edits a row. The log line is printed on every tick regardless.

The hysteresis is RE-DERIVED from the series rather than stored, which costs no table and no second source of truth. A reading counts for a day only while it is fresh, and fresh is per kind: two days for the six-hourly capacity readings, seventeen for the three DEX facts the weekly job writes. One bound for both would leave the market limb stale on five days of every seven and break every run that depends on it — the failure would be invisible, because the capacity series is long enough that nothing would report a shortfall either. The seventeen is arithmetic rather than a round number: the weekly cadence, times the runs the limb may miss (one), plus three days of margin for a run that is merely late. A gap that outlasts a kind's own bound BREAKS the run rather than being interpolated across, so one skipped Monday is survived and two consecutive ones restart the fortnight. Sizing it to a single cadence looked stricter and was worse: one missed run — an unset query id, a thrown execution, a spent Dune budget, an open rate-limit breaker — broke the run on the eleventh day, withdrew any standing proposal, pushed the decision it carried out by another fortnight, and paged the alert below every day until the following Monday fixed it unaided.

Two shortfalls are counted apart, because they mean different things: rows that disagree but have under fourteen days of series (the ordinary state of a young series), and rows that disagree today but whose window has a hole in it — a measurement leg that has stopped writing. Each is one count per run, not a line per token.

The same freshness rule bounds the day's verdict, not just the walk. The store answers with the newest row per kind whatever its age, so without the bound loopable would be written and re-affirmed every six hours off a market measurement months old, taken by a weekly job that had stopped running — and nothing would have reported it, because a stale reading that AGREES with the declared category never reaches the proposal path at all.

And this leg watches the weekly one, because nothing else can. The market measurement is a separate crontab line on a separate clock: a job that never runs cannot fail, and a crontab line nobody added has no log to grep. So the six-hourly leg counts the rows with no fresh market reading and puts them on its summary, and when NO row in scope has one it prints a [fail] line and posts to Telegram once a day while the condition stands. It waits until its own capacity series reaches back further than a market reading may be stale, because the first fortnight after a release is a young series rather than an outage.

Because a reading stands through one skipped Monday, nothing fresh left anywhere already means two runs in a row are gone, and the message says so in RUNS rather than in days — which is what lets it ask for the crontab line and the environment variables to be checked. A single miss has ordinary causes that fix themselves the following week, and paging for one would send a reader to a configuration that was never wrong.

loopable_verified is written beside the flag on every tick the verdict is known, even when the answer has not changed. The flag itself can never be retracted here — an unmeasured tick has nothing to say and must not publish a false — so once the readings behind it age out the last answer simply stands, and without a date a flag judged in June by a job that has since stopped reads exactly like one judged this morning.

Environment

Variable	Used by	Unset
DUNE_QUERY_ID_DEX_MARKET	the weekly market measurement	a [fail] line and a non-zero exit: the day count never arrives, so every verdict stays null, which is why an unset id is a failed run rather than a skip
COINGECKO_API_KEY	the weekly measurement's pool-depth read, its reported-volume read, and the live prices	the keyless endpoints, paced at four a minute instead of thirty. The pass still completes, and the pool half is ONE REQUEST PER PAGE rather than per token, plus one volume call per measured row on the same paced stream: 23 rows all answered on page 1 is 46 requests, about twelve minutes, and the three-page ceiling is 92, about twenty-three. Keyed the same pass is about two minutes. A key that is SET and refused (a 401 or a 403: expired, mistyped, wrong tier) falls the pool half back to the keyless door for the rest of the run, loudly, so a bad key costs a slow week rather than a lost one; the volume half answers nothing at all on a refused key and its limb says so
ETHEREUM_ARCHIVE_RPC_URL	the capacity readers' pinned block	the public dRPC fallback

Table conventions (agent-grade)

New tables follow the conventions piloted by the lending-positions migration (scripts/sql/026-lending-positions.sql): canonical keys (chain_id + lower-cased addresses; symbols are display attributes), block-anchored rows (block_number/block_at_snapshot alongside snapshot_ts), current-state tables (*_current) separated from snapshot history, and an explicit basis column stating the methodology of derived rows. DDL for every table lives in scripts/sql/001..043-*.sql. See Database & schema for the full table list and reader-side details.

Related docs

• Metrics: how & why — the annualizeRatio APY convention and per-metric formulas.
• Database & schema — table-by-table schema and the reader functions.
• Architecture — overall system shape.
• Operational processes — how the refreshed series feed the carry screener and the Fluid vault registry.