Replaying positions that closed inside the backfill window

Written because the v0.32.x validation wave's P5a carried an explicit feasibility gate and the work was over it; this page is the design and the cost estimate the gate asked for, kept for the reasoning. What shipped is neither Shape A nor Shape B below: the discovery pass reads the groups out of the event ledger the 100k-scale programme had since built, which answers Shape B's question with four indexed SQL statements instead of a whole-universe archive sweep, so the cost that stopped the work no longer exists. See Data pipeline → Historical group discovery from the ledger (D2), and the deep-history plan for what was actually built, with every place it departed from its own text.

The defect, in product terms

A new user registers. We rebuild roughly ninety days of their history so the chart is not born empty. We rebuild it only for the positions they hold on the day they register, a deliberate rule, so that a standalone position they closed and forgot about does not reappear as history they cannot recognise.

The rule has a consequence nobody chose. If the user exited a position inside those ninety days, the exit is not replayed, but the cash it paid out is: the dollars landing in the wallet are a tracked balance. With nothing to net against, the chart reports that cash as new money arriving from outside.

Measured on one of the campaign's wallets: a vault redemption of 28,832.81 USDC on 2026-07-04 at 10:01:11 is drawn as a deposit of +$28,828.57 at that same second. Twenty-one minutes later the same money went into another vault, and that deposit is drawn too. The wallet's opening book value understates what it actually held by about 39%, and the product's own explainer ("closing a position keeps what it earned") is false for that position, which kept nothing and re-entered as a deposit.

There is a second consequence, structural rather than numeric: no registration backfill can ever exercise the exit path at all, because a fully-closed position is by construction not in the set the replay is built from. The exit case is untested because it is unreachable.

What a fix has to do

Three things, in order:

1. Discover the position groups the wallet held at some point inside the window and no longer holds.
2. Replay them: read their balances at every grid point (they read as held, then as gone), and lay down their own flow rows.
3. Explain the disappearance. The engine's honesty rule is that a position that vanishes with no flow to explain it books nothing and raises an alarm. Once the exit's own flow row exists, the exit nets against the incoming cash in the same transaction and both sides come to roughly zero, which is the right answer.

Step 3 is already built and is not the problem. Steps 1 and 2 are.

Two shapes, and why the cheap one is only half of it

Shape A: read the wallet once at the window start

Add one full-universe read at the first grid point (ninety days back) and union what it finds with what the wallet holds today. Anything held at the window start and closed later is then replayed.

This is a small change: the same read the probe already performs at "now", issued once more at an archive block, plus a union of two group sets. The campaign's own wallet is fixed by it, because the vault it redeemed had been held since before the window opened.

It is also not the stated fix. A position opened and closed inside the window is invisible to both reads, so its exit still draws as a deposit. Shipping A and calling P5a done would leave a defect of the same shape live, under a changelog line saying it was fixed. That is the half-ship the gate forbids.

Shape B: sweep the window for everything the wallet touched

Scan the transfer streams of the whole universe over the window with the wallet as the filter, and admit every group the wallet touched. This is complete. It is also where the cost is:

• The universe is on the order of a thousand token addresses (every Morpho market's collateral and loan tokens, every vault share, every Aave/Spark aToken and debt token, the bare wallet tokens). The open-only design exists precisely to avoid scanning them all.
• The scanner passes its address list to one log query per block chunk. At ninety days and fifty-thousand-block chunks that is roughly thirteen chunks in each of two directions per wallet, with a thousand-address filter on an archive endpoint. Providers cap address arrays and degrade on wide ones, so the scanner would need address chunking it does not have today, and the chunked range backoff would multiply the call count when a provider pushes back.
• Every registering wallet pays this, on the archive endpoint, inside the drain that already parks wallets when it runs long.

What it puts at risk

Each of these is a live invariant that a change here can break quietly:

• The window start. First activity is currently the earliest flow of an open group. Admitting closed groups moves it earlier, which moves the date every new user's chart starts on.
• The "this wallet holds nothing" verdict, which is terminal and prunes history. It is decided from the current read alone; a discovery pass that finds historical groups sits right next to that decision and must not change it.
• The disappearance guard. Every replayed close is a leg death that the flow ledger must explain in the same interval. On the daily replay grid a death and its explaining flow land in the same day, but the guard's tolerance has never been exercised by a registration replay, because the case was unreachable.
• Idempotency. A re-run must produce identical rows. A discovery pass whose result depends on how a provider chunked its logs is a second answer to the same question.
• Archive cost and drain time, both already the subject of an open operational defect (wallets parked at the statement timeout when several drain at once).

Cost estimate

Piece	Estimate
Shape A (window-start read, union, tests)	~0.5 day
Shape B discovery: address chunking in the scanner, budget guard, determinism	~1 day
Replay wiring, first-activity and empty-verdict decisions, alarm handling	~0.5 day
Unit tests + a fixture wallet that exits a position inside the window	~0.5 day
Re-derivation of the affected wallets and verification against the real numbers	~0.5 day, and it needs the server
Total	~3 days, plus a server-side verification pass

Against a two-day gate, with the verification step outside this wave's remit (no server work), the honest call is to stop.

Recommendation

1. Do not ship a partial version. Shape A alone leaves the same defect reachable and would be recorded as a fix.
2. Schedule Shape A + B together as their own change, with the fixture wallet that exits inside the window as the acceptance case. That fixture is worth having on its own, because it makes the exit path testable for the first time.
3. Until then, know the exposure. It shows up only on a wallet that closed a tracked position inside its own backfill window, and it shows up as a deposit marker and an understated opening book value, never as fabricated yield: both performance lines net the flow out, so the total-return and accrual figures are unaffected. What is wrong is the capital-movement story, not the return.

Provenance

Prod validation campaign, 2026-08-06: verdicts S4-b3 (the redemption has no rows anywhere), S4-b2 (the deposit twenty-one minutes later graded as legitimate), F3 (the same wallet's window start), and the reviewer's note joining them.