Skip to content

built Built. This is a decision record, not documentation.

What is still current: Nothing describes current behaviour. It is the adjudication record of the v0.32.x validation campaign, including the renegotiated P3 acceptance figure; the engine those figures were measured against has since been replaced.

Landed: PR #560, released in v0.34.1 (2026-08-10)

Header updated 2026-09-14. The body below is frozen history. All plans.

The v0.32.x validation fix wave ​

Reviewed and adjudicated on fix/v0321-validation-wave. Every review finding is dispositioned (see "From the independent reviews"), and P3's acceptance was renegotiated in writing by the architect before merge rather than left open: the accepted figure is 14,454.06, not the ±1% band around 11,248 the plan first wrote. What that trades away, and what it obliged instead, is in P3 below.

A five-agent validation campaign ran the signed-in portfolio against a real prod account and seven real wallets in August 2026, deriving every number it checked independently (archive eth_call at anchor blocks resolved by timestamp bisection, prices read from the mirror over a read-only connection) rather than from the pipeline it was auditing. It found the two-line engine arithmetically exact and found seven defects around it, one of which is the single largest number the chart draws for the affected wallets.

This page is the plan it produced, and what was actually built against it. The campaign's own ground-truth figures are the acceptance criteria, quoted below in the units the chart draws.

What was wrong, and what it now does ​

P1 — a seizure financed in another currency drew the whole seizure ​

Before. When collateral in one denomination is financed by debt in another and the position is liquidated, the chart booked the ENTIRE collateral taken as a realized loss, and the debt the liquidator repaid on the reader's behalf was credited in no denomination at all. Measured on three real events: 14.376386 ETH drawn against a true 0.748885, 8.325848 against 0.431757, and 1,322.750 against 65.966. On the largest, a liquidation that cost 4.5% of the position read as a 90.6% one-day drawdown on a position still worth 135 ETH.

After. The repayment is valued in the collateral's own denomination off the same price bar that values the seizure, and netted exactly as a same-denomination seizure nets, per mark, floored at zero. A same-denomination control is unchanged to 1e-9 (0.277041752815838 ETH on its own event). If either price is unavailable the penalty is withheld rather than partially derived, and the marker is still drawn: the event is visible even where its magnitude is not.

The stored rows still carry the old magnitudes, which is what the repair script in Server steps is for.

P2 — a Fluid seizure vanished when its owner had touched the position ​

Before. Detection suppressed state-diff liquidation reading for any position its owner had operated anywhere in the scan window. Wallet 0x1ec6942b…'s NFT 8643 was liquidated in nine steps and the product served no liquidation at all, in six views, at both chart widths and in the aggregate, because of a routine deposit made a day earlier. The practical shape of it: the more actively somebody managed a position, the more certain its liquidation was to be invisible.

After. An owner operation explains the difference at ITS OWN block, not across the window. Both gate layers are mutation-tested, and the original purpose is intact: an owner operation at the same block or in the same transaction still wins, so an operation is never misread as a liquidation.

This changes what the pipeline WRITES, so the affected wallet needs the targeted re-derivation in Server steps; no repair script can create rows that were never detected.

P3 — an active restructure overstated total return ​

Before. On 0x60753e86… the total-return line read +16,309 where the mark-to-market truth is +11,248. A Fluid smart-pair restructure moves several legs of one position in one transaction, and the guard that refuses to book value it cannot explain was asking the question per LEG, so a genuine −1,855.29 booked zero.

After. For Fluid legs the question is asked at the POSITION, not the leg: the transaction's flows across the position's legs jointly explain its transition. Verified by replaying the wallet's real stored rows: exactly 1 of 84 intervals changed on each line, by exactly −1,855.300005, and the other 83 (including all 66 flow-free ones) are byte-identical to 1e-9. The honesty rule stands — a transition the grouped ledger still cannot explain books zero and reports itself.

The acceptance number was renegotiated, in writing, before merge. The plan attributed the whole −5,061.24 overstatement to this guard. The campaign's own evidence does not: the guard owns exactly one of the four restructuring days (2026-06-07, −1,855.30), which is 36.7% of it. So the served figure moves 16,309.36 → 14,454.06 and stops there.

The accepted outcome is 14,454.06 (architect ruling, 2026-08-07): this guard is required to recover exactly the 1,855.30 it withheld, and no more. The remaining +3,206 is not a defect in the guard, it is the six-hourly attribution convention the engine has always run on: a leg attributed from an index carries its return on the size present when the interval opened, so a balance migrated from one position to another mid-day keeps earning on its pre-move size until the next reading (the largest single day, 2026-07-23, is truth −82.01 against a drawn +1,972.32, and that position has no unexplained birth for the group guard to fire on). Changing that is a change to every wallet's accrual line and is explicitly OUT of scope for this wave.

What the ruling obliged instead: say so where the reader is. A convention is only honest if the number beside it does not claim otherwise, and the chart's explainer read as an exact identity ("everything the book holds, valued at what it is worth today, net of the money you moved in and out"). It now also states that both lines are read at intervals rather than continuously, that money moved between two readings is picked up at the next one, and that a position restructured partway through a day keeps counting on the size it held at the last reading. The same limit is stated on the portfolio page of these docs, beside the same identity. Pinned by the copy contract in series-labels.test.ts, so it cannot be dropped silently.

P4 — a $23M holding was invisible and undisclosed ​

Before. 0xa2c0f108… holds 12,065.12 stETH (~$23.0M) bare in the wallet. It was charted nowhere, which is correct (it is outside coverage), and it was absent from the outside-holdings list too, which is not: the statement of account simply did not mention $23M.

After. Bare stETH and eETH are disclosed under Other at a market-only value, read at display time, excluded from both performance lines exactly as the coverage policy requires. Neither performance line moves, by construction rather than by a filter: the disclosed legs never enter a snapshot, a flow ledger or a curve.

P5 — consistency and boundaries ​

  • a. A position closed inside its own backfill window replays its cash inflow as an external deposit (measured: a $28,833 vault redemption drawn as new money arriving). This item carried an explicit feasibility gate; the gate was invoked and the design delivered instead of a half-fix. See the design.
  • b. Both chart widths now serve the newest observation at its own instant, so one moment reads one value whichever width is on screen.
  • c. The history response's basis label described the markers while sitting beside a book value stamped in the other basis. It now reports the POINTS' basis, and a new additive field carries the markers'. No key was renamed.
  • d. Flow markers below $1 (0.0005 ETH, 0.00001 BTC) are no longer drawn: seven of eleven markers on one real wallet were under $5 and six under $1, none of them a capital move anybody made. Display only — the rows and the netting are untouched and the events ledger still serves every one.
  • e. Liquidation labels too close to read separately share one label carrying the count, so nothing is hidden and no label overprints; the 1W axis no longer collides on its last tick; a dust book no longer prints "−$0".
  • f. A marker sits with the point that draws the drop it explains, on both widths.

P6 — the pipeline around all of it ​

  • a. Backfill parking. Three wallets lost their history on prod while the pipeline worked exactly as designed: writers take one lock, a large replay holds it for minutes, and the role cancels any statement over 30 seconds — so writers were cancelled for QUEUEING rather than for working, and after two attempts the queue gave up. Each finished in 28-33 seconds the moment a human re-queued it. The wait now has its own budget (and hands the role's ceiling back before any write runs under it), the retry budget is four rather than two, and a wallet parked by that class of fault is re-queued by the drain itself after a rest, a bounded number of times. That last part matters most for a WATCHED wallet: the existing escape hatch fires on sign-in, and nobody signs in as an address somebody else added.
  • b. Price-mirror storm. One drain issued 8,747 rate-limited price look-ups in a single run, each degrading to the standing bar anyway, while 20 minute-precise look-ups lost their precision and 1,526 marks came off stale bars. The first refusal now pauses this process's look-ups for a cool-off and everything inside it is declined locally, before the socket. No served number changes; the traffic that was holding the limit tripped does.
  • c. Secret hygiene. Providers put the API key in the URL, and the backfill printed the endpoint once per wallet into a cron log that was world-readable on the prod box. Every logging site now prints through a redactor. The key itself still wants a rotation decision.

P7 — docs, fixtures, tests ​

The fixture gained a fourth wallet carrying both liquidation shapes and one sub-floor transfer, on its own account, so no existing figure moves. Five e2e specs assert them end to end through the real read path. Every behavior change above is reflected in the metrics, pipeline, deployment, external-dependency and process pages.

As built: where this differs from the plan ​

Each of these is a decision made against a measured fact, not a shortcut. They are listed in the order the work happened.

P1 — the redemption mark of a cross-currency seizure. The plan says to net "per mark". There is no par claim between two denominations (the redemption logic treats a dollar stablecoin as par in ANY book, which would have valued 23,985 USDT as 23,985 ETH and clamped a real penalty to zero), so the redemption mark nets the SAME market-bridged repayment the market mark nets, while the collateral keeps its own per-mark value. Made explicit with a flag so the same-denomination path keeps its strict rule; both behaviours are mutation-tested.

P1 — one scope edge left as it was. A liquidation whose DEBT reserve is absent from the registry carries no debt leg at all, so it still books the full seizure. Not in the plan, not reachable on the campaign's events, and fixing it needs a decimals source rather than a netting change. The existing test was renamed to say what that case now is.

P3 — births, not births and deaths. The plan words the group-level question as covering both. Measured rather than assumed: replaying the campaign wallet's real stored rows produces 3 birth anomalies, 0 death anomalies and 0 bridge anomalies over its whole history. The death branch already carries a group-shaped escape valve written for exactly this case, so a restructure withdrawal passes it and attributes normally, and extending the group test into the death branch would additionally have to unwind suspension state retroactively — the machinery that exists to keep the guard honest. No measured defect is left unaddressed by the narrower scope.

P4 — a curated read, not a registry row. A registry row with no book would have flipped stETH to excluded for EVERY venue, un-charting any stETH leg held on a covered venue and re-booking its stored history; a row with a book would have charted it, which the acceptance forbids. So the disclosure is a display-time read over a short list in code, and the wave needs no migration.

P5a — the feasibility gate was invoked. Measured before deciding: the discovery pass needs either one extra full-universe archive read at the window start (which fixes the campaign's wallet but leaves a position opened AND closed inside the window still drawing its exit as a deposit) or a whole-universe wallet-filtered sweep over ~1,000 token addresses across ~650k blocks, which needs address chunking the scanner does not have. It also sits on four live invariants, and its acceptance can only be confirmed by a server re-derivation this wave may not run. Estimated ~3 days plus a server pass against a ~2-day gate. Shipping the cheaper shape alone is the half-fix the gate forbids. Design and cost estimate: Replaying positions that closed inside the backfill window.

P5b — fixed at the open bucket, with a residual named. Both widths serve the newest observation at its own instant and the still-open bucket keeps the reading at its start, which is what the acceptance asks. A residual the plan does not cover: for a FINISHED day on a six-hourly history the daily point still carries that day's last reading stamped at its start. Removing that means re-stamping every daily point, which moves the whole shipped series, and that is not a consistency-batch change.

P5c — the wire, not the caption. The plan says to fix the field value and the hero caption. The caption is already true: the figure it captions is genuinely market-marked, so changing it to say otherwise would introduce the lie the item exists to remove. Only the wire was corrected, and with an added field rather than a renamed one — one field could not carry both answers truthfully.

P5e — merged, not offset. The plan allowed either. Markers too close to label separately share one label carrying the count, and every event keeps its own line, so nothing is hidden and no label overprints. The clearance a pair needs is measured against the labels that will actually be drawn, not against a flat share of the window: merging is what widens the label ("LIQ x2" is twice "LIQ"), so a threshold calibrated on the plain one let a counted flag and the next seizure overprint at the separations just past it. The plain pair's threshold is unchanged.

P5f — both widths, not the daily one. The same off-by-one exists on the six-hourly grid, the rule is one rule, and a marker carries no clock of its own on screen (the ledger keeps the real timestamp), so snapping the flag to the point that draws the drop misreports nothing on either width.

Two unplanned fixes, both consequences of the work above. A point off the daily grid now shows its clock time in the tooltip (after the tipping change a multi-wallet view carries one tip per wallet seconds apart, which under a date-only label would read as several values for one day — the same confusion P5b removes). And the disclosed-holdings read is bounded at 2.5 seconds with its empty answer cached: it is the only chain read the dashboard waits on, and the shared client retries three times on a 60-second timeout, so a node that hangs rather than refusing could have held the positions endpoint for over three minutes.

P6a — what "serialize the writers" turned into. The writers already serialize; the defect was that serialising was punished. So the acquire got its own wait budget rather than the worker pool getting capped, which keeps the pool's benefit (one slow child no longer blocks the tick) and removes the failure. The self-heal deliberately excludes provider faults, because a backfill run is a destructive full re-derivation and re-running one every quarter of an hour against a broken upstream forever is worse than a parked row and an alert.

P7 — the fixture covers the READ path for P2, not detection. A Fluid seizure on an owner-touched position is seeded and asserted end to end, but detection runs against chain state, which an offline fixture has none of. The detection gate itself is covered by mutation-tested unit tests, and its acceptance is the server re-derivation in Server steps.

P7 — one e2e item is not achievable offline. The suite points every RPC at a closed port, so the disclosed-holdings read (P4) correctly serves nothing in the fixture and "stETH in the Other tab" cannot be asserted there without a chain stub. Its unit coverage pins the acceptance figure through the real enrichment path.

From the independent reviews ​

Three reviewers read the built branch. Eleven findings, seven distinct after de-duplication. Six were confirmed against running code and fixed; one was refuted by the architect's ruling on P3. Each was re-verified against the running code before it was dispositioned, and every fix below is mutation-tested (revert the fix, the spec fails, and nothing else does). Verifying them turned up three more, listed after. What each changed:

P3's acceptance was MISSED by 28.5% and has been renegotiated: REFUTED as a defect. Sized above. The ruling is that 14,454.06 is the accepted figure and the residual is the attribution convention, out of scope. The obligation it carried instead was to stop the chart and the docs promising an exact identity across a mid-window move, which is done and pinned by a copy contract. Two consequences the plan did not anticipate, both stated rather than smoothed: the residual is a different mechanism from the one P3 fixes, and the fix's own −1,855.30 lands on the ACCRUAL line as well as the total-return line (M21 is one guard for both), moving this wallet's accrual headline by the same amount — about −18.6% of it. That is the correct treatment (a smart pool's re-mix cost already lands there) and it is a bigger move than the plan contemplated, so it is disclosed rather than discovered later.

A cross-currency Fluid seizure was about to store a fabricated zero — on the acceptance wallet. Making the seizure visible (P2) also made its VALUATION reachable, and that valuation subtracted dollars from ether: on NFT 8643 that is 33.06 ETH minus 48,930 dollars, clamped to nothing, i.e. a stored claim that a real seizure cost nothing. The mirror-image vault would have booked almost the whole seizure, the same overstatement class P1 removes. Fixed the way P1 fixes it: convert first, off the same price observation, then subtract; and withhold rather than zero when the conversion is unavailable. The chart still withholds a cross-denomination Fluid magnitude — rows written BEFORE this fix carry the unconverted figure and admitting them would draw those — so the corrected number reaches the statement of account, not yet the line. P2's "non-null loss on the total-return line" is therefore met for a same-denomination position and not for a cross-denomination one; the seizure itself is visible in every view, which was the defect. One tightening rides along and is worth stating: a leg that cannot be valued at all used to be skipped, quietly shrinking the side it belonged to; it now withholds that whole side, so a partly-priced position reports no magnitude rather than an understated one.

A seizure financed in a par stablecoin would have withheld its penalty forever. P1 asked the price mirror for a cross-denomination price of USDS / PYUSD / FRAX / crvUSD — assets that are marked at par precisely BECAUSE no price bar for them exists, so the answer is null every time, permanently, and each request spent a fetch-through lookup for a guaranteed blank. Those are ordinary borrow assets on Aave and SparkLend. The conversion now uses the pin itself (a dollar is a dollar) over the collateral denomination's own price, and issues no lookup that cannot answer.

A withheld penalty was served on the wire as loss: 0. This wave opened two new ways to reach a withheld penalty, and the response had no way to say "unknown" — a seizure that cost nothing and a seizure whose cost is unavailable are different statements. The field is now nullable. Additive; the marker draws either way.

The group-level explanation could have passed on cancellation alone. The joint test measured its allowance against the pair's COMBINED size while testing their DIFFERENCE, so a levered pair born with no flows at all passed on the netting: driven against the shipped engine, a position of 100,000 against 97,500 booked +2,500 of return nobody earned, with no alarm, and any position above about 96% of its borrowing limit qualifies. The allowance is now the widest member's own allowance, which keeps the real restructure explained and refuses that one. Every difference the joint test does book is now written to the operations log with its size.

The repair script would have overwritten verified figures with blanks during an outage. A re-derivation that comes back unpriced looks identical whether the price is durably unavailable or the archive timed out for a minute, and --execute re-derives from scratch rather than replaying the reviewed dry run, so the operator could not have caught it. Those rows are now left as stored, counted and printed, with --allow-null as the deliberate opt-in.

The backfill self-heal could not tell queueing from working. Postgres words "this writer was cancelled while waiting for the lock" and "this wallet's own write ran past the 30-second ceiling" identically, and the acquire hands that ceiling back before the writes — so a wallet whose write is genuinely too slow would have failed deterministically through four extra DESTRUCTIVE full re-derivations. The lock acquire now names its own failure, and only that shape heals.

Found while verifying the fixes, not by the reviews ​

The par shortcut read every par claim as a dollar, and two of them are not. The fix for the stablecoin-financed seizure above uses the par declaration instead of asking for a price bar that does not exist. But "par" in this codebase means one unit of the asset's own denomination, and the uncovered-par population is 17 dollar stablecoins AND two assets whose claim is something else entirely: the ether sentinel (one ETHER) and LBTC (one BITCOIN). Priced as a dollar, a repayment made in LBTC is credited at about one hundred-thousandth of its value, which books very nearly the FULL seizure — the exact overstatement this wave exists to remove, reintroduced by the shortcut meant to fix it. Reproduced at 6.5x on an ether-collateral position and pinned by a spec; the conversion now goes from the asset's own denomination, and the per-moment cache carries that denomination in its key so two par assets at one moment cannot swap bridges. Latent rather than observed — no such position is in the campaign's cover set — and found by enumerating the population rather than by reading the diff.

A withheld penalty had no spec, only a type. ?? 0 still type-checks against a nullable field, so nothing but a test could hold the wire honest. The marker mapping is now a named pure function with its own spec (mutation-tested: reinstate the coalesce and it fails).

Three doc statements the build had made untrue. The pipeline page still described a cross-denomination seizure as keeping the whole seizure value, still described the Fluid seizure gate as suppressing a whole position for the whole window, and the read path's own comment still explained the withheld cross-denomination magnitude by an arithmetic the write path no longer performs. All three now say what the code does, including why the read path still withholds (rows written before this wave).

Server steps ​

None of these has been run. Nothing in this wave changed anything on a server.

  1. Correct the stored seizure magnitudes (P1). scripts/repair/remark-liquidation-penalties.ts, dry-run by default, --execute to write, --wallet to scope. It re-runs the production classifiers and the production valuation rather than reimplementing the arithmetic, takes block timestamps from the stored rows (no archive reads) and disables the minute true-up (standing bars only, zero price-mirror credits). Fluid rows are skipped by design — they need rows CREATED, not corrected, which is step 2. Read the skipped-unpriced count before and after. A row that re-derives to a blank over a stored number is LEFT AS STORED and printed, because a transient archive or price-mirror outage is indistinguishable from a durably unavailable price and a wrong blank cannot be undone by re-running. A non-zero count means the run was degraded: re-run it. Only if the same rows repeat on a healthy run are they genuinely unpriceable, and --allow-null then writes those blanks deliberately.
  2. Re-derive one wallet's history (P2): 0x1ec6942b94bfa1f1c858fda515b66da74865fc7d, re-enqueued through the app's own requeue statement and drained SERIALLY (BACKFILL_WORKERS=1 PORTFOLIO_BACKFILL_MAX=1). Never the destructive repair path. This is what creates the missing Fluid liquidation rows.
  3. No migration. The wave adds none. (If a later one is needed it starts at 076: a closed branch already carries a file named 075 and the migration ledger keys by filename.)

Not in this wave ​

  • P3's remaining +3,206. Ruled out of scope: it is the six-hourly attribution convention, not the guard, and closing it means changing how an index leg is attributed across a mid-interval balance move, which moves every wallet's accrual line. The obligation the ruling attached instead (say the limit where the reader is) is done. If it ever becomes work it is its own item, with its own acceptance.
  • Admitting a cross-denomination Fluid seizure's magnitude onto the chart. The write path values it correctly now; the read path still withholds it, because rows stored before this wave carry the unconverted figure. It needs a re-derivation sweep over the wallets holding such rows first, and then the read-time withholding can go.
  • The prod release. This ships to staging and waits.
  • P5a's replay work (design delivered, see above).
  • A rotation decision on the archive provider key. The value was never reproduced anywhere, including in the campaign's own report, but it did sit in a world-readable file.

Private documentation. creddit.xyz