Portfolio phase 2 — full PT accounting + Fluid position tracking
Researched 2026-07-10. Every load-bearing on-chain claim below was verified live on mainnet during planning (resolver payloads decoded, event semantics matched to the wei against real transactions, archive reads exercised at 30d/90d-old blocks, event volumes measured). Repo claims were verified against the merged staging tree (PR #348).
Prereq reading for every executing agent: AGENTS.md, the shipped docs/plans/portfolio-read-only-plan.md (methodology M1–M10 is still law), docs/plans/portfolio-execution-log.md (append deviations there, same format), docs/metrics.md, src/lib/portfolio/pnl.ts and valuation.ts headers.
Scope: (A) complete Pendle PT accounting (honest entry fills, M6 pull-to-par redemption, PT-as-Aave/Spark-collateral valuation, maturity handling), and (B) full Fluid position tracking, T1 through T4. Each workstream is one PR into staging. Base branch: current staging (contains PR #348).
0. Architecture decision for Fluid (evaluated, settled)
Four candidate data paths were evaluated. Resolver-first wins, uniformly for all four vault types.
| Path | Verdict | Why |
|---|---|---|
| On-chain resolvers (chosen) | Primary for levels, events for flows | FluidVaultResolver 0xA5C3E16523eeeDDcC34706b0E6bE88b4c6EA95cC positionByNftId(id) returns one FULLY STATIC 109-word payload (~175k gas, ~0.3s) containing everything: owner, isLiquidated, interest-accrued supply/borrow, dustBorrow, tick, the vault address, vaultType (10000/20000/30000/40000 = T1–T4), both token tuples, CF/LT/penalty, oracle, exchange prices, borrowLimit/minimumBorrowing. Verified working at latest, 30d-ago and 90d-ago archive blocks; the resolver deployed at block 23,881,723 (2025-11-26), ~4.5 months older than any block the 90d backfill needs. Liquidations are fully absorbed by reads (verified across a real liquidation block: supply 44.3255→42.3455 shares). Smart-leg share→token conversion via FluidDexResolver 0x11D80CfF056Cef4F9E6d23da8672fE9873e5cC07 getDexState(dex) (static 30 words; words 26–29 = token0/1PerSupplyShare, token0/1PerBorrowShare, native decimals per 1e18 share), also archive-safe. |
| Fluid public API | Secondary only: discovery aid + QA cross-check | GET https://api.fluid.instadapp.io/v2/{chainId}/users/{user}/nfts exists (spec at /swagger), serves wei-exact JIT resolver reads (verified to the wei against the resolver). But: undocumented/unauthenticated third-party infra, 200-req rate limit, cannot serve block-anchored/historical reads, and its vault-scoped sibling /v2/1/vaults/{id}/nfts provably serves stale zeros for smart vaults — the backend index can silently rot. Never on the critical path; useful in QA to cross-check our share-decomposition math. |
| Dune | Dev-time reconciliation oracle only | The 0xfluid pipeline (result_fluid_user_reserves, q6901809) is DAILY and keyed by liquidity-layer user address (vault/DEX contract), with no wallet or NFT dimension — it structurally cannot serve per-wallet history. A custom scheduled per-wallet query at 6h cadence would burn ~1,200–2,400 of our ~4k monthly credits and adds decoded-table lag + third-party matview refresh risk (their q5011149's latest execution state is FAILED). Their own pipeline clamps greatest(cum_shares, 0) to hide event-sum drift. What we DO take from Dune is the unit conventions (§B.1) and a one-off ~10-NFT reconciliation (~100–200 credits, FWS5). |
| Indexing (rindexer) | Rejected (Fred: last resort) | Unnecessary: measured chain-wide event volume is ~120–150 LogOperate/day (two point-in-time samples, 2026-07) + ~70 factory events/day + rare LogLiquidate. A 6h cursor scan decodes ~200 logs/day. An indexer buys nothing here. |
Two structural facts drive the whole Fluid design; executors must internalize them:
- Fluid has no per-position liquidation event.
LogLiquidate(liquidator_, colAmt_, debtAmt_, to_)(topic00x80fd9cc6…cb66) is vault-aggregate; liquidated positions emit nothing. Per-position liquidation is detected by state diff between anchor reads, with LogLiquidate as the cheap trigger.isLiquidated=1in the resolver payload means "sits on a liquidated branch", NOT closed — verified: NFT 9266 had isLiquidated=1 with live, changing balances across two separate partial liquidations. - LogOperate amounts change meaning by vault type. Verified to the wei: T1 legs and the non-smart legs of T2/T3 carry signed TOKEN amounts (
colAmt −17552000000000000== 0.017552 weETHs ERC-20 transfer); smart legs (T2/T4 collateral, T3/T4 debt) carry signed 1e18 DEX shares (a T4 deposit moved 14.4168 GHO in calldata but emitted colAmt 6.5004e18 shares). One event can mix both semantics (T2: colAmt in shares, debtAmt in USDT tokens).
1. Methodology addenda (extend M1–M10; same locked status)
| # | Rule |
|---|---|
| M11 | PT entry basis. A PT acquisition flow is valued at the PT's own price at the flow block (TWAP getPtToAssetRate(market, 900) × underlying's book-unit price), NEVER at par of the underlying. The position's entry implied APY is derived from its acquisition fills (quantity-weighted when there are multiple buys; partial sells do not change it). A position whose entry predates our data window gets a synthetic basis = the first observed pt_to_asset_rate at its opening snapshot, labeled basis_synthetic in the API/UI. |
| M12 | PT redemption mark = pull-to-par at entry implied yield. value_redemption(t) = qtyPT × (1 + entryImpliedApy)^(−yearsToMaturity(t)) × underlyingRedemptionPrice(t) via the existing ptRedemptionAssetPerPt. At/past maturity the rate is exactly 1 (par). Never blended with the MARKET TWAP series. The M7 wedge for PTs is |
| M13 | PT maturity. A PT never silently leaves the universe while any tracked wallet holds it. At maturity the valuation rolls to par; the exit is the redemption burn (Transfer to 0x0), a normal withdraw flow at par. The 45-day matured-grace universe filter is replaced by a held-by-anyone check. |
| M14 | Fluid legs. T1 supply/borrow legs are accrual='index' with ComposedIndex = the VAULT-level exchange price (vaultSupplyExchangePrice/vaultBorrowExchangePrice from the resolver payload, 1e12 scale) composed with the accounting asset's book redemption rate. The LL-level exchange prices stored in fluid_ll_apy are NOT the position index (they ignore vault magnifiers); they serve quoted rates only. Smart legs are TWO PositionReads per side (one per pool token), qty = shares × tokenXPerShare at the anchor block, accrual='value' regardless of the token's par-ness (per-share amounts grow with trading fees; classifying a par-token smart leg 'none' silently drops fee yield). Key shape (load-bearing): the side suffix must stay TERMINAL because side is derived by positionKey.endsWith(':debt') in five places (assemble.ts sideForPositionKey, valuation-sources.ts sideForKey, api-data.ts ×2, quoted-rates.ts) plus labelForRow. T1: fluid:vault:<addr>:nft:<id>:supply|:debt (6 parts). Smart legs: fluid:vault:<addr>:nft:<id>:<tokenLower>:supply|:debt (7 parts, token BEFORE the side). groupKeyForLeg slices the first five parts, so both shapes group as one NFT. Pin with a test: a fluid smart-debt leg classifies side='debt' through assemble, api-data, and quoted-rates. Cross-book rule: a fluid group whose legs span more than one real book is EXCLUDED even when debt-free (new classifyLegs rule) — a debt-free USDC-ETH smart-collateral NFT would otherwise be included per-leg and pool composition drift would book phantom yield in one book and phantom loss in the other with no flow to net it. Fixtures both with and without debt. |
| M15 | Fluid liquidations. Trigger: LogLiquidate on the vault within the scan window. Classification: an NFT whose col/debt decreased across the window with no matching LogOperate for that nftId. Booked once, as a realized-loss event valued as equity destroyed (Δ collateral value − Δ debt value at the window anchors), never as user flows (M4). Deterministic provenance for idempotency: the synthesized row carries the tx_hash + log_index of the LAST LogLiquidate on that vault in the window. Mechanism (corrected): isLiquidationMechanic is a NO-OP for fluid (it excludes same-tx Transfer mechanics; liquidated Fluid positions emit no events) — do not extend it, verify it needs no extension. The machinery that must change is seizedKeysByInterval in buildBookCurve, which skips the value-series contribution of exactly one position_key: the synthesized fluid liquidation row's position_key is the NFT GROUP PREFIX (fluid:vault:<addr>:nft:<id>) and the seizure skip must PREFIX-MATCH fluid keys against it, so every 'value' leg of a smart NFT (2–4 legs) is covered. Without this, a seized smart-collateral leg double-counts the loss and a seized smart-DEBT 'value' leg books phantom POSITIVE yield. Fixture: a smart-col + smart-debt liquidation asserts the loss is booked exactly once, per leg. Note: a T1 'index' leg is qty-agnostic and safe — an acceptance test that only covers T1 would pass while the smart-leg double count ships. |
| M16 | Fluid NFT transfers. A factory ERC-721 Transfer between two wallets moves the entire levered position. Collateral legs: transfer_out/transfer_in valued at the flow block. Debt legs: emit the SENDER's debt leg as kind repay and the RECEIVER's as borrow (valued at the flow block) — a debt transfer_out would be signed −value by signedFlowValue with no side awareness, booking −(col+debt) instead of −(col−debt) at the sender and +2×debt phantom yield on 'value' debt legs. Pin with a test: an NFT transfer of a levered position nets to −equity at the sender, +equity at the receiver, and zero yield on every leg. Mint (0x0→user) and burn are not value flows; the same-tx LogOperate carries the value. |
2. Current-state facts executors must not rediscover wrong
PT (shipped state, verified):
readers/pendle.tsemits qtyRaw = PTbalanceOf,indexRaw=null, accountingAsset = market UNDERLYING, plus (not persisted) marketAddress, maturityTs, ptToAssetRate (TWAP at the anchor block, null on young-TWAP revert = valid read).snapshot.tspendle branch: value_market = qty × ptToAssetRate × book-unit price; value_redemption stored NULL always;ptRedemptionFallback(assemble.ts:117) patches redemption = market at the DB→read boundary. Consequence: the PT wedge is structurally zero today.- Flow-valuation defect (root of FWS1): PT flows are valued at PAR of the underlying in both marks (flows.ts values the PT transfer via the underlying's price with an implicit rate of 1). A PT bought at 0.95 books −0.05/PT of instant phantom negative yield that then accretes back.
- PT-as-Aave/Spark-collateral:
buckets.tsdeliberately resolves a known PT to EXCLUDED (long comment at lines ~32–43); the whole account lands in "Outside the yield book" (pinned by a regression test at pnl.test.ts ~line 678, which FWS1 will retire and replace). - The M6 engine exists and is tested but uncalled:
ptImpliedApyFromFill(entryAssetPerPt, yearsToMaturityAtEntry)(= p^(−1/T) − 1),ptRedemptionAssetPerPt(impliedApy, yearsToMaturity)(= (1+y)^(−T), returns 1 at T≤0),yearsBetween. - Universe:
loadPendleMarketskeeps matured markets forPENDLE_MATURED_GRACE_DAYS = 45, then they vanish from snapshot AND flow universes — a still-held PT would book its whole value as phantom negative yield.loadPtUnderlyingsmaps pt→underlying over ALL markets. pendle_market_state.pt_to_asset_raterpc-basis rows start 2026-07-02; the older pendle_api rows carry NO rate. The portfolio does not need that table for marks (it reads the TWAP at-block, including archive blocks); it MAY use it for synthetic basis lookups where available.registry.tsheader comment about PT promotion is STALE (says promotion exists; shipped code EXCLUDEs). Fix the comment in FWS1.
Fluid (verified live):
positionsNftIdOfUser(wallet)→ uint256[] across ALL vaults (30k gas; factory-wide; 18,560 NFTs exist in total).positionByNftIdis self-describing (returns its vault + type + tokens), so the reader needs no vault registry to value a position.- Vault-type encoding in ConstantViews: 10000=T1, 20000=T2, 30000=T3, 40000=T4.
- ETH appears as pseudo-token
0xeeee…eeeein Fluid token tuples (native ETH). getDexStateper-share token amounts are in NATIVE token decimals per 1e18 share and the composition DRIFTS with pool price (USDe-USDT went 1.213/0.910 → 0.762/1.362 per share between two observed blocks): smart-leg share amounts MUST be converted at the block being valued, never with today's rates.- Factory
0x324c5Dc1fC42c7a4D43d92dF1eBA58a54d13Bf2d: NewPositionMinted has vault/user/tokenId ALL indexed; ERC-721 Transfer standard (wallet filterable at topics 1/2, tokenId topic3). - LogOperate topic0
0xfef64760e30a41b9d5ba7dd65ff7236a61d89ed8b44c67a29e84db1a67513a1c, ZERO indexed params, data words [user, nftId, colAmt int256, debtAmt int256, to];useris msg.sender (often a DSA/wrapper) — never filter on it. - RPC ops: publicnode serves latest-block calls + recent logs; drpc serves all archive calls but free-tier getLogs needs ≤1000-block chunks with ~300ms pacing and retries. The chain-wide 90d Fluid event scan runs exactly ONCE, as the deploy-time seed of the
fluid_event_logcache (D7, ~650 chunks, attended, free tier); registrations and repairs read the cache. No paid key (D3, resolved). The JIT mini-scan's tail (events since the last 6h cron append, ≤ ~1800 recent blocks) is a live recent-range getLogs, which publicnode serves without archive access. - Repo plumbing already reserved:
Venueunion + SQL CHECKs include 'fluid';groupKeyForLeg(pnl.ts ~759) already groupsfluid:vault:<addr>:nft:<id>:<side>by the first five segments (one NFT = one group; the non-Aave debt-group rule applies: included iff all legs share one book — no e-mode gate, Fluid vaults are isolated pairs by construction).VENUE_INDEX_SCALEin valuation-math.ts has NO 'fluid' entry (a non-null indexRaw currently throws) — addfluid: 1e12. quoted-rates.tshas no fluid branch (returns null → dash in the earned-vs- advertised column).- Universe/labels:
carry_registry(protocol='Fluid') has vault_address/type/ labels/status but NO token decimals; the resolver payload has everything. Wound-down vaults (borrowLimit < minimumBorrowing) still hold live user debt: the reader must NOT filter them out (track all; annotate).
Dune conventions adopted (from q5011149/q6901809/q7490982/q7829121):
- Exchange prices are 1e12-scaled; normal = raw × px/1e12.
- LL LogOperate (liquidity layer,
0x52Aa8994…e497, topic00x4d93b232…8d15) data: [supplyAmount int256, borrowAmount int256, withdrawTo, borrowTo, totalAmounts packed, exchangePricesAndConfig packed]; supply/borrow exchange prices at bits 91/155 (64-bit each). totalAmounts = four coefficient<<8|exponent BigNumbers. - Do NOT copy their supply_rate formula (operator-precedence bug overstating by (px/1e12)^2) and do not replicate event-sum balances (their own
greatest(x,0)clamp admits drift). Events for flows, reads for levels. - On Fred's q7829121 (redemption-numeraire): the concept is exactly our REDEMPTION mark and the netflow-in-same-mark convention matches M5. Two flaws if it keeps being used: the
else 1.0catch-all rates ANY unmapped token at 1 ETH (fine for the weETH/ETH position it targets, wrong the moment it's pointed elsewhere), and the log-linked TWR silently contributes zero for a wiped/negative period instead of flooring at −100% (the samelinkTwrbug class we fixed in PR #348).
3. Workstreams
Order: FWS1 → FWS2 → FWS3 → FWS4 → FWS5. FWS1 is independent of the Fluid work and can run in parallel with FWS2. FWS2 ships gated (D8) so the FWS2→FWS3 staging interim never charts unflow-covered fluid legs. Each PR updates docs in the same commit and appends every deviation to docs/plans/portfolio-execution-log.md. The review discipline from #348 applies: independent review lenses per PR + adversarial verification of findings before fixes.
Acceptance prerequisites (every workstream, read before starting): a local Postgres with the creddit schema is required (recipe in the team memory / deployment docs; restore the staging nightly dump locally so pendle_markets, lending_reserves, morpho_market_registry, carry_registry, and token_yield_apy share-rate history are POPULATED — 'history'-mode valuation skips wrapper legs without them). Fixture wallets must exist as accounts: INSERT INTO onchain_credit.accounts (uid, created_at) VALUES (lower('<addr>'), now()) — backfillWallet throws on a missing accounts row (FK). Find real holders via factory/Transfer logs; a single wallet holding Aave + Fluid T1 + smart legs may not exist — compositing across two fixture wallets is sanctioned.
FWS1 — PT truth: entry fills, pull-to-par redemption, PT collateral, maturity
Goal: a PT position charts honestly in both marks, whether held directly or as Aave/Spark e-mode collateral; the flagship PT-loop carry stops being "Outside the yield book".
- Fix PT flow valuation (the phantom-yield bug). In the flow-valuation path, a
pendleflow's underlying amount and both mark values must use the PT's own rate at the flow block: TWAPgetPtToAssetRate(market, 900)at that block (archive read; a young-TWAP revert is a valid null → leave the flow unvalued per M9, never par). Regression test: a 100-PT buy at rate 0.95 books a +95 flow, not +100, and the first window shows ~0 yield, not −5. - Entry-basis derivation (pure). New pure module (e.g.
src/lib/portfolio/pt-basis.ts): given a position's ordered PT flows (acquisitions with per-flow assetPerPt implied by value/amount, disposals, opening balance) and maturityTs, produce{ entryImpliedApy, basisSynthetic }. Rules: quantity-weighted average fill across buys; sells don't move it; an opening balance (position predates the window) uses the opening snapshot's pt_to_asset_rate as a synthetic fill, flagged. Derived at READ time in api-data from the flow ledger — no schema change. Unit-test adversarially (multiple buys at different discounts, buy-sell-buy, pure opening balance, flow with null valuation). Concrete plumbing an executor otherwise discovers mid-build: (a)loadFlowRowsdoes not select amount_raw/amount_underlying — widenFlowRowLiteand the query, plus PT decimals, or per-fill assetPerPt cannot be implied; (b) the opening snapshot's rate is not stored — recover it asqty_underlying / (qty_raw / 10^decimals)from the opening row (pendle_market_state rpc-basis rows only start 2026-07-02, do not rely on them); (c)ptRedemptionFallbackhas TWO call sites (loadSnapshotRows, loadLiveRows) and its M12 replacement cannot stay per-row — it needs flows + maturity + entryImpliedApy, so apply it at loadContext level after the flow ledger loads, leaving stored value_redemption NULL; (d) the flow-block TWAP in step 1 needs marketAddress (and maturityTs, for par-at-maturity) on the pendle TransferTarget —buildTransferTargetscurrently drops both. - Wire M12 redemption. Replace
ptRedemptionFallback(market-as-redemption) with the real curve: at each snapshot ts,value_redemption = qtyPT × ptRedemptionAssetPerPt(entryImpliedApy, yearsBetween(ts, maturityTs)) × underlying par price in the book. Maturity comes frompendle_marketsjoined via the pt address embedded in position_key (nothing new persisted). The wedge computation now runs on genuinely distinct marks — verify a real PT shows a nonzero wedge pre-maturity and that it converges to zero at maturity in a fixture. - PT as Aave/Spark collateral. Flip
buckets.tsback to promoting a known PT to its underlying's book, now WITH the valuation path: in snapshot.ts, an aave/sparklend leg whose accounting asset is a known PT is valuedrayMul-descaled aToken qty × ptToAssetRate(market, at block) × underlying book price(MARKET) and via M12 (REDEMPTION). Accrual — settled, do not improvise: the stored venue index (RAY normalizedIncome) is used ONLY at write time to descale the aToken quantity into value_market/value_redemption; it is NEVER the attribution index.accrualForRow(assemble.ts) gains anisKnownPt(accountingAsset)input built fromptUnderlyings(which api-data.loadContext already loads): an aave/sparklend row whose accounting asset is a known PT classifiesaccrual='pt'even though index_raw is stored — attribution then runs on the mark's own value series net of flows, identical to a directly-held PT. This ripples the signature oflegSnapshotsFromRows/legSnapshotFromRow; name it in the PR. (A value- series leg cannot carry a ComposedIndex — do not attempt to compose one.) Retire the "PT-collateral is Outside" regression test (pnl.test.ts ~line 678) and replace it with: a PT-sUSDe collateral + USDT debt e-mode account is an INCLUDED USD-book carry whose collateral yield accretes pull-to-par. Unknown PTs (not in pendle_markets) remain EXCLUDED. Stored-history repair: rows written before this fix carry book='EXCLUDED' and NULL redemption; after merge, runbackfill-portfolio-wallet.tsas a windowed repair for tracked wallets holding PT collateral so the delete+insert rewrites them with the promoted book and both marks (list this as a server step in the PR body). - Maturity handling (M13). Universe: keep any market held by a tracked wallet regardless of the 45-day grace. The held-by-anyone join must cover BOTH holding shapes: the market's pt_address appearing in a snapshot's position_key (pendle venue) OR in accounting_asset (aave/sparklend PT-collateral legs) OR in any flow row's asset — a join written against pendle position keys alone silently drops exactly the flagship PT-loop case this workstream exists to fix. Post-maturity the TWAP oracle may revert — value at par instead (rate exactly 1). The redemption burn is a par-valued withdraw flow. Fixture test: hold through maturity, redeem two weeks later; the curve is flat at par in between, no phantom yield at the universe boundary.
- Docs: metrics.md M6 section rewritten to the shipped truth (it currently describes the fallback); portfolio.md; execution-log rows. Fix the stale registry.ts header comment.
Acceptance (run, don't claim): local DB + mainnet: snapshot a real PT whale (find via PT Transfer logs), backfill a short window, show: entry fill derived from a real acquisition, redemption curve ≠ market curve, wedge nonzero, npm test/tsc/build green. Show the Aave PT-loop fixture account moving from "Outside" to an included USD carry with sane APY.
FWS2 — Fluid read core: reader, valuation, snapshots, JIT
Goal: every Fluid position (T1–T4) of a tracked wallet is snapshotted and valued at any anchor block, self-describingly.
src/lib/portfolio/readers/fluid.ts.readFluidPositions(wallets, blockTag): per walletpositionsNftIdOfUser(at the anchor block) → per NFTpositionByNftId(batch via multicall3; the payload is fully static, decode with the published resolver ABI — precedent: vault-capacity.ts's getVaultEntireData tuple decode). Emit:- T1 / non-smart legs: one PositionRead per side, positionKey
fluid:vault:<addrLower>:nft:<id>:supply|:debt, qtyRaw = raw units (normal amount ÷ (vault exchange price/1e12)), indexRaw = the VAULT-level exchange price (M14), accountingAsset = the leg token (map0xeeee…→ WETH address for bucketing; it is the ETH par unit), decimals from the resolver payload. - Smart legs: TWO PositionReads per side, positionKey
fluid:vault:<addr>:nft:<id>:<tokenLower>:supply(and...:<tokenLower>:debt— token BEFORE the side; the terminal:debtsuffix is load-bearing per M14), qtyRaw = shares × tokenXPerShare at the SAME block (onegetDexState(dex)per DEX per anchor, shared across positions), indexRaw = null, accrual resolves to 'value' (M14 — requires the assemble.tsaccrualForRowfluid case so par tokens are NOT 'none'). - Closed positions (supply=0, borrow=0, dustBorrow=0) emit nothing.
- dustBorrow is NOT an edge case: exhibit NFT 9266 carries nonzero, interest-accruing dustBorrow at every sampled block (~32,687 raw USDT units at latest; beforeDustBorrow likewise nonzero). Sum it into the debt leg from day one.
- Interim gating: until FWS3's flow scanner is merged, fluid legs are classified "Outside the yield book" with a distinct
coverage-pendingreason (a single const flipped in FWS3) — 'value'-accrual legs charted without flow coverage would book any interim deposit as pure yield (the newborn-value-leg defect class fixed for pendle in #348).
- T1 / non-smart legs: one PositionRead per side, positionKey
- Plumbing:
VENUE_INDEX_SCALE.fluid = 1e12;groupKeyForLegalready groups per NFT — confirm the 7-part smart-leg keys still group on the first five parts (extend the test); registry.tsloadFluidState()is thin (no vault list needed for valuation) but the carry_registry join for labels + wound-down annotation is a REQUIRED deliverable here (FWS4's positions-table annotation depends on it; return{label, status}per vault, filter NOTHING out); snapshot.ts venue wiring;PAR_ACCOUNTING_ASSETS/buckets coverage for every token observed in the live vault set (report unmapped ones — they become EXCLUDED legs honestly). - JIT (live.ts): same reader at latest; positionsNftIdOfUser is one 30k-gas call, so the JIT path needs no NFT cache. Cross-check in dev against
GET /v2/1/users/{wallet}/nfts(Fluid API) — amounts must match to the wei; any divergence is a bug in OUR share math (the user-scoped API route was verified wei-exact; never use the vault-scoped route, it serves stale zeros). - Books/M1: a wstETH/ETH T1 NFT = ETH-book carry (included, 'same-book-debt'); a USDC-ETH smart-col NFT = cross-book → Outside; a USDe-USDT smart pool = USD book included. Add classifier fixtures for each.
Acceptance: live snapshot of ≥3 real NFTs spanning T1, T2/T4 smart-col and a smart-debt vault (find holders via factory logs), each cross-checked: (a) against the Fluid user API to the wei, (b) smart-leg token decomposition against getDexState arithmetic shown in the report, (c) one position at a 90d-old archive block. tsc/test/build green.
FWS3 — Fluid flows: operate, NFT transfers, state-diff liquidations
Goal: the flow ledger captures every Fluid boundary crossing at block precision, and liquidations are realized losses, never flows.
- Migration 044 — flow-leg discriminator. One Fluid LogOperate can carry a collateral delta AND a debt delta (one log → two ledger rows), and one NFT transfer moves multiple legs under one log. The current PK (chain_id, wallet, tx_hash, log_index) cannot hold them. Add a
legcolumn (text, NOT NULL, DEFAULT '') and swap the PK to (chain_id, wallet, tx_hash, log_index, leg). This is NOT plain expand/contract — treat it as a coordinated change: the live writers use an explicit 4-columnON CONFLICT (chain_id, wallet, tx_hash, log_index)in TWO places —scripts/refreshers/portfolio.ts(writeFlows, ~line 205) andsrc/lib/portfolio/live.ts(writeFlowRows, ~line 118) — and Postgres rejects every flow insert for ALL venues the moment the old PK disappears. In the SAME PR: widen both conflict targets to includeleg, addlegtoFlowRow/FlowDetected(flows.ts) and to all three INSERT paths (the refresher, live.ts, andwriteBackfillFlowsin backfill.ts — the backfill's windowed delete+insert has no conflict target and tolerates the change). Write the PK swap inside a guardedDO $$block (check pg_constraint for the 5-column PK before swapping) so re-runs are no-ops per the migrate.sh contract. Flag prominently in the PR body and docs/database.md that a CODE ROLLBACK after 044 is applied leaves the previous release's flow writers hard-broken (accepted, staging-first; the alternative two-release sequencing was considered and rejected as not worth the delay for a staging-gated feature). Migration 044 ALSO creates thefluid_event_logcache table per D7 (same GRANTs discipline), and this PR ships its two writers (deploy-time seed script + cron append) and points the backfill and JIT readers at it. - Scanner (flows.ts sibling, own cursor scopes
portfolio:events:fluid-operate,portfolio:transfers:fluid-nft):- Chain-wide LogOperate by topic0 (~120–150/day measured, point-in-time samples), decode all, filter by the tracked NFT set. The set is derived per scan run, no new table: (a) distinct fluid position_keys already in portfolio_position_snapshots, (b) positionsNftIdOfUser at the scan anchor, (c) factory Mint/Transfer logs for tracked wallets WITHIN the scan window — so an NFT acquired or shed mid-window is still matched. Signed colAmt/debtAmt → deposit/withdraw/borrow/repay rows; smart-leg amounts are SHARES and must be valued via
getDexStateat the FLOW block (archive read; composition drifts). Per-tx netting per group already handles loops. - Factory ERC-721 Transfers (wallet-filterable): user↔user transfers emit per-leg rows per M16 (collateral transfer_out/in; debt as repay/borrow); mints/burns skip (the same-tx LogOperate carries the value).
- JIT mini-scan (live.ts): factory Transfer scan for the wallet (wallet-filterable, cheap) plus a chain-wide LogOperate scan over the mini-window since the wallet's last snapshot (≤ ~1800 blocks) filtered in-process against positionsNftIdOfUser's ids — LogOperate cannot be wallet-filtered at the node.
- Chain-wide LogOperate by topic0 (~120–150/day measured, point-in-time samples), decode all, filter by the tracked NFT set. The set is derived per scan run, no new table: (a) distinct fluid position_keys already in portfolio_position_snapshots, (b) positionsNftIdOfUser at the scan anchor, (c) factory Mint/Transfer logs for tracked wallets WITHIN the scan window — so an NFT acquired or shed mid-window is still matched. Signed colAmt/debtAmt → deposit/withdraw/borrow/repay rows; smart-leg amounts are SHARES and must be valued via
- State-diff liquidations (M15). In each scan window, for vaults with a LogLiquidate: diff every tracked NFT on that vault across the window anchors; a col/debt decrease with no matching LogOperate for that nftId is a liquidation. Synthesize ONE realized-loss row valued as equity destroyed (Δcol value − Δdebt value at the anchors), provenance = the last LogLiquidate tx_hash/log_index on that vault in the window, leg per M15, position_key = the NFT group prefix. Multiple partial liquidations inside one window legitimately collapse into one row (document). Booked-once enforcement is in
seizedKeysByInterval(prefix-match for fluid keys, per M15) —isLiquidationMechanicneeds NO fluid extension (verify, don't extend); test with a smart-col + smart-debt fixture asserting the loss lands exactly once and no 'value' leg books phantom yield. - Backfill + repair: extend
backfill-portfolio-wallet.tsto the fluid venue: per 6h grid anchor, resolver reads (archive-safe to 2025-11-26, far beyond the 90d window); flows and liquidation triggers read from thefluid_event_logcache (D7) — a registration backfill NEVER re-scans the chain for Fluid events; factory transfers stay a direct wallet-filtered getLogs (cheap). Same windowed delete+insert repair semantics, same stale-running reclaim. Probe correction: the existing wallet-filtered probe sweep structurally cannot see LogOperate (zero indexed params) — the fluid probe uses factory events only (NewPositionMinted hasuserindexed; ERC-721 Transfer is wallet-filterable) for the empty decision and first-activity block; the cache is consulted only when the probe shows the wallet owns/owned NFTs. The one-time ~650-chunk chain-wide scan exists only as the deploy seed (D7) and as the guarded fallback for a window predating the seed's coverage start.
Acceptance: deterministic historical proofs, all shown with psql output: (a) a real T1 deposit+borrow tx produces exactly two correctly-signed flow rows under one log_index with distinct legs; (b) a real smart-leg operate valued at the flow block's getDexState (show the arithmetic); (c) the real vault-93 liquidation (block 25,493,556, tx 0xf87d…c545, NFT 9266) classified as ONE liquidation row with equity-destroyed value, and NOT as withdraw/repay; (d) idempotent re-scan; (e) a 5-day backfill of a real Fluid whale, re-run byte-identical.
FWS4 — Surfacing: quoted rates, labels, UI, docs
quoted-rates.tsfluid branch: T1 supply =fluid_ll_apysupply APY of the leg token (+ wrapper token_yield APY composed, both sides, matching the realized side); T1 debt = borrow APY (+ wrapper); smart legs = the pool'sfluid_dex_apyfee APY + LL leg rates, labeled approximate. Concrete wire changes (name them, don't discover them):QuotedRateTablesgainsfluidLl: Map<token, {supply, borrow}>andfluidDex: Map<pool, feeApy>;api-data.loadQuotedRatesloads both tables; the APIPositionRowgains the approximate-rate marker surfaced through api-types.ts.- Labels/symbols for fluid legs (
wstETH supply · Fluid #16, smart legs asUSDe-USDT LP (col)); "Outside the yield book" reasons already generic. Wound-down vaults annotated in the positions table (from carry_registry status), never hidden. - UI states: nothing new structurally (books, marks, wedge, events feed all inherit); verify a Fluid-heavy fixture renders sanely at the three shell-zoom breakpoints; remove/adjust the "Fluid not yet tracked" gap-notice if one shipped in the interim.
- Docs: database.md (leg column), data-pipeline.md (fluid scanner scopes + volumes + drpc pacing), metrics.md (M11–M16), portfolio.md (venue coverage table), deployment.md if any env var (paid dRPC key, D3) is introduced.
Acceptance: full staging-style walkthrough with a fixture wallet holding Aave + Fluid T1 + smart positions: books correct, earned-vs-advertised populated for fluid legs, wedge sane, docs build green.
FWS5 — Cross-validation and hardening (Dune, invariants)
- Dune reconciliation (dev-time, once). Budget ≤200 credits, inspect query cost before executing (house rule). Clone q7490982 parameterized over ~10 NFTs we track (mix of T1/T2/T4, include NFT 9266 with its liquidation); compare its daily net/PnL series against our engine's book curves. Expected discrepancies to EXPLAIN, not paper over: Dune's per-NFT event-sum misses per-NFT liquidation effects (our resolver reads are truth — for NFT 9266 we should DISAGREE with Dune by exactly the seized amounts); daily vs 6h granularity; end-of-day vs flow-block valuation. Write the reconciliation note into docs/metrics.md.
- Invariant sweep: the FWS1–3 additions must keep the global invariant (attributedYield == ΔbookValue − netFlow, redemption mark) green across the new accrual paths — extend the property test with a fluid T1 leg, a fluid smart 'value' leg, and a post-fix PT leg (entry at discount).
- Alert coverage: unknown Fluid tokens with nonzero value hit the WS8 unknown-asset alert; matured-PT valuation fallback logs distinctly.
4. Decisions taken (defaults; flag in PR if changed)
| # | Decision |
|---|---|
| D1 | PT entry basis is derived at read time from the flow ledger (no new table). If read-time derivation proves too slow at scale, add a small cache table in a follow-up — do not pre-optimize. |
| D2 | Fluid tracks ALL vaults a wallet touches (self-describing reads), including wound-down and non-screener vaults; carry_registry is annotation only. Anything else recreates the silent-invisibility trap. |
| D3 | RESOLVED (Fred, 2026-07-10): no paid dRPC key. With the seeded event cache (D7) the chain-wide getLogs scan runs exactly ONCE (the deploy-time seed, ~650 chunks, attended, free-tier pacing is fine). Remaining archive load is ordinary per-wallet eth_calls at grid anchors, which the free tier served reliably in testing. A paid key stays a break-glass option if free-tier reliability degrades. |
| D4 | Legacy vaults 1/2/4/5 (stale tick data in the RISK layer): positionByNftId per-position reads are expected to be correct there, but the executor MUST verify one real position on a legacy vault before trusting it; if wrong, exclude those four vaults with an honest annotation. |
| D5 | T3 smart-debt share semantics: VERIFIED during plan verification — T3 vault 46 (0x221e35b5655a1eeb3c42c4defc39648531f6c9cf, vaultType 30000), tx 0x9acafaf889510302873400788d6502d306c3ed82b034ef5bcc173eb924206ce2 (block 25,497,638) paid the user exactly 46.000000 USDT while LogOperate debtAmt = 20.66e18 shares; shares × getDexState words 28/29 at that block ≈ $45.98. 1e18 borrow-share semantics confirmed. FWS3 re-runs this exhibit as part of acceptance. |
| D6 | Fluid user_ in LogOperate is never used for anything. NFT ownership comes exclusively from factory events + resolver owner field (positions held via DSA proxies stay invisible unless the proxy is the registered wallet — same policy as the rest of the portfolio, M8). |
| D7 | RESOLVED (Fred, 2026-07-10): BUILD the decoded-event cache in FWS3. New table fluid_event_log in migration 044: (chain_id, tx_hash, log_index) PK, block_number, ts, vault, kind ('operate' | 'liquidate'), nft_id (NULL for liquidate), col_amt numeric (signed raw), debt_amt numeric (signed raw). It stores ALL Fluid vault events, not just tracked NFTs — future signups need arbitrary NFTs. Writers: (1) a one-time deploy seed script scans the trailing 90 days chain-wide and fills it (attended, free-tier pacing, listed as a server step in the FWS3 PR body); (2) the 6h cron scanner appends what it already decodes (same cursor, same 64-block safety margin, idempotent on the PK). Readers: registration backfills and the JIT mini-scan query THIS TABLE for operate/liquidate history and never re-scan the chain; the seed's coverage start is recorded so a backfill window predating it (impossible after day one, guarded anyway) falls back to a chain scan with a loud log line. ~150 events/day, ~55k rows/yr — negligible. |
| D8 | FWS2 ships with fluid legs gated to "Outside the yield book" (coverage-pending); FWS3 flips the gate. Staging never charts a fluid 'value' leg without flow coverage. |
5. Explicitly out of scope
Multichain Fluid (mainnet only, like everything else); per-NFT exact DEX fee attribution beyond the value-series capture (the share-value growth already embeds fees; a separate fee line item is v3); Fluid rewards/points (M8); pre-2025-11-26 Fluid history (resolver floor; irrelevant inside the 90d window); Dune as anything but a dev-time oracle; execution.
6. Risks
- Resolver ABI drift: Fluid ships versioned resolvers (the API was observed using
0x814c8C7c…while we pin0xA5C3E165…). Pin ours, but decode via ABI not word offsets where possible, and let the FWS2 acceptance catch drift. - getDexState per-flow archive reads add one archive call per (DEX, flow block); volumes are trivial today (~141 operate/day chain-wide, of which smart-leg a fraction) but the scanner must batch and pace.
- classifyLegs group-key width: smart-leg keys have SEVEN colon-parts (T1 keys have six); groupKeyForLeg slices the first five, so neither shape splits a group — pinned by an FWS2 test asserting against the real 7-part shape.
- PT TWAP null windows (young markets): both marks can be legitimately null → gaps render as gaps (M9), and the entry-basis derivation must skip unvalued flows rather than assume par.