Sydnee nightly — persistence audit — 2026-05-28 P0 findings: 0 P1 findings: 1 Risks: 3 - Area: Persistence - Bugs found (P0 / P1): 0 / 1 - Risks noted: 4 --- Full report (dev branch): https://github.com/kanex1/sydnee.signals/blob/dev/docs/audit_2026-05-28_persistence.md Reply FROM [email protected] to [email protected] to request fixes, e.g.: "code_task on sydnee-signals-dev: apply fix for the P0 about RVOL threshold in bot.py" Sydnee Agent will propose + you APPROVE (or plain 'approve') + auto-push to dev. --- Full audit below (first 12 KB) --- # Nightly Audit 2026-05-28 — PERSISTENCE ## Summary - Area: Persistence - Bugs found (P0 / P1): 0 / 1 - Risks noted: 4 --- ## Findings ### BUG [P1]: `_tv_confirm_state` not persisted + `tv_bxt_signal` has no TTL — stale signal retried after restart **File:** `bot.py:4020-4053` (confirm state), `bot.py:12245-12246` (signal set) **Evidence:** ```python # bot.py:12245-12246 — plain r.set(), no TTL r.set(f"tv_bxt_signal:{sym}", sig) r.set(f"tv_bxt_signal_ts:{sym}", str(int(time.time()))) # bot.py:4020-4037 — retry counter is in-memory only if not hasattr(self, "_tv_confirm_state"): self._tv_confirm_state = {} # ← resets to {} on every restart _st = self._tv_confirm_state.get(sym, {}) if _st.get("sig_ts") != _sigts: _st = {"sig_ts": _sigts, "attempts": 0, "last_check": 0.0} # ← resets to 0 _st["attempts"] = _st.get("attempts", 0) + 1 self._tv_confirm_state[sym] = _st if _st["attempts"] >= 5: self._tv_bxt_signal_clear(sym) # abandon after 5 ... return ``` `_tv_confirm_state[sym]` is a per-symbol dict tracking how many flow-confirmation retries have been attempted for the current TV-BXt signal. It lives entirely in memory — no Redis persist, no DB write. The `tv_bxt_signal:{sym}` Redis key is set via `r.set()` with no TTL, so it persists indefinitely until `_tv_bxt_signal_clear` is called (on position close or after 5 failed attempts). **Failure path on restart during a flow-confirmation retry:** 1. TV webhook fires → `tv_bxt_signal:{sym}` set in Redis (no TTL). 2. Flow not confirmed; `_tv_confirm_state[sym]` shows `attempts=3/5`. 3. Bot restarts (crash, redeploy, health-check eviction). 4. `tv_bxt_signal:{sym}` still in Redis — no expiry. 5. `_tv_confirm_state` is `{}` (reset). The `sig_ts` check re-initializes to `attempts=0`. 6. Fresh 5-attempt window begins. A signal that should have been abandoned after 2 more retries now gets 5 more. 7. If the WS sidecar is slow to reconnect on restart (`_ws_age > 5000ms`), the `_fresh = False` **fail-open** path bypasses flow confirmation entirely and triggers an immediate entry on a signal that may be 5–15 min old. **Impact:** On any restart during trading hours while a TV-BXt flow retry is in progress, the bot may enter on a stale signal. The fail-open path (stale WS on startup) is the critical sub-case — it short-circuits all confirmation logic, so even a signal that never confirmed flow (e.g., because it was a blowoff move) gets an immediate entry attempt on restart. **Fix (two changes):** ```python # 1. bot.py — give tv_bxt_signal keys a TTL (at signal-set site, bot.py:12245): r.setex(f"tv_bxt_signal:{sym}", 900, sig) # expire after 15 min r.setex(f"tv_bxt_signal_ts:{sym}", 900, str(int(time.time()))) # 2. bot.py — persist _tv_confirm_state to Redis (at update site, bot.py:4037): self._tv_confirm_state[sym] = _st if self._regime_redis: try: self._regime_redis.setex(f"tv_confirm:{sym}", 600, json.dumps(_st)) except Exception: pass # 3. bot.py — restore on startup (after Redis init, ~bot.py:790): if not hasattr(self, "_tv_confirm_state"): self._tv_confirm_state = {} if self._regime_redis: for _sym in self._tv_bxt_active_syms(): _raw = self._regime_redis.get(f"tv_confirm:{_sym}") if _raw: try: self._tv_confirm_state[_sym] = json.loads(_raw) except Exception: pass ``` Either fix alone significantly reduces the risk: the TTL prevents stale signals from surviving multi-minute restarts; the state persistence preserves the attempt counter so a 4/5 signal isn't retried fresh. --- ### RISK: `reconcile_import` positions now TV-managed without OBV trail state **File:** `bot.py:3907291 commit` (`_eval_tv_bxt_sym`), `bot.py:3735-3790` (`_obv_trail_stop_hit`) **Evidence:** ```python # 3907291 — my_pos now includes reconcile_import (fresh today) my_pos = next((t for t in open_here if self._is_tv_bxt_trade(t) or getattr(t, "strategy_reason", "") == "reconcile_import"), None) # _obv_trail_stop_hit — initializes OBV trail if _obv_peak not present # bot.py:3753-3770 r = getattr(self, "_regime_redis", None) if r is not None and not getattr(my_pos, "_obv_peak", None): _ot = r.get(f"obv_trail:{my_pos.trade_id}") # → None for reconcile_import (never set by _obv_trail_stop_hit before) ``` A reconcile_import trade was never processed by `_obv_trail_stop_hit`, so no `obv_trail:{trade_id}` key exists in Redis. When the TV engine now inherits such a position as `my_pos`, the first call to `_obv_trail_stop_hit` initializes `_obv_peak` and `_obv_stop` from the current price — not from the actual price action since entry. For a long that ran +2% before the bot restarted, the OBV trail starts from the top (fair). For a long that ran up and then gave back, the trail floor misses the true peak — the stop is set too tight or too loose depending on direction. **Impact:** The first OBV trail evaluation on a reconcile_import position uses a fresh start rather than accumulated history. Low probability of causing a stop-out in one bar, but the trail won't tighten correctly until a new peak is established post-adoption. For a TV position that was deep in profit pre-restart, this could allow more drawback before the trail fires. **Fix:** In `_eval_tv_bxt_sym`, after setting `my_pos` for a reconcile_import trade, try to restore OBV trail from Redis using the same logic as `_load_trades_from_db:1170-1192` (which already handles `obv_trail:{trade_id}` restore). Alternatively, skip OBV trail exit management for `reconcile_import` positions on the first bar and let the position fall through to price-stop logic only. --- ### RISK: `_fri_flat_queued` not persisted — RTH force-flat dedup missed (CARRIED OVER from 2026-05-22) **File:** `bot.py:4334-4339` **Evidence:** ```python if getattr(t, '_fri_flat_queued', False): continue # ← in-memory only; no _update_trade_field t._fri_flat_queued = True self._deferred_closes.append((t, ...)) ``` Contrast: sister flag `_fri_ah_close_queued` was persisted in 8845a24 (audit 2026-05-21 R4). On restart during the 3:50 PM ET Friday close window, the dedup guard resets and a second close is enqueued. The `is_open` guard in `_close_trade` provides secondary protection if the first fill processed. Unchanged from 2026-05-22 filing; narrow Friday-only window; no financial harm on paper. **Fix (identical to `fri_ah_close_queued` pattern):** ```python # core/database.py ensure_trade_state_columns — add: "ALTER TABLE trades ADD COLUMN IF NOT EXISTS fri_flat_queued BOOLEAN DEFAULT FALSE", # bot.py:4339 — add: self._update_trade_field(t.trade_id, fri_flat_queued=True) # bot.py _load_trades_from_db — add alongside fri_ah_close_queued: if row.get("fri_flat_queued"): t._fri_flat_queued = True ``` --- ### RISK: `_bxt1m_last_bar` not persisted — 1m BXt dedup replays on restart (CARRIED OVER from 2026-05-22) **File:** `bot.py:3891-3892`, `bot.py:4149-4155` **Evidence:** ```python if not hasattr(self, "_bxt1m_last_bar"): self._bxt1m_last_bar = {} # ← always empty on restart ... if self._bxt1m_last_bar.get(sym) == idx[-1]: return # dedup skips already-processed bar self._bxt1m_last_bar[sym] = idx[-1] # ← in-memory only ``` On restart, the dedup guard resets and the most recent 1m bar is re-evaluated. If `desired_dir == 0` (5m trend not aligned) at the moment of restart, an open BXT1m position is force-closed prematurely. Unchanged from 2026-05-22 filing. Gated behind `BXT_1M_NAKED_SYMBOLS` env var (empty by default = off on both dev and prod). Risk dormant if `BXT_1M_NAKED_SYMBOLS` is unset. **Fix (from 2026-05-22):** Persist `last_ts` to Redis with `setex(f"bxt1m_last_bar:{sym}", 120, str(last_ts))`; restore on init. --- ## OK (checked, working or recently fixed) **New since 2026-05-22 — TV-BXt feature commits reviewed:** - **`tv_bxt_signal` set/clear logic** — `_tv_bxt_signal_clear` correctly deletes both `tv_bxt_signal:{sym}` and `tv_bxt_signal_ts:{sym}` on close and on entry (line 3937, 3967). Deletion side correct; missing TTL filed as P1 above. - **`_tv_block_logged` dict** — not persisted, but purely cosmetic (dedup for a dashboard log line). On restart, at most one extra "manual position blocked" log per signal. No trading impact. ✓ (RISK too low to file) - **Flow-confirm `_ratio` / `_flow_min` / `_vol_min` checks** — purely computed from live WS overlay on each call; no per-trade state, no persistence needed. ✓ - **WS-touch marketable limit order kwargs** — `_close_order_kwargs` is a runtime attribute set on the Trade object in `_eval_tv_bxt_sym`. Not persisted. On restart, the trade reverts to market-order close. Acceptable: limit close is an optimization, not a correctness requirement. ✓ - **Spread gate `ws_spread_bps`** — read live from `_ws_live_overlay` on each entry evaluation; no cached state to persist. ✓ - **`reconcile_import` quarantine guard raised 25%→55% NLV** (`fix(reconcile)` 62f9195) — env-var math, computed fresh at each reconcile tick; no state. ✓ **From 2026-05-22 audit — all still OK:** - All 7 items fixed in 8845a24 (oracle TTL, consec_losses, tick_peak, tick_fire_ts, fri_ah_close_queued, signal_price/time restore) remain correct. ✓ - `_fri_flat_queued` and `_bxt1m_last_bar` remain open risks (carried above). Status unchanged. **Long-standing — re-confirmed no regression:** - `trimmed_1/2/3`, `eod_trimmed`, `obv_be_applied`, `extreme_since_trim`, `last_eod_trim_time`, `last_ah_trim_time`, `pending_reversal`, `force_trim_pending/2`, `force_trim_reason`, `tick_peak_signed_vol`, `fri_ah_close_queued`, `signal_price/time` — all correctly persisted to DB and restored in `_load_trades_from_db`. ✓ - OBV trail (`obv_trail:{trade_id}`) persisted to Redis with 7-day TTL (604800s); restored at startup for genuinely TV-opened trades. ✓ - `_pending_manual_orders` persisted to Redis hash; restored on restart; stale-cleanup patrol uses Redis inventory to skip known manual orders. ✓ - Bar snapshots (`bars:{sym}:{tf}`) saved to Redis every 5m via `_save_bars_to_redis` (86400s TTL); restored in `_load_bars_from_redis`. ✓ - `_deferred_closes` drain deduplicates by `trade_id` + `is_open` guard; double-close risk contained. ✓ - `ensure_trade_state_columns()` runs at `__init__` (line 733); idempotent `ADD COLUMN IF NOT EXISTS`. New columns from all recent audits confirmed present in migration. ✓ - `exit_reason` empty for FLAT_EXIT parent and reconcile closes — filed as RISK in 2026-05-26 exit_flow audit; no change since then. ✓ - `vix_at_entry/exit`, `position_size_pct` always 100.0, `_setup_mode` — analytics-only, carried from prior audits. ✓ - OBV divergence swing-point history — dormant (`OBV_DIVERGENCE_ENABLED=false`). ✓