Sydnee nightly — PERSISTENCE audit — 2026-08-13 P0 findings: 0 P1 findings: 4 Risks: 7 - Area: Persistence (Thursday theme) - Branch: `dev` (cda5ab3 — docs(peak-monitor): hourly log 2026-08-12 13:09 PT) - Files scanned: `bot.py` (24,173 lines), `core/database.py` (2,553 lines), `core/risk.py` (150 lines), `core/persistence.py` (153 lines), `core/schema.sql` (104 tables), `docs/strategy_decisions.md`, `docs/audit_2026-08-06_PERSISTENCE.md`, `git log --oneline -30` - **Bugs found (P0 / P1): 0 / 4 (all carried — no new bugs)** - **Risks noted: 7 (all carried; 2 corrections to prev audit code excerpts noted below)** **Cross-check:** `git log --oneline --since=2026-08-06` returns zero results for `bot.py`, `core/database.py`, `core/risk.py`, `core/persistence.py`. All commits since 2026-08-06 are audit docs + peak-monitor hourly logs. All 4 P1 bugs and all 7 risks from the 2026-08-06 PERSISTENCE audit re-verified at current line numbers — all present and unchanged. **Two corrections to 2026-08-06 audit excerpts (no code changes — prior auditor misquoted):** - `_safe()` (database.py:82) logs at `logger.warning`, NOT `logger.debug` as stated in the 2026-08-06 RISK. Partially reduces RISK-4 (write swallowing is still silent in terms of propagation, but operator-visible at warning level). - `_update_trade_field` (bot.py:1313) uses `setattr(t, k, v)`, NOT the `setattr(t, t, v)` typo claimed in RISK-3. The `hasattr` gate risk still applies (non-dataclass flags skip in-memory update) but the typo was never in the code. --- Full report (dev branch): https://github.com/kanex1/sydnee.signals/blob/dev/docs/audit_2026-08-13_PERSISTENCE.md Reply FROM [email protected] to [email protected] to request fixes, e.g.: "code_task on sydnee-signals-dev: apply fix for the P0 about RVOL threshold in bot.py" Sydnee Agent will propose + you APPROVE (or plain 'approve') + auto-push to dev. --- Full audit below (first 12 KB) --- # Nightly Audit 2026-08-13 — PERSISTENCE ## Summary - Area: Persistence (Thursday theme) - Branch: `dev` (cda5ab3 — docs(peak-monitor): hourly log 2026-08-12 13:09 PT) - Files scanned: `bot.py` (24,173 lines), `core/database.py` (2,553 lines), `core/risk.py` (150 lines), `core/persistence.py` (153 lines), `core/schema.sql` (104 tables), `docs/strategy_decisions.md`, `docs/audit_2026-08-06_PERSISTENCE.md`, `git log --oneline -30` - **Bugs found (P0 / P1): 0 / 4 (all carried — no new bugs)** - **Risks noted: 7 (all carried; 2 corrections to prev audit code excerpts noted below)** **Cross-check:** `git log --oneline --since=2026-08-06` returns zero results for `bot.py`, `core/database.py`, `core/risk.py`, `core/persistence.py`. All commits since 2026-08-06 are audit docs + peak-monitor hourly logs. All 4 P1 bugs and all 7 risks from the 2026-08-06 PERSISTENCE audit re-verified at current line numbers — all present and unchanged. **Two corrections to 2026-08-06 audit excerpts (no code changes — prior auditor misquoted):** - `_safe()` (database.py:82) logs at `logger.warning`, NOT `logger.debug` as stated in the 2026-08-06 RISK. Partially reduces RISK-4 (write swallowing is still silent in terms of propagation, but operator-visible at warning level). - `_update_trade_field` (bot.py:1313) uses `setattr(t, k, v)`, NOT the `setattr(t, t, v)` typo claimed in RISK-3. The `hasattr` gate risk still applies (non-dataclass flags skip in-memory update) but the typo was never in the code. --- ## Findings ### BUG [P1] (carried ×3 persistence audits, unfixed since 2026-07-14): `exit_reason` blank in DB — crash window between `_close_trade_in_store` and exit-reason write **File:** `bot.py:1383` (`_close_trade_in_store`); call sites `bot.py:8795`, `8900`, `8921`, `13279`, `2515`, `2528`, `5758` **Evidence:** ```python # bot.py:1380-1384 — _close_trade_in_store() always passes exit_reason="": self.db.close_trade( trade_id=trade_id, exit_price=exit_price, exit_time=datetime.now(ET).isoformat(), pnl=pnl, exit_reason="", ) # bot.py:8795-8798 — main exit path: reason written in a separate update_trade() call: pnl = self._close_trade_in_store(trade.trade_id, exit_price) self.db.update_trade(trade.trade_id, exit_reason=reason, ...) # bot.py:2515, 2528 — reconcile path: close_trade_in_store() with NO follow-up # update_trade(exit_reason=...) call at all — exit_reason is always "" for these. self._close_trade_in_store(t.trade_id, exit_px) ``` For the main exit path (lines 8795–8928), a crash between `_close_trade_in_store` and the subsequent `update_trade(exit_reason=...)` leaves the row permanently blank. For the reconcile path (lines 2515, 2528) there is no follow-up at all — those trades are always closed with `exit_reason=""` regardless of crash. **Impact:** All `CASE WHEN exit_reason ~* 'TRIM'` analytics queries misclassify reconcile-driven closes. Any crash in the exit path produces the same result for main-loop closes. **Fix:** Pass `exit_reason` as a parameter to `_close_trade_in_store()` and include it in the `close_trade()` call. Eliminates the two-step write and the race window. --- ### BUG [P1] (carried ×3 persistence audits, unfixed since 2026-07-14): `RiskManager.DailyState` (`trade_count`, `realized_pnl`) not restored on restart — daily caps reset to zero **File:** `bot.py:23458–23461` (startup restore); `core/risk.py:29` **Evidence:** ```python # core/risk.py:29-34 — DailyState always starts zeroed: @dataclass class DailyState: date: date = field(default_factory=date.today) realized_pnl: float = 0.0 trade_count: int = 0 open_positions: int = 0 # bot.py:23458-23461 — startup only calls set_open_positions(): open_count = len(self.open_trades()) if open_count > 0: self.risk.set_open_positions(open_count) logger.info("Restored %d open positions", open_count) # No restore of trade_count or realized_pnl from DB # core/risk.py:112-120 — caps that rely on these values: if self.state.realized_pnl <= -self.cfg.daily_loss_limit: return False, "Daily loss limit hit: ..." if self.state.trade_count >= self.cfg.max_daily_trades: return False, "Max daily trades reached: ..." ``` `RiskManager` has no `restore_daily_state()` method. After any restart mid-session, `trade_count` resets to 0 and `realized_pnl` resets to 0.0. **Impact:** Real-money risk. A bot at 10/10 max daily trades that restarts places new trades freely. A bot past the daily loss limit bypasses the circuit breaker for the remainder of the session. DB has all necessary data to restore via `SELECT COUNT(*), SUM(pnl) FROM trades WHERE trade_date = today AND exit_time IS NOT NULL`. **Fix:** Query completed trades for today at startup and call `self.risk.record_exit(pnl)` for each closed trade, or add `restore_daily_state(trade_count, realized_pnl)` to `RiskManager` and call it during `start()`. --- ### BUG [P1] (carried ×3 persistence audits, unfixed since 2026-07-14): `_ab_live_state` memory-only — Asian Breakout executor loses all state on restart, pending force-flat never fires **File:** `bot.py:20237–20238` **Evidence:** ```python # bot.py:20237-20238 — lazy-init, no DB restore: if not hasattr(self, "_ab_live_state"): self._ab_live_state = {} ``` `_ab_live_state` is a dict keyed by symbol holding bracket metadata (`side`, `qty`, `entry`, `stop`, `target`, `bracket_id`, `state`). On restart it is empty. The force-flat path (`bot.py:20126–20140`, `_ab_close_market`) reads `_ab_live_state[sym]` to construct the cancel+market-close order — with empty state the force-flat silently skips. The AB position continues as an untracked IBKR position. **Impact:** Live AB positions (low — `AB_EXECUTE` dev-paper only) become orphaned on restart. Real risk if `AB_EXECUTE` is promoted to production. **Fix:** Persist `_ab_live_state` to DB or Redis on each state change; restore at startup from `asian_breakout_signals` table (schema entry created via `ensure_trade_state_columns()` scope per prior audits). --- ### BUG [P1] (carried ×3 persistence audits, unfixed since 2026-07-14): AB bracket fills and closes not written to DB **File:** `bot.py:20109–20140` (`_ab_place_bracket`, `_ab_close_market`) **Evidence:** ```python # bot.py:20109-20124 — _ab_place_bracket(): places IBKR bracket, no DB write # bot.py:20126-20140 — _ab_close_market(): places IBKR market close, no DB write # Both functions only read/write _ab_live_state (memory) ``` Neither `_ab_place_bracket()` nor `_ab_close_market()` calls `db.insert_trade()` or `db.close_trade()`. AB P&L is invisible to `realized_pnl` in `RiskManager.DailyState` (compounding P1 bug #2 above — daily loss limit ignores AB losses). **Impact:** Risk double-counting (risk manager unaware of AB position size) + analytics blind spot. **Fix:** Insert a trade row on bracket fill confirmation; close it on `_ab_close_market`. --- ## Risks ### RISK (carried from 2026-08-06): `_tv_scale_state` memory-only — TV-BXt active scale-in window abandoned on restart **File:** `bot.py:4378–4379` (init); `bot.py:4144–4158` (scale logic) ```python # bot.py:4378-4379 — lazy-init, no Redis restore: if not hasattr(self, "_tv_scale_state"): self._tv_scale_state = {} ``` On restart during an active 10-slot (600s) scale-in window, `_tv_scale_state` is empty, `_scale_active=False` causes early return — remaining slots abandoned silently. No duplicate orders placed; parent trade continues normally. **Impact:** Position undersized if restart occurs during the 600s scale-in window. Silent — no log message on abandoned-slot path. --- ### RISK (carried since 2026-07-14): `TradeStore._load_today()` only loads today's date file — silently loses overnight-open trades **File:** `core/persistence.py:80–88` ```python def _load_today(self) -> None: path = self._file_for_date() # always date.today() ... ``` `TradeStore` is not used by `bot.py` (zero imports), so this is latent. If any future consumer uses it, positions opened before midnight are silently absent after the date rolls. --- ### RISK (carried since 2026-07-14, code excerpt corrected): `_update_trade_field` `hasattr` gate skips non-dataclass attribute updates in memory **File:** `bot.py:1306–1318` ```python def _update_trade_field(self, trade_id: str, **kwargs) -> None: with self._trades_lock: for t in self._trades: if t.trade_id == trade_id: for k, v in kwargs.items(): if hasattr(t, k): # ← skips attrs not on Trade dataclass setattr(t, k, v) # (correctly uses k, not t — prior audit had typo) break self.db.update_trade(trade_id, **kwargs) ``` The `hasattr` gate means DB and in-memory state diverge for any field not declared on the `Trade` dataclass. DB is updated correctly; in-memory `Trade` object is not. **Note:** The `setattr(t, t, v)` typo claimed in the 2026-08-06 audit does NOT exist in the current code — current code correctly uses `setattr(t, k, v)`. That part of the prior risk was a misquote. --- ### RISK (carried since 2026-07-14, log level corrected): `_safe()` swallows all write exceptions — DB failures are `warning` logged only, never re-raised **File:** `core/database.py:77–87` ```python def _safe(self, func, *args, default=None, **kwargs): try: return func(*args, **kwargs) except Exception as e: logger.warning("DB operation failed: %s", e) # WARNING level (not debug) try: self._local.conn = None except Exception: pass return default ``` All write operations (`insert_trade`, `close_trade`, `update_trade` at lines 127, 138, 146) go through `_safe()`. Failures return `None` and continue execution. **Note:** The 2026-08-06 audit stated `logger.debug` — current code uses `logger.warning`, which is operator-visible. The core risk remains: DB divergence accumulates without exception propagation or a metric counter. **Remaining fix:** Re-raise on critical write operations (`insert_trade`, `close_trade`, `update_trade`) so callers can detect and surface DB outages. --- ### RISK (carried since 2026-07-14): Schema drift — 7 columns and 7 tables in runtime `ensure_*` methods are absent from `core/schema.sql` **File:** `core/database.py:222` (`ensure_trade_state_columns`), `core/database.py:364` (`ensure_scalp_signal_log_table`); `core/schema.sql` Columns in `ensure_trade_state_columns()` not in `schema.sql` trades table: `obv_be_applied_at`, `order_lmt_price`, `tape_metadata`, `tick_peak_signed_vol`, `fri_ah_close_queued`, `fri_flat_queued` (6 of 7 from prior audit — `order_type` is in `orders` table in schema.sql, not in `trades`; confirmed absent from `trades` definition via grep). Tables created at runtime but absent from `schema.sql`: `nlv_snapshots`, `rrod_signal_log`, `asian_breakout_signals`, `stock_strategy`, `scalp_signal_log`, `futures_bars`, `futures_l2_snapshots` A fresh `psql -f schema.sql` deploy misses all of these. DR restore from schema alone silently fails on first write to missing columns/tables. --- ### RISK (carried since 2026-07-14): `_deferred_closes` memory-only — crash between trim-flag persist and order submit loses deferred close **File:** `bot.py:23276` ```python self._deferred_closes: list[tuple] = [] ``` Plain list, no Redis or DB backing. 15+ call sites append to this list (trim-1/2/3, Friday EOD, Friday AH, reversal, etc.). If the process crashes after persisting a trim flag to DB but before `_process_deferred_orders()` drains the list, the close order is lost — position remains open with trim flags set, misleading analytics. --- ### RISK (carried since 2026-07-14): `insert_bar()` omits `source` column **File:** `core/database.py:1110–1119` `insert_bar()` inserts into `bar_data (symbol, timeframe, timestamp, open, high, low, close, volume)` with no `source` field distinguishing IBKR vs Polygon bars. With sp