Sydnee nightly — PERFORMANCE audit — 2026-08-15 P0 findings: 0 P1 findings: 2 Risks: 13 - Area: Performance (Saturday theme) - Branch: `dev` (c930888 — docs(peak-monitor): hourly log 2026-08-14 13:10 PT) - Files scanned: `bot.py` (24,173 lines), `core/database.py` (2,553 lines), `futures_client.py`, `pages.py` (13,447 lines), `vianexus_client.py`, `docs/strategy_decisions.md`, `git log --oneline -30`, prior performance audits 2026-08-08, 2026-08-09 - **Bugs found (P0 / P1): 0 / 2 (both carried; 0 new)** - **Risks noted: 13 (1 new + 12 carried)** **Cross-check:** `git log --oneline -30` shows zero commits to `bot.py`, `core/`, `pages.py`, `strategy.py`, or any strategy/config file since 2026-07-17. All subsequent commits are docs-only (audit files + peak-monitor hourly logs). Both P1s confirmed unchanged at identical line numbers. 11 of 12 carried risks confirmed unchanged. 1 new risk identified: the 2026-08-09 audit's OK section stated `_regime_redis.decode_responses` was "not confirmed as True" — bot.py:803 confirms `decode_responses=True`, making the bytes branch in `_ws_live_overlay_ibkr` (bot.py:1535) and the `_dk` check in TAPE_SIG3_L2 (bot.py:18520) also permanently dead code. Both were incorrectly exempted from the carried bytes-check risk. --- Full report (dev branch): https://github.com/kanex1/sydnee.signals/blob/dev/docs/audit_2026-08-15_PERFORMANCE.md Reply FROM [email protected] to [email protected] to request fixes, e.g.: "code_task on sydnee-signals-dev: apply fix for the P0 about RVOL threshold in bot.py" Sydnee Agent will propose + you APPROVE (or plain 'approve') + auto-push to dev. --- Full audit below (first 12 KB) --- # Nightly Audit 2026-08-15 — PERFORMANCE ## Summary - Area: Performance (Saturday theme) - Branch: `dev` (c930888 — docs(peak-monitor): hourly log 2026-08-14 13:10 PT) - Files scanned: `bot.py` (24,173 lines), `core/database.py` (2,553 lines), `futures_client.py`, `pages.py` (13,447 lines), `vianexus_client.py`, `docs/strategy_decisions.md`, `git log --oneline -30`, prior performance audits 2026-08-08, 2026-08-09 - **Bugs found (P0 / P1): 0 / 2 (both carried; 0 new)** - **Risks noted: 13 (1 new + 12 carried)** **Cross-check:** `git log --oneline -30` shows zero commits to `bot.py`, `core/`, `pages.py`, `strategy.py`, or any strategy/config file since 2026-07-17. All subsequent commits are docs-only (audit files + peak-monitor hourly logs). Both P1s confirmed unchanged at identical line numbers. 11 of 12 carried risks confirmed unchanged. 1 new risk identified: the 2026-08-09 audit's OK section stated `_regime_redis.decode_responses` was "not confirmed as True" — bot.py:803 confirms `decode_responses=True`, making the bytes branch in `_ws_live_overlay_ibkr` (bot.py:1535) and the `_dk` check in TAPE_SIG3_L2 (bot.py:18520) also permanently dead code. Both were incorrectly exempted from the carried bytes-check risk. --- ## Findings ### BUG [P1] (carried × 7 performance audits, unfixed since 2026-07-11): `_generate_daily_summary()` uses in-memory `_build_performance()` — reports $0 P&L after any bot restart **File:** `bot.py:11681` (`perf = self._build_performance()`), `bot.py:11731` (`self.db.upsert_daily_summary(...)`), `bot.py:1156` (`def _load_trades_from_db`) **Evidence:** ```python # bot.py:1156 — startup loads only OPEN trades def _load_trades_from_db(self) -> None: db_trades = self.db.get_open_trades() # closed trades never loaded # bot.py:11681 — daily summary uses in-memory stats built from empty closed-trades cache perf = self._build_performance() today = perf["today"] # {pnl: 0, trades: 0, win_rate: 0} after restart # bot.py:11731-11737 — permanently writes wrong daily record to DB self.db.upsert_daily_summary( d=date.fromisoformat(today_str), pnl=today["pnl"], # ← $0 trade_count=today["trades"], # ← 0 ... ) ``` Also affected: `_agent_ledger()` (bot.py:9156) writes Redis `signals.ledger.state` with $0/0-trade values after every restart; `_ledger_eod_report()` (bot.py:9184) produces a $0 EOD summary; `_audit_scan()` win-rate regression detector (bot.py:2817–2828) silently disables post-restart until 30 trades accumulate in memory. **Impact:** Daily AI review at 4:15 PM ET receives "P&L: $0, Trades: 0, Win rate: 0%" context on any day with a bot restart. The `daily_summaries` row is permanently written with incorrect values. `signals.ledger` Redis state reflects $0 for up to 6 minutes after each restart. Win-rate regression alert is offline for the remainder of any session after a restart. **Fix:** Replace `_build_performance()` with the DB-backed path in `_generate_daily_summary()` — same pattern already used by `api_performance` (bot.py:14428): ```python today_str = datetime.now(ET).strftime("%Y-%m-%d") today = self.db.compute_performance(date_from=today_str) ``` --- ### BUG [P1] (carried × 7 performance audits, unfixed since 2026-07-11): FLAT_EXIT parent `pnl=0` counted as loss in `_compute_period_stats()` — win rate and total P&L understated **File:** `bot.py:8950–8951` (`_compute_period_stats`), `bot.py:5743–5760` (FLAT_EXIT parent close), `bot.py:1374` (`_close_trade_in_store` pnl calc) **Evidence:** ```python # bot.py:5743-5744: shares decremented BEFORE close-in-store remaining = t.shares - trim_shares # = 0 on full close self._update_trade_field(t.trade_id, shares=remaining) # t.shares → 0 if remaining == 0 and _flat_full_close: self._close_trade_in_store(t.trade_id, live_price) # bot.py:1374: pnl computed against already-decremented shares t.pnl = round((exit_price - t.entry_price) * t.direction * t.shares, 2) # t.shares=0 → pnl=0.0 even on a profitable trade # bot.py:8950-8951: losses includes pnl=0 parents wins = [p for p in pnls if p > 0] losses = [p for p in pnls if p <= 0] # ← pnl=0 lands here ``` **Impact:** With `FLAT_EXIT=true` (active on dev), every profitable mean-revert / extension-fade close appears as a $0 "loss" in in-session win-rate stats. `_generate_daily_summary()` and `daily_summaries` rows understate win_rate. `_audit_scan` win-rate regression (bot.py:2823–2828) may fire false alerts as $0 parents dilute the last-20 window's WR computation. **Fix:** Filter zero-pnl full-closed parents in `_compute_period_stats()`: ```python pnls = [t.pnl for t in trades if t.pnl is not None and not (t.pnl == 0.0 and t.exit_price is not None and t.shares == 0)] ``` Long-term: move all performance callers to the DB path (same root fix as P1 above). --- ## Risks ### RISK (NEW — first confirmed 2026-08-15): `_regime_redis` confirmed `decode_responses=True` — `_ws_live_overlay_ibkr` and TAPE_SIG3_L2 `_dk` bytes checks also permanently False; prior OK section incorrect **File:** `bot.py:803` (init), `bot.py:1535` (`_ws_live_overlay_ibkr._g()`), `bot.py:18520` (TAPE_SIG3_L2 `_dk`) **Evidence:** ```python # bot.py:800-803 — _regime_redis initialization: self._regime_redis = _redis_early.Redis( host='trading-redis', port=6379, decode_responses=True, # ← confirmed; always returns str keys socket_timeout=1 ) # bot.py:1535 — _ws_live_overlay_ibkr uses _regime_redis.hgetall(): v = h.get(k.encode()) if isinstance(next(iter(h)), bytes) else h.get(k) # ^^^ permanently False — dead code # bot.py:18520 — TAPE_SIG3_L2 block also uses _regime_redis.hgetall(): _dk = isinstance(next(iter(_d_book)), bytes) # ← permanently False _bv = _d_book.get(b"bids" if _dk else "bids") # ← b"bids" path is dead _av = _d_book.get(b"asks" if _dk else "asks") # ← b"asks" path is dead ``` The 2026-08-09 audit OK section stated `_regime_redis`'s `decode_responses` setting "was not confirmed as True — this check is potentially correct." The initialization at bot.py:803 refutes that: `decode_responses=True` was always set. Both locations should be added to the bytes-check dead-code cleanup already tracked for `_ws_live_overlay` (bot.py:1487) and `_tape_score_sym` (bot.py:18417). **Impact:** Two additional redundant `next(iter(h))` calls per invocation: - `_ws_live_overlay_ibkr` path: 5 `_g()` calls × per-position API poll - TAPE_SIG3_L2 block: 1 `_dk` check per tape tick per symbol (when `TAPE_SIG3_L2_SWEEP=true`) Both are cleanup / code quality — the CORRECT branch already executes correctly. **Fix:** Same as carried risks — remove bytes branch from `_ws_live_overlay_ibkr._g()`; simplify TAPE_SIG3_L2 to `_bv = _d_book.get("bids"); _av = _d_book.get("asks")`. --- ### RISK (carried × 1 from 2026-08-09): `_tape_score_sym` inner `_g()` helper at line 18417 has permanently-False bytes-key check — 19 redundant `next(iter(h))` calls per invocation, 7 call sites **File:** `bot.py:18417–18424`, call sites at `bot.py:4228, 15537, 16310, 16442, 21179, 22073, 22120` ```python def _g(k, cast=float, default=0.0): v = h.get(k.encode()) if isinstance(next(iter(h)), bytes) else h.get(k) ``` All Redis clients use `decode_responses=True` (bot.py:803, 14603, 14631, 14815, 23367, 23406, 23581). `h` comes from `hgetall` on one of these — always str keys. `isinstance(..., bytes)` permanently False. 49 syms × 1Hz × 19 calls = **931 redundant iterator-creation calls/second** during active tape sessions. **Fix:** `def _g(k, cast=float, default=0.0): v = h.get(k); ...` --- ### RISK (carried × 4 from 2026-08-02): `_build_positions_with_status` fires redundant `db.get_open_trades()` on every 3-second `/api/positions` poll **File:** `bot.py:18039`; `core/database.py:195–196`; `pages.py:7669` (`setInterval(upd,3000)`) 20 DB queries/minute during RTH; `entry_reason` (line 18048) is already available as `t.strategy_reason` in-memory. **Fix:** Read `entry_reason` from the in-memory `Trade` object; keep the DB call only for `trigger_type` which isn't on the dataclass. Add partial index: `CREATE INDEX idx_trades_open ON trades(entry_time) WHERE exit_price IS NULL`. --- ### RISK (carried × 4 from 2026-08-02): `_build_positions_with_status` computes fresh RSI(9) series per open position on every 3-second poll — redundant with `ss.last_rsi` **File:** `bot.py:18065–18075` `_rsi()` is an O(n) ewm pass over up to 500 rows; 5 positions × 20 calls/min = 100 redundant Series allocations/minute from the API path. `ss.last_rsi` already maintained by `_evaluate_symbol()`. Comment at 18059 acknowledges the issue. **Fix:** Gate the recompute on bar-staleness via a cached `ss._last_rsi_api_ts`; use `ss.last_rsi` otherwise. --- ### RISK (carried × 4 from 2026-07-25): `futures_client.py` — new `ThreadPoolExecutor` per call, no module-level cache **File:** `futures_client.py:70–75` ```python with ThreadPoolExecutor(max_workers=len(symbols)) as ex: # created + torn down each call ``` 1 call/min on `/` × 5 Yahoo fetches = 5/min pool create+destroy cycles. **Fix:** Module-level `ThreadPoolExecutor` with 15s TTL result cache. --- ### RISK (carried × 4 from 2026-07-25): `tickPrices()` full `innerHTML` DOM rebuild at 1s polling — 60 full rebuilds/min when SSE is unhealthy **File:** `pages.py:4811` (`el("symbols").innerHTML=sh`); `pages.py:6201–6207` (`setInterval(fn, 1000)`) Full positions grid destroyed and recreated every second when SSE is unhealthy. SSE- healthy path correctly returns early via `_maybeSkipPoll`. On SSE drop (network hiccup, container restart), reverts to 1s full-rebuild path; button handlers rebound each time. **Fix:** Shallow equality guard before innerHTML write, or targeted DOM updates for changed fields only. --- ### RISK (carried × 7 from 2026-07-11): `_bxt_regime()` caches `"unknown"` for the full trading day on Polygon failure — all BXt SHORTs silently blocked **File:** `bot.py:913–933` ```python except Exception: reg = "unknown" self._regime_cache[symbol] = (today_pt, reg) # cached all day; no retry TTL ``` A transient Polygon timeout at open caches `SPY → (today, "unknown")`; gate at bot.py:6326 blocks all BXt SHORTs for the rest of the day, forfeiting the +$13K/60d SHORT edge validated in commit `2227b52`. **Fix:** Cache `"unknown"` with a 5-min TTL and re-query. --- ### RISK (carried × 7 from 2026-07-11): `_tape_l2_sums_history` list rebuilt O(n) per symbol per tick **File:** `bot.py:18618` ```python _sh = [(t, b, a) for (t, b, a) in _sh if _now_ts - t <= 60] # new list every tick ``` Dormant when `TAPE_SIG3_L2_SWEEP=false` (default). Active on dev at ~1Hz × 20 symbols = 20 list comprehensions/second. **Fix:** Convert to `deque` with `popleft()` expiry. --- ### RISK (carried × 7 from 2026-07-11): 13+ `os.environ.get()` calls per `_tape_score_sym()` tick — 637+ env-reads/second **File:** `bot.py:18406–18700` (`_tape_score_sym` hotpath); also `bot.py:5057–5082` (OBV-BE inner block re-reads env per bar) 13 `os.environ.get`/`os.getenv` calls confirmed in lines 18406–18700 (at lines: 18480, 18514, 18530, 18545, 18548, 18584, 18586, 18587, 18589, 18591, 18592, 18611, 18625). 49 syms × 1Hz × 13 = **637+ env-reads/second** during active tape sessions. **Fix:** Cache flag-like env vars at startup or as class attributes. --- ### RISK (carried × 8 from 2026-06-25): `scalp_signal_summary` full-table `COUNT(*)` with no `WHERE` clause **File:** `core/database.py:506` ```python cur.execute("SELECT count(*)::int AS total FROM scalp_signal_log") # full scan ``` ~6M rows/month at 2Hz insert rate when `SCALP_SIGNAL_LOG=true`. Grows unbounded. **Fix:** Add `WHERE ts >= now() - make_interval(hours => 720)` to bound the scan. --- ### RISK (carried × 7 from 2026-07-11): `pd.concat + iloc[-500:]` GC pressure in 1Hz bar aggregation loop **File:** `bot.py:3069` (1m Polygon), `bot.py:3090` (TF IBKR tick), `bot.py:3219` (TF