Sydnee nightly — performance audit — 2026-05-30 P0 findings: 0 P1 findings: 1 Risks: 2 - Area: Performance (Saturday theme) - Branch: `dev` (0f470d7) - Files scanned: `bot.py` (18 140 lines), `core/database.py`, `docs/strategy_decisions.md`, prior audit chain (2026-05-23, 2026-05-16) - Bugs found (P0 / P1): 0 / 1 - Risks noted: 2 (1 new, 1 carried from 2026-05-23) --- Full report (dev branch): https://github.com/kanex1/sydnee.signals/blob/dev/docs/audit_2026-05-30_performance.md Reply FROM [email protected] to [email protected] to request fixes, e.g.: "code_task on sydnee-signals-dev: apply fix for the P0 about RVOL threshold in bot.py" Sydnee Agent will propose + you APPROVE (or plain 'approve') + auto-push to dev. --- Full audit below (first 12 KB) --- # Nightly Audit 2026-05-30 — PERFORMANCE ## Summary - Area: Performance (Saturday theme) - Branch: `dev` (0f470d7) - Files scanned: `bot.py` (18 140 lines), `core/database.py`, `docs/strategy_decisions.md`, prior audit chain (2026-05-23, 2026-05-16) - Bugs found (P0 / P1): 0 / 1 - Risks noted: 2 (1 new, 1 carried from 2026-05-23) --- ## Findings ### BUG [P1]: `_scalp_signal_log_loop` — forward-price horizons re-filled after tape stale, corrupting research data **File:** `bot.py:16126-16141` (introduced in commit 0f470d7, 2026-05-28) **Evidence:** ```python # bot.py:16170 — events initialized with None for all three horizons: pending.append({ "t0_ms": now_ms, ..., "price_30s": None, "price_60s": None, "price_120s": None, }) # bot.py:16132-16137 — fill loop every `period` seconds (default 2.0): if ev.get(key) is not None: # <-- None is not None == False → does NOT skip continue if now_ms - ev["t0_ms"] >= hsec * 1000: ev[key] = self._scalp_live_price(r, ev["symbol"], now_ms) # can return None else: done = False ``` `_scalp_live_price` returns `None` when the tape is stale (age > 10 s — normal during pre/post-market transitions and sidecar restarts). When this happens at the exact 30s window: 1. `ev["price_30s"] = None` (unchanged from initial state) 2. Next tick: `ev.get("price_30s") is not None` → `None is not None` → `False` → **re-enters the fill branch** 3. At e.g. t₀+62s, tape is live again → `ev["price_30s"] = P₆₂` (the 62s price, not the 30s price) 4. `ev["price_60s"]` may fill with the same `P₆₂` at the same tick Resulting DB row: `price_30s = P₆₂`, `price_60s = P₆₂`, `ret_30s` and `ret_60s` both equal the ~62s return, **not** the 30s and 60s returns. The research data is silently contaminated whenever the tape has any stale window (>10s) within the first 60 seconds after an event. **Impact:** The scalp-edge logger's core purpose is to answer "does the 4-signal checklist beat cost at +30s/+60s/+120s?" Mislabeled horizons make the 30s and 60s columns unreliable. If the tape is stale at many events' 30s mark (e.g., during any pre-market low-volume period), the averaged `avg_ret_30s` in `scalp_signal_summary` reflects an arbitrary-horizon return — the 30s column can only be trusted if the tape was live at every event's exact 30s window. **Fix:** ```python # bot.py:16170 — use a sentinel (False) to distinguish "not yet attempted": "price_30s": False, "price_60s": False, "price_120s": False, # bot.py:16132 — skip horizons that have been attempted (None=miss, float=price): if ev.get(key) is not False: # already attempted → skip regardless of result continue ``` With `False` as the "not yet tried" sentinel: - First attempt at the horizon window: `ev[key]` goes from `False` → `None` (tape stale) or `float` (price) - Subsequent ticks: `ev.get(key) is not False` is `True` for both `None` and `float` → `continue`, no re-try - `_ret(pf)` in `_scalp_finalize_event` already handles `None` → stores SQL NULL, which `avg()` skips correctly One additional change needed in `_scalp_finalize_event` (bot.py:16075): ```python def _ret(pf): if pf is None or pf is False or not p0: # False = never attempted (shouldn't occur) return None return round(d * (pf - p0) / p0, 6) ``` --- ### RISK: `api_scalp_signal_summary` has no response cache — 3 aggregation queries per call **File:** `bot.py:14185` **Evidence:** ```python @app.route("/api/scalp-signal-log/summary") def api_scalp_signal_summary(): ... return jsonify(self.db.scalp_signal_summary(hours=hours) or {}) ``` `scalp_signal_summary` runs 3 SQL aggregation queries (GROUP BY score, GROUP BY sig2, COUNT(*)) on `scalp_signal_log` with no response-level cache. The endpoint currently returns immediately (table is empty unless `SCALP_SIGNAL_LOG=true`). Once the logger accumulates data (days of 2-second ticks at 8 symbols), the table will grow into the tens-of-thousands of rows and these GROUP BY queries will add latency. **Current impact:** Low — endpoint is not auto-polled by the dashboard. Only relevant once the Phase-0 logger is running continuously and the summary page is being actively monitored. **Fix:** Add a 30-second cache mirroring `_perf_cache` at `bot.py:15872`: ```python _scalp_summary_cache = None _scalp_summary_cache_time = 0 # In api_scalp_signal_summary(): now = time.time() if self._scalp_summary_cache and now - self._scalp_summary_cache_time < 30: return jsonify(self._scalp_summary_cache) result = self.db.scalp_signal_summary(hours=hours) or {} self._scalp_summary_cache = result self._scalp_summary_cache_time = now return jsonify(result) ``` --- ### RISK [carried from 2026-05-23]: `pd.concat + iloc[-500:]` double-alloc GC pressure in bar handlers **File:** `bot.py:2918`, `bot.py:2939`, `bot.py:3068` **Evidence (unchanged):** ```python ss.bars[1] = pd.concat([ss.bars[1], new_row_1m]).iloc[-500:] # 2918 ss.bars[tf] = pd.concat([ss.bars[tf], new_row]).iloc[-500:] # 2939, 3068 ``` Each bar event allocates a new DataFrame (concat) then immediately discards a slice (iloc). At 8 symbols × 5 TFs × ~12 events/min: ~480 alloc+GC cycles/min. Minor at current scale; long-standing carry. Still unfixed. --- ## OK (checked, working or recently fixed) **Fixes from 2026-05-23 confirmed in HEAD (0f470d7):** - **`_tv_atr_obv` bar-count cache [RISK 2026-05-23]:** `ss._tv_atr_obv_cache = (n, atr, slope)` at `bot.py:3716–3730`. Recomputes only when `len(ss.bars[5])` increments. ✓ - **OBV trail ratchet Redis persist [RISK 2026-05-23]:** `setex(f"obv_trail:{trade_id}", 604800, ...)` at `bot.py:3788`. Ratcheted `_obv_peak`/`_obv_stop` survive restart. ✓ - **`_lazy_trend()` cache fix [RISK 2026-05-23]:** `pc.fetch_bars_cached(symbol, tf, dur)` at `bot.py:9216`. No longer bypasses the 15s `_bar_cache`. ✓ - **`api_trading_stocks` N+1 [RISK 2026-05-23]:** `get_stock_strategies("day_15m")` + `get_catalysts_for_symbols()` batch calls at `bot.py:15017–15020`. Loop now uses dict lookups; per-request DB cost is flat at 2 queries regardless of watchlist size. ✓ - **`api_trade_sim` O(1000) scan [RISK 2026-05-23]:** `get_trim_children(trade_id)` at `bot.py:13894` uses indexed `LIKE` prefix. ✓ **Working correctly (verified this audit):** - **`REVERSE_TIME_DECAY` sizing math** (`bot.py:5748–5757`): linear ramp from `MORNING_SIZE_SCALE` at `_open_et = 600` (10:00 ET) to 1.0 at `REVERSE_DECAY_FULL_ET = 780` (13:00 ET). `_frac` correctly clamped [0, 1]. Off-hours and last-hour caps handled separately by `_late_session_scale`. ✓ - **Hourly P&L timezone** (`bot.py:13088–13097`): `SET timezone = 'America/Los_Angeles'` on all DB connections (database.py:37,49). psycopg2 returns PT-aware datetimes; `str(dt)[11:13]` extracts PT hour, consistent with the team's operational timezone. ✓ - **`compute_performance` max drawdown** (`database.py:1561–1565`): running equity + peak tracking; initialises peak at 0 (correct — drawdown from flat capital baseline). ✓ - **`_perf_cache` 30-second TTL** (`bot.py:13062–13063`): cache-hit path returns `jsonify(_perf_cache)` without DB queries. ✓ - **`_scalp_live_price` stale guard** (`bot.py:16064`): `if now_ms - int(ts) > 10000: return None` — correctly rejects tape older than 10s. ✓ - **`scalp_signal_summary` net60_bps formula** (`database.py:368`): `avg(ret_60s * 10000 - spread_bps)` correctly subtracts one full spread as round-trip cost proxy. SQL `avg()` handles NULL returns. ✓ - **`_scalp_signal_log_loop` pending list growth**: bounded by 60s per-symbol cooldown (`SCALP_SIGNAL_LOG_COOLDOWN_SEC`). At 8 symbols × 1 event/60s, accumulates at most ~3,840 events over an 8h trading day — ~700 KB in-memory, negligible. ✓ - **`api_nav` 5-second cache** (`bot.py:11269–11270`): confirmed. ✓ - **`_status_precompute_loop` 5-second background precompute**: all `/api/status` requests return cached `_status_cache`. ✓