Judgment, Oversight & Execution
A five-role AI desk trading a paper account on live market data. In one week this month it computed its own alpha for the first time, found itself 129 basis points behind SPY, and shipped forty fixes for what that measurement exposed — a backtest engine, an out-of-band kill switch, hash-chained journals, evidence-driven signal retirement, and multiple-testing correction on its own research loop. Runs 24/7 on a Synology NAS.
Share this case study →The hard part: hard risk limits enforced in code the model can't override, and a self-improving loop where a repeated mistake gets promoted from a logged lesson into a permanent, hard-coded gate.
APScheduler-based; wakes each role on a market-driven schedule. This is the scheduler that actually runs the desk in production — distinct from the bounded ReAct orchestrator, which is built but has never been promoted past shadow mode.
Python, Postgres, Alpaca, DeepSeek, Qwen, Video Factory LLM gateway
An APScheduler-based orchestrator wakes each agent on a market-driven schedule. The Analyst runs at 08:30 ET on the quality tier — heavy reasoning, once daily — reading overnight news, the macro calendar and its own historical lessons before writing a pre-market brief. The Trader runs every 20 minutes from 09:10 to 15:50 ET on the fast tier, deciding which tickers to open or close, at what size, with what stop. The Evaluator runs at midday, re-examining every open position against its original thesis. The Reviewer runs at market close and drives the self-improvement loop below. Only the Trader can place an order.
A Challenger role sits in front of consequential opens as an adversarial second opinion, on the heavier tier, as its own separate call against its own prompt rather than a second voice folded into the same run. An audit this month found it had never once fired. Its consequentiality test read the order quantity from the action — a field the Trader's own prompt explicitly forbids the model from emitting — so the test saw nothing consequential, every time, while the config flag said the safety layer was armed. That is the kind of defect that only surfaces if you go looking for the absence of something, and it is exactly why the role-run counts are on the public docket: you can check whether it fired today without taking anyone's word for it.
Every trading decision function is pure — zero I/O, zero side effects, fully unit-testable. The 52-week-high momentum signal (citing George & Hwang 2004) always returns every diagnostic field, even for candidates that fail, so nothing is ever a black-box rejection. The core-sleeve rebalancer is pure too: no I/O, no LLM, just a list of orders for a thin wrapper to execute. The practical effect: roughly 90% of the trading logic is unit-tested without a live broker connection, without mocking, and without any risk of an accidental real order during a test run.
The Trader has the richest toolset of the five agents, but several deterministic gates constrain it regardless of what the model decides. A cooldown system prevents re-proposing the same ticker on consecutive ticks — 120 minutes after opening a position, 40 minutes after evaluating one without acting, and cooldowns only ever extend, never shorten. An exit manager assigns every closed position a standardized reason and flags anything held beyond 48 hours as overdue — a pure implementation of the Triple Barrier Method's vertical barrier. Above both sits a daily circuit breaker: if the portfolio drops 2% or more in a session, new entries halt for the rest of the day, no override. And if the Analyst's morning brief is missing or stale, the Trader hard-blocks new opens rather than trading blind.
Three of those gates are new this month, and each replaced something that only looked like a gate. An out-of-band kill switch can halt the desk from outside the agent loop — until this month the only stop was a risk posture the LLM itself wrote, which is not a stop at all. A numeric-claim grounding gate checks that the figures in a decision's reasoning actually trace to evidence, rather than only checking that an evidence field was populated. And position sizing was rebuilt after a live measurement showed the Kelly multiplier had collapsed to 0.0057, flattening every position to the same ~$500 stake on a $103k account: maximum conviction and minimum conviction produced an identical order.
The portfolio is split into two mechanically separate sleeves. The Core sleeve (SPY/QQQ/VOO, 60% allocation) runs zero-LLM drift-band rebalancing — pure math, an order only when drift exceeds 8%. The Satellite sleeve (Active, 40%) is fully LLM-driven, deciding every 20 minutes via DeepSeek V4 Flash. This guarantees that even if the satellite agent's reasoning goes completely off track, 60% of the portfolio is untouched and still tracking the market. Capital preservation is architected in, not just a policy.
Every trading day ends with the Reviewer computing P&L attribution, looking for patterns across trades, and writing dated, ID-tagged lessons into a journal — committed and pushed to Gitea, no vector database or RAG pipeline involved; the git history is the memory system. That journal is now hash-chained, so the record is tamper-evident by construction rather than by someone reading the diffs.
The loop used to end there, and that was its weakness: a lesson could inform tomorrow's brief but had no path to becoming a tested claim. It now closes. A backtest engine exists, so a risk parameter can be argued from a run rather than from a conversation — every value in the risk config had previously been reasoned into place and never measured. The signal-evaluation loop searches many hypotheses at once and now carries a multiple-testing correction, because a search that wide will always find something. Signals can retire: eleven of them could previously live forever regardless of evidence, and decay is now driven by what they actually predicted.
And the loop still graduates lessons into code. After several logged instances of opening positions below the 20-day moving average and losing, that threshold stopped being a note to remember and became a hard-coded check in the risk module — from "we noticed this went wrong" to "the code no longer allows it."
Python, DeepSeek API (quality and fast tiers through a shared LLM gateway), Alpaca, Finnhub, EDGAR, pgvector, APScheduler, Docker, Gitea Actions CI/CD (self-hosted).