Letting AI Write Your Research Code Without Letting It Grade Its Own Homework

AI agents now write, run, and debug research code end to end. That is a gift for infrastructure and a hazard for evaluation. Three guardrails keep the results trustworthy.

Coding agents have crossed a threshold. Models like Claude no longer just suggest code; they write it, execute it, read the error messages, and fix their own mistakes. For a solo quant researcher, that changes the economics of every project: work that used to eat a weekend now takes an afternoon.

For research infrastructure, this is close to a free lunch. Data loaders, API clients, plotting utilities, schema definitions: describe the shape, review the diff, done. There is no honest argument for hand-writing another requests wrapper in 2026.

The interesting question is what happens when the code being written is the code that evaluates the strategy. That is where AI-generated trading code goes from accelerant to accident, and it is exactly where a few structural guardrails earn their keep.

The failure mode nobody warns you about

Here is the scenario. You let an agent write a strategy and its backtest in one session. It reports a Sharpe of 2.1. Beautiful. Except the backtest has a bug: it enters positions at a closing price that was not knowable at decision time. The agent wrote both sides of the experiment, so there is no incentive structure anywhere in the loop that would ever surface the bug. The code runs, the number is impressive, and the number is fiction.

This is not a hypothetical about model dishonesty. It is the ordinary behavior of any optimizer given control of its own scorecard. Whoever writes both the strategy and the evaluation controls the verdict, and that is true whether the author is an agent, a junior analyst, or you on a deadline. The fix is the same one trading desks have used forever: separation of duties.

Guardrail 1: the model never sees the answer key

The single most important rule when an agent builds anything predictive: withhold the outcome data.

If you hand over a dataset that contains next-day returns and ask for a model that selects tickers, some path to peeking exists, and you should assume it will be found. Not out of malice; the optimization pressure just flows downhill, and leakage requires only access, never intent. So structure the assignment defensively:

• Provide features exactly as they were known at the time, point-in-time.
• Strip realized outcomes from anything the agent can read.
• Keep evaluation data physically separate, in files or tables the agent has no access to.

Point-in-time discipline was already the core skill of honest backtesting; it is the entire subject of look-ahead bias. With an agent in the loop it goes from good practice to non-negotiable, because you are no longer the only one who can leak the future into the past. An agent iterating at machine speed can find a leak in minutes that you might never have stumbled into.

Guardrail 2: it does not grade its own homework

Split the roles the way a desk splits them:

• The agent writes the strategy code, working only against point-in-time features.
• You own the evaluation harness: entry prices, transaction costs, the P&L calculation. Write it once, keep it boring, and reuse the same harness across every experiment.
• The agent's output flows into your harness. Never the reverse.

The harness being boring is a feature, not a compromise. A dull, stable evaluation layer that you understand line by line is worth more than a sophisticated one you cannot audit. Every experiment that passes through the same harness is comparable to every other, which is most of what makes a research program cumulative rather than a pile of one-off scripts.

One upgrade worth stealing: have a separate agent session review the strategy code adversarially. Same model, fresh context, one instruction: "find the lookahead bug in this." Generation and criticism can come from the same intelligence, but they must never come from the same conversation. A fresh session has no attachment to the code being right, and that detachment is the whole value.

Guardrail 3: rationale in writing, costs in the numbers

Two habits close out the loop.

First, keep requiring a written rationale for what the code does, the same demand you make during AI-assisted idea generation. When the stated logic and the implemented logic disagree, and occasionally they will, you have caught exactly the kind of bug that silently ruins results. The rationale is a tripwire: divergence between what the code claims and what it does is the dangerous defect, surfaced for free.

Second, make your harness charge costs by default: spreads, fees, borrow where relevant. Agents are relentless optimizers, and a frictionless simulator is an open invitation to strategies that only work in a frictionless world. If a strategy's edge cannot survive a realistic cost model, you want to learn that from your harness, not from your broker statement.

What this looks like in practice

A working session under these guardrails runs roughly like this:

• You define the hypothesis and assemble point-in-time features, outcomes excluded.
• The agent writes the strategy logic and a written rationale for it.
• A fresh agent session attacks the code, hunting for leaks and unknowable inputs.
• The surviving code runs through your harness, with costs on.
• You read the result with the same skepticism you would apply to any backtest. Our checklist for reading a backtest honestly applies unchanged; nothing about AI authorship earns a result softer scrutiny.

None of this is exotic. It is the standard separation of duties any research team runs, applied to a collaborator that types faster than you. The teams getting real value from these models are not the ones trusting them more; they are the ones who built the harness that makes trust unnecessary.

Notice also what the guardrails do not cost you. The agent still writes the loaders, the feature engineering, the strategy logic, and the plots, which is where nearly all of the typing lives. What you keep is small and stable: one evaluation harness, one review habit, one rule about where outcome data sits. The overhead is a rounding error next to the speedup, which is exactly why there is no excuse for skipping it.

Learn the full system

These guardrails come from From Idea to Code, With Guardrails, a lesson near the end of Alphanume Learn's course, Systematic Trading with Market Data. It closes a loop the course builds deliberately: ideas from the model, code from the model, judgment and evaluation from you. Everything runs in the browser against live market data from the Alphanume API, so the raw material for this workflow is one function call away, and the point-in-time habits the guardrails depend on are taught from the first module.

That first module is free, no account needed. The full course, roughly 18 hours of runnable lessons, is included with an Alphanume Pro membership alongside the data platform; see pricing or the syllabus. Theory is cheap. Run the code.