How to Build a Reproducible Pre-Market Drop-Risk Backtest

A practical test of a pre-open probability score, with the cutoff, cohort, missing outcomes, and trading constraints kept in the right order.

Alphanume's Pre-Market Drop Risk dataset publishes a ranked list of US microcaps using information available by 09:00 ET. The model estimates the probability that each stock will fall at least 5% from the regular-session open to the close. That sounds easy to backtest because the score and the outcome sit in the same row. The hard part is making sure the outcome never leaks into the decision.

A useful backtest should recreate the morning decision, preserve every eligible observation, and grade the probability before it asks whether a hypothetical short made money. This guide lays out that process and the checks that keep a promising result from becoming a data artifact.

Freeze the decision at 09:00 ET

The timestamp is the first rule. Each row is scored from the cumulative pre-market return path through 09:00 ET, recent price returns, the last pre-market price at or before the cutoff, and point-in-time market cap. The regular-session open, closing print, and forward returns happen later. None of those later fields belong in a feature, filter, or position-size rule.

The API documentation defines the target precisely: an unadjusted open-to-close return of -5% or worse. Keep that definition fixed. Replacing it with close-to-close return, the session low, or an after-hours exit changes the research question and makes the served probability hard to interpret.

• Decision time: 09:00 ET on the signal date.
• Entry anchor: the regular-session open, subject to a separate execution model.
• Classification target: open-to-close return at or below -5%.
• Later outcomes: attached only after the close or forward horizon can be observed.

Save the raw response exactly as retrieved, including rows you later exclude. A frozen input file lets you prove that a later rerun did not benefit from revised filters, a changing universe, or outcomes that matured after the first pull.

Build the cohort before grading the model

The published list is already a selected cohort. A ticker must be in the point-in-time microcap universe and have at least 20 pre-market minute bars at or before the cutoff before it can be scored. The absence of a ticker is therefore different from a low score. It may have been outside the size universe, inactive before the open, or otherwise ineligible.

Start with every published row over the test window. Add filters only after recording them as explicit research variants. The most important split is usually price: sub-dollar stocks are included and labeled rather than silently discarded. Those names can behave differently and often carry the worst borrow constraints, so pool them only if that is the strategy you actually intend to run.

Do not optimize all filters at once. Pre-register a small set such as all rows, top five ranks, probability at or above 0.70, sub-dollar names, and names priced at $1 or above. If you inspect dozens of thresholds and report only the winner, the threshold search becomes another source of overfitting.

Pull history without turning nulls into zeros

A recent row can have a probability and no return because the horizon has not elapsed. A mature row can also remain null when a halt, delisting, or missing required exchange print prevents the outcome from being computed. Both cases are information. Converting either one to zero invents a flat return and mechanically improves some summaries.

Use a fixed end date far enough before the retrieval date for the horizon you want to test. For the same-day classification, one completed session is enough. For the field named return_lead_30d, allow 20 trading sessions because that is the documented horizon. Then report how many rows still lack a result and investigate the reason before excluding them.

import requests
import pandas as pd

url = "https://api.alphanume.com/v1/premarket-drop-risk"
params = {
    "date_gte": "2025-01-01",
    "date_lte": "2026-07-31",
    "api_key": "alp_abc123",
}

rows = requests.get(url, params=params, timeout=30).json()["data"]
df = pd.DataFrame(rows)

df["target"] = (df["intraday_return_pct"] <= -5).astype("Int64")
mature = df[df["intraday_return_pct"].notna()].copy()
missing = df[df["intraday_return_pct"].isna()].copy()

print("published rows", len(df))
print("graded rows", len(mature))
print("ungraded rows", len(missing))

The sample keeps the missing rows in a separate audit table. A production pull should also paginate until has_more is false, preserve the response pages, and sort by date, rank, and ticker before any grouping.

Test calibration before simulated returns

The score is a probability, so the first question is whether observed frequencies line up with predicted frequencies. Group scores into pre-declared bands, then compare the average prob_drop with the share of stocks that actually fell at least 5%. A 0.70 score does not promise that every name will fall. Across a large sample, a calibrated 0.70 band should land near a 70% event rate.

1. Coverage: count rows, dates, unique tickers, and missing outcomes.
2. Calibration: compare mean probability with realized event rate by score band.
3. Discrimination: compare high-score bands with the full published cohort.
4. Daily lift: calculate whether top ranks beat the same day's lower ranks.
5. Stability: repeat the tables by year, price cohort, and market-cap bucket.

Add Brier score or log loss if you need one summary statistic, but keep the band table. A single metric can hide a model that works in one score range and fails in another. Daily lift also deserves special attention because cross-sectional rank can remain useful even when the raw probability level drifts.

Measure uncertainty around every rate. Bootstrap by date rather than by row so a volatile market morning stays together as one cluster. Treating 20 stocks from the same morning as 20 independent experiments usually makes confidence intervals look tighter than they are.

Add a return layer with real short constraints

Classification quality and trade profitability are separate questions. The dataset does not contain live borrow availability, locate price, financing cost, spread, available size, or buy-in risk. Those omissions matter most in microcaps and sub-dollar names, which are also where dramatic raw returns can make a backtest look best.

A defensible return simulation starts with an executable entry rule. If the forecast cutoff is 09:00 ET and the target begins at the official open, do not assume a fill at a stale pre-market price. Use the open or a documented post-open execution window, apply spread and slippage assumptions by liquidity bucket, and join historical borrow information when the strategy requires a short.

• Report gross open-to-close return before any cost assumptions.
• Report the fraction of candidates with a valid entry and exit print.
• Keep halted and delisted names in an exception ledger instead of quietly dropping them.
• Stress locate and slippage costs rather than selecting one favorable estimate.
• Cap daily and ticker exposure before looking at cumulative performance.

For a long-only process, the same score can be tested as an avoid list rather than a short portfolio. That use avoids borrow assumptions, but it still needs a precise benchmark and a rule for what capital does when a candidate is excluded.

Make the result reproducible

A complete research bundle should contain the raw API pages, the exact query parameters, a data dictionary, the filter configuration, the graded cohort, the missing-outcome ledger, and the final tables. Record the retrieval timestamp and software versions. A second researcher should be able to rebuild the same cohort without reading your notebook line by line.

Before trusting a result, rerun the test with one variable changed at a time: probability cutoff, rank cutoff, price cohort, test year, and cost assumption. A useful signal should degrade gradually rather than disappear when a threshold moves by one rank or five probability points.

The practical next step is small. Pull one fully matured year from the Pre-Market Drop Risk explorer, save the raw pages, and produce the calibration table before building a P&L curve. If the probability ranking survives that test, the current access options provide the history needed for a broader study.