Statistics for dummies

Every term on the desk, in plain words. Hover any dotted word on any page for the same explainer.

As-of (point in time): Reading the data as it looked on a given date: only observations published by then, and the version that was current then. Backtests that read today's cleaned data are reading the future.
Autocorrelation: When a series resembles its own recent past. Prices and overlapping returns do this strongly, which makes naive tests see patterns that are just persistence.
baseline_only: A game insight that predicts the result (who wins, by how much) rather than the result against the betting line. Real, but the line already prices it, so it is not an edge.
Benjamini-Hochberg: A rule for testing many hypotheses at once: sort the p-values, and accept only the ones small enough given their rank, so the share of false discoveries stays below a target (here 10%). Without it, screening 34,000 specs would 'find' thousands of relationships by chance.
Brier score: How wrong a probability was, squared. Say 70% and it happens: (0.7 − 1)² = 0.09. Say 70% and it doesn't: 0.49. Lower is better; always saying 50% scores 0.25, so anything above 0.25 is worse than a coin flip.
Calibration: When you say 30%, does it happen about 30% of the time? A seat that says 70% on things that happen half the time is overconfident. It takes dozens of resolved forecasts to tell.
Cluster bootstrap: A way to get a confidence interval by resampling whole groups (settlement days) with replacement, thousands of times, and seeing how much the average wobbles. Resampling individual forecasts would overstate the certainty when forecasts share a day.
Confidence interval: The range the true number probably sits in, given how much data there is. A 95% interval of [−0.17, −0.03] means: if we repeated this many times, 95% of such intervals would contain the truth, and here the whole range is negative.
Edge: Market Brier minus our Brier on one question. Positive means we did better on that question. It says nothing about money until there are many of them.
Effect size: How much the target moves per unit of the feature, from a regression. A beta of −0.0115 on a 21-day corn return means one unit higher in the feature went with about 1.2% lower return. Small effects can still be real; tiny ones are rarely worth trading.
Effective n: The number of truly independent observations. 700 daily rows of a 21-day return are more like 700 ÷ 21 ≈ 33 independent ones. Rules that need 'at least 60' mean effective, not raw.
Embargo: A gap of one horizon between the training and test data in a walk-forward, so a 21-day return that starts in training cannot end in test and leak the answer.
Family: The group of tests corrected together: every feature tested against one target at one horizon. The family size is stored with each result so nobody can shrink it later.
Feature: The thing that might predict: a weather anomaly, a positioning z-score, a funding rate, a team's recent form. Transformed and lagged so it was knowable before the target.
Grade: robust: significant after correction, out of sample, costs, stable, holdout agrees. promising: significant with out-of-sample excess, holdout not decisive. weak: screened but failed out of sample or the holdout. failed: was promising, no longer. Retired rows stay visible.
HAC (Newey-West) errors: Overlapping forward returns are not independent: tomorrow's 21-day return shares 20 days with today's. HAC standard errors widen the uncertainty to account for that, so the p-value is not fake-small.
Hit rate: The share of times the rule fired and the target beat the baseline. 0.55 means slightly better than even; 0.50 is a coin.
Holdout: The newest quarter of every sample, set aside before any screening or fitting. The development rule is applied to it once, at the end. Passing means the relationship kept its sign and beat the holdout's own average. It is the closest thing to a fresh test the data allows.
Horizon: How far ahead the target looks. A 21-day horizon means the return from today to 21 trading days from now.
Independent events: Three WTI strikes that all settle on the same day's price are really one roll of the dice, not three. We count settlement events, not forecasts, when asking how much evidence we have.
Influence (leave-one-out): Drop each observation in turn and recompute. If one extreme day is doing all the work, the result flips when it is removed, and it fails.
Kelly fraction: The bet size that maximizes long-run growth given your edge and odds: (p × b − q) ÷ b. Real bettors use a fraction of it because the edge estimate is noisy; here a cap of 2% of bankroll would apply.
Lag: How old the feature is allowed to be. A lag of 3 days means the feature value used is the last one published at least 3 days before the target's start.
line_relative: A game insight whose target is the result against the line: did the home side cover, or beat the Kalshi price. Only these can say anything about edge.
Log loss: Like the Brier score but it punishes confident misses much harder. Saying 99% and being wrong costs about 4.6; saying 60% and being wrong costs 0.9.
Log-odds pool: How the committee number is made: each seat's probability is turned into log-odds, averaged with calibration weights, and turned back. Deterministic. The portfolio manager explains it; it does not pick it.
Look-ahead bias: Using information that was not available at the time. A close stamped at midnight is 'known' 17 hours before the market closed; a seasonal average that includes next year leaks the future. Every read here filters on the time a number was published.
Market Brier: The same squared error, but for the Kalshi price at the moment the question was asked. It is the bar: beating it means knowing something the price did not.
Matched sample: Only the questions a market also priced, so both sides are scored on the same questions. Comparing our average over everything with the market's average over a subset is how a bad number hides.
Mean publication lag: On average, how many hours after an event its number became knowable. Zero for a price at its own close; about 30 for observed degree days; days for weekly reports.
OOS excess: The average target return when the rule fired, minus the average over the whole test period. Measuring against zero lets a rule 'win' just because prices drifted up or home teams win more often.
Out of sample (OOS): Tested on data the rule was not fitted on. We fit a rule on earlier data, then score it on later data it never saw. Anything can fit the past; only out-of-sample results count.
p-value: If there were truly no relationship, how often would random noise produce a result at least this strong? 0.01 means about one time in a hundred. Small is good, but with thousands of tests some small p-values are guaranteed by luck.
Percentile: Where today's value sits in its own history: the 96th percentile means it has been this high only 4% of the time. Signals fire in the top or bottom 30%.
Pre-registered: Written down before the result was seen. The miner's grid of features, targets, horizons and lags is enumerated in code before any test runs, so the search cannot be steered toward what looks good.
Published time: When a number became knowable to anyone. Different from when it happened (event time) and when the desk fetched it (ingested time). Futures closes publish at the session close; NOAA observed degree days about 30 hours after the day.
q-value (false-discovery rate): The p-value corrected for how many tests were run in the same family. q = 0.05 means that among findings called significant at this level, about 5% are expected to be flukes. The Benjamini-Hochberg procedure computes it.
Regime split: Testing the relationship separately in different periods (calendar thirds). A relationship that only exists in one regime is probably that regime, not a law.
Registry: An append-only file where every forecast and resolution is written with the previous line's hash and its own. Change or delete any line and the chain breaks. Corrections are new lines.
Retrospective run: A question written and run after its as-of date. The data was capped at that date but the model's knowledge was not, so it is labelled and kept out of the live scores.
Reviewer checklist: Seventeen questions a hostile statistician would ask (sample size, pre-registration, multiplicity, out of sample, costs, look-ahead, survivorship, revisions, regimes, autocorrelation, single observations, confounders), each answered from the numbers, never from prose.
Revision: A later version of the same data point. The old print stays; the new one is knowable only from the moment it arrived. Reads as of an earlier date still see the old print.
Signal: A library insight whose feature is currently in the tail that mattered historically, pointing up or down over its measured horizon. It is a measured relationship in play tonight, not a forecast of what will happen.
Skill vs market: 1 minus (our Brier ÷ the market's Brier). +20% means our errors are a fifth smaller than the market's; a negative number means the market is ahead.
Spearman correlation: Correlation on ranks instead of raw values: do high values of one series go with high values of the other? Runs from −1 to +1 and ignores outliers' size, only their order.
Stability: Split the history into three periods: in what share of them does the relationship keep the same sign as the full sample? 1.0 means all three agree; 0.33 means it flips around.
Survivorship bias: Studying only the things that survived. An injury history built from today's report misses everyone who recovered; a stock universe built from today's index misses the delisted.
Target: The thing being predicted: the forward return of a price over 5, 10 or 21 days, or a game's margin, win, or result against the line.
Walk-forward: Fit on the first chunk of history, test on the next, then grow the training window and repeat. Four folds here, with an embargo gap so overlapping returns cannot leak across the boundary.