What does it mean to validate a strategy?
A backtest answers one question: what would these rules have done on this particular stretch of history? Validation asks a harder one: is there a reason to expect the rules to behave similarly on data they were not built around?
The gap between the two is where most strategies fail. If you try enough ideas, or adjust one idea enough times, something will look good on past data by chance alone. Validation is the set of checks that separates a pattern that may be real from one that was simply fitted to the past. No check removes uncertainty, but each one removes a specific way of fooling yourself.
Does it still work on data it has never seen?
This is the most important test. Split your history into two parts. Build and adjust the strategy on the first part, often called the in-sample period. Then run it, once and without changes, on the second part, the out-of-sample period.
If performance falls apart on the unseen portion, the strategy was probably fitted to noise. Two rules keep this honest. First, decide the split before you look at results. Second, do not keep going back and tweaking after each out-of-sample run, because every tweak quietly turns the unseen data into data you have seen.
A stricter version is walk-forward testing, where the strategy is repeatedly tuned on one window and tested on the next one, rolling through the whole history. It produces many out-of-sample results instead of one.
Do the results survive realistic costs?
Many backtests assume every order fills instantly at the exact price you wanted, with no fees. Real trading has commissions, the bid-ask spread, and slippage, which is the difference between the price you expected and the price you got. Fast-moving or thinly traded markets make slippage larger.
Costs matter most for strategies that trade often. A small edge per trade can disappear entirely once a modest cost is subtracted from each one. Rerun the test with deliberately pessimistic cost assumptions. If the result only works when trading is free, it does not work.
Is the result robust to small changes?
Suppose a strategy uses a 20-day moving average and performs well. What happens at 18 days or 23 days? If results change sharply with tiny adjustments, the good number was likely a lucky spike rather than a stable feature of the idea.
A robust strategy tends to show a plateau: a broad range of nearby settings that perform similarly. A fragile one shows a single narrow peak surrounded by poor results. Test neighboring parameter values, shift the start and end dates slightly, and change position sizing assumptions. You are looking for gradual changes, not cliffs.
Also count the trades. A handful of trades is not enough to draw conclusions, no matter how attractive the return looks.
Does it work across different market conditions?
A strategy tested only during a rising market has not been tested against a falling one. Check results separately across periods of rising prices, falling prices, sideways movement, high volatility and low volatility, including any major shocks in your data.
You do not need the strategy to profit in every environment, but you should know when it struggles and how large the losses were. Pay particular attention to maximum drawdown, the largest peak-to-trough decline. It shows what holding the strategy through its worst stretch would have felt like, which is often what decides whether someone can stick with it.
Where possible, test on more than one instrument or market. A rule that works on a single ticker and nowhere else deserves extra suspicion.
Why paper trade before using real money?
Paper trading means running the strategy in real time with simulated money. It cannot reproduce every real-world condition, but it exposes problems a backtest cannot: rules that were ambiguous, data that arrives late, signals that look different when the day is still unfolding, and operational mistakes.
It also tests you. Following a rule set when it is losing, or when it has been quiet for weeks, is harder than it looks on a chart. Run the paper phase long enough to include a meaningful number of trades, and compare what happens with what the backtest would have predicted for the same period. Large, unexplained differences mean something in the process needs fixing.
What should you decide before going live?
Write down your rules for the live phase before it starts, while you are calm. Decide the maximum amount you are willing to lose, how you will size positions, what drawdown would make you stop and review, and what evidence would make you conclude the strategy has stopped working. Decisions made in advance are easier to follow than decisions made during a losing streak.
Also decide how you will measure live results against your validation results, and how often you will review them. If you cannot say what a failure would look like, you are not ready to start.
A validation checklist
- Write the rules down precisely, with entries, exits and sizing, so that anyone could reproduce them.
- Reserve out-of-sample data before you start tuning, and test on it only once.
- Rerun the test with realistic commissions, spreads and slippage.
- Vary key settings and dates slightly, and look for a stable range rather than a single peak.
- Review results by market condition, and record the maximum drawdown and the longest losing stretch.
- Confirm there are enough trades to support any conclusion, and check for lookahead and survivorship bias.
- Paper trade for a meaningful period and compare the results with the backtest.
- Set your loss limits, review schedule and stop conditions in writing before using real money.
Where Thulo fits
Thulo is an analytical platform. It lets you describe a strategy in plain English and backtest it against historical data, so you can run checks like the ones above without writing code. Thulo does not make decisions for you. What you test, how you interpret it and whether you act on it remain your choices.