Before anyone risks capital on a strategy, they ask: would this have worked in the past? Backtesting answers it — you run a set of rules against historical data and tally the result. Done honestly it's one of the most useful tools in quant work. Done carelessly it's one of the most dangerous, because a backtest is remarkably easy to make look brilliant while being worthless.
Why a great backtest is easy to fake:
• Look-ahead bias. The strategy is quietly allowed to use information it couldn't have had at the time. A rule that "buys near the monthly low" is trivial in hindsight and impossible in real time.
• Silent over-tuning. Try enough parameter sets and one will produce a spectacular curve — not because it found a real pattern, but because with enough attempts something always fits the noise. Failed experiments rarely get written down, so the winner looks like a first-try triumph.
The deepest version is overfitting: a strategy that memorized the past instead of learning a durable pattern. It nails history because it was shaped to history — and falls apart on data it has never seen.
What an honest backtest looks like:
• Hold out data — build on one slice, test once on a later slice it never saw.
• Walk it forward — retrain on a rolling window, test on the period right after.
• Model the frictions — costs, spread, slippage.
• Count your attempts — the more you tried, the more likely your best result is luck.
Even a careful backtest shares one weakness: it's graded on data that already exists, so the tester always knows how the story ends. The only way to fully escape that is to predict first and let reality grade you — a forward test.
That's the design of our public experiment: every forecast is locked and Bitcoin-timestamped before the event, then scored in the open — wins and losses alike. Nothing about a timestamped forward record can be quietly fitted to a past you already know.
Full record: https://neuportal.ai/experiment
Educational content only — not financial advice.