How often does a trading bot with great past results keep winning later?
Research question: When a trading bot shows strong past results, how often do its returns hold up in the months after it is released to the public?
The records do not give a rate for how often a bot's returns hold up after release. They do show that strong past results are a weak guide: a strategy that looks optimal in sample often performs very poorly out of sample because its parameters are overfit 3. In most runs of the authors' online simulator, a strategy optimized on one random walk performed poorly on a second 1. The records also state that a backtest only simulates the past and does not predict future behavior 2. The newest related item is an SEC fraud report dated 2026-09-29, about a month before today, which concerns AI trading bot scams rather than release performance 45.
Sources
"In most runs using our online tool, the “optimal” strategy derived from the first time series performs poorly on the second time series, demonstrating how hard it is not to overfit a backtest."Statistical Overfitting and Backtest Performance (2014)
"It is well known that a backtest is just a simulation over the past and does not predict future behavior of a trading system."Backtest of Trading Systems on Candle Charts
"Unfortunately, such an “optimal” strategy often performs very poorly “out of sample” (i.e., on another dataset), because the parameters of the invest strategy have been overfit to the in-sample data, a situation known as “backtest overfitting”."Statistical Overfitting and Backtest Performance (2014)
"TSAI Pro and its foundation used their website, WhatsApp chats and a public Facebook page to promise guaranteed profits from renting out AI-programmed trading bots."SEC Investment Fraud Charges Target AI Trading Scams
"Investors were directed to open accounts on a trading platform that, the SEC says, never executed a genuine trade."SEC Investment Fraud Charges Target AI Trading Scams