Back-testing tells us how a system performed in the past and that is all. If one assumes that systems that did not perform well in the past have high probability to fail in actual trading then back-testing can be turned into a process of hypothesis testing. The hypothesis to test is whether a specific trading system will not perform well in the future. If the results of backtesting are not acceptable, then the hypothesis is supported and as a result we reject the system.
Back-testing cannot tell us anything about future performance if the hypothesis is not supported, meaning that in the case that the trading system performance is acceptable nothing can be said about future performance. The reason is that we have no means of rejecting the hypothesis but of only supporting it. Rejecting the hypothesis is equivalent to showing that the system performance will be acceptable in the future. We have only one principle that we can use to support the hypothesis in the case that the back-testing results are not acceptable but nothing we can use to reject it and offer support to the alternative hypothesis that the trading system will perform well in the future. Thus, given that the hypothesis is not supported, the system may or may not be profitable in the future. Two comments on the above:
(A) Use of back-testing to support a hypothesis about the future failure of a trading system is based on a principle. The principle is that trading systems that do not perform well in the past based on a sufficiently large sample of trades have high probability not to perform well in the future. Obviously, the probability can never be equal to 1 and there will always be some false rejects, meaning systems that although failed to perform well in the past, it turns out that they perform well in the future for whatever reason.
(B) If the back-testing performance is acceptable, the hypothesis about future failure is not supported and the system may or may not be profitable in actual trading. It has probability p to stay profitable and 1-p to fail. Successful system traders have devised methods of selecting systems with high probability to stay profitable and for an extended period of time.
In another post I will comment on the pitfalls of back-testing due to the inappropriate mixing of data and systems, a mixing that often creates chaos and useless results. It appears that many traders are not aware of some of these pitfalls.
