Variations in the results generated by different backtesting platforms are common. In this article, I compare year-to-date backtesting results from Amibroker and Quantopian for a robust strategy that was machine designed by Price Action Lab software a few years ago. Small variation is found in both trades and returns.
In this article we will use the code of a long/short strategy for DIA ETF from an August 2013 article. In that article there is also a brief discussion of data-mining and selection bias. The DIA trading strategy was developed by Price Action Lab in April 2013 with in-sample data from the ETF inception to 12/31/2009.
Price Action Lab offers automatic code generation capability for both Amibroker the Quantopian platforms. Amibroker is a very popular platform for charting and backtesting. Quantopian provides a research environment to quantitative traders for developing strategies.
A variation of the DIA strategy that was machine designed by Price Action La software was used to enter the Quantopian trading contest, as was announced here.
Below is the year-to-date performance of the DIA trading strategy from Amibroker. This includes commission of $0.01 per share. Strategy is fully invested.
YTD return is 14.78% at -3.81% maximum drawdown and a total of 13 trades.
Below is the performance from the Quantopian platform. The entries are determined on daily data but the exits are determined on 1-minute data. There is also a liquidity algo for partial fills and commission estimation. In Amibroker backtest, entries are at the open and exits occur intraday and it is assumed all are immediately filled. This may not always be the case, especially for large orders. Of course, one could implement a partial fill algo in Amibroker but this is not done in this case.
In the Quantopian case, return is 15.6% with maximum drawdown of -3.4% and 10 trades. One would expect the performance from Quantopian to be lower but this is not the case. Return is better by about 80 basis points.
Apparently, the variation in the results from the two backtesters is due to the difference in the number of trades. Amibroker generates 13 trades, as shown below:
The three trades that are missing from the Quantopian backtest are highlighted in the above trade list. Below are the transaction details from Quantopian:
The three missing trades result in a combined loss of about 1.07%. This difference in backtest results could be due to a few factors, including lack of liquidity and variations in the data. We are not going to try to identify the exact causes here because that is not really important for the purpose of this article. There will be variations between the results of different backtesters almost always due to different assumptions, even if data series are the same. What is important in this case is that performance is close enough and within an acceptable error margin of 1%. That is good enough for practical purposes in the case of low frequency trading.
It is always a good idea to compare backtests from different platforms and quant traders should always do that to minimize probability of errors in strategy code.
If you have any questions or comments, happy to connect on Twitter: @mikeharrisNY
Subscribe via RSS or Email, or follow us on Twitter.
Charting and backtesting program: Amibroker
Disclaimer
Technical and quantitative analysis of Dow-30 stocks and 30 popular ETFs is included in our Weekly Premium Report. Market signals for longer-term traders are offered by our premium Market Signals service.
