In this article we list some of the reasons DLPAL does not include in trading strategy synthesis exits based on number of bars since entry. An example is included.
The objective of DLPAL is the identification of strategies for short-term market timing but in the context of Occam’s Razor. This principle states that among many different alternatives, the one that makes the fewer assumptions is the best. This principle has important application in trading strategy development because of the problem of over-fitting. As the number of indicators for entry and exit under consideration increases, so is the probability that the result will be over-fitted to historical data.
Note that out-of-sample testing has limited or even no effectiveness in machine design of trading strategies because of the multiple comparisons problem. In addition, over-fitted strategies almost always never fail Monte Carlo analysis. More advanced validation tests must be used with strategies developed by machines. The procedures used for validation are usually considered an integral part of an edge and rarely disclosed.
Since the objective of DLPAL is to identify strategies for short-term market timing, it offers only percentage and point stops based on the entry price. The goal is to specify small values for the exits for a given market with appropriate reward:risk ratio and let DLPAL identify the strategies. This maximizes the probability that the generated strategies have timing ability and are not over-fitted to the data.
Exits based on number of bars after entry and also on ATR multiples naturally fit to historical data. When this happens, the entry may become irrelevant and random strategies can be selected. In forward trading the probability of failure is high if the same distribution of returns for specified number of bars is not maintained. In the case of stops based on ATR, the strategies fit in-sample data but usually fail in out-of-sample for similar reasons.
An example
For this example we consider NQ futures data from 01/02/2014 to 02/08/2017. Note that the objective is not to identify a robust strategy but to study the effects of exits based on number of bars. The DLPAL results for 2% profit target and stop-loss are shown below. All other search parameters were left at default values for the purpose of the study.
DLPAL identified 28 strategies in this data sample, 23 long and 5 short. Again, this is not how strategies are developed in general. The objective here is to study the effect of exits based on number of bars.
Next we use the Amibroker system code generated by DLPAL that combines all those strategies and we obtain the results below.
As expected the in-sample CAR (bottom pane) is high. The average trade duration is 4 bars. Next we remove the 2% profit target and stop-loss and apply an exit of 4 bars. Below are the results of the backtest.
The CAR decreased and the maximum drawdown increased. This happened because there are no stops. Below is an example of a trade that generated a loss of about 8%.
From August 19 to 24, 2015 NQ dropped 12%. A long trade was generated for the open of August 20, 2015. Based on 2% stops, the trade lost about $2K per contract. But with the 4-day exit, it lost about $8K per contract. Had the trade been short, it would have generated an outlier that would artificially skew expectation. Note the performance deteriorates for all values of the exit based on number of bars between 1 and 10.
The main problem is that machine learning attempts to filter out large bad trades in the in-sample but these could appear in forward samples because there is no control over stops. In general, machine designed systems with exits based on number of bars will have excellent in-sample performance. If adverse conditions do not appear in the in-sample and validation is positive, the risk of failure in forward sample is high.
Separating development from application
Many trading system developers mix development and application. During development we want to make sure that our short-term strategies have timing ability. This requires refraining from using indicators and exits that naturally fit to the data. For example, moving averages and related indicators naturally attempt to fit to the data. Time-based and ATR exits fit to data by definition. Small percent or point stops help in avoiding over-fitting and reducing the probability of random strategies. However, they are not sufficient for robustness and extra validation tests must be applied.
A good but strict test (high probability of Type II error) involves a portfolio backtest on a group of securities. An advantage of exits based on number of bars is that they do not depend on point values, especially when futures are used. However, most futures markets are unique and have different dynamics, such as volatility. It may make no sense to use a 3-day exit for both stock and metals futures. According to our extensive research, portfolio backtests are relevant only when applied to a group of comparable securities. Otherwise the power of the test is low and risk of rejection of a good strategy is high.
Finally, after a strategy with possible timing ability is identified, one could test it with exits based on number of bars or of different type. But one must first make sure that profitability is not related to stops but to market timing ability. Otherwise, the strategy has high probability of failure in the future.
If you have any questions or comments, happy to connect on Twitter: @priceactionlab
You can download a demo of DLPAL from this link. For more articles about DLPAL and DLPAL PRO click here.
