In this example of machine generation of trading systems, the steps are presented for discovering price patterns with a high reward-risk ratio for SPY using the Price Action Lab search function. The in-sample results are tested on a portfolio of ETFs and also cross-validated in an out-of-sample to eliminate data-mining bias. The results from a long-short symmetrical system demonstrate the unique power of Price Action Lab in identifying non-random and robust patterns in a fully deterministic (reproducible) way.
This is the seventh in a series of posts that are intended to demonstrate the process of machine generation of trading systems based on price patterns. The first post was about SPY, the second was about QQQ, the third was about EC futures, the fourth post was about YM futures, the fifth about DIA and the sixth about the GBPUSD currency pair. Anyone interested in purchasing the code for these systems please follow this URL for the proper email address to use. Full system code is available for Tradestation and Amibroker (Charts created with AmiBroker – advanced charting and technical analysis software. http://www.amibroker.com/”) and condition code for several other platforms.
The results presented here, except the pattern code, can be confirmed by anyone with a demo version of Price Action Lab. Actually, the demo version was used to generate most of the results of this study.
Step 1: Create a target and stop file
In this example 4% will be used for the profit-target and 2% for the stop-loss for an expected average win to average loss ratio close to 2. A T/S file is created first with the values as shown below and then saved as T4S2.trs:
Step 2: Create a search workspace
The workspace is created by selecting the following:
a) The T/S file we created by the name T4S2.trs
b) The in-sample data file SPY. txt. Range: 01/03/2000 – 31/31/2009.
c) The trade parameters: ”%” is marked to indicate that the values in the selected T/S file stand for percentages of the entry price, which is selected as the open of the next bar under Inputs. The Delay input is kept marked off. (Using delay inputs will be the subject of another post).
d) Under search parameters we input 50.00 for the percent minimum win rate for both long and short patterns, 29 for the minimum number of trades, 1.50 for the minimum profit factor and 10 for the maximum number of consecutive losers. The other parameters stay at default values.
e) The date range in the data file is shown under File Date Range. In this case it corresponds to the in-sample range. The Search Range is left to 500. This means that Price Action Lab will demand that all patterns that are found to satisfy the performance criteria set in (a) – (d) must have at least one historical trade in the most recent 500 bars. Finally, the Extended search option is checked and we run the workspace.
Step 3: In-sample results
Price Action Lab will run for an interval of time depending on computer CPU speed but in this particular case it will complete the search after about 2 hours on the average. The output should look like the one below:
Each line in the above results corresponds to a price pattern that satisfies the performance parameters specified by the user. Index and Index Date are used internally to classify patterns. Trade on is the entry point, in this case the Open of next bar. P is the success rate of the pattern, PF is the profit factor, Trades is the number of historical trades, CL is the maximum number of consecutive losers, Type is LONG for long patterns and SHORT for short patterns , Target is the profit target, Stop is the stop-loss and C indicates whether % or points for the exits, in this case it is “%”. Last Date and First Date are the last and first date in the historical data file.
It may be seen from the results that Price Action Lab found 6 patterns, 3 long and 3 short, which satisfied the performance criteria specified on the workspace for the in-sample. (Please keep in mind that this is just one example for specific risk/reward criteria and performance parameters). One could argue that these patterns are random and a result of survivorship bias, i.e. they survived by chance alone and they have no predictive power. This is the reason we used an in-sample to search for the patterns and we will use the out-of-sample to see how these patterns performed in a regular backtest. Please note that:
– Price Action Lab does not look at the out-of-sample when searching the in-sample. This is very important because if a program looks at the out-of-sample and then selects patterns from the in-sample that also worked well in the out-of-sample this is an extremely bad (and even deceiving) practice that is known as data-snooping. Unfortunately, several programs that are sold to traders for discovering trading systems do allow this bad practice.
– Price Action Lab does not find systems that later allow varying their parameters for fitting their performance in the out-of-sample. All patterns found are parameter-free, eliminating an important but serious concern dealing with curve-fitting.
– Price Action Lab does not use genetic programming, neural networks, permutations or any other methods employed by some other programs that very often produce random and curve-fitted systems. Price Action Lab is based on a proprietary deterministic algorithm that produces the same output each time it encounters the same conditions and that is in compliance with the standards of scientific testing and analysis.
The Test Patterns function is used next to get a quick calculation of key performance statistics for the in-sample and out-of-sample performance. The results are as follows:
It may be seen that the win rate of this system of 6 patterns when all signals were taken is 54.08% with an average win to average loss ratio of 1.99, close to 2.00 as expected by the profit target and stop-loss levels selected. These parameters result in a profit factor equal to 2.35 for the in-sample for a total of 196 trades.
Portfolio backtesting
Before doing an out-of-sample testing, we can see how this systems has performed on a portfolio of 12 popular ETFs since their inception to 12/31/2009. This portfolio also includes SPY since inception. For the purpose the Portfolio Backtest tool is used:
It may be seen from the above portfolio backtest results that all patterns have a profit factor greater than 1. The fact that a portfolio backtest increases the trade samples of each pattern significantly allows to validate their robustness in the in-sample.
Step 4: Out-of-sample testing
With Price Action Lab it is also very easy to test all patterns found in the in-sample on an out-of-sample. Just click on Test Patterns and then select the location of the out-of-sample data file. The results with the Test Patterns output are shown below for the out-of-sample period from 01/04/2010 to 09/12/2012:
It may be seen that in the out-of-sample the win rate is 45.65%, the average win to average loss ratio is 2.00, as expected, and the profit factor is 1.68 on a total of 46 trades.
Unused data
How has this system performed on unused SPYT data, from inception to 12/31/1999? Often, trading system developers select data periods that suit them well while the system fails in the used data. Using the Test Pattern tool one can find out that the system has generated a profit in the 7 years before the start of the in-sample and 4 out of the 6 patterns had a profit factor greater than 1.
Equity curve
The equity curve below was obtained after code was generated for Amibroker (Charts created with AmiBroker – advanced charting and technical analysis software. http://www.amibroker.com/”) and a system was implemented in that platform:
The results show that a 20K starting capital of a fully invested system in SPY using the machine designed system grew to about 100K in the combined period resulting in a compounded annual return (CAR) close to 14%. Some other parameters of the system as calculated by Amibroker (Charts created with AmiBroker – advanced charting and technical analysis software. http://www.amibroker.com/”) are:
Win rate: 57.97%
Profit factor: 2.02
Sharpe ratio: 1.74
Maximum drawdown: -11.59%
Step 5: Add the systems to system tracking or generate code
The patterns Price Action Lab discovered can be added to the system tracking module of the program for receiving notifications when a signal is generated or, if the user wishes, code can be generated for implementing a system in some other popular platform.
Comments and FAQ
How long will these patterns remain profitable? This is unknown. However, the real value of Price Action Lab is that the search can be repeated again after six months or one year, for example, and the systems can be upgraded. This is not like getting a black-box and have to live with it.
Do I really need the out-of-sample test since it reduces available data for the in-sample? Actually, some people believe that out-of-sample testing is not needed after it is used to confirm that a certain process of finding trading systems leads to significant results. Thus, if it is shown than Price Action Lab can get significant patterns in an out-of-sample as long as the one in this post, then the process can be eliminated all together and the available data can be used as an in-sample to increase search flexibility. Instead of doing out-of-sample tests, the search is rerun every six months and the system is rebalanced. Portfolio backtesting can be used instead to study the robustness of the patters and eliminate data-mining bias.
Disclosure: no relevant position at the time of this post.
Charting program: Amibroker (Charts created with AmiBroker – advanced charting and technical analysis software. http://www.amibroker.com/”)
