In this article we list the validation tools provided by DLPAL. Validation methods are an important integral part of strategy development. DLPAL offers different validation methods to increase the significance of the results.
DLPAL offers the following validation methods:
From search results:
- Test Strategies for out-of-sample testing
- Robustness for analyzing sensitivity to exit variations
- Portfolio Backtest for determining performance in a group of securities
From scan results
- Robustness for analyzing sensitivity to exit variations
- Portfolio Backtest for determining performance in a group of securities
From P-indicator results
- Significance test
Summary of Validation Tools
The various functions are summarized on the program function chart below:
Due to data-mining bias, in-sample results are usually not significant because they may reflect a strategy suggested by the data instead of a strategy that has intelligence in timing the market. Although use of out-of-sample validation is consider a method of dealing with data-mining bias and over-fitted results, its effectiveness is limited by multiple testing. In essence, the effectiveness of out-of-sample testing diminishes as a function of the trials that use the same data to validate a strategy. This is the effect of data-snooping.
Robustness and Monte Carlo tests are limited by over-fitting. It is known that over-fitted strategies respond well to Robustness and Monte Carlo tests by virtue of low sensitivity to parameter changes. This does not make these methods completely useless but their limitations must be kept in mind.
Portfolio backtests are usually an effective but strict test. By demanding that a strategy is profitable over a large number of securities we effectively minimize Type I error (false discovery) but at the expense of Type II errors (missed discovery. Portfolio backtests are suitable for conservative strategy developers who are more interested in capital preservation.
DLPAL offers the above validation tools with the understanding that they all have certain limitations as discussed above. Trading strategy development is both an art and a science. More choices of validation tools offer enhanced capability of detecting errors before a strategy is used.
For more details about the validation tools offered by DLPAL please refer to the program manual.
You can download a demo of DLPAL from here. For more articles about DLPAL and DLPAL PRO click here.
If you have any questions or comments, happy to connect on Twitter: @mikeharrisNY
