Backtesting Methodology

I will utilize backtesting results often on this blog to illustrate points and that warrants a brief description of my technique and a disclaimer. The IBD100 is a dynamic universe of stocks and therefore, nearly impossible to backtest accurately. The purpose is not to try and predict exactly what a system will make but to understand how different concepts tend to affect results. We are not testing variables, we are testing concepts. I will not optimize anything and I will generally run the tests on the same basket of 300 IBD scan generated stocks over a 10 year period (1/1/1998-12/31/2007). The idea is to generate enough results for statistical relevancy. Except when backtesting money management and position sizing, I will use $1000 per trade, will not include slippage and broker’s fees, and will exit all open trades at the end of the test.

The backtesting software used for these tests allow split adjusted and raw historic data to be used together. One important point is one must use raw, unadjusted data to calculate entry and the number of shares that are purchased and split-adjusted data to calculate returns. This is important when backtesting stocks. If split-adjusted data is used for entry, it would reflect you purchasing many more shares than the funds allocated per trade would permit at that time. CSI is the data vendor, backtesting software is TradersStudio by Murray Ruggiero.

The data used is end of day only. Unless noted otherwise, all entries are market orders at open on the day following the entry signal. Protective and trailing stops are standing stop/market orders and profit target stops are standing limit orders that execute during market hours. You will note in the results that the largest loss is typically larger than our protective stops and this is becuase of large price shocks where the market gaps down at the open. These "real-world" situations are not omitted from the results and are part of the risk associated with holding positions overnight. These can wipe out an undercapitalized or over-leveraged account even with a good system with a positive expectancy.

All backtesting introducues survivorship bias and many of the stocks in the current IBD list were barely more than penny stocks 10 years ago, if around at all. The yearly results of these tend to show much better performance in rescent years as they work their way onto the IBD list. Again, I must stress, these tests are not attempting to predict future results, they are only to understand how different methods affect performance and I will only post results for test who's results are consistant across almost ANY basket of stocks.

The point is to find robust methods that work across as wide a universe as possible under a wide range of conditions. Sometimes I will run tests on other baskets of stocks or etfs to illustrate concepts of robustness.

In addition to William O’Neil’s books, another book I highly recommend is Dr. Van Tharp’s Trade Your Way to Financial Freedom. In discussing testing results, I will utilize his concepts of R-Multiples which is basically a way of evaluating risk-reward through the lens of expectancy. An overly simplified version is as follows:

Your R-value is the amount risked per trade. Often this is a fixed number but sometimes when backtesting variable exits like n-day lows, this number varies from trade to trade and an exact number is difficult to determine through backtesting results only and the Average Losing Trade value can be used as a substitute. A system’s R-value is evaluated in relationship to the Average Trade value to determine your expectancy per trade per dollar risk. If this is confusing, it should become clear later and a thorough read of Dr. Tharps’s book will clear up any confusion.