Backtesting Methodology
I want to shift gears a little this week and discuss a study I just wrapped up looking at the affects volume has on a trading system. We’ve all heard how important volume is but I’ve become increasingly skeptical of conventional wisdom and decided to take a closer look at it myself. We will change our data format a little this week as well to better facilitate an analysis of this. As usual, we will compare the results over three separate portfolios to confirm our results are systemic.
The entry system is the same one we have been using to date with one exception, Pradeep’s volume requirement has been replaced and we will be looking at the volume increase as a percentage over the average 10-day volume. I inevitable get asked about entries after each post and will only refer one to Stockbee for more details on the entry criteria. I do not want to take the liberty of detailing his criteria here.
For our average volume calculations, we could use a different time frame but the results will not vary substantially as this number changes. This is an area that is susceptible to curve fitting and in order to avoid this, I will always use a 10-day period for these types of calculations.
We will also look at the results with two sets of exits. The 8% protective+20% profit target and the 8% protective + 25% trailing profit stop. These exits are discussed in more detail in previous posts here and here. While probably not necessary, I’ve decided to test with multiple exits to confirm the results are independent of a particular exit strategy.
Let’s begin by looking at a chart of the expectancy per dollar risk (Avg. Trade/ Avg. Loss) against our different volume requirements followed up by the raw data.
8% Protective + 2o% Profit Target
8% Protective + 25% Trailing Profit Stop
To borrow an analogy from Quantifiable Edges, myth confirmed - sort of. Strong volume does appear to elevate our expectancy and the affect can be quite substantial. Looking at he 20% profit target with the IBD portfolio, a 200% volume increase raises our expectancy per dollar risked by nearly 36% over a 0% volume increase. The Nasdaq portfolio is even more dramatic with a 110% expectancy increase.
There are some interesting exceptions however. Above about a 250% volume increase, things start to get a little catawampus, particularly with the S&P portfolio. Part of this can perhaps be described by the lower number of trades at the higher volume breakouts which may be increasing the margin of error but does not fully explain why the S&P portfolio performance completely falls off a cliff between 250-300%. I can only speculate to why this is but feel it might be related to the nature of the large-cap stocks that make up the S&P500 and an over-reaction to a positive event or news item that would trigger this high a volume surge and the subsequent correction. This is only speculation however.
As usual, I encourage everyone to spend some time with the data and formulate your own conclusions and strategies for incorporating this data into your trading, if at all. To aid in this, I have posted the raw data in spreadsheet form to the following location for download.
20% profit target
http://spreadsheets.google.com/pub?key=prgyYW4ZqFw5nJ2l4JqiKZw
25% trailing profit stop stop
http://spreadsheets.google.com/pub?key=prgyYW4ZqFw4Lmo62uQVxNg
Saturday, March 29, 2008
Subscribe to:
Post Comments (Atom)
9 comments:
Very impressive spreadsheet - did you use Tradersstudio to create that? Or did you just get raw data from TS and then work it further in Excel? Just curious because I'd love to learn how to do a test like that.
Any help greatly appreciated!
I just took raw data out of TS Summary reports and worked it further in excel. I think TS can do this with its custom report feature but I haven't been able to get it to work yet. These more advanced features are very poorly documented and I am having a heck of a time trying to figure it out. Their support apparently thinks everyone is a programming expert, which I am not.
I agree with you - I've found the tool rather frustrating in that way. It also has no community to help out, unlike, say, Amibroker or Wealth-lab with has a really good community
It is extremely powerfull though and the price is good for what it can do. Just a steep learning curve. I had a tough time wrapping my head around the object-oriented components - macros, tradeplans, etc. but think I am finally starting to get a handle on it though.
Have you considered applying a good old fashioned T-test to your results?
(in addition to the other systemic filters you use)
Also, IBD is big on sibling confirmation. Perhaps a test that required several canslim type stocks to be present , and moving, to trigger entry would be interesting. Then again that would probably be very similar to simply testing in a bull market lead by growth stocks?
Just an idea, anyway great blog, I really enjoy it
Great stuff.
Is there any particular reason why you are using Pradeep's entry criteria? I ask, because since he charges a subscription for his information, it makes it costly for the rest of us to try and replicate anything. I'm just curious why you have selected that entry criteria as the only one you are testing with. If it is just testing for testing's sake, why not some moving avg. crossovers, breakouts, etc?
Anyway, I'm not complaining. I really like your blog.
martingale,
Part of the reason I started tracking the IBD100 index was eventually use it much as you describe to confirm leadership movement, etc. I'd love to run some student-t's on the data but not really sure how to do it with the tools I have. I will look into it.
woodshedder,
I recognize the problem. I originally started using his system as it was a "published" system that had 1 or 2 unique elements to it I liked and personally use. I have considered simplifiying it into something I am comfortable posting and will go ahead and do that moving forward. It will make things simplier. Thank you and I read and enjoy your blog daily as well.
bhh,
You won’t have any trouble with students -T. Once the data is in excel it will calculate the std deviations easily. Given the number of trades in your tests the degrees of freedom (n-1) will be well in excess of 1000 so anything around 1.6 and above will be in the safe zone at 90% confidence level. A nice little objective failsafe to add to a common sense approach. Great work
sorry, but I don't get what are you doing. Do you study the performance of IBD stocks, or the other baskets, as a function of the volume of the break up that gives the entry signal?.
BTW a possible idea to backtest. To study an exit based in a trailing stop at 3 or five days low. Similar to what Pradeep does.
Best.
Post a Comment