I haven’t been backtesting much lately. One problem I have noticed with backtesting trading systems, is that there’s no way to account for the effect that your trades have on the instruments you are trading. I have been noticing that effect more than ever, recently. If you look for it, I bet you can spot it, as well.
It should be obvious, if you think about it, that interacting with the markets can change their future direction. When you buy shares of a stock, no one else can get those shares anymore. So, if there are 2000 offered at a price, and you buy 1900 of them as other market orders come in, chances are good that the ask is about to go up. With all the backtesting systems I am aware of, though, you get your 1900 shares and the system keeps on playing back its canned market data. Unfortunately, in that data, someone else may have gotten the same 1900 shares as you did. In real life, that could not have happened.
But, it’s no big deal, right?
At this point, you might think, “so what? the ask moves up a cent, momentarily…” First off, on some stocks your buy order could eat through 5 or 10 cents of ask levels. Even on a small order, I’ve been victimized a number of times by pesky specialists who find a way to raise the ask, fill my order, and then drop it back down (probably laughing at me the whole time). But, that’s still a small effect, that you could just call slippage in a backtester, right?
I say, wrong. Sometimes even a single cent makes a big difference. It’s like how they say a butterfly flaps its wings in Kansas, and causes a typhoon in Asia (as if foreign countries needed more reasons to hate the US!). What if that one cent ripple is a new high that triggers a wave of computerized institutional buying? After a couple more cents, shorts get nervous about the volume and start to cover. Then, any idiot can see the stock is moving, and more money pours in. etc. etc. None of which would have happened if you hadn’t eaten up a few shares a couple minutes before.
To take it even further, what if all that buying in your stock pushes your sector’s index through an important technical level? Perhaps that causes a huge number of participants to pile on to several stocks, adding more fuel to the fire. Or perhaps not, but it grabs a lot of attention, regardless.
It’s Not Always A Rosy Experience
The previous example is the case we want, where our buying helps cause a wave of additional buying. Score! I’m far from the first to notice that this can happen. I was reading an old elite trader thread about tape reading the other day. In it, someone pointed out that when they are long a stock, they’ll buy 100 more shares at stalled new highs, to keep the ball rolling. (I don’t normally read the elite trader site, but this post from Tyro Trader pointed it out to me.)
However, it’s easy to imagine cases where your effect on the price action can work against you, as well. Take the case where you sell a stock short, and the specialist manages to pull that trick where the bid drops 5 cents just long enough to fill your order. Then, the bid pops right back up. That sucks, but it’s just part of the game, you think. Then, a couple buyers pick up some shares and the stock moves up a couple more cents. Hmmm… What does this look like on a 1-minute candlestick chart? If you sold enough shares short, it can look like a relatively high-volume hammer candle! That’s a bullish signal that could cause trigger-happy speculators to start buying more. The more they buy, the more interest is generated. Had you not shorted those shares, it would just be a flat 2 cent green candle that no one would care about. It might have even gone on to drop hard, just like you imagined when you shorted the shares in the first place!
Continuing the example, let’s say the stock keeps moving up, and you decide you must have been wrong, so you cover your shares. This eats through some ask levels (at which point you swear to yourself you will never trade this stock again!), and causes another wave of buying. The irony is, had you not covered, maybe the stock would have fallen back down.
Then again, maybe not. You’ll never know, and your backtester won’t tell you, either.
Another Example
Your order doesn’t even have to be filled to make a difference in the price action. How many times have you put in a limit order that splits the bid/ask, only to see other buyers step in one cent in front of you? You chase the bid, but they keep one-upping you before you can get filled. Who’s to say that would have happened if you hadn’t put in your greedy limit price?
Just backtest very liquid stocks?
Of course, your influence on a market is minimized if you trade instruments that are highly liquid. Even so, you do have an undeniable effect on the supply/demand when you trade. Further, only the most liquid stocks are truly liquid all the time. To minimize your effect, you should really only enter your trades at times when $/sec churning through the stock is pretty high. I mean, it doesn’t matter that 2 million shares traded in the opening hour, if only a few thousand shares are trading right now.
I’ll add that backtesting fewer shares might help preserve the integrity of the simulation, but it would also make the results useless. You can’t take those results, trade larger in the real markets, and expect proportional results.
Summary
Because of the deficiency I’ve described here, I’ve lost some of my interest in backtesting. Or, at least, I’m now more interested in scanning for potentially good setups, than I am in going through an entire trading simulation. The simulation simply cannot account for my effect on the price action, which is very real. Look for it in your own trades, and you may be able to discern the difference you are making. All too often, I can look at a chart and spot my trade on it. The effect may be pronounced for me, because I am primarily a breakout trader, and often my trades help fuel the breakout (and sadly, sometimes my trade represents the upper wick on the highest candle of the day).
great post richard…this weekend you mentioned that you were going to write a post that sounded interesting…it had to do with you needing to execute more trades to prove your system statistically…on the surface i disagree with that statement, but i still would like to read your thoughts on the matter
53 consecutive winning sessions seemed to prove (to me anyway) that your system was viable…imho your ability to stay out of low probability setups, coupled with an ability to execute very few (but highly probable/profitable) setups was proof that your system worked…using the logic you set forth above concerning price action, couldn’t it also be said that searching for more trades would have an impact on the viablility of your system…so therefore, you may statistically prove the viability of the new (more trading activity) system, but it does nothing to either prove or disprove what you were doing before…i understand that you desire a larger sample size to determine results, but couldn’t that come just as easy by doing what you were doing for a longer period of time
on another note, congratulations on the slumpbuster…
btw, i barely passed the statistics class at the local community college
Interesting perspective, Richard. There is an important factor to consider here which is the volume and general interest in the particular stock. Those stocks that do tens of millions of shares on thousands of trades a day are significantly less likely to be impacted by any single small trader’s order.
Yeah, there is smaller probability that you will have a lasting impact on a very liquid stock. But, even they get stalled out at tops and bottoms, and you can tip the scales and see if anyone follows you. I haven’t made a habit of trying that, but I’ve seen a couple references lately to people that do, so it got me thinking more about it.
@John, I just need to get the energy together to write up the article. Then you can decide for yourself what you think about it. I’m still mulling it over, but the experiments I did seem to show that more trading==more consistent profits.
richard: i am looking forward to it
Richard, great article! Reading it reminded me of the quote in the bookReminiscences of a Stock Operator.
“I remember very clearly how everyday I would buy cotton, more cotton. And why do you think I bought it? To keep the price from going down! If that isn’t a supersucker play, what is? I simply kept on putting up more and more money to lose eventually.”
Food for thought.
I daytrade 3-4 stocks a day on average. The issues you discussed above are interesting.
In my trading I don’t experience the issues you speak about. I trade between 1-3k shares on the same gappers that most day traders play. I might get 1-2 cent slippage once in a while but nothing like what you describe. I too will play breakouts.
What type of order do you send and when do you send it to get those order fill results?
@Zoomie, thanks
@GX you can see all my trades here on the site. Sometimes, even on stocks that trade millions of shares a day, the bid/ask levels can be really thin. You can see examples on the site where I’m filled in 100 share increments across 10 cents on high-volume stocks. There’s really no way to know, due to the L2 games that get played, how deep the ask really is. I do sometimes try to watch the T&S and make sure at least some larger prints are going through.
As for the specialist games, when a stock’s doing a ton of volume you won’t see them. It’s when you happen to catch a lull in the prints that these things start to show up.
I don’t play gappers, usually. I use market orders for almost everything.
And, I don’t want to give the impression that these weird things happen on every trade. It’s rare, but it happens. And, even without pricing anomalies, the overall point is that you can’t participate in the markets without altering their course. That is true in all cases.
Great post, Richard. I have noticed the same affects as you, especially when trading smaller cap stocks. It reminds me of the Heisenberg Uncertainty Principle, where it is impossible to precisely measure certain things in quantum mechanics because the experiment itself adversely affects the results.
But backtesting is useful for mutual fund trading analysis because slippage is not an issue, and your purchases/sales do not affect the execution price.
@George: yeah, I didn’t draw that analogy in the article, because you can observe the markets without affecting them. But, it certainly feels like a similar notion!
Re – 100 share increments across 10 cents – which broker are u using?
My experience is similar to that of GX with regards to the slippage but those gappers are usually very liquid.
@Eyal, I use Scottrade, but in the cases I am talking about, I don’t see what else they could do. I see the L1 ask run up as my order is filled. I am literally eating through the offered shares with my market order. Is there anything else they could do? I’m not watching L2 quotes normally, but I assume I’d see the same thing if I were.
I think really I am to blame for entering such a large order while the stock is trading so thinly. Sometimes I gauge the amount I can buy at once via T&S, and other times I do not.
I see. It sounds like you can see what’s going on with the orders and are aware of the relationship of your quantity vs. stock volume etc. Nothing wrong with thin stocks as long as you take it into account and are still able to make money :)