Jan 28

If you are trading with a goal of maximizing your long term profits, you should be concerned with maximizing your expectancy. But, if you are like me, and you want to keep your electricity on via trading stocks, expectancy isn’t the right measurement to use. In this article, I’ll explain how to gauge your consistency. In other words, what percentage of your days/weeks/months can you expect to be profitable? This is useful if you are designing a system, and also useful if you want to know more about the system you already use.

Note: Many of the ideas in this article are largely derived from the first 6 pages of this elitetrader thread, started by user Acrary. I’ve tried to expand on that material with more detail and graphs, to make it easier to understand. Plus, it’s a lot more fun to read without all the elitetrader noise…

Expectancy: the Premier Profitability Measure

If you want to know if your trading system is profitable over the long haul, you want to know about your expectancy. You can find lots of articles on the web about expectancy, so I won’t spend long explaining it here. Briefly, it is computed as:

Expectancy = win_rate * avg_win - loss_rate * avg_loss

For the purposes of this article, I am going to ignore exact breakeven trades. This means that the loss_rate is directly related to the win_rate, and expectancy becomes:

Expectancy = win_rate * avg_win - (1 - win_rate) * avg_loss

A positive expectancy means that if you took an infinite number of trades, you would have more money than you started with when you finished (assuming, of course, that you didn’t bust your account during a bad drawdown–see this article for more about the risk of ruin, and this article for more about comparing the risk at different account sizes). Similarly, a negative expectancy means you’d have less money after an infinite number of trades, and a 0 expectancy means you’d break even.

That’s great, but when’s the last time you made an infinite number of trades? :-) What I’d really like to know is, am I going to make money most weeks? months? years? Profitablility and consistency are not the same thing!

Consistency Graphs

So, to get started, let’s look an example consistency graph for a system with positive expectancy. There’s going to be a lot of graphs like this one in this article, so let me explain what they mean.

The horizontal axis describes a number of trades taken during a given time period. The vertical axis tells what percentage of those time periods will be profitable, given that you took that number of trades. This was computed via a monte-carlo type analysis, with 2000 trials per dot on that chart.

So, let’s say you trade the depicted system, and you tend to trade 20 times a month. This graph tells you that you will be profitable around 95% of months you trade. If you trade 30 times a day, then this graph tells you that you will be profitable about 97% of your trading days. If you trade 10 times an hour, then you will be profitable around 85% of the hours that you trade. So it works on any timeframe… you get the idea by now.

See how the graph starts out in the 70% range, and converges on 100% consistency as the number of trades goes up? In general, all graphs of positive expectancy systems look like this, in that they start out near the win rate % consistency, and converge on 100% consistency. The width of the path, and the speed of the convergence will vary.

Now, let’s look at a graph for a system with negative expectancy:

Pretty much the inverse of the positive expectancy chart. It basically says, the more you trade, the less chance you have of being profitable. As an aside, this is exactly why your best bet at a roulette table is to bet everything on one round. Roulette has a negative expectancy for the player, so the more you play, the less consistent your profits will be.

Finally, if your system has an expectancy of 0, the graphs all look like this:

… as the number of trades increases, the consistency % forms a band around 50%. Makes sense.

Effect of Expectancy Size on Consistency

So, by now, you should know that a positive expectancy is necessary both for long term profits, and consistent profits. You might wonder if a larger expectancy system will be more consistent than a smaller expectancy system. The answer, which surprises a lot of people, is: no.

Here are consistency graphs for a range of expectancies from $40/trade to $440/trade. They are only marginally different:

If you think about the meaning of expectancy, you will realize that the $440/trade system will make a lot more money than the $40/trade system. But, for any given set of trades, expectancy is clearly not the aspect of the system that governs the consistency of the profits. We’ll just have to keep looking….

Effect of Win Rate on Consistency

Well, surely, if my system wins more often, I will have more consistent profits? It turns out, winning more often increases expectancy, but does not necessarily do much for consistency of profits. Here are several consistency graphs for systems with increasing win rates. You can see that, after you get above 10 to 15 trades per day/week/month/whatever, the graphs all look about the same.




It makes sense that the win rate would have sway over the answers when there are fewer than 10 trades per trial. So few inputs go into the profitability calculation, that a little luck in either direction changes the answer. Also, at the extreme end, any 1-trade-per-trial run will have a consistency % equal to the win rate (since the single trade is profitable at exactly its win rate).

Effects of Profit Factor on Consistency

Now, if you are an astute reader, you’ve noticed that in the preceding two sections, each graph had a constant “Pf” label at the top. And, since each set of graphs had fairly constant conistency, you might conclude that this “Pf” is what really gauges the consistency of a trading system. You’d be right.

“Pf” stands for “Profit Factor.” The equation for it has the same terms as the expectancy equation, arranged differently:

Profit Factor = (win_rate * avg_win) / (loss_rate * avg_loss)

Since the equation is so similar, you can see by simple transformation that the profit factor will be 1 whenever the expectancy is 0:

  • (win_rate * avg_win) / (loss_rate * avg_loss) = 1
  • win_rate * avg_win = loss_rate * avg_loss
  • win_rate * avg _win - loss_rate * avg_loss = 0

Similarly, the profit factor will be greater than 1 whenever expectancy is greater than 0, and less than 1 whenever expectancy is less than 0.

Here are some charts of systems with increasing profit factors. They are randomly-generated systems with respect to win_rate, avg_gain, and avg_loss… the only thing I’m controlling is that the profit factor of each is rising. It’s easy to see that the profit factor is highly correlated with speed of consistency convergence of a system, and that higher profit factors require fewer trades per period to give a good guarantee of consistency.











If you think the graphs from 1.5 to 3.5 look very similar, check the y-axis! The convergence is getting much faster! (On some of these graphs, Mathematica cut off the early numbers, because it converges on the 99% range so fast that it decided to blow up the 99-to-100% range. Recall that the point for 1 trade-per-period will always be very close to whatever the win rate happens to be, no matter how fast it converges after that.)

Reading that last chart, you can see that if you find a system with a profit factor of 6, and you can trade 15 times a day with it, you will make money 99.95% of days you trade. That means you will have a losing day once every 10 years, if you trade 200 days a year. Oddly enough, I bet you feel pretty bad that day… so try not to let it throw you off! :-)

These trials confirm the advice given in the elitetrader thread, which gives the following guidelines for trading frequency versus profit factor:

Profit Factor # Trades Needed
for 95% Consistency
1.5 60
1.75 40
2.0 30
2.5 20
4.0 10

By looking at my graphs, you can see what the guidelines are for any level you want to target, for profit factors from 1.5 up to 6.

Simplifications

I already mentioned that I ignored breakeven trades. This has only a marginal effect on the outcome, while simplifying my job a bunch. If you make 10 trades a day, and 9 of them are exactly breakeven, then maybe you should do some mental translation when using graphs like these. If your truly breakeven trades are relatively rare (like they are for most people), then you will be fine.

I also assumed during the random trials that a win is always the avg_win, and a loss is always the avg_loss. I could have been more exact by making a probability distribution of wins and losses, via a standard deviation from the averages. Assuming the variance in the wins and losses is not very large, doing this wouldn’t affect the message of this article, and would just make the graphs a bit more noisy. And, I could eliminate that noise by ramping up the number of trials per dot up from 2000 to more like 1,000,000 anyway. So, I didn’t bother. However, if your trading is all over the place, then you could have much wider swings in your profitability than depicted here (especially if you don’t trade often). Do try to keep the variance in your gains and losses as small as possible, if only because it makes it easier to reason about and predict your own performance.

Conclusions

So, here are some guidelines you can take away from all this.

If you are designing a system for consistency:

  1. Maximize the profit factor.
  2. Double-check your win rate to make sure your risk of ruin is acceptable
  3. Trade as many times as you can, within the parameters of the system.
  4. Get the variability of your returns under control (using something like the modified sharpe ratio, for example)
  5. Tune the expectancy (or add additional systems) in order to make enough money.

If you are designing a system for maximum profit:

  1. Maximize the expectancy.
  2. Double-check your win rate to make sure your risk of ruin is acceptable
  3. Trade as many times as you can, within the parameters of the system
  4. Use the profit factor to frame your month-to-month profitability expectations. No reason to be down on yourself for a losing month, if your system should only win 60% of all months!

If you already trade a system, and you want to be more consistently profitable:

  1. Find more trades to take within your system’s parameters. This doesn’t mean you start taking questionable trades, just to get the count up. Instead, you have to be more efficient about finding and exploiting opportunities to trade.

(this is why you’ve seen me post every now and then about needing to find more trades to take… it’s the only way I know of to be more consistent, without changing my system!)

If you’d like to see the Mathematica notebook that I used for this investigation, you can read the html version of it.

Aug 4

Ever wonder why a 1:3 Risk:Reward ratio is so prevalent as a rule of thumb? Ever since I’ve been reading about trading, I’ve heard people say that they ideally look for at least a 1:3 ratio, even if they’ll settle for 1:2. In this article, I’ll talk about how you can derive this value form the expectancy equation. Further, I’ll show how you can predict what kind of Risk:Reward ratio is right for you.

If you are math-averse, you can skip the equations; examples are given and explained.

What Should My Minimum Typical Gain Be?
Let’s say the minimum acceptable trading performance is flat. That is, at a minimum, you should not be losing money over time. How much money should you be making on your winning trades to achieve this performance? If you know your historical win rate, and assume your typical loss is a full 1 R, then this is the equation you need:

gain = (1 - winrate) / winrate

This equation answers the question “What typical R gain will make my expectancy equal 0, assuming I always lose a full 1 R when I lose?” So, obviously, typical gains greater than this value will give you a positive expectancy.

Let’s work an example: Say my win rate is 40%. Then, the equation above is (1 - .40)/.40, or 1.5. That means I should only enter trades where I average at least a 1.5 R gain when I win. If I always trade at that minimum, my account will not grow or shrink, long-term. To intuitively see how this works, imagine I take 100 trades. I will win 40 of them for 1.5 R each. I will lose 60 of them for -1 R each. That’s a gain of 60 R and a loss of 60 R. My account hasn’t budged. Taking trades with a reward smaller than 1.5 R would cause my account to lose money over time, at this win rate. In the same way, taking trades with a reward greater than 1.5 R would cause my account to grow.

Here’s a few more values:

Win Rate Needed Gain
20% 4 R
30% 2.33 R
40% 1.5 R
50% 1 R
60% 0.67 R
70% 0.43 R
80% 0.25 R

Translate this to Risk:Reward
So, is the above saying a 40% winner should enter trades whenever they estimate a Risk:Reward of 1:1.5 or better? Probably NOT. You see, if you are like me, or any other traders I know, you don’t hit your estimated reward target all the time (or even most of the time). So, you need to understand, based on your trading history, what kind of adjustment to make in order to get the gains you need.

The adjustment is to divide the R gain you need by the percentage of your reward target that you tend to win. The example will make this clearer…

So, to continue the above example, recall that my win rate is 40%. I know from the table above that I need to average a minimum of 1.5 R gain on my winning trades. But what Risk:Reward estimate gets me a 1.5 R gain when I win? I’ll need to look through my trading journal. My records show that I tend to take home about 50% of the reward I estimate for winning trades. So, I’ll divide that 1.5 R by 0.5, to get 3 R. That means I should enter trades when I estimate a 1:3 Risk:Reward. That way, I’ll tend to win 1.5 R or greater, and my account will be healthy.

… And there’s our magic 1:3 number! My example case was not an accident. You see, I think the number is so prevalent because of what you can assume about the typical profitable trader. It’s pretty safe to assume that they generally win 40% or more of the time, and that they get around half the gains they thought they would. Given that, 1:3 pops right out as the kind of Risk:Reward they should be looking for.

And, it follows that if your stats aren’t anywhere close to that hypothetical trader’s, then 1:3 is a meaningless guideline for you. For instance, if you win 60% of the time, and tend to take home a third of the projected reward, then 1:2 is a more appropriate minimum ratio. Of course, we are calculating the minimum for profitability here… traders should always strive to find the best risk:reward trades available within their style of trading.

Summary
After reading this article, you should have an idea of why conventional wisdom says 1:3 Risk:Reward ratios are good. You should also have the tools you need to find out what your minimum Risk:Reward targets should be. If you need help with this calculation (which is not to be construed as investment advice), just drop me a line.

Jun 21

I was listening to this excellent audio presentation that ugly posted about, and it got me thinking yet again about expectancy. The audio was about how people have trouble with probability. One example characterizes lotteries as a stupidity tax, because the odds of winning are so remote. But, what is the expectancy of playing the lottery?

The odds of winning the Texas lottery (according to this site) are 1 in 15,890,700. So, simplifying away all the small cash prizes you can hit, the expectancy of a lottery ticket is:

(1/15890700)*jackpot - (15890699/15890700)*ticket_price

So, knowing the ticket price is $1, you can ask yourself “how big does the jackpot need to be before the lottery has a positive expectancy for me?” And, of course, the answer is: $15,890,701.

But does that mean it’s a good idea to play the lottery any time the jackpot gets that high (and it does get that high with some regularity)? What about a jackpot of $28mil? In that case, the expectancy is that you will make 76 cents of profit per ticket you buy! You should buy as many as you can, then, right? WRONG! Of course wrong. Because, like many statistical measures, expectancy only tends to “come true” as the number of times you play tends towards infinity.

In other words, if you had the funds and time to buy ticket after ticket, then after playing a few hundred million times, you would expect your winnings to converge on that 76 cents of profit per ticket figure. Do you have the time or money to do that? If so, what the heck are you doing playing the lottery?!?

Apply that to the Stock Market

This is just another way to point out that, yes, expectancy tells us we can be profitable stock traders while winning less than 50% of the time. But, you can’t stop there. A system with an expectancy of 76 cents profit for every dollar traded could be just as bad a deal as the lottery above. You should strongly prefer to win as much as possible, and not because it’s psychologically pleasant. Winning more is less risky–it will lessen the probability of financial ruin when strings of bad luck happen (and you must assume they WILL happen–see my previous post about this). I feel so strongly about this that I would prefer a system with a slightly lower expectancy, if it had a significantly better winning percentage. Trust me, the dull ache of slightly lower returns as the number of trades approaches infinity means little compared to the sharp pain of applying for a day job to rebuild your trading stake.

(note: I just keep hitting this point because I don’t ever see anyone else do it, and I think everyone should keep these facts in mind. Measures like expectancy live in an abstract world where you can trade with a negative account balance until your losing streak is over. In the real world, you are job hunting and eating ramen noodles long before that. All that said, I still like, use, and recommend expectancy as the first measure of a trading system’s viability. You just can’t stop there.)

May 20

We are all, theoretically, at risk of total financial ruin. Imagine you find a system with a 99.9% success rate, an average per-trade gain of 50% equity, and an average loss of only 0.01% of equity. You would use it, right? So would I! After all, the expectancy would be: .999 * 50 - (0.001 * 0.01) = 49.95% average equity gain on every trade!

BUT, it is not impossible for you to pick this system, and empty your account without making a penny. Think about it… you don’t expect to flip a coin and get tails 100 times in a row, but you know that it’s theoretically possible, right? Consider, too, that the average loss of 0.01% of equity is the AVERAGE loss. In theory, you could use the above system and blow your whole account on one trade. How likely this is depends on the variance associated with that average.

What can we learn from this line of thinking? Well, let’s start by getting it straight in our heads that, almost all the common statistical measures of trading success are only accurate under two conditions:

  • You make an Infinite number of trades
  • You can trade with a negative account balance

Obviously, neither of these conditions match reality. So, they can easily mislead you. The above example’s amazing expectancy is only guaranteed to materialize as the number of trades I make approaches infinity, and if I could somehow still trade with a $10 account.

Does this mean expectancy, and other such measures, are useless? No, but it does mean we need to take extra steps help ensure that our trading account doesn’t vanish. A small account doesn’t have the padding to weather long strings of losses. So early on, it can even make sense to choose a worse-performing system that has better-bounded worst case behavior. Right now, I’m thinking of three specific steps:

  • Always risk a % of current equity
  • Prefer systems with higher win rates and lower avg losses
  • Prefer systems with less variance in the average loss data

1. Always risk a % of current equity

Clearly, if you want to stay in this game, you have to make money in the long run. But, you also need to never go broke in the short run. I’m going to define broke as $25k, since at that point your pattern day-trading account will be frozen. The conclusions are the same if you define broke to be $0.

Let’s say you start with $100k, and you risk a fixed 2%, or $2k, per trade. Your account will be frozen if you lose 38R. In the worst case (assuming away slippage), this means losing 1R 38 times in a row. (100k - 38*2k) = 24k = busted

Now assume you start with $100k, and you risk 2% of current equity per trade. This way, you would have to lose 69 times in a row before your account is frozen. (100K * (0.98)^69) = 24.8k = busted

Obviously, it is more likely you will go bust with a fixed risk. In the coin toss analogy, 38 consecutive tails is much more likely than 69 consecutive tails. (Again, never forget that 100 million consecutive tails is still possible, no matter how unlikely).

2. Prefer systems with higher win rates and lower avg losses

When talking about expectancy, it’s common for people to point out that you can win less than half the time, and still make money. They should really add the qualifiers “in the long run, if you don’t go broke first.” We’ll start with an absurd example, and then get more realistic:

  • winRate = 3%
  • avgGain = 200%
  • avgLoss = 4%

… expectancy of 2.12 % equity gain per trade. Better than many systems! But, not usable in practice. At 4% avg loss, it would take 34 consecutive losing trades to bust a $100k stake. With a 3% win rate, there is (1 - 0.03)^34 = 35% chance that this will occur. WAY too risky. In general, the probability of busting an account without any wins is: (1 - winRate)^( (ln (25000/initialStake)) / (ln (1 - avgLoss)) ). Incidentally, that exponent will be the number of consecutive losing trades needed to bring down the account.

So, when comparing any two systems (or versions of the same system, if you are building one), it can be helpful to compare their probability of immediate ruin as well as their expectancy. In general (due to the equation above) higher win rates and lower average losses will perform better on the immediate ruin test.

Certainly, if two systems have the same expectancy, then you should prefer the one with the smaller probability of immediate ruin. That goes without saying. But, what about these two systems:

  • winRate = 40%
  • avgGain = 2%
  • avgLoss = 1%

… expectancy of 0.20% equity gain per trade.

  • winRate = 60%
  • avgGain = 0.6%
  • avgLoss = 0.5%

… expectancy of 0.16% equity gain per trade.

After 500 trades with a $100k stake, the first system is expected to make about $50k more than the second. But, it is also 2.8 x 10^79 times more likely to burn an account straight down to the $25k freeze point without winning once.

Now, at this point, you may do the math and point out that the probability of immediate ruin on the first system is only 2.5 x 10^-29 % in the first place. Isn’t that good enough that you wouldn’t want to give up $50k expected gains for more protection? I actually don’t know yet. It seems to me like the only way to answer would be to work out all 2^500 strings of trades and get a probability distribution of the outcomes. I can only say that minimizing that straight drop path’s probability should also reduce the probability of the other account busting paths, since all losing paths will have at least as many losses as the straight drop.

3. Prefer systems with less variance in the avg loss data

With the analysis we just did, on the probability of the account going straight down, we assumed we always experienced the average loss. This is another one of those statistical generalities that can mess you up. If the variance in the loss samples is large, then the average loss across a small number of trades could be way out of line with the overall average loss. Not only would that somewhat invalidate our supposed “worst case” analysis, but it could decimate our trading account!

For example, an average loss of 3% equity could have come about from either of these two strings of losses, for instance:

  • string1 = 10% 5% 1% 1% 0.7% 0.25%
  • string2 = 3% 4% 2% 3% 4% 2%

Across the first 2 losses, though, string1 averaged 7.5%, while string2 was still pretty much in line with 3.5%. And that’s just the difference between using 2 samples and 6! If 100’s of losses went into the expectancy calculation, then any string of 5 losses could could hurt your account far worse than the average would indicate.

As it happens, one way to remove some of the variation from the losses is to prefer systems that incorporate stops (lots of systems don’t!). This puts a cap on the loss amount, which has a side effect of making losses much more predictable. Further, in any worst-case analysis, you can assume all losing trades hit the stop, rather than assuming the average loss. That’s a very safe and conservative path to follow.

Why don’t all systems have stops, if they have so many desirable properties? Well, it would appear that lots of winning systems require you to ride out substantial paper losses on some of their trades, prior to exiting at a profit. Note that this is also why you can’t just add stops to a system that doesn’t have them… they wouldn’t be nearly as successful if they were stopped out in those cases.

But, for the small ($100k or less) account, I’m starting to think that systems without stops should be deemed too risky to use, no matter what the expectancy is. Small accounts just can’t take enough oversized losses to make it. My backtesting seems to bear this assumption out.