Finding the perfect trading strategy is a hard, hard problem. It’s equivalent to determining future prices, when you get down to it. By giving up on perfection, though, we can arrive at something profitable while going to a lot less effort. When you go reducing a problem like that, it really pays to understand what you are doing in a mathematical sense. Many pitfalls can be avoided that way.
So, let’s talk for a bit about a strategy scientists and mathematicians use all the time. The short version of the story is: they transform a problem into a simpler problem, while retaining enough of the original problem’s structure to get the same answer. Thinking about how this works and what it means can give you ideas about how to approach technical analysis.
I know there’s a lot of math-averse folks out there, so I’ve tried to keep the examples very simple and easy to follow.
Example
What if you were presented a series of natural numbers, and you are presented with the question: “which ones can be divided evenly by 5? ” Let’s say the numbers are:
3, 5, 17, 20, 45, 47
As usual, there are lots of ways to attack the problem. One obvious answer is to just divide all the numbers by 5 and see which answers have no remainder. We’d find that the 2nd, 4th, and 5th inputs are divisible. For our example, though, let’s assume that this solution is unavailable to us for some reason (cost, time, etc.). What to do?
Being a clever person, you might study the problem and notice that if a number ends in 0 or 5, it is a multiple of 5. So, we can transform the input into:
3, 5, 7, 0, 5, 7
… and transform the question into “which numbers are 0 or 5?” You can see that we get the same answer as we would solving the original problem.
Simplification
Before moving on, we should think about what we just did. We studied the issue, and used some clever insight to:
- Change the input
- Change the question into an easier question
- Get the same answer as the original question
Of course, not just any change will do… we couldn’t have done as well replacing all the numbers with pictures of cars, after all! A key is that the transformation cuts away aspects of the problem that don’t matter for the immediate question at hand. In the case above, it didn’t actually matter what the numbers were… all we absolutely needed to know was their last digit. From the perspective of divisibility by 5, enough of the underlying structure from the input was retained to answer the question, while everything else was jettisoned.
It may not be immediately obvious, but there are at least two costs to doing this. In particular:
- It takes work to transform the input
- You often can’t get back to the original problem space
So, obviously, if it takes more work to transform the problem than it does to solve the original problem, then the so-called “simplification” is not worth it. The second consequence is far more interesting. It means that you usually end up throwing away details to get at your answer, and you can’t recover those details later.
In our “divisible by 5” example, it might be natural to follow up with “how many times did 5 go into them?” Unfortunately, once we simplify the problem, we lose the ability to answer that type of question. This is because all we’re left with are those single digits. Simplification usually throws away aspects of the problem space that don’t contribute directly to the solution. Some might argue that you could make a point of tracking what the original numbers were, but that misses the point: you’d still have to go back and solve the original problem to find out how many times 5 goes into the numbers. And, once you have to go back and solve the original problem, doing the simplification is a waste of time.
By the way, this is why mathematicians are always proving “the existence of” things, rather than just finding the things themselves! Figuring out whether something exists is often a simplified version of the problem, and the simplification process cuts the cord that could have led us to the full answer.
Approximation
Now let’s get much closer to trading, by changing our story somewhat. This time, let’s say we study the problem and notice that numbers ending in 5 are multiples of 5. However, we don’t notice that numbers ending in 0 are also multiples of 5.
So, we transform the input just as before:
3, 5, 7, 0, 5, 7
… and transform the question into “which of the inputs are 5s?” Our answer is the 2nd and 5th number.
What can we say about this solution? Well, it does find numbers that are divisible by 5… it just doesn’t find all of them. Our solution produces false negatives. In other words, if the answer is “yes” we can believe it, and if the answer is “no” then all we can say is that it “probably is not divisible by 5.”
There are plenty of algorithms just like this… algorithms for primality testing come to mind (which can always be believed if they tell you a number is composite, but are only usually correct when they declare a number is prime), but there are many others. In fact, I’m about to point out that lots of trading systems can be thought of in terms of an approximate simplification of time-series forecasting.
Technical Analysis
That last section describes the kind of approximate solution that we often find as technical analysts. Let’s think through the steps:
We are given a series of prices, and we need to know when we can profitably go long. The “obvious” answer is to forecast the series into the future and find the forecasts that go up, but that’s awfully difficult. What to do?
A clever analyst might study the problem and notice that whenever a fast moving average crosses above a slower one, price is going up. So, the analyst transforms the input from prices to moving average values. Then, the question is changed to “have the moving averages crossed over?”
Just like our approximation example above, it’s clear that this simplified problem doesn’t identify every place you could profitably trade long. There are false negatives when price moves up enough for profitable long trades without the moving averages crossing. But, we can at least say that price has changed direction whenever the moving averages have crossed.
This solution does have an additional tiny defect, though… price doesn’t always follow through after the crossover. So, not only do you get false negatives (where you could have traded but the averages didn’t cross), but you also get false positives (where the averages cross but there’s no profitable trade to take).
You will generally have both types of error when dealing with uncertain future prices. Trading is a probability game, after all! However, you should note that the two types of error are often closely related. For example, you could go with very fast averages to cut down on false negatives (you spot every turning point), but in the process you’d get a lot more false positives (whipsaw trades). Maintaining an effective balance between the two types of error is an important part of strategy development. I tend to be conservative and minimize the false positives at the expense of missing out on potentially good trades, but that’s just me.
The crossover example was used because most traders are familiar with that type of trading. In fact, a whole lot of directional trading strategies can be outlined as a simplification of time-series forecasting. The thought process goes lke this:
- Ideally, we’d figure out where price is going to go
- That’s hard, though.
- All we really need to know is “where can we make money?”
- So, here is a two-part simplification of the problem:
- We need to know which direction price is going
- We need to know if price will follow through enough for us to profit
Like the naive moving average crossover trader, lots of traders focus on the direction, and forget about follow-through. However, if you can formulate a way to answer both questions, then you can trade profitably without solving the hard problem.
You might look at the system you are trading, and ask how well (or poorly!) it fits this framework. For instance, let’s say our solution is to go long when an oscillator leaves oversold territory. We infer that price is tending to go up, since it is no longer oversold. The system also posits that price needs a sizeable breather after being oversold, which would provide the follow-through we need to profit. (Again, it’s that darn follow-through assumption that’s weak! I bet you see that pattern in a lot of well-known trading ideas…)
Note that this is not the only way to characterize a directional trading system. If you like market profile-type stuff, you might simplify your prediction problem into a question of where participants recognize value. I am sure you can think of other examples.
I’d like to point out one last time that we’ve usually thrown away a lot of information when simplifying a problem. For example, imagine the perfect simplified solution, which identifies every place where we can put on a trade and close it profitably. We still don’t necessarily know what the future prices are, or even how many points of follow-through we are going to get. Fortunately, there’s plenty of time to ponder that from our private beach, as supermodels feed us grapes.
Summary
The moral of the story is: never try to predict prices when you can get by predicting a moving average of the price. Why? Because it’s a simpler problem (you can tell because lots of price configurations would produce identical moving average values). Just beware that when you simplify and approximate your solutions, you are cutting certain cords between you and your original problem. That much is usually ok. However, you are also potentially introducing both false positive and false negative answers. Whether these are acceptable depends on the context.
I’ve been thinking some of the same things about the whole false positive, missed positive etc lately, but not as clearly as yo have put it here. I’ve also had some vague inklings about watching for and predicting follow-through as well. Thanks for the article, it helped me a lot.
That’s the beauty of a mathematical perspective: clarity! It’s also fun to think of trading issues from an information-theory slant, for example.
Great article……… worthy of the Nobel Peace Prize :)
thanks
So we need all that fancy math just to figure out what common sense already tells us?
Just kidding – I love this site for articles like this.
Ugly! Great to hear from you, man. I hope all is going well with you.