You know, when I wrote Friday’s article, I wasn’t planning at all to get into a discussion about artificial intelligence in trading. I was just pleased with the notion that institutional program trading would either (1) never lock out the little guys and gals, or (2) trading would cease to serve its economic function, and would be replaced by something else. Other than an offhand statement that amateurs may be using super-fast AI agents one day, that part of the topic just wasn’t on my mind.
But, Ugly came out almost right away to remind me about the likelihood of eventual computer trading agents that surpass humans in trading ability. The way software tends to go, once that’s anywhere, it will be everywhere soon after. When the bar for the amateur trader is raised, you need to make sure you’re still on the right side of that bar!
Current program trading, to the best of my understanding, is mostly filling complex orders on behalf of humans, at the behest of humans. Then there are the quant funds, which at present occasionally still spaz out. I would expect both of these to continue to get more sophisticated, but they really aren’t in direct competition with the small daytrader.
Look around, though, and you’ll see that more and more scalping black-boxes are coming on-line. They are not too impressive right now (though they at least make a profit, which is admittedly better than most human traders manage to do). These mechanical trading systems will also improve over time, but they still aren’t quite what I think of when I picture the trading agent of the future.
AI Trading Agents
I don’t know what you picture when you think of these hypothetical advanced trading agents. I can tell you what I imagine. I imagine a discretionary trader just like myself, with notable exceptions:
- it spots opportunities faster
- it acts upon those opportunities faster
- it does so across multiple markets, simultaneously, 24 hours a day
- it never trades out of boredom, or revenge, or desperation
- it has a perfect memory of past events and performance, which it can use to continually learn and improve
- …you get the picture by now, I hope
In short, it has more of every quality that I think makes me a good trader. I would kill to have these qualities. As the percentage of traders with these qualities increases, I would expect my trading expectancy to decrease. At some point, my expectancy would drop to 0.
Is it really so hard to see why that is? You can readily see it in today’s markets. If you press the buy/sell button too late (or are too far back in the order queue), you either get filled at a worse price than you wanted or you don’t get filled at all. When you are filled at a worse price, your potential profit decreases, while the risk you take on increases. Your expectancy drops. And that’s assuming all of your analysis was correct, and the order wasn’t fat-fingered, etc. etc. Every time I try to trade while groggy or sick, I make less money. Hell, every time my internet connection gets a little laggy, I make less money. There’s not much room for error, even today.
There are people who conjecture that as a group of faster, smarter traders eats up the opportunities you have now, they will create new opportunities for profit in the process. Do you know who I would expect to find and exploit those new opportunities first? The smarter, faster traders, that’s who. Leaving me in the dust yet again.
There’s nothing special about the fact that we are talking about computer programs… I would expect the same result if I were slower and dumber than most other human traders. Since I don’t seem to fall into that category, based on my performance to date, I don’t fear human traders. But look at the progress of computers in cognitive and creative areas, though… something would have to stop the trend they’re on to keep them from catching up with me and then surpassing me.
I don’t know what that barrier would be, that can stop the progress of technology. I feel it’s safest to assume the trend will continue.
It Only Gets Harder
Up to now, we are mainly talking about computerized trading agents that are somewhat better than good human traders. If you fast-forward a bit, it gets much worse for the poor unassisted human. You see, as computers continue to gain speed and cognitive ability, the divide between them and plain old humans will continue to widen. So maybe earlier you felt like a high school basketball player playing in an NBA game. If you are one of the few very talented, you could probably still compete. Eventually, though, you will feel like my cat playing in an NBA game…. fearful of all the strange action and unable to comprehend what’s really going on.
I can imagine a future market where the only exploitable edges left have more features to consider than the human brain can easily process at once (see any cognitive psychology textbook for more about these limitations). At this point, the unassisted human literally cannot identify the opportunities in time to take advantage of them.
Doesn’t this raise computers into some sort of Magical/Mystical object?
I was really surprised when I saw this claim on Ugly’s comment section. The AI perspective is that there is nothing unique or special about human intelligence, and that its underlying mechanisms can be emulated and improved upon. To claim computers cannot duplicate human intelligence raises the human brain into some sort of magical/mystical object, as far as I’m concerned.
But Unlike Current AI Conquests, the Markets Are Always Changing…
Yeah, and chess is harder than checkers. And poker has more human elements than chess. The next step is always more complicated than the last step, every step of the way. And yet, progress is still accelerating. I mean, the software credibly challenging poker experts today runs on a cheap Apple laptop…
To me, wars and other news events are elements that make the markets hard for everyone to navigate. I guess, the people harping on this are coming from the perspective that humans currently have the advantage when adapting to unforseen elements. If you believe that intelligence on par with humanity can be duplicated, then it must be obvious to you that this will not always be the case.
But, The Markets Aren’t a Problem with a Solution
They don’t have to be. Software exists that is beyond problem solving via pre-set recipe. It has been around for a long time, in the ivory towers. In my opinion, it will become more mainstream relatively quickly. As cognitive and creative capability in software increases, computers will dynamically find and exploit opportunities based on the simple goal to make a profit. Just like I do. To claim that this is impossible makes my intelligence into something mystical and magical, and I just do not believe in that theory.
So, What Can We Do?
Keep up, that’s what. Like I said in Friday’s article, your goal is to stay at least one step ahead of most amateurs. The technology available to the small trader is getting more sophisticated all the time. Take advantage of it. Look at tools like trade-ideas, which help you spot all kinds of conditions in real-time, and their associated features to quantify which strategies seem to be working right now. Look at platforms like tradestation or multicharts that allow you to test and identify all kinds of complicated, cross-market cross-timeframe conditions. And on and on.
Learn some programming basics, so that you can offload as much of your analysis as you can onto your computer. Do this because more and more of your neighbors are doing it, and it’s making them faster and more accurate than you are. Slowly but surely, you will start to lose if you don’t.
Extrapolate far enough, and when most small players are autotrading, in my opinion it’s fairly easy to see that you had better be on board.
Where’s Your Proof?
There is no proof for any of this. I can’t prove that the sun will come up tomorrow, either. Like the magic 8-ball, I just know that all signs point to ‘yes.’ When you think about the future, you usually extrapolate from the present. In the recent threads, Ugly’s been kind enough to provide links to various current activities in this space. Even a little undergraduate work in AI will open up your eyes to what is possible, given time and processing power. This post is just my conjecture. You are free to reach your own conclusions.
Also feel free to leave either agreeing or disagreeing comments below. I certainly don’t have a lock on the future. The more the merrier. It’s fun to imagine the future of technology when the discussion is genuine and light-hearted. Last time, it got to the point that I sat through comments deconstructing my word choices. I will try my best not to participate in that level of discussion again–it frustrates and brings out the worst in me. Let’s try to be civil, and intelligent, this time. Thanks.
November 4th, 2007 at 2:12 pm
Nice job Richard - thanks for putting everything I was thinking into a nice structured post.
November 4th, 2007 at 2:24 pm
[...] think everyone should read Richard’s post about the importance of traders keeping up with technology (and what this may mean). I agree 100% with everything he says. I’m too lazy to write a long, [...]
November 4th, 2007 at 3:14 pm
when I say “everything I was thinking” - I just mean I agree with you, I didn’t mean to take any credit.
I think a lot of people resist the idea of computers becoming as smart as them because of ego reasons. It’s understandable, we all want to feel important and the idea of a computer being better than us at every detail can be humbling. I think this is especially true with traders.
November 4th, 2007 at 3:21 pm
I’ve been saying this for a while and usually get laughed at. We as small traders need to keep up with technology so that we can stay in the game. The money and the resources that big institutions put up for AI will always force us to take a back seat. But that doesn’t mean we should not be in the game.
If you have rules and can get those rules on paper, then you will eventually be able to get into code. While pure coding will not make you a good trader, it can as you mentioned, unload a certain portion of the work for us to concentrate on other things.
This is not about creating magical system. The human mind will always be more powerful but it will never be able to keep up with the number of tasks that a computer can. I have a friend that used to manage 60 positions at a time through TS. Can a single human do that?
Great post.
November 4th, 2007 at 5:09 pm
Richard,
Very good post, thank’s for your insights.
November 4th, 2007 at 5:47 pm
richard: i don’t see where any of tyro’s comments were out of line (either on this site or ugly’s)
November 4th, 2007 at 8:00 pm
It’s just me, I’m sure. I personally find Tyro’s style of “discussion” extremely tiring. I remember when we were discussing 0-sum games wishing I were dead. I’m sure other people (possibly including yourself) find his style delightful. It drives me crazy, though. I guess, I give him credit for more intelligence than he was putting on display. Not because he disagrees with me, but because he seemed incapable of understanding what ugly and I were trying to tell him. The focus on minutiae and allusions to mysticism, coupled with requests for logic-based evidence when his claims all take the form “I can’t imagine X” wear me down very quickly. As I said on ugly’s blog, my low tolerance for stuff like that is my limitation, and not his. I am truly sorry that I couldn’t keep going until I could help him out. Especially since I think some of his claims with regard to how breakouts work are way off base, and possibly hurting his performance in today’s markets.
November 4th, 2007 at 11:49 pm
i thought the original purpose what ugly and u (i think) were saying is that there would be no place for the small time trader after computers take over…when it comes to chess, and possibly even poker (i don’t know), a computer can beat a human every time…however, u may find this tiring, but as long as prices move, there will be opportunity — especially for a small time trader who can pick his spots…somehow the discussion seemed to get sidetracked about who is better at trading…of course computers surpass humans when it comes to certain things, but IMO discretionary trading is just that — i’m not sure that a computer can match that
a computer can trade better than me, but that is setting the bar fairly low :-)
November 4th, 2007 at 11:59 pm
excuse me for beating a dead horse, but i haven’t seen anyone address this issue…traders/investors all have different time frames, and profit/stop targets…i don’t have to tell u about all of the different styles…if your scenario is correct, how would that stop? wouldn’t that in itself provide opportunity for the small time trader?
November 5th, 2007 at 12:24 am
BAT: Ok, so Article 1 from friday says “there will always be room for the small time trader, or trading will become obsolete.” So, in my opinion, prices will always move, and opportunities will always exist in some form in some market.
Article 2 (this article) says “the small time trader is getting more and more sophisticated, and you need to keep up.” Today, most advanced tech is playing a game that you aren’t playing, so it’s mostly irrelevant to you… it’s just those annoying buy and sell programs that fuck up the charts a couple times a day. In the future, though, I imagine you will be competing directly against advanced tech as employed by other small traders, and I really think the only way to stay competitive is to fight tech with tech.
The rest of the article is just my personal speculation about what the advanced tech will look like. It could obviously be completely wrong, and even if it’s right I have no idea what kind of timetable it plays out on. It’s congruent with futurist books I have read by people who presumably have spent a lot more time than me researching the topic.
People who do not agree that tech can mature beyond a certain point will obviously think most of the second article is full of shit.
As far as I can reason, in the abstract sense, all opportunities are measured in shares. When the shares are gone, so is the opportunity. It’s hard to even speculate, but I would imagine increased tech competition will make edges even thinner than today. So, the human trader watching a couple markets and usually showing up last to the party will just not be viable compared to the faster tech-assisted guy next door trading tirelessly 20 markets 24hrs a day.
November 5th, 2007 at 7:16 am
Hey Richard, you’ll probably think that I am here to critize this post since I always completely agreed with Tyro regarding this issue, but I found your post suprisingly good. You make very good points and it shows that you have thought deeply about this. I completely agree that this development is going to make it harder for human day traders because software programs are indeed better executing day trading strategies. But most transactions in the markets are on a longer time frame (studies say 3-6 months on average) where execution doesn’t make much of a differece. So I don’t think that trading software is going to cause markets to cease. Not only that, but there are many (maybe oldfashioned) people who don’t trust computers in general. They hear computers and think LTCM or some other quant fund that blew up.
Of course, I still disagree about the intelligence and AI part. It’s not magical, it’s just extremely hard to do. I don’t think the limitation is processing power, but there is nobody smart enough to develop true artificial intelligence beyond some little stupid domain (like games with clearly defined rules), which do not require intelligence, but processing power. I don’t know what you think artifical intelligence in trading would look like in concrete terms, but I guess your version would be just a continously self-reoptimizing trading system, which we already have today and has it’s own problems, but again there is nothing intelligent about that.
November 5th, 2007 at 8:58 am
Hi Hermann, that was a great comment, and there’s nothing wrong with agreeing with Tyro :-) I am mainly concerned with what happens to daytraders because I am one. Longer term strategies do seem likely to be a more level playing field, at least initially. I guess it depends on how advanced you think the software will eventually get.
I think there will be lots of incremental steps taking who knows how many years, but in the end I would expect the AI trading would look just like mine, in concrete terms. Kind of a trading turing test situation, I guess! And then the next version would look better than mine…
November 5th, 2007 at 10:45 am
I’m personally most interested in systems design and system trading. I’ve spent a fair bit of time scouring websites. Two of the more interesting ones are the Neural Shell (www.neuroshell.com) and the performance of the futures systems designed by the minianalyst (www.minianalyst.com).
I’m sure there are humans that can achieve 300% returns for four years running but not many. The returns at minianalyst are rather remarkable.
November 5th, 2007 at 1:07 pm
Richard, I’ve never been asking for proof, just an argument that follows through and some minimal specifics. Maybe you can’t “prove” that the sun will rise tomorrow, but I think there’s a pretty big difference between the support for the sun rising and the support for computers taking over trading. And I wasn’t attacking your word choice, just the argument that because someone else was wrong, you are right. Hopefully you understand these points and aren’t too personally upset, and are just taking some lighthearted jabs at me. If not, I apologize for my wording.
Anyhoo, I tried to distill my ideas into a funny story with alcoholic fish in the hopes that it might explain why I still disagree with your conclusions even though I agree with many of your steps. I hope I didn’t go too far with the snide comments. I can’t think of anything more to say, so again hopefully this will be the end from me.
November 5th, 2007 at 5:11 pm
I will link to your article, which you obviously put some effort into. Hopefully anyone that cares about this argument will read it, and the different perspective will spark more thought.
Sigh… I never said you asked for proof, or that you attacked anything. Do you know what “deconstruct” means? Hint: it’s not “to attack.”
Though your post says all your requests were met with silence, I can think of three separate times I’ve tried to describe conditions which would be very hard on humans. One of those times is in this very post! You are either ignoring them or not noticing them, or something, because you have yet to so much as mention them… instead you focus on irrelevant side issues like whether or not anyone actually said that computers couldn’t do calculus long ago.
In other words, I think the reason you frustrate me so much, is that I can’t seem to communicate with you. For all I know, it’s mutual and you think I am misunderstanding everything you are saying. At this point, I can live with that! I just want to move on to something less aggravating.
November 5th, 2007 at 6:46 pm
And to clear things up, neither Richard or I said that computers are going to solve the market. Nobody knows what will happen. I just think, in my humble opinion, that (as Richard very nicely lays out above) computers and AI will make the markets so competitive that it will be impossible for an unassisted human to make consistent money. They don’t need to solve it to do that. A lot of people I respect (including Tyro) disagree - that’s fine. We can all have our own predictions.
But I think it is naive to say that AI can’t be developed “beyond some little stupid domain (like games with clearly defined rules).” It is already beyond this today.
November 6th, 2007 at 8:19 am
I will throw a different spin on what I mean - I believe that given enough computer power - no man should ever beat a computer at chess or checkers (maybe poker). Computers are a fact of life in investing.
I have heard every year of complaints about computer buy and sell programs. I do not know if you are old enough but before computers, I heard of complaints about buy and sell programs too. They were just not called that. More like young idiot gun slingers movig money around quickly.
Yes - computers can make a lot of money quickly figuring out how to price points are out of place. In ways that humans can not even figure out after hours. Yes using computers you can move money around like crazy.
My main two points are that computers will only be as good as the humans who program them and I do not see them making it harder for traders to make money. The markets and the tech to trade thme, is always changing and as long as you change with them, you have a chance.
If I understand what you guys are writting about, that of a few poeple on this planet will be able to trade and make money with in the next 20 years and the rest of us should take our toys and go home. I disagree agree with that because until you can program fear and greed, I think traders will be safe to make a living.
November 6th, 2007 at 9:18 am
Hi Jeff. Yeah if you look at some of the comments I’ve left above, or Friday’s article, you’ll see that I agree with you about buy and sell programs. They are really not a concern for the small trader, no matter how sophisticated they get, in my opinion.
So, where we start to disagree is when you say that computers will always only be as good as the humans that program them. For instance, I myself have made a chess-playing program that I cannot defeat. This doesn’t prove that someone can make a program better than me at trading, but it’s the kind of evidence I look at and extrapolate from.
I believe that eventually computers will understand fear and greed better than we do. I am a little more optimistic than your last paragraph… it won’t be that only a few people will be able to make money. I just believe that all money-making traders will be using advanced tech to trade. To me this is just like how many people used to believe that no one would want a “personal” computer in their home. Very quickly, they were everywhere…
No matter what happens, it will be very interesting to see how technology changes the game going forward. Thanks for your perspective!
November 6th, 2007 at 10:52 pm
I won’t rehash what has already been said, but instead I’d like to contribute this article, which I bring up to just about everyone I talk tech with.
http://www.kurzweilai.net/articles/art0134.html?printable=1
It has nothing to do with trading, it’s about the exponential rate of technological change. The jist is that technology progress is actually speeding up, to the point where we cannot foresee what will come next.
I for one welcome our computerized trading overlords.
November 6th, 2007 at 11:42 pm
Tecla: I agree 100%
November 7th, 2007 at 4:32 pm
One thing nobody has discussed and that is the fact that computerised trading will affect the nature of markets. What we will have will be akin to a simulation and the stock market will essentially be flat. This will be the consequence of everyone using an optimized startegy.
The prices of all shares will represent a rational value and variability will be ironed out.
November 10th, 2007 at 11:23 am
>The prices of all shares will represent a rational value and variability will be ironed out.
What you are talking about is the age-old ‘Random Walk’ theory of markets. There are a number of famous mathematicians that agree with you. The obvious problem is that everybody “knows someone” that makes more than an index fund does. I made *way* more than any index fund in the last 12 months. I assume that you did as well, so we’ll just call bull*hit on that theory. lol.
What is true is that once you are trading too many contracts on a particular system that you will not get fills anymore. Generally speaking, that’s called ‘arbitrage’. Eventually any flaw in the markets will be found and exploited by someone smart. They will rake in money until too many people figure out how the system works.
There’s always a fine line when an automated system company starts selling their system to others…it’ll stop working once too many people are trading too many contracts on it. That’s why most providers, like http://www.breakoutfutures.com, make it rather clear that they don’t personally use the system that they are selling…they use one that they will never sell. The reason is obvious.