Once I’ve finished talking about the EMA and a couple variations on the EMA, I’ll move on to some basic applications of wavelets in the trading world. I had done work with wavelet analysis for music (de)composition in college, and have always intended to apply some of those techniques to market data. I just haven’t gotten around to it, until now.
But, as with most things I’ve done in the past, I never got a thorough understanding of what I was up to… I just skimmed forward to parts I could try to code. So, I’m correcting that now. I’ve ordered An Introduction to Wavelets and Other Filtering Methods in Finance and Economics, and am also looking at Wavelet Methods for Time Series Analysis. Anyone have any experience with either of those books?
*insert pat on back for awesome* here. Couldnt find the button to do so.
Soon I can see “Richard’s Music Bars” or different systems named after composers and the like :)
E.
Yeah, I don’t know if the “pat on the back” button is coming back or not. I kinda like the way the site looks now, without adding a bunch of widgets to it. We’ll see…
Excellent. I’ve been wanting to look into wavelets but haven’t yet. I’m still not sure about the whole fixed frequency analysis thing, though… especially when we have what I consider a non-linear or at least arbitrary time axis. I wonder how many things would open up by messing around to find the right time axis. How about sumi(Ai+Bi*sin( sumj(Cj*sin(Dj*x+E)+F)… ah, you get the idea. Nested sin’s or a cubic in there or… whatever. I started messing with that–using genetic algorithm to find coefficients. Conventional (newton and the like) methods just can’t handle that–at least the engine I tried didn’t. Tried GA, didn’t work at first but after adding in a couple new evolutionary operators, I moved a step along the path. Not there yet, though–but fun stuff.
Yes, it is an interesting question. Since we’re assuming there’s a signal trapped in some noisy data, it should be differentiable from noise via nearly any sampling method, provided the data is oversampled. But there may be perspectives/transformations that make the process easier or the answer more useful. I’m not sure.
I think undersampling is probably the biggest problem most technical analysts deal with, whether they know it or not. I mean, how accurately can you really track turns on a 4-tick range bar chart, when there are like 0 bars of leeway on either side of it before the signal starts to be too late? That’s the really hard part about the indicators I release through eotpro… we use them on pretty charts, when slow settings on fast charts are easier to make accurate.
Yeah, one of my frustrations with ninjatrader is I can’t scrunch up a one tick chart indefinitely. There are indicators that are much easier to program on a 1 tick chart but then NT will only squeeze it so far, so… it gets spread out too much sometimes.
I have a “ZoomPanel” thing I developed where I can see the whole day’s data tick by tick, zoom in/out, scale, etc. I have recorded the full market data (trade/bid/ask/depth) so I can do analysis on things that typically have to be realtime in most platforms.
There are two schools of thought, though: is the signal in noisy data (signal processing approach), or are there behavior/patterns hidden in the low level data that shows up here and there that can show the direction (trade intensity idea is one example of such a thing).
Isn’t the second school just looking for a specific kind of signal?
Yeah, I wasn’t clear. I just realized what I was trying to say I think.
The one is a continuous signal, and the other is it’s all noise except for certain events that happen at discrete points.
Not sure… just something I’ve been thinking about. Kind of the difference between signal processing approach and what I’m calling “activity analysis” looking for “interesting” behavior in low level data.
I see what you mean. Maybe I’m too abstract, but I still see that as a signal that’s zero most of the time. Like the result of edge-detecting filters on 2D images, for example.
Richard: right, my comment was more from a perspective of how one goes about trying to find the signal: top down vs. bottom up. Looking at individual ticks and groups of them for behavior, vs. trying to filter or curve fit to the set of data over time.
Anyway… interesting stuff.
Hi Richard & taotree. Very interesting ideas, as usual for Richard’s blog :-). I’m no where near the math level of understanding that you guys are at, but I wanted to share some observations and see if it provokes any thoughts. (Sorry in advance if I’m off topic due to my misunderstanding) 1) concerning the assumption of a signal trapped in some noisy data: it’s interesting to me that on many institutional platforms (i.e. RCG Onyx and the many proprietary platforms), one built in feature is for the system to enter an iceberg order with the option to break up the order using a random size function and a random timing function within adjustable parameters/depending on other signals/etc. Now, I now the tricks get even trickier than that (i.e. selling a little, in order to buy more combined with random sizing), however, just the idea that many big players are executing orders with the assistance of a computer running a “random” algo gives us evidence of intentional noise and a true signal(s) behind it, right? The image of tracking a missile that has AI intentionally trying to fuzzy up any tracking devices ability by moving around all funny comes to mind. I understand that in addition to the data coming from such players not all such players are acting in unison and there are tons of other participants that create a more unintended noise, but so many participants are getting hip to tracking large group activity in certain common ways (volume, intensity, inventory levels, market breadth from equities) that perhaps there’s a sort of group mind signal that develops secondarily as a certain number of market participants decipher correctly the dominant signal inside of the intentional random algo order activity 2) I’m intrigued by the idea of market signals that are sequential & symbiotically structured in a sort of predator/food web hierarchy in the markets, where the elite lion (goldman sachs) due to their superior attributes is able to normally get the best of the catch (organ meats & best fill prices), then come the next level predator (wild dogs & the well equipped traders like Tradepointtechnologies) who benefit from the lion’s ability to get the fast prey down, then the smaller rodents, bugs, fungi (us) get our turn (maybe the fungi would be complex option players or would it be buy-and-holders?). In such a web of interactivity there’s bound to be all kinds of noise: weird weather, spaces between events, interruptions to the order, maybe chaotic new information like a certain player not showing up or showing up later than normal, a coke bottle falling out of a plane over head…but there’s definitely a series of signals there and well worn signal paths that are generally understood/accepted/expected by the participants amidst any noise around/between. What I also find fascinating in that scenario is that big players are actually helping smaller players by creating waves and stop/start points for waves and smaller players are helping larger players by buying/selling the bigger guys inventory off them on the way up and way down….wherein the big players do a bunch of hard work and if we can be in tune with that we can ride the pulse from the work and we help them finish the work….again signals within signals within noise. 3) according to some conversations I had with a fellow who developed his own zoompanel of every tick with bid/ask/depth/size and inventory analysis and watching his youtube videos (futurescalper), seems like in many markets there is definitely a signal in the individual sawtooths of a larger saw cutting through wood (his analogy)-where the whole market is the saw cutting through the wood and the signals at the individual tooth level are the low level signals seen at the tick by tick with time zoom level…it was interesting to watch his videos over and over and notice how when a decent # of orders would go by at the offer, often very quickly the market would drop several ticks a few times and then resume its upward movement-the interpretation given by fs that market maker algos take advantage of such moments and shake out any weak holders before taking the market higher. In his approach, he’s assuming that alot of this kind of control and subsequent signal trail is done by larger players’ super fast computers dominating the DOM order tiers so much that they can move the market just by lowering or raising the DOM tiers…so this could be an example of a low level signal that most folks looking at time/range bars might call noise yet it actually exists as a signal purposefully meant to be hidden to benefit players that have the ability to profit from bid/ask spreads all day long-perhaps in the food web metaphor such activity are microorganisms.
BG: Your comments toward the end reminded me of how people talk about the spoofing that goes on in the DOM. Some say that DOM is worthless because of it. However, isn’t there a possibility that the fact that someone is spoofing is meaningful? If they are trying to accomplish something by spoofing, then… might be “tipping their hand”? Otherwise, why spoof?
On that note, I saw some fun stuff the other day. Occasionally someone was adding/removing 500 contract blocks from bid/ask. Bid goes from 1400 to 400, back up again… all in split second. It takes some money to be able to do that even with the low margins these days, so… you know someone with decent sized pockets was doing that.
taotree: I’ve been thinking about similar things. The talented trader/programmer I mentioned earlier talked about how spoofing does exist, however, if your software is analyzing the DOM activity at a rate say 8x per second, then you could plot a statistical level of how much variance there is in a tier. So, in theory, if someone(s) are leaving their size in a tier more often than pulling, perhaps this is them tipping their hand to where they expect to receive orders. I like the idea you brought up, where spotting an intended spoof could be a tip. The programmer, regarding this, brought up the idea that AI of big players is reading sentiment of market participants and then generating high probability strategies that might include using size to further encourage the public to continue going short while big player’s AI is accumulating long. Perhaps, the AI uses spoofing on the DOM as a technique also. The reason I keep mentioning the AI is from thinking that only a computer could keep up all that high speed analysis and execution intraday with precision in my opinion. I think keeping the AI in mind when considering spoofing as a technique can help us in framing questions like “so we’re at a potentially cheap price, could be attractive for institutions to add to longs, could AI be spoofing to help that strategy?”. That’s cool that you spotted that big blocks dropping and reappearing, I’m betting it was a computer doing it if the size was like that and it moved that quick. It’s like Lockheed said on a recent Discovery channel show about their latest plane, where they hinted that this will be their last manned fighter, doesn’t seem fair :-) it’s Skynet but for real lol. In my novice view, seems like if you plotted that activity with an emphasis of highlighting how steady the size was over micro-time periods you might find neat stuff. I admire you programmer guys who had/have the patience/constitution to learn to code and make your own potions. I studied cultural anthropology in college and often find myself investigating interesting tribes of people (traders) and interviewing them to learn these kinds of perspectives. Thanks so much Richard, Taotree, and others for the openness and fun learning environment.