Standard-Deviation Technicals Adam Hamilton Archives December 12, 2003 As speculators and investors these days are rightfully in a jovial Christmas mood, I thought it would be a good week to delve into something light and fun the realm of hardcore statistics applied to technical analysis! OK, all joking aside, so maybe statistical technical analysis might not exactly be "light and fun," but it is certainly interesting, offering much food for thought! As both gunslinging speculators and long-term investors we are always searching for superior technical tools to help increase our probabilities of rapidly recognizing major turning points in the current dominant trends. Statistical analysis of existing trends via standard deviations offers a powerful secondary confirmation for the potential high-probability turning points that are of great interest to market players. Standard deviations?!? I suspect that the majority of market participants haven't thought much about the statistical concept of standard deviations since their university days. While statistics may have seemed dry and dreary in college, a necessary evil on the way to an education, when applied to the markets that we all have a passion for trading they are really quite fascinating. Standard deviations are a natural fit for the financial markets, as they are effectively a measure of volatility, which in and of itself is one of the primary variables that speculators zealously watch. A standard deviation is simply a mathematical way to express how tightly a set of data, like market prices, tends to cluster around an average. Standard deviations quantify core market volatility in a unique, comparable, and easy to understand way that no other indicator can touch, including the mighty VIX implied volatility index. According to statistical lore, Belgian Astronomer Royal Adolph Quetelet is credited with popularizing standard deviations in the nineteenth century. Quetelet collected data on the size and height measurements of soldiers and found that when he graphed these measurements they tended to form a symmetrical bell-shaped curve around their average. This bell curve, or normal distribution, eventually became famous and widely observed in all kinds of natural phenomena. While bell curves of the heights of soldiers may be shaped slightly differently than bell curves of gold prices, the normal-distribution concept is universal. In order to better describe how data like market prices vary from their average, the standard deviation was born. While it sounds cryptic, its name is really quite descriptive. The deviation part refers to the volatility of data away from its average, while the standard part refers to the notion that a known percentage of any dataset is always within defined multiples of a "standard" deviation. Standard deviations are easier to understand in a practical, rather than a theoretical, sense. Imagine importing one year of closing gold prices, about 250 or so data points, into a spreadsheet and calculating their average. If you take the latest year of data, you will come up with an annual average close to $360. This number alone is of limited use however. In isolation you have no way of knowing if gold traded in a tight range between $355 and $365 to make this average or a far more volatile $250 to $470. Using a standard deviation however, you can gain an idea of how volatile gold really was. The actual formula for computing this number is complex, but knowing it is not important these days since all spreadsheets like Excel can calculate the standard deviation of any dataset in the twinkling of an eye. When you query your spreadsheet for the standard deviation of your gold-price data, it will give you an excellent idea of the underlying volatility inherent in your dataset. If the standard deviation is large, say $25, then the volatility of your gold dataset is far greater than if it was only $5. While the deviation component helps you understand the magnitude of the raw volatility of your price data, the true magic lies in the standard part. By mathematical convention, 68.3% of your dataset is always within one standard deviation of its average. If you double your standard deviation, then 95.4% of your gold price data is always within plus or minus these two standard deviations from your average. At three standard deviations, 99.7% of your price data is included. This all sounds academic and abstract, until you realize just how practically it can be applied to trading and technical analysis. One of the core principles of market trading is the idea of mean reversions, that prices abhor extremes and always return to average levels over time. This mean-reversion concept can be applied over very long-term spans of time, like in the Long Valuation Waves, or in short-term trading. Standard deviations, especially when graphed, dovetail in beautifully with mean reversions. For example, if the price of gold, or the S&P 500, or any security, happens to be more than two standard deviations away from its average, then speculators know that this is an event that only happens about 1 in 22 trading days on average, fairly rare. If a market price stretches to three standard deviations away from its mean, this is extremely rare, something only witnessed statistically about 1 in 333 trading days on average. As speculators or even long-term investors looking for an entry point, probabilities for an excellent trade begin increasing at two standard deviations from the mean and grow very high at three standard deviations. If a price happens to stretch three standard deviations above or below its average, then odds are that a significant to major move in the opposite direction is probably imminent. Speculators can look for these rare three-standard-deviation readings as a secondary confirmation of a major interim high or low in a particular market. Used in conjunction with other technical indicators, the standard deviations are very effective in helping speculators decide when to launch a bet on a mean reversion of a particular market price that they happen to be following. OK, if you've made it this far and are still awake, congratulations! These standard deviations make much more sense when processed visually in charts. This week we created long-term standard-deviation graphs of the S&P 500, gold, and the HUI gold-stock index as starting points of embarkation for my initial essay on standard-deviation-based technical analysis. In all of our charts this week, the actual price data is graphed in blue, with the 200-day moving average of this raw price data in black. The first standard deviation above and below the 200dma is rendered in green, the second in yellow, and the third and most extreme in red. Arrows mark tradable trend changes that appeared soon after the relevant standard deviation was pierced. When viewed visually, this concept really starts to jell. While we would have no idea how rare a particular price on any given trading day happened to be without the standard-deviation bands, when they are added certain key turning points in the markets leap right out of this chart. Each time the S&P 500 hit or actually pierced its red outer +/-3 SD bands, a major short-term trend change was usually imminent or rapidly approaching. If speculators had been actively watching the outer +/-3 standard-deviation bands in real-time, they would have had a good chance of noting high-probability moments for major short-term trend changes well before they became readily apparent to all. The mathematically rigid standard-deviation numbers let us know when a market price is getting really stretched into statistically rare territory far away from its average, a great bit of knowledge to possess. The green bands above, representing plus or minus one standard deviation, contain 68.3% of this S&P 500 price data. If we expand out to the yellow +/-2 SD bands, we know that 95.4% of the market prices lie between these two lines, since this is the universal definition of two standard deviations. At the extreme red +/-3 SD bands, 99.7% of our price data is sandwiched between these outer lines, by statistical definition. Thus, if we encounter a price in real-time that is three standard deviations or more away from its average, we know that it is an extremely rare event, happening a little less than 0.3% of the time. This is the equivalent to 1 out of 333 trading days on average, an extraordinarily rare event. And when a price this rare comes along, speculators can be sure that the probabilities are very high that a tradable mean reversion back in the opposite direction is imminent or at least rapidly approaching. Now the graph above covers 1240 trading days, yet you will note that there are six arrows marking +/-3 SD trend reversals, an average of one every 207 trading days. If we take these six occurrences (which actually each encompassed a group of trading days in most cases) and multiply them by the 333 days average chance of such an extreme reading, we arrive at 1998 trading days. In other words, there are 60% more occurrences of +/-3 SD extremes in the S&P 500 since 1999 than statistics would suggest! Why? Provocatively, market-price data is not normally distributed! If you create histograms of price data it will look almost like a bell curve, but not exactly. A very meticulous examination will reveal what are known to statisticians as "fat tails". These "tails" of the bell curve, the tiny extremes on the far left and right below and above three standard deviations away from the mean, are actually larger in the financial markets than they mathematically should be. In other words, extreme price days are more common in the financial markets than a true normal distribution would suggest. There is an endless debate on the reasons for this, but the answers most probably lie in the popular emotions of greed and fear. When the thundering herd gets either really greedy or really scared, they tend to badger stocks far above or below where math and statistics suggest they really ought to end up before a short-term trend change. These group emotions work in the favor of prudent contrarians, however, as they provide us with both a warning of a coming extreme and also more actual occurrences of the rare +/-3 SD extremes to actively trade. Another important point to note on these charts is the volatility bulges. Standard deviations, since they are standardized, shrink with lower volatility in the markets and grow or bulge with increasing volatility. This information alone is useful, as higher volatility periods around interim market bottoms lead to bulges and lower volatility episodes near interim market tops lead to constrictions. Analysis could certainly be undertaken on the convergences and divergences of the positive and negative standard-deviation lines alone! Volatility profiles are not only highly variable within given markets, but also across different markets. While US stocks have had a rather volatile time so far in our Great Bear, gold has witnessed far less volatility in its own young Great Bull. Yet, standard-deviation technical analysis still applies and works really well. The standard-deviation bands are tighter with the lower inherent volatility of gold, but the +/-3 SD extremes still tend to mark very tradable short-term turning points. Even though it is hard to imagine two markets so vastly different than the global gold market and the American S&P 500 flagship stock index, the standard deviations seem to apply equally well in both! As is readily evident on the chart above, each time gold stretched three or more standard deviations away from its own average it tended to mark a tradable short-term turning point. When speculators witness these events in real-time, they can ratchet up their stops and prepare to be stopped out on the inevitable pullback if they were long. Investors can use these signals as well, as they would do best to not even think about deploying fresh long-term capital if gold happens to be approaching levels three standard deviations above its current average. No sense buying in on short-term extremes! Also interesting, with seven of these +/-3 SD readings regions noted above, fat tails in the normal distribution of gold prices in recent years are also very apparent. Nevertheless, even though gold prices are skewed away from true normality with their fat tails, the speculation signals spawned by these +/-3 SD extremes tend to be pretty good. I would certainly be very remiss in discussing standard-deviation technical analysis without paying tribute to its modern father, John Bollinger. Mr. Bollinger is a world-renowned technical analyst who appears on CNBC today from time to time. He has several websites including www.bollingerbands.com and has written a book called "Bollinger on Bollinger Bands" which is recommended reading for anyone digging deeper into standard-deviation technical analysis. Bollinger Bands, which bear Mr. Bollinger's name, are a widely-used formed of standard-deviation technical analysis. Mr. Bollinger popularized their use and continues to push the envelope in the practical deployment of standard-deviation Bollinger Bands for real-world speculations. While Bollinger Bands are not necessarily rigidly defined, they have evolved into a primary definitive form in real-world usage. Today Bollinger Bands are most often considered to be +/-2 SD bands above and below a 20-day moving average. In the past Mr. Bollinger has recommended 10-day moving averages for short-term trading, 20dmas for intermediate-term trading, and 50dmas for long-term trading. Sometimes on the longer-term 50dma Bollinger Bands, technical analysts expand the bands to +/-2.5 standard deviations. In contrast, all of the charts in this essay take a very long-term perspective in standard-deviation analysis. We used 200-day moving averages and 200-day standard deviations to create these graphs, as I have not very often seen standard-deviation analysis applied to longer periods of time. Our 200-day standard-deviation bands graphed above and below the 200dmas of these price datasets suggest that SD technical analysis is truly fractal in nature, applying just as easily and readily to long periods of time as to shorter periods. In future essays and research work at Zeal we are going to apply SD analysis to short-term trading in various markets in which we are speculating, more in line with Mr. Bollinger's conventions. But we needed to first check out the long-term strategic overview shown this week to provide a solid foundation off of which to launch our future standard-deviation technical journeys. Our final graph, using this same 200dma base, shows the power and relevance of these standard-deviation bands in even an extraordinarily volatile market like the gold-stock arena. The +/-3 SD range on the HUI is gargantuan, but even so the principles of the standard-deviation extremes and mean reversions still apply. Unlike the first two charts above, this HUI graph is rendered on a zeroed axis which makes its huge inherent volatility all the more amazing. You would be really hard-pressed to find a more volatile major sector in recent years than the gold stocks, so this really offers some excellent additional insights into SD bands in hyper-volatile markets. This chart is also important as it nicely illustrates some important caveats in incorporating SD technical analysis into your speculator toolbox. While there are +/-3 SD turning points above, there were also two separate times that the +3 SD line was hugged for months in a row! In both early 2002 and just recently in 2003 massive gold-stock rallies propelled the HUI up to breathtaking new heights. In these mammoth rallies the buying pressure was so immense that the HUI traded three standard deviations above its 200dma for long periods of time without crumbling. This extraordinarily strong bull-market behavior perfectly illustrates the primary caveat of using SD bands Standard-deviation analysis is not designed to be used alone in isolation to provide primary buy and sell signals! Don't forget this! SD bands, and formal Bollinger Bands, are best used by speculators as relativity gauges considered in conjunction with other technical signals. When an index like the HUI pushes +3 SD above its mean, it does not provide a sell signal in isolation. Instead it just lets speculators know that the HUI is on the relatively-high edge and they should be cautious. SD bands reveal relatively overbought and relatively oversold levels that are most useful when used as a secondary confirmation of other primary trading signals. For example in the HUI's case, last week in "Trading the Gold-Stock Bull 3" I discussed the Gold 50/200 MACD and the Relative HUI, which are both primary trading indicators. The SD bands are awesome tools to use to provide a secondary confirmation when a primary signal such as a Relative HUI neutrality level is tripped. If a price like the HUI's happens to be near or beyond three standard deviations out from its mean, and other more specific technical indicators are also flashing, only then should speculators seriously consider girding themselves for a major short-term trend change. Do not analyze SD bands in isolation to use as primary trading signals, as sometimes prices can hug these bands for weeks or months in exceptional markets. Incidentally, if trading gold stocks is your game, the current December issue of our Zeal Intelligence newsletter for our subscribers is flying off the shelves. Titled "Blue-Chip Golds," it analyzes all of the individual stocks in the elite HUI and XAU gold-stock indices, looking at their relative performances in each major rally to date as well as their relative volatility and risk levels. Subscribe today to read this analysis and see the half-dozen blue-chip-gold winners, which I am looking forward to analyzing fundamentally in future issues of ZI. Standard-deviation bands show us relatively overbought and oversold levels, but just as the HUI hugged +3 standard deviations for months above before correcting on two separate occasions, an SD extreme does not necessarily warn of a certain turn happening immediately. But it can reveal to us the general tenor of a market and how anomalous it happens to appear at the moment in statistical probability terms, a good thing to know! The farther out that a price happens to be in standard-deviation terms from its average, the rarer that such an event truly is, and the higher the probability that such an anomaly will not last long. If an SD extreme exists while other more well-defined technical indicators are calling for a trend change, speculators would do well to heed their combined message. SD bands nicely compliment and augment other forms of technical analysis. I am certainly looking forward to integrating more standard-deviation technical analysis into our research work at Zeal. It ought to provide an excellent secondary confirmation for other technical trading signals that we are watching and developing. Adam Hamilton, CPA email: zelotes@zealllc.com Archives December 12, 2003 So how can you profit from this information? We publish an acclaimed monthly newsletter, Zeal Intelligence, that details exactly what we are doing in terms of actual stock and options trading based on all the lessons we have learned in our market research. Please consider joining us each month for tactical trading details and more in our premium Zeal Intelligence service. Copyright ©2000-2003 Zeal Research All Rights Reserved (www.ZealLLC.com) 321gold Inc ref: 02998