Standard-Deviation
Technicals
Adam Hamilton
Archives
December 12, 2003
As speculators and investors
these days are rightfully in a jovial Christmas mood, I thought
it would be a good week to delve into something light and fun
the realm of hardcore statistics applied to technical analysis!
OK, all joking aside, so maybe
statistical technical analysis might not exactly be "light
and fun," but it is certainly interesting, offering much
food for thought!
As both gunslinging speculators
and long-term investors we are always searching for superior
technical tools to help increase our probabilities of rapidly
recognizing major turning points in the current dominant trends.
Statistical analysis of existing trends via standard deviations
offers a powerful secondary confirmation for the potential high-probability
turning points that are of great interest to market players.
Standard deviations?!? I suspect
that the majority of market participants haven't thought much
about the statistical concept of standard deviations since their
university days. While statistics may have seemed dry and dreary
in college, a necessary evil on the way to an education, when
applied to the markets that we all have a passion for trading
they are really quite fascinating.
Standard deviations are a natural
fit for the financial markets, as they are effectively a measure
of volatility, which in and of itself is one of the primary variables
that speculators zealously watch. A standard deviation is simply
a mathematical way to express how tightly a set of data, like
market prices, tends to cluster around an average. Standard deviations
quantify core market volatility in a unique, comparable, and
easy to understand way that no other indicator can touch, including
the mighty VIX implied volatility index.
According to statistical lore,
Belgian Astronomer Royal Adolph Quetelet is credited with popularizing
standard deviations in the nineteenth century. Quetelet collected
data on the size and height measurements of soldiers and found
that when he graphed these measurements they tended to form a
symmetrical bell-shaped curve around their average. This bell
curve, or normal distribution, eventually became famous and widely
observed in all kinds of natural phenomena.
While bell curves of the heights
of soldiers may be shaped slightly differently than bell curves
of gold prices, the normal-distribution concept is universal.
In order to better describe how data like market prices vary
from their average, the standard deviation was born. While it
sounds cryptic, its name is really quite descriptive. The deviation
part refers to the volatility of data away from its average,
while the standard part refers to the notion that a known percentage
of any dataset is always within defined multiples of a "standard"
deviation.
Standard deviations are easier
to understand in a practical, rather than a theoretical, sense.
Imagine importing one year of closing gold prices, about 250
or so data points, into a spreadsheet and calculating their average.
If you take the latest year of data, you will come up with an
annual average close to $360. This number alone is of limited
use however. In isolation you have no way of knowing if gold
traded in a tight range between $355 and $365 to make this average
or a far more volatile $250 to $470.
Using a standard deviation
however, you can gain an idea of how volatile gold really was.
The actual formula for computing this number is complex, but
knowing it is not important these days since all spreadsheets
like Excel can calculate the standard deviation of any dataset
in the twinkling of an eye. When you query your spreadsheet for
the standard deviation of your gold-price data, it will give
you an excellent idea of the underlying volatility inherent in
your dataset.
If the standard deviation is
large, say $25, then the volatility of your gold dataset is far
greater than if it was only $5. While the deviation component
helps you understand the magnitude of the raw volatility of your
price data, the true magic lies in the standard part.
By mathematical convention,
68.3% of your dataset is always within one standard deviation
of its average. If you double your standard deviation, then 95.4%
of your gold price data is always within plus or minus these
two standard deviations from your average. At three standard
deviations, 99.7% of your price data is included. This all sounds
academic and abstract, until you realize just how practically
it can be applied to trading and technical analysis.
One of the core principles
of market trading is the idea of mean reversions, that prices
abhor extremes and always return to average levels over time.
This mean-reversion concept can be applied over very long-term
spans of time, like in the Long Valuation Waves, or in short-term
trading. Standard deviations, especially when graphed, dovetail
in beautifully with mean reversions.
For example, if the price of
gold, or the S&P 500, or any security, happens to be more
than two standard deviations away from its average, then speculators
know that this is an event that only happens about 1 in 22 trading
days on average, fairly rare. If a market price stretches to
three standard deviations away from its mean, this is extremely
rare, something only witnessed statistically about 1 in 333 trading
days on average.
As speculators or even long-term
investors looking for an entry point, probabilities for an excellent
trade begin increasing at two standard deviations from the mean
and grow very high at three standard deviations. If a price happens
to stretch three standard deviations above or below its average,
then odds are that a significant to major move in the opposite
direction is probably imminent.
Speculators can look for these
rare three-standard-deviation readings as a secondary confirmation
of a major interim high or low in a particular market. Used in
conjunction with other technical indicators, the standard deviations
are very effective in helping speculators decide when to launch
a bet on a mean reversion of a particular market price that they
happen to be following.
OK, if you've made it this
far and are still awake, congratulations! These standard deviations
make much more sense when processed visually in charts. This
week we created long-term standard-deviation graphs of the S&P
500, gold, and the HUI gold-stock index as starting points of
embarkation for my initial essay on standard-deviation-based
technical analysis.
In all of our charts this week,
the actual price data is graphed in blue, with the 200-day moving
average of this raw price data in black. The first standard deviation
above and below the 200dma is rendered in green, the second in
yellow, and the third and most extreme in red. Arrows mark tradable
trend changes that appeared soon after the relevant standard
deviation was pierced.
When viewed visually, this
concept really starts to jell. While we would have no idea how
rare a particular price on any given trading day happened to
be without the standard-deviation bands, when they are added
certain key turning points in the markets leap right out of this
chart. Each time the S&P 500 hit or actually pierced its
red outer +/-3 SD bands, a major short-term trend change was
usually imminent or rapidly approaching.
If speculators had been actively
watching the outer +/-3 standard-deviation bands in real-time,
they would have had a good chance of noting high-probability
moments for major short-term trend changes well before they became
readily apparent to all. The mathematically rigid standard-deviation
numbers let us know when a market price is getting really stretched
into statistically rare territory far away from its average,
a great bit of knowledge to possess.
The green bands above, representing
plus or minus one standard deviation, contain 68.3% of this S&P
500 price data. If we expand out to the yellow +/-2 SD bands,
we know that 95.4% of the market prices lie between these two
lines, since this is the universal definition of two standard
deviations. At the extreme red +/-3 SD bands, 99.7% of our price
data is sandwiched between these outer lines, by statistical
definition.
Thus, if we encounter a price
in real-time that is three standard deviations or more away from
its average, we know that it is an extremely rare event, happening
a little less than 0.3% of the time. This is the equivalent to
1 out of 333 trading days on average, an extraordinarily rare
event. And when a price this rare comes along, speculators can
be sure that the probabilities are very high that a tradable
mean reversion back in the opposite direction is imminent or
at least rapidly approaching.
Now the graph above covers
1240 trading days, yet you will note that there are six arrows
marking +/-3 SD trend reversals, an average of one every 207
trading days. If we take these six occurrences (which actually
each encompassed a group of trading days in most cases) and multiply
them by the 333 days average chance of such an extreme reading,
we arrive at 1998 trading days. In other words, there are 60%
more occurrences of +/-3 SD extremes in the S&P 500 since
1999 than statistics would suggest! Why?
Provocatively, market-price
data is not normally distributed! If you create histograms of
price data it will look almost like a bell curve, but not exactly.
A very meticulous examination will reveal what are known to statisticians
as "fat tails". These "tails" of the bell
curve, the tiny extremes on the far left and right below and
above three standard deviations away from the mean, are actually
larger in the financial markets than they mathematically should
be. In other words, extreme price days are more common in the
financial markets than a true normal distribution would suggest.
There is an endless debate
on the reasons for this, but the answers most probably lie in
the popular emotions of greed and fear. When the thundering herd
gets either really greedy or really scared, they tend to badger
stocks far above or below where math and statistics suggest they
really ought to end up before a short-term trend change. These
group emotions work in the favor of prudent contrarians, however,
as they provide us with both a warning of a coming extreme and
also more actual occurrences of the rare +/-3 SD extremes to
actively trade.
Another important point to
note on these charts is the volatility bulges. Standard deviations,
since they are standardized, shrink with lower volatility in
the markets and grow or bulge with increasing volatility. This
information alone is useful, as higher volatility periods around
interim market bottoms lead to bulges and lower volatility episodes
near interim market tops lead to constrictions. Analysis could
certainly be undertaken on the convergences and divergences of
the positive and negative standard-deviation lines alone!
Volatility profiles are not
only highly variable within given markets, but also across different
markets. While US stocks have had a rather volatile time so far
in our Great Bear, gold has witnessed far less volatility in
its own young Great Bull. Yet, standard-deviation technical analysis
still applies and works really well. The standard-deviation bands
are tighter with the lower inherent volatility of gold, but the
+/-3 SD extremes still tend to mark very tradable short-term
turning points.
Even though it is hard to imagine
two markets so vastly different than the global gold market and
the American S&P 500 flagship stock index, the standard deviations
seem to apply equally well in both! As is readily evident on
the chart above, each time gold stretched three or more standard
deviations away from its own average it tended to mark a tradable
short-term turning point.
When speculators witness these
events in real-time, they can ratchet up their stops and prepare
to be stopped out on the inevitable pullback if they were long.
Investors can use these signals as well, as they would do best
to not even think about deploying fresh long-term capital if
gold happens to be approaching levels three standard deviations
above its current average. No sense buying in on short-term extremes!
Also interesting, with seven
of these +/-3 SD readings regions noted above, fat tails in the
normal distribution of gold prices in recent years are also very
apparent. Nevertheless, even though gold prices are skewed away
from true normality with their fat tails, the speculation signals
spawned by these +/-3 SD extremes tend to be pretty good.
I would certainly be very remiss
in discussing standard-deviation technical analysis without paying
tribute to its modern father, John Bollinger. Mr. Bollinger is
a world-renowned technical analyst who appears on CNBC today
from time to time. He has several websites including www.bollingerbands.com
and has written a book called "Bollinger on Bollinger Bands"
which is recommended reading for anyone digging deeper into standard-deviation
technical analysis.
Bollinger Bands, which bear
Mr. Bollinger's name, are a widely-used formed of standard-deviation
technical analysis. Mr. Bollinger popularized their use and continues
to push the envelope in the practical deployment of standard-deviation
Bollinger Bands for real-world speculations. While Bollinger
Bands are not necessarily rigidly defined, they have evolved
into a primary definitive form in real-world usage.
Today Bollinger Bands are most
often considered to be +/-2 SD bands above and below a 20-day
moving average. In the past Mr. Bollinger has recommended 10-day
moving averages for short-term trading, 20dmas for intermediate-term
trading, and 50dmas for long-term trading. Sometimes on the longer-term
50dma Bollinger Bands, technical analysts expand the bands to
+/-2.5 standard deviations.
In contrast, all of the charts
in this essay take a very long-term perspective in standard-deviation
analysis. We used 200-day moving averages and 200-day standard
deviations to create these graphs, as I have not very often seen
standard-deviation analysis applied to longer periods of time.
Our 200-day standard-deviation bands graphed above and below
the 200dmas of these price datasets suggest that SD technical
analysis is truly fractal in nature, applying just as easily
and readily to long periods of time as to shorter periods.
In future essays and research
work at Zeal we are going to apply SD analysis to short-term
trading in various markets in which we are speculating, more
in line with Mr. Bollinger's conventions. But we needed to first
check out the long-term strategic overview shown this week to
provide a solid foundation off of which to launch our future
standard-deviation technical journeys.
Our final graph, using this
same 200dma base, shows the power and relevance of these standard-deviation
bands in even an extraordinarily volatile market like the gold-stock
arena. The +/-3 SD range on the HUI is gargantuan, but even so
the principles of the standard-deviation extremes and mean reversions
still apply.
Unlike the first two charts
above, this HUI graph is rendered on a zeroed axis which makes
its huge inherent volatility all the more amazing. You would
be really hard-pressed to find a more volatile major sector in
recent years than the gold stocks, so this really offers some
excellent additional insights into SD bands in hyper-volatile
markets. This chart is also important as it nicely illustrates
some important caveats in incorporating SD technical analysis
into your speculator toolbox.
While there are +/-3 SD turning
points above, there were also two separate times that the +3
SD line was hugged for months in a row! In both early 2002 and
just recently in 2003 massive gold-stock rallies propelled the
HUI up to breathtaking new heights. In these mammoth rallies
the buying pressure was so immense that the HUI traded three
standard deviations above its 200dma for long periods of time
without crumbling. This extraordinarily strong bull-market behavior
perfectly illustrates the primary caveat of using SD bands
Standard-deviation analysis
is not designed to be used alone in isolation to provide primary
buy and sell signals! Don't forget this!
SD bands, and formal Bollinger
Bands, are best used by speculators as relativity gauges considered
in conjunction with other technical signals. When an index like
the HUI pushes +3 SD above its mean, it does not provide a sell
signal in isolation. Instead it just lets speculators know that
the HUI is on the relatively-high edge and they should be cautious.
SD bands reveal relatively overbought and relatively oversold
levels that are most useful when used as a secondary confirmation
of other primary trading signals.
For example in the HUI's case,
last week in "Trading the Gold-Stock Bull 3" I discussed
the Gold 50/200 MACD and the Relative HUI, which are both primary
trading indicators. The SD bands are awesome tools to use to
provide a secondary confirmation when a primary signal such as
a Relative HUI neutrality level is tripped.
If a price like the HUI's happens
to be near or beyond three standard deviations out from its mean,
and other more specific technical indicators are also flashing,
only then should speculators seriously consider girding themselves
for a major short-term trend change. Do not analyze SD bands
in isolation to use as primary trading signals, as sometimes
prices can hug these bands for weeks or months in exceptional
markets.
Incidentally, if trading gold
stocks is your game, the current December issue of our Zeal Intelligence
newsletter for our subscribers is flying off the shelves. Titled
"Blue-Chip Golds," it analyzes all of the individual
stocks in the elite HUI and XAU gold-stock indices, looking at
their relative performances in each major rally to date as well
as their relative volatility and risk levels. Subscribe today
to read this analysis and see the half-dozen blue-chip-gold winners,
which I am looking forward to analyzing fundamentally in future
issues of ZI.
Standard-deviation bands show
us relatively overbought and oversold levels, but just as the
HUI hugged +3 standard deviations for months above before correcting
on two separate occasions, an SD extreme does not necessarily
warn of a certain turn happening immediately. But it can reveal
to us the general tenor of a market and how anomalous it happens
to appear at the moment in statistical probability terms, a good
thing to know!
The farther out that a price
happens to be in standard-deviation terms from its average, the
rarer that such an event truly is, and the higher the probability
that such an anomaly will not last long. If an SD extreme exists
while other more well-defined technical indicators are calling
for a trend change, speculators would do well to heed their combined
message. SD bands nicely compliment and augment other forms of
technical analysis.
I am certainly looking forward
to integrating more standard-deviation technical analysis into
our research work at Zeal. It ought to provide an excellent secondary
confirmation for other technical trading signals that we are
watching and developing.
Adam Hamilton, CPA
email:
zelotes@zealllc.com
Archives
December 12, 2003
So how can you profit
from this information? We publish an acclaimed monthly newsletter,
Zeal
Intelligence,
that details exactly what we are doing in terms of actual stock
and options trading based on all the lessons we have learned
in our market research. Please consider joining us each month
for tactical trading details and more in our premium Zeal
Intelligence service.
Copyright ©2000-2003
Zeal Research All Rights Reserved (www.ZealLLC.com)
321gold Inc
ref: 02998
|