Gold Futures CoT 2

Adam Hamilton
Archives
May 26, 2007

I have loved reading my entire life, so when I am not studying the financial markets one of my favorite pastimes is reading great fiction. My favorite genre these days is the rich adventure/action stories spun out by brilliant authors like Clive Cussler, Jack Du Brul, and James Rollins. A good book makes even the very best movies seem like shallow grade-school plays by comparison.

Adventure stories often have history woven in as the heroes chase after some priceless artifact. Usually some ancient priest-type caste existed that hid the artifact away to protect it from a calamity in the past so our heroes can unlock its secrets in the present. These historical priests often used special knowledge that only they had, usually scientific in nature, to cement their power in the society.

As an example, some high priest might know through a lifetime of studying the heavens that a solar eclipse was due soon. So he would use this knowledge to gain temporal power. He would tell the king that the gods were displeased with him and were going to blacken the sun on an appointed date. Then after the event happened as prophesied and the king came groveling in fear, the priest could demand anything to bring back the sun.

Similar mysticism continues to cloak the futures markets today. Futures traders are very small in number compared to stock traders, so to stock traders the futures world seems esoteric and impossible to comprehend. Thus gurus have arisen to act as interpreters to stock traders to explain happenings in the futures world. Like the adventure-story priesthoods, disturbingly often the futures gurus tell half-truths and withhold information so they can retain or enhance their popular status as gurus.

In some circles the futures priesthood holds very little sway. The vast majority of mainstream investors, for example, probably aren't even aware of theories that exist alleging S&P 500 futures manipulation to help steer the stock markets briefly at key inflection points. They don't care what happens in stock futures and aren't worried about them adversely affecting their stock portfolios.

But in other areas of the markets, typically small underdog contrarian-dominated ones, the futures priesthood wields great influence. Stock investors worry constantly about what is happening in the futures and they look to gurus to interpret the esoteric signs in futures trading that may affect them. The key example of this I have observed for many years now exists in gold. Gold-stock traders tend to have great interest in, and trepidation for, what is happening in gold futures.

Sometimes this interest blossoms into full-blown paranoia. If I had an ounce of gold for every time I have read something like the following since this gold bull began six years ago, I could start my own central bank.

Remember reading endless comments like these? "Commercials are record short, they are capping gold." "Gold open interest is plummeting, Wall Street is trying to scare traders out of the market." "The large specs are setting the small specs up in a trap, the little guys are going to get slaughtered." "Paper gold is more important for the gold price than physical supply and demand." "Gold OI fell by 25,000 contracts today, a record. The smart money is abandoning ship."

In general, the overall theme the futures priesthood tries to impart to its stock-trading followers is negative. It says that little traders are always at the mercy of the big traders, so when little traders lose money in gold stocks it is not their fault. It wasn't because they made the wrong decisions on timing or stocks, but because powerful forces beyond their comprehension in the futures world were toying with them. People love avoiding accountability for their own actions.

Naturally with this victim-mentality focus, the futures priesthood's influence is greatest when gold stocks are the weakest and stock traders seek explanations for their falling portfolios. It is far more comfortable to attribute one's losses to "them" rather than simply one's own incorrect buy/sell timing. Unfortunately a sizeable subset of our Zeal subscribers get caught up in this futures mysticism when their holdings are not performing well, so I am writing this essay for them.

Although futures trading is more complex than stock trading, it is very straightforward and does not need to be cloaked in mysticism. Anyone can easily understand the broad trends in gold futures trading, and just like in every other market most of the day-to-day fluctuations in futures are just meaningless random noise. With a little bit of understanding, gold futures rapidly shed their sinister reputation in stock traders' minds.

The first thing that is crucial to understand is, unlike stocks, futures are a zero-sum game. This means that every dollar won in a gold futures trade is a direct dollar loss for the counterparty on the other side of the trade. In this type of trading, every long contract is perfectly offset by an opposing short contract. Thus the total number of longs and shorts at any given time is always equal, in perfect equilibrium.

This basic fact that everyone should know never ceases to amuse me. I can't count the times that I've read futures gurus warning that "record short positions exist in gold so it cannot go any higher". Well, obviously if record short positions exist then record long positions have to exist too! Perhaps the glass should be seen as half full rather than half empty. In a pure zero-sum game, there are always equal numbers of contracts betting for or against any given asset.

One of the key statistics the futures priesthood eagerly watches is open interest. Open interest is the futures term used to quantify the total number of futures contracts outstanding at any given time. Since each contract has an equal long and short side, OI only counts longs or shorts, but not both. To count both would overstate true open interest by 100%. Over time, OI trends reveal whether the capital involved in a futures market is growing or contracting.

This first chart takes a look at COMEX gold futures open interest in the US in our gold bull to date. Although gold is traded around the world on many different futures exchanges, so far COMEX gold prices have been the most-watched price. I don't think this US dominance will last forever given the US dollar's increasingly precarious technical position, but for now it still holds true.

Virtually all of the futures data that you will ever see analyzed, including this bull-to-date gold open interest, comes from the Commitments of Traders Report. Each Friday, the Commodity Futures Trading Commission releases its CoT report. It summarizes changes in futures positions held by the major classes of commodities traders. It is definitely useful, but it is complex so an air of mysticism has sprung up around it.

In terms of gold open interest reported in these weekly CoT reports, it has risen dramatically since gold bottomed in April 2001. Back in mid-2001, there were only 100k total contracts of COMEX gold outstanding. But in recent months gold OI broke 400k contracts for the first time in this bull market. Overall from trough to peak, COMEX gold futures OI has blasted 321% higher.

The higher gold goes, the more traders are interested in participating in its bull market. Investment assets are pretty unique in this regard in that higher prices drive higher demand, which is the opposite of most typical commodities. If corn prices triple, corn demand will drop as corn users seek lower-priced substitutes. But if gold prices triple, investors and speculators want even more as they seek to chase its momentum.

Gold prices have indeed nearly tripled since their April 2001 lows, and gold futures OI has quadrupled. It is wonderful to see gold futures OI rise with gold because it shows ever-wider interest in the gold bull from bigger pools of capital. Obviously the more capital that ultimately bids on gold, the higher this bull market will ultimately climb. The more investors and speculators that follow us in, the greater our profits will ultimately be.

I find it particularly fascinating that gold OI has risen in a very well-defined uptrend so far. Yes, gold OI will rise and fall depending on the tactical fortunes of the gold price, but over time these flows and ebbs have tended to form remarkably solid support and resistance lines. Although there is certainly no guarantee this trend will hold into the future, these OI extremes are tradable.

If you carefully examine gold OI versus the gold price above, you will note that it tends to rise when gold is strong and fall when gold is weak. This makes sense since traders are much more likely to be interested in participating in a hot market than a slow one. So if OI is near support, probabilities favor gold moving higher. If it is near resistance, probabilities favor gold consolidating sideways or correcting lower. While gold does not always follow these OI trading rules, they can help supplement signals offered by other more accurate trading tools.

And getting back to the futures priesthood always scaring the stock traders, an important observation is plainly evident above. While gold OI has risen on balance in this bull, it has also fallen sharply over a dozen times so far. Yet despite all these sharp OI plunges, gold has somehow managed to advance 181% higher in its bull to date. And these sharp plunges that drag OI back near support tend to mark low points in the gold price, great times to buy.

So the next time you hear some guru lament that gold OI is plummeting so the gold price must be in trouble, realize it is mystical nonsense. Gold OI falls sharply multiple times every year, yet the gold bull powers higher on balance despite this. Taking a sharp change in OI out of context is irrational and silly. Unless the OI change radically violates this uptrend one way or the other, it is probably not even worth thinking about.

Now the venerated CoT reports that the high priests graciously interpret for us mere mortals are indeed complicated. A couple years ago I downloaded CoT data for futures for a single year and the spreadsheet contained 129 columns and nearly 3500 rows of data. I have worked with spreadsheets of CoT data that weighed in at a whopping 40 megabytes! Raw CoT data is voluminous, complex, and intimidating.

But like most complex things, it can be distilled down into basics that stock traders can easily understand. The Commitments of Traders Report is exactly what it sounds like, it shows what three basic groups of traders currently have open in terms of gold futures positions. While the total number of gold longs and shorts is always perfectly equal, the internal proportions of these positions held by the three groups change over time and these changes are what is analyzed.

The raw CoT has two major groups of traders and a third minor one that acts as a plug to balance out the first two. The first category of traders is "non-commercial". These are large traders buying and selling gold futures that are not actually producing or consuming real physical gold. Hence they are the speculator side of the futures trade and are more commonly known as large speculators.

The second major group of traders is the "commercial" category. Commercials are large traders too, but they are theoretically actually producing or consuming real physical gold. Thus they are the hedger side of the futures trade. They are trying to lock in their gold price on the buy or sell side so they can better plan their business operations. Commercials, of course, are the notorious group that the futures priesthood drums up the most fear about.

The third group is quite small compared to the first two, and indeed it represents the mathematical difference between the large speculators' and hedgers' positions. These are the "nonreportable positions", or futures traders that are so small they don't have to report their trades to the Commodity Futures Trading Commission. These are more commonly known as small speculators.

Each group has longs and shorts outstanding, but if all of their longs and shorts are added together it yields a net-long or net-short figure. Over time these net positions can be charted and analyzed. And of course since futures are a zero-sum game, net longs and net shorts are always equal. If longs are thought of as positive and shorts as negative, the total positions always add up to zero.

Here are the three groups' net positions in gold futures charted over the entire duration of this gold bull. The futures priesthood loves to take one aspect of this perpetually balanced market, usually the commercials' net shorts, and blow it way out of proportion. But if you can maintain perspective, there is certainly nothing ominous in this data as we stock traders are often led to believe.

Now the strategic story of this chart is pretty simple. Over the course of this gold bull, commercial hedgers' net-short position has grown greater on balance. This naturally spooks gold-stock traders who fear gold futures. Offsetting this increase in hedger net shorts though is a trending-higher net-long position held by large speculators. And curiously small speculators', largely individual traders, net-long position has been essentially flatlined for years despite the powerful gold bull.

Does this mean that hedgers are getting more bearish on gold and speculators are getting more bullish? Not necessarily, especially in the case of the hedgers. While I personally hate hedging and wish no commodities producers ever did it, it is a fact of life in the gold-mining industry. Hedgers have good reason to continue hedging a portion of their production, even in a secular bull.

Gold mining is an expensive and risky business. Once a promising deposit is found, tens or hundreds of millions of dollars must be spent drilling it to define the ore body and develop an optimal mining plan. And once this mining plan is ready, hundreds of millions or billions of dollars must be spent to actually build the mine. All this cash necessary to finance such big projects has to come from somewhere. Gold companies have two choices to get it, either issue new shares to raise the cash or borrow the money.

Issuing new shares irritates gold-stock investors to no end. We do not want to be diluted, have our ownership percentage drop, as new shares are issued. Also, ultimately issuing new shares is the costliest way to finance a mining project. Instead of the new financiers being paid a fixed cost for their capital, these new shareholders have a full unlimited profit interest in future production. Stocks usually sell off sharply on major new share issuances because stock investors loathe them.

The only other alternative for raising cash is borrowing it. The debt markets are a far cheaper way to develop a gold mine over the long term. Debt investors get a fixed rate of return on their capital and that is it. If the gold mine is fantastically profitable, all profits above and beyond interest expenses go to existing shareholders. Over the long haul, shareholders in companies that finance their projects with debt usually fare a lot better than shareholders in companies that continually issue new equity.

But banks lending miners money are not hardcore secular commodities bulls like individual investors. They want assurance that they'll get their principal back when the mine goes live regardless of prevailing market conditions. Thus they almost always demand some level of hedging in order to issue the loan. Gold companies that borrow money to build mines usually have to hedge some fraction of future production as part of their debt covenants. Now they are obviously bullish since they are building a gold mine, yet they still have to effectively short some gold in order to finance the mine.

Another scenario where companies feel a legitimate need to sell gold before they produce it exists in operations where gold is the byproduct, not the primary metal. A big copper producer may end up mining a lot of gold with the copper ore it extracts and processes. But since it is not in the gold-mining business, it really doesn't care about gold prices and chooses to hedge its byproduct gold production.

Now once again as an investor and speculator I am not a fan of hedging and wish it didn't exist at all. But my point here is that it does exist and is not necessarily nefarious and anti-gold. If a company I own wants to build a major new gold mine that will increase my future profits radically, but it must hedge 10% of the production out of that particular mine to get the loan, I can live with that. Ultimately it is good for me as a stock investor.

Back to the chart, in late 2005 when the hedgers' net-short position reached record highs, the futures priesthood was going bananas. All of the futures gurus that I followed were very bearish back then. Gold had consolidated sideways for all of 2005 until that point and the record net-short position of the commercials must have meant that a big gold decline was imminent. "The commercials are always right", they said. The bearish hysteria back then due to futures mysticism was unreal.

But interpretation is very important in the markets. This next chart zooms in so these record net-short hedger positions can be more easily seen. At the very moment the commercials were heavily short, the large specs were heavily long. So was the glass half empty or half full? I was betting on the latter back then since gold was due for a major upleg after its long consolidation. It turns out the large specs were right in spades.

The record net-short hedger position in late 2005 occurred very early on in the biggest gold upleg we've seen by far in this entire bull market. This isn't always the case, as sometimes new record commercial net-short levels appear near tops, but it does illustrate that record commercial net shorts considered in isolation are nothing to fear. Hedgers will continue to hedge for valid business reasons until the end of time, regardless of the gold price trend.

While the futures priesthood frets about commercial shorts, which will continue to grow with open interest in this bull market, one thing I don't see them address is the flatlined small speculator net-long position. You would think that as this gold bull powers higher, more small speculators would start playing in gold futures and their net-long position would trend higher. So far this hasn't been the case though. While I don't know why, I do have some theories after watching this for years.

First, these three CFTC-reported trader categories are a bit nebulous and I really doubt their precision. The small specs are nonreportable, they are a mathematical construct necessary to make large specs and hedgers balance out. Viewed this way, as a kind of accounting plug rather than actually measured individual futures traders, it makes sense that this difference between hedgers and large specs would be fairly constant over time.

Second, even within the major hedger and large-spec categories, CoT classification is not rigorous. For example, imagine a big Wall Street bank trades in gold futures, some for its own account and others for its clients. The bank will probably be classified as a large spec. But it may have clients that are really hedgers, like a jewelry store for instance. The jewelry store will want to lock in gold prices for commercial reasons but its futures could be held in the bank's street name and hence classified as a large spec rather than a hedger. Similarly a small-spec client could be classified as a large spec if its positions are held in the large spec's street name.

For this reason I don't have a lot of confidence in the arbitrary CFTC line between hedgers and speculators on its CoT report. There is no doubt that some are classified in the wrong category. This realization is kind of amusing too, as the CoT data that futures gurus sweat over is likely fairly subjective. It is not rock-solid like price data, but is fuzzy based on classification.

Another problem with CoT data is it is just for one futures contract traded in one country. Gold is traded worldwide on many exchanges, so the US CoT is never an accurate reflection of global positions. And many gold contracts between major buyers and sellers are cut privately, off the exchanges, so they never show up in futures data anyway. This means the CoT has a very narrow and myopic focus compared to the global gold market's true scope. Thus it is too provincial to accurately reflect a global market.

Finally, there are good reasons why the vast majority of small speculators prefer betting on this gold bull with gold stocks instead of gold futures. Of course stocks are easier to trade than futures and are perceived as less risky since stocks don't have expiration dates and can't be bought on extreme margin like futures can. This is not an accurate perception if margin is excluded though, as an unleveraged stock trade is far riskier than a hypothetical unleveraged futures trade.

I prefer gold stocks to gold futures for my own personal speculations simply because the risks and rewards in gold stocks are much greater. Yes, a company can see mine X fail to produce at plan or country Y go all Marxist and steal its flagship project. But when a company is successful at mining gold, its stock-price appreciation dwarfs gold appreciation and even far exceeds margined gains on gold futures. Bull to date at best the gold price is up 181% but the leading gold-stock index, the HUI, is up 996%. So gold stocks have leveraged gold's gains by 5.5x so far in this bull market!

Stocks also have other advantages over futures. There is usually only one major gold futures contract traded in a given country, so all futures speculators study it relentlessly. Since they only have one price to follow, they all become experts on it and it is really hard to achieve information superiority and beat the professional traders at their own game. If their only focus is gold futures, they will get darned good at trading them.

On the other hand, there are many hundreds of gold stocks. With so many stocks to follow, most traders know very little about all of them and no one is an expert on many. This perpetual lack of information means there are countless opportunities to exploit information asymmetry. A promising new gold miner may be trading at deep bargain prices simply because not enough traders know about it yet. Opportunities like this just don't arise in the singular gold-futures world.

Another big advantage stocks have over futures is their proportion of irrational traders. Margined futures trading is extremely unforgiving, so if a new trader isn't really good he is going to go broke sooner or later. This leaves a high proportion of very competent professional traders dominating gold futures. They eat new guys for lunch. But in stocks, everyone trades them. While most futures traders are good, most stock traders are bad. They are naïve and dominated by emotions which leads to poor trading decisions and price anomalies on both the high and low sides that we rational traders can exploit for profits.

So while stocks are higher risk, their potential returns are much higher too. So I believe there is very good reason why individual speculators would prefer to trade in the gold-stock sector rather than in gold futures. There are just so many more opportunities in gold stocks since they aren't followed as closely and a vastly higher proportion of irrational traders drives countless exploitable price anomalies in them. Perhaps this helps explain why CoT small specs' net-long positions have remained flat for many years now.

Gold futures are a fascinating and important realm, but they do not deserve the level of mysticism and fear they seem to generate. The futures priesthood that "informs" gold-stock traders often takes events out of context and disseminates half truths designed to sway sentiment. While I find their babblings rather amusing myself, it bothers me when my clients take them as gospel and fear futures action when they have no need to.

Thus at Zeal, even though we are primarily stock speculators, we track gold and silver futures CoT data on a weekly basis. Our subscribers can log into our website and look at big high-resolution updated versions of the charts in this essay anytime they want. By seeing the entire bull-to-date futures action at one time, it is much easier to keep current developments in proper context and not be steered astray by some paranoid guru.

In fact, during times like these when futures gurus are seeing terrible portents of woe in gold CoT data, we like to buy gold stocks. The gold-futures being-used-to-manipulate-gold-stocks to-steal-from-the-little-guy theories only thrive when sentiment is bad enough for a bottom to likely be near. Thus we have been aggressively buying elite high-potential gold stocks near technical lows in both our weekly and monthly newsletters. Subscribe today before gold stocks soar again and this opportunity vanishes!

The bottom line is the US CoT data on gold futures is interesting, but it doesn't drive the gold price. Global gold physical supply and demand does. The CoT data merely offers a tiny nebulous view into classes of traders that aren't very well defined and constantly shift. And it ignores all the other non-COMEX gold trading worldwide. Therefore gold futures CoT reports should be taken with a big grain of salt.

If some guru is wailing about record commercial net shorts, then you can instantly know that this also means record large-spec net longs. Maybe the glass is half full. If someone is spewing fear because gold OI has collapsed, realize it has already done this over a dozen times yet gold has still nearly tripled. Gold futures CoT mysticism is irrational and misplaced, don't be swayed by it.

Adam Hamilton, CPA

May 25, 2007

Thoughts, comments, or flames? Fire away at zelotes@zealllc.com. Due to my staggering and perpetually increasing e-mail load, I regret that I am not able to respond to comments personally. I will read all messages though and really appreciate your feedback!

Copyright©2000-2025 Zeal Research All Rights Reserved.

321gold Ltd