GPUs and the new 'digital divide'

I spent Tuesday at Nvidia headquarters, attending the company's annual Analyst Day.

I've been to most of Nvidia's analyst events over the last decade or so, since I covered Nvidia almost from its inception while working as the graphics analyst at Microprocessor Report. These meetings are always a good way to get an update on the company's business operations, and sometimes--like this time--one provides exceptionally good insight into larger industry trends.

Nvidia has had a rough couple of quarters in the market, which CEO Jen-Hsun Huang blamed in part on a bad strategic call in early 2008: to place orders for large quantities of new chips to be delivered later in the year. When the recession hit, these orders turned into about six months of inventory, much of which simply couldn't be sold at the usual markup.

In response, Nvidia CFO David White outlined measures the company plans to take to increase revenue, sell a more valuable mix of products, reduce the cost of goods sold, and cut back on Nvidia's operating expenses.

Three things stood out for me in this presentation:

Nvidia is planning an aggressive transition to state-of-the-art ASIC fabrication technology at TSMC, the company's manufacturing partner. Within "two to three quarters," White said, about two-thirds of the chips Nvidia sells will be made using 40-nanometer process technology. (The first of these chips were announced Tuesday.)

White also acknowledged something that I've long assumed to be true: Nvidia receives "preferential allocation" on advanced process technology at TSMC. It's logical that Nvidia should get the red-carpet treatment, having been TSMC's best customer for many years, but I don't recall hearing Nvidia or TSMC put this fact on the record before.

The third notable point from White's presentation: the gross margins for Nvidia's Tegra, an ARM-based application processor--which Nvidia's Mike Rayfield, general manager of the Tegra division, says has already garnered 42 design wins at 27 companies--are much higher than I'd have guessed--at "over 45 percent." That's quite excellent for an ARM-based SoC; it's a very competitive market.

More surprises
The technical sessions at the event contained their own surprises.

For example, Nvidia effectively seized control of an old Intel marketing buzzword: "balanced."

For years, Intel used to talk about the value of a "balanced platform," referring to a balance between CPU performance, memory bandwidth, and I/O bandwidth. This balance was the responsibility of the PC chipset, which in those days controlled the memory and I/O.

Now, Nvidia says that the most critical element of balance in the PC platform is between the CPU and the GPU.

The concept of balance also has some personal significance to me because I was the senior analyst responsible for PC core logic at Microprocessor Report when Intel introduced this concept, and to be honest, I thought it was irrational claptrap.

First, the systems from Intel's competitors were using roughly similar DRAMs, I/O interfaces, and peripheral chips, so balance wasn't a big source of competitive advantage for Intel.

Second, there's no universal concept of balance. Intel was trying to make the point that if an application hit a bottleneck in memory or I/O, more CPU performance wasn't going to help. But the tradeoffs between CPU, memory, and I/O are different for every application. Every applications will hit one bottleneck before the others no matter how the system is designed.

It's the same way today in the balance between CPU and GPU performance. Some applications rely heavily on the CPU, others use the GPU. Few use both. No application I can think of requires a specific "balance" between CPU and GPU performance.

But balance in the old days was a matter of minor differences: for example, Intel made a big deal about each new generation of chipsets with 33 percent to 50 percent more DRAM bandwidth. That usually translated into a small net performance advantage for the Intel platform: 10 percent or less.

Much bigger difference
Today, systems equipped with a good GPU can run some applications five times faster than a system without one. A great GPU can be 20 timed faster than a CPU of the same price on the same tasks. (This is a good place to point out that while they can't necessarily run all the same software, Advanced Micro Devices' ATI Radeon graphics chips are also GPUs--though AMD doesn't use that term.)

The performance advantages of GPUs are big enough that some applications can't be run at all without a GPU. That creates at least a minimum reasonable definition of "balance": each subsystem should be fast enough to support all the popular software on the platform.

For example, a notebook based on Intel's Atom processor and GMA 950 integrated-graphics core can't play all HD video content, or many new 3D games--even some aimed at wide audiences, such as the new Sims 3 from Electronic Arts.

Even when a given task can be achieved without a GPU, the low performance of CPU-based software will mean that some users simply won't bother. One of the most effective demos at the Nvidia event required nothing but the public beta version of Microsoft's Windows 7. Nvidia showed how Windows 7 will recognize certain personal media players (the demo used a Sony Walkman) and offer to convert videos to a compatible format when the user simply drags and drops an HD video to the player on the desktop. If a GPU is available, Windows 7 will use it to accelerate the transcoding process.

Nvidia says a one-hour HD video can be converted to the Walkman's native format in 38 minutes using the GPU in the Ion (GeForce 9400M) chipset--or three hours using an Atom processor alone. Without a GPU, the computer is basically 100 percent occupied for three hours. Seriously, how many people will bother going through that process?

(If 38 minutes is too long to wait, a high-end GeForce graphics card can do the same work in 8 minutes, according to Nvidia.)

New kind of 'digital divide'
It seems to me that such dramatic performance differences create a new (and less socially significant) kind of "digital divide." As more applications learn to take advantage of GPU co-processing, the practical advantages of GPU-equipped systems will eventually become overwhelming.

Since the cost of adding a GPU to the system is so low--already well under $50 for an entry-level add-in card and much less for integrated and motherboard-level GPUs--there's no good reason for OEMs not to include one.

But there is one bad reason: according to Huang, Intel is forcing OEMs of low-cost notebooks and small-form-factor systems to pay for its inferior integrated-graphics chipsets, which don't include GPUs, even if the OEM intends to use Nvidia's Ion chipset instead.

That's an extraordinary allegation, tantamount to accusing Intel of violating the Sherman Antitrust Act, which prohibits tying the sale of an undesirable good to that of a desirable good. (At least that's my take on the situation, but I Am Not A Lawyer, and I have no personal knowledge of this situation beyond what I've heard in public statements, mostly from Nvidia executives.)

Personally, I no longer recommend the purchase of computers without GPUs. The lack of a GPU is what dooms most Netbooks to premature obsolescence. Long before the usual lifespan of three or four years has passed, GPU-less Netbooks will be perceived as useless even by the least demanding users.

Nvidia says it expects to see a veritable flood of Ion-based notebooks, some small enough and cheap enough to compete directly with existing Netbooks, to reach the market in the second half of 2009. I think these systems will finally live up to user expectations, with full Windows 7 compatibility (including that magic transcoding feature) and enough performance for Sims 3 and most popular games.

But Intel has announced no plans to compete directly with Nvidia or AMD/ATI in the GPU arena. The company's Larrabee project, a plan to build a GPU from multiple 3D-optimized x86 CPU cores, is still a year away from release (and one well-publicized rumor recently pushed this date back to 2011, though Intel denied the rumor and I don't find it credible myself).

In the meantime, Intel intends to release Clarkdale and Arrandale, processors with package-level integration of its graphics engines. To me, this is just a tactic to let Intel sidestep the question of whether its tying practices are legal: in Clarkdale and Arrandale, the CPU and the graphics engine are physically inseparable, one single product by any legal definition. But in practical terms, these products will work just the same as separately packaged chips, and they will have the same effect on OEM buying decisions.

I expect many consumers will be dissatisfied with Clarkdale and Arrandale systems when compared with GPU-equipped machines. Intel will presumably use its best available graphics engines, ensuring full Blu-ray compatibility and the ability to play mainstream games, but they still won't qualify as GPUs, and they won't deliver the kind of performance Nvidia is already shipping today.

As usual, it'll be up to users to make sure they understand what they're getting and whether it meets their needs. That much, at least, will never change.