Nvidia's Superman

Is Jensen Huang the Seymour Cray or the Steve Jobs of his era?

After last week’s post on Google’s TPU

Jon Metzler

suggested that it would be useful to take a look at the ‘thousand chips’ that were inspired by the TPU. Great idea, Jon! Lots more analysis and commentary on the TPUs and the chips it’s inspired coming up.

First though, something a little lighter. A look at how three great tech icons have presented themselves, starting with Nvidia’s own Jensen Huang.

Why didn't someone raise a hand? After a tense moment, one programmer replied ‘How do you talk to God'?’

Talking about Seymour Cray from ‘The Supermen’

Along with most of the rest of the tech world, it seems, I’ve just finished watching Jensen Huang’s keynote at Nvidia’s GPU Technology Conference (GTC).

If you haven’t watched it yet, then I’d recommend setting a couple of hours aside. I think you’ll find it worth the investment of your time.

I enjoyed the keynote immensely, but it left me pondering how Huang and Nvidia present themselves to the world, which is the topic of this post. For more on the technical advances presented at GTC, see the ‘Further Reading’ section at the end of the post.

---

Whilst the keynote was, of course, mainly about Nvidia’s new products and its vision of the future, Huang is also clearly very aware of his company’s history. Early on he presents a slide with key milestones for Nvidia and wider industry, starting with the IBM S/360 in 1964. It’s been a long journey for Huang - and the rest of us - to get here.

It certainly feels like a historic moment. Some commentators have been moved to reach for historical precedents for this GTC. Some have compared it with Woodstock. The Economist compares Huang with Jimi Hendrix:

In advance, Nvidia’s fans on Wall Street had dubbed it the “AI Woodstock”. It wasn’t that. The attendees were mostly middle- aged men wearing lanyards and loafers, not beads and tie-dyes. Yet as a headliner, there was a bit of Jimi Hendrix about Jensen Huang.

Huang is clearly a ‘rock star’ of his industry. But what industry is that? Nvidia may make ‘Graphics Processing Units’ but it’s certainly not just about graphics any more. I think you could argue that Nvidia’s main business now is creating ‘supercomputers’, the most powerful computers in the world. Supercomputers like Eos:

Revealed in November at the Supercomputing 2023 trade show, Eos — named for the Greek goddess said to open the gates of dawn each day — reflects NVIDIA’s commitment to advancing AI technology.

Each DGX H100 system is equipped with eight NVIDIA H100 Tensor Core GPUs. Eos features a total of 4,608 H100 GPUs.

(Note that the number of H100s has doubled in the slide)

Coincidentally, I’ve just finished reading Charles R, Murray’s book ‘The Supermen: The Story of Seymour Cray and the Technical Wizards behind the Supercomputer’.

This book traces the development of the world’s most powerful computers from the early post-WWII years through to the 1990s. From Engineering Research Associates, through Control Data Corporation to Cray Research and Cray Computer Corporation. The ‘Supermen’ of the title are the designers of those computers, and especially Seymour Cray.

From the vantage point of the technology we have in 2024, the computers that Murray describes in his book might seem a little quaint. The iconic Cray-1 of 1975 was built out of integrated circuits with, by modern standards, tiny numbers of gates:

The Cray-1 used only four different IC types, an ECL dual 5-4 NOR gate (one 5-input, and one 4-input, each with differential output), another slower MECL 10K 5-4 NOR gate used for address fanout …¹

The Cray-1 had just 8 MegaBytes of random access memory and ran at 80 MHz achieving 160 Million Floating Point Operations Per Second (FLOPS). Nvidia’s Grace Blackwell ‘superchip’, announced at GTC, delivers 1.4 ExaFLOPS, an eight billion-fold increase over the Cray², and these individual chips are just components in data center sized systems.

The Cray-1 was a very ‘human scale’ machine, at 8 1/2 feet wide and 6 1/2 feet tall. It even had human-friendly (computationally heated) seats arranged around its central tower.

Cray - 1 at the Science Museum in London - By Irid Escent - 20180227_132902, CC BY-SA 2.0, via Wikipedia

Today, powerful computers have gotten both smaller and bigger. It’s a bit of a cliche that a Raspberry Pi can outperform a Cray-1. On the other hand modern data centers are, of course, much, much bigger than the facilities that housed the Cray and its successors.

There is another sense in which ‘supercomputers’ have got bigger. I don’t think it’s unreasonable to describe supercomputer of the Cray-1’s era as a ‘cottage industry’ in 2024 terms. Only around a hundred Cray-1’s were ever built. By contrast Nvidia shipped over half a million of its latest GPUs last year.

Those one hundred or so Crays were used to try to tackle a small number of problems in physics, cryptanalysis, weather forecasting and so on. Today, challenges such as weather forecasting are still a focus for Nvidia’s supercomputers, as we saw in the keynote.

But the plans that buyers of Nvidia’s systems have are much more far reaching and pervasive than was ever the case for Cray. Huang’s 2024 GTC included, early on, a slide with the names of the firms presenting at GTC.

Oil companies, auto makers, advertising, cosmetics, and on and on. Even Huang seems a little staggered by this list. These firms believe that AI, and by extension Nvidia and its products, will be pervasive.

We saw in Moore on Moore that Gordon Moore extrapolated the output of the semiconductor industry and projected it surpassing the world’s output of goods and services.

It’s worth recalling Moore’s own commentary on this projection:

I do not know how much of the GWP we can be, but much over one percent would certainly surprise me. I think that the information industry is clearly going to be the biggest industry in the world over this time period, but the large industries of the past, such as automobiles, did not approach anything like a percent of the GWP. Our industry growth has to moderate relatively soon. We have an inherent conflict here. Costs are rising exponentially and revenues cannot grow at a commensurate rate for long.

With the list of companies and the range of industries represented on Huang’s slide above perhaps Moore would be a little less surprised at the continued growth of his industry.

If Jensen Huang, as the modern-day supercomputer builder, is today’s ‘Superman’ how does he compare with Seymour Cray?

Cray certainly inspired awe. ‘The Supermen’ recounts a story about a Cray:

After a rare speech at the National Centre for Atmospheric Research in Boulder, Colorado, in 1976, programmers in the audience had suddenly fallen silent when Cray offered to answer questions. He stood there for several minutes, waiting for their queries, but none came. When he left, the head of NCAR’s computing division, chided the programmers. ‘Why didn't someone raise a hand?’ After a tense moment, one programmer replied. ‘How do you talk to God?’

Cray was also famously reclusive, basing himself well out of the public and corporate limelight at Chippewa Falls in Wisconsin. Here is one of a small number of recorded examples of Cray speaking in public. Do try to watch even just a couple of minutes of this. You get a real sense of Cray’s combination of dry charm and uncompromising focus.

Cray was relatively conservative in his choice of technologies. By contrast Nvidia uses close to ‘leading edge’ process nodes from TSMC. Cray was an engineer who shunned the business and worked ‘hands-on’ with small teams to design his computers. Huang is CEO of a company with around 30,000 employees.

And contrasting with Cray, Huang seems excited to present two-hour long keynotes to large, highly appreciative, audiences. I doubt that Seymour Cray would ever have exclaimed ‘God, I love Cray’ after a demonstration, in the way that Huang exclaims affection for his company.

If Huang’s presentation style isn’t like Cray’s then it’s much more like, Steve Jobs’s, seen below introducing the iPhone at Apple’s own developer conference, WWDC in 2007, another, by any measure, historic event.

Jobs was a very different individual to Nvidia’s CEO. But the parallels between the two men and their firms are stronger than one might expect. Both men took the companies that they founded to enormous financial success. Both men spotted a transformational change in technology and positioned their firms to take advantage of it. Both men saw their firms have ‘near death’ experiences and had, as Huang has recently recounted, ”… plenty of opportunities for setbacks and suffering”. Both men had signature clothing preferences, leather jackets for Huang, and New Balance sneakers for Jobs.

And Jobs knew full well the importance of the silicon that went into his products. He set Apple on the course that would see it design its own ‘System-on-Chips’ and make it the biggest customer of the biggest semiconductor foundry. Nvidia is now the second biggest.

And there are perhaps parallels in the development of their firms’ business models too. In amongst the technological innovations on display in the GTC keynote, there were shifts towards Nvidia claiming greater control and ‘lock-in’ as it offers more products and services built on its GPUs. Doesn’t that sound a little like Apple?

So overall the picture I have is a company that's closer to Apple in spirit, that wants total vertical integration over the ML stack, that purposefully gimps their products like disabling NVLink support for 4090 whose only goal is to get you to buy more GPUs.

GPUs were meant to be the next PC but instead we got the next iPad.

Mark Saroufim

I love my iPad, but I’m less keen on the control that Apple and Nvidia look to exert. And it’s not really new. Huang and Nvidia have known the importance of controlling their stack for a long time. Their investment in CUDA mirrors the investment that Apple has made in many areas of their ecosystem: the Safari browser, the Swift programming language, the Xcode development environment and so on. It’s been a major factor in Apple’s success.

But there is a notable difference between how Huang and Nvidia today and Jobs and his Apple. It’s a difference that is epitomised for me by a short speech that a frail Jobs made at his penultimate ‘Stevenote’.

I've said this before and I thought it was worth repeating. It's in Apple's DNA that the technology alone is not enough. That its technology married with liberal arts, married with the humanities that yields us the result that makes our heart sing.

Steve Jobs, March 2011

Technology is not enough.

And that shone through Jobs’s keynotes. There was always delight not just in the technology but in what it could do for people.

And Jobs regularly shared the stage with other people. Including the CEOs of companies that Apple worked with. Here is Jobs in 2006 sharing the stage with Intel’s Paul Otellini as Apple switched to Intel processors for the Mac.

As fas as I know, Jobs never shared the stage with Huang, however he was certainly happy to share the limelight with Nvidia’s products.

What about Nvidia? As Nvidia builds much of the hardware that seems likely to, as a minimum, fundamentally change the human race’s relationship with technology, did the keynote reflect that change?

Was there any discussion at all of the complex and profound implications of the application of Nvidia’s technology? If there was then I missed it.

Notably, humans, apart from Huang, were absent from the stage at the GTC.

The press release for Blackwell has the CEOs of Alphabet, Amazon, Meta, Microsoft, OpenAI, Tesla and others paying tribute to Nvidia new ‘Superchip’. But none earned a place on stage at the keynote. Nvidia is happy to partner with these firms. They all need Nvidia, but it doesn’t need them, at least not individually.

So who appeared on stage with Huang? Our new robot overlords in the shape of GR00T.

Is Huang today’s Seymour Cray or Steve Jobs? He’s his own man, of course, but I sense that he enjoys being a more likeable, approachable, ebullient, enthusiastic and extrovert Cray, designing and wielding systems with the power of billions of Cray’s supercomputers.

But in 2024, his role has changed. Nvidia is right at the center of the most important technology of our era. Huang is now in the limelight and has influence that reaches into billions of lives, just like Jobs and Apple’s iPhone. Actually, with the possibility of more influence than even the iPhone has had. Nvidia’s products could profoundly change humanity’s relationship with technology. And with that increased influence, perhaps, the nature of the role changes. It’s not just about the technology any more.

Further Reading on GTC

For an overview of GTC,

Doug O'Laughlin

‘s post is a great starting point.

Jensen's World: Compressing Reality

More on the pricing of Nvidia’s new products from

Dylan Patel

Nvidia B100, B200, GB200 - COGS, Pricing, Margins, Ramp - Oberon, Umbriel, Miranda

1. https://en.wikipedia.org/wiki/Cray-1

2. Note that Blackwell’s FLOPS are lower precision than the Cray’s