(2020-09-15) Thompson: Nvidia's Integration Dreams

Ben Thompson: Nvidia's Integration Dreams. Over the weekend Nvidia consummated the biggest chip deal in history when it acquired Arm from Softbank for around $40 billion in stock and cash. Nvidia founder and CEO Jensen Huang wrote in a letter to Nvidia employees: We are joining arms with Arm to create the leading computing company for the age of AI. Oct'2021 update: EU is investigating this, it still hasn't gone through.

Uniting NVIDIA’s AI computing with the vast reach of Arm’s CPU, we will engage the giant AI opportunity ahead and advance computing from the cloud, smartphones, PCs, self-driving cars, robotics, 5G, and IoT.

once you start digging into the details of Nvidia and ARM, it is rather difficult to see what strategy might be driving this acquisition.

It was in 2001 that Nvidia released the GeForce 3, which had the first pixel shader; instead of a hard-coded GPU that could only execute a pre-defined list of commands, a shader was software, which meant it could be programmed on the fly. This increased level of abstraction meant the underlying graphics processing unit could be much simpler, which meant that a graphics chip could have many more of them

This level of scalability makes sense for video cards because graphics processing is embarrassingly parallel

It turns out, though, that graphics are not the only embarrassingly parallel problem in computing. Another obvious example is encryption

this is why graphics cards are very popular for blockchain applications; miners are basically endlessly brute-forcing encryption keys.

What is most enticing for Nvidia, though, is machine learning. Training on large datasets is an embarrassingly parallel problem.

The CUDA platform allows programmers to access the parallel processing power of Nvidia’s video cards via a wide number of languages, without needing to understand how to program graphics.

Here the kicker: CUDA is free, but that is because the integration is so tight. CUDA only works with Nvidia video cards

Arm’s business model could not be more different. The company, founded in 1990 as a joint venture between Acorn Computers, Apple, and VLSI Technology, doesn’t sell any chips of its own; rather, it licenses chip designs to companies which actually manufacture ARM chips. Except even that isn’t quite right: most ARM licensees actually contract with manufacturers like TSMC to make physical chips, which are then sold to OEMs.

It also licenses less than full designs: Apple, for example, designs its own chips, but uses the ARM Instruction Set Architecture (ISA) to communicate with them. The ARM ISA is the platform that ties this entire ecosystem together; programs written for one ARM chip will run on all ARM chips

In 1987, Morris Chang founded Taiwan Semiconductor Manufacturing Company (TSMC) promising “Integrity, commitment, innovation, and customer trust”. Integrity and customer trust referred to Chang’s commitment that TSMC would never compete with its customers with its own designs: the company would focus on nothing but manufacturing.

the fact that TSMC existed created the conditions for an explosion in “fabless” chip companies that focused on nothing but design.

That article was about TSMC overtaking Intel in fabrication, but a similar story can be told about Arm overtaking Intel in mobile

The end result made everyone a winner (except Intel): nearly every smartphone in the world runs on an ARM-derived chip (either directly or, in the case of companies like Apple, the ARM ISA), which is to say that Arm makes money when everyone else in the mobile ecosystem makes money.

neutrality is gone under Nvidia ownership, at least in theory: now Nvidia has early access to ARM designs, and the ability to push changes in the ARM ISA; to put it another way, Nvidia is now a supplier for many of the companies it competes with, which is a particular problem given Nvidia’s reputation for both pushing up prices and being difficult to partner with.

By far the best articulation of the upside of this deal came, unsurprisingly, from Huang.

fairly obvious question: why does Nvidia need to own ARM, instead of simply license it

Huang’s voice was halting and uncertain, as if he were scared of his own ambition.

Number one is this: as you know, we would love to take Nvidia’s IP through ARM’s network

Number two, we would like to lean in very hard into the ARM CPU datacenter platform. There’s a fundamental difference between a datacenter CPU core and a datacenter CPU chip and a datacenter CPU platform

make ARM a first-class data center platform, from the chips to the GPUs to the DPUs to the software stack, system stack, to all the application stack on top, we want to make it a full out first-class data center platform. Well, before we do that, it would be great to own it.

Third reason, we want to go invent the future of cloud to edge.

Huang’s argument is that it is the lack of software — a platform, as opposed to simply a chip or a core — that is limiting ARM in the data center, and that Nvidia intends to build that software.

I don’t know if it will work; data centers are about the density of processing power, which is related to but still different than performance-per-watt, ARM’s traditional advantage relative to Intel

This vision of Arm’s future is in many ways incompatible with ARM’s neutral past, but the truth is Arm is already facing disruption of its own. RISC-V is an open-source ISA that is increasingly popular for embedded controllers in particular, in large part because it not only gets rid of Arm control, but also Arm license fees. I would expect investment in RISC-V to accelerate on this news, but it’s worth noting that it is just that — an acceleration of what was inevitable in the long run.