1.
It's easy to start a company, it's hard to keep a company alive.
In order to keep NVIDIA alive, co-founder and CEO Jensen Huang has made many decisions over the past 29 years that were extremely risky at the time.
Amazingly, the vast majority of these decisions turned out to be correct: from TNT, to programmable Shaders, to the CUDA architecture and its open source and forward compatibility, and the level of computing power and storage performance that always exceeded the demands of mainstream applications in the current graphics market— these key technologies and products, as well as innovative strategic decisions, have time and time again pushed NVIDIA's position in the computing market to new heights.
Fueled by marketing strategies, people today honor Huang Jenxun as the "godfather of AI" and exaggeratedly describe Nvidia's graphic computing products as "nuclear bombs." In reality, however, Huang Jen-hoon did not have any illusions of being a god— many of the decisions he made were made in order to avoid the things he was most afraid of:
Nvidia was reduced to a mediocre supplier of PC components.
"There are so many pixels on the screen, and there are so many functions that can be added to the chip, but the number of transistors is still increasing." In this way, at a certain point in time, the graphics computing performance can be completely enough for anyone to use. If you look at today's various integrated graphics cards, good performance, and no money, you will find that this analysis is completely correct."
"Then, if we don't reinvent computer graphics computing, don't revolutionize our own lives, and don't completely liberate the capabilities of this graphics processor, the end is that NVIDIA will inevitably be commoditized."
—Jen-Hoon Wong, via Stratechery
With such fear, Huang Jenxun embarked on a pioneering path of self-destruction. This road, to this day, has been nearly thirty years.
2.
Recently, Nvidia held its annual technology conference GTC 2022, releasing the latest generation of commercial-grade H100 GPUs and the Hopper architecture named after renowned computer scientist Grace Hopper. The new architecture uses TSMC's 4nm process with a memory bandwidth of 3 TERA per second, which is three times faster than the A100 GPU of the previous generation Ampere architecture in 32- and 64-bit floating-point computing.
The H100 has become the latest generation of "nuclear bombs," but Nvidia has many more killer tools:
Through the latest interconnection technology NVLink technology, 8 H100 single card connections constitute the DGX H100 modular supercomputer, the computing power reached an astonishing 1EFlops, and NVLink Switch technology has also achieved an epic improvement in scalability, supporting up to 256 H100 GPU interconnects.
Because GPUs require a lot of memory bandwidth for deep learning calculations, NVIDIA even partnered with ARM to develop a CPU with a new Grace architecture, which is specifically designed to assist the GPU in bandwidth allocation, so that the GPU can be fully started for computation, and will not be affected by the limitation of memory bandwidth.
Nvidia, which has long been a leader in deep learning computing, has once again created new GPUs, CPU architectures, and interconnect technologies. Today' company is no longer a mere graphics technology company — it has pushed the power of graphics-accelerated computing to the limits, opening up new possibilities for deep learning computing.
At gtcover, Jen-Hsun Huang believed that humanity would usher in an era in which AI was created by AI (not just humans), calling it "intelligence manufacturing."
After the meeting, Jen-Hsun Huang was interviewed by Ben Thompson, lead writer/well-known analyst at Stratechery's blog. In particular, in this interview, Jen-Hsun Huang talked about some topics that were rarely discussed in the past — especially the fear of mediocrity of the company and the perception that NVIDIA's three decades of entrepreneurial road to today.
In his opinion, there is nothing more terrifying than turning Nvidia into an ordinary supplier in the "Wintel" ecosystem.
In 2009, Jen-Hsun Wong gave a lecture at Stanford University called "Vision Matters." Among them, he recalled that the company launched a programmable pixel shader (programmable pixel shader) in 2000, which almost killed the company at that time.
But if that decision hadn't been made, Nvidia might not have gotten where it is today.
As the main driver of the concept of independent graphics processors, Nvidia with more than 20 million US dollars from investors such as red shirts painstakingly developed GPU technology, although the first two products NV1, NV2 GPU have suffered failures, fortunately NV3 (officially named RIVA 128) due to relatively advanced technology, low price, launched less than 1 year to achieve 1 million shipments. On the basis of the RIVA 128, NVIDIA launched riva TNT the following year, which was significantly better than the products of competitor 3dfx at the time, and successfully won a number of graphics card manufacturers to join its camp.
The excellent results of RIVA 128 and TNT helped NVIDIA to be successfully listed in 1999, but Huang Renxun was no longer optimistic about the technical concept represented by these two products.
Early GPU products, including RIVA TNT, were fixed-function chips. The advantage of this type of chip is that it is very efficient to run the fixed function.
However, in the future that Wong sees, improving the pure performance of GPUs will become meaningless. Because the total number of pixels on the screen is limited, the functions that can be put into a processor are also limited. As a result, one day people will be satisfied with the performance of existing GPUs and no longer need to update faster GPUs... At that point, NVIDIA will be reduced to a mediocre PC parts supplier.
Thus, Huang Renxun began NVIDIA's first "deviant": the introduction of programmable shader.
The idea behind NVIDIA's actions is to change the identity of the GPU as a fixed-function processor, turning it into a programmable processor, allowing users to do more creative work on NVIDIA GPUs, including 3D rendering, special effects production, game development, etc. - so that Nvidia GPU users will not only be ordinary consumers, but also developers.
3.
This transformation almost cost Nvidia its life, not because it went in the wrong direction — it was correct, it just happened too early.
As mentioned earlier, processors designed for specific functions run efficiently, and if the GPU is to be programmable, the GPU will run less efficiently than before, and additional costs will be added in terms of computing power, memory and so on.
Nvidia made programmable shaders and adaptive graphics cards, but graphics developers aren't ready to pay for the future. The games and graphics computing applications they developed could not benefit from programmable shader technology at the time. As a result, NVIDIA's new technologies and products are great and powerful, but too expensive and useless from a consumer's point of view.
"This processor architecture is brand new, programmable pixel shaders have never been seen before, programmable GPU processors and programming models are unprecedented — all these awkward realities that we can only swallow into our stomachs." Huang Jenxun said.
GTC 2020, Jen-Hsun Huang took out the upcoming graphics card from the oven
"After swallowing, we then formed a compiler team, studied SDKs and libraries, went everywhere to find developers to talk to them about our new architecture, and made them realize the benefits of this set of things — we even had to develop libraries ourselves and show developers how easy it was to import their applications into us, what the benefits were; we even used marketing budgets to help developers market new products they developed with our architecture to create market demand..."
Over time, Nvidia ceased to be a hardware company. It can be said that it is a technology platform company that has to achieve vertical integration of "hardware technology + software experience + development ecology" in order to survive with dignity.
Later GeForce, CUDA, Tensor Core, etc., each of which wrote a strong note on NVIDIA's technology development process, are similar to programmable shaders in some ways: advanced technical capabilities, high programmable freedom, and business models open to ecological partners.
But at the same time, they were actually launched by Huang Jenxun to avoid NVIDIA going to mediocrity. Without Shader, CUDA, RTX, DGX and other technologies that appear to be "deviant" at their respective birth points, today's NVIDIA may have long been reduced to an ordinary graphics card company, and today's Brands that produce graphics cards under the name of GeForce are not much different.
4.
Nvidia's full commitment to AI technology in recent years is also the result of avoiding the company's mediocrity.
According to Huang Jenxun, the several milestone key technology launches in NVIDIA's history are actually "generalize" the development results of its own GPU technology, and then find that it can do more different things.
As a result, Nvidia has built a new, GPU-based computer science architecture as it continues to migrate and generalize GPU capabilities.
When the era of artificial intelligence comes, the new architecture built by NVIDIA is very suitable for accelerating the task of deep learning.
It is under such conditions that Huang Jenxun has changed from the previous graphics card king to today's "godfather of AI". But if anyone thinks that NVIDIA just happened to catch up with this wave of AI, he is dead wrong.
As early as ten years ago, Huang Jenxun already believed that graphics computing acceleration achieved the same as the early NVIDIA, and the new era of NVIDIA should be fully committed to the acceleration of AI computing, and no other company is more suitable for this than NVIDIA.
One of the most typical examples is robotics. "A classic robotics problem involves perception, reasoning, planning, and many different tasks that follow, According to Huang. These tasks involve large amounts of real-time data from multiple sensors; and for the purposes of diversification and redundancy, processors need to be processed with different algorithms. "The characteristics of these tasks are exactly what Nvidia's GPU architecture is good at.
Including AI, autonomous driving, data center/high-performance computing, supercomputers, industrial edge computing, metaverse-related fusion reality interactive technologies, and more... Now if we go to NVIDIA's website, we will see that this company is simply omnipotent, and it is no longer the graphics card company that most ordinary consumer users know.
These newer businesses can be seen as a reflection of NVIDIA's avoidance of mediocrity and constantly pushing its boundaries.
In the interview, Huang Renxun said that the greatest gift of his life was to have a group of the world's richest and most talented colleagues around him. And his own greatest talent is perseverance.
"I've been on this path longer than anyone else, but that's only because I'm patient. As long as I choose a path, I can always walk on it for a long, long time. That's my patience. ”