Due to the boom in artificial intelligence, the chip world is undergoing a huge transformation.
The demand for chips that can train AI models faster, as well as chips that can run models on mobile phones, allows us to use these models without revealing private data.
Governments, tech giants, and startups all want a piece of the growing semiconductor market.
Here are four trends for the year ahead that will define what the chips of the future will look like, who will make them, and what new technologies they will unlock.
(Source: AI-generated, picture-text unrelated)
Chips Bills around the world
In the suburbs of Phoenix, two of the world's largest chipmakers, TSMC and Intel, are building factories in the desert that they hope will bolster America's chip-making capacity.
The common denominator behind these efforts is funding. In March 2024, U.S. President Joe Biden announced $8.5 billion in federal funding and $11 billion in loans for Intel's expansion in the United States. A few weeks later, he gave TSMC another $6.6 billion.
These grants are just part of the U.S. subsidies to the chip industry, based on the $280 billion CHIPS and Science Act signed in 2022.
This huge funding means that any company that ventures into the semiconductor ecosystem is thinking about how to restructure its supply chain and then benefit from the grant.
While much of the funding is aimed at boosting chip manufacturing in the U.S., everything from device manufacturers to niche materials startups can get involved.
But the U.S. isn't the only country trying to build (partial) chip manufacturing supply chains at home. Japan has allocated $13 billion to its own chip bill, and Europe will spend more than $47 billion. Earlier in 2024, India also announced a $15 billion investment in the construction of chip factories.
Chris Miller, a professor at Tufts University in the United States and author of the book "Chip Wars," said the roots of this trend can be traced back to 2014.
"This creates a dynamic in which other governments conclude that they have no choice but to offer incentives," he said. ”
This sense of unease, combined with the rise of the AI boom, has led Western governments to fund alternatives. In the coming year, this could have a snowball effect, with more countries launching their own chip projects for fear of falling behind.
Miller said the money is unlikely to create entirely new chip competitors or fundamentally shake the chip-making industry. Instead, it will primarily encourage industry leaders like TSMC to set up shop in multiple countries.
But funding alone wasn't enough to do that quickly, and TSMC's efforts to build a factory in Arizona, USA, ran into a quagmire, with repeated delays to completion, labor disputes, and Intel also failing to meet its promised deadlines.
It is not yet known when and even if these plants will be able to start operating, and whether their equipment and staff will be at the same level as the most advanced factories abroad of these companies.
"Supply chains are only going to slowly shift over years, if not decades," Miller said. But the situation is changing. ”
More focus on AI at the edge
Currently, most of our interactions with AI models like ChatGPT take place via the cloud.
This means that when you ask ChatGPT to pick out an outfit (or be your boyfriend), your request is sent to OpenAI's servers, where the model on the server processes it and completes the reasoning (like reaching a conclusion) before replying to you.
There are some drawbacks to relying on cloud services. For example, it needs to be connected to the internet, which also means that some of your data is to be shared with model operators.
That's why people and money are interested in edge computing for AI. In edge computing, AI models are invoked and computed on your device, such as a laptop or smartphone.
The industry is making a big push for AI models that know users better. OpenAI's CEO, Sam Altman, once described to me what he sees as a killer AI application that "knows exactly my whole life, every email I have, every conversation."
As a result, there is a demand for faster edge computing chips that can run models without sharing private data.
These chips face different constraints than data center chips, and they often have to be smaller, cheaper, and more energy-efficient.
The U.S. Department of Defense is funding a lot of research into fast, private edge computing. In March 2024, its research arm, the Defense Advanced Research Projects Agency (DARPA), announced a partnership with chipmaker EnCharge AI to develop a super-powerful edge computing chip for AI inference.
EnCharge AI is working to create a chip that protects privacy and can also operate at very little power. This will make it suitable for military applications such as satellite and off-grid surveillance equipment. The company expects to launch them in 2025.
Some applications of AI models will always rely on the cloud, but new investments and interest in improving edge computing may lead to faster chips for our everyday devices, leading to more AI.
If edge chips are small enough and cheap enough, we are likely to see more AI-powered smart devices at home and in the workplace. Today, AI models are mostly confined to data centers.
Naveen Verma, co-founder of EnCharge AI, said, "Many of the challenges we see in the data center will be solved. I'd like to see [industry] focus on the edge, which I think is critical to deploying AI at scale. ”
Tech giants move into chip manufacturing
Companies in every industry, from fast fashion to lawn care, are paying high computational costs to create and train AI models for their businesses.
Current applications include models that scan and summarize documents, as well as external-facing technologies such as virtual agents that can guide you on how to repair your refrigerator. This means that the demand for using cloud computing to train these models has peaked.
The companies that provide most of the cloud computing services are Amazon, Microsoft, and Google. For years, these tech giants have been looking to improve their profit margins by using self-developed chips in their data centers, rather than buying chips from companies like Nvidia.
Nvidia has a near-monopoly on the most advanced AI training chips, with a market capitalization that exceeds the GDP of 183 countries.
Amazon acquired startup Annapurna Labs in 2015 and began its journey to develop its own chips.
Google launched its own TPU chip in 2018. Microsoft launched its first AI chip in 2023, and Meta launched a new version of its AI training chip in 2024.
This trend is likely to affect Nvidia's share. But in the eyes of big tech companies, Nvidia is an indispensable supplier in addition to playing the role of a competitor.
Whether or not their own internal efforts succeed, the cloud computing giant's data centers still need its chips.
That's because their own chip manufacturing capabilities can't meet all the demand, and their customers want to use the best performing Nvidia chips themselves.
Rani Borkar, head of Microsoft's Azure hardware division, said: "It's really about giving customers a choice. ”
She said she couldn't imagine a future where Microsoft would use all of its own chips in its cloud services: "We will continue to maintain strong partnerships and deploy chips from all the partners we work with." ”
At the same time that the cloud computing giant is trying to steal market share from chipmakers, Nvidia is taking similar action.
In 2023, the company launched its own cloud service so that customers can bypass Amazon, Google, and Microsoft and access cloud services directly on Nvidia chips.
As this fierce battle for market share unfolds, the question for the year ahead will be how customers perceive Big Tech's chips as on par with Nvidia's most advanced chips or more like its spare tire.
Nvidia fights startups
Despite Nvidia's dominance of the chip industry, there has been a wave of investment going to startups that are aiming to beat Nvidia in some areas of the chip market of the future.
These startups all promise to accelerate the training of artificial intelligence, from quantum to photons to reversible computing, and they have different ideas about these fresh computing technologies.
Murat Onen, 28, is the founder of chip startup Eva. The company was born out of his Ph.D. work at MIT, and he described what it's like to start a chip company now.
"Nvidia stands at the top of the mountain, and this is the world we live in." He said.
许多初创公司,如 SambaNova、Cerebras 和 Graphcore,正试图改变芯片的底层架构。
Imagine an AI accelerator chip constantly moving data back and forth between different regions.
A piece of information is stored in storage, but must be moved to the processing area, where it is computed, and then sent back to the storage for safekeeping. All of these activities take time and energy.
Improving the efficiency of this process will provide customers with faster and cheaper AI training conditions, but only if the chipmaker has good enough software to enable AI training companies to seamlessly transition to new chips.
If the software transformation is too unwieldy, model makers such as OpenAI, Anthropic, and Mistral may choose big-name chip manufacturers.
This means that companies that take this approach, such as SambaNova, spend a lot of time not only on chip design, but also on software design.
Ao Nang proposes a deeper change. As traditional transistors became smaller and more efficient over the decades, he used a new component called a proton-gated transistor.
He said that Eva has designed this component specifically for the mathematical needs of AI training.
It allows devices to store and process data in the same place, saving time and calculating energy consumption. The idea of using such components for AI inference dates back to the 60s of the 20th century, but researchers at the time were unable to find a way to use it for AI training.
Part of the reason is that the material is not advanced enough, and it requires a material that can precisely control the conductivity at room temperature.
One day in the lab, Ao Nan said, "by optimizing the numbers, we were very lucky to get the material we wanted." Suddenly, the device was different and no longer a research project. ”
This increases the likelihood of large-scale use of such components. After months of data confirmation, he founded Eva. A paper related to this work was published in Science.
But in an industry where many founders have promised but failed to topple the dominance of leading chipmakers, Ao Nang candidly admits that it will be a few years before he knows if his designs will work as intended, and whether manufacturers will agree to produce.
Leading a company through this uncertainty, he said, requires flexibility and standing up to the skepticism of others.
"I think sometimes people get too attached to their ideas and lead to insecurity," he said. They will think that if the idea fails, there is nothing. I don't think so, I'm always looking for people who challenge us and say we're doing it wrong. ”
Support: Ren
Operation/Typesetting: He Chenlong