What has the breakthrough of Tsinghua AI optical chip by swiping the screen?

The semiconductor industry is vertical

2024-04-26 18:20Posted in Beijing, creators in the field of science and technology

What has the breakthrough of Tsinghua AI optical chip by swiping the screen?

Recently, there has been good news from Tsinghua University. The first AI optical chip architecture and the development of a new AI "optical chip" - Taichi (Taichi), which can achieve 160 TOPS/W general intelligent computing, and the energy efficiency is 1000 times that of H100.

训练下一代万亿级参数大模型的高效芯片诞生了。目前，相关研究论文以“Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence”为题，已发表在权威科学期刊 Science 上。

Address: https://www.science.org/doi/10.1126/science.adl1203

What are the results?

Currently, there are growing signs that LLMs will not be the ultimate path to AGI.

Computers have long been the world's energy giants, and as more and more power-hungry artificial intelligence is put into use, the energy demand for computers is also skyrocketing.

In the case of the NVIDIA H100, the peak power consumption is 700 watts, which is equivalent to the average power consumption of a U.S. household (assuming 2.51 people per household) at an annual utilization rate of 61%. Some experts predict that after the massive deployment of H100, the total power consumption will be comparable to that of a large American city, and even more than some small European countries.

If a chip can be invented that saves a lot of energy consumption, the performance of LLMs may be even improved in the future.

Tai Chi, on the other hand, may make artificial general intelligence (AGI) a reality.

According to the official website of Tsinghua University, the Tsinghua team designed a large-scale optical chip based on an integrated diffraction interference heterogeneous design and a general distributed computing architecture, Taiji, which has the capacity of tens of millions of neurons and realizes 160 trillion times per second · watt (TOPS/W) of general intelligent computing.

In addition, 1000 category-level classification on the chip (91.89% accuracy on the 1623-class Omniglot dataset) and high-fidelity AI-generated content were achieved in the experiments of the Taiji Optical chip, which improved the efficiency by two orders of magnitude.

According to the researchers, "Tai Chi" paves the way for large-scale photonic computing and advanced tasks, further exploring the flexibility and potential of photonics in modern AGI.

Science commented on the study: "The team from Tsinghua University explored a distributed diffraction interference hybrid optical computing architecture to effectively increase the scale of optical neural networks (ONNs) to the million-neuron level. An ONN of 13.96 million neurons on a chip was experimentally realized for complex, thousand-level classification and AI-generated content tasks. This work is a promising step towards real-world optical computing, supporting a variety of applications in artificial intelligence. ”

What is it, what is it for?

Bottlenecks in electronic chips

As electrons pass through transistors and other traditional integrated circuit components, they encounter resistance and generate heat. As designers continue to add various components to the chip, the heat generated by the chip naturally increases. This electronic property has even become an obstacle to the performance of microchips, and it is also the main reason why computers consume so much energy.

The development of technology with electronics as the carrier has approached the physical limit, the problem of "power wall" when the chip size is reduced to the extreme, the inability to store and calculate a large amount of information under the memory access bottleneck, and the improvement of the performance of electronic chips while reducing the cost performance.

In electrical circuits, the idea of replacing electrons with photons has been around for a long time.

In the sixties and seventies of the 20th century, researchers began to develop photonic chips. At that time, some experts expected photonic chips to be miniaturized as quickly as traditional integrated chips.

The reduction in power consumption of the circuit is also due to the nature of light. There is no resistance problem with photonic chips. This is because the photons generated by the laser can quickly pass through the waveguide, modulator, reflector, and other elemental arrays. As a result, photonic chips generate less heat and consume less energy.

A computing chip with light as the carrier

Optical computing, as the name suggests, is to change the computing carrier from electricity to light, and use the propagation of light in the chip to calculate.

From the perspective of energy consumption, silicon photonics technology can effectively improve the overall performance of GPUs, greatly reduce their power consumption, and effectively solve the current computing power bottleneck. In other words, the next generation of computing power is likely to be photonic computing or even quantum computing.

The problem is that current integrated photonic computing, especially optical neural networks (ONNs), typically contains hundreds to thousands of parameters, dozens of which are tunable parameters, and only supports basic tasks such as simple pattern recognition and vowel recognition. It can perform some simple tasks and shallow models, but cannot support the intelligent computing of complex large models that require high computing power and high energy efficiency.

Optical computing has higher requirements for integration, but its technical difficulties are not only in integration, but also in the recycling of computing units and nonlinear processing.

It is understood that in a huge AI model, each layer of the network needs to perform matrix operations, and a large matrix may need to be split into several small matrices for calculation. In other words, in a large model, matrix multiplication and addition is an iterative, cyclical calculation process, and with each cycle, the weights of the elements on the matrix are updated.

The matrix multiplication and addition of light is very fast as compared to electrons, but the speed of light slows down when it comes to updating the weights.

To achieve large-scale, energy-efficient photonic computing, it is not practical to simply scale up existing photonic neural network chips, as the inevitable analog noise expands exponentially as the number of neural network layers increases. Scaling up an existing architecture doesn't improve performance proportionally.

New architecture: distributed diffraction-interference hybrid photonic computing architecture

The Tsinghua team conducted architectural research and redesigned a new architecture suitable for optical computing "from 0 to 1".

According to the paper, the Tsinghua team built a network structure with a shallow depth but a wide width for Taiji, which uses distributed computing.

Unlike traditional methods of stacking a series of layers for deep computing, Taichi distributes computing resources across multiple independent clusters, organizes clusters separately for subtasks, and eventually synthesizes those subtasks to accomplish complex high-level tasks.

Specifically, the fully connected nature of the optical diffraction layer can provide greater deformation capacity than convolutional layers in traditional DNNs.

This means that it is possible for an optical network to achieve the same transformation with fewer layers than an electronic system.

Figure (B) shows the Taiji chip, which includes a dual diffraction unit for large-scale input and output data, and an adjustable matrix multiplication of MZI arrays for reconfigurable feature embedding and hardware multiplexing.

Xu Zhihao, the first author of the paper and a doctoral student in the Department of Electronics, introduced: "In the "Taiji" architecture, the top-down coding split-decoding reconstruction mechanism simplifies complex intelligent tasks and splits them into multi-channel and high-parallel subtasks. ”

AI optical chips: interfero-diffraction fusion computing chips

In this work, the team designed a large-scale diffraction-interference hybrid photonic AI chip with a flexible distributed computing architecture, Taiji.

According to the paper, the "Taiji" optical chip has an area efficiency of 879 T MACS/mm² and an energy efficiency of 160 TOPS/W, achieving an energy efficiency improvement of up to two orders of magnitude. For the first time, optical computing is empowered to realize complex AI tasks such as thousands of types of objects in natural scenes and cross-modal content generation.

What is the situation of domestic optical chip companies?

Among the potentially disruptive technologies facing the "post-Moore era", photonic chips have entered people's field of vision. Its advantages of high speed, low energy consumption, and relatively mature process technology can effectively break through the bottleneck of the physical limit of traditional integrated circuits and meet the technical needs of artificial intelligence, Internet of Things, cloud computing and other industries for information acquisition, transmission, calculation, storage and display in the new round of scientific and technological revolution.

At present, the global photonic chip industry has just started, and as a new integration technology independent of electronic integration technology, its technical barriers have not yet been formed. The development level of the photonics industry in the mainland is in parallel with the world, and it has certain advantages in the basic theoretical research and technological development of photonics.

At present, China's local high-power laser chips and some high-speed laser chips (10G, 25G, etc.) are in the stage of accelerating the breakthrough of localization, while optical detection chips and high-speed laser chips above 25G have just started.

It is worth noting that Huawei also has a layout in terms of optical chips.

In March this year, Huawei announced an invention patent for "optical chips, their preparation methods, and communication equipment". The publication number of the application is: CN117616316A, and the patent application date is September 18, 2021. The abstract shows that the embodiment of the present application provides an optical chip and a preparation method thereof and a communication apparatus that relates to the field of optical communication technology to solve the problem that the tip of the optical waveguide in the existing optical chip is easy to break during the preparation process.

The profits of optical chip companies declined

Domestic optical chip manufacturers include Yuanjie Technology, Changguang Huaxin and Focuslight Technology. In the 2023 annual performance report, all three companies mentioned the impact of the macro environment and industry development, which led to a decrease in product demand and a reduction in prices.

Yuanjie Technology is a leading optical chip IDM manufacturer in China, with products covering indium phosphide laser chips from 2.5G to 50G. Judging from the performance of Yuanjie Technology in 2023, the operating income totaled about 144 million yuan, a decrease of 48.96% compared with the same period last year, and the annual attributable net profit was 19.5 million yuan, a year-on-year decrease of 80.58%.

Judging from the performance report released by Changguang Huaxin, it is also in a state of loss. The operating income was about 292 million yuan, a year-on-year decrease of 24.2%, and the net profit loss attributable to shareholders of the listed company was 86.1017 million yuan. From January to June 2023, Changguang Huaxin's operating income is composed of: high-power single-tube series accounted for 90.16%, high-power bar series accounted for 8.24%, other businesses accounted for 0.9%, and VCSEL chip series accounted for 0.7%.

Focuslight's business covers upstream "generating photons" and "modulating photons", as well as midstream automotive, pan-semiconductor, and medical and health fields, and has reached cooperation with a number of well-known companies in the industry. Focuslight released its 2023 annual performance report, with operating income of about 561 million yuan, an increase of 1.69% year-on-year, and net profit attributable to shareholders of listed companies of 89.68 million yuan, a year-on-year decrease of 29.44%

Yuanjie Technology said that "the sales of the telecom market and data centers are less than expected", Changguang Huaxin said that "due to the impact of macroeconomic environment and other factors, the market confidence is insufficient, the demand for laser market continues to be weak, and the industry competition is intensifying", and Focuslight Technology said that "the price of some upstream components of the company has decreased, and the comprehensive gross profit margin has declined".

View original image 741K