The Tsinghua team developed an optoelectronic fusion chip with more than 3,000 times the computing power of commercial GPUs, promoting the construction of an eco-friendly AI computing framework

A few days ago, the team of academician Dai Qionghai of Tsinghua and the team of associate researcher Qiao Fei welcomed the new results of the chip, and they created a photoelectric fusion chip called ACCEL. The system-level computing power and energy efficiency of the chip are measured to reach more than 3,000 times that of high-performance industrial-grade GPUs and more than 4,000,000 times that of energy efficiency, with the characteristics of ultra-high computing power and ultra-low power consumption.

The minimum linewidth of the optical part of the ACCEL chip is only 100 nanometers, while the circuit part only uses the 180nm Complementary Metal Oxide Semiconductor (CMOS) process, which has achieved orders of magnitude performance improvement over the GPU of the 7nm process.

The researchers said: "Figuratively speaking, if the original power can support the operation of existing high-performance chips for one hour, then the ACCEL chip can work for more than 500 years under the same power supply. ”

Figure | Group photo of some team members (Source: Infographic)

The experimental demonstration in the paper shows that the successful development of the chip proves the superiority of photonic computing in many AI tasks (that is, photon supremacy), and also opens up a new path to solve the slowdown in the growth rate of Moore's Law and build an eco-energy-friendly large-scale AI computing framework.

Figure | ACCEL chip (Source: Research Group)

In the paper, the researchers used "All-analog Chip Combining Electronics and Light" to describe the characteristics of this photoelectric fusion chip. The English initials are simply ACCEL, which happens to mean "acceleration".

At present, human beings are in the era of explosive growth in computing power demand, and ultra-high-performance computing architectures have a lot of use. The researchers are keen to quickly put the ACCEL chip into practice.

At present, they are exploring a series of applications based on the optoelectronic computing framework of ACCEL chips, such as autonomous driving, field monitoring, IoT sensor networks, computer vision, etc.

At present, they have carried out the exploration of ultra-high-speed image computing for signal encoding and decoding and error correction in optical fiber communication, which is expected to reduce the delay of end-to-end signal processing of optical fiber communication by four orders of magnitude.

Once the calculation time changes from three hours to three seconds, many daily life applications and scientific computing tasks will change qualitatively.

Figure | Computing principle and chip architecture of optoelectronic computing chip ACCEL (Source: Nature)

According to reports, the ACCEL chip realizes the calculation of neural networks by fusing optical domain computing and analog domain electrical computing. In the optical domain, the ACCEL chip uses a multilayer optical diffraction neural network to perform feature extraction and data dimensionality reduction at the speed of light for the input high-resolution image.

The output of the diffraction network is received by an array of photodiodes and converted into an analog current signal by photoelectric effects. With this photometric processing, the data dimension can be greatly reduced, thereby reducing the scale of photoelectric conversion.

Among them, the photocurrent generated by each photodiode flows into the corresponding calculation node according to the weight parameters of the electrical network, and the electrical calculation of the simulation domain is realized based on Kirchhoff's law.

At this time, through the photodiode, an ultra-high-speed, low-power photoelectric interface, the optical network and the electrical network are connected, so that the photoelectric fusion computing system can achieve direct and efficient integration.

So, in vision tasks such as autonomous driving, how is the generalization ability of ACCEL chips? Generalization ability usually refers to the ability of a model to adapt to new samples and new scenarios.

ACCEL not only shows good generalization ability on different test sets, but also has excellent generalization under different working conditions. For example, the same is used for traffic scene computing, if there are extremely low light, ultra-high frame rate and other scenarios, compared with light calculation alone or electrical calculation, ACCEL chip shows excellent robustness under the anti-noise training algorithm.

In addition, existing optical computing systems are often designed for specific tasks, which limits their range of applications. The ACCEL chip integrates optical domain computing and analog domain electrical computing, which can be easily reconstructed.

When an ACCEL chip is designed and prepared for a specific task, the ACCEL chip can retrain the parameters of the electrical network to be adapted to different tasks with little to no compromise on final accuracy thanks to the ease of programming in the electrical signal domain.

Figure | Performance of the optoelectronic computing chip ACCEL under different tasks and light intensity (Source: Nature)

Optical chips, what's the difference?

Compared to traditional electronic chips, optical chips use photons to complete relevant calculations. Compared with traditional electronic chips, it does not use electricity as a carrier to complete digital signal processing, but calculates through the information changes in the propagation and interaction of light.

For example, in the famous Young's double-slit experiment in the history of physics, after the coherent light passes through the baffle with two slits, it will get light and dark stripes on the back detection plate. If the coherent parallel light is regarded as the input and the pattern on the detection plate is regarded as the output, the above experiment can be simply abstracted as follows: the baffle regulates the input light, and realizes the processing of the input light signal through the propagation of light between the baffle and the detection surface.

For existing optical computing, many ideas are similar to the above process. That is, by finely adjusting the process of light propagation, the physical properties such as optical phase, optical amplitude, and optical polarization at the receiving position are changed, so as to realize the calculation and signal processing of the optical domain.

The advantages of optical computing chips are the high speed, low energy consumption and large bandwidth of photons, which can provide a promising solution for massively parallel computing and high-speed data transmission.

At the same time, in a large number of visual tasks and daily life scenes, the original signal itself is an optical signal. With traditional solutions, it is necessary to process it with electronic chips after the sensor has been captured, which adds photoelectric conversion, storage, and calculation steps. In contrast, using light to perform calculations directly is a more natural and efficient way.

Optical chips, what are the shortcomings?

In recent years, in the face of the slowdown in the growth rate of Moore's Law and the failure crisis, optical computing, as a new computing paradigm, has received widespread attention and high expectations. Compared with current electronic devices, by directly processing the original visual information in the light domain, optical computing can be improved by orders of magnitude in speed and energy efficiency. However, the current optical computing system faces international problems such as complex nonlinear implementation and energy consumption of optoelectronic interface, which makes it difficult to implement and apply the high-performance advantages evaluated by many scientific research works.

Based on this, the team aimed at overcoming the bottlenecks in the current field of optical computing, so that the ultra-high performance of optical computing can move from the laboratory to daily life.

Optical chips, how to improve?

In order to solve the above international problems, this study proposes a deep fusion of optical computing and analog electrical computing for the first time, and establishes a fully simulated chip computing framework.

In order to overcome the pain points of existing optical computing systems, the researchers turned their attention to the electric domain simulation calculation, which is also a simulation calculation: it uses Kirchhoff's voltage and current law and charge conservation law to achieve calculations. When the optical signal is converted into an analog electrical signal through the photoelectric effect, there is an intrinsic nonlinear relationship.

Based on this, they propose a new computing paradigm: ACCEL integrates diffractive neural networks for large-scale extraction of visual features and purely analog electronic computing based on Kirchhoff's law within the same chip framework. This bypasses the physical bottlenecks of the analog-to-digital converter that limit the speed, accuracy and power consumption, so that the three key bottlenecks of large-scale computing unit integration, efficient nonlinearity, and high-speed optoelectronic interface can be broken through in a single chip. Achieve high computing energy efficiency and computing speed while guaranteeing high task performance.

Figure | ACCEL is expected to be used for ultra-low-power face wake-up schematic GIFs for electronic devices (Source: Tsinghua University)

An online conference that gave birth to a Nature paper

In the corresponding papers of this work, there are as many as four corresponding authors, who come from different teams. This starts with an online conference in 2020, when Qiao Fei, an associate researcher in the Department of Electronics at Tsinghua University, listened to a report on optical computing by assistant professor Wu Jiamin.

Later, in the discussion between the two research groups, the idea came up: since they are both in the field of analog computing, can they jointly solve the bottleneck problem in the field through in-depth cooperation?

Soon they decided on the subject. Subsequently, theoretical modeling and simulation verification were carried out, and physical derivation, physical simulation and chip design were carried out for the computational model of diffractive light network, nonlinear model simulation of photoelectric effect, and computational model of electrical network.

Later, in order to overcome the error accumulation and noise in the actual system deployment after the actual tape-out and chip measurement, they modeled these non-ideal factors, and developed a set of systematic correction algorithms to deal with non-ideal factors such as weak light noise, alignment, and processing errors. In this way, a high accuracy rate of experiments consistent with the simulation results is achieved.

Since then, they have evaluated the energy efficiency and computing power at the system-on-chip level. The measured results show that the ACCEL chip is more than a thousand times and more than a million times higher than the current high-performance commercial industrial GPUs in terms of system-level computing power and energy efficiency, respectively.

To ensure the reliability of such astonishing data, the researchers did a particularly solid job of measuring and validating it.

They not only measured the end-to-end system-level energy consumption data and latency data of the ACCEL chip, but also further proposed the concept of equivalent computing power. Measure calculations directly from an accuracy perspective, eliminating the impact of different physical modeling methods.

It can truly achieve the same accuracy as digital convolutional neural networks even on complex data sets, while reducing end-to-end system-level time consumption by a thousand times and energy consumption by a million times. It dispels the concerns of the industry about the "validity" of optical computing power.

Finally, the related paper was published in Nature[1] with the title "All-analog photoelectronic chip for high-speed vision tasks", with doctoral student Chen Yitong, doctoral student Maimeti Nazamat, and Dr. Xu Han as co-authors, and Academician Dai Qionghai, Associate Professor Fang Lu, Associate Researcher Qiao Fei, and Assistant Professor Wu Jiamin of Tsinghua University as co-corresponding authors.

Figure | Related papers (Source: Nature)

Subsequently, they will study larger and more powerful analog domain photoelectric fusion systems, which require higher-level joint design optimization at the algorithm level and hardware level.

In addition, an efficient hardware computing platform based on a large language model and based on a new AI algorithm is also one of the directions worth studying.

There is no doubt that the improvement of hardware computing power is one of the important engines leading today's AI wave. The researchers believe that the framework based on fully simulated photoelectric fusion computing has a very good application prospect.

In order to further expand the scope of application, it is necessary to build an ecosystem from software to hardware. A sound ecological environment needs to be jointly created by academia and industry, so they are very much looking forward to the industry deploying relevant businesses in this direction, so that cutting-edge academic achievements can be accelerated into products and complete the further leap of the efficient computing platform paradigm.

Resources:

1.Chen, Y. et al. All-analog photoelectronic chip for high-speed vision tasks. Nature https://doi.org/10.1038/s41586-023-06558-8 (2023).