In 1965, Gordon Moore hypothesized that the number of transistors on a microchip would double approximately every two years. The past few decades have shown that this is an accurate prediction, as more transistors are encapsulated on each newly released chip and node sizes shrink dramatically.
Nevertheless, there is a limit to the size of semiconductor nodes without negatively affecting the functionality of the device. According to the MIT review, the semiconductor industry has acknowledged that process node sizes will soon stop shrinking — as we know Moore's Law is coming to an end.
A promising way to effectively circumvent node limitations is chip-level heterogeneous integration. This means connecting several dedicated, smaller semiconductor devices in a package to create a system-in-package (SiP) rather than a system-on-chip (SoC). By splitting the chip's capabilities into smaller devices called chiplets, semiconductor manufacturers can achieve a higher level of integration than a monolithic SoC.
Displays chip-based ICs. Image courtesy of APEC
In recent years, as chip-based processors such as AMD's Zen 2 have grown in popularity, industry research and development has focused on improving inter-chip interconnect capabilities in heterogeneous architectures.
The rise of SiP chip architectures
In addition to improving yield, heterogeneous chip architectures allow manufacturers to create optimized processors by combining different types of cores in a single package. For example, heterogeneous mobile processors can have both high-performance, high-power cores and low-performance, low-power cores on separate chiplets. This allows operating system programs called schedulers to determine which programs (threads) are dedicated to a certain type of core and optimize overall power and performance.
Flowcharts show how modern operating system schedulers take advantage of heterogeneous CPU architectures. Image courtesy of the University of Manchester
Even so, chip-based designs have their own technical challenges. One of the main obstacles to SiP chip architectures is building cost-effective, high-performance, and energy-efficient die-to-die interconnects.
Parallel interconnects for inter-chip communication
As with other systems, there are two broad types of physical layer chip-to-chip interconnects: parallel and serial. Both parallel and serial interconnects have important advantages and are used according to the geometry of the SiP. In general, there are three types of SiP geometries: 2D, 2.5D, and 3D.
NASA's Electronic Parts and Packaging Program (NEPP) demonstrates progress in chip-on-chip packaging. Image courtesy of NASA
Historically, chip-based architectures and SoC architectures have typically used 2D package shapes. For such geometries, where two small chips may be far apart, serial SerDes PHY is typically associated with clocks and data transmitted using only one wire.
SerDes is a serializer/deserializer system. The system receives parallel clock and data signals from one chip, serializes them into a single line, and transmits them through the substrate to another chip at very high data rates. This is useful for longer transmission distances, such as those in 2D geometric SiP systems, because it eliminates timing bias between clock and data lines that occur in parallel interconnects. The timing deviation in this article refers to the difference in time when the data and clock signals arrive at the receiver due to the propagation delay in the transmission line.
SerDes interconnect between two small chips. Image courtesy of Microchip Technology and NASA
However, SerDes interconnects come at a cost: they typically consume more power due to the serial data and clock and the complex circuitry required to subsequently recover these two signals at the receiver.
To solve this problem, semiconductor design companies have begun to investigate 2.5D and 3D chip geometries using parallel interconnects and intermediary layers. The intermediary layer allows chiplets to be stacked and greatly reduces the distance between the data and clock signals that need to be transmitted between the chiplets. In addition, the inserter allows for very high density of parallel connections.
The high-density intermediary layer facilitates parallel interconnection between two small chips. Image courtesy of Microchip Technology and NASA
Because 2.5D and 3D geometries reduce the distance between dies through vertical stacking, timing deviations are not as problematic as they are in 2D geometries. As a result, parallel interconnects provide the best combination of power efficiency while still being able to match the bandwidth of the SerDes method used in 2D geometry.
In addition, parallel interconnects enable much lower latency transmissions because there is no longer any overhead associated with serialization, deserialization, encoding, and decoding in the SerDes system. In fact, Intel has demonstrated that its Advanced Interface Bus (AIB) standard has a much lower total latency (latency) than SerDes systems.
The table shows the lower latency of the AIB standard compared to the SerDes system. Image courtesy of Intel
Standardize the Chiplet interconnect protocol
With the rise of high-performance computing and machine learning, the workloads that heterogeneous processors have to handle have increased dramatically. As a result, a new protocol standard called Universal Chiplet Interconnect Express (UCIe) was announced to help build an open ecosystem of small chip chips across the semiconductor industry. UCIe is a layered protocol that specifies the physical layer, die-to-die adapter layer, and protocol layer. It allows 2D and 2.5D for encapsulated geometries, as shown in the following figure.
UCIe layer and encapsulation options. Image courtesy of Universal Chiplet Interconnect Express
UCIe seeks to be the energy-saving and cost-effective standard used throughout the semiconductor industry and may play a key role in future heterogeneous architectures.