Before reading the article, please click "Follow", which is convenient for you to discuss and share, and can bring you a different sense of participation, thank you for your support.
Wen 丨 small cai rapeseed oil
Editor丨Small Cai rapeseed oil
preface
Reconfigured devices provide the ability to program electronic circuits on demand. In this work, the creation of artificial neurons, synapses and storage capacitors on demand in post-fabricated perovskite NdNiO is demonstrated in a device that can simply be reconfigured for a specific purpose with a single electrical pulse.
The sensitivity of the electronic properties of perovskite nickelate to the local distribution of hydrogen ions makes these results possible. Using the experimental data of the stored capacitors, the simulation results of the reservoir computing framework show excellent performance in tasks such as digital recognition and ECG heartbeat activity classification. Using reconfigurable artificial neurons and synapses, simulated dynamic networks outperform static networks in incremental learning scenarios. The ability to design brain-inspired computer building blocks on demand opens up new directions for adaptive networks.
The continuous learning of AI is a daunting challenge. Models are typically trained on smooth data distributions, so when new data increments are presented to neural networks, this interferes with previously learned knowledge, resulting in poor performance, which is known as catastrophic forgetting, and remains an active area of research.
One of the main ways to solve this problem is to actively adjust the structure of the network itself as new data becomes available. Not only does adjusting the architecture of the network based on the distribution of inputs allow the network to efficiently manage its resources, recent findings also show that dynamic networks can show better performance than static networks when the same resources are provided.
Smart technology
As intelligent edge devices become more integrated into society, they will need to implement complex networks in hardware that is constrained by chip area and power consumption. Being able to dynamically reallocate network resources to perform a variety of tasks in a changing environment is critical. Having programmability in hardware could be a game-changer for the computers of the future, designed to be inspired by the intelligence of animal brains.
In this work, perovskite nickelate, a class of quantum materials that undergo room-temperature electron phase transitions after hydrogen doping, was demonstrated to provide a versatile, reconfigurable hardware platform for adaptive computing. Individual devices made from H-doped neodymium NiO can be electrically reconfigured as needed to assume the functions of neurons, synapses, or storage capacitors.
This versatile adjustability is achieved by the synergistic combination of a large number of metastable configurations of protons in the perovskite lattice, which can also be voltage controlled. Although various ionoelectronic switches for neuromorphic computation are being explored, a complete reconfiguration of neuromorphic functions remains elusive.
To demonstrate the example application in AI, experimental data from storage capacitors were used in the reservoir computing framework, a brain-inspired machine learning architecture, and the simulation results showed excellent performance comparable to theoretical and experimental reservoirs. The experimental characteristics of neurons and synapses obtained by perovskite nickelate devices and their runtime reconfigurability were used to design adaptive dynamic on-demand growth networks.
Driven by data processing in the cerebral cortex, GWR networks provide an unsupervised approach to lifelong learning in real-world scenarios with limited availability of training samples that may be missing or noisy labels. It is proved that such a network can take advantage of the creation and deletion of network nodes, providing greater representation power and efficiency compared to static networks
Perovskite nickelates are a class of quantum materials whose electronic properties are mediated by strong electron interactions. The original NNO is a related metal at room temperature. Hydrogen dopants as electron donors can result in orders of magnitude reduction in conductivity by modifying the Ni orbital configuration. Gentle redistribution of hydrogen ions that have been doped in the lattice by an electric field can systematically alter conductivity to produce multiple electronic states.
By annealing the NNO device in hydrogen, hydrogen can be intermittently doped into the NNO lattice close to the electrodes. The hydrogen atoms then donate electrons to the Nid orbital, which changes the filling state of the NNOd band and causes a phase change where resistivity changes by orders of magnitude. Protons in the lattice can use a large number of metastable states, and their distribution and local concentration can then be modulated by an electric field applied to the electrode. The switching mechanism of the H-NNO device is compared to the traditional non-filamentous resistive memory device in Table S1.
Artificial neurons and synapses were created from the same device to study spike neuron behavior in H-NNO devices in an electronic state. A continuous electrical stimulus is applied to the device, and once a critical level is reached, a sudden change in the resistance of the device is observed. The nonvolatile neuronal response of nickelate devices to electrical stimulation depends on the pulse voltage and pulse width.
Perovskite nickelate
Synaptic behavior under electron state IV in nickelate devices was demonstrated by continuous voltage sweep. Under LRS, a small threshold pulse field is sufficient to modulate the device resistance, which is suitable for analog behavior where the resistance changes gradually. This analog update of the device's resistance suppresses sudden jumps in resistance required for spikes. In HRS, V is much higher than a thousand and requires changing resistance in favor of stimulating neuronal behavior.
In order to understand the nanoscale mechanism to achieve electrical reconstruction, representative H-NMO devices corresponding to synaptic and neuronal states, respectively, were deeply characterized in LRS and HRS. Confocal Raman spectra ranging from 300 to 550 cm−1 were first collected from two control samples: the original NNO film near the Pd electrode and a large number of doped NNO films near the Pd electrode. The pattern of TheT2 g NNO exists at about 439 cm−1 for the original NNO, while for heavily doped NNO, it disappears, indicating a dense proton concentration near the Pd electrode.
Two-dimensional Raman imaging was performed in the rectangular area of this boundary of the H-NNO device at LRS and HRS in Figure 2I. The relative peak intensity of T of 2 g is 0.77 for the H-NNO device at LRS, while for HRS, this mode drops to 0.68, indicating that the local proton distribution of H-NNO at HRS near the Pd electrode is higher. Near-field tip enhanced Raman scattering is performed on HRS near the LRS and Pd electrodes of the H-NNO device.
Nickelate devices with 100nm gap dimensions can demonstrate scalability, durability, repeatability and ultra-low energy consumption. In scaling devices, electrical reconfiguration can be achieved with electrical pulses of <10ns. The energy cost of a single synaptic renewal is about 2fJ, which is comparable to the energy cost in the brain. To demonstrate compatibility with CMOS technology, nickelate devices were fabricated on SiO 2 on a silicon substrate by sputtering and ALD.
The application of adaptive nickelate hardware applies experimental memristor and memrization capacitance behaviors in RC, a brain-inspired machine learning architecture that solves the training complexity and parameter explosion problems common in traditional recurrent neural networks by adapting only to simple output layers. RC explains higher-order cognitive function and the interaction of short-term memory with other cognitive processes.
For baseline comparison, the performance of the evaluated H-NNO device compared to theoretical models and experimental reports was used for three distinct tasks: MNIST digital recognition, isolated speech digital recognition, and ventricular heartbeat classification on ECG datasets.
Simulation results from A to C show that the H-NNO reservoir has comparable performance over three tasks compared to theoretical and experimental reservoirs. The performance-to-equipment ratio results show that the H-NNO reservoir is on average superior to the theoretical and experimental reservoirs in terms of MNIST, isolated voice digital and ECG heartbeat, respectively.
Having neuronal and synaptic functions in a single type of device enables compact and energy-efficient neuromorphic system design. In addition, the ability to reconfigure devices for multiple neuromorphic functions opens up their innovative uses in next-generation AI, namely in the emerging field of dynamic neural networks. One such example is the GWR network, which creates new nodes and their interconnections based on competitive Hebbian learning.
GWR networks extend the concept of self-organizing neural networks, adding or removing network nodes in an unsupervised manner to accurately approximate the input space, sometimes even more concise than static self-organizing mappings. Dynamic GWR can be compared with a static self-organizing network that uses the same Hebbian learning scheme but with a fixed number of nodes, initialized randomly at the beginning.
The network was trained on a subset of two prototype datasets used to assess literature performance, MNIST and CUB-200, to simulate how such a network would behave in flight.
For the dataset and network, two sets of simulations were performed using experimental data from H-NNO devices: incremental learning, in which the network shows updated categories of data over time, and assessing the impact of growing or shrinking compared to static networks - GWR represents the efficiency of the input space.
In an incremental learning scenario, when training two datasets for each new class. Dynamic networks were observed to retain their learned representations better than static networks, and the final accuracy test resulted in 212% higher MNIST accuracy and 250% higher CUB-200 accuracy. By scaling it up, the network avoided the pain of catastrophic forgetting, and as the number of classes increased, performance showed only a smooth decline.
Choose a static network whose size is equal to the maximum number of nodes required for the GWR network. This arrangement ensures that the observed differences are not due to differences in size between the two networks, but rather to the growth and learning capacity of dynamic networks. The ability of a GWR network to dynamically change its size to fit the input space.
The network is initially shown the first half of the total number of classes in the dataset, with the size growth and saturation of the GWR. Later, when the network presents the entire dataset, GWR rapidly scales up to accommodate the change. Static networks can't do this, so they can't learn new data, and they also suffer performance degradation in the initial class.
Dynamic networks achieved better accuracy on the test set compared to static networks: 170% for MNIST and 4% for CUB-200. Next, it is demonstrated that GWR is able to allocate its resources efficiently compared to large static networks. At the beginning, the network is shown all the classes of the dataset. After the learning occurs, half of the categories are removed, allowing the GWR network to reduce its size and reach a balanced number of nodes.
GWR was found to be able to maintain a similar level of performance to large static networks on a subset of interest and demonstrate higher efficiency by reducing the size of MNIST by 27% and CUB-200 by 4% In addition to simulation studies, proof-of-concept experiments were conducted to demonstrate the hardware reconfigurability of H-NNO devices in incremental learning scenarios compared to static networks.
conclusion
Artificial neurogenesis has already been demonstrated in perovskite electronics: the ability to reconfigure hardware building blocks for brain-inspired computers on demand within a single device platform. Dynamic deep learning networks simulated using experimental measurement characteristics of nickelate devices consistently outperform static counterparts.
The results demonstrate the potential of reconfigurable perovskite quantum electronic devices in emerging computing paradigms and artificial intelligence machines. In addition, the room temperature operation of semiconductor technology-compatible ALDs and test chips on silicon platforms can further enable the widespread adoption of perovskite quantum materials in mainstream integrated circuit manufacturing.
bibliography
- J.Kirkpatrick， R. Pascanu， N. Rabinowitz， J. Veness， G. Desjardins， A. A. Rusu， K. Milan， J. Quan， T. Ramalho， A. Grabska-Barwinska， D. Hassabis， C. Clopath， D. Kumaran， R. Hadsell， Overcome catastrophic forgetting in neural networks. Proc Natl Acad Sci U S A 114, 3521–3526 (2017).
- G. I. Parisi， R. Kemker， J. L. Part， C. Kanan， S. Wermter， Continual Lifelong Learning with neural Networks： A review.神经网络。 113， 54–71 （2019）.
- K.O. Stanley, J. Clune, J. Lehman, and R. Miikkulainen, designed neural networks through neuroevolution. Natmacher. 1， 24–35 （2019）.
- N. K. Upadhyay, H. Jiang, Z. Wang, S. Asapu, Q. Xia, J. J. Yang, Emerging memory devices for neuromorphic computing. Advanced Alma Mater Technology. 4， 1800589 （2019）.
- W. Yi, K. K. Tsang, S. K. Lam, X. Bai, J. A. Crowell, E. A. Flores, Biological rationality and randomness in scalable VO2 active memristor neurons. National Commune. 9， 4661（2018）.