Reports from the Heart of the Machine
Machine Heart Editorial Department
The phone of 2022 looks like this: it will automatically help you sort out all the messages, scan the QR code without unlocking, change the antenna that receives the signal according to the holding method, and listen to the sound of the speech to determine whether you have a disease.
From heterogeneous computing power to upgradeable GPU drivers, Qualcomm's mobile phone chips can always lead the new direction of the industry.
On November 30, Qualcomm released a new generation of Snapdragon 8 mobile platform (Snapdragon 8 Gen 1) let us have expectations for the upcoming Android flagship phone, from the recently unbanned running score, this chip GPU peak performance increased by 59.7%, on the GFXBench score is even higher than the iPhone 13 Pro A15.
This is not the biggest, the new generation of Snapdragon 8 can be described as a comprehensive upgrade, its 7th generation AI engine has achieved a four-fold performance performance, more than one step, bringing a new experience.
In addition to the engineering machine on the booth can play "Original God" for 60 frames, at the Snapdragon Technology Summit, the concept of artificial intelligence runs through the whole process, and Snapdragon has been built into the core of all mobile terminals working together.
Multiple AI models run at the same time
In the past, apps typically ran only one or two neural networks at the same time. But for a better experience, more and more machine learning models are being added to our applications, and these neural networks are becoming more complex every year as terminal features evolve.
Qualcomm says the 7th generation AI engine can handle these needs while maintaining low power consumption.
Snapdragon's AI engine is powering 1.8 billion people as a major mobile phone chip maker, and some of the powerful features that have been implemented include machine learning algorithms that automatically help people adjust camera settings for optimal imaging results when taking photos. AI can also recognize and understand user speech and complete tasks based on user commands. On some brands of mobile phones, system applications are even able to translate people's conversations into different languages in real time.
Snapdragon's "dedicated AI processing unit" Hexagon processor processes emotion judgments up to 30 times faster than CPUs, allowing phones to perform local processing such as the Transformer architecture model.
Qualcomm showed that Hugging Face analyzes the content in all the drop-down notifications on the phone through natural language processing model BERT processing, and the AI will analyze the tone and mood in each message text, and classify it according to urgent, important, non-urgent, etc., so that you will not miss the email urged by your boss. If too many messages can't be found, you can also use voice commands to search for specific types of messages.
On the sensor hub module, Snapdragon 8 also originally added an "always on" chip module, this extremely low power ISP can be associated with the terminal's camera, driving the camera 7× 24 hours a day to start, thus bringing some convenience and security features - if a two-dimensional code is detected, the phone will automatically wake up the application for recognition; when you use the phone on the desktop, it will automatically lock the screen; if you hold the phone, someone peeking over your shoulder, the phone will automatically lock the screen Or if you want to share the content on the home screen of your phone screen to someone else, the sms prompt banner will be automatically hidden.
Many people have something to do when buying a mobile phone is to turn off the various information screen displays and hand-raising wake-up functions to pursue power saving, but on the Snapdragon 8 you don't have to do this, the new generation of sensor module AI computing power increased by 75%, energy consumption reduced by 50%. Qualcomm also promised that the new ISP would not take pictures or videos while working, and the data would not leave the module unit, ensuring the privacy and security of users.
Qualcomm showcased camera AI applications based on the 7th generation AI engine. The phone's camera supports better autofocus, auto exposure, and auto white balance adjustments, gesture detection allows users to take pictures without pressing any keys, and FEATURES SUCH AS AI-BASED video background bokeh.
Background Bokeh + Facial Recognition + Gesture Recognition + Gestures, the effects of multiple neural networks can now be completed in real time on the mobile phone side at the same time.
Details on the human face are also important. For the first time, Qualcomm Spectra has completely replaced the traditional algorithms that form the basis of photography, allowing AI to completely take over the task of shooting, which can recognize facial features and determine whether the subject's eyes are open or even smiling. The new Facial Feature Engine detects 300 feature points and is trained for more facial features, including eyebrows, lips, jaw lines, and more.
In addition to focus and special effects, the AI engine and camera engine can work together to process 3.2 billion pixels per second for greater zoom capabilities. Last year's "Night Mode" on the Snapdragon 888 combined about six frames of photographs to form a composite image. The Snapdragon 8's camera multi-frame large engine will combine about 30 frames to provide "five times the night mode". Because more pixels are tracked in low-light conditions, the camera can intelligently filter out blur caused by hand shaking for a sharper image. At the same time, artificial intelligence can also eliminate the distortion of the picture in wide-angle lenses.
This was followed by virtual reality technology, where Qualcomm demonstrated a new direction in which AI transformed the shopping experience. Through AR technology, people can decorate virtual furniture and try on sneakers at home to achieve a more personalized shopping experience. These applications are optimized through the Qualcomm AI engine.
On the Snapdragon 8 mobile platform, Qualcomm also integrated AI technology into modems and RF systems, bringing the world's first AI antenna tuning technology to detect the position of the user's hand when holding the terminal and switch the antenna receiving the signal, aiming for faster data transmission, better network coverage and longer battery life.
The 7th generation AI Engine doubles performance and energy efficiency
How is this done? We know that to be able to run AI models requires powerful computing power. On the AI engine of the Snapdragon 8 mobile platform, Qualcomm has achieved the largest AI computing power improvement to date.
Specifically, the Gen 7 AI engine consists of multiple low-power AI subsystems consisting of next-generation Adreno GPUs, Hexagon processors, Kryo CPUs, and 3rd Gen Qualcomm Sensor Hubs, each handling different computing needs. Multiple compute units across the SoC implement heterogeneous computing power of 6TOPS.
On some widely used AI models, such as inceptionv3, mobilenet_v2, Resnet50, VDSR, etc., the inference speed is 4 times faster than the previous generation OF AI engines, and the energy efficiency of AI operations is 1.7 times faster than that of the previous generation.
The 7th generation of AI engines is able to achieve this magnitude of improvement thanks in large part to the new generation of AI software: in addition to doubling the tensor computing performance of the Hexagon processor and doubling the shared memory, its AI software performance has also increased by 2x, and now supports the hybrid precision computing of INT8 + INT16.
Larger memory means that the chip can run larger, deeper neural networks, and higher precision can make the results of the algorithm's inference more accurate. On the development tools, the new neural network processing SDK has updated the operator support, and Qualcomm AI Engine Direct can directly access the mobile deep learning framework TensorFlow Lite and Android NN API, which is convenient for developers to apply their AI models faster on the mobile side.
Use your phone to complete tasks on your desktop
New AI technologies are emerging, how do you cram machine learning models into your phone? This is a very challenging task for engineers, and using AI technology to automate it is a promising direction. Qualcomm and Google have partnered to bring Neural Network Architecture Search (NAS) to the Snapdragon platform and have been integrated into the 7th generation of Qualcomm AI engines.
Just like AutoML's thinking, Qualcomm provides a NAS tool Neual Processing SDK that can input hardware configuration and work requirements (such as frame rate, accuracy) as parameters, and AI algorithms help engineers quickly achieve the optimal design of deep learning models.
AI can start at the most basic level, based on the initial network search and optimize to build a model that conforms to mobile hardware. It takes into account latency, network scale, accuracy, power consumption, and more, allowing models to run faster on mobile phones, while developers can reduce their working time from months to weeks.
Snapdragon is the world's first mobile platform to support Google Cloud Vertex AI NAS. Previously, Qualcomm's research on refining the best neural network architecture donna was also accepted by ICCV 2021, the academic summit of artificial intelligence. In the future, Qualcomm hopes to enable NAS capabilities to help with multi-terminal deployment efforts, allowing AI models to be quickly deployed to mobile phones, laptops, VR devices, and cars.
In addition to the technologies and use cases mentioned above, what other black technologies can the AI engine of Snapdragon 8 implement? For example, Qualcomm said they are working with Sonde Health to develop a diagnostic technology based on audio recognition, so that the phone only needs to listen to your voice to determine whether there are health problems such as depression and asthma.
And, of course, the super-resolution technology in the game screen:
By leveraging historical frame data and GPU-optimized motion estimation, the Adreno Graphic Motion Engine allows some games to run at double frame rates while maintaining nearly the same power consumption.
Today, there are more than 3 billion gamers worldwide, and mobile games account for 52% of the annual revenue. Among them, the chipsets supporting Snapdragon Elite Gaming have shipped more than 800 million units, and Qualcomm's new technology will benefit many players. According to Qualcomm, it is to let mobile gamers enjoy the terminal game experience.
Qualcomm and various OEMs and technology companies have delivered more than 200 machine learning models this year alone. In the future, we can also see more technologies that were unimaginable in the past on mobile phones.
Revolutionizing the mobile experience requires state-of-the-art technology. Next year, on the flagship mobile phone equipped with Snapdragon 8, what kind of experience will AI technology with powerful performance, reduced power consumption and rich features bring to us? Let's wait and see.