Yifan from the Vice-Pilot Temple
Smart Car Reference | Official account AI4Auto
The elite is exhausted, and Huawei is crazy.
Huawei Developer Conference 2024 (hereinafter referred to as HDC 2024) opened with a shocking opening, with "pureblood" Hongmeng, Pangu large model 5.0, programming language Cangjie, and 5.5G...... New achievements have exploded one after another.
The Science and Technology Spring Festival Gala is high-energy, and the technical feast is full of dry goods.
In particular, the transformation of the Pangu model to autonomous driving has attracted the attention of the industry.
AI-generated videos are so realistic that autonomous driving doesn't have to rely on road tests.
Huawei's Pangu model reshapes autonomous driving
At HDC 2024, Huawei officially announced that the Pangu model will be upgraded to 5.0, greatly improving its multimodal capabilities.
The so-called multimodality, that is, the Pangu model 5.0 can better and more accurately understand the physical world, including text, pictures, videos, radar, infrared and remote sensing, and the generated content is more in line with the laws of reality.
From the perspective of application implementation, focusing on the autonomous driving industry, Pangu Large Model 5.0 will support flexible customization to generate training video data for different road conditions, different lighting, and different weather, with a duration of up to 1 minute.
At the scene, Huawei also demonstrated this capability:
It can be flexibly customized, whether you want to generate an empty street or a complex road situation with multiple vehicles:
Weather conditions can also increase intensity:
If you look closely, you can see that when the weather changes, the details of the generated vehicles are different:
In the same car, the lights are on in the rain.
How do you do it so realistically and intelligently?
Behind it is the innovative controllable spatiotemporal generation technology (STGC), the core of which is both data and model.
For training data, Huawei uses six cameras with a 360-degree view, collecting and managing a total of 200,000 frames of high-quality data.
In terms of models, Huawei has added three input modules to the VAE and DiT architectures for video generation.
- 3D边界框编码器(3D bounding box Encoder)
- BEV encoder for road networks
- Camera track encoder
Then, the algorithm is used to process the 3D bounding box and the BEEV road network diagram, so that the Pangu large model can realize multi-view correlation learning, and learn the multi-view space and timing consistency. to generate the scenarios required for autonomous driving training.
This helps to solve two of today's challenges in autonomous driving:
Generalization of datasets and edge scene generation.
The training of autonomous driving systems is very complex, and in the past, it relied on road tests, which required a large investment in deploying fleets, high cost and long time cycles for road test data collection.
At the same time, the growth of autonomous driving capabilities is limited by the edge scenarios encountered, which is relatively rare.
Moreover, when encountering edge scenarios, after adjusting the system in a targeted manner, it may not be possible to reproduce and verify, after all, road conditions and vehicle conditions are changing at any time.
And extreme edge scenes, more dangerous.
For these problems, in fact, the industry has also proposed a route for simulation testing earlier.
However, in the past, modeling first and then rendering resulted in high cost and high accuracy errors in constructing simulation data.
Now, Huawei has used the Pangu model to transform modeling and simulation into AI generation.
It helps autonomous driving get rid of the dependence on road tests, collects data and does not necessarily have to be on the road, and reduces the cost of autonomous driving training.
Edge scenarios can be generated and reproducible, accelerating the endgame of autonomous driving.
Previously, startups such as Wayve and Waabi have attracted attention because of their related capabilities, and giants such as NVIDIA have invested.
Now that Huawei has officially announced its entry, it is likely to push this track to a new climax.
Autonomous driving is just the tip of the iceberg of Pangu's large-scale model to reshape thousands of industries.
The Pangu model is just one of the main dishes of the HDC 2024 science and technology feast.
HDC 2024, reshape, upgrade and leapfrog
In addition to autonomous driving, Huawei also mentioned the empowerment of the Pangu model for all walks of life, including industrial design, architectural design, embodied intelligence, and automotive design.
Huawei also specifically mentioned that the Pangu model is good for the styling design of the new car.
The Pangu model can greatly shorten the original design cycle of 1-2 years, and the stylist can use his own inspiration to interact with the large model through dialogue and drawing to generate a 3D digital model.
Stylization adjustments are also available, as well as component editing, color changes, and more.
The generated digital model can also be directly exported to 3D files, supporting more than 10 mainstream formats.
Designers can directly 3D print samples, reducing the number of rounds of making clay models, which greatly saves cost and time.
In addition, in fact, the visual ability of the Pangu model has been used to detect defects in auto parts for a long time.
The Pangu model was upgraded to the bombing field, and the official upgrade was also 5G technology.
Yu Chengdong announced at the scene that 5.5G has lower latency and more connections, and this year will be the first year of 5.5G commercialization.
In Huawei's official warm-up video, the latest model Pura 70 has appeared 5.5G network logo, which means self-explanatory.
Let's continue to talk about the impact of new upgrades on smart cars and autonomous driving.
According to public reports, in fact, since last year, some car companies, such as Great Wall, have been experimenting with "5G-A (5.5G) ultra-reliable and low-latency automotive flexible production lines" to try to improve production efficiency.
In the field of autonomous driving, the ultra-massive data generated by unmanned vehicles every day can also be transmitted back to the background computing platform in real time with the help of 5.5G to speed up algorithm training.
Compared with the Pangu model and 5.5G, perhaps for ordinary users, the most concerned and the most direct feeling is still "pure-blooded Hongmeng", that is, Harmony Next.
The new generation of HarmonyOS is not only independent of kernel, database and file system, but also supporting compiler, programming language, development IDE and framework.
From the core hardware at the bottom to the front-end of user interaction, each development link is autonomous, and the ecosystem supported by 900 million devices has shaken the pattern of the two strong operating systems.
However, at the conference site, when it comes to the upgrade of Harmony Next, Huawei will introduce more about the impact on terminals.
But it can also be inferred from it that in the future, the fluency of Hongmeng car machine will continue to improve, and there will be new imagination space for car machine interaction.
and Huawei's practice in autonomous driving, which has been fed back to the terminal.
At the conference, Yu Chengdong used the concept of L1-L5 to describe the reasoning and planning ability of voice assistant Xiaoyi after the upgrade of the "pure-blood" Hongmeng.
He also said excitedly at the scene:
Today, Hongmeng has set a general trend.
No one can extinguish the starry sky.
In the latter sentence, Yu Chengdong said at the Huawei Developer Conference 4 years ago:
No one can extinguish the starry sky, and the spark can start a prairie fire.
Nowadays, we see that the Pangu model is reshaping thousands of industries, 5G technology continues to upgrade, and the HarmonyOS ecosystem is evolving by leaps and bounds.
The spark 4 years ago has really become a prairie fire and converged into a bright galaxy.
The wind and snow have been pouring for four or five years, and the light boat has passed over the ten thousand mountains.
After Wanzhong Mountain, continue to lead.