From the advent of ChatGPT in 2023 to the birth of the Sora model this year, end-to-end large models have shown strong potential in many fields, including in the automotive industry.
What is "end-to-end"?
In the field of deep learning, "end-to-end" usually refers to an AI model that directly outputs the final result by simply inputting raw data. By training on a large amount of high-quality data, the end-to-end large model can gradually improve its intelligence. In the automotive industry, this technology can be applied to the field of autonomous driving, thus replacing traditional autonomous driving technology.
Traditional modular autonomous driving systems usually divide perception, decision-making, and control into independent modules, each module focuses on solving specific problems, which simplifies the difficulty of system development and facilitates problem backtracking and R&D iteration, which is the current mainstream solution.
However, the drawback of this approach is that the manually programmed code can only handle a limited number of driving scenarios. No matter how many lines of code you add to the system, it will not cover all special cases, making it difficult to achieve full autonomous driving.
In contrast, the end-to-end large model is a monolithic model, closer to the model of human driving. Instead of trying to process each driving scenario through manual programming, it uses a large amount of data for training, allowing the AI to discover the driving rules hidden in the data on its own, so that it can cover a wider range of driving scenarios and is expected to achieve full autonomous driving.
However, today's large end-to-end models have incomprehensible flaws. Even after a lot of training, large models can still become "artificially retarded" in some scenarios, taking decisions far worse than those of humans. At the same time, the decision-making process of the end-to-end large model has the characteristics of "black box", the internal logic is not public, and the problems in the decision-making are difficult to locate, which has a negative impact on R&D iteration and problem solving.
In addition, the amount of data, computing power, and the streamlining and optimization of large models are all important factors to promote the development of end-to-end large models.
First, autonomous driving systems require large amounts of high-quality training data. This data includes images, videos, and sensor data for various driving scenarios, weather conditions, and traffic conditions. Collecting, annotating, and maintaining the quality and diversity of this data is a challenge, especially to ensure that the data covers all possible driving scenarios. Musk has said: "It took Tesla about a quarter to complete the training of 10 million video clips. After training 1 million video cases, it barely works, 2 million is slightly better, 3 million is Wow, and at 10 million it becomes incredible. ”
Second, computing power is an indispensable resource when feeding massive amounts of data to end-to-end large models. The Intelligent Computing Center needs to be expanded to meet the growing demand for computing power.
Finally, when the large model in the cloud is trained, it needs to be streamlined. Cloud servers have a large number of high-performance hardware resources, which support large-scale parallel processing and data storage. However, in-vehicle computing resources are limited, and in order to adapt to them, the model needs to be optimized and energy consumption reduced.
Tesla's end-to-end large model and computing power deployment
Tesla is leading the way in the field of end-to-end large models. In August 2023, Musk showed the beta version of FSD V12 in a live broadcast, repeatedly emphasizing that the version uses massive video data for training, and driving decisions are generated by AI algorithms. He also tweeted on X that the V12 beta version had drastically reduced the manually programmed C++ control code by two orders of magnitude (to 3,000) from 300,000 lines.
In March this year, Tesla's non-employee users in North America received the FSD V12.3 version push. This version is better at dealing with complex scenarios such as obstacles and lane change games, but when dealing with some simple scenarios, such as driving on open roads, there will be outrageous acceleration or deceleration issues.
In terms of computing power planning, Tesla deployed more than 10EFLOPS of computing power last year and is expected to reach 100EFLOPS by the end of this year.
End-to-end large-scale model and computing power deployment of new EV manufacturers
New domestic car-making forces have followed up with FSD V12, but they have not yet achieved mass production on the car. At the national intelligent driving launch conference in January this year, He Xiaopeng said that Xiaopeng Motors will realize the full launch of the end-to-end large model. It is reported that the ideal new model will also be launched this year, and NIO will launch end-to-end based active security functions within the year.
At the same time, all three have a layout in terms of intelligent computing centers.
Xpeng and Alibaba Cloud jointly built the Fuyao Intelligent Computing Center, with a training power of 600 PFLOPS (0.6 EFLOPS).
The intelligent computing center jointly built by Li Auto and Volcano Engine has a training computing power of 1200PFLOPS (1.2EFLOPS).
NIO has integrated the technical resources of Alibaba Cloud, NVIDIA and other partners to build the NIO Cloud Intelligent Computing Center, with a computing power of 1400PFLOPS (1.4EFLOPS).
epilogue
The end-to-end model of the car shows good potential, but it is not yet mature and requires a security strategy. But don't worry, it's just getting started. With the continuous optimization of algorithms and hardware, and the use of more data for deep learning, end-to-end large models will gradually be improved and widely used in autonomous driving.