【CSDN Editor's Note】The popularity of the autonomous driving battlefield has attracted countless developers and practitioners to integrate into it. However, there is a big difference with the traditional Internet, autonomous driving is the product of the deep integration of the automobile industry with artificial intelligence, Internet of Things, cloud computing and other new generation of information technology, so when many developers first enter it or even wait and see, the dazzling emerging technologies, lack of underlying tools, complicated knowledge points and other pain points have to make it bow to reality and stop at the moment.
In "New Programmer 004", we invited a senior technical person rooted in the Internet for 20 years, based on his personal experience in transforming from the Internet backend to the research and development of autonomous vehicle infrastructure, and deeply shared the software and hardware, entry and growth experience of transformation to automatic driving, and also hoped that this article can be beneficial to more peers.
Author | Karin Immortals (pen name)
Produced by | Editorial board of New Programmer
As an Internet programmer who entered the IT industry in 2000 and has been engaged in back-end research and development for more than a decade, I carefully considered five years ago and turned to the field of self-driving car infrastructure research and development, and decided to start over in a new industry and a new technical direction.
When I went from facing terminals and browsers every day to dealing with various embedded development boards and various types of sensors, often needing to debug on the road, and finally seeing a car driving autonomously according to code expectations, I can really feel a completely different experience and sense of accomplishment from my previous work.
Since entering the autonomous driving industry, I have participated in the development of mass production models equipped with advanced driver assistance systems (ADAS) on the market, here, I also hope to summarize the experience of the past few years, to the interested participation in automatic driving research and development, but still waiting and watching hesitant peers to refer to, to help you find the right positioning for you faster.
Introduction to autonomous driving technology, must know the division of labor and job functions
The operating environment of the vehicle's autonomous driving system is very different from that of Internet services, which is neither as rich as the system resources of server clusters, nor as diverse as the development tools of browsers and mobile terminals.
The automatic driving system needs to work on the existing data and control interface of the vehicle, first read the various inputs of the body and sensors, after the system processing, and then converted into the longitudinal (acceleration and deceleration) and lateral (steering) control instructions for the chassis control interface output, and finally the vehicle executes these instructions to complete the action.
In bottom-up order, self-driving cars are first broadly divided into hardware directions and software directions.
A hardware platform for autonomous driving
Vehicle platform
For the controller and software system of the vehicle's own body, cockpit and chassis, the current automotive industry has a very mature ECU (Electronic Control Unit) controller industry chain and automotive open system architecture (AUTOSAR) software system.
They are responsible for processing the underlying control logic of the vehicle, providing data and control interfaces for the upper-level autonomous driving system, and are the basis for the autonomous driving system to be able to control the vehicle.
Autonomous driving hardware platform
The computing power demand of the autonomous driving system is much higher than that of the low-power chips commonly used in the automotive industry, and a dedicated high-computing power and low-power embedded computing platform is required. At present, the industry generally uses 64-bit ARM architecture chips (aarch64), automatic driving also relies on various types of cameras and radar sensors in various frequency bands, requiring professionals to be responsible for the drive adaptation of various hardware and sensors, as well as gpus, FPGAs, DSPs and other special computing acceleration equipment operator development, this part of the work is responsible for embedded hardware engineers.
Because the industry background knowledge is relatively different, programmers with Internet backgrounds usually do not choose to participate in hardware direction work, so we need to pay attention to software direction positions.
Software platform for autonomous driving
Vehicle-side infrastructure
In a broad sense, the autonomous vehicle infrastructure covers all parts of "hardware-driven and algorithm application", and the vehicle-side infrastructure runs through the hardware layer and the application layer, playing a role in carrying forward the upper and lower levels.
Vehicle infrastructure engineers often need to solve complex problems with mixed hardware and software factors and difficult to reproduce, and the recruitment requirements for a similar position in a friendly business are very straightforward: "Use your expertise to debug hard-to-reproduce whole-system problems." It is more suitable for Internet back-end programmers who are familiar with the C++ technology stack, and based on the recruitment experience in recent years, I think C++ engineers in the communications industry are also generally more suitable.
The vehicle-side infrastructure mainly includes the following directions:
Middleware: Provide basic services such as communication and task management for upper-level applications, while ensuring the unification of interfaces between different models, sensors and operating systems.
Toolchain: Data acquisition, playback and debugging toolsets supported by car-side middleware and cloud data platforms to develop multipliers for iterative efficiency and problem location efficiency.
Operating system: At present, the automotive autonomous driving domain is dominated by commercial real-time operating systems and customized versions of Linux, and it is expected that there will be a dedicated automatic driving operating system in the future.
End-of-the-car algorithms and applications
According to the current industry consensus, the autonomous driving car-side algorithm is generally divided into perception, positioning, decision-making, planning, control and other directions, which is more suitable for engineers with the background of algorithm strategies in the Internet industry.
In layman's terms, each algorithm direction solves the following problems:
Perception: Based on the input of cameras and various radars in all directions, determine what you are seeing and how far away.
Positioning: Based on satellite positioning, inertial navigation, HIGH-precision maps and environmental characteristics, determine which lane of which road you are on.
Decision-making and planning: where you should go and how to get there safely.
Control: Controls vehicle steering and acceleration and deceleration while ensuring the passenger experience.
Cloud services
In addition to the car side, the importance of the autonomous driving cloud business is also very high, and the perfection and maturity of the cloud system are the key factors that directly affect the iterative efficiency of the car side system. Because it is not limited by the hardware environment of the car, it can directly follow the mature Internet technology stack, so the automatic driving cloud is the most suitable technical direction for Internet programmers to transform automatic driving. Cloud services have the following services:
Data platform: A very large amount of data will be generated during the development and operation of automatic driving, which is the key factor to ensure R&D iteration and troubleshooting, and together with the vehicle tool chain, it constitutes the core infrastructure.
Virtual simulation and automated testing: On-the-road testing is expensive, and virtual simulation scenarios can be run in parallel in large numbers, which can improve test efficiency and reduce test costs.
OTA remote update: The security and reliability requirements of the vehicle for the remote update software are much higher than those of the mobile phone, and the robustness of the remote update service is also put forward higher.
Remote monitoring: Remote monitoring and takeover based on 5G infrastructure is a relatively new technology direction, and may be a key factor in the landing of Robotaxi (self-driving taxi) services.
Smart Cockpit
Although the intelligent car cockpit direction does not participate in vehicle control, it assumes the responsibility of bridging the gap between the user and the automatic driving system, which is directly related to the whole experience of the vehicle passengers.
At present, the cockpit generally uses large-screen car machines and voice control based on the Android system as the main human-machine interface, so it is more suitable for mobile or front-end developers in the Internet industry to consider.
HD map
It is expected that in the future for a long time, high-precision maps are a key dependence of the automatic driving system, high-precision map business includes data collection and processing, cloud services, car-side services, etc., itself has covered all aspects of the technical links from the car to the cloud, if you have a GIS (Geographic Information System) professional background, or Internet map service project experience, it is more suitable for high-precision map direction positions.
In addition to the above mentioned these, automatic driving also has recruitment needs for R&D supporting positions such as information security, configuration management, system integration, and software testing, and it is recommended that interested peers can keep an eye on it.
Characteristics and differences of autonomous driving systems
Obviously, autonomous driving systems have higher requirements for the stability, safety and performance of the code. In addition to this, you need to adapt to these differences:
The difference with the Internet
As an Internet programmer, the daily working environment is a back-end server or a client browser, and the development, testing and operation environment is very mature, stable, and more and more standardized. As a developer, you only need to pay attention to the operating system interface level, and generally do not need to go further into the hardware level.
When engaged in the development of automatic driving systems, the software system and the vehicle platform, vehicle-side hardware and sensors are closely coupled, all characteristics must be coordinated by the software and hardware system to operate normally, and the automatic driving industry is in a stage of rapid growth of drastic evolution, models and sensor models are numerous, technical standards and interfaces are changeable, so we need to have a certain understanding of all levels and links of knowledge from the bottom up, from hardware to the system, It is best to have the opportunity to personally participate in the whole process of hardware configuration and software deployment and debugging, in order to deepen the overall understanding and avoid blind people touching the elephant.
In my personal opinion, if you have a certain number of years of driving experience, it will be more helpful to understand the user's concerns and demands from the driver's point of view.
Vehicle rules and functional safety
The new terms that every Internet programmer new to the autonomous driving industry pays the most attention to are necessarily "car rules" and "functional safety".
Briefly speaking, vehicle regulation is a series of industry standards and industry specifications accumulated by the automotive industry over the years, from code specifications to development processes, and Functional Safety is a technical term that refers specifically to a set of research and development process methodologies in the automotive industry to avoid casualties and losses.
Among them, the most closely related to autonomous driving systems are the international standards ISO26262 - Functional Safety of Road Vehicles, the C/C++ code specification MISRA C and MISRA C++ (Motor Industry Software Reliability Association). Because the latest version of the MISRA C++ specification was developed in 2008 and cannot cover the dramatic changes in language standards since C++11, you can also choose to refer to AUTOSAR C++14, a code specification that supports higher language standards.
ISO26262 V R&D process model
If the functional safety standards and methodology are followed, the final result is the vehicle safety integrity level (ASIL, Automotive Safety Integrity Level) of the software, which is divided into FOUR levels: ASIL-A, ASIL-B, ASIL-C, ANDIL-D, and the safety level is gradually increased, and if the ASIL-D is passed, it means that the highest level of functional safety is met. For example, some components of the current functionally safe version of the microkernel real-time operating system QNX OS for Safety (system kernel, part system services, C compiler and standard library) have passed the ASIL-D standard certification.
Here are a few requirements from the ASIL-D standard, and I believe that industry practitioners can appreciate the complexity and workload of passing ASIL-D certification:
Formal validation
Restrict the use of pointers
There is no hidden data flow or control flow
The branch coverage, function coverage, and call coverage of the test require 100%
Software functional safety is a more professional topic, usually the autonomous driving team will be equipped with professional functional safety engineers, developers need to do the main thing to understand the standard outline and familiar with the supporting tools, follow the requirements and suggestions of professionals. In addition, because the cost of passing the highest level of certification is quite high, in practice, the applicable functional safety level is generally selected according to the needs of the project and for different modules.
Real-time
In the Internet industry, real-time systems generally refer to systems that process or respond quickly enough, and do not have clearly defined and quantitative indicators.
In the autonomous driving industry, real-time (real-time) refers to the ability of the system to complete the operation on time without interference from external factors such as task scheduling and resource scrambling within the promised time window or scheduling period. I personally think a more appropriate description would be "certainty".
For example, a task module designed to output the result of the operation at 20Hz (that is, 20 times per second) can accurately achieve that the interval between each output is 50 milliseconds, which meets the real-time index of the design requirements. However, if the output spacing is uneven or significantly jittery, even if the output frequency of 30Hz or even higher can be achieved, it is not satisfied with the real-time performance index.
To improve the real-time nature of software systems, first use an operating system that supports real-time scheduling policies, such as BlackBerry QNX or Wind River VxWorks, or at least a custom version of Linux with a kernel real-time patch upgraded. Secondly, the application layer code should pre-allocate various resources through the pooling strategy to avoid jitter caused by dynamic resource allocation and cache invalidation at runtime, scientifically provision the affinity and priority of each computing task, and reduce the frequency of being switched by the operating system task scheduling service.
When many Internet programmers first entered the autonomous driving industry, they may have heard rumors that "the vehicle specification code does not allow dynamic application for memory at runtime", which is actually derived from real-time requirements. All new/delete and malloc/free calls, including the create smart pointer object operation after C++11, are flagged as violating the MISRA C/C++ code specification when the code is statically scanned.
In addition to functional safety and real-time, autonomous driving is also gradually learning from the mature experience of the industry in the field of traditional high-reliability software such as aerospace.
Study recommendations and recommended bibliographies
At present, the industry already has high-quality open source projects such as ROS2 and Apollo, and the project documents are rich enough that basic peers can refer to and learn about the technical details:
ROS2：https://docs.ros.org/
Apollo：https://github.com/ApolloAuto/apollo
The following books introduce autonomous driving from the perspective of industry background and technical overview, which is more suitable for beginners:
Driverless: https://book.douban.com/subject/27058912/
The Age of Automation: Driverless Reshaping the World: https://book.douban.com/subject/35101057/
The First Book of Driverless Technology: https://book.douban.com/subject/27038410/
Introduction to Driverless Cars: https://book.douban.com/subject/26629406/
The automotive industry has a hundred years of history, and autonomous driving is a new direction, so if you have just entered the industry, you will inevitably hear new terms every day and see new abbreviations every day.
In the first few months, it is necessary to make full use of the rich information resources of the Internet and supplement industry knowledge with high intensity. To be diligent in searching, asking questions, and writing code, autonomous driving is a discipline that deeply combines software and hardware, code and experience, so it must be added: diligently get on the car.
Message
As a newly born technology, no one can know in advance what autopilot should do, and everyone is exploring from scratch at the same starting line.
I remember five years ago, I was still hesitating whether to transform from the Internet to autonomous driving, worried about whether I could start from scratch in a new direction, when an industry senior once enthusiastically called: "Don't worry about what you don't know, we all start from nothing to do autonomous driving." ”
In addition, as a person who has experienced many ups and downs in the Internet industry, we must also remind everyone that although the current automatic driving is on fire, there are more opportunities for money, and it is still necessary to be psychologically prepared for cyclical fluctuations. I just transformed into autonomous driving after encountering two years of industry troughs, if you just to invest in the next outlet, then these two years will inevitably be very painful, even regret not the beginning. But if you, like me, figure out what you want, then all the difficulties are no longer obstacles, but tests before reaching the goal.
About the author: Kalin Xianren (pen name), engaged in the research and development of Internet back-end technology from 2000 to 2017, engaged in the research and development of autonomous vehicle infrastructure since 2017, and has worked in Companies such as Sina and Baidu.
—END—
This article is the content of "New Programmer 004", "New Programmer 004" is about to be released, so stay tuned.
![Goodbye, old system! | Geek time[fig]](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=)