Since 2018, Unity China has been deeply engaged in the field of intelligent cockpits and has launched a number of innovative solutions. Today, Unity China's solutions have been adopted by nearly 30 OEMs and have enabled more than 60 production models. Among them, behind all the models on sale in NIO, its intelligent cockpit is empowered by the Unity engine.
During the Beijing Auto Show, Unity China invited Lei Jinliang, Senior Director of NIO's Infotainment Software Department and Senior Product Experience Expert, to share NIO's development experience with the audience. In the interview, Mr. Lei said, "OEMs are still very short of Unity development and design talents, and the integration and interaction of 3D and 2D is a major trend, and we hope that more 3D talents will join in." ”
So what kind of developer is NIO looking forward to, Mr. Lei's interview record gives you direction~
Lei Jinliang
Senior Director of Infotainment Software Department of NIO, Senior Product Experience Expert
Wang Chen: Hello everyone, I am very happy to have our old friend, Lei Jinliang, Senior Director of NIO's Infotainment Software Department and Senior Product Experience Expert. The Unity engine is widely used in all NIO models on sale, and we very much hope that OEMs like NIO, which have their own design and R&D teams, can unleash the potential of Unity as creators and developers. I also want to know what kind of new products NIO will release at the Beijing Auto Show this time?
Lei Jinliang: At this year's Beijing Auto Show, we participated in the exhibition with the theme of "Lead the Change", and for the first time, NIO's booth showed the public a full-stack layout of 12 technologies, including smart energy, smart manufacturing, chips, and on-board intelligent hardware, as well as NIO's battery swap station and 640kW fully liquid-cooled ultra-fast charging pile. Eight of the NIO brand's 2024 models were unveiled for the first time at the show, and of course, the most important is our 2024 ET7 smart electric high-end executive sedan.
Wang Chen: Every time I go to NIO's NIO House, it gives people a sense of home, allowing users to take a break and get together while looking at the car. If you travel a lot, you can also see a NIO House at Shanghai's Hongqiao Airport, which is equivalent to a VIP lounge exclusive to NIO users.
Lei Jinliang: Yes, we have an exclusive lounge area for NIO House near Gate 56 of Terminal 2 of Hongqiao Airport, and NIO has once again brought NIO House to the site at this auto show, bringing a rich experience to the audience with more than 300 user volunteers through the NIO Café and the user area on the second floor of the booth. BLUE SKY LAB products are made from sustainable materials left over from automotive manufacturing to the public, fulfilling the vision of clear skies.
Everyone is welcome to visit the NIO booth.
Wang Chen: I think the influence of NIO's user community in the industry is obvious to all. I think the cooperation between Unity China and NIO should be nearly 3 years old. Lei Jinliang: Yes, remember that we have been working together since 2021. Wang Chen: I was also an employee of NIO before, and I was a product manager in charge of vehicle control software at that time. Although now Party A jumped Party B and left NIO. However, my cooperation with NIO in Unity China has not stopped, including NIO's subsequent brands and models, and I am very much looking forward to it! Through this conversation, we would like to let the audience and our developers know what kind of blessings do you think NIO's 3D HMI technology, or Unity's technology, can bring to our intelligent cockpit operating system? LEI JINLIANG: IN FACT, IN THE DIGITAL COCKPIT, 3D AND 2D FUSION INTERACTION IS A VERY BIG TREND, IT SHOULD BE SAID THAT THERE ARE MORE AND MORE EXPERIENCES USING 3D HMI TECHNOLOGY, SUCH AS AD HMI & ESD, VEHICLE HEALTH, AIR CONDITIONING COMFORT PANEL, 3D MAP, HANDCAR INTERCONNECTION, VARIOUS SCENE MODES AND MOBILE APPS (NIO APP), ETC. 3D HMI can give users a very good intuitive interactive experience, which not only enhances the immersion of human-computer interaction, the sense of technology, but also improves the interaction efficiency.
Wang Chen: We also very much agree that in order to facilitate user control, we will also combine 2D UI, voice interaction (VUI) and physical buttons (TUI) when making 3D HMI. On the basis of taking into account both efficiency and ease of use, we use Unity's technology to innovate, such as the comfort panel we used to do 3D car control, which can integrate our seat comfort (ventilation, heating, massage), air conditioning, and fragrance through functions such as one-shot to the end and silky transition.
Lei Jinliang: Yes, 3D HMI technology will indeed make the vehicle control experience very extreme, of course, this requires the atomic capacity building of the whole vehicle, in fact, NIO users can already experience it on the car machine; We are also continuing to improve the atomization capability of the whole vehicle, including the atomization capability of the exterior and the atomization of the interior, which can be expressed very well through 3D HMI; For example, on May 15, NIO will release the Ledao brand for the mainstream household market, and the first model will define a new standard for pure electric family SUVs, and I believe you will also see more extreme 3D HMI experiences, so stay tuned!
Wang Chen: That's right, I'm also looking forward to Ledao's models, and I wish NIO and Ledao great sales! We all know that NIO has experience with multiple platforms, multiple models, and multiple languages, do you think you have any experience in using Unity for multi-platform and multi-vehicle adaptation?
Lei Jinliang: How to improve the efficiency of multi-platform and multi-model development is very important, that is, whether a development can be reused as much as possible for multiple platforms and models, thanks to Unity's mature editor, rich plug-ins, good cross-system support capabilities, mature resource loading capabilities, and active communities, 3D HMI development can be better migrated to multiple models and platforms. In addition, Unity has performed well in improving development efficiency and reducing maintenance costs, and has greatly optimized the startup speed and stability.
Wang Chen: Actually, I think we have only played about 40% of the potential of 3D HMI in the industry at present. In fact, there is more space in the back, such as the game cockpit, XR cockpit, AR HUD, seamless experience of tablet-phone-car machine, more advanced art effects, etc., we still have a lot to look forward to, and I think there are many directions that can be explored in the future cooperation.
Lei Jinliang: I just watched your latest demo, I have to ask you about this, if we say that we have multiple 3D applications at the same time, I feel that this is still a large consumption of system resources, how does Unity solve this problem?
Wang Chen: That's right, because if we only do application-level innovation now, we will inevitably encounter bottlenecks or ceilings, for example, if five 3D applications are opened at the same time, then five different Unity runtimes are called at the same time, which is obviously unreasonable. It doesn't matter in the game, because you only play one game at a time, but in the car it has a lot of safety, comfort, and even safety regulations, so that we can't take up more resources, which is why Unity China's teams in Shanghai and Guangzhou have teams dedicated to engine technology research and development.
Engine technology includes architectural innovations, and our innovative Unity Engine URAS Service (Unity Render as Service) does this, where we use a single Unity engine to drive multiple different 3D applications at the same time. This architecture solves the problem of whether our CPU, memory, and turn-on time can be optimized when there are multiple 3D applications running in our car at the same time, and I think the future development of this technology is very worth looking forward to. In fact, designers and product managers have a lot of ideas. We want to say that these features that are valuable to users are not limited by technology.
Lei Jinliang: That's right. This is still quite expected, and this is indeed some of the problems that plague us.
Wang Chen: Some time ago, I saw our NOMI GPT, which can already use functions such as car Q&A, AI car control, AI encyclopedia Q&A, and atmosphere magic, such as asking NOMI to help me generate an atmosphere light color of the sea. I would like to ask Mr. Lei, how big is the impact of AI technology on our intelligent cockpit?
Lei Jinliang: AI is indeed very popular, but I think the impact of AI on the cockpit may be continuous and all-round, but it is also gradual, because it always has to be combined with user scenarios, that is, the essence of what kind of incremental user value we have created, such as how much efficiency has been improved, how much user costs have been reduced, what is the improvement of experience, and it can also be said that AI is a means, and the purpose is to help solve user problems.
To give you a few examples, for example, we recently released NOMI GPT, and there are a lot of new experiences for generative AI applications, including the ambient light color that you just mentioned about generating a sea; For another example, NIO has always had a concept, that is, we say to create a pleasant lifestyle, we believe that the car is not only a means of travel, but also a mobile living space, this auto show, we showed a cockpit function, that is, microphone-free karaoke, that is, users do not need to hold a microphone, microphone-free karaoke wants to sing, this is the mobile karaoke room; Through full-stack self-developed AI filters, AI intelligent sound effects and other technologies, we have brought users the ultimate experience beyond expectations, I don't know if you have experienced it.
Wang Chen: I have experienced it, and I have also seen Mr. Wang Feng's in-car karaoke video, with the blessing of this technology, it feels like entering a concert.
Lei Jinliang: Yes, with the help of AI technology, it does bring users a different experience, for example, AI filter technology, when some tunes are difficult to sing, AI will help us pick them up, so that it sounds good; We hope that NIO's users will "everyone is a strong singer" and make everyone fall in love with singing, so the impact of AI technology on the cockpit is indeed very large.
I think in the next few years, there may be more and more AI capabilities, it may not only be presented in the form of Wensheng pictures and videos, it may be just combined with our usual intelligent driving, voice interaction experience, Face ID or gesture recognition, mobile living space, etc., if you are interested in NIO, you can experience our new technology.
Wang Chen: We feel that the hardware is now full, and the hardware actually determines the upper limit of our experience. However, how to use appropriate software and algorithms to achieve the upper limit of these hardware is a topic worth exploring, which is why our product manager team, including designers and development students, has silently paid a lot.
Thank you very much Mr. Lei for sharing today, the last question is about innovation, because I have been doing cockpit innovation, I think a good innovation direction, which can directly affect the user's car experience. I would like to ask how to create differentiation in the current intelligent cockpit, so involution? What does Mr. Lei think?
Lei Jinliang: This topic is actually quite big, let's talk about a few feelings, talk about differentiated innovation, I think the first thing is to return to the insight and identification of user needs, that is, what is Wants that we often say internally? What are Needs?
For example, when we talk about cars, some people say that we want a smart electric car with 150 kWh, which is Wants, but when it really lands, you will find that there are more users who choose to buy 75 kWh. After careful analysis, you will find that in fact, his real demand may not be a 150-degree battery pack, his need is a better battery life or energy replenishment experience, this is Needs, when you get this point, the differentiated innovation focuses on how to solve the problem of energy replenishment, so that the energy replenishment experience becomes better, so our 3-minute battery replacement scheme will come out, and the same is true for the intelligent cockpit, so the insight into user needs is very important, and no matter how much effort is in the wrong direction, it will be in vain.
Secondly, pay attention to listening to users, run in small steps, and constantly iterate to polish the experience. We have a very complete user voice research and feedback mechanism, every day we are collecting a large number of user feedback and user voices, and then we spend a lot of time to do data analysis, through the high attention to user voices, we have made a lot of industry premiere experiences, such as NOMI, quick scenes, microphone-free karaoke, etc.; At the same time, once the direction of the experience is determined, we will put user satisfaction as an extremely important North Star indicator; Of course, it is difficult to do innovation right the first time, we will adopt the method of running in small steps, starting from a thorough point, and polishing the experience through continuous iteration.
Finally, let's talk about the whole experience that exceeded expectations. We summarize the whole user experience into 14 touchpoints, in general, each contact point may be responsible for different departments, and often cross-departmental, cross-touch points can produce a full experience beyond user expectations, this is actually an opportunity for differentiated innovation, experience managers from the whole experience of closed-loop thinking, and through the team model to promote the landing, this is actually a very unique place in our corporate culture, we call experience managers, different from the traditional sense of product managers, The experience manager will pull through each experience domain from the dimension of the whole experience, which makes it easier to generate differentiated and innovative experiences.
Wang Chen: Yes, first of all, we need to understand the needs of users, not their requirements. In fact, the user's requirements are actually a solution on the surface, but in fact, its essential requirements may be completed in a more efficient way. At the same time, our kind of innovation, such as systematic innovation, as well as system-level innovation, relies on the cross-domain cooperation of multiple teams.
In fact, like the car control team, when I went back, I heard that there were already 100 people, in fact, it should have different development classmates, and at the same time in different teams, so that it can use a lot of interoperable information on the platform, which is also a very efficient experience.
In fact, after I came to Unity, one of the biggest impressions was that a lot of technology has been used in the original game, no wonder it's called the Ninth Art, although I don't play many games myself, but I started playing after coming to Unity, and it's also very good to see if there are any benefits.
Thank you very much Mr. Lei for sharing, and finally we have a small link, which is to help millions of our Unity developers in China to ask, if I am a development engineer or 3D artist with no experience in the automotive field, but I want to go to the OEM to develop and do the functions of intelligent cabin and intelligent driving, do you have any comments and suggestions for these Unity partners?
Lei Jinliang: First of all, we are still very short of Unity development and design talents in OEMs, and I remember that building a 3D team a few years ago was a bit individual, and it took a long time to find the right talent. Of course, it's still a very good window of opportunity, and I just mentioned that the convergence of 3D and 2D is a big trend, and I hope that more 3D talents will join in.
The second point is that for developers with certain Unity development experience, some of the development capabilities done in the past can be completely reused, and most of the car machine systems are still based on the Android system framework.
Wang Chen: Thank you Mr. Lei for sharing. If you are interested in Unity, you are very welcome to submit your resume to the official website of NIO, and maybe one day you will become a colleague of Mr. Lei.
Lei Jinliang: Very welcome.