AI fusion
Lenovo Ventures and Pencil Road jointly launched the "AI Fusion" column, focusing on new insights, new trends, and new opportunities in the AI era.
10 years before the release of the sci-fi blockbuster "Robot Story", a robot appeared in the Hospital of the Chinese University of Hong Kong. It's an "errand runner" who helps nurses carry samples from the ward to the lab that need to be tested. Along the way, there are always people who can't help but touch it or even let it go.
This humble "Wall-E" was hammered out by Liu Yunhui, a professor in the Department of Mechanical and Automation Engineering at the university, with his students, and died after only three months of work due to lack of maintenance funds, but Liu Yunhui faintly realized that the end of the technology was no longer a few laboratories.
At that time, the Chinese University of Hong Kong, under the impetus of the old president and the father of optical fiber, Charles Kao, developed engineering education that kept up with the future wave of science and technology, recruited top engineering talents from all over the world, and pursued the concept of "becoming a source of knowledge power for Hong Kong and serving the society". In 1995, Liu Yunhui resigned from the National Institute of Electronics Technology and joined the Chinese University of Hong Kong to engage in the research of robot control theory.
More than 20 years later, China's installed industrial robot capacity has exceeded 50% of the global total, becoming the largest industrial robot market. Liu Yunhui led the students to serve the industry with research, and founded a number of robot companies, including VisionNav Robotics, Connosten, Zhucheng, etc., to open up a "new continent" in the era of robot exploration.
Founded in 2016, VisionNav Robotics is the first robot company co-founded by Liu Yunhui, with unmanned forklifts as its main products, and has received financing from Meituan, ByteDance, Wuyuan Capital, Lenovo Venture Capital, etc.; Connastar is engaged in the research and development of high-end surgical robots, and will receive 800 million yuan of financing from Daohe Technology Investment and Lenovo Venture Capital in 2023; Zhucheng mainly develops construction robots including intelligent spraying robots for exterior walls, and has received Series A financing from XVC, Lenovo Venture Capital and other institutions.
What are the factors behind the birth of star enterprises in batches from Liu Yunhui's laboratory, and can we continue to replicate more Future Robotics, Connaissten, and Zhucheng? What are the opportunities for the robot industry as a new round of artificial intelligence technology revolution sweeping the world? What are the key roles of industrial capital represented by Lenovo Venture Capital in the rise of hard technology start-ups?
Recently, Pencil Road met with Liu Yunhui at the Hong Kong Science Park. In addition to participating in major company decisions, he spends most of his time doing research at the university and serving as the director of the Tianshi Robotics Research Institute of the University of Hong Kong Chinese. He says he enjoys the process of generating ideas more than dealing with the trivial aspects of running a business. "Creativity is what I like the most, combining technology and applications to solve corner cases. ”
01
He is a scientist and an entrepreneur
Pencil Road: When did you move from theoretical research to application of robots?
Liu Yunhui: Around 1997, the dean of the medical school approached me and asked the Prince of Wales Hospital to take a large number of samples from the ward to the laboratory every day, and the nurses were very busy, so could they make a robot to transport the test samples?
The medical school gave about 200,000 yuan, and the students and I developed it for about half a year. At that time, there was no lidar - lidar was very expensive, costing hundreds of thousands - and visual navigation was used, and cameras relied entirely on natural road signs (to distinguish directions). In terms of technology, it is very advanced and should be regarded as one of the earliest applications of visual navigation robots. After three months of use in the hospital, the robot had to be stopped, although it worked well, because there was no one and no follow-up funds to maintain its operation.
At that time, robots were very novel, and many patients would use their hands to block the top (lens) and knock on the touch screen, which was broken several times. But it made me realize that visual navigation can be applied to real-world scenarios. I will continue to do research in the future, hoping to combine visual feedback with robot control.
The school has an "unround lake" with a lot of leaves and garbage on the surface of the lake. Around 2000, the campus management department asked if it was possible to develop robots to clean up fallen leaves and other garbage, and also gave about 200,000 yuan. We researched for a while and found that it was too difficult, and the garbage near the corners of the shore was especially difficult to collect. It makes everyone realize that solving real-world problems is very different from academic research.
Both of these have a lot of industrialization opportunities, and the biggest problem is that venture capital was not active at that time, and there was no money at all. After more than ten years, (service robots, unmanned ships) have all risen. It also showed me that the R&D in the laboratory can be market-oriented.
Liu Yunhui introduces the visual navigation unmanned forklift (Source: Communications and Public Relations Office of Chinese University)
Pencil Road: Why did you think of making unmanned forklifts?
Liu Yunhui: After testing in the hospital, we felt that vision, as the main sensor, was very helpful for robot control and walking. Around 2010, we began to study the robot control technology with vision as the main feedback, put a robot in the room, no GPS signal, no radar signal, let it run in a straight line, circle, the accuracy can be controlled to a very high level. Later, there was more research on autonomous driving in the industry - the principle of autonomous driving in cars is the same as that of laboratory robot navigation, but the difference between high speed and low speed is the same. I feel that the opportunity for low-speed industrial robots may come earlier, and the technical difficulty of autonomous driving in enclosed spaces is lower than that in open spaces.
We have done research, and the AGV (Automated guided vehicle) of automatic forklift is a good opportunity, and the market space is very large. There are millions of forklifts in China, which are basically driven by people, and they will definitely be unmanned in the future.
Moreover, automatic forklifts have technical thresholds, and the core perception and control technology are our strengths. It may be fine if the car is parked by 10 centimeters, but the forklift parking error must be within one or two centimeters. Again, the car weighed two tons, and the person weighed several hundred kilograms when he sat on it. Forklifts can carry up to half or more of their own weight. With different vehicle inertia, it is even more challenging when it comes to control technology. Choose visual navigation, first, there were few people doing it at that time, and second, lidar is very expensive, forklifts need to measure the location of goods with high precision, and the visual measurement accuracy is relatively high.
Pencil Road: How did the unmanned forklift get out of the laboratory?
Liu Yunhui: There is a Hong Kong-funded enterprise ABC in Foshan that makes sanitary napkins, and it needs to move products from the production line to the warehouse, and is willing to try it and open the warehouse to us.
At the beginning, there were many problems, such as visual navigation is greatly affected by natural light, and windows will refract light, causing the forklift to make mistakes in judgment and get stuck in doors or aisles. The navigation and control algorithms need to be constantly adjusted, and one or two engineers collect data analysis every day.
The biggest problem is the control error. Forklifts are often not parked in place, and the original setting error is plus or minus one centimeter, but the actual error is often plus or minus five to ten centimeters. The reasons include visual calibration, forklift parameter calibration, etc., some changes in the physical parameters of each forklift, as well as friction and other factors. The visual scene in the warehouse is much more complex than in the laboratory, and the model and concept are fine, but the forklift is moving somewhere. By repeatedly collecting data, the control algorithm of the system was improved, and the controller was redesigned to improve the accuracy.
It was tested in the ABC warehouse for more than a year, solved technical and engineering problems, and sorted out the product positioning.
The unmanned forklift is commissioned at the ABC warehouse (Source: Communications and Public Relations Office of the Chinese University of Chinese)
Pencil Road: What is your role in founding Future Robotics?
Liu Yunhui: The core of the VisionNav Robotics team, including the CEO, CTO (Li Luyang, Fang Mu) and the main R&D personnel, are all doctoral students who have come out of our laboratory.
At the beginning, the team did not have a lot of experience and resources, and the financing ability was relatively weak. I help them in terms of resource matching, financing, team practice, risk assessment, and general technical direction. The forklift was debugged in the warehouse, and the doctor went to the site, and I followed the guidance and analyzed and discussed together.
Now I mainly grasp the general direction of the company and make suggestions for the development of the company.
Including Connaught (CTO Wang Zerui is Liu Yunhui's doctoral student), Zhucheng, discussed the direction together in the early days, and then there were consultants. The company is run by them, and I spend more time in school.
Pencil Road: At what stage of the enterprise, more hands are left to them?
Liu Yunhui: Generally about two or three years, the company has entered the scale and the technology is reliable. In the first year or two, I still have to participate, and their experience and resources are not enough in all aspects, and they may take some detours.
It's hard to get me fully involved in company management, but my creativity is still on the same frequency as young people. Originally, we mainly wrote papers, but the results of the papers or simply explained the technology clearly (it is not enough), and there are still many places that need to be broken through in the final actual product.
Pencil Road: Why don't you just come out of school and start a business like other scientists?
Liu Yunhui: When VisionNav Robotics was founded, everyone said let me come, and then I thought the CEO (Li Luyang) was very good, so I said you should do it. In 2015, the president gave me two tasks: first, I hope to make the institute internationally renowned, and second, to be able to apply technology to the industry. At that time, we wanted to incubate some companies in these fields, including medical, logistics, and construction. I don't have the skills to do it myself, so I support students or teachers who have the ability to do it (start a company).
Pencil Road: Does managing a company full-time don't match your interests and abilities?
Liu Yunhui: I have a lot of technical skills. Technically help the company, the help will be a little bigger.
If you have a technical idea, you enjoy it very much. In the past few years, I have learned more about the industry, and I found that there are many corner cases in the industry that everyone can't solve. When you go to the industry to do this kind of particularly difficult case, you need to bear the pressure of operation, which will make you impatient enough, and you will not be able to make a really good product.
If you do a lot of exploration in the school first, there is government support behind it, and when all aspects are mature, then integrate innovative technologies and applications to solve these corner cases. I enjoyed the process.
Pencil Road: I don't have much experience in industry with my students, but the business I started is relatively successful.
Liu Yunhui: I think it's luck. There are many things that you can't control if you start a successful business. It is said that a lot of planning should be done in the early stage of entrepreneurship, but in fact, business planning cannot keep up with changes, and I don't know if you can find customers or whether the technology can be implemented.
It also includes the social environment. In the past few years, we have been very fortunate because of the great support from the government and the support from venture capital. The company often encounters (tight funds), but is very fortunate to have the support of investment institutions at critical moments.
Some students have more business ideas and learn quickly, and some teams work well together, CEOs are more business sense, and CTO and R&D are completely technology-oriented, which is a very good match. It also takes luck to meet such a team. We're lucky.
Pencil Road: How does industrial capital like Lenovo Venture Capital help enterprises?
Liu Yunhui: Very big, especially in the early days. A lot of the money in the early days of venture capital was life-saving, and without them, there would be no way for [the company] to do it. In the early days of Lenovo Venture Capital, it was very important to the entire industry and especially hard technology.
Lenovo's factory is also open to us. Industrial capital is more patient, and it will not be said that it will come out in a few years, and industrial capital is very important to the development of the industry.
Many start-up CEOs and CTOs have no experience, and Lenovo Venture Capital organizes some forums, industry banks, and seminars, which are very meaningful to improve their capabilities.
We hope that Lenovo Ventures will invest more in Hong Kong's early-stage science and technology enterprises, combine the resources of the mainland, and the hard technology will definitely go to the Greater Bay Area and the mainland.
02
The study of humanoid robots is important
Pencil Road: What do you think of humanoid robots?
Liu Yunhui: (In the next ten years) the application of robots will definitely develop rapidly, but it does not have to be in the form of a humanoid (it can be discussed), including this thing made by Tesla. But Tesla has a job of operating screws, there is no need to use humanoid robots, and now many things are made into human shapes, and I personally think that there is not much need.
But research is important, and this is mainly reflected in anthropomorphism. Because anthropomorphic problems are very difficult, including things like walking stability control, mechanical design, etc. Every piece of design is a technical challenge, how to coordinate the stability of so many motors? First, this (research) can cultivate a large number of good talents. Second, it may promote the development of some of the following component industries, such as motors, sensors, etc., as well as some cognitive software and hardware. This time (Tesla) is also right to do humanoid research, and we are also laying out, but going to industrialization is not as simple as Musk said.
Pencil Road: The machine is in the form of a human, and the technical requirements are too high?
Liu Yunhui: It's very simple, like autonomous driving, there are only two control quantities: the steering wheel and the accelerator. The car only runs on flat roads, and there are only three degrees of freedom. Autonomous driving has been researched since 2000 years ago, and now more than 20 years later, how many have actually landed? And humanoid robots, twenty or thirty motors, twenty or thirty degrees of freedom. The scene is more complicated, not only walking on the road, but also a lot of bumpy roads in the wild. The difficulty of walking humanoid robots in natural scenarios is many orders of magnitude higher than that of autonomous driving. Unless in the short term hardware technology (there is a revolutionary upgrade).
This is not the same as a computer or a large model. Robots involve hardware, and it is much more difficult to interact with the actual scene than in the virtual world of computers. The research of humanoid robots, (realization) walking and running may be no problem, but there are still many problems (to be solved) if they are really used in the industry, and I don't know if we can find a landing scene. If you really want to do service, there is still a long way to go.
The multi-finger manipulator studied by Liu Yunhui (Source: Communications and Public Relations Office, Chinese University)
Pencil Road: Which form of robot has the potential for a breakthrough in application?
Liu Yunhui: There are still a lot of advantages to the robotic arm, and the robotic arm is combined with some mobile platforms - you don't have to make it into a leg. But the robotic arm is also challenging, like doing the simplest thing at all: it is difficult to grasp an object very accurately in any situation. At an academic symposium in Singapore last September, several experts predicted that grasping anything would be difficult to achieve in the next 10 or 5 years. Although the human arm is small, but the muscles are very strong, you have to make a robotic arm with the same strength to be much larger than the human hand, it is too small, and many things cannot be handled.
It can find (more) landing opportunities in the industry, because the robotic arm was also first implemented in the industry. But now on the production line, the main application of the robotic arm is to take one thing and put it in another.
Pencil Road: It's still clumsy.
Liu Yunhui: If you can improve the intelligence level of the robotic arm and make it more flexible, it will have many application scenarios. It is to put the industrial scene (extended) to the service scene, such as simply cleaning up the dishes in the restaurant, doing some simple kitchen (work), such as cooking noodles, such as some kind of standardized stir-fry.
There is also the elderly, taking medicine is a big problem for the elderly, it is very troublesome to take medicine every day, can you get a robot to help him take medicine? The elderly have bad eyes, it is difficult to break the medicine, can the robot help the elderly to do something very important, but it is not so difficult.
Pencil Road: If the application scenarios of robots are to be expanded, which technologies need to be further developed?
Liu Yunhui: I have been doing research on the fusion control of vision and robot action, and the first thing is to let the robot understand the environment well, and then it can interact effectively. If a person is blind, he cannot do many things. Now 90% of the people working in the factory are blind robots. If we get the robot to open its eyes, a lot of things can be done.
Pencil Road: Many robots are still blind now?
Liu Yunhui: The sensor is difficult, our understanding algorithm is still not good, and the reliability and accuracy cannot meet the requirements. Environmental understanding, etc., relies on algorithms. It is very important to consider how to support visual perception and motion control at the whole system level, and the core algorithms and software used in this are very important.
Pencil Road: Where is the development of artificial intelligence reflected in the improvement of robots?
Liu Yunhui: Interaction. Talking to a service robot, you may have to do program one, two, three, four. Now the big model is coming, you have to ask it to take something, it can do the task decomposition on its own, know what to do in the first step, the second step, the third step, the large model is very suitable for helping the robot to do high-level planning.
However, the application of large models is still very challenging for low-level control problems such as action and movement. There is a pen on the table, and it is easy for a person to pick it up, but it is not so easy for a robot. (A lot of things that seem simple) need to be understood very finely, and very quickly. We humans not only understand, but also move our hands while looking at the pen when we grasp it. To realize these vision-driven motion control on robots, breakthroughs in core technologies are also needed.
Pencil Road: Over the past 20 years, what proportion of the progress of robotics technology has been brought about by artificial intelligence?
Liu Yunhui: It's hard to say this ratio. Depending on how you define artificial intelligence, some computer algorithms, such as perception algorithms, and robot-oriented machine intelligence, are they considered artificial intelligence? The boundary between robots and artificial intelligence is not very clear, and it can be regarded as the best hardware platform for artificial intelligence algorithm testing.
In recent years, key hardware such as motors, sensors, chips, and cameras have developed rapidly, and the volume has become smaller and smaller, the accuracy has become higher and higher, and the cost has also come down, which has promoted the rapid development of robots. For example, SLAM technology (synchronous localization and mapping, mainly used to solve the positioning and mapping problems of robots in unknown environments) has developed rapidly in the past two decades, especially for the application development of mobile robots.
Another example is the robotic arm, from the industrial arm to the current collaborative arm, which has been reduced a lot, and the reliability and accuracy have also been improved a lot, but the application of the robotic arm with real intelligent force control is still very limited.
03
The DNA of innovation: openness
Pencil Road: Hong Kong is not known for its industry and cutting-edge manufacturing, but the research level of machinery, electronics and robotics of CUHK and HKU is among the highest in Asia. What is the reason?
Liu Yunhui: It has to do with the government's support for universities. For more than 20 years, the government has been completely open to basic research, and no one plans what research you are going to do, but what the government has done is to continue to invest in research.
Second, a lot of talent has returned to Hong Kong [from all over the world], including those from the Mainland. Before the 90s, all universities in Hong Kong were completely educational, and there was no research at all. After the 90s, our president (Charles Kao) decided to make it a research university and established many engineering colleges in Chinese University. HKU and HKUST (related faculties) were also established at that time. At that time, our equipment was very good and attracted a lot of excellent students. Hong Kong is still relatively open, and schools are relatively free.
Again, since 2010, the government has vigorously promoted the development of science and technology. If we had made technology a major industry since 2000, there might have been a good opportunity for Hong Kong's technology industry, but unfortunately it was missed. At that time, finance and real estate were too developed, and they were missed for more than ten years. But now there is still an opportunity to do some high-end industries through science and technology, but it is inseparable from some cooperation with the Greater Bay Area and the mainland.
We often compare Hong Kong to Singapore, and I am also an advisor to Singapore's National Robotics Development Program, and I am involved in all project evaluations and have a good understanding of their robotics research and industry. Compared to Hong Kong, Singapore's supply chain is not deep enough. The robot has to do a lot of testing, and if I can't find a suitable factory in Malaysia, I have to go to Suzhou. We went straight to Shenzhen to do it. Hong Kong has a better talent pool than Singapore, which doesn't have as many engineers. We went to Shenzhen to open a company, and we were able to recruit very good engineers. Singapore's high-tech industry is still relatively difficult to compete with Hong Kong.
Pencil Road: What is the relationship between basic scientific research and innovation?
Liu Yunhui: Very important. To do 0 to 1, we must invest in basic scientific research, do not think about whether it can be done at all, and do not think too much about returns. To do basic research, we must cast a wide net. There is a problem in the mainland, where money and technicians are concentrated in a certain area at every turn. Too many resources are concentrated in the so-called big expert team, and it is very important for basic scientific research to let all teachers, young teachers, participate more. Give them independent space to study.
Let young people explore, study what they are interested in, provide a basic research platform, do it slowly, and if someone does it well, don't expect everyone to have potential. There are a lot of things that really can't be planned, research and planning are not much necessary, and technology can't be planned. It's even harder to get creative.
Chinese University Science Museum (Source: Chinese University Communications and Public Relations Office)
Pencil Road: Can the qualities that creativity requires be acquired in education?
Liu Yunhui: Education can help you do something, and students receive more training role. To be a top talent, it is important to do what you do at the PHD stage. I am more free-range with my students, and some of the students with poor abilities find it difficult. I give him some methods, I give him some ideas, but I don't go into great detail to figure them all out. If students have a good understanding, they can learn a lot in the process.
When I was studying for a Ph.D. at the University of Tokyo, my tutor basically didn't care about me, but I talked to him about what I thought. First, he can judge whether the idea is good or not, and second, he creates something with you that can make you think deeply and make you realize that the original good person thinks the way the problem is. To allow students to explore on their own, I also screen students in this way.
Pencil Road: How many of your students start their own businesses?
Liu Yunhui: About 30 percent of them started businesses, and there were more in the past few years. Before 2015, 80% of my students were teaching at universities, including in the United States, the United Kingdom, Hong Kong, and the Mainland. After 2015, many of them are in the corporate world or starting a business. Recently, many doctoral students want to enter the school again.
Pencil Road: Around 2015, what changes have been made in the country's support for the robot industry?
Liu Yunhui: After 2015, hard technology, especially robots, has become an unavoidable technical direction. In terms of labor costs, robots are a must to improve the degree of automation in the manufacturing industry. Including the future in the service, especially the pension industry, is really a big problem. Now (robots) are mainly in the factory, and there is still a lot of space in the factory, but (robots) are slowly entering the society and into the family, it is certain. But for home service robots, we are still researching and doing a lot of exploration. Whether it will be a humanoid or not, there is no (conclusive), and there are still many difficulties in technology.