Editor: Editorial Department
The fire of the large model ignited the enthusiasm of the entire robot industry. Throughout the world, in the past few months, the cooperation between large model manufacturers and robot startups at home and abroad has also been moving. It seems that the first year of robots has really come!
Large models have been involved in the field of robotics.
In early March, the startup Figure released the first demonstration of Figure 01, a humanoid robot powered by OpenAI's large model.
With the LLM "brain", Figure 01 can see apples on the table, clean up the dishes, and communicate seamlessly with humans.
On the day of the official announcement of the electric Atlas, the start-up Mentee Robotics also showcased the first Menteebot humanoid robot, which can communicate through natural language.
Similarly, Menteebot, which is blessed by a large model, can interpret commands, think, make decisions, and complete tasks.
Under the wave of large models, similar cases have emerged in an endless stream in the past year.
On the other hand, in China, with the help of large models, the field of robotics is also hot.
At the beginning of the year, UBTECH, known as the "first stock of humanoid robots", recorded a skyrocketing trend of 3 times in 2 days, echoing the recent cooperation between large model manufacturers and the robot industry.
Some industry experts say that the era of robots with multimodal LLM brains has finally arrived. These robots will be able to understand instructions and perform tasks: they can use laptops, wash dishes, brew coffee, and get a proper AGI!
Obviously, large-scale model empowerment robots have huge potential and have become one of the few consensus of major technology manufacturers.
The first year of the explosion of "embodied intelligence" has come
People say that 2024 is the first year of robots.
The advent of the large model has undoubtedly made the robot industry a bright spot in the research and industry again.
Let's take a look at a picture first, and intuitively feel the current progress of robot companies around the world.
As many people expect, on the day when AGI really arrives, "embodied intelligence" is an indispensable and important hardware carrier.
From the past year to the present, the investment market in the field of robotics has continued to heat up and ushered in a highlight moment.
In the first three months of this year, robotics startups raised $3.2 billion, compared with $1.7 billion in the same period last year, according to research firm Robot Report.
In foreign countries, OpenAI's two humanoid robot startups, 1X and Figure, have received a new round of financing.
There are also robot startups such as Bear Robotics, a service robot in Silicon Valley, Physical Intelligence, which develops a brain for robots, and Skild, which has not yet generated revenue, which are also sought after by capital.
In China, represented by UBTECH, it became the first stock of humanoid robots to be listed at the end of December 2023, and as mentioned earlier, the stock has been climbing.
Not long ago, the industrial version of the humanoid robot Walker S has entered the workshop and started to work.
In addition, Unitree humanoid robot completed nearly 1 billion yuan of B2 round of financing in February this year, and its product Unitree H1 is even more popular abroad.
According to the statistics of netizens, in the first quarter of this year alone, there were more than 20 financing cases in the domestic robot market.
Source: Internet
It can be seen that the injection of a large amount of capital has pushed the boom of the robot industry to the highest point.
In fact, robots are nothing new for everyone.
Why did the addition of the large model usher in the ChatGPT moment?
Why do you need a multimodal LLM?
As we all know, traditional robots have one obvious limitation – they require clear instructions.
It is not difficult for them to master individual skills, such as opening doors, pulling drawers, picking up and manipulating something.
However, it is very difficult to get a robot to complete a task that combines multiple skills.
This is why the emergence of large models solves the rigid problem that traditional robots need clear instructions in order to perform tasks.
In simple terms, LLMs can map loosely defined instructions to specific task sequences within the scope of a robot's skills.
For example, when you nod your head to a robot, how do you make it nod to you in a friendly way?
GenEM, developed by the University of Toronto, Google DeepMind and other institutions, uses GPT-4's rich knowledge to transform the abstract behavior of "nodding" into a specific action that can be output by the robot.
However, on the road to embodied intelligent AGI, relying on large language models alone is not enough.
This is because various multimodal information such as images, text, voice, and video constitute our real world. Human feelings, communication, are all carried out in a multimodal form.
For intelligent robots, it is also a must-have ability.
For example, when a robot wants to correctly execute the command "I'm a little tired, help me get a refreshing drink", the most important thing is to complete "multimodal understanding".
Seeing a pile of food on the table, which is the coffee?
After understanding the voice command and disassembling the task steps, the robot needs to recognize the objects in the "line of sight" and determine which bottle of drink needs to be taken according to the meaning of the instruction.
Not only that, intelligence such as smell and taste will also be the field of robots that will gradually expand in the future.
All in all, multimodal capabilities are essential for robots that are actually going to enter the home, and multimodal understanding in particular is particularly important.
Domestic top stream: iFLYTEK Spark large model
It can be seen that the market's enthusiasm for robots has been completely ignited by AI models.
A 25-year-old company in China launched the "Superbrain 2030 Plan" in 2022, imagining that robots can enter every home.
The plan is divided into three phases, and through gradual progress, AI can understand knowledge, learn well, and evolve.
For a long time, based on the continuous exploration of cutting-edge technology, the company behind it, iFLYTEK, is moving towards the goal.
First, at the 2022 iFLYTEK Global 1024 Developer Conference, the "iFLYTEK Robot Superbrain Platform" was released for the first time.
The platform provides a full-stack toolchain for developers, including model training, asset generation, and software and hardware access.
After ChatGPT detonated the large model craze, iFLYTEK released the "iFLYTEK Spark Large Model" for the first time in May 23, and completed 5 iterations in more than a year.
The original iFLYTEK Xinghuo large model has seven dimensions that embody general artificial intelligence, including text generation, language understanding, knowledge question and answer, logical reasoning, mathematical ability, code ability and multimodal ability.
In the following June, August, and October, as well as in January this year, the iFLYTEK Xinghuo large model completed the iterative upgrade from V1.5, V2.0, to V3.0 and V3.5.
The large model breaks the ceiling of traditional cognitive intelligence and lays a solid foundation for robots to understand the world more deeply.
However, the full integration of these advanced technologies to facilitate significant advances in human-machine interaction, task planning, and environmental adaptation for robots also requires reliance on specialized systems.
Inject soul into the "Robot Superbrain Platform".
To this end, in July 2023, iFLYTEK officially integrated the iFLYTEK Xinghuo large model with the "Robot Superbrain Platform".
Specifically, the iFLYTEK Robot Superbrain Platform is based on iFLYTEK SuperBrain 2030 technology, a robot development platform for the physical world, virtual world and metaverse.
It focuses on multimodal perceptual expression, open semantic understanding, large and small brain collaborative motion control, and software and hardware access, and can help developers quickly build physical robots and virtual digital human products.
Nowadays, humanoid robots face more challenges when it comes to practical applications.
Do you often encounter robots in shopping malls, talking to chickens and ducks, and you can't speak two sentences at all, and you are driven crazy?
In such a noisy environment, the interaction between humans and robots is simply more difficult.
Or, when you ask the robot to tell a joke, that serious broadcast tone is simply an embarrassing atmosphere group online.
The key to solving these problems is to make the robot better "pick up", and the speech no longer has such a heavy "machine flavor".
The iFLYTEK Robot Superbrain Platform solves this problem from two aspects: "multi-modal perception interaction of audio-visual fusion" and "large model understanding and decision-making".
The first is to create a new paradigm of robot interaction.
To this end, the iFLYTEK robot superbrain platform integrates multi-dimensional information such as speech, vision, and semantics.
By upgrading the microphone array algorithm to fuse voice, face, and lip shape information, even in noisy and noisy scenes, the robot can accurately pick up voices and achieve "clear hearing" in complex scenes.
At the same time, through the large voice model and super-anthropomorphic synthesis technology, the robot can "hear accurately" and "answer naturally", making the robot more vibrant and infectious!
Secondly, it is the interactive brain of the robot.
This brain, undoubtedly based on the iFLYTEK Xinghuo large model, realizes the unification of multiple interactive scenarios such as control-level instructions, official skills, core business functions, quick knowledge questions and answers, and chatting and companionship.
For every child, if the robot can tell bedtime stories in the mother's voice and tone, it can be called a real "companion" robot.
But for this to happen, the robot also needs to be emotionally and expressively compositable.
In this regard, the iFLYTEK Robot Superbrain platform has also designed a multi-style and multi-emotional AI persona, which can make each robot unique.
By the way, the virtual human-driven protocol of the iFLYTEK Robot Superbrain platform has been fully opened.
By accessing the drive protocol, the third-party digital human product can achieve the same interactive effect as the iFLYTEK digital human.
The iFLYTEK Robot Superbrain platform has empowered 398 robot customers in four major fields and deeply linked 13,000 robot developers.
Official website address: https://aibot.xfyun.cn
With the blessing of sparks, the robot soars
In order to further expand the cooperation ecosystem, on April 15, 2024, iFLYTEK officially launched the "Galaxy Action" plan to recruit ecological partners to jointly promote the prosperity of the robotics industry.
Prior to this, there were many domestic head robot manufacturers from different segments that had already used the powerful capabilities of the iFLYTEK Xinghuo model.
Currently, in the opinion of many, humanoid robots are the best universal embodied form.
When the robot is on the ground, the humanoid form is not necessarily the best form to complete the task. It can be a robotic arm or a wheeled ......
iFLYTEK and Pangolin Robotics have been cooperating for a long time, and have been jointly exploring the integration of AI technology and robots, and constantly expanding the new boundaries of service robot applications.
The company's new AI service robots such as Xiaoyu, Amy, and Xiaoxue are all equipped with the iFLYTEK robot superbrain platform and iFLYTEK Xinghuo large model technology.
Based on iFLYTEK's advanced speech recognition and AI technology, the robot can smoothly conduct natural language conversations, quickly understand and give accurate answers.
At the same time, they are able to provide a wide range of information and advice thanks to their extensive technical reserves.
Pangolin robots have been widely used in catering, government affairs, education, medical and other fields
Of course, there are also home service robots, and Optimism is one of them.
By accessing the iFLYTEK Spark model and multi-modal interaction, this desktop black technology robot has become an intimate "little housekeeper" for children.
With just one call, whether it is entertainment or learning and life problems, it can accompany children with unprecedented interactive ability.
Among the head humanoid robots, iFLYTEK's core technology is also indispensable.
As we are familiar with, companies such as Zhiyuan Robot, UBTECH, and Unitree Technology have all received the blessing of iFLYTEK's full-link voice and iFLYTEK Xinghuo large model.
There is also the EX robot that has recently officially announced the cooperation, which perfectly combines the respective advantages of both parties.
Based on the iFLYTEK robot superbrain platform, the iFLYTEK Spark large model and multi-modal interaction technology are applied to the EX bionic robot, realizing the same thinking, dialogue and action as people.
In addition to the above cases, the "circle of friends" of iFLYTEK's ecosystem is still expanding.
Robot development, the best of times
We should also see that the development of robots not only relies on the technical iteration of the AI brain, but also requires the parallel of the "body".
With the gradual development of the robot industry, the supply of components in the robot industry has also begun to develop in the direction of specialization and maturity.
On April 2, at the Chinese Human Robot Ecological Conference held in Shanghai, it can be seen that robot component manufacturers have also made great progress——
Manufacturers such as humanoid robot universal base, robot dog universal base, robot super brain board, multi-modal voice interaction, 3D vision chip, flexible manipulator, robot servo motor, 3D printing frame and other manufacturers participated in the exhibition.
A number of special reports were held at the venue, such as "multimodal + large model, building a new interaction of humanoid robots", "humanoid robot perception technology and development" and other sub-fields of achievement display.
This will be the best time for entrepreneurs and developers in the robotics industry!
A large number of modular, rapidly integrated industry platforms and components make it no longer necessary to make algorithms from scratch as in the past.
In particular, the iFLYTEK robot superbrain platform has introduced a general open platform with large models, which further reduces the difficulty of the development of conventional algorithms and human-computer interaction of robots to the point of "ready-to-use".
In addition, the iFLYTEK robot superbrain platform has opened up the mature robot hardware system of the docking link company (Yushu, Zhiyuan, EX Robot, etc.).
This means that secondary application development will become one of the fastest ways to enter the industry and meet customer needs with products.
Obviously, the underlying hardware of the robot has been initially equipped, and the LLM technology is the core of the robot interaction.
Next, the content of customer demand mining, pain point solutions and user personal services needs to be further polished.
This will be the beginning of entrepreneurship in the robot industry.
Large model + robot, the prospect is very good
The next step is how to promote commercialization after the accelerated integration of LLMs and robots?
From the perspective of cognitive ability, the development of AI robots is getting closer to humans. Even, in terms of appearance, it has become more human-like.
The Boston Consulting Group (BCG) estimates that the global robotics market will reach $160 billion to $260 billion by 2030.
In other words, the future market prospect of LLM+ robots is very broad, and it can be deeply applied to various industrial fields and human life scenarios.
In manufacturing, robots on assembly lines are able to produce goods of higher quality and consistency that human workers can't match.
In warehouses and logistics companies, AI robots are able to perform heavy lifting tasks, such as transporting products, placing them on shelves, and other tasks, greatly reducing the burden on human labor.
For example, the army of 750,000 robots in Amazon's fulfillment warehouses has been fully put into use.
In addition to robots in the industrial field, AI medical robots can also help doctors perform surgeries, make more accurate diagnoses, and guide patients through physical therapy and rehabilitation.
Looking to the future, AI robots will be fully covered in many scenarios such as restaurants, space exploration, education, and nursing homes.
It is not difficult to predict that the dawn of robotics has arrived, and it is reshaping the whole world in ways that we could only imagine a few decades ago.
What iFLYTEK has done is to use technological innovation to bring robots into every home.