FSD empowers AI decision-making.
Author |
Edited by Tian Siqi
On May 5, Tesla shared the latest progress of the Optimus (Optimus Prime) humanoid robot on the X account, and the capabilities of Optimus have been fully upgraded.
In the video, Optimus uses an end-to-end neural network to perform basic factory tasks, including accurately sorting and inserting the Type 4680 battery cells into trays, and is walking at a much faster rate of more than 30 percent from December, which is now around 0.6 m/s. In addition, Tesla has made a special emphasis on the autonomy of robots, and Optimus is even able to correct its own mistakes at work.
Milan Kovac, an engineer on Tesla's robotics team, elaborated on the newly upgraded Optimus:
- Tesla developed and deployed a neural network that allows the Optimus robot to perform more practical tasks, such as picking up Type 4680 batteries from a conveyor belt and accurately placing them in the battery tray.
- This neural network operates entirely end-to-end, using data from the robot's own 2D camera and tactile and pressure sensors to directly generate sequences that control the joints. The network runs entirely on the robot's Full Self-Driving (FSD) controller, powered by the robot's battery.
- Optimus is designed to perform multiple tasks with a single neural network. During the training process, Tesla introduced a lot of different data.
- Optimus is currently imperfect and has a slower response time, but its success rate is gradually improving. Tesla is also training Optimus on how to recover from failure, and Optimus has been observed to correct errors on its own.
- Tesla has deployed several Optimus robots in one factory, which are tested daily in a real-world production environment and continuously improved. Optimus is now also able to walk for long periods of time in the office without falling.
- Tesla is further researching how to increase the robot's movement speed so that it can adapt to more complex terrain, all of which have not affected its humanoid properties. At the same time, Tesla is also working to improve the replicability of the entire robot fleet, training the neural network to handle dynamic calibration and small differences between robots.
Since its debut in 2021, Tesla has shared updates to Optimus every few months for three years, mainly to show how fast it is walking and how accurate it is in performing tasks. Tesla has also previously demonstrated the robot's ability to fold clothes, sort objects autonomously, and perform physical movements such as yoga. Tesla's CEO, Elon Musk, also said in December that these "steel men" could hold the needle in a year.
On April 23, during Tesla's Q1 2024 earnings call, Musk emphasized that Optimus is already on factory duty and is expected to be more widely used in the coming years. It is expected that by the end of this year, Optimus will be carrying out a series of work tasks in the factory. And by the end of 2025, "robots will be sold externally".
Regarding the price, Musk said in March: "The price of the Optimus will end up being less than $25,000 or $30,000, and he expects its production cost to be less than half the cost of the car." ”
1. More flexible hands
Jim Fan, senior research scientist at NVIDIA, praised the Optimus upgrade as a remarkable update. "We were able to get a glimpse of them collecting human data, which is one of the great strengths of Optimus," he said. ”
What are the key elements needed to establish such an efficient data collection process?
The first is the hand of the Optimus robot. Its hands are one of the most advanced five-fingered dexterous robotic hands, with tactile sensing and the ability to perceive and process a variety of objects.
Compared to many competitors with only 6 to 7 degrees of freedom (i.e., the number of joints that can move independently), the Optimus has 11 degrees of freedom, which greatly improves flexibility and precision. The high degree of freedom of the hand also allows Optimus to perform more complex tasks.
The upgraded Optimus is sorting the battery, source: Tesla
At the same time, Optimus is robust enough to handle frequent object interactions, reducing the need for routine maintenance. Tesla has made improvements to the robot's vestibular system, foot movement trajectory, and ground contact logic. At the same time, its motion planner has been upgraded, adding slight torso and arm swings, as well as shortening the robot's cycle delay time. As a result of these technological improvements, Optimus has seen a significant overall improvement in stability and confidence while exercising.
The second is the remote operation software used by Optimus. The operator controls the robot by wearing VR glasses and gloves, enabling precise control with very low latency. The complexity of this setup lies in the fact that a large number of real-time video streams and control signals are processed at the same time, ensuring that the operator's movements are reflected on the robot instantly and accurately. Even the smallest delays can significantly affect the intuition and efficiency of operations, so it is critical to reducing latency in human-computer interactions.
It is important to emphasize that Optimus uses Tesla's Full Self-Driving (FSD) controller in its control system. FSD provides advanced vision processing capabilities and real-time decision-making capabilities that enable robots to autonomously complete complex tasks without direct human supervision.
The key to FSD lies in its advanced neural networks, which are able to collect data from the car's cameras and sensors and translate it into driving commands such as steering, accelerating, and braking.
Tesla uses an end-to-end neural network in its FSD system, where the entire process, from data input to driving decisions, is done automatically, without the need for human-written code. Such a system not only reduces reliance on hard-coded rules, but also improves its performance through continuous learning and adaptation.
One of the main reasons for using end-to-end AI in FSD systems is its ability to significantly improve the speed and accuracy of decision-making. By learning directly from real-world driving data, this technology can more accurately simulate and predict human driving behavior, leading to safer and more efficient driving in a variety of driving environments.
It can be seen that Tesla's use of FSD technology on robots is mainly to make robots more independent and effective in performing tasks with the help of its powerful data processing and AI decision-making capabilities. Leverage FSD's ability to process high-speed, high-precision data streams for smoother and more precise robot motion control.
Moreover, FSD has been widely used and verified in automotive autonomous driving, and its high-performance computing capabilities can fully support the complex data processing and real-time decision-making needs of Optimus robots.
In addition, Optimus' operating system includes not only advanced hardware, but also complex operational management. A large swarm of robots requires multiple robots running simultaneously to collect data in parallel, in addition to trained human contractors working in shifts around the clock, as well as on-call maintenance teams to ensure the continued efficient operation of the system. The diverse application scenarios of the Optimus robot can also be seen in the video, whether it is moving batteries in a factory environment, or handling laundry and organizing everyday items in a home environment, Optimus has shown great adaptability and functionality.
Optimus walking in Tesla's office after the upgrade, source: Tesla
This broad integration of technologies not only demonstrates Optimus' capabilities as an advanced robotics platform, but also begs an important question about the future of humanoid robotics: In the face of limited budget and resources, how does Tesla choose the tasks that maximize skill transfer and generalization?
While remote operation offers a possible path to solving the problem of humanoid robots, the scalability of this approach is limited. In the future, it is also necessary to explore how to further improve the autonomy and versatility of robots while maintaining operational accuracy and low latency. Machine learning algorithms, augmented perception systems, and the physical design of the robot all need to be further optimized.
2. The controversial "Optimus Prime"
Since August 2021, when the concept machine Tesla Bot was first publicly demonstrated at Tesla's AI DAY, the Optimus robot project has had mixed reputations, and many people only regard him as a gimmick for Maskra's investment.
The Optimus prototype announced by Tesla in 2021, source: Tesla
At the time of its debut, roboticists pointed out that the Optimus robot showed limited mobility and was a big gap to the existing humanoid robots of companies such as Boston Dynamics.
Shaun Azimi, head of NASA's Dexterity Robotics team, pointed out that autonomous driving is not as simple as people think, and for humanoid robots, the challenge is even greater. He stresses that the robot's ability to respond flexibly is particularly critical when unexpected situations arise.
At the same time, Nancy Cooke, a professor of human systems engineering at Arizona State University in the United States, mentioned that to prove the success of the robot, Musk needs to demonstrate that the robot is capable of performing diverse and non-specific instructions. Just having a robot walk and dance doesn't give it a full show.
But amid a series of controversies, Optimus is also moving forward:
- In February 2022, Tesla unveiled a prototype of its humanoid robot, marking an important step from concept to reality. By April 2022, the prototype completed its first gait walk.
- At the second AI DAY in September of the same year, Optimus publicly demonstrated for the first time actions such as walking upright, carrying and sprinkling water.
- In February 2023, Optimus was shown through a video at Investor Day, where simple jobs such as freewalking and screwing were showcased.
- At the shareholder meeting in May 2023, Musk showed the walking, waving, and rocking movements of the Optimus and shared a video of its application in a car factory.
- In September of the same year, Tesla released a video on social platforms showing Optimus' self-calibration, multitasking, attitude control and self-balancing capabilities.
- In December at the end of 2023, the Optimus Gen-2 video was released, demonstrating more agile walking and finer movements, such as two-finger holding of an egg and left-right hand transfer, demonstrating advanced movement and centroid control.
Tesla Optimus development node, source: Medium
The Optimus Gen2 robot is a significant improvement over its predecessor in several ways. The Optimus Gen2 features actuators and sensors designed in-house by Tesla. Among them, the robot's neck has been increased to 2 degrees of freedom, making the head movement more natural; Walking speed increased by 30%; The weight of the robot is 10 kg lighter than the previous generation.
The Optimus Gen2 also adds foot force and torque sensing, and features an articulated toe section that better mimics the geometry of the human foot, which improves the robot's stability and adaptability on a variety of grounds. The most striking thing is the above-mentioned fact that the hands of this generation of robots have 11 degrees of freedom.
In the past three years, Tesla's Optimus humanoid robot has achieved a transformation from a conceptual prototype to a technological breakthrough. Its improvement in walking speed, autonomy and stability indicates that humanoid robots will be more flexible and efficient, so that they can enter more industries to improve production efficiency and improve the quality of life of human beings.
A more intelligent and humanized era of robots may soon come.
(Cover image source: Tesla)