Author|Yang Xiao
Editor|Su Jianxun
The Stanford Chinese team's Mobile ALOHA "all-round housework robot" has become popular on the Internet.
It demonstrates a variety of household skills such as cooking, making beds, watering flowers, and is also able to cook shrimp, wipe red wine on the table, wash dishes, and automatically open the elevator...... It can be described as a fully compound "nanny talent". This made netizens call the era of robots coming, because it "has work in its eyes".
However, the complex skills demonstrated by Mobile ALOHA are actually operated by humans remotely.
In the early morning of January 6th, after discovering that netizens were overwhelmed by the autonomous and intelligent capabilities presented by Mobile ALOHA, two R&D personnel of Mobile ALOHA hurriedly "refuted the rumors", saying that some of the actions were displayed by the robot autonomously, and some were completed by human remote control.
In order to faithfully present the robot's capabilities, they also specially released a collection of Mobile ALOHA's "stupid" actions in a fully automatic state, saying that "robots are not ready to take over the world".
When you think the robot can lift a glass of red wine with silky grace, it can only hold the glass tremblingly, watch the glass fall to the ground, and spill the red wine all over the table.
Wine spilled all over the table
When performing well, the robot can familiarly hold the pot with its left hand and the spatula with its right hand to complete the preparation of a dish. But a few times, the spatula that was originally free to swing was not operated on the bottom of the iron pot, or waved in the pot but could not touch the dish.
The case of a failed stir-fry
In the fully automatic state, it could not effectively control a pan, and after the pot was lifted, the entire pan inevitably fell, and the researchers who were guarding the side were burned in order to save the overturned iron pot.
Failed to pour the dish
Even a red ballpoint pen that Mobile ALOHA couldn't pick up.
Take a marker
There is still a gap between reality and ideals, but the ability of Mobile ALOHA to complete delicate and complex tasks is still worth paying attention to, which makes future robots more imaginative.
Remote-controlled quilt folding, cooking, and laundry skills
What exactly can Mobile ALOHA do?
After 20~50 rounds of demonstration and imitation Xi, the robot can autonomously complete some complex mobile control tasks. It is able to pour oil into the pan in the order of human cooking, add another shrimp, flip the spatula to fry it, and pick up the pan, turn around and pour the shrimp into the plate that has been prepared behind it.
Stir-fried shrimp
It can automatically press the elevator button and successfully enter the elevator;
Automatically open the elevator
You can also pinpoint a two-door cabinet and lift your hand to place a 3-pound pot in the cabinet and close the door.
Place the pot in the cabinet door
It is also possible to extend a "hand" after an encounter with a person and give a high-five.
High fives with people
At present, the robot can complete the action of wiping the table with a wine glass for 9 consecutive times, and call the elevator autonomously for 5 consecutive times, and has the ability to learn skills. However, this skill is not 100% accurate and intelligent, and errors will occur, presenting the "failure state" shown above.
For a robot that wants to enter the home scene, safety is a more important prerequisite than skill richness. A bulky, steel robot can bring far more lethality to the home than pets such as dogs.
Robots are not ready to take over the world, but they have potential.
To explore the potential of Mobile ALOHA, Dr. Foo took the robot home and remotely controlled the robot to perform some complex movements. It can complete the cooking skills of making smooth egg shrimp, dried scallop roast chicken, and oyster sauce lettuce, which involves delicate actions such as chopping garlic, washing lettuce, and beating eggs.
Make scalloped roast chicken
In the remote control state, it can pull back the curtains, open the window, and hold the kettle for watering;
Zhaoshui
You can hold the Dyson vacuum cleaner and clean the whole house;
Vacuum the room
It can complete the entire shaving action while the owner is standing still, pinching the round razor.
Shaved Huzi
You can put the clothes in the washing machine, turn the round cap of the laundry detergent, pour the laundry detergent into the washing machine, and then operate to complete the laundry action.
wash clothes
It is also possible to extract a piece of paper from the drawer and crumple it into a ball, and then wipe the ash off the glass, and control the contact between the machine and the glass without harming the glass surface.
Take paper and wipe the glass
Although the aforementioned housework is done by humans remotely, the physical possibilities of robots to complete complex, fine motor movements are a huge step forward.
It also solves a major technical problem in the field of humanoid robots - in the face of fabric, a soft and thin material, how can the robot complete the actions that can be easily completed by hand, such as folding quilts and clothes.
In the remote control state, it can complete the action of flattening the bed sheet and putting pillowcases on the pillow, and the bed laid by the robot can be described as meticulous, far beyond the scope of bed storage capacity of contemporary young people.
Change the bed linen
It can also unfold a piece of clothing, pinch the zipper pull, close the zipper on the clothes, hold it up with a clothes brace, and put it in the closet.
Organize your clothes
In the robot world, actions that can be easily performed by human hands are more difficult for machines. Robots can't do "work in their eyes", but they can do meticulous work with their hands.
Tony Z. Zhao, co-developer of Mobile ALOHA, also cited the failed compilation of the 2015 DARPA Robotics Challenge, jokingly claiming that the fall video of the Hope Tesla robot falling.
Netizens were also very tolerant of Mobile ALOHA's failed highlights, calling them "stupid and cute", and some people said that "compared to remote control, autonomy is really much more difficult".
The software and hardware are open source, and the full set of robots only costs $32,000
The greatest value of Mobile ALOHA is the advancement in physical manipulation capabilities, which demonstrates the potential of robots to perform repetitive, delicate manipulation tasks.
In order to realize the two-handed operation skills, Dr. Fu's team designed a low-cost full-body remote operating system "Mobile ALOHA" based on the ALOHA remote operating system generated by Google Deepmind.
Mobile ALOHA is an open-source operating system that learns Xi principles of teleoperation and imitation Xi, that is, through the Xi of learning and mastering skills of human repetitive movements, it is a full-body remotely operated two-handed mobile operating system.
Google's ALOHA remote operating system is more about thinking on the desktop, and Mobile ALOHA adds a mobile chassis on the bottom to increase the training of robot mobility and flexibility.
With the integrated mobile base and full-body remote operation interface, Mobile ALOHA can collect more data on the robot's interaction with the real world, and learn to Xi and mimic human movements during the interaction.
The research team found that with just 50 demonstrations of each task, Mobile ALOHA had a 90% success rate in learning to move, such as 9 consecutive rubs of red wine and 5 consecutive successful use of the elevator.
According to Dr. Fu, the ability to achieve this in less than 50 demos is due to Mobile-ALOHA's combination of imitation Xi and static manipulation data for collaborative training.
Synergistic training effect
With this joint training approach, the success rate of robot training can be significantly improved, especially when performing tasks that require precise actions. For example, in the task of wiping red wine, raising the wine glass and wiping are the most important sub-movements, and the success rate is only 58% when there is no joint training, but after the joint training, the success rate can be increased to 95%.
Difference in training effect 1
This method of joint data training has good results in different simulation Xi methods such as VINN, diffusion strategy, and ACT. For example, under the diffusion strategy, the success rate of the entire mission was originally only 35%, but after joint training, it can be increased to 65%.
Difference in training effect 2
In addition, the robustness and stability of collaborative training have been improved in different dimensions under different data combinations.
Difference in training effect 3
In terms of hardware, its external image is not the hottest humanoid robot form, and the entire body is composed of a mobile chassis, an operating interface, two robotic arms, feedback drive, two wrist cameras and a top camera. The whole machine is designed with a reverse drive structure, and the controller can control the robotic arm through the operator console to guide the Mobile ALOHA to complete the action. It has a battery pack installed at the bottom, which can be used when the power is off.
Hardware design for Mobile ALOHA
The robot and operating system were developed by a team of three people at Stanford, Zipeng Fu and Tony Z. Zhao, Ph.D. students in computer science, and their supervisor is Chelsea Finn.
They have open-sourced both software and hardware, published the code, hardware structure and data, and the team also made a specific list of hardware, including the cost of the entire set of robots, which is only 32,000 US dollars, or 227,000 yuan.
This means that anyone who is interested can conduct experiments on robots based on their research results to train them to achieve the actions they want to achieve. Songling Robot, the sports chassis manufacturer of the hardware, has said on its official website that it can customize the same Stanford Mobile Aloha robot.
Mobile Aloha机器人
2024 is considered to be the first year of humanoid robots, whether it is embodied intelligent players such as Tesla and Boston Dynamics, or Google's AutoRT control system, or RT-2 robot models, all of which are promoting the development of humanoid robots from different dimensions. In the long-term imagination, humanoid robots entering the home will bring huge market space.
Originally, Tesla, Zhiyuan Robot and other humanoid robots regarded the To B scenario as the first to land in the future, and the To C family scene was more difficult, but Mobile ALOHA's robots made the To C scene see the light of day.
Although the era of fully automatic robots has not yet arrived, Mobile ALOHA can complete difficult actions such as laundry and cooking with only two fingers, which gives housework robots and humanoid robots a more concrete imagination.