Reported by the Heart of the Machine
Editor: Chen Ping, Dapan Chicken
A busy day for robots.
This year, the robot is really going to become a refinement, and after bringing it home, it is a little expert in housework.
Cook a few delicacies at your fingertips, and after a while, you can make a quick meal: shrimp with eggs, lettuce with oyster sauce, roast chicken with scallops, and those who don't know think it's made by a real chef:
The preparation of the dish is also done in a good way, I saw it skillfully take out a lettuce and cut off the roots, and then gently beat the egg and put it in the bowl:
The egg-beating link also knows to throw the eggshell aside, it seems to be a fastidious robot, and he is no longer afraid of the fright of accidentally eating the eggshell when eating an omelette:
The robot keeps stir-frying in the frying process to avoid sticking to the pan:
Don't forget to infuse the oyster sauce lettuce with soul minced garlic. This time, we don't use a kitchen knife to shoot garlic, and ordinary fruit knives can also cut out the garlic paste, which shows that the knife is very good:
Finally, the finished sauce is poured over the lettuce, and a home-cooked dish with full flavor and color is perfectly served:
You think that the robot can cook food is already its limit, but what you don't know is that it is also a good cleaner.
Get up in the morning to water the green plants on the balcony, even if you are away on a business trip, you are not afraid of the flowers and plants "dying of thirst":
There is always work in the eyes, and when the floor is dirty, the floor is mopped:
Brewing a cup of coffee with a capsule coffee machine will allow you to start the day with 100% energy:
Help the boy shave, if the strength is not accurate, this job really dare not be handed over to the robot:
The cat owner is no longer lonely, and there is another "shovel officer" at home to play with it:
If the clothes are dirty, they will help you wash your clothes, and open and screw the lid of the laundry detergent by yourself:
Even technical work such as pillowcases is not difficult for it:
Hang the washed clothes, and the robot can accurately find the zippers, fasten the zippers together, help you pull the clothes, and then hang them:
Show you how to open a bottle cap in public:
Help you tuck the quilt at bedtime and turn off the lights and have a good night. The robot's day is finally over.
Seeing this, I really can't believe that this is what a robot can do, a full-time housekeeper.
The robot's name is Mobile ALOHA, and the research team from Stanford is a team of three people working together.
The project is co-led by Zipeng Fu, a Ph.D. student in computer science at the AI Lab at Stanford University, where he studied with Prof. Chelsea Finn, and Tony Z. Zhao, a Ph.D. student in computer science at Stanford University, where he was mentored by Chelsea Finn. Together, the three of them completed the study.
According to the list of materials of the robot, the total cost of this robot is about 32,000 US dollars, equivalent to about 220,000 yuan, and the software and hardware are all open source.
Bill of Materials
Introduction to the study
In the field of robotics, imitation Xi from human demonstrations has achieved remarkable results. However, most of the results are focused on desktop operations, lacking the mobility and flexibility to accomplish general tasks. So how does the $32,000 (about 220,000 yuan) personal butler and "royal" chef accomplish this flexible and meticulous work? Let's take a look at the technical details behind it.
In this work, the researchers developed a system for mimicking two-arm movement manipulation tasks that require whole-body control. The full-body remote manipulation system used for data collection is the mobile ALOHA. It enhances the ALOHA system with a mobile base and a full-body remote operation interface. Using the data collected by Mobile ALOHA, the study conducted supervised behavioral cloning and found that co-training with existing static ALOHA datasets improved the performance of mobile operations tasks.
By giving 50 demonstrations of each task, the joint training increased the success rate by 90%, allowing Mobile ALOHA to autonomously complete complex mobile manipulation tasks such as stir-frying shrimp, opening a double-door wall cabinet to store a heavy cooking pot, calling and entering the elevator, and gently rinsing a used pan using a kitchen faucet.
- Address: https://mobile-aloha.github.io/resources/mobile-aloha.pdf
- Project Address: https://mobile-aloha.github.io/
- Technical Documentation: https://docs.google.com/document/d/1_3yhWjodSNNYlpxkRCPIlvIAaQ76Nqk2wsqhnEVM6Dc/edit
Mobile ALOHA 硬件
Mobile ALOHA is a low-cost mobile manipulator that can perform a variety of household tasks. Mobile ALOHA inherits the benefits of the original ALOHA system, which is a low-cost, smart, serviceable, dual-arm remote operating unit, while extending its capabilities beyond desktop operation. The researchers focused on four key factors during the study
1. Mobility: The speed of movement is comparable to that of a human walking, which is about 1.42 m/s.
2. Stability: It remains stable when operating heavy household items, such as pots and cabinets.
3. Full body remote control: All degrees of freedom can be operated remotely at the same time, including both arms and moving base.
4. Cordless: It has an on-board power supply and computing equipment
Figure 2 (left) illustrates what the researchers found that a design that tied the operator's waist to the mobile base was the simplest and most straightforward solution.
The data in Figure 2 (center) shows that the manipulator has a vertical height of 65 cm to 200 cm relative to the ground, can extend 100 cm from the base, can lift an object weighing 1.5 kg, and can exert a pulling force of 100 N at a height of 1.5 m. This design allows Mobile ALOHA to accomplish a wide range of tasks, including physical cooking, housekeeping, human-computer interaction, and more.
More technical specifications for Mobile ALOHA are listed in Figure 2 (right). In addition to off-the-shelf robots, the researchers have open-sourced all software and hardware components and provided detailed tutorials, including 3D printing, assembly, and software installation. We can find this information on the project page.
Leverage static ALOHA data for federated training
In this work, the investigators used a joint training pipeline to leverage existing static ALOHA datasets to improve the mimicry Xi performance of mobile operations, especially two-arm movements. The static ALOHA dataset has a total of 825 demonstration actions, with tasks including sealing a sealed bag, picking up a fork, wrapping candy, tearing paper towels, opening a plastic bottle with a lid, playing table tennis, handing out duct tape, using a coffee maker, handing over a pencil, and operating a screwdriver, among other things.
It's important to note that static ALOHA data is collected on a black desktop, with both arms facing each other fixedly. This setup is different from moving ALOHA, where the background of the mobile ALOHA changes as the mobile base changes, with the arms parallel to the front. In the joint training, the investigators did not use any special data processing techniques for RGB observations or arms movements in static ALOHA data.
task
The researchers selected 7 tasks that cover a variety of functions, objects, and interactions that may arise in real-world applications. Figure 3 illustrates these tasks, which are wiping wine, boiling shrimp, rinsing pots, using cupboards, calling elevators, pushing chairs, and high-fives.
experiment
The purpose of the experiment was to answer two core questions:
(1) Can Mobile ALOHA acquire complex mobile manipulation skills through collaborative training and a small amount of mobile manipulation data?
and (2) whether Mobile ALOHA can be used with different types of imitation Xi, including ACT, Diffusion Policy, and retrieval-based VINN.
Table 1 shows the results of co-training to improve the ACT. With the help of collaborative training, the robot has a 95% success rate of wiping wine, a 95% success rate of calling elevators, and an 80% success rate of washing dishes.
Collaborative training improved the robot's performance in 5 of the 7 tasks, by 45%, 20%, 80%, 95%, and 80%, respectively. For the remaining two tasks, the success rate of co-training was comparable for those without co-training. The study found that collaborative training was more helpful for subtasks where precise manipulation was the bottleneck, such as pressing a button, flipping a shrimp, and turning on a faucet.
The success rate of Mobile ALOHA in co-training and no-co-training on 2 real-world tasks (polishing and pushing chairs) is reported in Table 2.
For more detailed technical details, please refer to the original article.