Recently, Chelsea Finn, the mentor of the project and a professor in the Department of Computer Science and Electrical Engineering at Stanford, announced at X that she has officially started a business with several other Berkeley scholars and Google DeepMind scientists to build intelligent brains for robots.
The name of the new company is Physical Intelligence, or Pi or π for short. The goal is to develop a set of "software that adds advanced intelligence to a wide range of mechanical devices" and ultimately build a general-purpose AI model that can control any robot to perform any task.
Chelsea Finn explains that this is challenging, requiring the integration of cross-platform strategies for bots, transfer learning from vision and language models, and the acquisition of flexible skills through imitation learning.
Co-founder and CEO Karol Hausman is also excited that the project will collect robot data on an unprecedented scale, improve algorithms and train very large models, and tackle all the technologies needed to bring AI into the physical world. To this end, they have assembled a "world-class team" and can't wait to embark on this new adventure.
According to public information, less than a month after its establishment, Pi has been locked in advance by a number of venture capital institutions, including OpenAI and its early investors Khosla Ventures, Sequoia Capital, and Lux Capital, and won a high financing of $70 million. This is not only because the company's technical prospects in the robotics track are extremely optimistic, but also because it is a confident bet on the strength of the founding team.
A team of ten people, one god per capita, and one Chinese member
Pi's official website page introduces itself as follows:
"Physical Intelligence is a new company that brings artificial general intelligence into the physical world.
We are a group of engineers, scientists, roboticists, and company founders who are developing the foundational models and learning algorithms that drive today's robots and tomorrow's physical devices. It's still early days, and interested partners are welcome to join!"
There are currently only ten members listed:
Although the presentation is "too plain", the team lineup is actually quite luxurious, and it is almost a god per capita. Before joining Pi, they each had a lot of research and development achievements, and several of them were well-known names in the industry.
首先必须介绍的是除Chelsea Finn和Karol Hausman的另一位联创Sergey Levine。
Sergey Levine is currently an assistant professor in the Department of Electrical Engineering and Computer Science at UC Berkley, focusing on general-purpose algorithms that enable autonomous agents to learn complex behaviors, with a focus on machine learning for decision-making and control. He has also developed end-to-end deep neural network training strategies, and has led the team to jointly develop the RT-X robotics project with Google, and is considered one of the leaders in the field of reinforcement learning.
图源：MIT Technology Review
However, what made this man even more famous was his nickname of "academic maniac". Sergey Levine has more than 130,000 academic citations on Google, and has published a large number of research papers in top international conferences and journals.
For example, at NeurIPS 2019 and 2020, 12 papers were accepted, ranking first in the NeurIPS list. In 2019, ICML was tied for second in terms of the number of papers received. In 2022, 30 papers were submitted to ICML, and 16 of them were received.
In addition, Sergey Levine is also a popular "Internet celebrity professor" in Berkeley, and his educational achievements are outstanding. His Deep Reinforcement Learning (codenamed CS285) course has been very well received by students. The online video can be watched on YouTube and Station B, and it is widely disseminated.
In the "Founding Statement" for the new company Pi, he said that he hopes to bring a general solution to the field of robotics similar to "large language models for natural language processing".
"We've seen many times in the past that machine learning faces a huge difference in large datasets versus small datasets. Our research is of great practical value and I believe it will also open the door to breakthroughs in basic research. ”
Chelsea Finn, mentioned at the beginning, is also one of the co-founders. She received her Ph.D. from MIT at Berkeley and won the 2018 ACM PhD Dissertation Award for her dissertation on meta-learning algorithms, which was supervised by Sergey Levine.
Chelsea Finn is currently an assistant professor of computer science and electrical engineering at Stanford University, where she focuses on learning and interacting to develop a wide range of intelligent behaviors for agents such as robots. Examples include end-to-end visual perception and robot manipulation, autonomous learning of generic skills from collected experience, and meta-learning algorithms for rapid learning of new concepts and behaviors.
Chelsea doesn't have as many Google Scholar citations as Sergey, but it's also prominent, with more than 49,000 citations. She also spent 5 years as a research scientist at Google Brain, developing robot depth prediction models.
Pi's CEO, Karol Hausman, is a senior research scientist at Google Brain and an adjunct professor at Stanford University. His research interests focus on enabling robots to autonomously acquire generic skills in the real world with minimal supervision, and he received the 2023 IEEE Society for Robotics and Automation Industry Career Award for "making significant contributions to scalable robotic learning algorithms."
Hausman's self-description at X is funny: "I love robotics, AI, the NBA, philosophy, football, and almond croissants. ”
In addition to these three, the team also brings together Brian Ichter, a former Google research scientist who specializes in robot motion planning and basic models, Anduril Industries, a distinguished engineer of Pakistani origin and a former Tesla autonomous driving and hardware expert who designed the Model X's unique upturned falcon door, Anduril Industries, now senior vice president and head of electrical engineering at Anduril Industries, and Chelsea's proud protégé, machine learning at Toyota Research Institute. Suraj Nair, a research scientist in robotics and computer vision, and Lachy Groom, a former executive at payments company Stripe and a well-known technology investor, are among the biggest names in the industry.
Another thing that caught our attention was that there was also a Chinese member in this list, Lucy Shi. The girl, who received her bachelor's degree in computer science from USC, is now a student researcher at Stanford, supervised by Professor Chelsea Finn. He has worked with Yoke Zhu, Senior Research Scientist and Head of the Universal Embodied Intelligence Team at NVIDIA, and Jim Fan, Senior R&D Manager.
She recently unveiled the Stanford-Berkeley Yell At Your Robot (YAY Robot) project, which showcases the results of robots improving in real time from speech correction, learning and improving based on human spoken feedback, and performing dexterous manipulation tasks.
Lucy Shi happily shared the news that she has joined Physical Intelligence as the "first intern" on her personal page.
In her introduction, she wrote, "I have a broad interest in robotics learning. The goal of the research is to create general-purpose robots that can seamlessly perform complex, long-term tasks in our daily lives....I am convinced of human creativity and the potential of artificial intelligence. In the next 20 years, I hope to become a university professor and build a new generation of Bell Labs, a factory of innovative ideas that will change the world. "We are also pleased to see another young scholar with both wisdom and ideals, and a promising future.
The creator of the brain of intelligent robots that rise to the challenge
In science fiction and movies for a long time, people have always dreamed of a robot that really understands their needs. It thinks, has emotions, can be by our side, can help solve all kinds of problems in life, and is as versatile as a human friend. However, although real-world robots can carry heavy loads in factories and clean homes, the range of tasks they can perform is much more limited than that of chatbots, which are becoming more general.
The rise of chatbots and LLMs has benefited from the massive amount of data in Internet corpora. OpenAI and Google can train large language models by feeding them billions of human language samples. However, it is extremely difficult to collect data of this scale from the real world, which has also limited the progress of AI in the field of physical robotics in recent years.
Physical Intelligence believes that now is the time to adopt a new approach to general-purpose robotics.
Figure 01 realizes the intelligent "seeing, listening, speaking" interaction of humanoid robots by connecting to ChatGPT, allowing people to see the great potential of combining large models and robots. Pi also hopes to combine the advanced technology of building language models with its own machine control and instruction technology to create a general AI system with a wide range of task execution capabilities that can be used on any hardware and any platform.
The team said that Pi is not focused on specific types of robotic arms or industrial robots, but plans to develop software that can be applied to many types of robots. They also don't make their own hardware, and their first step is to solve engineering problems, build models, and buy and train on a variety of different robots, with the goal of accumulating the largest amount of robot data to date.
In a public interview, Karol Hausman emphasized that the team wanted to develop a general-purpose model to bring artificial intelligence from computers to the physical world, "which can power any hardware device for any application." ”
And this is clearly not just Pi's vision. In addition to facing competition from companies like Figure AI and Tesla that make humanoid robots, efforts have been made for decades to improve the software that drives robots.
In the same week that Pi was announced, Covariant, a seven-year-old company founded by renowned AI scientist Pieter Abbeel and his three Chinese PhD students, launched the basic model RFM-1 to provide ChatGPT-like language understanding and generation capabilities for robots. After the joint training of general Internet data and rich real-world interaction data, RFM-1 can allow the robot to understand natural language instructions and generate corresponding actions, and can deal with some unexpected situations, which has received a lot of praise.
Now that they have joined forces and officially joined the battlefield, it is time for the team to gather the outstanding results accumulated by the team for many years. Coupled with the support behind OpenAI, can you get together Dragon Ball to summon the dragon and promote a new era in the field of general robotics?
"Our goal is to bring basic human capabilities to machines. "I think building humanoid robots is a really cool thing. But it's the brain, not the hardware, that fundamentally makes humans interesting – we're the ultimate generalists. ”
(封面图:Template/Moment RF)