Excerpted from spectrum.IEEE
By Evan Ackerman
Machine Heart Compilation
Editor: Chen Ping
Eliminating high-end, energy-intensive components that require only a monocular camera, some neural networks, a basic GPU system, and some simple hints in the form of a very basic top view that is readable to humans, the robot can navigate very well.
Most robots navigate very differently from most humans. Robots perform best when they have a comprehensive understanding of their environment, a complete geometric reconstruction of everything around them, and an accurate understanding of their position and orientation. With lidar, pre-existing maps, powerful computers, and even motion capture systems, it's fair to say that the "demand" for autonomous robots is never-ending.
But it's clear that these things don't scale well, and of course it may be that research can't afford it.
With this in mind, in a recent paper, ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints, Dhruv Shah, an AI PhD student at UC Berkeley, and his mentor Sergey Levine explored a different way for robots to navigate.
They argue that the elimination of high-end, energy-intensive components of robot navigation require only a monocular camera, some neural networks, a basic GPU system, and some simple hints in the form of a very basic top view that humans can read.
Such hints may not sound as impactful, but they enable a very simple robot to efficiently and intelligently traverse unfamiliar environments to distant destinations.
Address of the paper: https://arxiv.org/pdf/2202.11271.pdf
Project Home: https://sites.google.com/view/viking-release
Specifically, the study proposes a learning-based approach that integrates learning and planning and can use auxiliary information such as schematic roadmaps, satellite maps, and GPS coordinates as planning heuristics. ViKiNG combines a local traversability model that looks at the robot's current camera-based observations and a potential sub-target to infer how easy it is to reach the sub-target.
In addition, ViKiNG includes a heuristic model that looks at the top view and tries to estimate the distance from various sub-targets to destinations. ViKiNG does not perform explicit geometry reconstruction, using only the topological representation of the environment.
Although it has never seen a trajectory larger than 80 meters in the ViKiNG training dataset, it can take advantage of image-based learning controllers and goal-directed heuristics to navigate to targets up to 3 kilometers away in previously unseen environments and exhibit complex behaviors.
ViKiNG is also robust to unreliable maps and GPS, as the underlying controller ultimately makes decisions based on its own image observations, while maps serve only as a heuristic for planning.
The navigation of the ViKiNG robot goes like this:
Introduction to ViKiNG
If that little robot looks familiar, it's because we met it a few years ago through Levine's student Greg Khan. At the time, the robot was named BADGR, and its special skill was to learn to navigate new environments based on simple images and life experiences — or any robot equivalent to life experiences.
ViKiNG's predecessor, BADGR.
BADGR has now evolved into ViKiNG, which stands for vision-based kilometer-level navigation with geographic cues. While BADGR is free to walk around small areas, its successor, ViKiNG, aims to travel long distances to find targets, an important step toward practical applications.
Navigation, very broadly, includes understanding where you are, where you want to go, and how you want to get there. For robots, this equates to a long-term goal. Some distant GPS coordinates can be achieved by achieving a series of short-term goals, such as staying on a specific path within the next few meters. Reach enough short-term goals and you reach your long-term goals. But there's also a medium-term goal in it, which is particularly tricky because it involves making more complex and abstract decisions about what the best path might be. Or, to put it another way, which combination of short-term goals best fits the mission to achieve long-term goals.
Overview of the methods.
That's where ViKiNG's tip lies. Using satellite maps or road maps, robots can make more informed choices about short-term goals, greatly increasing the likelihood of achieving them. Even with a roadmap, ViKiNG isn't limited to roads; it's just that it might benefit the roads because that's the information it has.
Satellite imagery, including roads and other terrain, provides more information for the robot. These maps are hints, not descriptions, which means ViKiNG can adapt to obstacles it didn't anticipate. Of course, maps can't tell the robot exactly where to go on a smaller scale (whether these short-term targets can be crossed), but ViKiNG can handle itself with its monocular camera.
ViKiNG performance is stunning, as you can see in the figure, the blue line is the ViKiNG navigation path, which is the best route to the target in general. It's worth mentioning that the researchers didn't provide ViKiNG with a map of the surroundings, it did this with basic GPS, plus you needed to provide a photo, target GPS coordinates, monocular camera, and map. The image above shows the robot traversing a short path where ViKiNG can navigate autonomously.
Sergey Levine, corresponding author, assistant professor at UC Berkeley and reinforcement learning bull, said: "The study is exciting because the whole process is very simple. Unlike autonomous driving systems, which use a large number of software stacks and interactions, the system uses two neural networks (one processing first-person images and one processing map images) and a planning algorithm to determine the robot's walking path.
It can be said that this research is significant, because today's robot navigation systems are very complex and cannot be deployed on a large scale. If simple learning-based systems can match or exceed complex manual design methods, this could point the way for future machine navigation applications.