The copyright of the original article belongs to the WeChat public account
"Bring science home" all
撰文 | Cloud
Proofreading | Ah Xian
"Hello! Tomorrow is Mother's Day, and my mom passed away about eight years ago. I don't know if you can tell where this picture was taken, because I was too young to remember. ”
图片来源:georainbolt
Trevor Rainbolt, 25, is a well-known expert when it comes to determining where photos were taken, and he often receives requests from netizens to help him find old photo locations. With excellent judgment, he helped a lot of people.
Rainbolt is a professional player of GeoGuessr, a game that looks at photos and guesses the location, and his popularity has also made many people know about this game. He has played the game for more than 10,000 hours, four to eight hours a day during the pandemic.
However, in May of this year, he lost to the AI.
This awesome AI player was developed by three graduate students at Stanford University and took about two months to develop. During training, it was 92% correct in guessing the country where the photo was taken, and the average score for playing GeoGuessr was 4525 out of 5000, ranking in the top 0.01% of global players.
AI有时候接近满分 | 图片来源：georainbolt
In the face of this AI opponent, Renin Bolt is also willing to bow down. However, the privacy risks that this AI may bring have also made many people uneasy.
How to determine the position of an image
Like Wrenbolt, the three AI developers are also veteran players of GeoGuessr.
GeoGuessr is an online geoguessing game with around 50 million players, developed in 2013 by a software engineer in Sweden. When entering the game, players are randomly placed in a random place on Google Street View, guessing their geographic location based on the information on the screen, and finally placing a pin somewhere on the world map to mark their guessed location. Most of the game's players are young, with some of the best players even being as young as 14.
The image that appeared in the first round of Wrenbolt against the AI, you can also guess | Image courtesy of Georainbolt
The accuracy and speed of a player's guess determines the score. The closer the guessed position is to the actual position, the more points the player wins. In addition, the faster the player completes the guess, the higher the score.
Round 1 Match Score | Image courtesy of Georainbolt
Any detail in the Google Street View imagery provided by the game is the basis for the player's judgment. Clues such as soil on the ground, telephone poles, street signs, road markings, people's clothing, plants, and visible landscapes can all be used to determine the location. Of course, the player's personal intuition is also important.
As for judgment skills, Rainbolt would recommend looking for Turi's bollards and utility poles first, which have a very unique design for each country or region. For example, the Danish bollards have yellow tops, and the German ones have black ones.
Swipe left to see more bollards
Swipe left to see more poles
For example, the materials used in the telephone poles are also particular, some are wood, and some are concrete. The shape of the pole also varies, and the sticker on the pole is also a clue. "Ninety-five per cent of Australia's concrete utility poles are in Victoria," said Wrenbolt.
In Tasmania, Australia, utility poles have devices to prevent possums from climbing Image courtesy of Wired
Professional players will learn multiple languages and memorize different things in different countries, such as knowing which countries use triple white road lines, which countries have dotted road lines, which countries have green road signs, what kind of font is on the signs, the construction materials used for the road, the length of the road, the number plate and type of car on the road, and the type of vegetation on both sides of the road...... All the seemingly ordinary and small details are clues to the game.
图片来源:georainbolt
In addition to the content of the picture, there is also a difference in the quality of the image captured by Google Street View. As the smallest republic in the world, San Marino is a country surrounded by Italy, and the street scene here looks very similar to Italy. However, the image quality of the two is not the same – San Marino's is a bit worse.
San Marino, surrounded by Italy | wikipedia
In addition, you will sometimes see a small part of the Google Street View car, and if you see a black tape on the top of the Street View car, it means that the location of the image is Ghana, a country in West Africa.
Black tape on a street view car | Image source: Geoguessr
Players also share clues, but with more than 220 billion Google Street View images, it's not realistic to remember all the hints, and master gamers tend to rely on intuition.
AI doesn't have the sixth sense of a human gamer, but it can see not only the obvious features that humans can see, but also the small details that are overlooked.
AI beats human players
Last year, a group of graduate students at Stanford University were Xi the AI course Deep Xi Multitask and Meta Learning, and three GeoGuessr students needed a project, so they decided to start with a common hobby and try to build an AI player that could be more powerful than a human gamer.
The name of their project is "Predicting Image Geolocations", which is simply called PIGEON (pigeon) based on the initials of the letters. They used a CLIP neural network made by OpenAI (the company that also developed ChatGPT) to understand and analyze images by reading text. They then trained the system with Google Street View Imagery. The trained dataset contains about 500,000 Google Street View images.
The AI divides the image into very small squares for analysis, and is very good at picking out those special squares. For AI, a picture is not only an image, but also a corresponding text message. By combining visual information such as buildings, street layouts, vegetation, and landmarks in the image, as well as other ancillary information such as weather conditions, seasons, and climate, the AI can also quickly predict where the image will be taken, just like a top player.
Faced with an image, the AI analyzes both the text and the image | References[1]
Despite the relatively small size of the dataset, the AI system performed very well in the end, with a high accuracy rate and a smaller error than human guessing, and it could be located within about 40 kilometers of the actual location. In the match against Renin Bolt, the AI easily won many rounds.
The degree to which the AI model pays attention to different areas in the image can be seen as a stain on the camera References[1]
"We're not the first AI to play against Raynebolt, we're just the first AI to beat Raynebolt. Michal Skreta, one of the developers of the AI system, said.
The location guessed by the AI is closer to the actual location | References[1]
They see the technology as having a variety of potential applications, such as the ability to identify roads or power lines that need to be repaired and quickly spot invasive plants. In future studies, they will further improve the model to improve the accuracy of positioning, and will expand the geolocation dataset to cover more geographic regions and environments to improve the model's generalization ability (the ability of the model to perform in the face of unseen data or new situations).
Possible risks
This app seems to be able to geolocate photos outside of Street View. "I gave it some photos from a road trip from over a decade ago, and it figured out most of them. It guessed a campground in Yellowstone, about 55 kilometers away. Another photo was taken on a street in San Francisco, and it guessed the location just a few blocks away. This has caused concern among some experts. Geoff Brumfiel, science editor at NPR, said.
In the face of increasingly powerful and unregulated AI, it seems that it is becoming more and more difficult to protect personal privacy security. From a privacy perspective, our location can be a very sensitive set of information. In the past, people could remove the GPS location tag of a photo, and now this may not work either.
With so much potential for a low-budget student program, businesses and larger institutions may be able to spy on ordinary people more easily, and those with bad intentions can more easily stalk and harm others.
Since you can't win against the AI, let's enjoy the game and guess where this photo came from?