The AI That Can Guess Where a Photo Was Taken

Have you ever looked at a photo online and wondered where in the world it was taken? You're not alone. Pinpointing the location of a photo is something even humans find tricky. But scientists are developing AI systems that can figure it out.

Researchers have created two AI models called PIGEON and PIGEOTTO that can predict where a photo was taken just by looking at it. The technology could one day power apps that automatically tag where your vacation photos were taken. But how does it work?

Playing a Guessing Game

PIGEON was trained using images from Google Street View to get good at the online game Geoguessr. This game shows you photos of random street scenes and you have to guess the location on a map. PIGEON was able to rank in the top 0.01% of human players!

Geolocalization error of PIGEON against human players of various in-game skill levels across 458 multi-round matches. The Champion Division consists of the top 0.01% of players. PIGEON’s error is higher than in Table 1 because Geoguessr round difficulties are adjusted dynamically, increasing with every round.

Image Source: Haas, L., Skreta, M., Alberti, S., & Finn, C. (2023). PIGEON: Predicting Image Geolocations. arXiv preprint arXiv:2307.05845.

The AI looks for little clues in the photos like signs, vegetation, buildings, and terrain. It also has learned strategies from guides that top Geoguessr players use. This allows it to make smart guesses about the country, region, or city based on subtle details.

Seeing the World Through Photos

Meanwhile, PIGEOTTO was trained on a dataset of over 4 million photos from Flickr and Wikipedia. This allowed it to learn from a diverse set of scenes worldwide.

Remarkably, PIGEOTTO is the first system that can generalize well, meaning it can locate new photos it wasn't directly trained on. Prior AI models struggled with this.

Both systems use a clever approach called multi-task learning. This allowed them to learn from geographic data and captions in addition to photos, giving more context.

Pinpointing Places Globally

Tests showed PIGEON could get within 25 km of the right spot 40% of the time when analyzing Google Street View scenes. And PIGEOTTO achieved record results on benchmark tests, cutting the median distance error by up to 50% compared to previous AI systems.

This level of accuracy could open up new possibilities for image geotagging, travel apps, social media, navigation systems, and more.

But with any new technology, there is also potential for misuse. The researchers caution that safeguards should be put in place to prevent military or invasive uses of the AI. Overall though, these advanced systems demonstrate AI's potential to pinpoint images globally.

So the next time you're wondering where a photo was taken, AI might soon provide the answer. These virtual geo-guessing whizzes are only getting smarter!

Key Takeaways

AI systems like PIGEON and PIGEOTTO can predict the geographic location of a photo using visual clues.
Multi-task learning allows the AI to learn from diverse data like captions and maps, improving accuracy.
PIGEON ranked in the top 0.01% of human players at the geo-guessing game Geoguessr.
PIGEOTTO achieved state-of-the-art results on benchmark tests, cutting median errors by up to 50%.
The technology could have applications in travel, social media, navigation, and more.
Safeguards against misuse are needed as precision improves.

This article was supported by KoalaAI, a high quality SEO-optimiser, powered by GPT4, it combines SERP analysis with real-time data to support in creating content that ranks.

Check out our list of over 1,000 custom GPTs here.

Looking for prompts? We have the world's best prompts here.

Check out our image prompt database here.

Want more blogs? Find more here.

Citation: Haas, L., Skreta, M., Alberti, S., & Finn, C. (2023). PIGEON: Predicting Image Geolocations. arXiv preprint arXiv:2307.05845.