AI Models Show Promise in Geolocation but Human Oversight Remains Crucial

Bellingcat has recently concluded a series of benchmark tests evaluating the geolocation capabilities of various large language models (LLMs) and other artificial intelligence systems. The investigation aimed to assess how these technologies perform compared to human experts in pinpointing the exact locations from which images were taken, particularly under challenging conditions.

Since June 2025, Bellingcat has been running periodic tests on AI geolocation tools as LLMs began to compete with traditional reverse image search methods for this task. Initial rounds of testing revealed that GPT-4o-mini from OpenAI was the frontrunner in early stages, but later iterations saw Google's AI mode taking the lead. However, the latest round of tests introduced a new level of complexity by providing models with increasingly difficult images and requiring them to provide only one set of coordinates.

The test included both closed-weight models such as Claude, Grok, Gemini, ChatGPT, and Mistral, alongside open-weight models from Moonshot AI's Kimi, commercial provider GeoSeer, and the non-generative model GeoClip. Each model was tasked with analyzing 52 unique photos that had never been published online before.

The results indicated a significant improvement in geolocation capabilities among current AI models. Notably, Anthropic’s Opus and Fable models, along with Google's Gemini 4 Argon, outperformed others by rapidly narrowing down locations to the correct general area under specific conditions. However, despite these advancements, no model achieved perfect accuracy in pinpointing exact coordinates.

The necessity of human oversight was highlighted as a critical factor due to the inherent limitations of AI models. While some models demonstrated impressive performance in identifying broad geographic areas, none could reliably determine precise locations with absolute certainty. This level of precision is crucial in contexts such as military operations where incorrect information can have severe consequences, potentially leading to significant loss of life.

Moreover, organizations like Bellingcat rely heavily on the accuracy and reliability of geolocation data for investigative journalism purposes. Their reputation hinges on delivering pinpoint factual accuracy, which underscores the importance of human verification alongside AI assistance.

To evaluate model performance rigorously, Bellingcat employed two metrics: GeoGuessr Score and Mean Log Score. The GeoGuessr Score, based on a popular online game, measures whether models can systematically identify general areas from images. However, this metric does not distinguish well between guesses that are close to the precise location versus those that are merely in the vicinity.

To address this limitation, Bellingcat introduced the Mean Log Score, which assesses how accurately models pinpoint locations within the final few kilometers of a search. This custom metric assigns scores from 0 (completely inaccurate) to 100 (perfect model), providing a more nuanced evaluation of AI geolocation capabilities.

In conclusion, while advancements in AI technology have significantly enhanced geolocation tasks, human expertise remains indispensable for ensuring accuracy and reliability in critical applications such as military operations and investigative journalism.