When AI Knows Where You Are—Controlling Geolocation Privacy in Vision Language Models

Introduction

Imagine you upload a photo of your lunch to social media. You want your friends to know you are enjoying a trip to Paris, but you definitely don’t want strangers figuring out the exact street corner you are standing on, let alone the specific restaurant that might reveal your hotel’s location.

For years, privacy on the internet was often treated as binary: either you share something, or you don’t. However, with the rise of powerful Vision Language Models (VLMs) like GPT-4v, the line has blurred. These models possess an uncanny ability to analyze visual data—identifying landmarks, reading blurry text on background signs, and recognizing architectural styles—to pinpoint locations with frightening accuracy.

This capability, known as image geolocation, is no longer the domain of expert investigators or specialized software. It is an emergent property of the AI tools we use every day. This creates a massive privacy gap. How do we allow users to interact with these powerful models without accidentally doxxing themselves?

In this post, we dive deep into a research paper titled “Granular Privacy Control for Geolocation with Vision Language Models.” The researchers tackle this exact problem by introducing a framework for granular privacy—allowing users to set “dials” on how much location information an AI is allowed to reveal. They developed a new benchmark, collected extensive dialogue data, and fine-tuned models to act as privacy guards. Let’s explore how they did it and what it means for the future of AI safety.

The Geolocation Threat

Before we discuss the solution, we must understand the magnitude of the problem. You might assume that unless a photo contains the Eiffel Tower or the Statue of Liberty, an AI wouldn’t know where it was taken. This assumption is increasingly dangerous.

Modern VLMs are trained on vast swaths of the internet. They have “seen” millions of street views, restaurant menus, and architectural patterns. When prompted correctly, they can combine these subtle cues to deduce location coordinates.

The researchers demonstrated this by testing GPT-4v against state-of-the-art specialized geolocation models on the standard IM2GPS benchmark.

As shown in Figure 2, the results are startling. GPT-4v (specifically using a “Least-to-Most” prompting strategy) outperforms specialized models like GeoDecoder and PIGEOTTO. It achieves street-level accuracy (within 1 km) nearly 24% of the time. It also has the lowest median distance error of just 13 km.

This isn’t just about identifying landmarks. It involves complex reasoning: reading a phone number on a food truck, noticing the language on a poster, and recognizing the vegetation in the background. If a VLM can do this, any application built on top of it poses a potential privacy risk involving stalking, spear-phishing, or inferring a user’s broader activity patterns.

The Concept of Granular Privacy

The core contribution of this paper is moving away from a binary “safe/unsafe” toggle toward Granular Privacy Control.

Privacy norms are contextual. A travel influencer might want to share their exact coordinates to drive traffic to a location. A private citizen might want to share that they are in “Japan” but nothing more specific. The researchers propose a hierarchical approach to moderation, breaking location data down into five levels of granularity:

1. Country (e.g., United States)
2. City (e.g., Atlanta)
3. Neighborhood (e.g., Midtown)
4. Exact Location Name (e.g., The Varsity Restaurant)
5. GPS Coordinates (e.g., 33.77, -84.39)

The goal is to build a “Moderation Agent”—an AI system that sits between the user and the powerful VLM. This agent monitors the conversation and intervenes if the VLM is about to reveal information deeper than the user’s specified comfort level.

Figure 1 illustrates this workflow. On the left, a user (or attacker) asks questions about an image. The VLM (GPT-4v) generates a response. Before that response reaches the user, it passes through a Moderation Agent. This agent checks the response against the “Admin Configuration.” If the user set the privacy level to “City,” and the VLM tries to mention a specific food truck’s name, the agent flags the message and blocks it.

Building the Benchmark: GPTGEOCHAT

To train and test these moderation agents, the researchers needed data. Existing datasets weren’t sufficient because they usually focus on static image-to-GPS tasks, not the conversational back-and-forth where information leaks often happen.

They created GPTGEOCHAT, a dataset of 1,000 images and accompanying multi-turn conversations.

Data Collection

The researchers hired human annotators to collect stock images that resembled realistic social media photos—everyday scenes, street corners, and shop interiors, rather than just famous monuments. Crucially, 85% of these images contained text, testing the model’s OCR (Optical Character Recognition) capabilities alongside visual recognition.

The annotators then engaged in a dialogue with GPT-4v, trying to coax the location out of the model. They annotated every turn of the conversation, marking exactly when new location information was revealed and at what granularity.

Table 1 shows the diversity of these images. You can see examples ranging from a specific brewery in Germany to a political campaign in Texas. In the Texas example (top middle), the model uses the names on the political signs (“Julie Johnson,” “John Biggan”) to triangulate the location to Irving, Texas. This highlights the multi-step reasoning these models perform.

Synthetic Data Generation

Collecting high-quality human data is expensive (~$6.40 per dialogue). To scale up training, the researchers generated a synthetic dataset called GPTGEOCHAT-Synthetic.

They automated the process by having two instances of GPT-4v talk to each other:

1. The Questioner: Prompted to act as a “detective” using a Belief-Update technique. It maintains an internal belief of where the image is and asks questions to narrow it down (e.g., “I know it’s in France; what specific city has this cathedral?”).
2. The Answerer: Prompted with the ground-truth location data (to ensure it doesn’t hallucinate) and asked to answer visual questions.

This allowed them to create thousands of training examples at a fraction of the cost (~$0.26 per dialogue), which proved essential for fine-tuning the moderation models.

The Moderation Agents: Prompting vs. Fine-Tuning

With the dataset in hand, the researchers evaluated different ways to build the “Moderation Agent.”

1. Prompted Agents

The simplest approach is to take an off-the-shelf VLM (like GPT-4v, LLaVA, or IDEFICS) and give it a system prompt: “You are a content moderator. Does this response reveal the location more specifically than the City level?”

2. Fine-Tuned Agents

The more advanced approach involves taking a smaller open-source model (LLaVA-1.5-13b) and fine-tuning its weights specifically for this moderation task using the GPTGEOCHAT data. They trained separate versions of the model for each granularity level (one model is an expert at detecting City leaks, another at detecting Neighborhood leaks).

Experimental Results

The researchers evaluated the agents on two main fronts: Message-Level Moderation (looking at a single response in isolation) and Conversation-Level Moderation (looking at the flow of a chat).

Message-Level Performance

The primary metric here is the F1-score, which balances precision (not flagging safe messages) and recall (catching all unsafe messages).

Figure 3 reveals several key insights:

• Fine-Tuning Wins on Nuance: The fine-tuned agents (the green/teal bars) generally perform on par with or better than the massive GPT-4v (purple bar), particularly at the “Exact Location Name” level. This suggests that for specific policy enforcement, a smaller, specialized model is more effective than a giant generalist model.
• The Coordinate Anomaly: Look at the “GPS Coordinates” column on the far right. The simple Regex-Baseline (a regular expression script that just looks for number patterns like “33.74, -84.38”) outperforms almost every AI model. This highlights that sometimes, simple rule-based systems are better than AI for structured data detection.
• Prompting Open-Source Models Fails: The prompted versions of smaller models (like LLaVA-13b without fine-tuning) performed poorly, barely beating a random guess. They simply couldn’t follow the complex instruction without training.

Conversation-Level Trade-offs

A moderation agent faces a constant tug-of-war between Privacy (blocking leaks) and Utility (letting the user have a helpful conversation).

• Leaked Location Proportion: How often did the agent fail and let a secret slip?
• Wrongly Withheld Proportion: How often did the agent panic and block a harmless message?

Figure 4 visualizes this trade-off. The ideal agent would be at the bottom-left corner (0 leaks, 0 wrongly withheld).

• The Blue Region (Over-Censorship): Prompted GPT-4v and IDEFICS tend to be “paranoid.” They block too much information, favoring privacy at the cost of being helpful.
• The Pink Region (Under-Censorship): Prompted LLaVA tends to be too loose, letting information leak to maintain utility.
• The Sweet Spot: The fine-tuned agents (represented by the dots clustered near the center/diagonal) offer the best balance. They allow the conversation to flow while clamping down on specific leaks more effectively than the base models.

The “External Tools” Problem

There is a catch. Even if a moderation agent successfully blocks the VLM from saying “You are at 123 Main Street,” the VLM might still say, “You are at the statue of the founder, next to the bank.”

A clever user can take that “safe” text, plug it into Google Maps or a Geocoding API, and find the coordinates themselves. The researchers tested this by feeding the moderated dialogues into a Geocoding API to see if the location could still be recovered.

Figure 5 shows the Cumulative Density Function (CDF) of the prediction error. In this graph, a line that shoots up quickly on the left is bad—it means many images were geolocated to within a few kilometers (high accuracy) despite moderation. A line that stays low and flat is good—it means the moderation successfully kept the location vague.

The results are sobering. Even with the best moderation agent (Prompted GPT-4v in this specific test), about 2-3% of images could still be geolocated to within 20km using external tools. While this is a massive improvement over unmoderated chat, it shows that “safe” text descriptions can still be dangerous in the hands of a determined searcher. This is known as a multi-modal inference attack.

Conclusion and Future Implications

The paper “Granular Privacy Control for Geolocation with Vision Language Models” serves as a wake-up call and a roadmap. It proves that the “wait and see” approach to AI privacy is insufficient. VLMs are already capable of acting as digital detectives, and without intervention, they will inadvertently compromise user privacy.

The researchers successfully demonstrated that:

1. Granular control is possible: We don’t have to disable geolocation entirely; we can regulate it based on user preference.
2. Specialization beats scale: Small, fine-tuned models can moderate privacy as well as, or better than, massive proprietary models, offering a cheaper and more deployable solution.
3. Context is king: A simple keyword search isn’t enough. Models need to understand the dialogue history to know if a specific detail tips the scale from “vague” to “doxxed.”

However, the “External Tools” experiment highlights a lingering challenge. We are moving toward a world of Agentic AI—systems that can browse the web, use maps, and call APIs. As these systems become more integrated, preventing privacy leaks will require looking beyond just the text output of the model and considering the broader information ecosystem.

For students and researchers entering this field, this paper opens up a fascinating domain: Privacy-Preserving NLP. The challenge isn’t just making models smart; it’s making them discreet. The future of AI isn’t just about what it can tell you—it’s about what it knows to keep to itself.