Multi-Agent AI Systems: Grounding with Google Maps in Genkit

In my previous article, we explored how to build a multi-agent AI concierge using Angular and Google's Genkit. We discussed the architectural benefits of the multi-agent pattern and how to use Google Search Retrieval to ground our agents in real-world data.

Today, we're taking it a step further. We'll explore Grounding with Google Maps, a powerful feature in Genkit that allows your AI to not only talk about locations but to provide interactive, contextual map experiences directly within your application.

What is Grounding with Google Maps?

Grounding with Google Maps allows Gemini to access real-time place information, coordinates, and reviews. More importantly, it can generate a Contextual Token that your frontend can use to render an interactive Google Maps widget.

Instead of just getting a text description of a restaurant, your user gets an interactive card with photos, ratings, and a "Get Directions" button; all powered by the same data the model used to generate its response.

Enabling the Google Maps APIs

Before you can use these features, you must enable the necessary APIs in the Google Cloud Project associated with your application. This ensures that the Maps JavaScript API and the related Place services are available to your application.

1. Go to the Google Cloud Console and select the project associated with your app.
2. Navigate to APIs & Services > Library.
3. Search for and enable the Maps JavaScript API.
4. Once enabled, go to the APIs & Services tab within the Maps JavaScript API dashboard and ensure the following are enabled (these are sub-APIs under the main one):
   • Places API
   • Places API (New)
   • Maps Embed API

Obtaining and Restricting the API Key

Once the APIs are enabled, you need a way to authenticate your requests while keeping your project secure.

1. Create the Key

• In the Google Cloud Console, navigate to Google Maps Platform > Keys & Credentials.
• Click Create Credentials and select API Key.
• Copy your new API key (you'll need it for the next step).

2. Restrict the Key (Crucial!)

To prevent unauthorized use of your key by devious actors, you must restrict it to only be usable by your specific websites.

• Click on the newly created key to edit its settings.
• Under Application restrictions, select Websites (HTTP referrers).
• Add your website URLs (e.g., https://your-app.web.app/* and http://localhost:4200/* for local development).
• Under API restrictions, select Restrict key and choose the Maps and Places APIs you enabled earlier.
• Click Save.

Backend Implementation: The "Find & Navigate" Agent

In our concierge system, we have a specialized agent called the findAndNavigateAgentTool. This tool is specifically configured to use Google Maps grounding.

1. Enabling the Google Maps Tool

In Genkit, you enable Maps grounding by adding it to the tools array in the ai.generate configuration.

export const _findAndNavigateAgentToolLogic = ai.defineTool(
  {
    name: 'findAndNavigateAgentTool',
    description: 'Assists with finding the best routes and transportation options',
    inputSchema: z.object({
      input: z.string(),
      history: z.array(conversationMessageSchema).optional(),
    }),
    outputSchema: z.object({
      text: z.string(),
      mapsWidgetToken: z.string().optional(),
    }),
  },
  async ({input, history}) => {
    const response = await ai.generate({
      system: TRANSPORT_AGENT_PROMPT,
      messages: [
        ...toGenkitMessages(history ?? []),
        {role: 'user', content: [{text: input}]},
      ],
      config: {
        tools: [
          {
            googleMaps: {enableWidget: true} // This is the magic line
          }
        ]
      },
    });

    // ... extraction logic
  }
);

2. Extracting the Maps Widget Token

When enableWidget: true is set, Gemini returns a googleMapsWidgetContextToken in the grounding metadata. We need to extract this token and pass it back to our Angular frontend.

const mapsWidgetToken = (response.custom as any)
  ?.candidates?.[0]
  ?.groundingMetadata
  ?.googleMapsWidgetContextToken as string | undefined;

return { text: response.text, mapsWidgetToken };

Frontend: Rendering the Interactive Widget

On the Angular side, we receive this mapsWidgetToken. To turn this token into a visual map, we use the Google Maps JavaScript API and specifically the PlaceContextualElement.

The Maps Widget Component

We created a standalone MapsWidget component that handles the library loading and element creation.

@Component({
  selector: 'app-maps-widget',
  template: `<div #mapElement class="map-container"></div>`,
  schemas: [CUSTOM_ELEMENTS_SCHEMA], // Needed for custom elements
})
export class MapsWidget implements AfterViewInit {
  @ViewChild('mapElement') container!: ElementRef<HTMLElement>;
  readonly token = input<string>(''); // The token from Genkit

  async ngAfterViewInit() {
    await this.mapsLoader.importLibrary('places');
    this.renderWidget(this.token());
  }

  private renderWidget(token: string) {
    const places = (window as any)['google']?.maps?.places;

    // Create the contextual element using the token
    const el = new places.PlaceContextualElement({ 
      contextToken: token 
    });

    this.container.nativeElement.appendChild(el);
  }
}

Why use PlaceContextualElement?

The PlaceContextualElement (or <gmp-place-contextual>) is a specialized Web Component provided by Google Maps. It’s designed to work with these grounding tokens. It ensures that the place displayed on the map is exactly what the AI was talking about, maintaining "chain of custody" for the information.

Securely Loading the Maps API

To render the PlaceContextualElement, we need to load the Google Maps JavaScript API. Instead of hardcoding the API key in our frontend, we fetch it dynamically from a secure Firebase Function. This allows us to keep the key as a Firebase Secret and only expose it to authenticated users if needed.

1. The Backend: Providing the API Key

To ensure that the API keys are not committed to the repository, we store them as Firebase Secrets. This keeps them out of our source code and only injects them into the function's environment at runtime.

Step 1: Deploy the Secret

Use the Firebase CLI to securely upload your API key:

firebase functions:secrets:set MAPS_API_KEY

When prompted, paste the API key you generated in the Google Cloud Console.

Step 2: Access the Secret in your Code

In our functions/src/index.ts, we define the secret and then retrieve its value within our onCall function.

import { defineSecret } from 'firebase-functions/params';

// Define the secret
const MAPS_API_KEY = defineSecret('MAPS_API_KEY');

export const loadGoogleMaps = onCall(
  {
    ...GENKIT_FUNCTION_CONFIG,
    secrets: [MAPS_API_KEY], // Explicitly grant the function access to this secret
  },
  () => {
    // Access the value securely
    return { key: MAPS_API_KEY.value() };
  }
);

2. The Frontend: The GoogleMapsLoaderService

We created a central service in Angular to handle the asynchronous loading of the Maps libraries. It uses the @googlemaps/js-api-loader package for a clean, Promise-based initialization.

@Injectable({ providedIn: 'root' })
export class GoogleMapsLoaderService {
  private readonly functions = inject(Functions);
  private initialized = false;

  private async ensureInitialized() {
    if (this.initialized) return;

    // 1. Fetch the API key from our Firebase Function
    const loadGoogleMaps = httpsCallable<unknown, { key: string }>(
      this.functions,
      'loadGoogleMaps'
    );
    const { data } = await loadGoogleMaps();

    // 2. Configure the JS API Loader with the key
    setOptions({ 
      key: data.key, 
      v: 'alpha', // Contextual elements often require alpha/beta versions
      libraries: ['places'] 
    });

    this.initialized = true;
  }

  async importLibrary(library: string) {
    await this.ensureInitialized();
    return importLibrary(library);
  }
}

Why this approach?

1. Security: The API key isn't stored in environment.ts or hardcoded in index.html. It stays in Firebase Secrets.
2. On-Demand Loading: We only load the Maps JS SDK when it's actually needed (e.g., when the first map widget is about to render).
3. Consistency: Centralizing the loader ensures that all components use the same API version and configuration.

Key Features & Benefits

1. Contextual Accuracy: The widget isn't just a random map search; it's linked to the specific place the AI identified.
2. Interactive UX: Users can see ratings, hours, and photos without leaving your chat interface.
3. Low Friction: You don't need to manually manage Place IDs or coordinates; Genkit and the Maps API handle the handshake via the token.
4. Modern Angular Integration: By using Signals and input(), the component reacts instantly to new tokens generated during a conversation.

Conclusion

Grounding with Google Maps transforms your AI from a text-based assistant into a rich, spatial guide. By combining Genkit's powerful tool-calling orchestration with Angular's component-based architecture, you can build concierge experiences that feel truly native to the physical world.

Check out the project, Concierge AI, here: https://agents-concierge.web.app/
GitHub Repo: https://github.com/waynegakuo/concierge

Happy Coding!