DEV Community: Connie Leung

Build Real-Time Client-Side TTS in Angular Using Firebase AI Logic and Gemini

Connie Leung — Tue, 01 Sep 2026 10:52:37 +0000

In my previous article on Dev.to, I showed how to build expressive text-to-speech using Gemini and Firebase Cloud Functions. While that setup worked beautifully, it required building and deploying custom backend endpoints. Firebase AI Logic enables running Gemini TTS client-side with production security, eliminating custom backend code and automating deployment via Git integration.

1. Technical Stack

Project technical stack:

Angular 22: The latest version as of August 2026.
Node.js LTS: The LTS version as of May 2026.
Firebase Remote Config: Manages dynamic parameters, such as TTS model names, and conditionally configures limited-use tokens.
Firebase AI Logic: Interfaces with Gemini generative models.
Firebase App Check: Prevents abuse and verifies token authenticity.
Firebase App Hosting: Streamlined deployment to the Firebase App Hosting environment. With a single code push, the latest changes deploy to the production environment.

The public Google Gemini Developer API is restricted in my region (Hong Kong). However, the Agent Platform Gemini API (Google Cloud) offers enterprise access that works reliably here, so I chose the Agent Platform Gemini API for this Firebase AI Logic demo.

1.1 Install global dependency

npm install -g firebase-tools

Install or update firebase-tools globally using npm.

firebase logout

firebase login

Log out and re-authenticate with Firebase.

1.2 Install dependencies

npm i --save-exact firebase
npm i --save-exact --save-dev firebase-tools serve

Install the dependencies to call the Firebase AI Logic API, use the Firebase CLI to generate files, and serve the production build.

firebase init

Execute firebase init and follow the prompts to set up the Firebase AI Logic, Emulators, App Hosting, and Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

After completing the setup steps, the Firebase tools generate the configuration files such as .firebaserc and firebase.json. You can view the .firebaserc and firebase.json in the GitHub repo.

1.3. Generate Firebase Configuration File

We asked antigravity-cli (a terminal-first AI coding agent released by Google) and the Gemini Flash model to create two Node.js scripts to do the following:

Generate default Remote Config values in JSON format and save them to public/remote-config-defaults.json. You can read the complete listing of the script.
Populate the Firebase configuration object and write it to public/firebase.config.json. If we hardcode the public keys and push them to GitHub, GitHub triggers a false alarm. You can read the complete listing of the script and the environment variable template.

# generated firebase configuration
firebase.config.json

Add firebase.config.json to .gitignore to prevent accidental commits.

During build time, Angular bundles both JSON files into the dist directory to provide initial configuration values.

2. Architecture

Users submit text to Firebase AI Logic to synthesize speech. Firebase generates the complete L16 audio payload and returns it to the client. Because the HTML audio element does not support the L16 format, the application converts the audio to a WAV Blob before binding the Blob URL to the element source.

The second flow streams the audio and sends the L16 chunks to the Angular application. The audio player creates an AudioBufferSourceNode to play the chunk data and cleans up resources to prevent memory leaks.

3. Source Code

While the full codebase is available in the ng-firebase-tts repository, the application relies on Firebase Remote Config to manage configuration, App Check to prevent abuse, and App Hosting to deploy the Angular application.

The following sections illustrate how to initialize the Firebase app and App Check, and how to activate Remote Config values.

4. Firebase Integration

The public/firebase.config.json file contains the public Firebase API key, the sensitive reCAPTCHA Enterprise key, and the App Check debug token to bypass device attestation in local development. These values are critical in Firebase App and App Check initialization.

4.1. Firebase Infrastructure

@Service()
export class ConfigService {
  #app: FirebaseApp | undefined = undefined;
  #remoteConfig: RemoteConfig | undefined = undefined;

  /*... getter methods are omitted... */

  get appConfig(): AppRemoteConfig {
    return this.#appConfig;
  }

  get aiBackend(): AI {
    return this.#aiBackend;
  }

  async initialize(): Promise<void> {
    this.#app = initializeApp(firebaseConfig.app);

    (globalThis as any).FIREBASE_APPCHECK_DEBUG_TOKEN = firebaseConfig.appCheckDebugToken || true;

    initializeAppCheck(this.#app, {
      provider: new ReCaptchaEnterpriseProvider(firebaseConfig.recaptchaEnterpriseKey),
      isTokenAutoRefreshEnabled: true,
    });

    this.#remoteConfig = getRemoteConfig(this.#app);
    this.#remoteConfig.defaultConfig = remoteConfigDefaults;
    await fetchAndActivate(this.#remoteConfig);

    this.#appConfig = {
      vertexAILocation: getValue(this.#remoteConfig, 'vertexAILocation').asString(),
      useLimitedUseAppCheckTokens: getValue(
        this.#remoteConfig,
        'useLimitedUseAppCheckTokens',
      ).asBoolean(),
      geminiTTSModelName: getValue(this.#remoteConfig, 'geminiTTSModelName').asString(),
    };

    this.#aiBackend = getAI(this.#app, {
      backend: new AgentPlatformBackend(this.#appConfig.vertexAILocation),
      useLimitedUseAppCheckTokens: this.#appConfig.useLimitedUseAppCheckTokens,
    });
  }
}

The initialize method initializes Firebase App, configures App Check, sets up Firebase AI, and assigns Remote Config values to appConfig.

The appConfig object holds the location, the Gemini TTS model name, and the limited-use App Check token flag.

4.2. Remote Config and Construct Agent Platform Backend

Configure the TTS model name and the limited-use App Check token parameters in Firebase Remote Config. Both parameters have conditional values. When the Firebase web application is firebase-ai-logic-tts, the TTS model is gemini-3.1-flash-tts-preview, and the limited-use App Check token flag is set to true.

export const AI_BACKEND = new InjectionToken<AI>('AI_BACKEND');

export function provideFirebase() {
  return makeEnvironmentProviders([
    {
      provide: AI_BACKEND,
      useFactory: () => inject(ConfigService).aiBackend,
    },
  ]);
}

The AI_BACKEND injection token provides a factory function to return the Firebase AI from ConfigService.

export const appConfig: ApplicationConfig = {
  providers: [
    ... other providers ...
    provideAppInitializer(async () => await inject(ConfigService).initialize()),
    provideFirebase(),
  ],
};

provideAppInitializer and provideFirebase initialize Firebase and configure Firebase AI during application startup.

5. TextToSpeechService Definition in Angular

The private createModel method calls getGenerativeModel to create a generative model configured with speechConfig. The subsequent workflows use this model to convert L16 audio to WAV or stream L16 playback directly.

@Service()
export class TextToSpeechService {
  readonly #configService = inject(ConfigService);
  readonly #modelName = this.#configService.appConfig.geminiTTSModelName;

  private createModel(voiceName: string) {
    return getGenerativeModel(this.#aiBackend, {
      model: this.#modelName,
      generationConfig: {
        responseModalities: [ResponseModality.AUDIO],
        speechConfig: {
          voiceConfig: {
            prebuiltVoiceConfig: { voiceName },
          },
          languageCode: 'en-US',
        },
      },
    });
  }
}

5.1 Flow 1: L16 Audio to WAV conversion

async synthesize(text: string, voiceName: string): Promise<Blob> {
  const model = this.createModel(voiceName);
  const result = await model.generateContent([text]);
  const chunk = this.extractValidChunkData(result.response);
  const { data, mimeType } = chunk;
  return convertToWav(decodeBase64(data), mimeType);
}

The extractValidChunkData method extracts binary audio data and the MIME type from the response payload.

The synthesize method retrieves the entire audio payload in L16 format. However, the HTML audio element does not support L16, so the application must convert the data to WAV format before assigning the Blob URL to the element source.

See the conversion to WAV code for the full implementation.

This implementation remains straightforward because it avoids managing response streams and incremental audio chunks. However, users suffer from latency when the text is long and it produces a long audio stream. To eliminate playback latency, the next section explores streaming chunks directly via the Web Audio API AudioContext.

5.2 Flow 2: L16 Audio Instant Playback

async *synthesizeStream(text: string, voiceName: string): AsyncGenerator<RawAudioBinary | undefined> {
    const model = this.createModel(voiceName);
    let firstMimeType = '';
    let sampleRate = DEFAULT_SAMPLE_RATE;

    const responseStream = await model.generateContentStream([text]);
    for await (const chunk of responseStream.stream) {
      const chunkData = this.extractValidChunkData(chunk);
      if (chunkData) {
        const { data, mimeType } = chunkData;
        const decodedData = decodeBase64(data);
        if (!firstMimeType && mimeType) {
          firstMimeType = mimeType;
          sampleRate = parseMimeType(firstMimeType).sampleRate;
        }

        yield { decodedData, sampleRate };
      }
    }

    yield undefined;
}

The synthesizeStream method returns an asynchronous generator that yields the raw binary data and the sample rate. The MIME type is audio/l16; rate=24000; channels=1, so parseMimeType extracts the sample rate for the audio player's AudioContext.

While AudioPlayerService is beyond the scope of this article, the source code demonstrates how to play back audio without using an HTML audio element.

Next, we will build a reactive user interface in Angular that renders an HTML audio element to play audio.

6. Build Reactive UI in Angular

The TextToSpeechComponent delegates audio management to the view service to maintain separation of concerns.

6.1 Text to Speech Component

The TextToSpeechComponent displays three buttons to generate speech from text across three different scenarios:

Scenario 1: Synthesizes the entire audio payload and binds the resulting Blob URL to an HTML audio element.
Scenario 2: Streams audio chunks incrementally and plays the speech immediately via the Web Audio API. This mode hides the HTML audio element because audio plays directly through the Web Audio API.

@Component({
  selector: 'app-text-to-speech',
  templateUrl: './text-to-speech.component.html',
  styleUrl: './text-to-speech.component.css',
  imports: [SpinnerIconComponent, NgTemplateOutlet],
  providers: [TextToSpeechViewService],
})
export class TextToSpeechComponent {
  private readonly speechService = inject(TextToSpeechViewService);

  interestingFact = input<string | undefined>(undefined);
  audioPrompt = input.required<string>();
  voice = input.required<string>();

  async generateSpeech(mode: GenerateSpeechMode) {
    const fact = this.interestingFact();
    await this.speechService.generateSpeech(mode, {
      prompt: this.audioPrompt(),
      voice: this.voice(),
      fact: this.interestingFact(),
    });
  }
}

6.2 TextToSpeechView Definition in Angular

The TextToSpeechViewService encapsulates TextToSpeechService and AudioPlayerService to coordinate speech synthesis and audio playback.

@Injectable()
export class TextToSpeechViewService {
  private readonly speechService = inject(TextToSpeechService);
  private readonly audioPlayerService = inject(AudioPlayerService);

  #audioUrl = signal<string | undefined>(undefined);
  audioUrl = this.#audioUrl.asReadonly();

  private processStreamChunk(isInitialized: boolean, playbackRate: number, chunk: RawAudioBinary) {
    if (!isInitialized) {
      this.audioPlayerService.initialize(chunk.sampleRate, playbackRate);
      isInitialized = true;
    }
    this.audioPlayerService.processChunk(chunk.decodedData);
    return isInitialized;
  }

  private async handleSync(promptArgs: FactConfig) {
    const blob = await this.speechService.synthesize(promptArgs.prompt, promptArgs.voice);
    this.setAudioUrl(blob);
  }

  private async handleStream(promptArgs: FactConfig) {
    let isInitialized = false;    
    const { prompt, voice } = promptArgs;

    for await (const chunk of this.speechService.synthesizeStream(prompt, voice)) {
      isInitialized = this.processStreamChunk(isInitialized, 1, chunk);
    }
  }

  private setAudioUrl(finalBlob: Blob | undefined) {
    if (finalBlob) {
      const createdUrl = URL.createObjectURL(finalBlob);
      this.#audioUrl.set(createdUrl);
      return createdUrl;
    }
    return undefined;
  }

  async generateSpeech(mode: GenerateSpeechMode, promptArgs: FactConfig) {
    revokeBlobURL(this.#audioUrl());
    this.#audioUrl.set(undefined);

    switch (mode) {
      case 'sync':
        await this.handleSync(promptArgs);
        break;
      case 'web_audio_api':
        await this.handleStream(promptArgs);
        break;
    }
  }
}

handleSync invokes the Firebase SDK to synthesize audio using the Gemini TTS model, generates a Blob URL, updates #audioUrl, and renders the HTML audio element.

handleStream calls the Firebase SDK to use the Gemini TTS model to stream the audio instead. The audio context receives the chunk data, plays it immediately, and avoids rendering the HTML audio element.

Conclusion

The integration of text-to-speech with Firebase AI Logic empowers Angular applications for real-time audio generation.

The Angular application handles text-to-speech entirely on the client. Pushing changes to Git triggers an automatic deployment to Firebase App Hosting.

Try cloning the GitHub repository, uploading an image to generate an obscure fact, and using the Gemini 3.1 Flash TTS preview model to speak it with the specified scene, emotion, and pace.

Resources

Go Completely Offline: Build a Privacy-First Personal Finance Assistant with LiteRT and Gemma 4

Connie Leung — Thu, 27 Aug 2026 01:07:01 +0000

Imagine having a personal financial planner that analyzes your sensitive transaction history, flags spending anomalies, and suggests budget optimizations—all while running 100% offline, directly inside your web browser.

In this guide, we will build an offline Retrieval-Augmented Generation (RAG) assistant powered by the LiteRT LM Web API and the highly efficient, on-device Gemma 4 E2B small language model. By caching the model weights directly in the browser's Cache Storage, your application will achieve complete privacy, zero token costs, and genuine offline capability.

1. Prerequisites

LiteRT LM Web API
Gemma 4 E2B model
Angular 22
TailwindCSS
Node 24

While Angular and TailwindCSS provide the user interface and styling, the LiteRT LM Web API and the on-device small model power this demo's AI capabilities.

npm i --save-exact @litert-lm/core tailwindcss postcss @tailwindcss/postcss jsonrepair dexie
npm i --save-exact --save-dev angular-eslint husky lint-staged serve @commitlint/cli

We install additional dependencies for on-device AI, CSS Styling, JSON response streaming, and IndexedDB storage. The dev dependencies help automatically catch code smells, enforce code quality, and serve the production bundle.

2. Architecture

We designed this application to process transactions entirely on-device, ensuring financial data privacy inside the browser. The process operates in three key phases:

Model and Data Storage: First, the application downloads and caches the Gemma 4 E2B model weights directly in the browser's Cache Storage so they are available offline. When users log expenses, they are saved locally in IndexedDB.
Arithmetic Grounding: Small models hallucinate when processing arithmetic queries, so we aggregate expenses into a monthly total, a daily total, and an expense-by-category breakdown. We supply the precomputed values to the Gemma WebGPU engine as ground truth, along with the user query.
Conversation Management: To prevent exceeding browser token limits, the application resets the conversation after three turns but retains a client-side memory of the last two queries. On reset, the application re-primes the model with immutable financial grounding data before replaying those queries. This maintains conversational continuity while keeping memory usage within on-device limits.

Next, let's look at how we instantiate this LiteRT LM engine, define the insight service, and design the Angular user interfaces to generate these insights.

3. Source Code

While the full codebase is available in the NG Personal Finance Assistant repository, our application relies on an IndexedDB database to track expenses offline. You can find the database implementation in the service file, which is injected via an injection token and initialized at startup using provideAppInitializer.

With this local database configured to log expenses, we can now focus on the AI integration. The following sections illustrate how to cache the Gemma 4 weights and leverage Angular 22's reactive features to generate secure, local insights directly from this transaction data.

4. Gemma Model Caching and Engine Initialization

Running LLMs locally in the browser requires two steps:

Caching the Weights: Storing the model weights (~2GB) in the browser's Cache Storage so the application can run 100% offline.
Engine Bootstrapping: Creating a single, reusable instance of the LiteRT LM engine so we can stream responses with WebGPU speed and avoid costly re-initialization latencies.

4.1 Caching Gemma 4 to Cache Storage in the Browser

When discussing on-device AI, people often assume offline operation, zero server dependency, and total privacy. However, this is only partially true for Web AI. Most tutorials make a fetch request to retrieve the Gemma model from HuggingFace, which works well when online. This fails if the user's connection is unstable or unavailable.

This illustrates the benefit of caching a small Gemma model in Cache Storage. The device only needs to be online once to download the model, and the application runs 100% offline.

The demo has a download button for users to download the Gemma 4 E2B model that is around 2GB.

After a successful download, go to Application > Storage > Usage to verify that the model occupies 2GB in storage.

Let's implement this logic in AiModelCacheService:

export const AI_CACHE_NAME = 'JMWebAIModels';
export const DEFAULT_MODEL_FILENAME = 'gemma-4-E2B-it-web.litertlm';
export const GEMMA_MODEL_URL = `https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/${DEFAULT_MODEL_FILENAME}`;

@Service()
export class AiModelCacheService {
  readonly #modelUrl = GEMMA_MODEL_URL;
  readonly #state = signal({ status: 'not-downloaded' });

  async downloadModel(): Promise<string> {
    if (this.#state().status === 'cached') {
      return await this.getModelUrl();
    }

    this.#state.set({ status: 'downloading' });
    const localBlobUrl = await FileProxyCache.loadFromURL(this.#modelUrl);
    this.#state.set({ status: 'cached' });

    return localBlobUrl;
  }

  async getModelUrl(): Promise<string | null> {
    return await FileProxyCache.loadFromURL(this.#modelUrl);      
  }
}

The AiModelCacheService class leverages FileProxyCache (an open-source utility by Jason Mayes designed specifically for caching large Web AI weights). Instead of using a standard browser fetch (which can crash or fail with large ~2GB assets), FileProxyCache handles chunked downloads and saves the Gemma model directly to the browser's Cache Storage. In subsequent runs, it instantly loads the cached local file and passes its blob URL to the LiteRT engine, completing the offline capability.

4.2. Lazy-Initializing the LiteRT LM Engine

The application has a single instance of the LiteRT LM engine. Initializing and destroying an engine for an individual query is an expensive operation. This introduces latency while waiting for the engine to become available, start a new conversation, and accept a message.

We lazily initialize the engine using GemmaEngineService, which we then inject into our insight service in the next section.

@Service()
export class GemmaEngineService {
  readonly #cacheService = inject(AiModelCacheService);
  #engine: Engine | null = null;

  private async initializeEngine(): Promise<Engine> {
    const localBlobUrl = await this.#cacheService.getModelUrl();
    const instance = await Engine.create({
      model: localBlobUrl,
      mainExecutorSettings: {
        maxNumTokens: 4096,
      },
    });

    this.#engine = instance;
    return instance;
  }

  getEngine(): Promise<Engine> {
    if (this.#engine) {
      return Promise.resolve(this.#engine);
    }

    return this.initializeEngine();
  }

  ... other methods and lifecycle methods ...
}

If initialized, the method returns the engine immediately. Otherwise, the initializeEngine helper method loads the model from Cache Storage, constructs the LiteRT LM engine, and returns it.

5. Insight Service Definition in Angular

We define the system, priming, and user prompts to analyze and derive deep insights into personal finance and spending. You can view our exact prompts here.

To ensure high-quality, structured output from Gemma 4, the model must return a JSON response adhering to the InsightsResponse interface. You can view the response and the Insight interface here.

Let's implement our InsightService to coordinate the streaming process:

@Service()
export class InsightService {
  readonly #engineService = inject(GemmaEngineService);

  #conversation: Conversation | null = null;
  #lastPrimedExpenses: Expense[] | null = null;
  #turnsCount = 0;
  #previousQueries: string[] = [];

  async *streamInsights(userQuery: string, expenses: Expense[]): AsyncGenerator<InsightsResponse> {

    const isContextDifferent = this.#lastPrimedExpenses !== expenses;
    const isContextExhausted = this.#turnsCount >= 3;

    // Reset & prime context when threshold is met
    if (!this.#conversation || isContextDifferent || isContextExhausted) {
      await this.primeContext(expenses, isContextExhausted ? this.#previousQueries : undefined);
    }

    const userPrompt = INSIGHTS_USER_PROMPT(userQuery);
    const stream = await this.#conversation.sendMessageStreaming(userPrompt);

    this.#previousQueries.push(userQuery);
    this.#turnsCount = this.#turnsCount + 1;

    // Yield repaired, streamed JSON chunks (using jsonrepair)       
    yield* this.processStream(stream);
  }

  /* processStream, lifecycle, and other helper methods */
}

Our primeContext method formats transaction data, computes high-level spending statistics, and appends them to the query context—guaranteeing that Gemma 4 generates a reliable, grounded JSON response.

The processStream method returns an asynchronous generator that iterates over the stream to extract each chunk's content. Because LLM response streaming delivers data fragment-by-fragment, the raw buffer is frequently incomplete (e.g., an open brace or unclosed bracket). Attempting to run standard JSON.parse() on these partial chunks would immediately throw a syntax error. To solve this, the jsonrepair library dynamically patches and closes the partial JSON string on the fly, allowing the application to successfully parse and render UI updates incrementally.

6. Reactive UI Construction in Angular

6.1 History Insights Facade

HistoryInsightsService acts as a clean facade between our UI component and the core InsightService, letting us easily query expenses and stream insights based on user questions.

Our component injects this service to trigger and render the JSON response asynchronously.

Source code of HistoryInsightsService

6.2 History Insights Component

HistoryInsightsComponent comprises child components and a pipe. However, the primary method that communicates with Gemma 4 to generate a list of insights is onAskGemma.

@Component({
  selector: 'app-history-insights',
  ...
})
export default class HistoryInsightsComponent {
  protected readonly vm = inject(HistoryInsightsService);

  readonly expenses = signal<Expense[]>([]);
  readonly streamingResponse = signal<InsightsResponse>({ insights: [] });

  async onAskGemma(query: string): Promise<void> {
    const trimmed = query.trim();
    if (!trimmed) {
      return;
    }

    this.streamingResponse.set({ insights: [] });

    // The expenses are populated after a search query
    // Consume the stream generator and update the UI signal incrementally
    const generator = this.vm.streamInsights(trimmed, this.expenses());
    for await (const response of generator) {
      this.streamingResponse.set(response);
    }
  }
}

The onAskGemma event handler invokes HistoryInsightsService's streamInsights method to retrieve the streamed response and incrementally updates the streamingResponse signal. The derived aiState signal computes the current status, captures any errors, and binds the values directly to the child component to render the streamed response in real-time.

The full listing of HistoryInsightsComponent

7. Conclusion

This concludes the journey of building an on-device personal finance assistant using the LiteRT LM Web SDK, the Gemma 4 E2B small-scale model, and Angular. This blog post shows how to build an offline RAG-based application by caching the model weights locally. Even when Wi-Fi is turned off, the personal finance assistant provides insights for user queries.

We mitigated the lack of tool-calling capability in the API and the mathematical hallucinations of small models by providing precomputed aggregated data as the LLM context. This prevents incorrect "60 + 40.5 = 200.2" expressions in the final results.

Engineers can build web applications with local models to perform financial analysis without leaking personal data to the training datasets of AI vendors.

Resources

Go Completely Offline: Build a Privacy-First Personal Finance Assistant with LiteRT and Gemma 4 [GDE]

Connie Leung — Thu, 27 Aug 2026 01:00:03 +0000

1. Prerequisites

LiteRT LM Web API
Gemma 4 E2B model
Angular 22
TailwindCSS
Node 24

While Angular and TailwindCSS provide the user interface and styling, the LiteRT LM Web API and the on-device small model power this demo's AI capabilities.

npm i --save-exact @litert-lm/core tailwindcss postcss @tailwindcss/postcss jsonrepair dexie
npm i --save-exact --save-dev angular-eslint husky lint-staged serve @commitlint/cli

2. Architecture

We designed this application to process transactions entirely on-device, ensuring financial data privacy inside the browser. The process operates in three key phases:

Model and Data Storage: First, the application downloads and caches the Gemma 4 E2B model weights directly in the browser's Cache Storage so they are available offline. When users log expenses, they are saved locally in IndexedDB.
Arithmetic Grounding: Small models hallucinate when processing arithmetic queries, so we aggregate expenses into a monthly total, a daily total, and an expense-by-category breakdown. We supply the precomputed values to the Gemma WebGPU engine as ground truth, along with the user query.
Conversation Management: To prevent exceeding browser token limits, the application resets the conversation after three turns but retains a client-side memory of the last two queries. On reset, the application re-primes the model with immutable financial grounding data before replaying those queries. This maintains conversational continuity while keeping memory usage within on-device limits.

Next, let's look at how we instantiate this LiteRT LM engine, define the insight service, and design the Angular user interfaces to generate these insights.

3. Source Code

4. Gemma Model Caching and Engine Initialization

Running LLMs locally in the browser requires two steps:

Caching the Weights: Storing the model weights (~2GB) in the browser's Cache Storage so the application can run 100% offline.
Engine Bootstrapping: Creating a single, reusable instance of the LiteRT LM engine so we can stream responses with WebGPU speed and avoid costly re-initialization latencies.

4.1 Caching Gemma 4 to Cache Storage in the Browser

This illustrates the benefit of caching a small Gemma model in Cache Storage. The device only needs to be online once to download the model, and the application runs 100% offline.

The demo has a download button for users to download the Gemma 4 E2B model that is around 2GB.

After a successful download, go to Application > Storage > Usage to verify that the model occupies 2GB in storage.

Let's implement this logic in AiModelCacheService:

export const AI_CACHE_NAME = 'JMWebAIModels';
export const DEFAULT_MODEL_FILENAME = 'gemma-4-E2B-it-web.litertlm';
export const GEMMA_MODEL_URL = `https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/${DEFAULT_MODEL_FILENAME}`;

@Service()
export class AiModelCacheService {
  readonly #modelUrl = GEMMA_MODEL_URL;
  readonly #state = signal({ status: 'not-downloaded' });

  async downloadModel(): Promise<string> {
    if (this.#state().status === 'cached') {
      return await this.getModelUrl();
    }

    this.#state.set({ status: 'downloading' });
    const localBlobUrl = await FileProxyCache.loadFromURL(this.#modelUrl);
    this.#state.set({ status: 'cached' });

    return localBlobUrl;
  }

  async getModelUrl(): Promise<string | null> {
    return await FileProxyCache.loadFromURL(this.#modelUrl);      
  }
}

4.2. Lazy-Initializing the LiteRT LM Engine

We lazily initialize the engine using GemmaEngineService, which we then inject into our insight service in the next section.

@Service()
export class GemmaEngineService {
  readonly #cacheService = inject(AiModelCacheService);
  #engine: Engine | null = null;

  private async initializeEngine(): Promise<Engine> {
    const localBlobUrl = await this.#cacheService.getModelUrl();
    const instance = await Engine.create({
      model: localBlobUrl,
      mainExecutorSettings: {
        maxNumTokens: 4096,
      },
    });

    this.#engine = instance;
    return instance;
  }

  getEngine(): Promise<Engine> {
    if (this.#engine) {
      return Promise.resolve(this.#engine);
    }

    return this.initializeEngine();
  }

  ... other methods and lifecycle methods ...
}

If initialized, the method returns the engine immediately. Otherwise, the initializeEngine helper method loads the model from Cache Storage, constructs the LiteRT LM engine, and returns it.

5. Insight Service Definition in Angular

We define the system, priming, and user prompts to analyze and derive deep insights into personal finance and spending. You can view our exact prompts here.

To ensure high-quality, structured output from Gemma 4, the model must return a JSON response adhering to the InsightsResponse interface. You can view the response and the Insight interface here.

Let's implement our InsightService to coordinate the streaming process:

@Service()
export class InsightService {
  readonly #engineService = inject(GemmaEngineService);

  #conversation: Conversation | null = null;
  #lastPrimedExpenses: Expense[] | null = null;
  #turnsCount = 0;
  #previousQueries: string[] = [];

  async *streamInsights(userQuery: string, expenses: Expense[]): AsyncGenerator<InsightsResponse> {

    const isContextDifferent = this.#lastPrimedExpenses !== expenses;
    const isContextExhausted = this.#turnsCount >= 3;

    // Reset & prime context when threshold is met
    if (!this.#conversation || isContextDifferent || isContextExhausted) {
      await this.primeContext(expenses, isContextExhausted ? this.#previousQueries : undefined);
    }

    const userPrompt = INSIGHTS_USER_PROMPT(userQuery);
    const stream = await this.#conversation.sendMessageStreaming(userPrompt);

    this.#previousQueries.push(userQuery);
    this.#turnsCount = this.#turnsCount + 1;

    // Yield repaired, streamed JSON chunks (using jsonrepair)       
    yield* this.processStream(stream);
  }

  /* processStream, lifecycle, and other helper methods */
}

6. Reactive UI Construction in Angular

6.1 History Insights Facade

HistoryInsightsService acts as a clean facade between our UI component and the core InsightService, letting us easily query expenses and stream insights based on user questions.

Our component injects this service to trigger and render the JSON response asynchronously.

Source code of HistoryInsightsService

6.2 History Insights Component

HistoryInsightsComponent comprises child components and a pipe. However, the primary method that communicates with Gemma 4 to generate a list of insights is onAskGemma.

@Component({
  selector: 'app-history-insights',
  ...
})
export default class HistoryInsightsComponent {
  protected readonly vm = inject(HistoryInsightsService);

  readonly expenses = signal<Expense[]>([]);
  readonly streamingResponse = signal<InsightsResponse>({ insights: [] });

  async onAskGemma(query: string): Promise<void> {
    const trimmed = query.trim();
    if (!trimmed) {
      return;
    }

    this.streamingResponse.set({ insights: [] });

    // The expenses are populated after a search query
    // Consume the stream generator and update the UI signal incrementally
    const generator = this.vm.streamInsights(trimmed, this.expenses());
    for await (const response of generator) {
      this.streamingResponse.set(response);
    }
  }
}

The full listing of HistoryInsightsComponent

7. Conclusion

Engineers can build web applications with local models to perform financial analysis without leaking personal data to the training datasets of AI vendors.

Resources

Build Hybrid Image Analysis Application with Angular, Firebase AI Logic, and Gemini Nano [GDE]

Connie Leung — Fri, 07 Aug 2026 14:54:45 +0000

This tutorial demonstrates how to build a hybrid image analysis application using Angular, Firebase AI Logic, and Gemini. Users upload an image, click the button, and wait for either the Gemini Nano or the Gemini 3.5 Flash model to generate tags, alternative texts, and recommendations.

What makes Firebase AI Logic so powerful is that the Hybrid and On-device Web SDK decides to use either the on-device or cloud AI model. In this demo, the SDK downloads and uses Gemini Nano when it is executed in the Chrome browser. When it runs on other browsers, the Gemini 3.5 Flash model handles the image analysis task. The underlying model being used is completely transparent to the user and receives a high-quality response regardless of the backend.

1. Prerequisites

Angular 22
TailwindCSS
Node 24
gemini-3.5-flash
Firebase AI Logic
Firebase Remote Config
Firebase Local Emulator Suite
Firebase App Hosting
Firebase App Check

While Angular and CSS provide the user interface and styling, this demo's AI capability is powered by Firebase services such as Firebase AI Logic and Firebase Remote Config.

npm i -g firebase-tools

Install firebase-tools globally using npm.

npm i --save-exact firebase tailwindcss postcss @tailwindcss/postcss jsonrepair
npm i --save-exact --save-dev angular-eslint firebase-tools husky lint-staged serve

Install additional dependencies for Firebase AI Logic, CSS Styling, and JSON response streaming. The dev dependencies are installed to detect code smells, ensure code quality, and call the Firebase CLI to perform ad-hoc tasks.

2. Source Code

The full source code for this project is available in the NG Firebase Image Analyzer; however, the following sections describe the Firebase Project setup and provide code examples of the Firebase Hybrid and On-device Web SDK.

The following sections walk through the step-by-step Firebase project configuration.

3. Setup Firebase Project in Angular

Set up a Firebase project in an Angular project before any Angular service makes requests to Firebase AI Logic to perform AI tasks.

3.1 Access and Manage Firebase Projects

firebase logout

firebase login

Note: Logging out and logging back in to Firebase before configuring a new project is recommended to refresh authentication.

Create a firebase folder to store .firebasesrc and .firebase.json, which are Firebase configuration files. You can use the Antigravity CLI and Gemini to generate these configuration files within the folder. Antigravity CLI is a Google coding agent that uses AI models to tackle complex problems in the command line. You can find the information on this page.

In my prompt, I instructed the AI model what the default Firebase Project ID was, and the services to install. The demo utilizes Firebase AI Logic, App Hosting for production deployment, and the Firebase Local Emulator Suite to simulate hosting, accelerating the development cycle.

The following is a very simple firebase/.firebasesrc:

{
  "projects": {
    "default": "<Firebase Project ID>"
  }
}

The following is the full configuration firebase/firebase.json:

{
  "apphosting": [
    {
      "backendId": "ng-firebase-image-analyzer",
      "rootDir": "..",
      "ignore": ["node_modules", ".git", "firebase-debug.log", "firebase-debug.*.log", "functions"]
    }
  ],
  "emulators": {
    "apphosting": {
      "port": 5005,
      "rootDirectory": "..",
      "startCommand": "npm run start"
    },
    "ui": {
      "enabled": true
    }
  }
}

The apphosting block shows that the root directory of the Angular code is the parent folder and ignores a list of folders and files during upload.

The emulators block enables app hosting, the Angular application runs at port 5005 and the URL is http://localhost:5005.

The following is a simple apphosting.yaml:

runConfig:
  minInstances: 0
  maxInstances: 2

The YAML file limits the active instances to between 0 and 2, preventing unexpected cloud costs from infinite scaling.

Next, create a new firebase/scripts folder to store NodeJS scripts to facilitate Firebase integration with Angular.

mkdir -p firebase/scripts

3.2 Fetch Default Firebase Remote Config with Firebase CLI

Rather than downloading the default Firebase Remote Config JSON file from the Firebase Console manually, I instructed the Antigravity CLI and Gemini to download the Firebase Remote Configuration values, transforms the response to a JSON key-value pairs, and writes the file to ../public/remote-config-defaults.json. Then, the Angular application can access the JSON file in code during development and in the production build.

const { execSync } = require('child_process');
const fs = require('fs');
const path = require('path');

const firebaseDir = path.resolve(__dirname, '..');

const output = execSync('npx firebase remoteconfig:get --json --project default', {
  encoding: 'utf-8',
  cwd: firebaseDir,
});
const config = JSON.parse(output);
const parameters = config?.result?.parameters;

const defaults = {};
if (parameters) {
  for (const [key, paramObj] of Object.entries(parameters)) {
    if (paramObj.defaultValue && paramObj.defaultValue.value !== undefined) {
      defaults[key] = paramObj.defaultValue.value;
    }
  }
}

const outputPath = path.join(firebaseDir, '..', 'public', 'remote-config-defaults.json');
fs.writeFileSync(outputPath, `${JSON.stringify(defaults, null, 2)}\n`, 'utf-8');
console.log(`Successfully wrote ${outputPath}`);

Add a script in package.json to execute the NodeJS script.

"scripts": {
    "config:fetch": "node firebase/scripts/get-firebase-remote-config.js"
}

The output looks like the following:

{
  "geminiModelName": "gemini-3.5-flash",
  "vertexAILocation": "global",
  "thinkingLevel": "LOW"
}

The complete code can be found in this source file.

3.3 Generate Firebase App Object for App Initialization

The Angular application requires initializing a Firebase App and App Check during bootstrap before making any request to Firebase AI Logic. A Node.js script generates the Firebase App configuration object, reCAPTCHA Enterprise Key, and App Check Debug Token from .env and writes them to public/firebase.config.json. I gitignored the file because both the Recaptcha Enterprise Key and App check debug token are highly sensitive, and must not be revealed.

Copy the .env.example to .env and provide the valid information.

FIREBASE_API_KEY="<Firebase API Key>"
FIREBASE_AUTH_DOMAIN="<Firebase Auth Domain>"
FIREBASE_PROJECT_ID="<Firebase Project ID>"
FIREBASE_STORAGE_BUCKET="<Firebase Storage Bucket>"
FIREBASE_MESSAGING_SENDER_ID="<Firebase Messaging Sender ID>"
FIREBASE_APP_ID="<Firebase App ID>"
FIREBASE_RECAPTCHA_ENTERPRISE_KEY="<Recaptcha Enterprise Key>"
FIREBASE_APPCHECK_DEBUG_TOKEN="<App Check debug token>"

const fs = require('fs');
const path = require('path');

const envPath = path.resolve(__dirname, '../.env');
process.loadEnvFile(envPath);

const app = {
  apiKey: process.env.FIREBASE_API_KEY,
  authDomain: process.env.FIREBASE_AUTH_DOMAIN,
  projectId: process.env.FIREBASE_PROJECT_ID,
  storageBucket: process.env.FIREBASE_STORAGE_BUCKET,
  messagingSenderId: process.env.FIREBASE_MESSAGING_SENDER_ID,
  appId: process.env.FIREBASE_APP_ID,
};

// Check if recaptcha enterprise key is missing
if (!process.env.FIREBASE_RECAPTCHA_ENTERPRISE_KEY) {
  console.warn('Warning: Recaptcha Enterprise key is missing');
}

const config = {
  app,
  recaptchaEnterpriseKey: process.env.FIREBASE_RECAPTCHA_ENTERPRISE_KEY,
  appCheckDebugToken: process.env.FIREBASE_APPCHECK_DEBUG_TOKEN || '',
};

const outputPath = path.resolve(__dirname, '../../public/firebase.config.json');
fs.writeFileSync(outputPath, JSON.stringify(config, null, 2), 'utf-8');
console.log(`Successfully generated ${outputPath} from .env`);

Add a script in package.json to execute the NodeJS script.

"scripts": {
  "config:generate": "node firebase/scripts/generate-firebase-config.js"
}

The JSON object is shown as follows:

{
  "app": {
    "apiKey": "firebase api key",
    "authDomain": "auth domain",
    "projectId": "project key",
    "storageBucket": "storage bucket",
    "messagingSenderId": "message sender id",
    "appId": "app id"
  },
  "recaptchaEnterpriseKey": "<Recaptcha key>",
  "appCheckDebugToken": "<app check debug key>"
}

The complete code can be found in this source file.

3.4 App Hosting

App Hosting is configured in the Firebase Console.

Select your web project
Click App Hosting from the left-side menu
Link your GitHub repository and select your region (e.g., asia-east1).
Click Settings
Click the Domain item to examine the GitHub account, repository, and live branch to roll out.
Click the Environment item to enter the environment variables. These environment variables are the same ones from .env.
Click the Rollouts item to keep the default settings.
Click the Automatic base image updates item and choose the latest Node.js runtime, which is Node 24.

4. Architecture

The Angular application initializes a Firebase App and constructs HybridParams with a cloud-based Gemini model, on-device parameters, in-cloud parameters, a system prompt, and safety settings. Then, the user uploads an image and a user prompt to prompt the model to perform image analysis to generate a response.

The Firebase AI Logic Hybrid and On-device Web SDK determines whether the browser supports Gemini Nano to handle the image-to-text task. If Gemini Nano is capable, then it is used to analyze the image to generate tags, alternative texts, and recommendations. Otherwise, the request is sent to Cloud and processed by the Gemini 3.5 Flash model.

Next, we are ready to integrate Firebase with Angular by initializing the Firebase app, fetching Remote Config, and enabling security via App Check.

5. Initialize a Firebase App in Angular

Angular initializes the Firebase App during bootstrap. We declare a few injection tokens to inject the navigator, window, and Firebase AI instance. Then, we define Angular services that encapsulate the Firebase SDK to initialize a Firebase app, fetch and activate Firebase Remote Config values, and protect AI resources from abuse via App Check.

5.1. Injection Tokens

import { isPlatformBrowser } from '@angular/common';
import { inject, InjectionToken, PLATFORM_ID } from '@angular/core';
import { AI } from 'firebase/ai';

export const FIREBASE_AI = new InjectionToken<AI>('FirebaseAI');

export const WINDOW = new InjectionToken<Window | null>('WINDOW', {
  providedIn: 'root',
  factory: () => {
    const platformId = inject(PLATFORM_ID);
    return isPlatformBrowser(platformId) ? window : null;
  },
});

export const NAVIGATOR = new InjectionToken<Navigator | null>('NAVIGATOR', {
  providedIn: 'root',
  factory: () => {
    const win = inject(WINDOW);
    return win ? win.navigator : null;
  },
});

When the platform is a browser, the factory functions return the Window and Navigator objects, respectively.

5.2. Provide Firebase AI Model

import { FIREBASE_AI } from '@/features/ai/constants/ai.const';
import { ConfigService } from '@/features/ai/services/config.service';
import { EnvironmentProviders, inject, makeEnvironmentProviders } from '@angular/core';
import { getAI, VertexAIBackend } from 'firebase/ai';
import { getValue } from 'firebase/remote-config';

export function provideFirebaseAI(): EnvironmentProviders {
  return makeEnvironmentProviders([
    {
      provide: FIREBASE_AI,
      useFactory: () => {
        const configService = inject(ConfigService);
        const location = getValue(configService.RemoteConfig, 'vertexAILocation').asString() || 'global';
        return getAI(configService.firebaseApp, {
          backend: new VertexAIBackend(location),
        });
      },
    },
  ]);
}

The provideFirebaseAI provider uses the FIREBASE_AI injection token to construct a Firebase AI having Vertex AI as its backend. Vertex AI is utilized in this architecture because its endpoints are directly accessible in regions like Hong Kong, whereas the standard Gemini API requires a VPN.

5.3. Config Service

The core method of ConfigService is initialize(), which initializes the Firebase app, configures an instance of App Check, and fetches and activates the Firebase Remote Config values.

import { inject, InjectionToken, isDevMode } from '@angular/core';
import { WINDOW, NAVIGATOR } from '@/core/constants/navigator.const';

export const LOCAL_DOMAINS = new InjectionToken<string[]>('LOCAL_DOMAINS', {
  providedIn: 'root',
  factory: () => ['localhost', '127.0.0.1', '::1', '[::1]'],
});

export function injectIsLocalhost(): () => boolean {
  const win = inject(WINDOW);
  const localDomains = inject(LOCAL_DOMAINS);

  return () => !!win && localDomains.includes(win.location.hostname);
}

export function configureAppCheckDebugToken(configToken?: string, isLocalhost?: boolean): void {
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  (globalThis as any).FIREBASE_APPCHECK_DEBUG_TOKEN = configToken || isDevMode() || !!isLocalhost;
}

injectIsLocalhost creates and returns a higher-order function that determines whether the application is running on localhost.

export function injectOnlineStatus(): () => boolean {
  const navigator = inject(NAVIGATOR);
  return () => navigator?.onLine ?? true;
}

On the other hand, injectOnlineStatus returns a higher-order function that determines whether the application is running online or offline.

import firebaseConfig from '@/public/firebase.config.json';
import remoteConfigDefaults from '@/public/remote-config-defaults.json';
import { isDevMode, Service } from '@angular/core';
import { FirebaseApp, FirebaseOptions, initializeApp } from 'firebase/app';
import { AppCheck, initializeAppCheck, ReCaptchaEnterpriseProvider } from 'firebase/app-check';
import { fetchAndActivate, getRemoteConfig, RemoteConfig } from 'firebase/remote-config';

@Service()
export class ConfigService {
  #app: FirebaseApp | undefined = undefined;
  #remoteConfig: RemoteConfig | undefined = undefined;
  #isOnline = injectOnlineStatus();
  #isLocalhost = injectIsLocalhost();

  protected initializeAppCheckInstance(app: FirebaseApp, key: string): AppCheck {
    return initializeAppCheck(app, {
      provider: new ReCaptchaEnterpriseProvider(key),
      isTokenAutoRefreshEnabled: true,
    });
  }

  protected setupRemoteConfig(app: FirebaseApp): RemoteConfig {
    const rc = getRemoteConfig(app);
    rc.defaultConfig = remoteConfigDefaults;
    rc.settings.minimumFetchIntervalMillis = isDevMode() ? 0 : 3600000;
    return rc;
  }

  async initialize(): Promise<void> {
    this.#app = this.initializeApp(firebaseConfig.app);

    const isOnline = this.#isOnline();
    const isLocalhost = this.#isLocalhost();

    if (isOnline && firebaseConfig.recaptchaEnterpriseKey) {
      configureAppCheckDebugToken(firebaseConfig.appCheckDebugToken, isLocalhost);
      this.initializeAppCheckInstance(this.#app, firebaseConfig.recaptchaEnterpriseKey);
    }

    this.#remoteConfig = this.setupRemoteConfig(this.#app);

    if (isOnline) {
      await this.fetchAndActivate(this.#remoteConfig);
    }
  }
}

After Firebase app initialization, the online status and the Enterprise reCAPTCHA key are examined. When both conditions are met, an instance of Firebase App Check is initialized. Finally, the Remote Config is fetched and activated to overwrite the default values.

Tips:
Angular 22 Architecture Best Practice:
The Angular team now recommends the @Service() decorator over @Injectable({ providedIn: 'root' }) for global singletons because it is cleaner, contains less boilerplate, and is built specifically to pair with modern inject() functional dependency injection.
Note: @Injectable() is still fully supported and necessary if you require constructor-based dependency injection or scoped providers (such as component-level scoping).

5.4. Bootstrap the application

import { provideFirebaseAI } from '@/features/ai/providers/ai.provider';
import { ConfigService } from '@/features/ai/services/config.service';
import { ApplicationConfig, inject } from '@angular/core';

export const appConfig: ApplicationConfig = {
  providers: [
    ... other providers ... 
    provideAppInitializer(() => inject(ConfigService).initialize()),
    provideFirebaseAI(),
  ],
};

Include provideFirebaseAI and provideAppInitializer in the providers array of appConfig.

import { bootstrapApplication } from '@angular/platform-browser';
import { appConfig } from './app/app.config';
import { App } from './app/app';

bootstrapApplication(App, appConfig).catch((err) => console.error(err));

Calling bootstrapApplication ensures that the Firebase AI model, Firebase app, App Check, and Firebase Remote Config values are initialized properly during startup.

Once the Firebase integration is complete, we shift the focus to Angular AI services and JSON response streaming in the Angular component.

6. Image Analysis Service in Angular

6.1. AI Service with JSON Response Streaming

The Hybrid Image Analysis application prompts the Gemini Nano or Gemini 3.5 Flash model to return a structured output, which is a JSON object. Therefore, the application streams the JSON response and uses the jsonrepair library to repair the incomplete JSON object into a valid, partial JSON object.

import { EnhancedGenerateContentResponse } from 'firebase/ai';

export interface PartialResponse<T> {
  partialData: Partial<T>;
  response?: EnhancedGenerateContentResponse;
}

import { inject, Service } from '@angular/core';
import { GenerativeModel, SchemaRequest, TypedSchema } from 'firebase/ai';
import { jsonrepair } from 'jsonrepair';
import { GenerateContentParams, PartialResponse } from '../types/ai.types';
import { AiModelCacheService } from './ai-model-cache.service';

@Service()
export class AiService {
  #cacheService = inject(AiModelCacheService);

  private getCachedModel({ schema, systemInstruction }: GenerateContentParams) {
    const model = this.#cacheService.getOrCreateModel({
      schema,
      systemInstruction,
    });

    return model;
  }

  private parseStreamJSONResponse<T>(text: string, schema?: TypedSchema | SchemaRequest) {
    if (schema) {
      try {
        return JSON.parse(text) as Partial<T>;
      } catch {
        return JSON.parse(jsonrepair(text)) as Partial<T>;
      }
    }

    return { alternativeTexts: [text] } as unknown as Partial<T>;
  }

  private async downloadDeviceModel(model: GenerativeModel) {
    if (model) {
      await model.initializeDeviceModel((val) => console.log(`Download progress: ${Math.round(val * 10000) / 100}%`));
    }
  }

  private constructRequest(params: GenerateContentParams) {
    return typeof params.contents === 'string' || Array.isArray(params.contents) ? params.contents : [params.contents];
  }

  async *generateContentStream<T>(params: GenerateContentParams): AsyncGenerator<PartialResponse<T>> {
    const model = this.getCachedModel(params);
    const request = this.constructRequest(params);

    await this.downloadDeviceModel(model);

    const result = await model.generateContentStream(request);
    let accumulatedText = '';

    for await (const chunk of result.stream) {
      accumulatedText = accumulatedText + chunk.text();
      const parsed = this.parseStreamJSONResponse<T>(accumulatedText, params.schema);
      yield {
        partialData: parsed,
      };
    }

    const finalResponse = await result.response;
    const finalParsed = this.parseStreamJSONResponse<T>(finalResponse.text(), params.schema);
    yield {
      partialData: finalParsed,
      response: finalResponse,
    };
  }
}

The downloadDeviceModel helper method downloads the Gemini Nano model to the Chrome browser. When users request the image analysis task, it runs on-device instead of on a cloud server. Sensitive user data does not leave the local computer, and Cloud AI usage costs remain at zero.

The generateContentStream generator yields partial JSON objects and the final response.

6.2. Image Analysis Service, Prompts and Structured Output

The ImageAnalysisService delegates the image analysis task to AiService to obtain the generated tags, alternative texts, and recommendations.

export async function fileToGenerativePart(file: File | Blob, mimeType?: string): Promise<Part> {
  const resolvedMimeType = mimeType || file.type;
  if (!resolvedMimeType) {
    throw new Error('MIME type must be specified or present on the File/Blob.');
  }

  const dataUrl = await readFileAsDataURL(file);
  const base64Data = dataUrl.split(',')[1];

  return {
    inlineData: {
      data: base64Data,
      mimeType: resolvedMimeType,
    },
  };
}

The fileToGenerativePart utility function reads the uploaded image and converts it into inline base64 data. The function returns the data and MIME type required by the Gemini model.

export const SYSTEM_INSTRUCTION = `You are an expert image analyst, accessibility specialist, and professional photo editor.

Your behavior rules:
1. Always output strictly in raw JSON matching the provided schema, with no markdown formatting or conversational text.
2. For descriptions, prioritize screen-reader accessibility guidelines.
3. For tags, always provide an objective visual reason and platform-specific variants.`;

export const IMAGE_ANALYSIS_USER_PROMPT =
  'Analyze the attached image and generate the alternative descriptions, tags, and styling suggestions according to your defined system instructions and the response schema.';

SYSTEM_INSTRUCTION is the system prompt and IMAGE_ANALYSIS_USER_PROMPT is the user prompt.

The system prompt defines the AI's persona as an expert photo editor, while the user prompt describes the input image and specifies what structured data is expected in return.

import { Schema } from 'firebase/ai';
import { AltTextsSchema } from './alt-texts.schema';
import { RecommendationsSchema } from './recommendation.schema';
import { TagsSchema } from './tag.schema';

export const ImageAnalysisSchema = Schema.object({
  properties: {
    alternativeTexts: AltTextsSchema,
    tags: TagsSchema,
    recommendations: RecommendationsSchema,
  }
});

ImageAnalysisSchema was the structured output of the user prompt that consisted of alternative texts, tags, and recommendations.

import { fileToGenerativePart } from '@/core/utils/base64.util';
import { AiService } from '@/features/ai/services/ai.service';
import { ImageAnalysisSchema } from '@/features/image-analysis/schemas/image-analysis.schema';
import {
  ImageAnalysisResponse,
  StreamingAnalysisWithMetadata,
} from '@/features/image-analysis/types/image-analysis-metadata.type';
import { inject, Service } from '@angular/core';
import { IMAGE_ANALYSIS_USER_PROMPT, SYSTEM_INSTRUCTION } from '../prompts/image-analysis.prompt';

@Service()
export class ImageAnalysisService {
  #aiService = inject(AiService);

  async *analyzeImageStream(file: File | Blob, customPrompt?: string): AsyncGenerator<StreamingAnalysisWithMetadata> {
    const imagePart = await fileToGenerativePart(file);
    const userPrompt = customPrompt ? customPrompt : IMAGE_ANALYSIS_USER_PROMPT;

    const generator = await this.#aiService.generateContentStream<ImageAnalysisResponse>({
      systemInstruction: SYSTEM_INSTRUCTION,
      contents: [userPrompt, imagePart],
      schema: ImageAnalysisSchema,
    });

    for await (const update of generator) {
      const partialData = update.partialData;
      const analysis = {
        alternativeTexts: partialData?.alternativeTexts,
        tags: partialData?.tags,
        recommendations: partialData?.recommendations,
      };

      yield {
        analysis,
      };
    }
  }
}

The analyzeImageStream is a generator that returns the output of the image analysis.

7. Image Analysis User Interface in Angular

The ImageAnalysis component features an image uploader to select an image for analysis. When the button is clicked, it invokes triggerAnalysis to stream the image analysis response and sets the value of the analysisData signal.

import { ImageAnalysisService } from '@/features/image-analysis/services/image-analysis';
import { StreamingAnalysisWithMetadata } from '@/features/image-analysis/types/image-analysis-metadata.type';
import { ImageUploader } from '@/shared/ui/image-uploader/image-uploader';
import { Component, computed, inject, signal } from '@angular/core';
import { ImageAnalysisPanel } from './image-analysis-panel/image-analysis-panel';
import { TagList } from './tag-list/tag-list';
import { ImageTag } from './tag-list/types/image-tag.type';

@Component({
  selector: 'app-image-analysis',
  imports: [ImageUploader, TagList, ImageAnalysisPanel],
  templateUrl: './image-analysis.html',
  styleUrl: './image-analysis.css',
})
export default class ImageAnalysis {
  imageAnalysisService = inject(ImageAnalysisService);

  // Local reactive states
  imageUrl = signal<string | null>(null);
  selectedFile = signal<File | null>(null);
  analysisData = signal<StreamingAnalysisWithMetadata | null>(null);

  tags = computed<ImageTag[]>(() => {
    const data = this.analysisData();
    if (!data) {
      return [];
    }
    return (data.analysis.tags || []).map((t) => {
      return {
        label: t.name,
        tooltip: t.sentence,
      };
    });
  });

  onFileSelected(file: File) {
    this.selectedFile.set(file);
    this.analysisData.set(null);
    this.errorMessage.set('');
  }

  async triggerAnalysis() {
    const file = this.selectedFile();
    if (!file) {
      return;
    }

    const stream = this.imageAnalysisService.analyzeImageStream(file);
    for await (const update of stream) {
      this.analysisData.set(update);
    }
  }

  onImageRemoved() {
    this.selectedFile.set(null);
    this.analysisData.set(null);
    this.errorMessage.set('');
  }
}

When analysisData is updated, the HTML template of the component displays the latest values.

<div class="workspace-container">
  <app-image-uploader
    [(imageUrl)]="imageUrl"
    (fileSelected)="onFileSelected($event)"
    (imageRemoved)="onImageRemoved()"
  ></app-image-uploader>

  @if (imageUrl() && !isLoading()) {
    <div class="action-container">
      <button (click)="triggerAnalysis()" class="primary-action-btn">
        <span class="material-symbols-outlined">psychology</span>
        Analyze Image ({{ seconds() }}s)
      </button>
    </div>
  }

  @if (tags().length > 0) {
    <!-- Semantic Tag Cloud -->
    <div class="tag-cloud-section animate-fade-in">
      <h4 class="section-subtitle">Detected Classifications</h4>
      <app-tag-list [tags]="tags()"></app-tag-list>
    </div>

    @if (analysisData()) {
      <!-- Advanced Analysis Results Panel -->
      <app-image-analysis-panel
        class="animate-fade-in block"
        [data]="analysisData()"
        [imageUrl]="imageUrl()"
        [source]="source()"
      ></app-image-analysis-panel>
    }
  }
</div>

The template passed the inputs to app-tag-list and app-image-analysis-panel components.

<div class="tag-list-wrapper">
  <div class="tag-list-grid">
    @for (tag of tags(); track tag.label) {
      <app-badge [label]="tag.label" [tooltip]="tag.tooltip" />
    }
  </div>
</div>

The above is the HTML template of the tag list component. This component renders a list of tags. When a tag is hovered, a tooltip shows to explain why it is chosen.

@if (data(); as analysis) {
  @let innerAnalysis = analysis.analysis;
  @let alternativeTexts = innerAnalysis?.alternativeTexts;
  @let recommendations = innerAnalysis?.recommendations;

  <div class="analysis-panel-container">
    <app-tab-group>
      <app-tab id="thought" label="Thought Summary">
        <app-thought-summary
          [alternativeTexts]="alternativeTexts"
        />
      </app-tab>

      <app-tab id="detailed" label="Recommendations">
        <app-image-recommendation [recommendations]="recommendations" />
      </app-tab>
    </app-tab-group>
  </div>
}

The app-image-analysis-panel renders alternative texts and recommendations in separate tabs for users to view.

8. Conclusion

This concludes the journey of building a Hybrid Image Analysis Application using Angular, Firebase AI Logic Hybrid and On-device Web SDK, and Gemini models.

Engineers can build AI applications that run locally and use the Gemini Nano model in the Chrome browser. When the network is unstable, the application also works offline because AI processing never leaves the local machine for the cloud infrastructure.

Resources

Build Firebase AI Logic Application with Antigravity CLI and Stitch MCP Server [GDE]

Connie Leung — Wed, 15 Jul 2026 06:50:10 +0000

Build Firebase AI Logic with Antigravity CLI

Note: Google Cloud credits are provided for this project.

In this blog post, I demonstrate how to use the Antigravity CLI (an agentic AI assistant integrating directly with development workflows via skills and servers) to build an image analysis demo using Angular, the Firebase Hybrid & On-device Inference Web SDK, and Gemini models. Users upload an image and use a Gemini model to analyze it to generate a few alternative texts, tags, recommendations, and CSS tips to enhance the image quality.

When the demo is running in Chrome 148+, the Hybrid & On-device SDK leverages the Prompt API of the on-device Gemini Nano model to perform the image-to-text tasks, and the token usage is 0. When other browsers, such as Safari or Firefox, execute the same tasks, the SDK falls back to Cloud AI (Gemini 3.5 Flash model), which consumes tokens.

Next, I describe how to install the skills in my Angular project and register the Angular and Stitch MCP servers in the Antigravity CLI to develop the infrastructure, services, and UI design of my demo.

1. Workflow

This is my entire workflow from implementing features, generating UI screens, and mapping the screens to Angular components.

2. Skills

I installed the grill-with-docs, angular, and firebase skills in my project for the following reasons:

grill-with-docs: Conduct a rigid Q&A session to generate a specification for a feature, refactor, or critical fix. AI is responsible for performing thorough analysis, and putting in more efforts to generate code to achieve the task.
domain-modeling: The skill is referenced in the SKILL.md of the grill-with-docs skill, so a copy of it is required.
code-review: Spawn two sub-agents to review changes to detect code smells and verify that the changes align with the specification.
angular: Provide the best practices of modern Angular architecture, such as using signals and signal forms.
firebase: Provide the skills for Firebase AI Logic, Firebase Remote, etc.

3. Wire the Stitch MCP Server with the Antigravity CLI

Stitch is a Google design-to-code tool that lets developers and designers generate interactive screens using natural language prompts.

Hong Kong users need to enable a VPN to access Stitch with AI due to geofencing. Then, I can visit https://stitch.withgoogle.com/ to create a Stitch project and generate new screens in it using natural language.

On the other hand, there is no geofencing restriction on the Antigravity CLI, so I can register the Stitch MCP server to its global MCP configuration file.

This is how I made the CLI talk to the Stitch MCP server after trial and error:

Navigate to https://github.com/davideast/stitch-mcp and read the Quick Start to initialize the MCP server.
Visit https://stitch.withgoogle.com/settings, scroll to API Key and create a new key. Copy the key because we need it in the mcp configuration file.

Open ~/.gemini/config/mcp_config.json to register the Stitch MCP server globally. Replace <Stitch API Key> with your actual API key.

"mcpServers": {
   "stitch": {
        "command": "npx",
        "args": ["-y", "@_davideast/stitch-mcp@latest", "proxy"],
        "env": {
            "STITCH_API_KEY": "<Stitch API Key>",
            "STITCH_USE_SYSTEM_GCLOUD": "1",
            "GOOGLE_CLOUD_PROJECT": "<google cloud project id>",
            "CI": "1",
            "DOTENV_CONFIG_QUIET": "true"
        }
   }
}

Quit the Antigravity CLI and launch again to verify the integration is established correctly.
In the terminal, ask a question such as "List all my Stitch Projects", and wait for the response. If the CLI returns a correct response, then it can communicate with the Stitch MCP server.

The CLI is equipped with the skills and the MCP server. Next, I can use the CLI and the Gemini model to implement features and generate Angular components.

4. Wire the Angular MCP Server

Similarly, I also registered the Angular MCP server to find Angular best practices and search documentation.

"mcpServers": {
  "angular-cli": {
       "command": "npx",
       "args": ["-y", "@angular/cli", "mcp"]
    }
}

The server exposes the following tools to ease Angular development:

ai_tutor: Asks the Angular AI tutor for help with code generation or questions.
get_best_practices: Retrieves recommended guidelines and architectural practices for Angular.
search_documentation: Searches the official Angular documentation.
list_projects: Lists the projects defined in your Angular workspace configuration (angular.json).
onpush_zoneless_migration : Helps migrate parts of your application to OnPush change detection and zoneless mode.
run_target : Runs any configured architect target (e.g., build, test, lint) using the Angular CLI.
devserver.start, devserver.stop , and devserver.wait_for_build : Manage and interact with the local Angular development server.

Now, the CLI is equipped with the agent skills and MCP servers to coordinate the development of the new features in the Angular application.

5. Implement features with skills and MCP servers

After implementing a few features in my Firebase AI Logic application, I developed two workflows and chose either one depending on whether I wanted to develop muscle memory.

Workflow 1: Use /grill-with-docs to generate an Architecture Decision Record (ADR), and save it to a path for human review and documentation. Then, use /goal to ask the coding assistant to implement the ADR, and accept the changes.

For example, I type this prompt to ask the coding assistant to implement an ADR and include relevant skills.

Prompt: "/goal implement <ADR file name>, use Angular skill and Angular MCP server to adopt modern Angular architecture in the code"

Workflow 2: Use /grill-with-docs to generate an ADR, and save it to a path for human review and documentation. I implement the ADR to build up muscle memory, and commit the changes to a git branch. Next, use /code-review to fetch a commit or all changes of a Git branch, and request the coding assistant to spawn two sub-agents to review changes. The first sub-agent detects code smells, and the second sub-agent validates the changes against the ADR to find scope creep, missing, or incorrect details.

For example, I type this prompt to ask the coding assistant to fetch a Git branch, and validate against an ADR.

Prompt: "/code-review fetch all the commits of <Git branch>, and validate the changes against <ADR filename>."

The sub-agents run and print out a report with violations. I fix the violations, rebase the branch, and repeat the workflow until the report shows 0 violations. Finally, I merge the feature branch to main, push the main branch to the remote repository to deploy to Firebase App Hosting.

After implementing the features, I have the enums, types, interfaces, utilities, and services. In the next section, I use the Stitch MCP server to generate UI screens and map them to Angular components.

6. Generate UI Screens with Stitch

This web demo allows users to upload an image and prompt a Gemini model to analyze it to generate tags, alternative text, CSS filters, and image crops.

Note: Enable VPN to access the Stitch site

I use natural language to prompt Stitch to generate two pages for my web application. The web application has a landing page and an image analysis page.

First, I created a new Stitch project to host the UI screens.

Then, I wrote prompts to generate both the home page and the image analysis page

After clicking the Analysis Image button, the page displays the tab group with some tabs.

Next, I need to use the Antigravity CLI to communicate with the Stitch MCP server to generate the Design.md files.

Prompt: "Use the Stitch MCP server to retrieve a Stitch UI Screen from my Stitch Project. Analyze its visual layout, extract the design tokens (colors, typography, and spacing), and generate a DESIGN.md file in the project folder path."

For example, I saved the DESIGN.md of the home screen to the root directory and the DESIGN.md of the image analysis page in the image-analysis folder.

After reviewing the DESIGN.md and ensuring everything looks good, I use /goal again to map the DESIGN.md to the Angular components. In the prompt, I try to provide as much context as possible, so that the coding assistant does not generate all the CSS styles, HTML template, and logic in a single component.

Prompt: "/goal You are an expert Angular developer. Your goal is to implement the UI layout defined in DESIGN.md into production-ready Angular components. Fully read DESIGN.md first to extract all design tokens, spacing guidelines, colors, and layout instructions, and apply them using the following rules: Insert the context and rules...."

7. Live Site Testing

The use case is to upload an image, perform image analysis, and display the token consumption of the analysis.

On Chrome 148+,

Step 1: Upload an image
Step 2: Click the button to generate tags and alternative texts
Step 3: Click the Token usage tab to observe how tokens are spent to achieve the tasks.

On Firefox, repeat the same procedure

Step 1: Upload an image
Step 2: Click the button to generate tags and alternative texts
Step 3: Click the Token usage tab to observe how tokens are spent to achieve the tasks.

8. Conclusion

This concludes the journey of using the Antigravity CLI, agent skills, and Stitch MCP server to build an image analysis application.

When the demo is run in Chrome 148+, the Gemini Nano model is used, and the token usage of the cloud AI is 0. When the demo is run on other browsers, the Gemini 3.5 Flash model is used to generate the response, and tokens are consumed in the cloud.

Please examine the Firebase AI Hybrid & On-Device Web SDK, and leverage the on-device model to perform simple AI tasks to reduce costs.

Resources

Build Firebase AI Logic Application with Antigravity CLI

Connie Leung — Fri, 10 Jul 2026 15:45:18 +0000

Note: Google Cloud credits are provided for this project.

1. Workflow

This is my entire workflow from implementing features, generating UI screens, and mapping the screens to Angular components.

2. Skills

I installed the grill-with-docs, angular, and firebase skills in my project for the following reasons:

grill-with-docs: Conduct a rigid Q&A session to generate a specification for a feature, refactor, or critical fix. AI is responsible for performing thorough analysis, and putting in more efforts to generate code to achieve the task.
domain-modeling: The skill is referenced in the SKILL.md of the grill-with-docs skill, so a copy of it is required.
code-review: Spawn two sub-agents to review changes to detect code smells and verify that the changes align with the specification.
angular: Provide the best practices of modern Angular architecture, such as using signals and signal forms.
firebase: Provide the skills for Firebase AI Logic, Firebase Remote, etc.

3. Wire the Stitch MCP Server with the Antigravity CLI

Stitch is a Google design-to-code tool that lets developers and designers generate interactive screens using natural language prompts.

On the other hand, there is no geofencing restriction on the Antigravity CLI, so I can register the Stitch MCP server to its global MCP configuration file.

This is how I made the CLI talk to the Stitch MCP server after trial and error:

Navigate to https://github.com/davideast/stitch-mcp and read the Quick Start to initialize the MCP server.
Visit https://stitch.withgoogle.com/settings, scroll to API Key and create a new key. Copy the key because we need it in the mcp configuration file.

Open ~/.gemini/config/mcp_config.json to register the Stitch MCP server globally. Replace <Stitch API Key> with your actual API key.

"mcpServers": {
   "stitch": {
        "command": "npx",
        "args": ["-y", "@_davideast/stitch-mcp@latest", "proxy"],
        "env": {
            "STITCH_API_KEY": "<Stitch API Key>",
            "STITCH_USE_SYSTEM_GCLOUD": "1",
            "GOOGLE_CLOUD_PROJECT": "<google cloud project id>",
            "CI": "1",
            "DOTENV_CONFIG_QUIET": "true"
        }
   }
}

Quit the Antigravity CLI and launch again to verify the integration is established correctly.
In the terminal, ask a question such as "List all my Stitch Projects", and wait for the response. If the CLI returns a correct response, then it can communicate with the Stitch MCP server.

The CLI is equipped with the skills and the MCP server. Next, I can use the CLI and the Gemini model to implement features and generate Angular components.

4. Wire the Angular MCP Server

Similarly, I also registered the Angular MCP server to find Angular best practices and search documentation.

"mcpServers": {
  "angular-cli": {
       "command": "npx",
       "args": ["-y", "@angular/cli", "mcp"]
    }
}

The server exposes the following tools to ease Angular development:

ai_tutor: Asks the Angular AI tutor for help with code generation or questions.
get_best_practices: Retrieves recommended guidelines and architectural practices for Angular.
search_documentation: Searches the official Angular documentation.
list_projects: Lists the projects defined in your Angular workspace configuration (angular.json).
onpush_zoneless_migration : Helps migrate parts of your application to OnPush change detection and zoneless mode.
run_target : Runs any configured architect target (e.g., build, test, lint) using the Angular CLI.
devserver.start, devserver.stop , and devserver.wait_for_build : Manage and interact with the local Angular development server.

Now, the CLI is equipped with the agent skills and MCP servers to coordinate the development of the new features in the Angular application.

5. Implement features with skills and MCP servers

After implementing a few features in my Firebase AI Logic application, I developed two workflows and chose either one depending on whether I wanted to develop muscle memory.

For example, I type this prompt to ask the coding assistant to implement an ADR and include relevant skills.

Prompt: "/goal implement <ADR file name>, use Angular skill and Angular MCP server to adopt modern Angular architecture in the code"

For example, I type this prompt to ask the coding assistant to fetch a Git branch, and validate against an ADR.

Prompt: "/code-review fetch all the commits of <Git branch>, and validate the changes against <ADR filename>."

6. Generate UI Screens with Stitch

This web demo allows users to upload an image and prompt a Gemini model to analyze it to generate tags, alternative text, CSS filters, and image crops.

Note: Enable VPN to access the Stitch site

I use natural language to prompt Stitch to generate two pages for my web application. The web application has a landing page and an image analysis page.

First, I created a new Stitch project to host the UI screens.

Then, I wrote prompts to generate both the home page and the image analysis page

After clicking the Analysis Image button, the page displays the tab group with some tabs.

Next, I need to use the Antigravity CLI to communicate with the Stitch MCP server to generate the Design.md files.

For example, I saved the DESIGN.md of the home screen to the root directory and the DESIGN.md of the image analysis page in the image-analysis folder.

7. Live Site Testing

The use case is to upload an image, perform image analysis, and display the token consumption of the analysis.

On Chrome 148+,

Step 1: Upload an image
Step 2: Click the button to generate tags and alternative texts
Step 3: Click the Token usage tab to observe how tokens are spent to achieve the tasks.

On Firefox, repeat the same procedure

Step 1: Upload an image
Step 2: Click the button to generate tags and alternative texts
Step 3: Click the Token usage tab to observe how tokens are spent to achieve the tasks.

8. Conclusion

This concludes the journey of using the Antigravity CLI, agent skills, and Stitch MCP server to build an image analysis application.

Please examine the Firebase AI Hybrid & On-Device Web SDK, and leverage the on-device model to perform simple AI tasks to reduce costs.

Resources

How to Securely Manage AI Prompts in Angular with Firebase [GDE]

Connie Leung — Mon, 15 Jun 2026 18:31:32 +0000

Migrate to Firebase Server Prompt Template in Angular using Dependency Injection

Firebase released Server Prompt Templates to host prompt templates in its infrastructure. The template follows the DotPrompt format and syntax, so the content can have one or more of the following:

Model name
Model configuration
Input validation and schema
Output schema
Tool user
System instruction
User prompt

Moreover, the team offers the TemplateGenerativeModel class, which allows engineers to call either the generateContent or generateContentStream method with a template ID and optional template variables to generate responses. This simplifies the process of constructing text and inline data parts programmatically, passing the parts array and the generation configuration to GenerativeModel to obtain the same results.

Server Prompt Templates resolve several key enterprise AI pain points.

Pain Point	Description
Better Security	The prompt text is stored in the server side, so it cannot be exposed in the network call. Users cannot open the Network tab of the Chrome browser and inspect the prompt text in the payload.
Better Guardrail	Prompt texts are not revealed, so malicious users cannot modify the prompt easily to trigger prompt injection and other attacks to Gemini models
No Prompt Drift	Engineer A edits a prompt locally, forgets to commit, and deploys the code changes. Engineer B uses the old prompt for development, and there are two versions scattered around. Server prompt templates ensure engineers use the same version for development. When the prompt is updated on the server, it is propagated to all instances of the client application.
Testing in Console	Engineers can verify the prompts are working in the Firebase Console before writing a line of code.
Less Deployments	When prompts are updated in the server side, client applications receive the prompt updates without redeployment.

I have listed the benefits of Firebase AI Logic Server Prompt Templates. Next, I will demonstrate how to migrate an existing prompt to use Server Prompt Templates in Angular using Dependency Injection.

Note: Currently, Firebase AI Logic Server Prompt Template is in Preview, please do not use it in production until it reaches General Availability (GA) status. However, it is an interesting technology to explore.

1. Prerequisites

Angular 19
TailwindCSS
Node 22
gemini-3.1-flash-image (also known as Nano Banana 2)
Firebase AI Logic
Firebase Cloud Functions
Firebase Remote Config
Firebase Local Emulator Suite

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and re-login to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the screens to set up Firebase Cloud Function, Firebase Local Emulator Suite and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

After completing the step-by-step, the Firebase tools will generate function and remote config templates, and configuration files such as .firebaserc and firebase.json.

The next section has the details of the implementation repository.

2. Source Code

The full source code for this project is available in the NG Firebase AI Nano Banana, however, the following sections describe the code changes made to migrate to Firebase Server Prompt Templates.

3. Architecture

The application matches the URL paths and routes to different components. When the URL path matches template-prompt/:featureId, the route creates GenMediaService at the route level and injects IMAGE_GENERATOR_TOKEN using the route's injection context. The token is mapped to ServerTemplateService. On the other hand, other routes use the GenMediaService in the root injector and inject a global IMAGE_GENERATOR_TOKEN that maps to FirebaseService. The implementation will be shown later in the blog.

4. Server Prompt Template Creation

You can create a server prompt template in the Firebase Console. This guide assumes an existing Firebase project named vertexai-firebase. Click "AI Logic" from the left sidebar, and click the "Prompt templates (PREVIEW)" tab.

Users can click the Create Template button to create a new prompt on the server side.

A template is configured to generate a glass bottle image from inline image data. The unique template ID is glass-bottle-souvenir-v0-0-1, and the template name is glass-bottle-souvenir.

4.1. Model Configuration

---
model: "gemini-3.1-flash-image"
config:
  candidateCount: 1
  safetySettings:
    - category: HARM_CATEGORY_HARASSMENT
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_HATE_SPEECH
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_SEXUALLY_EXPLICIT
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_DANGEROUS_CONTENT
      threshold: BLOCK_ONLY_HIGH
input:
  schema:
    inlineImages?(array, inline image data):
      type: object
      properties:
        mimeType: string
        data: string  # inline data must be base64-encoded
    aspectRatio?: string, the aspect ratio of the image
    resolution?: string, the resolution of the image
---

The configuration specifies the model name, model configuration, and input schema and validations.

Section	Configuration	Description
model	gemini-3.1-flash-image	The Gemini model name of Nano Banana 2.
config	candidateCount: 1	The model returns at most 1 image
safetySettings	BLOCK_ONLY_HIGH	Safety category of harassment, hate speech, sexually explicit content, and dangerous content
input	schema	Input schema and validation

This prompt expects an array of inlineImages of type object. Each inline image contains a MIME type and inline data. Moreover, the prompt accepts an optional aspect ratio and resolution.

4.2. System Instructions

The prompt parts has {{role "system"}} syntax to specify the system instructions, and {{role "user"}} to specify the user prompt.

{{role "user"}}
A 1/7 scale commercialized collectible ... with realistic lighting and shadows.
{{#if aspectRatio}}
Apply this aspect ratio to the image: {{aspectRatio}}.
{{/if}}
{{#if resolution}}
Apply this resolution to the image: {{resolution}}.
{{/if}}

{{#each inlineImages}}
  {{media type="mimeType" data="data"}}
{{/each}}

The user prompt generates a souvenir glass bottle image from the uploaded inline image.

When the aspect ratio is provided, "Apply this aspect ratio to the image: {{aspectRatio}}." is appended to the prompt.

When the resolution is provided, "Apply this resolution to the image: {{resolution}}." is appended to the prompt.

The loop iterates the inlineImages list to specify the mime type and the inline data.

4.3. Testing the Prompt in Firebase Console

// Prompt Input
{
   "inline_images": [{
    "mime_type": "image/png",
    "contents": "iVBORw0KGgoAAAANSUhEUgAAARAAAABcCAYAAACm+q2AAAXGElEQVR4Ae1dC5QcVZm..."
  }],
   "aspectRatio": "4:1",
   "resolution": "512"
}

The prompt input includes an image, aspect ratio, and resolution for testing before writing a line of code.

In the Firebase UI Console, choose the Gemini API provider from the dropdown list. The Create formatted test request button allows users to verify the request is correct before the actual execution. The Run prompt text button executes the request to generate a 512px and 4:1 image.

The test request generates a souvenir glass bottle with the expected aspect ratio.

Next, I will define two new injection tokens: the first one injects an image generator and the second one injects a TemplateGenerativeModel. I also create a new Server Prompt Template service to generate an image based on the template ID and template variables.

5. Server Prompt Template Service Implementation

5.1. Image Generator Interface

export type BaseGenerateParam = {
  aspectRatio?: string;
  resolution?: string;
  imageFiles: File[];
}

export type GenerateImageParam = BaseGenerateParam &  {
  prompt?: string;
  templateId?: string;
}

The GenerateImageParam type provides aspect ratio, resolution, uploaded images, and template ID to the Gemini model to generate an image.

export type ImageResponseWithoutId = {
  data: string;
  mimeType: string;
  inlineData: string;
}

export type ImageResponse = ImageResponseWithoutId & {
  id: number;
}

export type ImageTokenUsage = {
  image: ImageResponse,
}

The ImageTokenUsage type stores inline image data, mime type, and a dummy image ID.

import { GenerateImageParam } from '@/features/ai/types/generate-image-param.type';
import { ImageTokenUsage } from '@/features/ai/types/image-response.type';

export interface ImageGenerator {
  generateImage(param: GenerateImageParam): Promise<ImageTokenUsage | undefined>;
}

ImageGenerator interface is a contract that must implement a generateImage method to accept a GenerateImageParam parameter and output a promise of ImageTokenUsage or undefined.

5.2. Injection Token for Image Generator

import { FirebaseService } from '@/features/ai/services/firebase.service';
import { ImageGenerator } from '@/shared/ui/gen-media/interfaces/image-generator.interface';
import { InjectionToken, inject } from '@angular/core';

export const IMAGE_GENERATOR_TOKEN = new InjectionToken<ImageGenerator>('IMAGE_GENERATOR_TOKEN', {
  providedIn: 'root',
  factory: () => inject(FirebaseService)
});

The IMAGE_GENERATOR_TOKEN injection token uses the factory function to inject FirebaseService by default. It can be overridden to use the ServerTemplateService when the URL path is template-prompt/:featureId.

5.3. Injection Token for Server Template Model

import { InjectionToken } from '@angular/core';
import { AI, TemplateGenerativeModel } from 'firebase/ai';

export const SERVER_TEMPLATE_MODEL = new InjectionToken<TemplateGenerativeModel>('SERVER_TEMPLATE_MODEL');

The SERVER_TEMPLATE_MODEL injection token injects an instance of TemplateGenerativeModel

Then, the provideFirebase function is updated to instantiate a TemplateGenerativeModel and provide it.

export function provideFirebase() {
    return makeEnvironmentProviders([
        {
          provide: VERTEX_AI_BACKEND,
          useFactory: () => {
            const configService = inject(ConfigService);
            const vertexAILocation = getValue(configService.remoteConfig, 'vertexAILocation').asString();
            const ai = getAI(configService.app, {
              backend: new VertexAIBackend(vertexAILocation)
            });

            return ai;
          }
        },
        {
          provide: SERVER_TEMPLATE_MODEL,
          useFactory: () => {
            const ai = inject(VERTEX_AI_BACKEND); 
            return getTemplateGenerativeModel(ai);
          }
        }
    ]);
}

5.4. Server Prompt Template Service

export async function makeTemplateVariables({ imageFiles, aspectRatio, resolution }: GenerateImageParam) {
  const imageParts = await resolveImageParts(imageFiles);
  const inlineImages = imageParts.map(part => part.inlineData);
  return {
    inlineImages,
    aspectRatio,
    resolution
  }
}

The makeTemplateVariables function converts Files[] to an array of inline image data before returning an object of inline images, aspect ratio, and resolution.

function processImageGeneratedContent(result: GenerateContentResult): ImageTokenUsage {
  const response = result.response;
  const inlineDataParts = response.inlineDataParts();

  if (inlineDataParts?.length) {
    const images = inlineDataParts.map(({inlineData}, index) => {
      const { data, mimeType } = inlineData;
      return {
        id: index,
        mimeType,
        data,
        inlineData: `data:${mimeType};base64,${data}`
      };
    });

    if (images.length <= 0) {
      throw new Error('Error in generating the image.');
    }

    return {
      image: images[0],
    };
  }

  throw new Error('Error in generating the image.');
}

export async function getTemplateBase64Images({ model, templateId, templateVariables }: TemplateImageOptions): Promise<ImageTokenUsage> {
  const result = await model.generateContent(templateId, templateVariables);
  return processImageGeneratedContent(result);
}

The getTemplateBase64Images function uses the model to generate an image, calls processImageGeneratedContent to post-process the result, and returns the ID, MIME type, inline data, and Base64-encoded string.

import { SERVER_TEMPLATE_MODEL } from '@/features/ai/constants/firebase.constant';
import { GenerateImageParam } from '@/features/ai/types/generate-image-param.type';
import { ImageTokenUsage } from '@/features/ai/types/image-response.type';
import { getTemplateBase64Images } from '@/features/ai/utils/generate-image.util';
import { makeTemplateVariables } from '@/features/ai/utils/inline-image-data.util';
import { inject, Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class ServerTemplateService  {
    private readonly serverTemplateModel = inject(SERVER_TEMPLATE_MODEL);

    async generateImage(genImageParameter: GenerateImageParam): Promise<ImageTokenUsage | undefined> {
        const { templateId } = genImageParameter;
        if (!templateId) {
          return undefined;
        }

        const templateVariables = await makeTemplateVariables(genImageParameter);
        return getTemplateBase64Images({
          model: this.serverTemplateModel,
          templateId,
          templateVariables,
        });
    }
}

The ServerTemplateService fulfills the contract of ImageGenerator and implements generateImage to call serverTemplateModel.

6. Angular Route Definition

import { ServerTemplateService } from '@/features/ai/services/server-template.service';
import { IMAGE_GENERATOR_TOKEN } from '@/shared/ui/gen-media/constants/image-generator.token';
import { GenMediaService } from '@/shared/ui/gen-media/services/gen-media.service';
import { Routes } from '@angular/router';

export const routes: Routes = [
  {
    path: 'predefined-prompt/:featureId',
    loadComponent: () => import('./features/predefined-prompt-editor/predefined-prompt-editor.component'),
  },
  {
    path: 'template-prompt/:featureId',
    loadComponent: () => import('./features/predefined-prompt-editor/predefined-prompt-editor.component'),
    providers: [
      GenMediaService,
      { provide: IMAGE_GENERATOR_TOKEN, useExisting: ServerTemplateService }
    ],
  },
  ... other routes ...
];

The routes array specifies a list of paths to route to different components to demonstrate use cases of image generation. The PredefinedPromptEditorComponent consists of an uploader that allows users to upload at least one image to prompt gemini-3.1-flash-image to generate a new image.

Use this component in two scenarios: programmatically passing the prompt text, or using Firebase Server Prompt Templates.

When the path is predefined-prompt/:featureId, the prompt text is submitted to gemini-3.1-flash-image directly. When the path is template-prompt/:featureId, the server prompt template is used.

In the former case, the component uses the FirebaseService that IMAGE_GENERATOR_TOKEN provides in its factory function. In the latter case, the route creates an instance of GenMediaService and does not use the global one. It also provides ServerTemplateService to IMAGE_GENERATOR_TOKEN.

@Injectable({
  providedIn: 'root'
})
export class GenMediaService {
  private readonly imageGenerator = inject(IMAGE_GENERATOR_TOKEN);

  ... the rest of the service ...
}

When GenMediaService injects IMAGE_GENERATOR_TOKEN, imageGenerator is mapped to the ServerTemplateService instead of FirebaseService.

Next, update the navigation menu to use /template-prompt/bottle to call the new template.

7. Update the Navigation Menu

"modeling": {
    "figurine": {
      "path": "/predefined-prompt/figurine",
      "customPrompt": "... custom prompt ..."
    },
    "bottle": {
      "path": "/template-prompt/bottle",
      "templateConfigName": "glassBottleSouvenirTemplateId"
    },
  }

In the features JSON file, the path of bottle is updated to /template-prompt/bottle. Delete customPrompt and add templateConfigName to store the Firebase Remote Config name.

glassBottleSouvenirTemplateId references the template Id, glass-bottle-souvenir-v0-0-1, to load the template to generate the image.

When the Angular application makes the request to Firebase AI Logic, the network payload does not reveal the prompt text.

8. Verify the Network Request

The network payload includes the aspect ratio, resolution, and inline image data. Firebase hides the prompt text, preventing it from being stored as a static value in the JSON file. If prompt text is sensitive data of an application, it is secured in the Firebase's infrastructure.

9. Conclusion

This concludes the journey of migrating the static prompt text to Firebase AI Logic Server Prompt Template.

After the migration, the Angular application does not require redeployment when the server prompt is modified. Users reload the page and they can use the latest prompt to generate images.

Engineers can build AI applications with Firebase AI Logic Server Prompt Templates to perform tasks beyond image generation, such as summarization, text generation, and tool use via Google Search and Google Maps.

Resources

How to Securely Manage AI Prompts in Angular with Firebase

Connie Leung — Mon, 15 Jun 2026 18:29:52 +0000

Firebase released Server Prompt Templates to host prompt templates in its infrastructure. The template follows the DotPrompt format and syntax, so the content can have one or more of the following:

Model name
Model configuration
Input validation and schema
Output schema
Tool user
System instruction
User prompt

Server Prompt Templates resolve several key enterprise AI pain points.

Pain Point	Description
Better Security	The prompt text is stored in the server side, so it cannot be exposed in the network call. Users cannot open the Network tab of the Chrome browser and inspect the prompt text in the payload.
Better Guardrail	Prompt texts are not revealed, so malicious users cannot modify the prompt easily to trigger prompt injection and other attacks to Gemini models
No Prompt Drift	Engineer A edits a prompt locally, forgets to commit, and deploys the code changes. Engineer B uses the old prompt for development, and there are two versions scattered around. Server prompt templates ensure engineers use the same version for development. When the prompt is updated on the server, it is propagated to all instances of the client application.
Testing in Console	Engineers can verify the prompts are working in the Firebase Console before writing a line of code.
Less Deployments	When prompts are updated in the server side, client applications receive the prompt updates without redeployment.

1. Prerequisites

Angular 19
TailwindCSS
Node 22
gemini-3.1-flash-image (also known as Nano Banana 2)
Firebase AI Logic
Firebase Cloud Functions
Firebase Remote Config
Firebase Local Emulator Suite

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and re-login to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the screens to set up Firebase Cloud Function, Firebase Local Emulator Suite and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

After completing the step-by-step, the Firebase tools will generate function and remote config templates, and configuration files such as .firebaserc and firebase.json.

The next section has the details of the implementation repository.

2. Source Code

The full source code for this project is available in the NG Firebase AI Nano Banana, however, the following sections describe the code changes made to migrate to Firebase Server Prompt Templates.

3. Architecture

4. Server Prompt Template Creation

Users can click the Create Template button to create a new prompt on the server side.

A template is configured to generate a glass bottle image from inline image data. The unique template ID is glass-bottle-souvenir-v0-0-1, and the template name is glass-bottle-souvenir.

4.1. Model Configuration

---
model: "gemini-3.1-flash-image"
config:
  candidateCount: 1
  safetySettings:
    - category: HARM_CATEGORY_HARASSMENT
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_HATE_SPEECH
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_SEXUALLY_EXPLICIT
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_DANGEROUS_CONTENT
      threshold: BLOCK_ONLY_HIGH
input:
  schema:
    inlineImages?(array, inline image data):
      type: object
      properties:
        mimeType: string
        data: string  # inline data must be base64-encoded
    aspectRatio?: string, the aspect ratio of the image
    resolution?: string, the resolution of the image
---

The configuration specifies the model name, model configuration, and input schema and validations.

Section	Configuration	Description
model	gemini-3.1-flash-image	The Gemini model name of Nano Banana 2.
config	candidateCount: 1	The model returns at most 1 image
safetySettings	BLOCK_ONLY_HIGH	Safety category of harassment, hate speech, sexually explicit content, and dangerous content
input	schema	Input schema and validation

This prompt expects an array of inlineImages of type object. Each inline image contains a MIME type and inline data. Moreover, the prompt accepts an optional aspect ratio and resolution.

4.2. System Instructions

The prompt parts has {{role "system"}} syntax to specify the system instructions, and {{role "user"}} to specify the user prompt.

{{role "user"}}
A 1/7 scale commercialized collectible ... with realistic lighting and shadows.
{{#if aspectRatio}}
Apply this aspect ratio to the image: {{aspectRatio}}.
{{/if}}
{{#if resolution}}
Apply this resolution to the image: {{resolution}}.
{{/if}}

{{#each inlineImages}}
  {{media type="mimeType" data="data"}}
{{/each}}

The user prompt generates a souvenir glass bottle image from the uploaded inline image.

When the aspect ratio is provided, "Apply this aspect ratio to the image: {{aspectRatio}}." is appended to the prompt.

When the resolution is provided, "Apply this resolution to the image: {{resolution}}." is appended to the prompt.

The loop iterates the inlineImages list to specify the mime type and the inline data.

4.3. Testing the Prompt in Firebase Console

// Prompt Input
{
   "inline_images": [{
    "mime_type": "image/png",
    "contents": "iVBORw0KGgoAAAANSUhEUgAAARAAAABcCAYAAACm+q2AAAXGElEQVR4Ae1dC5QcVZm..."
  }],
   "aspectRatio": "4:1",
   "resolution": "512"
}

The prompt input includes an image, aspect ratio, and resolution for testing before writing a line of code.

The test request generates a souvenir glass bottle with the expected aspect ratio.

5. Server Prompt Template Service Implementation

5.1. Image Generator Interface

export type BaseGenerateParam = {
  aspectRatio?: string;
  resolution?: string;
  imageFiles: File[];
}

export type GenerateImageParam = BaseGenerateParam &  {
  prompt?: string;
  templateId?: string;
}

The GenerateImageParam type provides aspect ratio, resolution, uploaded images, and template ID to the Gemini model to generate an image.

export type ImageResponseWithoutId = {
  data: string;
  mimeType: string;
  inlineData: string;
}

export type ImageResponse = ImageResponseWithoutId & {
  id: number;
}

export type ImageTokenUsage = {
  image: ImageResponse,
}

The ImageTokenUsage type stores inline image data, mime type, and a dummy image ID.

import { GenerateImageParam } from '@/features/ai/types/generate-image-param.type';
import { ImageTokenUsage } from '@/features/ai/types/image-response.type';

export interface ImageGenerator {
  generateImage(param: GenerateImageParam): Promise<ImageTokenUsage | undefined>;
}

ImageGenerator interface is a contract that must implement a generateImage method to accept a GenerateImageParam parameter and output a promise of ImageTokenUsage or undefined.

5.2. Injection Token for Image Generator

import { FirebaseService } from '@/features/ai/services/firebase.service';
import { ImageGenerator } from '@/shared/ui/gen-media/interfaces/image-generator.interface';
import { InjectionToken, inject } from '@angular/core';

export const IMAGE_GENERATOR_TOKEN = new InjectionToken<ImageGenerator>('IMAGE_GENERATOR_TOKEN', {
  providedIn: 'root',
  factory: () => inject(FirebaseService)
});

5.3. Injection Token for Server Template Model

import { InjectionToken } from '@angular/core';
import { AI, TemplateGenerativeModel } from 'firebase/ai';

export const SERVER_TEMPLATE_MODEL = new InjectionToken<TemplateGenerativeModel>('SERVER_TEMPLATE_MODEL');

The SERVER_TEMPLATE_MODEL injection token injects an instance of TemplateGenerativeModel

Then, the provideFirebase function is updated to instantiate a TemplateGenerativeModel and provide it.

export function provideFirebase() {
    return makeEnvironmentProviders([
        {
          provide: VERTEX_AI_BACKEND,
          useFactory: () => {
            const configService = inject(ConfigService);
            const vertexAILocation = getValue(configService.remoteConfig, 'vertexAILocation').asString();
            const ai = getAI(configService.app, {
              backend: new VertexAIBackend(vertexAILocation)
            });

            return ai;
          }
        },
        {
          provide: SERVER_TEMPLATE_MODEL,
          useFactory: () => {
            const ai = inject(VERTEX_AI_BACKEND); 
            return getTemplateGenerativeModel(ai);
          }
        }
    ]);
}

5.4. Server Prompt Template Service

export async function makeTemplateVariables({ imageFiles, aspectRatio, resolution }: GenerateImageParam) {
  const imageParts = await resolveImageParts(imageFiles);
  const inlineImages = imageParts.map(part => part.inlineData);
  return {
    inlineImages,
    aspectRatio,
    resolution
  }
}

The makeTemplateVariables function converts Files[] to an array of inline image data before returning an object of inline images, aspect ratio, and resolution.

function processImageGeneratedContent(result: GenerateContentResult): ImageTokenUsage {
  const response = result.response;
  const inlineDataParts = response.inlineDataParts();

  if (inlineDataParts?.length) {
    const images = inlineDataParts.map(({inlineData}, index) => {
      const { data, mimeType } = inlineData;
      return {
        id: index,
        mimeType,
        data,
        inlineData: `data:${mimeType};base64,${data}`
      };
    });

    if (images.length <= 0) {
      throw new Error('Error in generating the image.');
    }

    return {
      image: images[0],
    };
  }

  throw new Error('Error in generating the image.');
}

export async function getTemplateBase64Images({ model, templateId, templateVariables }: TemplateImageOptions): Promise<ImageTokenUsage> {
  const result = await model.generateContent(templateId, templateVariables);
  return processImageGeneratedContent(result);
}

import { SERVER_TEMPLATE_MODEL } from '@/features/ai/constants/firebase.constant';
import { GenerateImageParam } from '@/features/ai/types/generate-image-param.type';
import { ImageTokenUsage } from '@/features/ai/types/image-response.type';
import { getTemplateBase64Images } from '@/features/ai/utils/generate-image.util';
import { makeTemplateVariables } from '@/features/ai/utils/inline-image-data.util';
import { inject, Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class ServerTemplateService  {
    private readonly serverTemplateModel = inject(SERVER_TEMPLATE_MODEL);

    async generateImage(genImageParameter: GenerateImageParam): Promise<ImageTokenUsage | undefined> {
        const { templateId } = genImageParameter;
        if (!templateId) {
          return undefined;
        }

        const templateVariables = await makeTemplateVariables(genImageParameter);
        return getTemplateBase64Images({
          model: this.serverTemplateModel,
          templateId,
          templateVariables,
        });
    }
}

The ServerTemplateService fulfills the contract of ImageGenerator and implements generateImage to call serverTemplateModel.

6. Angular Route Definition

import { ServerTemplateService } from '@/features/ai/services/server-template.service';
import { IMAGE_GENERATOR_TOKEN } from '@/shared/ui/gen-media/constants/image-generator.token';
import { GenMediaService } from '@/shared/ui/gen-media/services/gen-media.service';
import { Routes } from '@angular/router';

export const routes: Routes = [
  {
    path: 'predefined-prompt/:featureId',
    loadComponent: () => import('./features/predefined-prompt-editor/predefined-prompt-editor.component'),
  },
  {
    path: 'template-prompt/:featureId',
    loadComponent: () => import('./features/predefined-prompt-editor/predefined-prompt-editor.component'),
    providers: [
      GenMediaService,
      { provide: IMAGE_GENERATOR_TOKEN, useExisting: ServerTemplateService }
    ],
  },
  ... other routes ...
];

Use this component in two scenarios: programmatically passing the prompt text, or using Firebase Server Prompt Templates.

When the path is predefined-prompt/:featureId, the prompt text is submitted to gemini-3.1-flash-image directly. When the path is template-prompt/:featureId, the server prompt template is used.

@Injectable({
  providedIn: 'root'
})
export class GenMediaService {
  private readonly imageGenerator = inject(IMAGE_GENERATOR_TOKEN);

  ... the rest of the service ...
}

When GenMediaService injects IMAGE_GENERATOR_TOKEN, imageGenerator is mapped to the ServerTemplateService instead of FirebaseService.

Next, update the navigation menu to use /template-prompt/bottle to call the new template.

7. Update the Navigation Menu

"modeling": {
    "figurine": {
      "path": "/predefined-prompt/figurine",
      "customPrompt": "... custom prompt ..."
    },
    "bottle": {
      "path": "/template-prompt/bottle",
      "templateConfigName": "glassBottleSouvenirTemplateId"
    },
  }

In the features JSON file, the path of bottle is updated to /template-prompt/bottle. Delete customPrompt and add templateConfigName to store the Firebase Remote Config name.

glassBottleSouvenirTemplateId references the template Id, glass-bottle-souvenir-v0-0-1, to load the template to generate the image.

When the Angular application makes the request to Firebase AI Logic, the network payload does not reveal the prompt text.

8. Verify the Network Request

9. Conclusion

This concludes the journey of migrating the static prompt text to Firebase AI Logic Server Prompt Template.

After the migration, the Angular application does not require redeployment when the server prompt is modified. Users reload the page and they can use the latest prompt to generate images.

Resources

Building Dynamic Audio with Emotion & Pace: Gemini 3.1 Flash TTS, Angular & Firebase Cloud Functions

Connie Leung — Sun, 07 Jun 2026 13:26:05 +0000

Google released the Gemini 3.1 Flash TTS Preview model for AI audio generation in the Gemini API, Gemini in Vertex AI, and Gemini AI Studio. This model introduces a new Audio tags feature to exhibit expressive human emotion, pace, and style.

This application explores Firebase AI Logic to analyze an uploaded image to generate recommendations, description, alternative tags, and an obscure fact. The obscure fact is sent to a Firebase Cloud Function to generate an audio using a Gemini TTS model. The Cloud Function returns the stream to an Angular application that converts it to a Blob URL object. An audio player sets the URL to the source that users can click the Play button to play the stream.

In this blog post, I migrate my application to use the Gemini 3.1 Flash TTS Preview model and create a signal form in Angular to input a scene, emotion, and pace. Then, the Angular application provides the form values and the obscure fact to the Firebase Cloud Function to generate an expressive voice using the GenAI TypeScript SDK.

Prerequisites

The technical stack of the project:

Angular 21: The latest version as of May 2026.
Node.js LTS: The LTS version as of May 2026.
Firebase Remote Config: To manage dynamic parameters.
Firebase Cloud Functions: To generate an expressive human voice when called by the frontend.
Firebase Local Emulator Suite: To test the functions locally at http://localhost:5001.
Gemini in Vertex AI: To generate videos and store them in Firebase Cloud Storage.

The public Google AI Studio API is restricted in my region (Hong Kong). However, Vertex AI (Google Cloud) offers enterprise access that works reliably here, so I chose Vertex AI for this demo.

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and log in again to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the prompts to set up Firebase Cloud Functions, the Firebase Local Emulator Suite, Firebase Cloud Storage, and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

In both cases, the Firebase CLI automatically installs the firebase-admin and firebase-functions dependencies.

After completing the setup steps, the Firebase tools generate the functions emulator, functions, a storage rules file, remote config templates, and configuration files such as .firebaserc and firebase.json.

Angular dependency

npm i firebase

The Angular application requires the firebase dependency to initialize a Firebase app, load remote config, and invoke the Firebase Cloud Functions to generate videos.

Firebase dependencies

npm i @cfworker/json-schema @google/genai @modelcontextprotocol/sdk

Install the above dependencies to access Gemini in Vertex AI. @google/genai depends on @cfworker/json-schema and @modelcontextprotocol/sdk. Without these, the Cloud Functions cannot start.

With our project configured, let's look at how the frontend and backend communicate.

Architecture

A user uploads an image in an Angular application and prompts the Gemini 3.1 Flash Lite Preview model to generate a few recommendations for improving the image, a description, and alternative tags. The user also uses the same model and the Google Search tool to find an obscure fact related to the image.

A user inputs a scene, an emotion, and a pace in an experimental signal form. When a user clicks the generate audio button, the Angular application sends the form values and the obscure fact to the Firebase Cloud Function to generate an expressive voice using the GenAI TypeScript SDK and Gemini 3.1 Flash TTS Preview model.

Limitations of Gemini 3.1 Flash TTS Preview Model

The model can only accept text inputs and generate audio outputs.
The context window is 32K tokens
TTS does not support streaming.
The supported languages can be found in https://ai.google.dev/gemini-api/docs/speech-generation#languages. My mother tongue, Cantonese, is currently unsupported.

Firebase Integration

1. Configure Environment Variables

Defining the environment variables in the Firebase project ensures the functions know the region of the Google Cloud project, the Firebase Cloud Function location, and the required TTS model.

.env.example

GOOGLE_CLOUD_LOCATION="global"
GOOGLE_FUNCTION_LOCATION="asia-east2"
GEMINI_TTS_MODEL_NAME="gemini-3.1-flash-tts-preview"
WHITELIST="http://localhost:4200"
REFERER="http://localhost:4200/"

Variable	Description
GOOGLE_CLOUD_LOCATION	The region of the Google Cloud project. I chose `global` so that the Firebase project has access to the newest Gemini 3.1 Flash TTS preview model.
GOOGLE_FUNCTION_LOCATION	The region of the Firebase Cloud Functions. I chose `asia-east2` because this is the region where I live.
WHITELIST	Requests must come from http://localhost:4200.
REFERER	Requests originate from http://localhost:4200/.

http://localhost:4200 is the host and port number of my local Angular application.

2. Validating Environment Variables

Before the Cloud Function proceeds with any AI calls, it is critical to ensure that all necessary environment variables are present. I implemented an AUDIO_CONFIG IIFE (Immediately Invoked Function Expression) to validate environment variables like the TTS model name, Google Cloud Project ID, and location.

import logger from "firebase-functions/logger";

export function validate(value: string | undefined, fieldName: string, missingKeys: string[]) {
    const err = `${fieldName} is missing.`;
    if (!value) {
        logger.error(err);
        missingKeys.push(fieldName);
        return "";
    }

    return value;
}

export const AUDIO_CONFIG = (() => {
    logger.info("AUDIO_CONFIG initialization: Loading environment variables and validating configuration...");

    const env = process.env;

    const missingKeys: string[] = [];
    const location = validate(env.GOOGLE_CLOUD_LOCATION, "Vertex Location", missingKeys);
    const model = validate(env.GEMINI_TTS_MODEL_NAME, "Gemini TTS Model Name", missingKeys);
    const project = validate(env.GCLOUD_PROJECT, "Google Cloud Project", missingKeys);

    if (missingKeys.length > 0) {
        throw new HttpsError("failed-precondition", `Missing environment variables: ${missingKeys.join(", ")}`);
    }

    return {
        genAIOptions: {
            project,
            location,
            vertexai: true,
        },
        model,
    };
})();

I am using Node 24 as of May 2026. Since Node 20, we can use the built-in process.loadEnvFile function that loads environment variables from the .env file.

In env.ts, the try-catch block attempts to load the environment variables from the .env file.

try {
    process.loadEnvFile();
} catch {
    // Ignore error if .env file is not found (e.g., in production where env vars are set by the platform)
}

In src/index.ts, the first statement imports the env.ts before importing other files and libraries.

import "./env";

... other import statements ...

If you are using a Node version that does not support process.loadEnvfile, the alternative is to install dotenv to load the environment variables.

npm i dotenv

import dotenv from "dotenv";

dotenv.config();

Firebase provides the GCLOUD_PROJECT variable, so it is not defined in the .env file.

When the missingKeys array is not empty, AUDIO_CONFIG throws an error that lists all the missing variable names. If the validation is successful, the genAIOptions and model are returned. The genAIOptions is used to initialize the GoogleGenAI and model is the selected TTS model name.

3. Sanitize the Prompt Inputs

The Cloud Function sanitizes the scene and transcript before composing the audio prompt.

The sanitizeScene function accepts the scene by escaping the newline character ('\n') with the '\\n'. The newline character creates a blank line and often signals the end of a block. The sanitization effectively flattens the scene into one continuous line of data and the LLM's Markdown parser recognizes it as a single, safe paragraph. The sanitization also removes all Markdown headers that are injected into the scene.

function sanitizeScene(text: string): string {
    return (text || "").trim().replace(/\r?\n/g, "\\n").replace(/^[#\s]+/gm, "");
}

The sanitizeTranscript function accepts the transcript by removing all Markdown headers and triple quotes that are injected into it.

function sanitizeTranscript(text: string): string {
    return (text || "").trim().replace(/^#+/gm, "").replace(/"""/g, '"');
}

4. Build an Audio Prompt

The AudioPrompt interface encapsulates the scene, emotion, pace, transcript, and voice option to set the location, audio tags, text, and persona of the audio.

export type AudioPrompt = {
  scene: string;
  emotion: string;
  pace: string;
  transcript: string;
  voiceOption: string;
}

The SCENE_DICTIONARY is an array of scenes. When the user does not provide a scene, a scene is randomly selected from the array.

export const SCENE_DICTIONARY = [
    "A dimly lit, dusty library filled with ancient leather-bound books.\n" +
        "The air is thick with history. A scholarly archivist is leaning closely into a warm, vintage ribbon microphone.\n" +
        "They speak with an infectious, hushed intensity, eager to share a forgotten secret they just uncovered in a decaying manuscript.",

    "It is 10:00 PM in a glass-walled studio overlooking the moonlit London skyline, but inside, it is blindingly bright.\n" +
        "The red 'ON AIR' tally light is blazing. The speaker is standing up, bouncing on the balls of their heels to the rhythm of a thumping backing track.\n" +
        "It is a chaotic, caffeine-fueled cockpit designed to wake up an entire nation.",

    "A meticulously sound-treated bedroom in a suburban home.\n" +
        "The space is deadened by plush velvet curtains and a heavy rug, creating an intimate, close-up acoustic environment.\n" +
        "The speaker delivers the information like a trusted friend sharing an inside joke.",

    "A high-tech, minimalist laboratory humming with servers.\n" +
        "Crisp, clean acoustics reflect off glass and steel.\n" +
        "A brilliant but eccentric scientist is pacing back and forth, speaking rapidly and enthusiastically into a headset microphone, excited to explain a complex phenomenon.",
];

I define a buildAudioPrompt function to construct the advanced audio prompt.
When an emotion is defined, the tag is [<emotion>]. When a pace is defined, the tag is [<pace>]. The combined audio tag is [<emotion>] [<pace>]<a space> to create a proper token boundary.

The insertAudioTagsToTranscript uses a regular expression to split the transcript into lines, inserts the combined audio tag before each line, and then joins them with an empty string.

The buildAudioPrompt concatenates the scene and the expressive transcript into a string before returning it.

import { SCENE_DICTIONARY } from './constants/scenes.const';
import { AudioPrompt } from './types/audio-prompt.type';

function makeTag(value: string) {
    const trimmedValue = value.trim();
    return trimmedValue ? `[${trimmedValue}] ` : "";
}

function insertAudioTagsToTranscript({ transcript, pace, emotion }: AudioPrompt): string {
    const audioTags = `${makeTag(emotion)}${makeTag(pace)}`;
    const cleanedTranscript = sanitizeTranscript(transcript);

    const parts = cleanedTranscript.split(/(?<!\b(?:Mr|Mrs|Ms|Dr|St|i\.e|e\.g))([.!?\n\r]+[”"’']*\s*)/);
    return parts
        .map((text, i, arr) => {
            if (i % 2 !== 0) {
                return ""; // Skip delimiters, they are appended to the text blocks
            }
            const delimiter = arr[i + 1] || "";
            return text.trim() ? `${audioTags}${text.trim()}${delimiter}` : delimiter;
        })
        .join("");
}

export function buildAudioPrompt(data: AudioPrompt): string {
    const randomIndex = Math.floor(Math.random() * SCENE_DICTIONARY.length);
    const selectedScene = SCENE_DICTIONARY[randomIndex];

    const trimmedScene = (data.scene || "").trim() || selectedScene;
    const escapedScene = sanitizeScene(trimmedScene);
    const transcript = insertAudioTagsToTranscript(data);

    return `## Scene:
${escapedScene}

## Transcript:
"""
${transcript}
"""
`;
}

The output of the prompt looks like:

## Scene:
<scene>

## Transcript:
[<emotion>] [<pace>] <sentence 1>[<emotion>] [<pace>] <sentence 2>...[<emotion>] [<pace>] <sentence N>

5. Generating an Expression Human Audio in a Firebase Cloud Function

The createVoiceConfig function constructs an instance of GenerateContentConfig that outputs a speech narrated by the given voice name.

import { GenerateContentConfig } from "@google/genai";

export function createVoiceConfig(voiceName = "Kore"): GenerateContentConfig {
    return {
        responseModalities: ["audio"],
        speechConfig: {
            voiceConfig: {
                prebuiltVoiceConfig: {
                    voiceName,
                },
            },
        },
    };
}

const splitList = (whitelist?: string) => (whitelist || "").split(",").map((origin) => origin.trim());

export const whitelist = splitList(process.env.WHITELIST);
export const cors = whitelist.length > 0 ? whitelist : true;
export const refererList = splitList(process.env.REFERER);

All Cloud Functions enforce App Check, CORS, and a timeout period of 600 seconds. If WHITELIST is unspecified, CORS defaults to true. While acceptable in a demo environment, configure CORS to a specific domain or false in production to prevent unauthorized access.

The readFact cloud function delegates to readFactStreamFunction when isStreaming is true. Otherwise, it is delegated to readFactFunction.

The readFactFunction function returns a Promise<string> that is the encoded base64 string.

The readFactStreamFunction functions returns a Promise<number[] | undefined> that represents a buffer of WAV header bytes.

import { onCall } from "firebase-functions/v2/https";
import { cors } from "../auth";
import { buildAudioPrompt } from './audio-prompt';
import { readFactFunction, readFactFunctionStream } from "./read-fact";
import { createVoiceConfig } from './voice-config';

const options = {
    cors,
    enforceAppCheck: true,
    timeoutSeconds: 600,
};

export const readFact = onCall(options, (request, response) => {
    const { data, acceptsStreaming } = request;
    const isStreaming = acceptsStreaming && !!response;
    const prompt = buildAudioPrompt(data);
    const voiceOption = createVoiceConfig(data.voiceOption);

    return isStreaming
        ? readFactStreamFunction(prompt, voiceOption, response)
        : readFactFunction(prompt, voiceOption);
});

The withAIAudio function is a high-order function that calls the callback to generate an audio stream.

async function withAIAudio(callback: (ai: GoogleGenAI, model: string) => Promise<string | number[] | undefined>) {
    try {
        const variables = AUDIO_CONFIG;
        if (!variables) {
            return "";
        }

        const { genAIOptions, model } = variables;
        const ai = new GoogleGenAI(genAIOptions);
        return await callback(ai, model);
    } catch (e) {
        if (e instanceof HttpsError) {
            throw e;
        }
        throw new HttpsError("internal", "An internal error occurred while setting up the AI client.", {
            originalError: (e as Error).message,
        });
    }
}

generateAudio is a callback function that uses the Gemini 3.1 Flash TTS Preview model to generate a response. getBase64DataUrl invokes extractInlineAudioData to extract the raw data and the mime type from the response. The encodeBase64String function first converts the raw data to WAV format, then encodes it to base64 format, and finally returns the base64 string.

The createAudioParams function constructs a parameter with the Gemini TTS model, the audio prompt, and the speech configuration.

async function generateAudio(aiTTS: AIAudio, prompt: string, voiceOption: GenerateContentConfig) {
    try {
        const { ai, model } = aiTTS;
        const response = await ai.models.generateContent(createAudioParams(model, prompt, voiceOption));
        return getBase64DataUrl(response);
    } catch (error) {
        console.error(error);
        throw error;
    }
}

function createAudioParams(model: string, prompt: string, config?: GenerateContentConfig) {
    return {
        model,
        contents: [
            {
                role: "user",
                parts: [
                    {
                        text: prompt,
                    },
                ],
            },
        ],
        config,
    };
}

function extractInlineAudioData(response: GenerateContentResponse): {
    rawData: string | undefined;
    mimeType: string | undefined;
} {
    const { data: rawData, mimeType } = response.candidates?.[0]?.content?.parts?.[0]?.inlineData ?? {};

    return { rawData, mimeType };
}

function getBase64DataUrl(response: GenerateContentResponse) {
    const { rawData, mimeType } = extractInlineAudioData(response);

    if (!rawData || !mimeType) {
        throw new Error("Audio generation failed: No audio data received.");
    }

    return encodeBase64String({ rawData, mimeType });
}

export function encodeBase64String({ rawData, mimeType }: RawAudioData) {
    const wavBuffer = convertToWav(rawData, mimeType);
    const base64Data = wavBuffer.toString("base64");
    return `data:audio/wav;base64,${base64Data}`;
}

generateAudioStream is a callback function that uses the Gemini 3.1 Flash TTS Preview model to stream a list of audio chunks. The chunks are iterated so that each chunk is passed to the extractInlineAudioData function to extract the raw data and the mime type. The function converts the chunk's raw data into a buffer and sends it to the client; the byte length accumulates to determine the total size of all chunks.

After all the chunks are sent to the client, the createWavHeader function uses the total byte length and the audio options to construct a WAV header and returns it.

async function generateAudioStream(
    aiTTS: AIAudio,
    prompt: string,
    voiceOption: GenerateContentConfig,
    response: CallableResponse<unknown>,
): Promise<number[] | undefined> {
    try {
        const { ai, model } = aiTTS;
        const chunks = await ai.models.generateContentStream(createAudioParams(model, prompt, voiceOption));
        let byteLength = 0;
        let options: WavConversionOptions | undefined = undefined;
        for await (const chunk of chunks) {
            const { rawData, mimeType } = extractInlineAudioData(chunk);
            if (!options && mimeType) {
                options = parseMimeType(mimeType);
                response.sendChunk({
                    type: "metadata",
                    payload: {
                        sampleRate: options.sampleRate,
                    },
                });
            }

            if (rawData && mimeType) {
                const buffer = Buffer.from(rawData, "base64");
                byteLength = byteLength + buffer.length;
                response.sendChunk({
                    type: "data",
                    payload: {
                        buffer,
                    },
                });
            }
        }

        if (options && byteLength > 0) {
            const header = createWavHeader(byteLength, options);
            return [...header];
        }

        return undefined;
    } catch (error) {
        console.error(error);
        throw error;
    }
}

The readFactFunction invokes the withAIAudio high-order function to generate a base64-encoded string.

The readFactStreamFunction function calls the withAIAudio high-order function to write chunks to the response body and send them to the client. Then, the generateAudioStream function returns the bytes of the WAV header.

export async function readFactFunction(prompt: string, voiceOption: GenerateContentConfig) {
    return withAIAudio((ai, model) => generateAudio({ ai, model }, prompt, voiceOption));
}

export async function readFactStreamFunction(prompt: string, voiceOption: GenerateContentConfig, response: CallableResponse<unknown>) {
    return withAIAudio((ai, model) => generateAudioStream({ ai, model }, prompt, voiceOption, response));
}

6. Firebase App Configuration and reCAPTCHA Site Key

I implemented a FIREBASE_APP_CONFIG IIFE (Immediately Invoked Function Expression) to run once to validate the environment variables of the Firebase app.

export const FIREBASE_APP_CONFIG = (() => {
    const env = process.env;
    const missingKeys: string[] = [];
    const apiKey = validate(env.APP_API_KEY, "API Key", missingKeys);
    const appId = validate(env.APP_ID, "App Id", missingKeys);
    const messagingSenderId = validate(env.APP_MESSAGING_SENDER_ID, "Messaging Sender ID", missingKeys);
    const recaptchaSiteKey = validate(env.RECAPTCHA_ENTERPRISE_SITE_KEY, "Recaptcha site key", missingKeys);
    const projectId = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);

    if (missingKeys.length > 0) {
        throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
    }

    return {
        app: {
            apiKey,
            appId,
            projectId,
            messagingSenderId,
            authDomain: `${projectId}.firebaseapp.com`,
            storageBucket: `${projectId}.firebasestorage.app`,
        },
        recaptchaSiteKey,
    };
})();

The getFirebaseConfig function caches the FIREBASE_APP_CONFIG for an hour before returning it to the Angular application.

The Angular application receives the Firebase app configuration and reCAPTCHA site key from the Cloud Function to initialize Firebase AI Logic and protect resources from unauthorized access and abuse.

export const getFirebaseConfig = onRequest({ cors }, (request, response) => {
    if (!validateRequest(request, response)) {
        return;
    }

    try {
        response.set("Cache-Control", "public, max-age=3600, s-maxage=3600");
        response.json(FIREBASE_APP_CONFIG);
    } catch (err) {
        console.error(err);
        response.status(500).send("Internal Server Error");
    }
});

7. Local Development with Emulators

For local development, I used the Firebase Local Emulator Suite to save cost and time. In the bootstrapFirebase process, the application calls connectFunctionsEmulator to link to the Cloud Functions running at http://localhost:5001.

The port number defaulted to 5001 when firebase init was executed.

function connectEmulators(functions: Functions, remoteConfig: RemoteConfig) {
  if (location.hostname === 'localhost') {
    const host = getValue(remoteConfig, 'functionEmulatorHost').asString();
    const port = getValue(remoteConfig, 'functionEmulatorPort').asNumber();
    connectFunctionsEmulator(functions, host, port);
  }
}

loadFirebaseConfig is a helper function that makes request to the Cloud function to obtain the Firebase App configuration and the reCAPTCHA site key.

{
  "getFirebaseConfigUrl": "http://127.0.0.1:5001/vertexai-firebase-6a64f/us-central1/getFirebaseConfig"
}

export type FirebaseConfigResponse = {
  app: FirebaseOptions;
  recaptchaSiteKey: string
}

import { HttpClient } from '@angular/common/http';
import { inject } from '@angular/core';
import { catchError, lastValueFrom, throwError } from 'rxjs';
import config from '../../public/config.json';
import { FirebaseConfigResponse } from './ai/types/firebase-config.type';

async function loadFirebaseConfig() {
  const httpService = inject(HttpClient);
  const firebaseConfig$ =
    httpService.get<FirebaseConfigResponse>(config.getFirebaseConfigUrl)
      .pipe(catchError((e) => throwError(() => e)));
  return lastValueFrom(firebaseConfig$);
}

The bootstrapFirebase function initializes the FirebaseApp and App Check, loads the Firebase remote configuration and cloud functions, and stores them in the config service for later use.

export async function bootstrapFirebase() {
    try {
      const configService = inject(ConfigService);
      const firebaseConfig = await loadFirebaseConfig();
      const { app, recaptchaSiteKey } = firebaseConfig;
      const firebaseApp = initializeApp(app);
      const remoteConfig = await fetchRemoteConfig(firebaseApp);

      initializeAppCheck(firebaseApp, {
        provider: new ReCaptchaEnterpriseProvider(recaptchaSiteKey),
        isTokenAutoRefreshEnabled: true,
      });

      const functionRegion = getValue(remoteConfig, 'functionRegion').asString();
      const functions = getFunctions(firebaseApp, functionRegion);
      connectEmulators(functions, remoteConfig);

      configService.loadConfig(firebaseApp, remoteConfig, functions);
    } catch (err) {
      console.error(err);
    }
}

The AppConfig remains unchanged.

import { ApplicationConfig, provideAppInitializer } from '@angular/core';
import { bootstrapFirebase } from './app.bootstrap';

export const appConfig: ApplicationConfig = {
  providers: [
    provideAppInitializer(async () => bootstrapFirebase()),
  ]
};

8. Angular Implementation

8.1 Audio Tags Component

I create an AudioTagsComponent and a new signal form to input the scene, emotion, pace, and voice name in the Angular frontend.

<div>
  <h3>
    <span class="text-xl">🎙️</span> Customize Audio Generation
  </h3>

  <div class="grid grid-cols-1 md:grid-cols-2 gap-4">
    <!-- Scene -->
    <div class="flex flex-col gap-1.5 md:col-span-2">
      <label for="scene">Scene Description</label>
      <textarea id="scene" [formField]="audioPromptForm.scene"
      ></textarea>
    </div>

    <!-- Emotion -->
    <div class="flex flex-col gap-1.5">
      <label for="emotion">Vocal Emotion</label>
      <input type="text" id="emotion" [formField]="audioPromptForm.emotion"
        placeholder="e.g., panicked, whispers"
      />
    </div>

    <!-- Pace -->
    <div class="flex flex-col gap-1.5">
      <label for="pace">Speaking Pace</label>
      <input type="text" id="pace" [formField]="audioPromptForm.pace"
        placeholder="e.g., very slow, rapid"
      />
    </div>

    <!-- Voice Option -->
    <div class="flex flex-col gap-1.5 md:col-span-2">
      <label for="voiceOption">AI Voice Model</label>
      <select id="voiceOption" [formField]="audioPromptForm.voiceOption"
      >
        <option value="" disabled selected>Select a voice...</option>
        @for (option of sortedVoiceOptions(); track option.name) {
          <option [value]="option.name" class="bg-slate-800">{{ option.label }}</option>
        }
      </select>
    </div>
  </div>
</div>

import { ChangeDetectionStrategy, Component, computed, signal } from '@angular/core';
import { form, FormField } from '@angular/forms/signals';
import { VOICE_OPTIONS } from './constants/voice-options.const';
import { AudioPromptData } from './types/audio-prompt-data.type';

@Component({
  selector: 'app-audio-tags',
  imports: [FormField],
  templateUrl: './audio-tags.component.html',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class AudioTagsComponent {
    #audioPromptModel = signal<AudioPromptData>({
      scene: 'A news anchor reading the news in a busy newsroom',
      emotion: 'professional, slightly serious',
      pace: 'moderate, clear enunciation',
      voiceOption: 'Kore'
    });
    audioPromptForm = form(this.#audioPromptModel);

    sortedVoiceOptions = computed(() => {
      const sortedList = VOICE_OPTIONS.sort((a, b) => a.name.localeCompare(b.name));

      return sortedList.map(option => ({
        name: option.name,
        label: `${option.name} - ${option.description}`
      }));
    });

    audioPromptModel = this.#audioPromptModel.asReadonly();
}

The AudioTagsComponent is imported into ObscureFactComponent such that users can input values into the experimental signal form.

In the HTML template of ObscureFactComponent, the <app-audio-tags> has a template variable audioTags, and audioTags.audioPromptModel() resolves to an instance of AudioPromptData. The data is assigned to the audioTags property of the generateSpeech method.

<div class="w-full mt-6">
    <app-audio-tags #audioTags />

    <h3>A surprising or obscure fact about the tags</h3>
    @if (interestingFact()) {
      <p>{{ interestingFact() }}</p>

      <app-error-display [error]="ttsError()" />

      <app-text-to-speech
        [isLoadingSync]="isLoadingSync()"
        [isLoadingStream]="isLoadingStream()"
        [isLoadingWebAudio]="isLoadingWebAudio()"
        [audioUrl]="audioUrl()"
        (generateSpeech)="generateSpeech({ mode: $event, audioTags: audioTags.audioPromptModel() })"
        [playbackRate]="playbackRate()"
      />
    } @else {
      <p>The tag(s) does not have any interesting or obscure fact.</p>
    }
</div>

import { AudioPromptData } from './audio-prompt-data.type';
import { GenerateSpeechMode } from '../../generate-audio.util';

export type ModeWithAudioTags = {
  mode: GenerateSpeechMode;
  audioTags: AudioPromptData;
};

export type AudioPrompt = {
  scene: string;
  emotion: string;
  pace: string;
  transcript: string;
  voiceOption: string;
};

The generateSpeech method uses the fact and audioTags to contruct an instance of AudioPrompt. When the mode is stream, the SpeechService calls generateAudioBlobURL to use the audioPrompt to construct a blob URL. When the mode is sync, the SpeechService calls generateAudio to use the audioPrompt to generate an encoded base64 string. When the mode is web_audio_api, the AudioPlayerService calls playStream to stream the audio.

import { SpeechService } from '@/ai/services/speech.service';
import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { ChangeDetectionStrategy, Component, inject, input, OnDestroy, signal } from '@angular/core';
import { revokeBlobURL } from '../blob.util';
import { AudioTagsComponent } from './audio-tags/audio-tags.component';
import { ModeWithAudioTags } from './audio-tags/types/mode-audio-tags.type';
import { generateSpeechHelper, streamSpeechWithWebAudio, ttsError } from './generate-audio.util';
import { AudioPlayerService } from './services/audio-player.service';

@Component({
  selector: 'app-obscure-fact',
  templateUrl: './obscure-fact.component.html',
  imports: [
    TextToSpeechComponent,
  ],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class ObscureFactComponent implements OnDestroy {
  interestingFact = input<string | undefined>(undefined);

  speechService = inject(SpeechService);
  audioPlayerService = inject(AudioPlayerService);

  isLoadingSync = signal(false);
  isLoadingStream = signal(false);
  isLoadingWebAudio = signal(false);

  audioUrl = signal<string | undefined>(undefined);

  ttsError = ttsError;

  async generateSpeech({ mode, audioTags }: ModeWithAudioTags) {
    const fact = this.interestingFact();

    if (fact) {
      revokeBlobURL(this.audioUrl);
      this.audioUrl.set(undefined);

      const audioPrompt = {
          ...audioTags,
          transcript: fact,
      };
      if (mode === 'sync' || mode === 'stream') {
        const loadingSignal = mode === 'stream' ? this.isLoadingStream : this.isLoadingSync;
        const speechFn = (audioPrompt: AudioPrompt) => mode === 'stream' ?
            this.speechService.generateAudioBlobURL(audioPrompt) :
            this.speechService.generateAudio(audioPrompt);

        await generateSpeechHelper(audioPrompt, loadingSignal, this.audioUrl, speechFn);
      } else if (mode === 'web_audio_api') {
        await streamSpeechWithWebAudio(
          audioPrompt,
          this.isLoadingWebAudio,
          (audioPrompt: AudioPrompt) => this.audioPlayerService.playStream(audioPrompt));
      }
    }
  }

  ngOnDestroy(): void {
    revokeBlobURL(this.audioUrl);
  }
}

8.2 Call Firebase Cloud Functions directly

The SpeechService has a generateAudio method that calls the readFact cloud function to obtain the encoded base64 string.

Similarly, the service has a generateAudioBlobURL method that streams the chunks to create a buffer and prepend it with the WAV header. The constructBlobURL creates a blob URL from the Blob Part array.

export function constructBlobURL(parts: BlobPart[]) {
  return URL.createObjectURL(new Blob(parts, { type: 'audio/wav' }));
}

import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { constructBlobURL } from '@/photo-panel/blob.util';
import { inject, Injectable } from '@angular/core';
import { Functions, httpsCallable } from 'firebase/functions';
import { StreamMessage } from '../types/stream-message.type';
import { ConfigService } from './config.service';

@Injectable({
  providedIn: 'root'
})
export class SpeechService  {
    private configService = inject(ConfigService);

    private get functions(): Functions {
      if (!this.configService.functions) {
        throw new Error('Firebase Functions has not been initialized.');
      }
      return this.configService.functions;
    }

    async generateAudio(audioPrompt: AudioPrompt) {
      const readFactFunction = httpsCallable<AudioPrompt, string>(
        this.functions, 'textToAudio-readFact'
      );

      const { data: audioUri } = await readFactFunction(audioPrompt);
      return audioUri;
    }

    async generateAudioStream(audioPrompt: AudioPrompt) {
      const readFactStreamFunction = httpsCallable<AudioPrompt, number[] | undefined, StreamMessage>(
        this.functions, 'textToAudio-readFact'
      );

      return readFactStreamFunction.stream(audioPrompt);
    }

    async generateAudioBlobURL(audioPrompt: AudioPrompt) {
      const { stream, data } = await this.generateAudioStream(audioPrompt);

      const audioParts: BlobPart[] = [];
      for await (const audioChunk of stream) {
        if (audioChunk && audioChunk.type === 'data') {
          audioParts.push(new Uint8Array(audioChunk.payload.buffer.data));
        }
      }

      const wavHeader = await data;
      if (wavHeader && wavHeader.length) {
        audioParts.unshift(new Uint8Array(wavHeader));
      }

      return constructBlobURL(audioParts);
    }
}

Similar to SpeechService.generateAudioBlobURL, the playStream method of AudioPlayerService also calls generateAudioStream to get a stream of chunks and play each of them immediately.

import { SpeechService } from '@/ai/services/speech.service';
import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { inject, Injectable, OnDestroy, signal } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class AudioPlayerService implements OnDestroy  {

  async playStream(audioPrompt: AudioPrompt) {
      const { stream } = await this.speechService.generateAudioStream(audioPrompt);
      for await (const audioChunk of stream) {
        ... process each chunk ...
      }
  }


  ngOnDestroy(): void {
    ... release resources to prevent memory leak ...
  }
}

This is the end of the walkthrough for the demo. You should now be able to input different combinations of scene, emotion, and pace to create a unique personality to say the given text in an audio clip.

9. Caveats and Lessons Learned: Avoiding the Dynamic Prompt Trap

The examples in Gemini AI Studio and Vertex AI Studio use static audio tags and transcripts and they work correctly for me.

When I applied dynamic audio tags and transcripts in the demo, the Gemini 3.1 TTS Flash Preview model ignored the audio tags. The issue was resolved after debugging in Gemini CLI for hours.

Here are the Caveats and Lessons Learned:

The Token Boundary Trap. The code originally concatenated tags and transcript without a space (for example, "[giggle][slow]Before"). The LLM tokenizer failed to recognize the instruction to change the behavior and pace of the audio. My fix was to insert a space between the tags and the transcript, which was "[giggle] [slow] Before".
Sanitize inputs before injecting into the prompt template. The sanitize functions remove Markdown headers (#) and triple quotes from the scene and transcript. The cleansed scene and transcript are injected into the prompt template to construct the final audio prompt.
LLM does not understand idiom. I typed "at a snail's pace" in the signal form and inserted "[at a snail's pace]" before the line. However, the model vocalized the tag literally, and no pace change occurred.
"Repetitive Weighting" is a Real Strategy. If standard tags like [slow] and [fast] are not dramatic enough, prepend the pace with "very" to increase the dramatic effect of the pace. It was evident when [very, very, very slow] generated a longer audio than [slow].
Replace newline character (\n) with \\n. to flatten the lines into a single paragraph. When the scene and transcript are cleansed and escaped, they are injected into the prompt template while the structure is preserved for the LLM parser.

Conclusion

The integration of text-to-speech with Firebase's serverless scalability empowers Angular applications for real-time audio generation.

First, the Angular application neither requires the genai dependency nor stores the Vertex AI environment variables in a .env file. The client application calls the Cloud Functions to perform the text to speech tasks to generate an audio stream.

The Cloud Functions receive arguments from the client, and execute a TTS operation to either return the entire audio as an encoded base64 string or stream the audio bytes in chunks. During local development, the Firebase Emulator calls the functions at http://localhost:5001 instead of the ones deployed on the Cloud Run platform to save cost.

Try cloning the GitHub repository, uploading an image to generate an obscure fact, and using the Gemini 3.1 Flash TTS preview model to speak it with the specified scene, emotion, and pace.

Resources

Building Dynamic Audio with Emotion & Pace: Gemini 3.1 Flash TTS, Angular & Firebase Cloud Functions [GDE]

Connie Leung — Sat, 02 May 2026 14:11:45 +0000

Prerequisites

The technical stack of the project:

Angular 21: The latest version as of May 2026.
Node.js LTS: The LTS version as of May 2026.
Firebase Remote Config: To manage dynamic parameters.
Firebase Cloud Functions: To generate an expressive human voice when called by the frontend.
Firebase Local Emulator Suite: To test the functions locally at http://localhost:5001.
Gemini in Vertex AI: To generate videos and store them in Firebase Cloud Storage.

The public Google AI Studio API is restricted in my region (Hong Kong). However, Vertex AI (Google Cloud) offers enterprise access that works reliably here, so I chose Vertex AI for this demo.

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and log in again to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the prompts to set up Firebase Cloud Functions, the Firebase Local Emulator Suite, Firebase Cloud Storage, and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

In both cases, the Firebase CLI automatically installs the firebase-admin and firebase-functions dependencies.

Angular dependency

npm i firebase

The Angular application requires the firebase dependency to initialize a Firebase app, load remote config, and invoke the Firebase Cloud Functions to generate videos.

Firebase dependencies

npm i @cfworker/json-schema @google/genai @modelcontextprotocol/sdk

Install the above dependencies to access Gemini in Vertex AI. @google/genai depends on @cfworker/json-schema and @modelcontextprotocol/sdk. Without these, the Cloud Functions cannot start.

With our project configured, let's look at how the frontend and backend communicate.

Architecture

Limitations of Gemini 3.1 Flash TTS Preview Model

The model can only accept text inputs and generate audio outputs.
The context window is 32K tokens
TTS does not support streaming.
The supported languages can be found in https://ai.google.dev/gemini-api/docs/speech-generation#languages. My mother tongue, Cantonese, is currently unsupported.

Firebase Integration

1. Configure Environment Variables

Defining the environment variables in the Firebase project ensures the functions know the region of the Google Cloud project, the Firebase Cloud Function location, and the required TTS model.

.env.example

GOOGLE_CLOUD_LOCATION="global"
GOOGLE_FUNCTION_LOCATION="asia-east2"
GEMINI_TTS_MODEL_NAME="gemini-3.1-flash-tts-preview"
WHITELIST="http://localhost:4200"
REFERER="http://localhost:4200/"

Variable	Description
GOOGLE_CLOUD_LOCATION	The region of the Google Cloud project. I chose `global` so that the Firebase project has access to the newest Gemini 3.1 Flash TTS preview model.
GOOGLE_FUNCTION_LOCATION	The region of the Firebase Cloud Functions. I chose `asia-east2` because this is the region where I live.
WHITELIST	Requests must come from http://localhost:4200.
REFERER	Requests originate from http://localhost:4200/.

http://localhost:4200 is the host and port number of my local Angular application.

2. Validating Environment Variables

import logger from "firebase-functions/logger";

export function validate(value: string | undefined, fieldName: string, missingKeys: string[]) {
    const err = `${fieldName} is missing.`;
    if (!value) {
        logger.error(err);
        missingKeys.push(fieldName);
        return "";
    }

    return value;
}

export const AUDIO_CONFIG = (() => {
    logger.info("AUDIO_CONFIG initialization: Loading environment variables and validating configuration...");

    const env = process.env;

    const missingKeys: string[] = [];
    const location = validate(env.GOOGLE_CLOUD_LOCATION, "Vertex Location", missingKeys);
    const model = validate(env.GEMINI_TTS_MODEL_NAME, "Gemini TTS Model Name", missingKeys);
    const project = validate(env.GCLOUD_PROJECT, "Google Cloud Project", missingKeys);

    if (missingKeys.length > 0) {
        throw new HttpsError("failed-precondition", `Missing environment variables: ${missingKeys.join(", ")}`);
    }

    return {
        genAIOptions: {
            project,
            location,
            vertexai: true,
        },
        model,
    };
})();

I am using Node 24 as of May 2026. Since Node 20, we can use the built-in process.loadEnvFile function that loads environment variables from the .env file.

In env.ts, the try-catch block attempts to load the environment variables from the .env file.

try {
    process.loadEnvFile();
} catch {
    // Ignore error if .env file is not found (e.g., in production where env vars are set by the platform)
}

In src/index.ts, the first statement imports the env.ts before importing other files and libraries.

import "./env";

... other import statements ...

If you are using a Node version that does not support process.loadEnvfile, the alternative is to install dotenv to load the environment variables.

npm i dotenv

import dotenv from "dotenv";

dotenv.config();

Firebase provides the GCLOUD_PROJECT variable, so it is not defined in the .env file.

3. Sanitize the Prompt Inputs

The Cloud Function sanitizes the scene and transcript before composing the audio prompt.

function sanitizeScene(text: string): string {
    return (text || "").trim().replace(/\r?\n/g, "\\n").replace(/^[#\s]+/gm, "");
}

The sanitizeTranscript function accepts the transcript by removing all Markdown headers and triple quotes that are injected into it.

function sanitizeTranscript(text: string): string {
    return (text || "").trim().replace(/^#+/gm, "").replace(/"""/g, '"');
}

4. Build an Audio Prompt

The AudioPrompt interface encapsulates the scene, emotion, pace, transcript, and voice option to set the location, audio tags, text, and persona of the audio.

export type AudioPrompt = {
  scene: string;
  emotion: string;
  pace: string;
  transcript: string;
  voiceOption: string;
}

The SCENE_DICTIONARY is an array of scenes. When the user does not provide a scene, a scene is randomly selected from the array.

export const SCENE_DICTIONARY = [
    "A dimly lit, dusty library filled with ancient leather-bound books.\n" +
        "The air is thick with history. A scholarly archivist is leaning closely into a warm, vintage ribbon microphone.\n" +
        "They speak with an infectious, hushed intensity, eager to share a forgotten secret they just uncovered in a decaying manuscript.",

    "It is 10:00 PM in a glass-walled studio overlooking the moonlit London skyline, but inside, it is blindingly bright.\n" +
        "The red 'ON AIR' tally light is blazing. The speaker is standing up, bouncing on the balls of their heels to the rhythm of a thumping backing track.\n" +
        "It is a chaotic, caffeine-fueled cockpit designed to wake up an entire nation.",

    "A meticulously sound-treated bedroom in a suburban home.\n" +
        "The space is deadened by plush velvet curtains and a heavy rug, creating an intimate, close-up acoustic environment.\n" +
        "The speaker delivers the information like a trusted friend sharing an inside joke.",

    "A high-tech, minimalist laboratory humming with servers.\n" +
        "Crisp, clean acoustics reflect off glass and steel.\n" +
        "A brilliant but eccentric scientist is pacing back and forth, speaking rapidly and enthusiastically into a headset microphone, excited to explain a complex phenomenon.",
];

The insertAudioTagsToTranscript uses a regular expression to split the transcript into lines, inserts the combined audio tag before each line, and then joins them with an empty string.

The buildAudioPrompt concatenates the scene and the expressive transcript into a string before returning it.

import { SCENE_DICTIONARY } from './constants/scenes.const';
import { AudioPrompt } from './types/audio-prompt.type';

function makeTag(value: string) {
    const trimmedValue = value.trim();
    return trimmedValue ? `[${trimmedValue}] ` : "";
}

function insertAudioTagsToTranscript({ transcript, pace, emotion }: AudioPrompt): string {
    const audioTags = `${makeTag(emotion)}${makeTag(pace)}`;
    const cleanedTranscript = sanitizeTranscript(transcript);

    const parts = cleanedTranscript.split(/(?<!\b(?:Mr|Mrs|Ms|Dr|St|i\.e|e\.g))([.!?\n\r]+[”"’']*\s*)/);
    return parts
        .map((text, i, arr) => {
            if (i % 2 !== 0) {
                return ""; // Skip delimiters, they are appended to the text blocks
            }
            const delimiter = arr[i + 1] || "";
            return text.trim() ? `${audioTags}${text.trim()}${delimiter}` : delimiter;
        })
        .join("");
}

export function buildAudioPrompt(data: AudioPrompt): string {
    const randomIndex = Math.floor(Math.random() * SCENE_DICTIONARY.length);
    const selectedScene = SCENE_DICTIONARY[randomIndex];

    const trimmedScene = (data.scene || "").trim() || selectedScene;
    const escapedScene = sanitizeScene(trimmedScene);
    const transcript = insertAudioTagsToTranscript(data);

    return `## Scene:
${escapedScene}

## Transcript:
"""
${transcript}
"""
`;
}

The output of the prompt looks like:

## Scene:
<scene>

## Transcript:
[<emotion>] [<pace>] <sentence 1>[<emotion>] [<pace>] <sentence 2>...[<emotion>] [<pace>] <sentence N>

5. Generating an Expression Human Audio in a Firebase Cloud Function

The createVoiceConfig function constructs an instance of GenerateContentConfig that outputs a speech narrated by the given voice name.

import { GenerateContentConfig } from "@google/genai";

export function createVoiceConfig(voiceName = "Kore"): GenerateContentConfig {
    return {
        responseModalities: ["audio"],
        speechConfig: {
            voiceConfig: {
                prebuiltVoiceConfig: {
                    voiceName,
                },
            },
        },
    };
}

const splitList = (whitelist?: string) => (whitelist || "").split(",").map((origin) => origin.trim());

export const whitelist = splitList(process.env.WHITELIST);
export const cors = whitelist.length > 0 ? whitelist : true;
export const refererList = splitList(process.env.REFERER);

The readFact cloud function delegates to readFactStreamFunction when isStreaming is true. Otherwise, it is delegated to readFactFunction.

The readFactFunction function returns a Promise<string> that is the encoded base64 string.

The readFactStreamFunction functions returns a Promise<number[] | undefined> that represents a buffer of WAV header bytes.

import { onCall } from "firebase-functions/v2/https";
import { cors } from "../auth";
import { buildAudioPrompt } from './audio-prompt';
import { readFactFunction, readFactFunctionStream } from "./read-fact";
import { createVoiceConfig } from './voice-config';

const options = {
    cors,
    enforceAppCheck: true,
    timeoutSeconds: 600,
};

export const readFact = onCall(options, (request, response) => {
    const { data, acceptsStreaming } = request;
    const isStreaming = acceptsStreaming && !!response;
    const prompt = buildAudioPrompt(data);
    const voiceOption = createVoiceConfig(data.voiceOption);

    return isStreaming
        ? readFactStreamFunction(prompt, voiceOption, response)
        : readFactFunction(prompt, voiceOption);
});

The withAIAudio function is a high-order function that calls the callback to generate an audio stream.

async function withAIAudio(callback: (ai: GoogleGenAI, model: string) => Promise<string | number[] | undefined>) {
    try {
        const variables = AUDIO_CONFIG;
        if (!variables) {
            return "";
        }

        const { genAIOptions, model } = variables;
        const ai = new GoogleGenAI(genAIOptions);
        return await callback(ai, model);
    } catch (e) {
        if (e instanceof HttpsError) {
            throw e;
        }
        throw new HttpsError("internal", "An internal error occurred while setting up the AI client.", {
            originalError: (e as Error).message,
        });
    }
}

The createAudioParams function constructs a parameter with the Gemini TTS model, the audio prompt, and the speech configuration.

async function generateAudio(aiTTS: AIAudio, prompt: string, voiceOption: GenerateContentConfig) {
    try {
        const { ai, model } = aiTTS;
        const response = await ai.models.generateContent(createAudioParams(model, prompt, voiceOption));
        return getBase64DataUrl(response);
    } catch (error) {
        console.error(error);
        throw error;
    }
}

function createAudioParams(model: string, prompt: string, config?: GenerateContentConfig) {
    return {
        model,
        contents: [
            {
                role: "user",
                parts: [
                    {
                        text: prompt,
                    },
                ],
            },
        ],
        config,
    };
}

function extractInlineAudioData(response: GenerateContentResponse): {
    rawData: string | undefined;
    mimeType: string | undefined;
} {
    const { data: rawData, mimeType } = response.candidates?.[0]?.content?.parts?.[0]?.inlineData ?? {};

    return { rawData, mimeType };
}

function getBase64DataUrl(response: GenerateContentResponse) {
    const { rawData, mimeType } = extractInlineAudioData(response);

    if (!rawData || !mimeType) {
        throw new Error("Audio generation failed: No audio data received.");
    }

    return encodeBase64String({ rawData, mimeType });
}

export function encodeBase64String({ rawData, mimeType }: RawAudioData) {
    const wavBuffer = convertToWav(rawData, mimeType);
    const base64Data = wavBuffer.toString("base64");
    return `data:audio/wav;base64,${base64Data}`;
}

After all the chunks are sent to the client, the createWavHeader function uses the total byte length and the audio options to construct a WAV header and returns it.

async function generateAudioStream(
    aiTTS: AIAudio,
    prompt: string,
    voiceOption: GenerateContentConfig,
    response: CallableResponse<unknown>,
): Promise<number[] | undefined> {
    try {
        const { ai, model } = aiTTS;
        const chunks = await ai.models.generateContentStream(createAudioParams(model, prompt, voiceOption));
        let byteLength = 0;
        let options: WavConversionOptions | undefined = undefined;
        for await (const chunk of chunks) {
            const { rawData, mimeType } = extractInlineAudioData(chunk);
            if (!options && mimeType) {
                options = parseMimeType(mimeType);
                response.sendChunk({
                    type: "metadata",
                    payload: {
                        sampleRate: options.sampleRate,
                    },
                });
            }

            if (rawData && mimeType) {
                const buffer = Buffer.from(rawData, "base64");
                byteLength = byteLength + buffer.length;
                response.sendChunk({
                    type: "data",
                    payload: {
                        buffer,
                    },
                });
            }
        }

        if (options && byteLength > 0) {
            const header = createWavHeader(byteLength, options);
            return [...header];
        }

        return undefined;
    } catch (error) {
        console.error(error);
        throw error;
    }
}

The readFactFunction invokes the withAIAudio high-order function to generate a base64-encoded string.

export async function readFactFunction(prompt: string, voiceOption: GenerateContentConfig) {
    return withAIAudio((ai, model) => generateAudio({ ai, model }, prompt, voiceOption));
}

export async function readFactStreamFunction(prompt: string, voiceOption: GenerateContentConfig, response: CallableResponse<unknown>) {
    return withAIAudio((ai, model) => generateAudioStream({ ai, model }, prompt, voiceOption, response));
}

6. Firebase App Configuration and reCAPTCHA Site Key

I implemented a FIREBASE_APP_CONFIG IIFE (Immediately Invoked Function Expression) to run once to validate the environment variables of the Firebase app.

export const FIREBASE_APP_CONFIG = (() => {
    const env = process.env;
    const missingKeys: string[] = [];
    const apiKey = validate(env.APP_API_KEY, "API Key", missingKeys);
    const appId = validate(env.APP_ID, "App Id", missingKeys);
    const messagingSenderId = validate(env.APP_MESSAGING_SENDER_ID, "Messaging Sender ID", missingKeys);
    const recaptchaSiteKey = validate(env.RECAPTCHA_ENTERPRISE_SITE_KEY, "Recaptcha site key", missingKeys);
    const projectId = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);

    if (missingKeys.length > 0) {
        throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
    }

    return {
        app: {
            apiKey,
            appId,
            projectId,
            messagingSenderId,
            authDomain: `${projectId}.firebaseapp.com`,
            storageBucket: `${projectId}.firebasestorage.app`,
        },
        recaptchaSiteKey,
    };
})();

The getFirebaseConfig function caches the FIREBASE_APP_CONFIG for an hour before returning it to the Angular application.

export const getFirebaseConfig = onRequest({ cors }, (request, response) => {
    if (!validateRequest(request, response)) {
        return;
    }

    try {
        response.set("Cache-Control", "public, max-age=3600, s-maxage=3600");
        response.json(FIREBASE_APP_CONFIG);
    } catch (err) {
        console.error(err);
        response.status(500).send("Internal Server Error");
    }
});

7. Local Development with Emulators

The port number defaulted to 5001 when firebase init was executed.

function connectEmulators(functions: Functions, remoteConfig: RemoteConfig) {
  if (location.hostname === 'localhost') {
    const host = getValue(remoteConfig, 'functionEmulatorHost').asString();
    const port = getValue(remoteConfig, 'functionEmulatorPort').asNumber();
    connectFunctionsEmulator(functions, host, port);
  }
}

loadFirebaseConfig is a helper function that makes request to the Cloud function to obtain the Firebase App configuration and the reCAPTCHA site key.

{
  "getFirebaseConfigUrl": "http://127.0.0.1:5001/vertexai-firebase-6a64f/us-central1/getFirebaseConfig"
}

export type FirebaseConfigResponse = {
  app: FirebaseOptions;
  recaptchaSiteKey: string
}

import { HttpClient } from '@angular/common/http';
import { inject } from '@angular/core';
import { catchError, lastValueFrom, throwError } from 'rxjs';
import config from '../../public/config.json';
import { FirebaseConfigResponse } from './ai/types/firebase-config.type';

async function loadFirebaseConfig() {
  const httpService = inject(HttpClient);
  const firebaseConfig$ =
    httpService.get<FirebaseConfigResponse>(config.getFirebaseConfigUrl)
      .pipe(catchError((e) => throwError(() => e)));
  return lastValueFrom(firebaseConfig$);
}

The bootstrapFirebase function initializes the FirebaseApp and App Check, loads the Firebase remote configuration and cloud functions, and stores them in the config service for later use.

export async function bootstrapFirebase() {
    try {
      const configService = inject(ConfigService);
      const firebaseConfig = await loadFirebaseConfig();
      const { app, recaptchaSiteKey } = firebaseConfig;
      const firebaseApp = initializeApp(app);
      const remoteConfig = await fetchRemoteConfig(firebaseApp);

      initializeAppCheck(firebaseApp, {
        provider: new ReCaptchaEnterpriseProvider(recaptchaSiteKey),
        isTokenAutoRefreshEnabled: true,
      });

      const functionRegion = getValue(remoteConfig, 'functionRegion').asString();
      const functions = getFunctions(firebaseApp, functionRegion);
      connectEmulators(functions, remoteConfig);

      configService.loadConfig(firebaseApp, remoteConfig, functions);
    } catch (err) {
      console.error(err);
    }
}

The AppConfig remains unchanged.

import { ApplicationConfig, provideAppInitializer } from '@angular/core';
import { bootstrapFirebase } from './app.bootstrap';

export const appConfig: ApplicationConfig = {
  providers: [
    provideAppInitializer(async () => bootstrapFirebase()),
  ]
};

8. Angular Implementation

8.1 Audio Tags Component

I create an AudioTagsComponent and a new signal form to input the scene, emotion, pace, and voice name in the Angular frontend.

<div>
  <h3>
    <span class="text-xl">🎙️</span> Customize Audio Generation
  </h3>

  <div class="grid grid-cols-1 md:grid-cols-2 gap-4">
    <!-- Scene -->
    <div class="flex flex-col gap-1.5 md:col-span-2">
      <label for="scene">Scene Description</label>
      <textarea id="scene" [formField]="audioPromptForm.scene"
      ></textarea>
    </div>

    <!-- Emotion -->
    <div class="flex flex-col gap-1.5">
      <label for="emotion">Vocal Emotion</label>
      <input type="text" id="emotion" [formField]="audioPromptForm.emotion"
        placeholder="e.g., panicked, whispers"
      />
    </div>

    <!-- Pace -->
    <div class="flex flex-col gap-1.5">
      <label for="pace">Speaking Pace</label>
      <input type="text" id="pace" [formField]="audioPromptForm.pace"
        placeholder="e.g., very slow, rapid"
      />
    </div>

    <!-- Voice Option -->
    <div class="flex flex-col gap-1.5 md:col-span-2">
      <label for="voiceOption">AI Voice Model</label>
      <select id="voiceOption" [formField]="audioPromptForm.voiceOption"
      >
        <option value="" disabled selected>Select a voice...</option>
        @for (option of sortedVoiceOptions(); track option.name) {
          <option [value]="option.name" class="bg-slate-800">{{ option.label }}</option>
        }
      </select>
    </div>
  </div>
</div>

import { ChangeDetectionStrategy, Component, computed, signal } from '@angular/core';
import { form, FormField } from '@angular/forms/signals';
import { VOICE_OPTIONS } from './constants/voice-options.const';
import { AudioPromptData } from './types/audio-prompt-data.type';

@Component({
  selector: 'app-audio-tags',
  imports: [FormField],
  templateUrl: './audio-tags.component.html',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class AudioTagsComponent {
    #audioPromptModel = signal<AudioPromptData>({
      scene: 'A news anchor reading the news in a busy newsroom',
      emotion: 'professional, slightly serious',
      pace: 'moderate, clear enunciation',
      voiceOption: 'Kore'
    });
    audioPromptForm = form(this.#audioPromptModel);

    sortedVoiceOptions = computed(() => {
      const sortedList = VOICE_OPTIONS.sort((a, b) => a.name.localeCompare(b.name));

      return sortedList.map(option => ({
        name: option.name,
        label: `${option.name} - ${option.description}`
      }));
    });

    audioPromptModel = this.#audioPromptModel.asReadonly();
}

The AudioTagsComponent is imported into ObscureFactComponent such that users can input values into the experimental signal form.

<div class="w-full mt-6">
    <app-audio-tags #audioTags />

    <h3>A surprising or obscure fact about the tags</h3>
    @if (interestingFact()) {
      <p>{{ interestingFact() }}</p>

      <app-error-display [error]="ttsError()" />

      <app-text-to-speech
        [isLoadingSync]="isLoadingSync()"
        [isLoadingStream]="isLoadingStream()"
        [isLoadingWebAudio]="isLoadingWebAudio()"
        [audioUrl]="audioUrl()"
        (generateSpeech)="generateSpeech({ mode: $event, audioTags: audioTags.audioPromptModel() })"
        [playbackRate]="playbackRate()"
      />
    } @else {
      <p>The tag(s) does not have any interesting or obscure fact.</p>
    }
</div>

import { AudioPromptData } from './audio-prompt-data.type';
import { GenerateSpeechMode } from '../../generate-audio.util';

export type ModeWithAudioTags = {
  mode: GenerateSpeechMode;
  audioTags: AudioPromptData;
};

export type AudioPrompt = {
  scene: string;
  emotion: string;
  pace: string;
  transcript: string;
  voiceOption: string;
};

import { SpeechService } from '@/ai/services/speech.service';
import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { ChangeDetectionStrategy, Component, inject, input, OnDestroy, signal } from '@angular/core';
import { revokeBlobURL } from '../blob.util';
import { AudioTagsComponent } from './audio-tags/audio-tags.component';
import { ModeWithAudioTags } from './audio-tags/types/mode-audio-tags.type';
import { generateSpeechHelper, streamSpeechWithWebAudio, ttsError } from './generate-audio.util';
import { AudioPlayerService } from './services/audio-player.service';

@Component({
  selector: 'app-obscure-fact',
  templateUrl: './obscure-fact.component.html',
  imports: [
    TextToSpeechComponent,
  ],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class ObscureFactComponent implements OnDestroy {
  interestingFact = input<string | undefined>(undefined);

  speechService = inject(SpeechService);
  audioPlayerService = inject(AudioPlayerService);

  isLoadingSync = signal(false);
  isLoadingStream = signal(false);
  isLoadingWebAudio = signal(false);

  audioUrl = signal<string | undefined>(undefined);

  ttsError = ttsError;

  async generateSpeech({ mode, audioTags }: ModeWithAudioTags) {
    const fact = this.interestingFact();

    if (fact) {
      revokeBlobURL(this.audioUrl);
      this.audioUrl.set(undefined);

      const audioPrompt = {
          ...audioTags,
          transcript: fact,
      };
      if (mode === 'sync' || mode === 'stream') {
        const loadingSignal = mode === 'stream' ? this.isLoadingStream : this.isLoadingSync;
        const speechFn = (audioPrompt: AudioPrompt) => mode === 'stream' ?
            this.speechService.generateAudioBlobURL(audioPrompt) :
            this.speechService.generateAudio(audioPrompt);

        await generateSpeechHelper(audioPrompt, loadingSignal, this.audioUrl, speechFn);
      } else if (mode === 'web_audio_api') {
        await streamSpeechWithWebAudio(
          audioPrompt,
          this.isLoadingWebAudio,
          (audioPrompt: AudioPrompt) => this.audioPlayerService.playStream(audioPrompt));
      }
    }
  }

  ngOnDestroy(): void {
    revokeBlobURL(this.audioUrl);
  }
}

8.2 Call Firebase Cloud Functions directly

The SpeechService has a generateAudio method that calls the readFact cloud function to obtain the encoded base64 string.

export function constructBlobURL(parts: BlobPart[]) {
  return URL.createObjectURL(new Blob(parts, { type: 'audio/wav' }));
}

import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { constructBlobURL } from '@/photo-panel/blob.util';
import { inject, Injectable } from '@angular/core';
import { Functions, httpsCallable } from 'firebase/functions';
import { StreamMessage } from '../types/stream-message.type';
import { ConfigService } from './config.service';

@Injectable({
  providedIn: 'root'
})
export class SpeechService  {
    private configService = inject(ConfigService);

    private get functions(): Functions {
      if (!this.configService.functions) {
        throw new Error('Firebase Functions has not been initialized.');
      }
      return this.configService.functions;
    }

    async generateAudio(audioPrompt: AudioPrompt) {
      const readFactFunction = httpsCallable<AudioPrompt, string>(
        this.functions, 'textToAudio-readFact'
      );

      const { data: audioUri } = await readFactFunction(audioPrompt);
      return audioUri;
    }

    async generateAudioStream(audioPrompt: AudioPrompt) {
      const readFactStreamFunction = httpsCallable<AudioPrompt, number[] | undefined, StreamMessage>(
        this.functions, 'textToAudio-readFact'
      );

      return readFactStreamFunction.stream(audioPrompt);
    }

    async generateAudioBlobURL(audioPrompt: AudioPrompt) {
      const { stream, data } = await this.generateAudioStream(audioPrompt);

      const audioParts: BlobPart[] = [];
      for await (const audioChunk of stream) {
        if (audioChunk && audioChunk.type === 'data') {
          audioParts.push(new Uint8Array(audioChunk.payload.buffer.data));
        }
      }

      const wavHeader = await data;
      if (wavHeader && wavHeader.length) {
        audioParts.unshift(new Uint8Array(wavHeader));
      }

      return constructBlobURL(audioParts);
    }
}

Similar to SpeechService.generateAudioBlobURL, the playStream method of AudioPlayerService also calls generateAudioStream to get a stream of chunks and play each of them immediately.

import { SpeechService } from '@/ai/services/speech.service';
import { AudioPrompt } from '@/ai/types/audio-prompt.type';
import { inject, Injectable, OnDestroy, signal } from '@angular/core';

@Injectable({
  providedIn: 'root'
})
export class AudioPlayerService implements OnDestroy  {

  async playStream(audioPrompt: AudioPrompt) {
      const { stream } = await this.speechService.generateAudioStream(audioPrompt);
      for await (const audioChunk of stream) {
        ... process each chunk ...
      }
  }


  ngOnDestroy(): void {
    ... release resources to prevent memory leak ...
  }
}

9. Caveats and Lessons Learned: Avoiding the Dynamic Prompt Trap

The examples in Gemini AI Studio and Vertex AI Studio use static audio tags and transcripts and they work correctly for me.

When I applied dynamic audio tags and transcripts in the demo, the Gemini 3.1 TTS Flash Preview model ignored the audio tags. The issue was resolved after debugging in Gemini CLI for hours.

Here are the Caveats and Lessons Learned:

The Token Boundary Trap. The code originally concatenated tags and transcript without a space (for example, "[giggle][slow]Before"). The LLM tokenizer failed to recognize the instruction to change the behavior and pace of the audio. My fix was to insert a space between the tags and the transcript, which was "[giggle] [slow] Before".
Sanitize inputs before injecting into the prompt template. The sanitize functions remove Markdown headers (#) and triple quotes from the scene and transcript. The cleansed scene and transcript are injected into the prompt template to construct the final audio prompt.
LLM does not understand idiom. I typed "at a snail's pace" in the signal form and inserted "[at a snail's pace]" before the line. However, the model vocalized the tag literally, and no pace change occurred.
"Repetitive Weighting" is a Real Strategy. If standard tags like [slow] and [fast] are not dramatic enough, prepend the pace with "very" to increase the dramatic effect of the pace. It was evident when [very, very, very slow] generated a longer audio than [slow].
Replace newline character (\n) with \\n. to flatten the lines into a single paragraph. When the scene and transcript are cleansed and escaped, they are injected into the prompt template while the structure is preserved for the LLM parser.

Conclusion

The integration of text-to-speech with Firebase's serverless scalability empowers Angular applications for real-time audio generation.

Try cloning the GitHub repository, uploading an image to generate an obscure fact, and using the Gemini 3.1 Flash TTS preview model to speak it with the specified scene, emotion, and pace.

Resources

Extending a Video with Angular, Veo 3.1 Lite, Firebase Cloud Functions, and Firebase Cloud Storage

Connie Leung — Fri, 01 May 2026 03:17:34 +0000

Google released the Veo 3.1 Lite model for AI video generation in the Gemini API, Gemini in Vertex AI, and Gemini AI Studio. This model solves a common developer pain point: generating high-quality videos quickly and at a lower cost.

In this blog post, I migrate my application to use the Veo 3.1 Lite model and implement a new Firebase Cloud Function to extend a video using the GenAI TypeScript SDK.

The application supports image-to-video generation, video interpolation using the first and last frames, and extending Veo videos.

Prerequisites

The technical stack of the project:

Angular 21, the latest version as of April 2026.
Node LTS, the LTS version as of April 2026.
Firebase Remote Config: To manage dynamic parameters.
Firebase Cloud Functions: To be called by the frontend to generate a video, interpolate a video between two images, or extend a Veo video.
Firebase Cloud Storage: To host the generated video files in the default Firebase Storage bucket.
Firebase Local Emulator Suite: To test the functions locally at http://localhost:5001.
Gemini in Vertex AI: Use Gemini in Vertex AI to generate videos and store them in Firebase Cloud Storage.

The public Google AI Studio API is restricted in my region (Hong Kong). However, Vertex AI (Google Cloud) offers enterprise access that works reliably here, so I chose Vertex AI for this demo.

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and log in again to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the prompts to set up Firebase Cloud Functions, the Firebase Local Emulator Suite, Firebase Cloud Storage, and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

In both cases, the Firebase CLI automatically installs the firebase-admin and firebase-functions dependencies.

Angular dependency

npm i firebase

The Angular application requires the firebase dependency to initialize a Firebase app, load remote config, and invoke the Firebase Cloud Functions to generate videos.

Firebase dependencies

npm i @cfworker/json-schema @google/genai @modelcontextprotocol/sdk

Install the above dependencies to access Gemini in Vertex AI. @google/genai depends on @cfworker/json-schema and @modelcontextprotocol/sdk. Without these, the Cloud Functions cannot start.

With our project configured, let's look at how the frontend and backend communicate.

Architecture

The frontend application is built with Angular. It relies on Firebase AI Logic to generate images using the Gemini 3.1 Flash Image Preview model. Then, the text prompt and the image are submitted to a Firebase Cloud Function to generate a video, store it in a Firebase Cloud Storage bucket, and return the GCS URI, MIME type, and HTTP URL to the client.

When the client extends the Veo video, it provides the prompt, the GCS URI, and the MIME type to another Firebase Cloud Function to generate an extended video, store it in the Firebase Cloud Storage bucket, and return the GCS URI, MIME type, and HTTP URL to the client.

Similarly, the HTML video player element plays the HTTP URL in the client application.

Difference between Veo 3.1 Lite in Gemini AI Studio and Gemini in Vertex AI

Difference	Gemini AI Studio	Gemini in Vertex AI
Model Name	veo-3.1-lite-generate-preview	veo-3.1-lite-generate-001
Number of extensions	Extend by 7 seconds and up to 20 times	Extend by 7 seconds and up to 4 times
Video length	Extend 141 seconds and the total length is 148 seconds	Extend 28 seconds and the total length is 36 seconds

What Veo 3.1 Does Not Support in Gemini Vertex AI

Unlike Veo 3.1 and Veo 3.1 Fast (which only lack support for Reference style images), Veo 3.1 Lite does not support either Reference asset images or Reference style images.
Supported input resolution does not support 4K.
Supported output resolution does not support 4K.

This limitation does not impact our application, as the demo focuses exclusively on video extension and the resolution of generated videos is hardcoded to 720p.

Firebase Integration

1. Configure Environment Variables

I define the environment variables in the Firebase project. This ensures the functions know the regions for storage, function hosting, and the Veo model to use for video generation.

.env.example

GOOGLE_CLOUD_LOCATION="us-central1"
GEMINI_VIDEO_MODEL_NAME="veo-3.1-lite-generate-001"
IS_VEO31_USED="true"
POLLING_PERIOD_MS="10000"
GOOGLE_FUNCTION_LOCATION="us-central1"
WHITELIST="http://localhost:4200"
REFERER="http://localhost:4200/"

Variable	Description
GOOGLE_CLOUD_LOCATION	The location of the bucket. I chose `us-central1` because the bucket is always free in this region.
GEMINI_VIDEO_MODEL_NAME	The name of the Gemini video model.
IS_VEO31_USED	Whether Veo 3.1 is used. If false, it falls back to generating a video instead of interpolation.
POLLING_PERIOD_MS	The polling period of the video operation in milliseconds.
GOOGLE_FUNCTION_LOCATION	The region of the Cloud Functions. I chose `us-central1` so the functions and the bucket are in the same region.
WHITELIST	Requests must come from http://localhost:4200.
REFERER	Requests originate from http://localhost:4200/.

2. Validating Environment Variables

Before the Cloud Function proceeds with any AI calls, it is critical to ensure that all necessary environment variables are present. I implemented a VIDEO_CONFIG IIFE (Immediately Invoked Function Expression) to run once to validate environment variables such as the polling period, whether Veo 3.1 is used, the Veo model name, the project ID, and the location.

import logger from "firebase-functions/logger";

export function validate(value: string | undefined, fieldName: string, missingKeys: string[]) {
  const err = `${fieldName} is missing.`;
  if (!value) {
    logger.error(err);
    missingKeys.push(fieldName);
    return "";
  }

  return value;
}

export const VIDEO_CONFIG = (() => {
  process.loadEnvFile();

  const env = process.env;
  const isVeo31Used = (env.IS_VEO31_USED || "false") === "true";
  const pollingPeriod = Number(env.POLLING_PERIOD_MS || "10000");

  const missingKeys: string[] = [];
  const model = validate(env.GEMINI_VIDEO_MODEL_NAME, "Gemini Video Model Name", missingKeys);
  const project = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);
  const location = validate(env.GOOGLE_CLOUD_LOCATION, "Google Cloud Location", missingKeys);

  if (missingKeys.length > 0) {
    throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
  }

  return {
    genAIOptions: {
      project,
      location,
      vertexai: true,
    },
    aiVideoOptions: {
      model,
      storageBucket: `${project}.firebasestorage.app`,
      isVeo31Used,
      pollingPeriod,
    },
  };
})();

I am using Node 24 as of April 2026. Since Node 20, we can use the built-in process.loadEnvFile function that loads environment variables from the .env file.

In env.ts, the try-catch block attempts to load the environment variables from the .env file.

try {
    process.loadEnvFile();
} catch {
    // Ignore error if .env file is not found (e.g., in production where env vars are set by the platform)
}

In src/index.ts, the first statement imports the env.ts before importing other files and libraries.

import "./env";

... other import statements ...

If you are using a Node version that does not support process.loadEnvfile, the alternative is to install dotenv to load the environment variables.

npm i dotenv

import dotenv from "dotenv";

dotenv.config();

Firebase provides the GCLOUD_PROJECT variable, so it is not defined in the .env file.

When the missingKeys array is not empty, VIDEO_CONFIG throws an error that lists all the missing variable names. If the validation is successful, the genAIOptions and aiVideoOptions are returned. The genAIOptions is used to initialize the GoogleGenAI and aiVideoOptions contains parameters for extending a Veo video.

3. Extending a Video and Storing in Firebase Storage

The extendVideo Cloud Function passes the payload to the extendVideoFunction function.

All Cloud Functions enforce App Check, CORS, and a timeout period of 600 seconds. If WHITELIST is unspecified, CORS defaults to true. It is acceptable in a demo, but it is safer to default to false or a specific domain in production.

const cors = process.env.WHITELIST ? process.env.WHITELIST.split(",").map((origin) => origin.trim()): true;
const options = {
  cors,
  enforceAppCheck: true,
  timeoutSeconds: 600,
};

export const extendVideo = onCall( options,
  ({ data }) => extendVideoFunction(data)
);

extendVideoFunction delegates to the extendVideoByPolling function to construct the video arguments and poll the video operation until it finishes. When the function completes successfully, it returns the GCS URI and the MIME type. In the error case, the function throws an error.

import { GoogleGenAI } from "@google/genai";
import { ExtendVideoRequest } from "./types/video.type";
import { extendVideoByPolling, VIDEO_CONFIG } from "./video.util";

export async function extendVideoFunction(data: ExtendVideoRequest) {
  const { genAIOptions, aiVideoOptions } = VIDEO_CONFIG;

  try {
    if (!aiVideoOptions.isVeo31Used) {
      throw new Error("Video extension is only supported for Veo 3.1 model");
    }

    const ai = new GoogleGenAI(genAIOptions);
    return await extendVideoByPolling({ ai, ...aiVideoOptions }, {
      prompt: data.prompt,
      video: data.video,
      config: data.config,
    });
  } catch (error) {
    console.error("Error generating video:", error);
    throw new Error("Error generating video");
  }
}

import { GenerateVideosConfig, GoogleGenAI, Video } from "@google/genai";

export type AIVideoBucket = {
  ai: GoogleGenAI;
  model: string;
  storageBucket: string;
  isVeo31Used: boolean;
  pollingPeriod: number;
}

export type ExtendVideoRequest = {
  prompt: string;
  video: Video;
  config?: GenerateVideosConfig;
}

export async function extendVideoByPolling(
  aiVideo: AIVideoBucket,
  request: ExtendVideoRequest,
) {
  return processVideoPolling(aiVideo, {
    prompt: request.prompt,
    config: request.config,
    video: request.video,
  });
}

async function processVideoPolling(
  { ai, model, storageBucket, pollingPeriod }: AIVideoBucket,
  mediaParams: VideoMediaParams
) {
  const genVideosParams: GenerateVideosParameters = {
    model,
    ...mediaParams,
    config: {
      ...mediaParams.config,
      numberOfVideos: 1,
      outputGcsUri: `gs://${storageBucket}`,
    },
  };

  return getVideoUri(ai, genVideosParams, pollingPeriod);
}

The extendVideoByPolling function invokes the processVideoPolling function to poll the video operation until it completes.

The processVideoPolling function constructs the video parameters and invokes the getVideoUri function to return the GCS URI and the MIME type. Most importantly, the config property specifies that the number of generated videos is one, and the outputGcsUri keeps the generated video in the storage bucket.

4. Asynchronous Polling

Video extension is a long-running task. Because Vertex AI processes them asynchronously, the functions must poll the operation status in a while loop until the done flag is true.

The Gemini API cannot see the Cloud Storage for Firebase Local Emulator, so it requires a real output GCS URI, which is gs://${storageBucket}.

When the done flag is true, the operation ends and one of three outcomes occurs:

Outcome 1: The error is true, and the video failed to generate. Therefore, the function throws an error.
Outcome 2: The video is successfully stored in the GCS bucket. The function returns the GCS URI and the MIME type to the client application.
Outcome 3: Neither happens. There is no error and no GCS URI, so the function returns an unknown error.

import { GenerateVideosParameters, GoogleGenAI } from "@google/genai";

async function getVideoUri(
  ai: GoogleGenAI,
  genVideosParams: GenerateVideosParameters,
  pollingPeriod: number,
): Promise<{ uri: string, mimeType: string }> {
  let operation = await ai.models.generateVideos(genVideosParams);

  while (!operation.done) {
    await new Promise((resolve) => setTimeout(resolve, pollingPeriod));
    operation = await ai.operations.getVideosOperation({ operation });
  }

  if (operation.error) {
    const strError = `Video generation failed: ${operation.error.message}`;
    console.error(strError);
    throw new Error(strError);
  }

  const video = operation.response?.generatedVideos?.[0]?.video || {};
  const { uri, mimeType } = video;
  if (uri && mimeType) {
    console.log("video uri", uri, mimeType);
    return { uri, mimeType };
  }

  const strError = "Video generation finished but no uri was provided.";
  console.error(strError);
  throw new Error(strError);
}

The environment variable, POLLING_PERIOD_MS, sets the polling period to 10 seconds. If the wait time is too long, the polling period can be decreased. If the polling cost is expensive or too frequent, the polling period can be increased.

Note: For demo purposes, polling is a decent solution to handle asynchronous video generation. However, it is expensive and creates unnecessary load and latency. For production usage, you may consider push notifications such as WebSockets and Server-Sent Events.

5. Firebase App Configuration and reCAPTCHA Site Key

getFirebaseConfig is a Firebase Cloud Function that returns both the Firebase app configuration and the reCAPTCHA site key.

import logger from "firebase-functions/logger";
import { validate } from "./validate";

function validateFirebaseConfigFields(env: NodeJS.ProcessEnv) {
  const missingKeys: string[] = [];
  const apiKey = validate(env.APP_API_KEY, "API Key", missingKeys);
  const appId = validate(env.APP_ID, "App Id", missingKeys);
  const messagingSenderId = validate(env.APP_MESSAGING_SENDER_ID, "Messaging Sender ID", missingKeys);
  const recaptchaSiteKey = validate(env.RECAPTCHA_ENTERPRISE_SITE_KEY, "Recaptcha site key", missingKeys);
  const projectId = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);

  if (missingKeys.length > 0) {
    throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
  }

  return {
    app: {
      apiKey,
      appId,
      projectId,
      messagingSenderId,
      authDomain: `${projectId}.firebaseapp.com`,
      storageBucket: `${projectId}.firebasestorage.app`,
    },
    recaptchaSiteKey,
  };
}

export const getFirebaseConfigFunction = () => {
  logger.info("getFirebaseConfig called");

  process.loadEnvFile();

  const variables = validateFirebaseConfigFields(process.env);
  if (!variables) {
    return undefined;
  }

  return variables;
};

6. Local Development with Emulators

For local development, I used the Firebase Local Emulator Suite. In the bootstrapFirebase process, the app calls connectFunctionsEmulator to link to the Cloud Functions running at http://localhost:5001.

The port number defaulted to 5001 when firebase init was executed.

Note: While the Cloud Function runs locally (at zero cost), the Storage emulator is not used. This is because the Gemini API requires an actual accessible GCS bucket to store the generated videos.

export function connectEmulators({ remoteConfig, functions }: FirebaseObjects) {
  const useEmulators = getValue(remoteConfig, 'useEmulators').asBoolean();
  if (useEmulators) {
    console.log('Connecting to emulators...');
    const host = getValue(remoteConfig, 'functionEmulatorHost').asString();
    const port = getValue(remoteConfig, 'functionEmulatorPort').asNumber();
    console.log('functionEmulator', `${host}:${port}`);
    connectFunctionsEmulator(functions, host, port);
  }
}

loadFirebaseConfig is a helper function that makes request to the Cloud function to obtain the Firebase App configuration and the reCAPTCHA site key.

{
  "getFirebaseConfigUrl": "http://127.0.0.1:5001/vertexai-firebase-6a64f/us-central1/getFirebaseConfig"
}

import { HttpClient } from '@angular/common/http';
import { inject } from '@angular/core';
import { initializeAppCheck, ReCaptchaEnterpriseProvider } from 'firebase/app-check';
import { catchError, lastValueFrom, throwError } from 'rxjs';
import config from '../../public/config.json';
import { ConfigService } from './ai/services/config.service';
import { FirebaseConfigResponse } from './ai/types/firebase-config.type';
import { connectEmulators, initFirebaseApp } from './firebase.util';

async function loadFirebaseConfig() {
  const httpService = inject(HttpClient);
  const firebaseConfig$ =
    httpService.get<FirebaseConfigResponse>(`${config.getFirebaseConfigUrl}`)
      .pipe(catchError((e) => throwError(() => e)));
  return lastValueFrom(firebaseConfig$);
}

export async function bootstrapFirebase() {
    try {
      const configService = inject(ConfigService);
      const firebaseConfig = await loadFirebaseConfig();
      const { app, recaptchaSiteKey } = firebaseConfig;
      const firebaseObjects = await initFirebaseApp(app);
      const { firebaseApp } = firebaseObjects;

      initializeAppCheck(firebaseApp, {
        provider: new ReCaptchaEnterpriseProvider(recaptchaSiteKey),
        isTokenAutoRefreshEnabled: true,
      });

      connectEmulators(firebaseObjects);
      configService.loadConfig(firebaseObjects);
    } catch (err) {
      console.error(err);
    }
}

The AppConfig remains unchanged.

import { ApplicationConfig, provideAppInitializer } from '@angular/core';
import { bootstrapFirebase } from './app.bootstrap';

export const appConfig: ApplicationConfig = {
  providers: [
    provideAppInitializer(async () => bootstrapFirebase()),
  ]
};

7. Firebase Remote Configuration

New variables are introduced for extending a Veo video in Firebase Remote Config. For the Gemini API, a video can be extended as many as 20 times. For Gemini in Vertex AI, a video can be extended as many as 4 times because the video length exceeds 30 seconds. Therefore, this is externalized into the maxVideoExtendAllowed variable.

Variable	Description	Value
maxVideoExtendAllowed	Maximum number of extensions allowed	4

8. Video Player Component in Angular

The Angular frontend triggers the process using httpsCallable. Once the function returns the Cloud Storage path and the MIME type, the app fetches the download URL for playback.

The ConfigService stores the Firebase app, Remote Config, and functions to be used throughout the application.

import { FirebaseApp } from 'firebase/app';
import { Functions } from 'firebase/functions';
import { RemoteConfig } from 'firebase/remote-config';

export type FirebaseObjects = {
  firebaseApp: FirebaseApp;
  remoteConfig: RemoteConfig;
  functions: Functions;
}

import { Injectable } from '@angular/core';
import { FirebaseObjects } from '../types/firebase-objects';

@Injectable({
  providedIn: 'root'
})
export class ConfigService  {
    firebaseObjects: FirebaseObjects | undefined = undefined;

    loadConfig(firebaseObjects: FirebaseObjects) {
      this.firebaseObjects = firebaseObjects;
    }
}

The retrieveVideoUri method calls the Cloud Function directly to retrieve the GCS URI and the MIME type.

The downloadVideoUriAndUrl method resolves the URI to an HTTP URL so that an HTML video player can play it immediately.

export type VideoGenerationResponse = {
  uri: string;
  url: string;
  mimeType: string;
}

export type DownloadVideoResponse = {
  uri: string;
  mimeType: string;
}

import { inject, Injectable } from '@angular/core';
import { httpsCallable } from 'firebase/functions';
import { getDownloadURL, getStorage, ref } from 'firebase/storage';
import { GenerateVideoRequest } from '../types/video.type';
import { ConfigService } from './config.service';

@Injectable({
  providedIn: 'root'
})
export class VeoService {
  private readonly storage = getStorage();
  private readonly configService = inject(ConfigService);

  private async retrieveVideoUri<T = GenerateVideoRequest>(request: T, methodName: string): Promise<DownloadVideoResponse> {
    try {
      const { functions } = this.configService.firebaseObjects || {};
      if (!functions) {
        throw new Error('Functions does not exist.');
      }

      const downloadGcsUri = httpsCallable<T, DownloadVideoResponse>(
        functions, methodName, { timeout: 600000 }
      );
      const { data } = await downloadGcsUri(request);
      return data;
    } catch (err) {
        console.error(err);
        throw err;
    }
  }

  async downloadVideoUriAndUrl<T = GenerateVideoRequest>(request: T, methodName: CallableNames = 'videos-generateVideo'): Promise<VideoGenerationResponse> {
    const { uri, mimeType } = await this.retrieveVideoUri(request, methodName);

    if (!uri) {
      throw new Error('Video operation completed but no URI was returned.');
    }

    return getDownloadURL(ref(this.storage, uri))
      .then((url) => {
        console.log("download url", url);
        return { uri, url, mimeType };
      })
      .catch((error) => {
        switch (error.code) {
          case 'storage/object-not-found':
            throw new Error("File doesn't exist");
          case 'storage/unauthorized':
            throw new Error("User doesn't have permission to access the object");
          case 'storage/canceled':
            throw new Error("User canceled the upload");
          case 'storage/unknown':
            throw new Error("Unknown storage error occurred, inspect the server response");
        }
        throw new Error("Unknown error occurred");
      });
  }
}

The VideoPlayerComponent has a required videoUrl signal input that is assigned to the source of the video player. It has an "Extend Video" button that emits an extendVideo custom event when clicked. This event notifies the parent component to extend the video in the videoUrl signal input.

import { LoaderComponent } from '@/shared/loader/loader.component';
import { ChangeDetectionStrategy, Component, input, output } from '@angular/core';
import { ExtendVideoIconComponent } from '../../icons/extend-video-icon.component';

@Component({
  selector: 'app-video-player',
  imports: [LoaderComponent, ExtendVideoIconComponent],
  template: `
@if (isGeneratingVideo()) {
  <div class="mt-6">
    <app-loader [loadingText]="loadingText()">
      <p class="text-sm">This can take several minutes. Please be patient.</p>
    </app-loader>
  </div>
} @else if (videoUrl()) {
  <div class="mt-6 flex flex-col gap-4">
    <div class="video-container">
      <video [src]="videoUrl()" controls autoplay loop class="w-full rounded-md"></video>
    </div>

    <div class="flex justify-center">
      <button
        (click)="extendVideo.emit()"
        aria-label="Extend video"
        title="Extend video"
        class="extend-btn"
      >
        <app-extend-video-icon />
        <span>Extend Video</span>
      </button>
    </div>
  </div>
}
  `,
  styleUrl: './video-player.component.css',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class VideoPlayerComponent {
  isGeneratingVideo = input(false);
  videoUrl = input.required<string>();
  extendVideo = output<void>();

  loadingText = input('Generating your video...');
}

9. Angular Integration with Gemini to Extend Video

import { VideoPlayerComponent } from './video-player/video-player.component';

@Component({
  selector: 'app-gen-media',
  imports: [
    VideoPlayerComponent,
  ],
  template: `
  <app-video-player
    [isGeneratingVideo]="isGeneratingVideo()" 
    [videoUrl]="videoUrl()"
    (extendVideo)="extendVideo()"
    [loadingText]="loadingVideoText()"
  />`,
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class GenMediaComponent {
  private readonly genMediaService = inject(GenMediaService);
  private readonly genVideoService = inject(GenVideoService);

  genMediaInput = input<GenMediaInput>();
  videoUrl = this.genVideoService.videoUrl;
  isGeneratingVideo = this.genVideoService.isGeneratingVideo.asReadonly();
  loadingVideoText = signal('Generating your video...');

  trimmedUserPrompt = computed(() => this.genMediaInput()?.userPrompt.trim() || '');

  async extendVideo() {
    if (this.trimmedUserPrompt()) {
      try {
        this.loadingVideoText.set('Extending your video...');
        await this.genVideoService.extendVideo(this.trimmedUserPrompt());
      } finally {
        this.loadingVideoText.set('Generating your video...');
      }
    }
  }
}

The GenMediaComponent calls extendVideo with the prompt and the Veo video URL to create an extended video.

import { ConfigService } from '@/ai/services/config.service';
import { VeoService } from '@/ai/services/veo.service';
import { ExtendVideoRequest, GenerateVideoFromFramesRequest, GenerateVideoRequest, VideoGenerationResponse } from '@/ai/types/video.type';
import { computed, inject, Injectable, signal, WritableSignal } from '@angular/core';
import { getValue } from 'firebase/remote-config';

@Injectable({
  providedIn: 'root'
})
export class GenVideoService {
  private readonly veoService = inject(VeoService);
  private readonly configService = inject(ConfigService);

  videoError = signal('');
  videoResponse = signal<VideoGenerationResponse>({ uri: '', url: '', mimeType: '' });
  #extendVideoCounter = signal(0);
  isGeneratingVideo = signal(false);

  videoUrl = computed(() => this.videoResponse().url);

  isVideoExtensionAllowed(counter: number) {
    const remoteConfig = this.configService.firebaseObjects?.remoteConfig;
    if (!remoteConfig) {
      console.warn('Remote config does not exist.');
      return false;
    }

    const max_extend_allowed = getValue(remoteConfig,'maxVideoExtendAllowed').asNumber();
    if (counter >= max_extend_allowed) {
      console.warn('Maximum extension limit reached.');
      return false;
    }

    return true;
  }

  async extendVideo(prompt: string): Promise<void> {
    try {
      const result = await this.extendInterpolatedVideo(
        prompt,
        this.#extendVideoCounter(),
        this.videoResponse()
      );

      if (!result) {
        return;
      }

      this.videoResponse.set(result);
      this.#extendVideoCounter.update(count => count + 1);
      console.log(`Video extended successfully. Current extension count: ${this.#extendVideoCounter()}`);
    } catch (e) {
      console.error(e);
      const errMsg = e instanceof Error ?
        e.message :
        'An unexpected error occurred in video generation.'
      this.videoError.set(errMsg);
    } finally {
      this.isGeneratingVideo.set(false);
    }
  }

  async extendInterpolatedVideo(
    prompt: string,
    counter: number,
    customVideo: Pick<VideoGenerationResponse, 'uri' | 'mimeType'>,
    generatingSignal?: WritableSignal<boolean>,
    error?: WritableSignal<string>
  ) {
    const { uri, mimeType } = customVideo;

    if (!mimeType || !uri) {
      console.warn('No video to extend. Please generate a video first.');
      return null;
    }

    if (!prompt) {
      console.warn('Prompt is required to extend the video.');
      return null;
    }

    if (!this.isVideoExtensionAllowed(counter)) {
      return null;
    }

    const actualErrorSignal = error || this.videoError;
    const actualGeneratingSignal = generatingSignal || this.isGeneratingVideo;
    try {
      actualErrorSignal.set('');
      actualGeneratingSignal.set(true);

      const extendVideoParams: ExtendVideoRequest = {
        prompt,
        video: { uri, mimeType },
      }

      return await this.veoService.downloadVideoUriAndUrl(extendVideoParams, 'videos-extendVideo');
    } catch (e) {
      console.error(e);
      throw e;
    } finally {
      actualGeneratingSignal.set(false);
    }
  }
}

The extendVideo and extendInterpolatedVideo methods validate that an extension is allowed by comparing the #extendVideoCounter signal value against the maxVideoExtendAllowed Remote Config value. If the check fails, nothing happens. If the check succeeds, the videos-extendVideo Cloud Function is called to extend the video. Then, the #extendVideoCounter is incremented by 1 until the extension limit is reached. When extending a Veo video that is too long, Gemini throws an error, and I want to block this from happening on the client side.

This is the end of the walkthrough for the demo. You should now be able to extend generated videos in a Cloud Function, store them securely in a bucket, and play them in a video player within a user interface.

Conclusion

Combining Veo 3.1 with the serverless scalability of Firebase is a powerful workflow.

First, the Angular application neither installs the genai dependency nor maintains the Vertex AI environment variables in a .env file. The client application calls the Cloud Functions to perform intensive tasks and waits for the results.

The Cloud Functions receive arguments from the client, execute complex AI operations like generation, interpolation, and video extension, and write the videos to the dedicated bucket securely. During local development, the Firebase Emulator calls the functions at http://localhost:5001 instead of the ones deployed on the Cloud Run platform.

Try cloning the GitHub repository, generating images, and using those images to generate and interpolate videos. Then, you can extend these Veo videos to create extended videos that are more than 7 seconds long.

Resources

Extending a Video with Angular, Veo 3.1 Lite, Firebase Cloud Functions, and Firebase Cloud Storage [GDE]

Connie Leung — Sat, 18 Apr 2026 06:50:21 +0000

In this blog post, I migrate my application to use the Veo 3.1 Lite model and implement a new Firebase Cloud Function to extend a video using the GenAI TypeScript SDK.

The application supports image-to-video generation, video interpolation using the first and last frames, and extending Veo videos.

Prerequisites

The technical stack of the project:

Angular 21, the latest version as of April 2026.
Node LTS, the LTS version as of April 2026.
Firebase Remote Config: To manage dynamic parameters.
Firebase Cloud Functions: To be called by the frontend to generate a video, interpolate a video between two images, or extend a Veo video.
Firebase Cloud Storage: To host the generated video files in the default Firebase Storage bucket.
Firebase Local Emulator Suite: To test the functions locally at http://localhost:5001.
Gemini in Vertex AI: Use Gemini in Vertex AI to generate videos and store them in Firebase Cloud Storage.

The public Google AI Studio API is restricted in my region (Hong Kong). However, Vertex AI (Google Cloud) offers enterprise access that works reliably here, so I chose Vertex AI for this demo.

npm i -g firebase-tools

Install firebase-tools globally using npm.

firebase logout

firebase login

Log out of Firebase and log in again to perform proper Firebase authentication.

firebase init

Execute firebase init and follow the prompts to set up Firebase Cloud Functions, the Firebase Local Emulator Suite, Firebase Cloud Storage, and Firebase Remote Config.

If you have an existing project or multiple projects, you can specify the project ID on the command line.

firebase init --project <PROJECT_ID>

In both cases, the Firebase CLI automatically installs the firebase-admin and firebase-functions dependencies.

Angular dependency

npm i firebase

The Angular application requires the firebase dependency to initialize a Firebase app, load remote config, and invoke the Firebase Cloud Functions to generate videos.

Firebase dependencies

npm i @cfworker/json-schema @google/genai @modelcontextprotocol/sdk

Install the above dependencies to access Gemini in Vertex AI. @google/genai depends on @cfworker/json-schema and @modelcontextprotocol/sdk. Without these, the Cloud Functions cannot start.

With our project configured, let's look at how the frontend and backend communicate.

Architecture

Similarly, the HTML video player element plays the HTTP URL in the client application.

Difference between Veo 3.1 Lite in Gemini AI Studio and Gemini in Vertex AI

Difference	Gemini AI Studio	Gemini in Vertex AI
Model Name	veo-3.1-lite-generate-preview	veo-3.1-lite-generate-001
Number of extensions	Extend by 7 seconds and up to 20 times	Extend by 7 seconds and up to 4 times
Video length	Extend 141 seconds and the total length is 148 seconds	Extend 28 seconds and the total length is 36 seconds

What Veo 3.1 Does Not Support in Gemini Vertex AI

Unlike Veo 3.1 and Veo 3.1 Fast (which only lack support for Reference style images), Veo 3.1 Lite does not support either Reference asset images or Reference style images.
Supported input resolution does not support 4K.
Supported output resolution does not support 4K.

This limitation does not impact our application, as the demo focuses exclusively on video extension and the resolution of generated videos is hardcoded to 720p.

Firebase Integration

1. Configure Environment Variables

I define the environment variables in the Firebase project. This ensures the functions know the regions for storage, function hosting, and the Veo model to use for video generation.

.env.example

GOOGLE_CLOUD_LOCATION="us-central1"
GEMINI_VIDEO_MODEL_NAME="veo-3.1-lite-generate-001"
IS_VEO31_USED="true"
POLLING_PERIOD_MS="10000"
GOOGLE_FUNCTION_LOCATION="us-central1"
WHITELIST="http://localhost:4200"
REFERER="http://localhost:4200/"

Variable	Description
GOOGLE_CLOUD_LOCATION	The location of the bucket. I chose `us-central1` because the bucket is always free in this region.
GEMINI_VIDEO_MODEL_NAME	The name of the Gemini video model.
IS_VEO31_USED	Whether Veo 3.1 is used. If false, it falls back to generating a video instead of interpolation.
POLLING_PERIOD_MS	The polling period of the video operation in milliseconds.
GOOGLE_FUNCTION_LOCATION	The region of the Cloud Functions. I chose `us-central1` so the functions and the bucket are in the same region.
WHITELIST	Requests must come from http://localhost:4200.
REFERER	Requests originate from http://localhost:4200/.

2. Validating Environment Variables

import logger from "firebase-functions/logger";

export function validate(value: string | undefined, fieldName: string, missingKeys: string[]) {
  const err = `${fieldName} is missing.`;
  if (!value) {
    logger.error(err);
    missingKeys.push(fieldName);
    return "";
  }

  return value;
}

export const VIDEO_CONFIG = (() => {
  process.loadEnvFile();

  const env = process.env;
  const isVeo31Used = (env.IS_VEO31_USED || "false") === "true";
  const pollingPeriod = Number(env.POLLING_PERIOD_MS || "10000");

  const missingKeys: string[] = [];
  const model = validate(env.GEMINI_VIDEO_MODEL_NAME, "Gemini Video Model Name", missingKeys);
  const project = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);
  const location = validate(env.GOOGLE_CLOUD_LOCATION, "Google Cloud Location", missingKeys);

  if (missingKeys.length > 0) {
    throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
  }

  return {
    genAIOptions: {
      project,
      location,
      vertexai: true,
    },
    aiVideoOptions: {
      model,
      storageBucket: `${project}.firebasestorage.app`,
      isVeo31Used,
      pollingPeriod,
    },
  };
})();

I am using Node 24 as of April 2026. Since Node 20, we can use the built-in process.loadEnvFile function that loads environment variables from the .env file.

In env.ts, the try-catch block attempts to load the environment variables from the .env file.

try {
    process.loadEnvFile();
} catch {
    // Ignore error if .env file is not found (e.g., in production where env vars are set by the platform)
}

In src/index.ts, the first statement imports the env.ts before importing other files and libraries.

import "./env";

... other import statements ...

If you are using a Node version that does not support process.loadEnvfile, the alternative is to install dotenv to load the environment variables.

npm i dotenv

import dotenv from "dotenv";

dotenv.config();

Firebase provides the GCLOUD_PROJECT variable, so it is not defined in the .env file.

3. Extending a Video and Storing in Firebase Storage

The extendVideo Cloud Function passes the payload to the extendVideoFunction function.

const cors = process.env.WHITELIST ? process.env.WHITELIST.split(",").map((origin) => origin.trim()): true;
const options = {
  cors,
  enforceAppCheck: true,
  timeoutSeconds: 600,
};

export const extendVideo = onCall( options,
  ({ data }) => extendVideoFunction(data)
);

import { GoogleGenAI } from "@google/genai";
import { ExtendVideoRequest } from "./types/video.type";
import { extendVideoByPolling, VIDEO_CONFIG } from "./video.util";

export async function extendVideoFunction(data: ExtendVideoRequest) {
  const { genAIOptions, aiVideoOptions } = VIDEO_CONFIG;

  try {
    if (!aiVideoOptions.isVeo31Used) {
      throw new Error("Video extension is only supported for Veo 3.1 model");
    }

    const ai = new GoogleGenAI(genAIOptions);
    return await extendVideoByPolling({ ai, ...aiVideoOptions }, {
      prompt: data.prompt,
      video: data.video,
      config: data.config,
    });
  } catch (error) {
    console.error("Error generating video:", error);
    throw new Error("Error generating video");
  }
}

import { GenerateVideosConfig, GoogleGenAI, Video } from "@google/genai";

export type AIVideoBucket = {
  ai: GoogleGenAI;
  model: string;
  storageBucket: string;
  isVeo31Used: boolean;
  pollingPeriod: number;
}

export type ExtendVideoRequest = {
  prompt: string;
  video: Video;
  config?: GenerateVideosConfig;
}

export async function extendVideoByPolling(
  aiVideo: AIVideoBucket,
  request: ExtendVideoRequest,
) {
  return processVideoPolling(aiVideo, {
    prompt: request.prompt,
    config: request.config,
    video: request.video,
  });
}

async function processVideoPolling(
  { ai, model, storageBucket, pollingPeriod }: AIVideoBucket,
  mediaParams: VideoMediaParams
) {
  const genVideosParams: GenerateVideosParameters = {
    model,
    ...mediaParams,
    config: {
      ...mediaParams.config,
      numberOfVideos: 1,
      outputGcsUri: `gs://${storageBucket}`,
    },
  };

  return getVideoUri(ai, genVideosParams, pollingPeriod);
}

The extendVideoByPolling function invokes the processVideoPolling function to poll the video operation until it completes.

4. Asynchronous Polling

Video extension is a long-running task. Because Vertex AI processes them asynchronously, the functions must poll the operation status in a while loop until the done flag is true.

The Gemini API cannot see the Cloud Storage for Firebase Local Emulator, so it requires a real output GCS URI, which is gs://${storageBucket}.

When the done flag is true, the operation ends and one of three outcomes occurs:

Outcome 1: The error is true, and the video failed to generate. Therefore, the function throws an error.
Outcome 2: The video is successfully stored in the GCS bucket. The function returns the GCS URI and the MIME type to the client application.
Outcome 3: Neither happens. There is no error and no GCS URI, so the function returns an unknown error.

import { GenerateVideosParameters, GoogleGenAI } from "@google/genai";

async function getVideoUri(
  ai: GoogleGenAI,
  genVideosParams: GenerateVideosParameters,
  pollingPeriod: number,
): Promise<{ uri: string, mimeType: string }> {
  let operation = await ai.models.generateVideos(genVideosParams);

  while (!operation.done) {
    await new Promise((resolve) => setTimeout(resolve, pollingPeriod));
    operation = await ai.operations.getVideosOperation({ operation });
  }

  if (operation.error) {
    const strError = `Video generation failed: ${operation.error.message}`;
    console.error(strError);
    throw new Error(strError);
  }

  const video = operation.response?.generatedVideos?.[0]?.video || {};
  const { uri, mimeType } = video;
  if (uri && mimeType) {
    console.log("video uri", uri, mimeType);
    return { uri, mimeType };
  }

  const strError = "Video generation finished but no uri was provided.";
  console.error(strError);
  throw new Error(strError);
}

5. Firebase App Configuration and reCAPTCHA Site Key

getFirebaseConfig is a Firebase Cloud Function that returns both the Firebase app configuration and the reCAPTCHA site key.

import logger from "firebase-functions/logger";
import { validate } from "./validate";

function validateFirebaseConfigFields(env: NodeJS.ProcessEnv) {
  const missingKeys: string[] = [];
  const apiKey = validate(env.APP_API_KEY, "API Key", missingKeys);
  const appId = validate(env.APP_ID, "App Id", missingKeys);
  const messagingSenderId = validate(env.APP_MESSAGING_SENDER_ID, "Messaging Sender ID", missingKeys);
  const recaptchaSiteKey = validate(env.RECAPTCHA_ENTERPRISE_SITE_KEY, "Recaptcha site key", missingKeys);
  const projectId = validate(env.GCLOUD_PROJECT, "Project ID", missingKeys);

  if (missingKeys.length > 0) {
    throw new Error(`Missing environment variables: ${missingKeys.join(", ")}`);
  }

  return {
    app: {
      apiKey,
      appId,
      projectId,
      messagingSenderId,
      authDomain: `${projectId}.firebaseapp.com`,
      storageBucket: `${projectId}.firebasestorage.app`,
    },
    recaptchaSiteKey,
  };
}

export const getFirebaseConfigFunction = () => {
  logger.info("getFirebaseConfig called");

  process.loadEnvFile();

  const variables = validateFirebaseConfigFields(process.env);
  if (!variables) {
    return undefined;
  }

  return variables;
};

6. Local Development with Emulators

The port number defaulted to 5001 when firebase init was executed.

export function connectEmulators({ remoteConfig, functions }: FirebaseObjects) {
  const useEmulators = getValue(remoteConfig, 'useEmulators').asBoolean();
  if (useEmulators) {
    console.log('Connecting to emulators...');
    const host = getValue(remoteConfig, 'functionEmulatorHost').asString();
    const port = getValue(remoteConfig, 'functionEmulatorPort').asNumber();
    console.log('functionEmulator', `${host}:${port}`);
    connectFunctionsEmulator(functions, host, port);
  }
}

loadFirebaseConfig is a helper function that makes request to the Cloud function to obtain the Firebase App configuration and the reCAPTCHA site key.

{
  "getFirebaseConfigUrl": "http://127.0.0.1:5001/vertexai-firebase-6a64f/us-central1/getFirebaseConfig"
}

import { HttpClient } from '@angular/common/http';
import { inject } from '@angular/core';
import { initializeAppCheck, ReCaptchaEnterpriseProvider } from 'firebase/app-check';
import { catchError, lastValueFrom, throwError } from 'rxjs';
import config from '../../public/config.json';
import { ConfigService } from './ai/services/config.service';
import { FirebaseConfigResponse } from './ai/types/firebase-config.type';
import { connectEmulators, initFirebaseApp } from './firebase.util';

async function loadFirebaseConfig() {
  const httpService = inject(HttpClient);
  const firebaseConfig$ =
    httpService.get<FirebaseConfigResponse>(`${config.getFirebaseConfigUrl}`)
      .pipe(catchError((e) => throwError(() => e)));
  return lastValueFrom(firebaseConfig$);
}

export async function bootstrapFirebase() {
    try {
      const configService = inject(ConfigService);
      const firebaseConfig = await loadFirebaseConfig();
      const { app, recaptchaSiteKey } = firebaseConfig;
      const firebaseObjects = await initFirebaseApp(app);
      const { firebaseApp } = firebaseObjects;

      initializeAppCheck(firebaseApp, {
        provider: new ReCaptchaEnterpriseProvider(recaptchaSiteKey),
        isTokenAutoRefreshEnabled: true,
      });

      connectEmulators(firebaseObjects);
      configService.loadConfig(firebaseObjects);
    } catch (err) {
      console.error(err);
    }
}

The AppConfig remains unchanged.

import { ApplicationConfig, provideAppInitializer } from '@angular/core';
import { bootstrapFirebase } from './app.bootstrap';

export const appConfig: ApplicationConfig = {
  providers: [
    provideAppInitializer(async () => bootstrapFirebase()),
  ]
};

7. Firebase Remote Configuration

Variable	Description	Value
maxVideoExtendAllowed	Maximum number of extensions allowed	4

8. Video Player Component in Angular

The Angular frontend triggers the process using httpsCallable. Once the function returns the Cloud Storage path and the MIME type, the app fetches the download URL for playback.

The ConfigService stores the Firebase app, Remote Config, and functions to be used throughout the application.

import { FirebaseApp } from 'firebase/app';
import { Functions } from 'firebase/functions';
import { RemoteConfig } from 'firebase/remote-config';

export type FirebaseObjects = {
  firebaseApp: FirebaseApp;
  remoteConfig: RemoteConfig;
  functions: Functions;
}

import { Injectable } from '@angular/core';
import { FirebaseObjects } from '../types/firebase-objects';

@Injectable({
  providedIn: 'root'
})
export class ConfigService  {
    firebaseObjects: FirebaseObjects | undefined = undefined;

    loadConfig(firebaseObjects: FirebaseObjects) {
      this.firebaseObjects = firebaseObjects;
    }
}

The retrieveVideoUri method calls the Cloud Function directly to retrieve the GCS URI and the MIME type.

The downloadVideoUriAndUrl method resolves the URI to an HTTP URL so that an HTML video player can play it immediately.

export type VideoGenerationResponse = {
  uri: string;
  url: string;
  mimeType: string;
}

export type DownloadVideoResponse = {
  uri: string;
  mimeType: string;
}

import { inject, Injectable } from '@angular/core';
import { httpsCallable } from 'firebase/functions';
import { getDownloadURL, getStorage, ref } from 'firebase/storage';
import { GenerateVideoRequest } from '../types/video.type';
import { ConfigService } from './config.service';

@Injectable({
  providedIn: 'root'
})
export class VeoService {
  private readonly storage = getStorage();
  private readonly configService = inject(ConfigService);

  private async retrieveVideoUri<T = GenerateVideoRequest>(request: T, methodName: string): Promise<DownloadVideoResponse> {
    try {
      const { functions } = this.configService.firebaseObjects || {};
      if (!functions) {
        throw new Error('Functions does not exist.');
      }

      const downloadGcsUri = httpsCallable<T, DownloadVideoResponse>(
        functions, methodName, { timeout: 600000 }
      );
      const { data } = await downloadGcsUri(request);
      return data;
    } catch (err) {
        console.error(err);
        throw err;
    }
  }

  async downloadVideoUriAndUrl<T = GenerateVideoRequest>(request: T, methodName: CallableNames = 'videos-generateVideo'): Promise<VideoGenerationResponse> {
    const { uri, mimeType } = await this.retrieveVideoUri(request, methodName);

    if (!uri) {
      throw new Error('Video operation completed but no URI was returned.');
    }

    return getDownloadURL(ref(this.storage, uri))
      .then((url) => {
        console.log("download url", url);
        return { uri, url, mimeType };
      })
      .catch((error) => {
        switch (error.code) {
          case 'storage/object-not-found':
            throw new Error("File doesn't exist");
          case 'storage/unauthorized':
            throw new Error("User doesn't have permission to access the object");
          case 'storage/canceled':
            throw new Error("User canceled the upload");
          case 'storage/unknown':
            throw new Error("Unknown storage error occurred, inspect the server response");
        }
        throw new Error("Unknown error occurred");
      });
  }
}

import { LoaderComponent } from '@/shared/loader/loader.component';
import { ChangeDetectionStrategy, Component, input, output } from '@angular/core';
import { ExtendVideoIconComponent } from '../../icons/extend-video-icon.component';

@Component({
  selector: 'app-video-player',
  imports: [LoaderComponent, ExtendVideoIconComponent],
  template: `
@if (isGeneratingVideo()) {
  <div class="mt-6">
    <app-loader [loadingText]="loadingText()">
      <p class="text-sm">This can take several minutes. Please be patient.</p>
    </app-loader>
  </div>
} @else if (videoUrl()) {
  <div class="mt-6 flex flex-col gap-4">
    <div class="video-container">
      <video [src]="videoUrl()" controls autoplay loop class="w-full rounded-md"></video>
    </div>

    <div class="flex justify-center">
      <button
        (click)="extendVideo.emit()"
        aria-label="Extend video"
        title="Extend video"
        class="extend-btn"
      >
        <app-extend-video-icon />
        <span>Extend Video</span>
      </button>
    </div>
  </div>
}
  `,
  styleUrl: './video-player.component.css',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class VideoPlayerComponent {
  isGeneratingVideo = input(false);
  videoUrl = input.required<string>();
  extendVideo = output<void>();

  loadingText = input('Generating your video...');
}

9. Angular Integration with Gemini to Extend Video

import { VideoPlayerComponent } from './video-player/video-player.component';

@Component({
  selector: 'app-gen-media',
  imports: [
    VideoPlayerComponent,
  ],
  template: `
  <app-video-player
    [isGeneratingVideo]="isGeneratingVideo()" 
    [videoUrl]="videoUrl()"
    (extendVideo)="extendVideo()"
    [loadingText]="loadingVideoText()"
  />`,
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class GenMediaComponent {
  private readonly genMediaService = inject(GenMediaService);
  private readonly genVideoService = inject(GenVideoService);

  genMediaInput = input<GenMediaInput>();
  videoUrl = this.genVideoService.videoUrl;
  isGeneratingVideo = this.genVideoService.isGeneratingVideo.asReadonly();
  loadingVideoText = signal('Generating your video...');

  trimmedUserPrompt = computed(() => this.genMediaInput()?.userPrompt.trim() || '');

  async extendVideo() {
    if (this.trimmedUserPrompt()) {
      try {
        this.loadingVideoText.set('Extending your video...');
        await this.genVideoService.extendVideo(this.trimmedUserPrompt());
      } finally {
        this.loadingVideoText.set('Generating your video...');
      }
    }
  }
}

The GenMediaComponent calls extendVideo with the prompt and the Veo video URL to create an extended video.

import { ConfigService } from '@/ai/services/config.service';
import { VeoService } from '@/ai/services/veo.service';
import { ExtendVideoRequest, GenerateVideoFromFramesRequest, GenerateVideoRequest, VideoGenerationResponse } from '@/ai/types/video.type';
import { computed, inject, Injectable, signal, WritableSignal } from '@angular/core';
import { getValue } from 'firebase/remote-config';

@Injectable({
  providedIn: 'root'
})
export class GenVideoService {
  private readonly veoService = inject(VeoService);
  private readonly configService = inject(ConfigService);

  videoError = signal('');
  videoResponse = signal<VideoGenerationResponse>({ uri: '', url: '', mimeType: '' });
  #extendVideoCounter = signal(0);
  isGeneratingVideo = signal(false);

  videoUrl = computed(() => this.videoResponse().url);

  isVideoExtensionAllowed(counter: number) {
    const remoteConfig = this.configService.firebaseObjects?.remoteConfig;
    if (!remoteConfig) {
      console.warn('Remote config does not exist.');
      return false;
    }

    const max_extend_allowed = getValue(remoteConfig,'maxVideoExtendAllowed').asNumber();
    if (counter >= max_extend_allowed) {
      console.warn('Maximum extension limit reached.');
      return false;
    }

    return true;
  }

  async extendVideo(prompt: string): Promise<void> {
    try {
      const result = await this.extendInterpolatedVideo(
        prompt,
        this.#extendVideoCounter(),
        this.videoResponse()
      );

      if (!result) {
        return;
      }

      this.videoResponse.set(result);
      this.#extendVideoCounter.update(count => count + 1);
      console.log(`Video extended successfully. Current extension count: ${this.#extendVideoCounter()}`);
    } catch (e) {
      console.error(e);
      const errMsg = e instanceof Error ?
        e.message :
        'An unexpected error occurred in video generation.'
      this.videoError.set(errMsg);
    } finally {
      this.isGeneratingVideo.set(false);
    }
  }

  async extendInterpolatedVideo(
    prompt: string,
    counter: number,
    customVideo: Pick<VideoGenerationResponse, 'uri' | 'mimeType'>,
    generatingSignal?: WritableSignal<boolean>,
    error?: WritableSignal<string>
  ) {
    const { uri, mimeType } = customVideo;

    if (!mimeType || !uri) {
      console.warn('No video to extend. Please generate a video first.');
      return null;
    }

    if (!prompt) {
      console.warn('Prompt is required to extend the video.');
      return null;
    }

    if (!this.isVideoExtensionAllowed(counter)) {
      return null;
    }

    const actualErrorSignal = error || this.videoError;
    const actualGeneratingSignal = generatingSignal || this.isGeneratingVideo;
    try {
      actualErrorSignal.set('');
      actualGeneratingSignal.set(true);

      const extendVideoParams: ExtendVideoRequest = {
        prompt,
        video: { uri, mimeType },
      }

      return await this.veoService.downloadVideoUriAndUrl(extendVideoParams, 'videos-extendVideo');
    } catch (e) {
      console.error(e);
      throw e;
    } finally {
      actualGeneratingSignal.set(false);
    }
  }
}

Conclusion

Combining Veo 3.1 with the serverless scalability of Firebase is a powerful workflow.