DEV Community: dllewellyn

Flame Shield: Protect Your Firebase Budget

dllewellyn — Tue, 25 Feb 2025 18:55:27 +0000

Stop Unexpected Firebase Costs with Flame Shield

Unexpected Firebase costs can quickly spiral out of control, especially as usage scales. Without strict budget monitoring, developers often find themselves dealing with costly surprises. This is where Flame Shield comes in.

Flame Shield is a powerful yet simple tool designed to help you manage your Firebase project costs. By setting strict monthly spending limits and enabling an automatic "kill switch," you can prevent unexpected expenses and maintain better budget control.

This guide will walk you through setting up Flame Shield and setting up a "kill switch" - a hard limit after which your billing account will be disconnected

Because Flames Shield relies on google billing updates, there is a delay between your billing trigger and the notification - so it can go over the billing limit

Get Started with Flame Shield

Head over to https://flamesshield.com to begin protecting your Firebase budget today!

How to Set Up Flame Shield

1. Add Your Firebase Project

Open the Flame Shield app. Flames Shield app
Tap the + button to add a new project.
Enter your Firebase Project ID and tap Add Project.

2. Re-authenticate Your Account

After adding your project, tap Re-authenticate. This ensures that Flame Shield can set up necessary alerts and protect your account.

Under the hood, we are requesting enhanced permissions temporarily to set up the necessary access to your billing acount

Review the requested permissions and tap Re-authenticate again.
Select your Google account and grant access.

3. Set Up Billing Protection

On the Dashboard, tap Setup Billing Protection.
Enter your desired monthly spending limit and tap Activate Billing Protection. You can update this limit at any time.

Testing the Kill Switch

It's generally a good idea to test your kill-switch at a less stressful time than 3am to make sure you know what the process is to re-connect.

Flame Shield’s kill switch automatically suspends your project if your spending exceeds the set limit. Here’s how to test this feature:

On the Dashboard, locate the Service Running status and tap Test Kill Switch.
Confirm that you want to proceed with the test.
Flame Shield will simulate an overspending scenario, and you’ll receive a Billing Account Detached alert.
To re-enable your project, tap Re-enable Billing Account.

Monitoring Current Spend

Flame Shield allows you to track your project’s expenses directly from the Dashboard. The Current Spend section provides an overview of your usage and ensures you stay within budget.

How It Works

Spend data updates roughly every hour as alerts come in from Google.
Notifications are sent out when:
- Billing limits are reached.
- The kill switch is no longer active (which can happen for various reasons). It’s essential to ensure that it remains active at all times.
- The kill switch is triggered, indicating that spending has exceeded the set limit.

By keeping an eye on the Current Spend section and ensuring the kill switch remains active, you can proactively manage your Firebase budget and avoid unexpected charges.

Some other good things

You can configure alerting - this will notify you when your billing limit is at 50%, when your 'ping' is down and when your kill-switch has kicked in
Ping - there are all sorts of ways that the service could stop working. You might delete the pub/sub, change the billing account or otherwise change the permissions. We ping the system every hour to make sure that kill-switch is always operational when you need it (and let you know what action to take is not)

By following these steps, you can efficiently manage your Firebase project costs and prevent unexpected overspending. Get started today at https://flamesshield.com!

How to Prevent Firebase Runaway Costs

dllewellyn — Thu, 06 Feb 2025 18:34:36 +0000

Firebase is an incredibly powerful platform that lets developers build robust applications quickly. However, users are rightly worried about the eye-watering costs, which are usually unexpected, and the fear of hitting that kind of bill puts off a lot of people for whom Firebase is a brilliant tool.

In this post, we’ll explore the main Firebase services that tend to incur the highest costs, discuss the events that can cause sudden billing spikes, and share strategies for avoiding these Firebase bills, helping you keep your spending under control.

Check out the full post:

How to Prevent Firebase Runaway Costs

Firebase authentication - Best practices for password requirements

dllewellyn — Fri, 17 Jan 2025 07:43:37 +0000

While building out an up-coming security and compliance dashboard for Firebase, some of the rules we looked at were around authentication settings in Firebase which are 'insecure' - we found a fair few that are defaults which was surprising!

How to evaluate the safety and security of LLM Applications?

dllewellyn — Wed, 18 Dec 2024 18:13:39 +0000

I've written a guide on essentially how to test LLM apps for security and safety. Looking forward to hearing what you think!

How to evaluate the safety and security of LLM Applications?

Build a serverless AI App with Cloud run, Python, Gemini and Vertex AI

dllewellyn — Wed, 25 Sep 2024 11:17:44 +0000

This is part 1 in a series!

Show me the code:

https://github.com/dllewellyn/hello-gemini

Introduction

In this series of tutorials, we’re going to use a suite of google tools — AI, cloud and dev — in order to create and deploy an AI powered application. We’ll steer clear on chatbots, because they are a tedious in the extreme and focus on building something more interesting.

Creating a “Hello World” Flask Application with Google IDX and the Gemini API

To get started, we’re going to use the ‘hello world’ IDX — Gemini and flask application. This gives us a really quick way to get setup with some AI tooling.

Project Setup

Navigate to Google IDX: Begin by accessing the Google IDX platform in your web browser.
Select the Gemini API Template: From the IDX welcome screen, locate and click on the “Gemini API” template under the “Start something new with a template” section.

Configure New Workspace: A “New Workspace” window will appear.
Name your workspace: I’ve just called it “hello-gemini.”
Environment: Choose the “Python Web App (Flask)” option from the dropdown menu.

Create Workspace: Once configured, click the “Create” button to initialise the workspace creation.
Await Setup Completion: IDX will set up the necessary environment for your Flask application.

With your workspace ready, we’ve got a basic ‘gemini’ application

Looking through the Hello World application

Obtain a Gemini API Key

Before you begin, you’ll need an API key to access the Gemini API.

Visit the Google Cloud Console.
Navigate to the ‘API Keys’ section.
Click ‘Create API Key’.
Choose an existing Google Cloud project or create a new one.
Copy the generated API key. Remember to store your API key securely!

Set up the Flask Application

Check the existing app in main.py — update this with your actual API key.

This is the basic setup of the gemini API

import os
import json
from google.generativeai import genai
from flask import Flask, render_template, request, jsonify

# Replace 'YOUR_API_KEY' with your actual API key
API_KEY = 'YOUR_API_KEY'
genai.configure(api_key=API_KEY)
app = Flask(__name__)

Check theHTML Template

Take a look in index.htmlto serve as the front-end for your web application. This template will display images of baked goods, an input field for the prompt, and a results section.

<!DOCTYPE html>
<html>
<head>
    <title>Baking with the Gemini API</title>
</head>
<body>
    <h1>Baking with the Gemini API</h1>
    <div>
        <img src="images/baked-good-1.jpg" alt="Baked Good 1">
        <img src="images/baked-good-2.jpg" alt="Baked Good 2">
        <img src="images/baked-good-3.jpg" alt="Baked Good 3">
    </div>
    <div>
        <label for="prompt">Provide an example recipe for the baked goods in:</label>
        <input type="text" id="prompt" name="prompt">
        <button onclick="generateRecipe()">Go</button>
    </div>
    <div id="results">
        <h2>Results will appear here</h2>
    </div>
</body>
</html>

You’ll see three images on the screen and a prompt input.

Define Flask Routes and Functions

In your main.py file, define the routes and functions to handle requests from the front-end and interact with the Gemini API.

@app.route('/')
def index():
  return render_template('index.html')

@app.route("/api/generate", methods=["POST"])
def generate_api():
    if request.method == "POST":
        try:
            req_body = request.get_json()
            content = req_body.get("contents")
            model = genai.GenerativeModel(model_name=req_body.get("model"))
            response = model.generate_content(content, stream=True)
            def stream():
                for chunk in response:
                    yield 'data: %s\n\n' % json.dumps({ "text": chunk.text })

            return stream(), {'Content-Type': 'text/event-stream'}

        except Exception as e:
            return jsonify({ "error": str(e) })

In this block of code, you’ll notice that we take the ‘model’ from the input, and the ‘contents’ which have been fired to the API from javascript.

We ‘stream’ the response — and that means we can pass that streamed content back to the frontend so they’re not waiting for everything to finish before showing it to the user.

Run the Flask Application

If you’re inside IDX, you can just view it in the ‘preview’ window

Now, you should be able to access your web application in your browser, select an image of a baked good, enter a prompt, and generate a baking recipe using the Gemini API. If you’re running inside of IDX, you can just run the ‘web preview’ — click Cmd+Shift+P and enter ‘Web preview’ (if on a mac) and you’ll see the preview window.

Hit generate and you’ll see a recipe based on the image you’ve selected:

Analysing the javascript

If you look through main.js you’ll see this file:

import { streamGemini } from './gemini-api.js';

let form = document.querySelector('form');
let promptInput = document.querySelector('input[name="prompt"]');
let output = document.querySelector('.output');

form.onsubmit = async (ev) => {
  ev.preventDefault();
  output.textContent = 'Generating...';

  try {
    // Load the image as a base64 string
    let imageUrl = form.elements.namedItem('chosen-image').value;
    let imageBase64 = await fetch(imageUrl)
      .then(r => r.arrayBuffer())
      .then(a => base64js.fromByteArray(new Uint8Array(a)));

    // Assemble the prompt by combining the text with the chosen image
    let contents = [
      {
        role: 'user',
        parts: [
          { inline_data: { mime_type: 'image/jpeg', data: imageBase64, } },
          { text: promptInput.value }
        ]
      }
    ];

    // Call the multimodal model, and get a stream of results
    let stream = streamGemini({
      model: 'gemini-1.5-flash', // or gemini-1.5-pro
      contents,
    });

    // Read from the stream and interpret the output as markdown
    let buffer = [];
    let md = new markdownit();
    for await (let chunk of stream) {
      buffer.push(chunk);
      output.innerHTML = md.render(buffer.join(''));
    }
  } catch (e) {
    output.innerHTML += '<hr>' + e;
  }
};

In it you can see that when the form is submit, it passes the model and the ‘contents’ — this contents object is what gemini expects and in python it mostly does a pass through.

content = req_body.get("contents")
model = genai.GenerativeModel(model_name=req_body.get("model"))
response = model.generate_content(content, stream=True)

It’s quite a straightforward setup — we base64 encode the image, upload it along with the prompt to gemini and then stream the response.

You can also see the stream response is coming from the gemini-api.js file:

/**
 * Calls the given Gemini model with the given image and/or text
 * parts, streaming output (as a generator function).
 */
export async function* streamGemini({
  model = 'gemini-1.5-flash', // or gemini-1.5-pro
  contents = [],
} = {}) {
  // Send the prompt to the Python backend
  // Call API defined in main.py
  let response = await fetch("/api/generate", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model, contents })
  });

  yield* streamResponseChunks(response);
}

/**
 * A helper that streams text output chunks from a fetch() response.
 */
async function* streamResponseChunks(response) {
  let buffer = '';

  const CHUNK_SEPARATOR = '\n\n';

  let processBuffer = async function* (streamDone = false) {
    while (true) {
      let flush = false;
      let chunkSeparatorIndex = buffer.indexOf(CHUNK_SEPARATOR);
      if (streamDone && chunkSeparatorIndex < 0) {
        flush = true;
        chunkSeparatorIndex = buffer.length;
      }
      if (chunkSeparatorIndex < 0) {
        break;
      }

      let chunk = buffer.substring(0, chunkSeparatorIndex);
      buffer = buffer.substring(chunkSeparatorIndex + CHUNK_SEPARATOR.length);
      chunk = chunk.replace(/^data:\s*/, '').trim();
      if (!chunk) {
        if (flush) break;
        continue;
      }
      let { error, text } = JSON.parse(chunk);
      if (error) {
        console.error(error);
        throw new Error(error?.message || JSON.stringify(error));
      }
      yield text;
      if (flush) break;
    }
  };

  const reader = response.body.getReader();
  try {
    while (true) {
      const { done, value } = await reader.read()
      if (done) break;
      buffer += new TextDecoder().decode(value);
      console.log(new TextDecoder().decode(value));
      yield* processBuffer();
    }
  } finally {
    reader.releaseLock();
  }

  yield* processBuffer(true);
}

In this file, the response is streamed from the server and rendered into markdown as the data comes back to the frontend.

That’s a wrap

We’ve covered a fair bit in this hello-world application, explaining how to get gemini setup in a python and flask app inside IDX, how to stream the response and how to use images as part of it!

Build a serverless AI App with Cloud run, Python, Gemini and Vertex AI

dllewellyn — Mon, 23 Sep 2024 17:57:54 +0000

In this series of tutorials, we’re going to use a suite of google tools - AI, cloud and dev - in order to create and deploy an AI powered application. We’ll steer clear on chatbots, because they are a tedious in the extreme and focus on building something more interesting.

Creating a "Hello World" Flask Application with Google IDX and the Gemini API

To get started, we're going to use the 'hello world' IDX - Gemini and flask application. This gives us a really quick way to get setup with some AI.

Project Setup

Navigate to Google IDX: Begin by accessing the Google IDX platform in your web browser.
Select the Gemini API Template: From the IDX welcome screen, locate and click on the "Gemini API" template under the "Start something new with a template" section.
Configure New Workspace: A "New Workspace" window will appear. Here, customise the following settings:
- Name your workspace: Provide a descriptive name for your project, such as "hello-gemini."
- Environment: Choose the "Python Web App (Flask)" option from the dropdown menu.

Create Workspace: Once configured, click the "Create" button to initialise the workspace creation.
Await Setup Completion: IDX will set up the necessary environment for your Flask application. This process might take a few moments, and a progress bar will indicate the status.

Next Steps

With your workspace ready, you can proceed to develop your Flask application within the provided environment.

This simple setup using Google IDX provides a foundation for building and deploying Flask applications that interact with the Gemini API.

Getting started with IDX

Go to https://idx.google.com/ - you'll need a google account to get started, but you'll also need it for the rest of the gemini/ vertex stuff too.

Looking through the Hello World application

Obtain a Gemini API Key

Before you begin, you'll need an API key to access the Gemini API.

Visit the Google Cloud Console.
Navigate to the 'API Keys' section.
Click 'Create API Key'.
Choose an existing Google Cloud project or create a new one.
Copy the generated API key. Remember to store your API key securely!

Set up the Flask Application

Create a Python file (e.g., main.py) and install the necessary libraries:

import os
import json
from google.generativeai import genai
from flask import Flask, render_template, request, jsonify

# Replace 'YOUR_API_KEY' with your actual API key
API_KEY = 'YOUR_API_KEY'
genai.configure(api_key=API_KEY)
app = Flask(__name__)

# ... (Rest of the code will be added in the following steps)

Create the HTML Template

Create an HTML file (e.g., index.html) to serve as the front-end for your web application. This template will display images of baked goods, an input field for the prompt, and a results section.

<!DOCTYPE html>
<html>
<head>
    <title>Baking with the Gemini API</title>
</head>
<body>
    <h1>Baking with the Gemini API</h1>
    <div>
        <img src="images/baked-good-1.jpg" alt="Baked Good 1">
        <img src="images/baked-good-2.jpg" alt="Baked Good 2">
        <img src="images/baked-good-3.jpg" alt="Baked Good 3">
    </div>
    <div>
        <label for="prompt">Provide an example recipe for the baked goods in:</label>
        <input type="text" id="prompt" name="prompt">
        <button onclick="generateRecipe()">Go</button>
    </div>
    <div id="results">
        <h2>Results will appear here</h2>
    </div>

    <script>
        function generateRecipe() {
            // ... (JavaScript code to handle user input and send requests to the Flask app)
        }
    </script>
</body>
</html>

Define Flask Routes and Functions

In your main.py file, define the routes and functions to handle requests from the front-end and interact with the Gemini API.

Build a serverless AI App with Cloud run, Python, Gemini and Vertex AI

This is part 1 in a series!

Show me the code:

https://github.com/dllewellyn/hello-gemini

Introduction

Creating a “Hello World” Flask Application with Google IDX and the Gemini API

To get started, we’re going to use the ‘hello world’ IDX — Gemini and flask application. This gives us a really quick way to get setup with some AI tooling.

Project Setup

Navigate to Google IDX: Begin by accessing the Google IDX platform in your web browser.
Select the Gemini API Template: From the IDX welcome screen, locate and click on the “Gemini API” template under the “Start something new with a template” section.

!https://cdn-images-1.medium.com/max/800/1*3UbWMkF7ijeiRDG11z7KDw.png

Configure New Workspace: A “New Workspace” window will appear.
Name your workspace: I’ve just called it “hello-gemini.”
Environment: Choose the “Python Web App (Flask)” option from the dropdown menu.

!https://cdn-images-1.medium.com/max/800/1*s9j9Z4H7RirlczOCKohNiw.png

Create Workspace: Once configured, click the “Create” button to initialise the workspace creation.
Await Setup Completion: IDX will set up the necessary environment for your Flask application.

With your workspace ready, we’ve got a basic ‘gemini’ application

Looking through the Hello World application

Obtain a Gemini API Key

!https://cdn-images-1.medium.com/max/800/1*c6tBFPL_FQN9mSHMu84AiQ.png

Before you begin, you’ll need an API key to access the Gemini API.

Visit the Google Cloud Console.
Navigate to the ‘API Keys’ section.
Click ‘Create API Key’.
Choose an existing Google Cloud project or create a new one.
Copy the generated API key. Remember to store your API key securely!

Set up the Flask Application

Check the existing app in main.py — update this with your actual API key.

This is the basic setup of the gemini API

import os
import json
from google.generativeai import genai
from flask import Flask, render_template, request, jsonify

# Replace 'YOUR_API_KEY' with your actual API key
API_KEY = 'YOUR_API_KEY'
genai.configure(api_key=API_KEY)
app = Flask(__name__)

Check theHTML Template

Take a look in index.htmlto serve as the front-end for your web application. This template will display images of baked goods, an input field for the prompt, and a results section.

<!DOCTYPE html>
<html>
<head>
    <title>Baking with the Gemini API</title>
</head>
<body>
    <h1>Baking with the Gemini API</h1>
    <div>
        <img src="images/baked-good-1.jpg" alt="Baked Good 1">
        <img src="images/baked-good-2.jpg" alt="Baked Good 2">
        <img src="images/baked-good-3.jpg" alt="Baked Good 3">
    </div>
    <div>
        <label for="prompt">Provide an example recipe for the baked goods in:</label>
        <input type="text" id="prompt" name="prompt">
        <button onclick="generateRecipe()">Go</button>
    </div>
    <div id="results">
        <h2>Results will appear here</h2>
    </div>
</body>
</html>

You’ll see three images on the screen and a prompt input.

Define Flask Routes and Functions

In your main.py file, define the routes and functions to handle requests from the front-end and interact with the Gemini API.

@app.route('/')
def index():
  return render_template('index.html')

@app.route("/api/generate", methods=["POST"])
def generate_api():
    if request.method == "POST":
        try:
            req_body = request.get_json()
            content = req_body.get("contents")
            model = genai.GenerativeModel(model_name=req_body.get("model"))
            response = model.generate_content(content, stream=True)
            def stream():
                for chunk in response:
                    yield 'data: %s\n\n' % json.dumps({ "text": chunk.text })

            return stream(), {'Content-Type': 'text/event-stream'}

        except Exception as e:
            return jsonify({ "error": str(e) })

In this block of code, you’ll notice that we take the ‘model’ from the input, and the ‘contents’ which have been fired to the API from javascript.

We ‘stream’ the response — and that means we can pass that streamed content back to the frontend so they’re not waiting for everything to finish before showing it to the user.

Run the Flask Application

If you’re inside IDX, you can just view it in the ‘preview’ window

!https://cdn-images-1.medium.com/max/800/1*Xl8qXDvkkDJ9PoozstwupA.png

Hit generate and you’ll see a recipe based on the image you’ve selected:

!https://cdn-images-1.medium.com/max/800/1*9hBTJhjNHWv4lmZ-yFvMAw.png

Analysing the javascript

If you look through main.js you’ll see this file:

import { streamGemini } from './gemini-api.js';

let form = document.querySelector('form');
let promptInput = document.querySelector('input[name="prompt"]');
let output = document.querySelector('.output');

form.onsubmit = async (ev) => {
  ev.preventDefault();
  output.textContent = 'Generating...';

  try {
    // Load the image as a base64 string
    let imageUrl = form.elements.namedItem('chosen-image').value;
    let imageBase64 = await fetch(imageUrl)
      .then(r => r.arrayBuffer())
      .then(a => base64js.fromByteArray(new Uint8Array(a)));

    // Assemble the prompt by combining the text with the chosen image
    let contents = [
      {
        role: 'user',
        parts: [
          { inline_data: { mime_type: 'image/jpeg', data: imageBase64, } },
          { text: promptInput.value }
        ]
      }
    ];

    // Call the multimodal model, and get a stream of results
    let stream = streamGemini({
      model: 'gemini-1.5-flash', // or gemini-1.5-pro
      contents,
    });

    // Read from the stream and interpret the output as markdown
    let buffer = [];
    let md = new markdownit();
    for await (let chunk of stream) {
      buffer.push(chunk);
      output.innerHTML = md.render(buffer.join(''));
    }
  } catch (e) {
    output.innerHTML += '<hr>' + e;
  }
};

In it you can see that when the form is submit, it passes the model and the ‘contents’ — this contents object is what gemini expects and in python it mostly does a pass through.

content = req_body.get("contents")
model = genai.GenerativeModel(model_name=req_body.get("model"))
response = model.generate_content(content, stream=True)

It’s quite a straightforward setup — we base64 encode the image, upload it along with the prompt to gemini and then stream the response.

You can also see the stream response is coming from the gemini-api.js file:

/**
 * Calls the given Gemini model with the given image and/or text
 * parts, streaming output (as a generator function).
 */
export async function* streamGemini({
  model = 'gemini-1.5-flash', // or gemini-1.5-pro
  contents = [],
} = {}) {
  // Send the prompt to the Python backend
  // Call API defined in main.py
  let response = await fetch("/api/generate", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model, contents })
  });

  yield* streamResponseChunks(response);
}

/**
 * A helper that streams text output chunks from a fetch() response.
 */
async function* streamResponseChunks(response) {
  let buffer = '';

  const CHUNK_SEPARATOR = '\n\n';

  let processBuffer = async function* (streamDone = false) {
    while (true) {
      let flush = false;
      let chunkSeparatorIndex = buffer.indexOf(CHUNK_SEPARATOR);
      if (streamDone && chunkSeparatorIndex < 0) {
        flush = true;
        chunkSeparatorIndex = buffer.length;
      }
      if (chunkSeparatorIndex < 0) {
        break;
      }

      let chunk = buffer.substring(0, chunkSeparatorIndex);
      buffer = buffer.substring(chunkSeparatorIndex + CHUNK_SEPARATOR.length);
      chunk = chunk.replace(/^data:\s*/, '').trim();
      if (!chunk) {
        if (flush) break;
        continue;
      }
      let { error, text } = JSON.parse(chunk);
      if (error) {
        console.error(error);
        throw new Error(error?.message || JSON.stringify(error));
      }
      yield text;
      if (flush) break;
    }
  };

  const reader = response.body.getReader();
  try {
    while (true) {
      const { done, value } = await reader.read()
      if (done) break;
      buffer += new TextDecoder().decode(value);
      console.log(new TextDecoder().decode(value));
      yield* processBuffer();
    }
  } finally {
    reader.releaseLock();
  }

  yield* processBuffer(true);
}

In this file, the response is streamed from the server and rendered into markdown as the data comes back to the frontend.

That’s a wrap

We’ve covered a fair bit in this hello-world application, explaining how to get gemini setup in a python and flask app inside IDX, how to stream the response and how to use images as part of it!

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/generate', methods=['POST'])
def generate_recipe():
    if request.method == 'POST':
        selected_image = request.form['selected_image']
        prompt = request.form['prompt']

        # ... (Code to process the image and prompt, and send a request to the Gemini API)

        response = genai.generate_text(
            model='models/gemini-1.5-pro-vision',
            prompt=prompt,
            image=image_data,
            stream_output=True
        )

        # ... (Code to stream the response back to the front-end)

        return jsonify({'status': 'success', 'response': streamed_response})
        ```
{% endraw %}


### Handle User Input and Display Results

In your {% raw %}`index.html`{% endraw %} file, add JavaScript code to handle user input, send requests to the Flask app, and display the streamed response from the Gemini API.
{% raw %}


```javascript
// ... (Previous code from step 3)

function generateRecipe() {
    const selectedImage = // ... (Get the URL of the selected image)
    const prompt = document.getElementById('prompt').value;

    fetch('/generate', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/x-www-form-urlencoded'
        },
        body: `selected_image=${selectedImage}&prompt=${prompt}`
    })
    .then(response => response.json())
    .then(data => {
        if (data.status === 'success') {
            const resultsDiv = document.getElementById('results');
            resultsDiv.innerHTML = ''; // Clear previous results

            // ... (Code to display the streamed response in the resultsDiv)
        } else {
            // ... (Handle errors)
        }
    })
    .catch(error => {
        // ... (Handle errors)
    });
}

Run the Flask Application

Run your Flask app using the following command:

#!/bin/sh

python -m flask --app main run -p $PORT --debug

Now, you should be able to access your web application in your browser, select an image of a baked good, enter a prompt, and generate a baking recipe using the Gemini API. If you're running inside of IDX, you can just run the 'web preview' - click Cmd+Shift+P and enter 'Web preview' (if on a mac) and you'll see the preview window.

A wrap for now

That’s a wrap for now, in the next post we’ll talk through changing this to use vertex AI instead

Build a serverless AI App with Cloud run, Python, Gemini and Vertex AI

This is part 1 in a series!

Show me the code:

https://github.com/dllewellyn/hello-gemini

Introduction

Creating a “Hello World” Flask Application with Google IDX and the Gemini API

To get started, we’re going to use the ‘hello world’ IDX — Gemini and flask application. This gives us a really quick way to get setup with some AI tooling.

Project Setup

Navigate to Google IDX: Begin by accessing the Google IDX platform in your web browser.
Select the Gemini API Template: From the IDX welcome screen, locate and click on the “Gemini API” template under the “Start something new with a template” section.

!https://cdn-images-1.medium.com/max/800/1*3UbWMkF7ijeiRDG11z7KDw.png

Configure New Workspace: A “New Workspace” window will appear.
Name your workspace: I’ve just called it “hello-gemini.”
Environment: Choose the “Python Web App (Flask)” option from the dropdown menu.

!https://cdn-images-1.medium.com/max/800/1*s9j9Z4H7RirlczOCKohNiw.png

Create Workspace: Once configured, click the “Create” button to initialise the workspace creation.
Await Setup Completion: IDX will set up the necessary environment for your Flask application.

With your workspace ready, we’ve got a basic ‘gemini’ application

Looking through the Hello World application

Obtain a Gemini API Key

!https://cdn-images-1.medium.com/max/800/1*c6tBFPL_FQN9mSHMu84AiQ.png

Before you begin, you’ll need an API key to access the Gemini API.

Visit the Google Cloud Console.
Navigate to the ‘API Keys’ section.
Click ‘Create API Key’.
Choose an existing Google Cloud project or create a new one.
Copy the generated API key. Remember to store your API key securely!

Set up the Flask Application

Check the existing app in main.py — update this with your actual API key.

This is the basic setup of the gemini API

import os
import json
from google.generativeai import genai
from flask import Flask, render_template, request, jsonify

# Replace 'YOUR_API_KEY' with your actual API key
API_KEY = 'YOUR_API_KEY'
genai.configure(api_key=API_KEY)
app = Flask(__name__)

Check theHTML Template

Take a look in index.htmlto serve as the front-end for your web application. This template will display images of baked goods, an input field for the prompt, and a results section.

<!DOCTYPE html>
<html>
<head>
    <title>Baking with the Gemini API</title>
</head>
<body>
    <h1>Baking with the Gemini API</h1>
    <div>
        <img src="images/baked-good-1.jpg" alt="Baked Good 1">
        <img src="images/baked-good-2.jpg" alt="Baked Good 2">
        <img src="images/baked-good-3.jpg" alt="Baked Good 3">
    </div>
    <div>
        <label for="prompt">Provide an example recipe for the baked goods in:</label>
        <input type="text" id="prompt" name="prompt">
        <button onclick="generateRecipe()">Go</button>
    </div>
    <div id="results">
        <h2>Results will appear here</h2>
    </div>
</body>
</html>

You’ll see three images on the screen and a prompt input.

Define Flask Routes and Functions

In your main.py file, define the routes and functions to handle requests from the front-end and interact with the Gemini API.

@app.route('/')
def index():
  return render_template('index.html')

@app.route("/api/generate", methods=["POST"])
def generate_api():
    if request.method == "POST":
        try:
            req_body = request.get_json()
            content = req_body.get("contents")
            model = genai.GenerativeModel(model_name=req_body.get("model"))
            response = model.generate_content(content, stream=True)
            def stream():
                for chunk in response:
                    yield 'data: %s\n\n' % json.dumps({ "text": chunk.text })

            return stream(), {'Content-Type': 'text/event-stream'}

        except Exception as e:
            return jsonify({ "error": str(e) })

In this block of code, you’ll notice that we take the ‘model’ from the input, and the ‘contents’ which have been fired to the API from javascript.

We ‘stream’ the response — and that means we can pass that streamed content back to the frontend so they’re not waiting for everything to finish before showing it to the user.

Run the Flask Application

If you’re inside IDX, you can just view it in the ‘preview’ window

!https://cdn-images-1.medium.com/max/800/1*Xl8qXDvkkDJ9PoozstwupA.png

Hit generate and you’ll see a recipe based on the image you’ve selected:

!https://cdn-images-1.medium.com/max/800/1*9hBTJhjNHWv4lmZ-yFvMAw.png

Analysing the javascript

If you look through main.js you’ll see this file:

import { streamGemini } from './gemini-api.js';

let form = document.querySelector('form');
let promptInput = document.querySelector('input[name="prompt"]');
let output = document.querySelector('.output');

form.onsubmit = async (ev) => {
  ev.preventDefault();
  output.textContent = 'Generating...';

  try {
    // Load the image as a base64 string
    let imageUrl = form.elements.namedItem('chosen-image').value;
    let imageBase64 = await fetch(imageUrl)
      .then(r => r.arrayBuffer())
      .then(a => base64js.fromByteArray(new Uint8Array(a)));

    // Assemble the prompt by combining the text with the chosen image
    let contents = [
      {
        role: 'user',
        parts: [
          { inline_data: { mime_type: 'image/jpeg', data: imageBase64, } },
          { text: promptInput.value }
        ]
      }
    ];

    // Call the multimodal model, and get a stream of results
    let stream = streamGemini({
      model: 'gemini-1.5-flash', // or gemini-1.5-pro
      contents,
    });

    // Read from the stream and interpret the output as markdown
    let buffer = [];
    let md = new markdownit();
    for await (let chunk of stream) {
      buffer.push(chunk);
      output.innerHTML = md.render(buffer.join(''));
    }
  } catch (e) {
    output.innerHTML += '<hr>' + e;
  }
};

In it you can see that when the form is submit, it passes the model and the ‘contents’ — this contents object is what gemini expects and in python it mostly does a pass through.

content = req_body.get("contents")
model = genai.GenerativeModel(model_name=req_body.get("model"))
response = model.generate_content(content, stream=True)

It’s quite a straightforward setup — we base64 encode the image, upload it along with the prompt to gemini and then stream the response.

You can also see the stream response is coming from the gemini-api.js file:

/**
 * Calls the given Gemini model with the given image and/or text
 * parts, streaming output (as a generator function).
 */
export async function* streamGemini({
  model = 'gemini-1.5-flash', // or gemini-1.5-pro
  contents = [],
} = {}) {
  // Send the prompt to the Python backend
  // Call API defined in main.py
  let response = await fetch("/api/generate", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ model, contents })
  });

  yield* streamResponseChunks(response);
}

/**
 * A helper that streams text output chunks from a fetch() response.
 */
async function* streamResponseChunks(response) {
  let buffer = '';

  const CHUNK_SEPARATOR = '\n\n';

  let processBuffer = async function* (streamDone = false) {
    while (true) {
      let flush = false;
      let chunkSeparatorIndex = buffer.indexOf(CHUNK_SEPARATOR);
      if (streamDone && chunkSeparatorIndex < 0) {
        flush = true;
        chunkSeparatorIndex = buffer.length;
      }
      if (chunkSeparatorIndex < 0) {
        break;
      }

      let chunk = buffer.substring(0, chunkSeparatorIndex);
      buffer = buffer.substring(chunkSeparatorIndex + CHUNK_SEPARATOR.length);
      chunk = chunk.replace(/^data:\s*/, '').trim();
      if (!chunk) {
        if (flush) break;
        continue;
      }
      let { error, text } = JSON.parse(chunk);
      if (error) {
        console.error(error);
        throw new Error(error?.message || JSON.stringify(error));
      }
      yield text;
      if (flush) break;
    }
  };

  const reader = response.body.getReader();
  try {
    while (true) {
      const { done, value } = await reader.read()
      if (done) break;
      buffer += new TextDecoder().decode(value);
      console.log(new TextDecoder().decode(value));
      yield* processBuffer();
    }
  } finally {
    reader.releaseLock();
  }

  yield* processBuffer(true);
}

In this file, the response is streamed from the server and rendered into markdown as the data comes back to the frontend.

That’s a wrap

We’ve covered a fair bit in this hello-world application, explaining how to get gemini setup in a python and flask app inside IDX, how to stream the response and how to use images as part of it!

Gen AI + Already proven practices = better Dev teams

dllewellyn — Mon, 23 Sep 2024 09:09:03 +0000

Using Generative AI to change how we do product engineer

A failure of imagination

Since the Generative AI starting gun was first fired, we've started to see a lot of people trying to do the things they already do better, but using AI. To be fair, the results are positive - the pragmatic engineer laid out some of the ways that people are already trying to use generative AI, especially github co-pilot, to make them more productive

https://newsletter.pragmaticengineer.com/p/ai-tooling-2024

But what's missing, is re-thinking the whole process from scratch

If I ask people what they wanted, they'd have said a faster horse - Henry Ford

It's hard to break out from the constraints that come from working in a particular environment, in a particular way for a long time - developers will look at how they can make themselves more productive and write higher quality software, product managers will look at how they can research better or build better personas, business analysts will look at capturing requirements and writing better tickets, and designers will look at how they can build their high-fidelity stories better.

Without getting people out of thinking of their current jobs, the plateau from AI is likely to come really quickly - its probably not going to be trusted to build large complex software any time soon, so all it can do it is make it a bit faster (or maybe a lot faster).

There are probably a few killer ideas out there, and I'm sure we're going to see some revolutionary new approach soon - but in the mean time, there are ideas that we could and should already be doing, which have already been proven to work, and which are much easier to adopt when you have AI to help.

There are also some ideas that are currently acknowledged as being things we shouldn't do, writing comments for every function is a good example, but the desire to do so persists - there is value in it, and AI can likely help to make it easier, more effective and more palatable.

Things we should already be doing

Small teams

The problem

Brooks's law is an observation about software project management that "Adding manpower to a late software project makes it later." - Mythical Man Month

We see the concepts of small teams over and over, it's a core part of Agile and DevOps, 'Getting real' even advocate a team as small as 3 for version 1.0. That small teams are needed isn't really that controversial - the challenge comes when we add two other factors:

Teams should be mostly self-sufficient - Accelerate demonstrates that hand-offs to separate QA teams for testing are bad, are architecture review boards are bad. The entire concept of DevOps rests on the idea that dev and ops hand-offs are bad.
We have a lot of job-titles and specialisms - If we need a few cloud engineers for the infrastructure, an architect to design it, a UI/UX designer for the UI design, a few frontend engineers for the frontend, a few backend engineers for the backend, a security person, a business analyst, a product manager, someone to look after delivery, a QA for testing, maybe an automation engineer for the pipelines (and on and on it goes) - how can we ever include all those roles in one team? ### The solution

The logical answer is to make your team a team of generalists.

Small teams need people who can wear different hats. You need designers who can write. You need programmers who understand design. - Getting Real, Signal37

By cross-training and growing engineering skills, generalists can do orders of magnitude more work than their specialist counterparts and it also improves our overall flow of work by removing queues and wait times - The DevOps Handbook - Gene Kim et al

So, the solution is easy - make everyone generalists right? This is as much a cultural challenge as anything else; but from a purely skills-based point of view there's a great opportunity to use AI.

AI as a solution

People might balk at the idea of asking AI to help find security issues, asses design against user personas, look for edge cases when using API libraries, generate automated tests or help write IaC - but by focusing on 'knowing when to ask for help' rather than knowing how to do everything perfectly, it means that you end up with much more efficient teams that are much more likely to focus on the right tasks at the right time.

This is obviously controversial, as is the basic idea of a full-stack engineer, but I would argue with some discipline from the individual engineers to ensure they understand what they've generated, and to get AI to double check things, it's a perfectly safe and very productive way to work.

One thing that AI can't help with though (or maybe I'm just drawing a blank...), is the cultural changes needed to make it happen

Design to Dev hand-offs

The problem

Let's expand on the generalists point, to include not just engineers but other roles in the team. DevOps is heavily influenced by the idea of how difficult handing off from dev to ops is, but design to dev is hardly a smooth transition - especially if weeks went into finalising the design and agreeing it with stakeholders.

The idea

There's already another school of thought on this:

In their book "Getting real" (https://basecamp.com/gettingreal/06.3-from-idea-to-implementation) the 37 Signals team lay out their process for getting to new features:

Paper sketches > Create HTML screens > Code It

Throughout the book, they emphasise the going straight from paper sketches to HTML - a sentiment that is repeated in rework and is evident in their hotwired suite of open source tools.

Getting some HTML, CSS and javascript out to customers for them to try out is a much more true test than 'clickable' prototypes. 37Signals advocate the idea of having devs doing a bit of design, and designers doing bits of html - that can be a bit of a tall order, but...

AI as a solution

With some prompting, designer's don't even need to learn much html/css. Some of my colleagues at Create Future (@angus Allan in particular) have produced some pretty impressive websites and games with just prompting - so, what if we skip the 'high fidelity' designs that get passed over to developers? AI makes it's easy to change too, you can sit with a customer live and modify your page, refresh - "How's that?" - much better to iterate in minutes than in weeks.

Even better, you can take your hand-drawn sketches, upload to chat GPT and ask for it to turn it into a website for you.

Picking the right tech-stack for the problem

The problem

In monolithic architectures and code-bases, we usually have to pick a single tech-stack and accept that it's going to be the best fit for some things, and not for others. If we're lucky, we optimised for the big problems - getting the best tech stack for the problems we face most often; if we're not - we're battling our setup to make it work better

The solution

With a system composed of multiple, collaborating microservices, we can decide to use different technologies inside each one. This allows us to pick the right tool for each job rather than having to select a more standardized, one-size-fits-all approach that often ends up being the lowest common denominator. - Sam Newman, Building Microservices, 2nd Edition

But what happens in reality is that teams tend to pick a particular stack and only work in that. Sometimes the driver is outside the team - it's reasonable to want to make it easy for engineers to move between teams and not have the programming language or tech as a barrier, but often it's from the teams themselves - people just pick the tech they know.

AI as a solution

One of the benefits of micro-services is allowing teams to pick the best tech for their specific problem. Whilst it's difficult to break people out of the comfort of their tech stack - gen AI can at least help with the skills.

If you 'know enough' of a coding language to get things done, AI can help find various issues in you're code, if you don't know much about the programming language's ecosystem you can research various libraries people use, assess your code against best practices, suggest how you might convert from a language you know to one you don't, debug code or explain how you can debug it.

It can also help with picking up other people's projects - all the big providers of AI coding assistants have a /explain option. Probably the biggest thing it can do though, is help teams make the right decision in the first place - outlining your service and its functionality, and getting suggestions from AI about what the best technology to use can take away some of the habitual nature of tech selection that engineers often exhibit.

Challenging existing assumptions

As well as making some of the 'best practices' (or just alternative ways) more do-able, gen AI can start to challenge some of our the existing assumptions and make us more productive by adopting things we currently think of as a bit antiquated now

Don't write comments

Why am I so down on comments? Because they lie. Not always, and not intentionally but too often. The older a comment is, the further it is from the code it described - Clean Code, Uncle Bob

It's difficult to argue with the statement that comments don't get updated and reviewed in the same way as regular code - but one neat feature in intellij AI assistant is to generate doc comments from the function itself. What if every time you saved a file, it checks if your comment is still true and just updates it if not?

This is sort-of useful for most development, it means that you can get some good information about what a function does without reading it - it's really useful for public APIs where you often do that, and it's super useful for documenting UI components and other types of 'interface' with your team.

Functional specs are fantasies

Architectural diagrams, UIs in figma, documentation, threat models, test cases, runbooks and application catalogues - they're all undoubtedly very useful, but the problem is that even the most simple project has multiple versions of the truth. The common approach to this for a long while has been to assume nothing is true, spend onerous amounts of time reconciling it with reality, or trying (valiantly, but ultimately, unsuccessfully) to generate it automatically from code.

We should re-visit these things, we could scan jira tickets, test tickets, videos of demos, designs and look for inaccuracies and update them automatically with gen AI - the real benefit is that the people who need the docs (and they really do sometimes) have them, and people can trust what they're looking at is up-to-date

Putting it all together

So what do you have if you put it all together?

You have a smaller team, capable of doing more tasks within them - your designers upload their hand-drawn sketches to chatGPT and turn it into HTML to test with end-users, they then use documentation automatically generated from components and open API specs to make it functional.

When they're done, the devs can quickly clean it up while the team converts the ideas into tickets without too much hassle.

The devs are programming in languages that are really optimised for the jobs that they are doing, and are living the devops dream - getting AI to help with tests, IaC and all the other benefits of gen AI. They release the code, and it updates architecture diagrams, flags anything that wasn't in the original designs / tickets, highlights any test cases that are missing - and then all the other supporting teams have their docs and user guides ready to roll.

It's probably not the end-state, but it's a good start

What bits of software engineering faff would you want to get rid of?

dllewellyn — Sun, 22 Sep 2024 12:33:00 +0000

I was having to do a bunch of how to videos recently and then to convert them to written tutorials with images I made a little Web app to generate user guides from videos.

It got me thinking though, what other things do devs have to faff with that they'd rather just film and describe ? I had a few ideas:

Generate test cases for features
Release notes
SOPs / Runbooks
Architecture diagram update
Onboarding docs
User manuals

What does everyone else think?