DEV Community: Patrick Loeber

Gemini Embedding 2: Our first natively multimodal embedding model

Patrick Loeber — Tue, 10 Mar 2026 21:01:28 +0000

Today we're releasing Gemini Embedding 2, our first fully multimodal embedding model built on the Gemini architecture, in Public Preview via the Gemini API and Vertex AI.

Expanding on our previous text-only foundation, Gemini Embedding 2 maps text, images, videos, audio and documents into a single, unified embedding space, and captures semantic intent across over 100 languages. This simplifies complex pipelines and enhances a wide variety of multimodal downstream tasks—from Retrieval-Augmented Generation (RAG) and semantic search to sentiment analysis and data clustering.

New modalities and flexible output dimensions

The model is based on Gemini and leverages its best-in-class multimodal understanding capabilities to create high-quality embeddings across:

Text: supports an expansive context of up to 8192 input tokens
Images: capable of processing up to 6 images per request, supporting PNG and JPEG formats
Videos: supports up to 120 seconds of video input in MP4 and MOV formats
Audio: natively ingests and embeds audio data without needing intermediate text transcriptions
Documents: directly embed PDFs up to 6 pages long

Beyond processing one modality at a time, this model natively understands interleaved input so you can pass multiple modalities of input (e.g., image + text) in a single request. This allows the model to capture the complex, nuanced relationships between different media types, unlocking more accurate understanding of complex, real-world data.

Like our previous embedding models, Gemini Embedding 2 incorporates Matryoshka Representation Learning (MRL), a technique that “nests” information by dynamically scaling down dimensions. This enables flexible output dimensions scaling down from the default 3072 so developers can balance performance and storage costs. We recommend using 3072, 1536, 768 dimensions for highest quality.

To see these embeddings in action, try out our lightweight multimodal semantic search demo.

State-of-the-art performance

Gemini Embedding 2 doesn't just improve on legacy models. It establishes a new performance standard for multimodal depth, introducing strong speech capabilities and outperforming leading models in text, image, and video tasks. This measurable improvement and unique multimodal coverage give developers exactly what they need for their diverse embedding needs.

Unlocking deeper meaning for data

Embeddings are the technology that power experiences in many Google products. From RAG where embeddings can play a crucial role in context engineering to large-scale data management and classic search/analysis, some of our early access partners are already using Gemini Embedding 2 to unlock high-value multimodal applications:

Start building today

Get started with the Gemini Embedding 2 model through Gemini API or Vertex AI.

from google import genai
from google.genai import types

# For Vertex AI:
# PROJECT_ID='<add_here>'
# client = genai.Client(vertexai=True, project=PROJECT_ID, location='us-central1')

client = genai.Client()

with open("example.png", "rb") as f:
    image_bytes = f.read()

with open("sample.mp3", "rb") as f:
    audio_bytes = f.read()

# Embed text, image, and audio 
result = client.models.embed_content(
    model="gemini-embedding-2-preview",
    contents=[
        "What is the meaning of life?",
        types.Part.from_bytes(
            data=image_bytes,
            mime_type="image/png",
        ),
        types.Part.from_bytes(
            data=audio_bytes,
            mime_type="audio/mpeg",
        ),
    ],
)

print(result.embeddings)

Learn how to use the model in our interactive Gemini API and Vertex AI Colab notebooks. You can also use it through LangChain, LlamaIndex, Haystack, Weaviate, QDrant, ChromaDB, and Vector Search.

By bringing semantic meaning to the diverse data around us, Gemini Embedding 2 provides the essential multimodal foundation for the next era of advanced AI experiences. We can't wait to see what you build.

Gemini 3.1 Flash-Lite: Developer guide and use cases

Patrick Loeber — Tue, 03 Mar 2026 18:55:00 +0000

Gemini 3.1 Flash-Lite is the high-volume, affordable powerhouse of the Gemini family. It’s purpose-built for large-scale tasks where speed and cost-efficiency are the main priorities, making it the ideal engine for background processing. Whether you're handling a constant stream of user interactions or need to process massive datasets with tasks like translation, transcription, or extraction, Flash-Lite provides the optimal balance of speed and capability.

This guide walks through seven practical use cases for Flash-Lite using the google-genai Python SDK.

Setup

Install the SDK and configure your API key:

# pip install -U google-genai
from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

1. Translation

If you're processing user-generated content at scale, such as chat messages, reviews, or support tickets, you need fast, cheap translation. Flash-Lite handles high-volume translation well, and you can use system instructions to constrain it to output only the translated text with no extra commentary.

text = "Hey, are you down to grab some pizza later? I'm starving!"

response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    config={
        "system_instruction": "Only output the translated text"
    },
    contents=f"Translate the following text to German: {text}"
)

print(response.text)
# Hey, hast du Lust, später eine Pizza essen zu gehen? Ich habe riesigen Hunger!

2. Transcription

Flash-Lite supports multimodal inputs and handles speech-to-text tasks fast and at scale, allowing you to pass audio files such as recordings, memos, or voice inputs directly for transcription. Furthermore, you have the option to leverage prompting in the same step to get the transcript in a specific format, making it ready for downstream tasks like agent hand-offs or other workflows.

# URL = "https://storage.googleapis.com/generativeai-downloads/data/State_of_the_Union_Address_30_January_1961.mp3"

# Upload the audio file to the GenAI File API
uploaded_file = client.files.upload(file="sample.mp3")

prompt = "Generate a transcript of the audio."
# prompt = "Generate a transcript of the audio. Remove filler words such as 'um', 'uh', 'like'."

response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    contents=[prompt, uploaded_file]
)

print(response.text)

3. Lightweight Agentic Tasks and Data Extraction

Flash-Lite supports structured JSON output, which makes it a good fit for entity extraction, classification, and lightweight data processing pipelines. You define your output schema (here using Pydantic) and the model returns valid JSON that conforms to it.

In this example, we extract structured data from an e-commerce customer review, including the specific product aspect mentioned, a summary quote, a sentiment score, and the customer's likelihood of returning.

from pydantic import BaseModel, Field

prompt = "Analyze the user review and determine the aspect, sentiment score, summary quote, and return risk"
input_text = "The boots look amazing and the leather is high quality, but they run way too small. I'm sending them back."

class ReviewAnalysis(BaseModel):
    aspect: str = Field(description="The feature mentioned (e.g., Price, Comfort, Style, Shipping)")
    summary_quote: str = Field(description="The specific phrase from the review about this aspect")
    sentiment_score: int = Field(description="1 to 5 (1=worst, 5=best)")
    is_return_risk: bool = Field(description="True if the user mentions returning the item")

response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    contents=[prompt, input_text],
    config={
        "response_mime_type": "application/json",
        "response_json_schema": ReviewAnalysis.model_json_schema(),
    },
)

print(response.text)
# {
#  "aspect": "Size",
#  "summary_quote": "they run way too small",
#  "sentiment_score": 2,
#  "is_return_risk": true
# }

4. Document Processing & Summarization

Flash-Lite handles high-volume document tasks with ease, from parsing PDFs for concise summaries to performing cross-source comparisons. It is also an ideal fit for document processing pipelines that require quick triage, enabling you to categorize incoming files, run simple pass/fail checks, or perform standard data extraction.

import httpx

# Download PDF document
doc_url = "https://storage.googleapis.com/generativeai-downloads/data/med_gemini.pdf"
doc_data = httpx.get(doc_url).content

prompt = "Summarize this document"
response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    contents=[
        types.Part.from_bytes(
            data=doc_data,
            mime_type='application/pdf',
        ),
        prompt
    ]
)

print(response.text)

5. Model routing

You don't want to send every request to your most expensive model. A common pattern is to use a fast, cheap model as a classifier that routes queries to the appropriate model based on task complexity. Flash-Lite works well for this because the routing call itself needs to be low-latency and low-cost.

A real-world example of this pattern is the open-source Gemini CLI, which uses Flash-Lite to classify task complexity and route to Gemini Flash or Pro. The following example is adapted from the CLI’s classifier strategy.

FLASH_MODEL = 'flash'
PRO_MODEL = 'pro'

CLASSIFIER_SYSTEM_PROMPT = f"""
You are a specialized Task Routing AI. Your sole function is to analyze the user's request and classify its complexity. Choose between `{FLASH_MODEL}` (SIMPLE) or `{PRO_MODEL}` (COMPLEX).
1.  `{FLASH_MODEL}`: A fast, efficient model for simple, well-defined tasks.
2.  `{PRO_MODEL}`: A powerful, advanced model for complex, open-ended, or multi-step tasks.

A task is COMPLEX if it meets ONE OR MORE of the following criteria:
1.  High Operational Complexity (Est. 4+ Steps/Tool Calls)
2.  Strategic Planning and Conceptual Design
3.  High Ambiguity or Large Scope
4.  Deep Debugging and Root Cause Analysis

A task is SIMPLE if it is highly specific, bounded, and has Low Operational Complexity (Est. 1-3 tool calls).
"""

user_input = "I'm getting an error 'Cannot read property 'map' of undefined' when I click the save button. Can you fix it?"

response_schema = {
  "type": "object",
  "properties": {
    "reasoning": {
      "type": "string",
      "description": "A brief, step-by-step explanation for the model choice, referencing the rubric."
    },
    "model_choice": {
      "type": "string",
      "enum": [FLASH_MODEL, PRO_MODEL]
    }
  },
  "required": ["reasoning", "model_choice"]
}

response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    contents=user_input,
    config={
        "system_instruction": CLASSIFIER_SYSTEM_PROMPT,
        "response_mime_type": "application/json",
        "response_json_schema": response_schema
    },
)

print(response.text)
# {
#   "reasoning": "The user is reporting an error symptom without a known cause. This requires investigation to identify the root cause, which falls under 'Deep Debugging & Root Cause Analysis'.",
#   "model_choice": "pro"
# }

6. Thinking with Gemini Flash-Lite

Flash-Lite supports configurable thinking levels, allowing the model to allocate additional compute to internal reasoning before producing a final response. This is ideal for tasks that benefit from step-by-step logic, such as math, coding, or multi-constraint problems, where you need higher accuracy while maintaining the efficiency of the Flash-Lite model. By default, Flash-Lite’s thinking level is set to minimal, but it can be adjusted to low, medium, or high depending on the complexity of your task.

response = client.models.generate_content(
    model="gemini-3.1-flash-lite-preview",
    contents="How does AI work?",
    config={
        "thinking_config": {"thinking_level": "high"}
    },
)

print(response.text)

For more on configuring thinking levels, see the Gemini API docs.

7. Batch API

If you have large volumes of data to process and low latency isn't a priority, the Gemini Batch API is the perfect companion for Flash-Lite. It is designed specifically for asynchronous, high-throughput tasks at 50% of the standard cost. The target turnaround time is 24 hours, but in the majority of cases, it is much quicker.

You can implement the Batch API in your workflow using the following pattern:

# Create a JSONL file with your requests and upload it
uploaded_batch_requests = client.files.upload(file="batch_requests.json")

# Create the batch job
batch_job = client.batches.create(
    model="gemini-3.1-flash-lite-preview",
    src=uploaded_batch_requests.name,
    config={'display_name': "batch_job-1"}
)

print(f"Created batch job: {batch_job.name}")

# Wait for up to 24 hours 
if batch_job.state.name == 'JOB_STATE_SUCCEEDED':
    result_file_name = batch_job.dest.file_name
    file_content_bytes = client.files.download(file=result_file_name)
    file_content = file_content_bytes.decode('utf-8')

    for line in file_content.splitlines():
        print(line)

Conclusion

Gemini 3.1 Flash-Lite excels at the "boring but big" tasks that define high-scale production. It serves as a versatile workhorse for everything from data extraction to agentic routing, enabling you to build more balanced and efficient AI architectures. By leveraging Flash-Lite for high-volume background processing, you can maximize your impact while keeping operational costs in check.

See the following resources to learn more:

How to build with Nano Banana: Complete Developer Tutorial

Patrick Loeber — Thu, 04 Sep 2025 17:24:31 +0000

Google has recently released Gemini 2.5 Flash Image, a powerful new model for image generation and editing, also known by its codename, Nano Banana. This model introduces state-of-the-art capabilities for creating and manipulating images, unlocking a wide range of new applications.

This guide provides a comprehensive walkthrough for developers looking to integrate Gemini 2.5 Flash Image aka Nano Banana into their applications using the Gemini Developer API.

This guide will cover:

Using Nano Banana in AI Studio
Project setup
Image creation
Image editing
Photo restoration
Multiple input images
Conversational image editing
Aspect ratios
Image-only outputs
Best practices and effective prompting
Community examples and inspiration
Resources

Here's an example of what you'll build in this tutorial:

prompt = "Restore and colorize this image from 1932"

response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=[prompt, image],
)

Let's get started!

If you'd prefer a video version of this post, you can watch it here:

Update Nov 2025: Nano Banana Pro, a higher-fidelity model for studio-quality image generation is now available. Learn how to build with it in the Nano Banana Pro Developer Tutorial.

1) Using Nano Banana in Google AI Studio

While end-users can access Nano Banana in the Gemini app, the best environment for developers to prototype and test prompts is Google AI Studio. AI Studio is a playground to experiment with all available AI models before writing any code, and it's also the entry point for building with the Gemini API.

You can use Nano Banana free of charge within AI Studio. To get started, go to aistudio.google.com, sign in with your Google account, and select Nano Banana from the model picker.

For direct access, use this link to start a new session with the model:

ai.studio/banana

Tip: You can also vibe code Nano Banana web apps directly in AI Studio at ai.studio/apps, or explore the code and remix one of the existing apps.

2) Project setup

To follow this guide, you will need the following:

An API key from Google AI Studio.
Billing set up for your project.
The Google Gen AI SDK for Python or JavaScript/TypeScript.

Step A: Generate an API Key

Follow these steps:

In Google AI Studio, click Get API key in the left navigation panel.
On the next page, click Create API key.
Select an existing Google Cloud project or create a new one. This project is used to manage billing for API usage.

Once the process is complete, your API key will be displayed. Copy and store it securely.

Step B: Enable Billing

While prototyping in AI Studio is free, using the model via the API is a paid service. You must enable billing on your Google Cloud project.

In the API key management screen, click Set up billing next to your project and follow the on-screen instructions.

How much does Nano Banana cost?

Image generation with Nano Banana costs $0.039 per image *. For $1, you can generate approximately 25 images.

* The official pricing is $0.30/1M input tokens and $30/1M output tokens. A standard 1024x1024px output image consumes 1290 tokens, which equates to $0.039 per image. For details, refer to the Gemini 2.5 Flash Image pricing table.

Step C: Install the SDK

Choose the SDK for your preferred language.

Python:

pip install -U google-genai
# Install the Pillow library for image manipulation
pip install Pillow

JavaScript / TypeScript:

npm install @google/genai

The following examples use the Python SDK for demonstration. Equivalent code snippets to use Nano Banana in JavaScript are provided in this GitHub Gist.

3) Image Generation from Text

Use Nano Banana to generate one or more images from a descriptive text prompt. Use the model ID gemini-2.5-flash-image for all API requests.

from google import genai
from PIL import Image
from io import BytesIO

# Configure the client with your API key
client = genai.Client(api_key="YOUR_API_KEY")

prompt = """Create a photorealistic image of an orange cat
with a green eyes, sitting on a couch."""

# Call the API to generate content
response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=prompt,
)

# The response can contain both text and image data.
# Iterate through the parts to find and save the image.
for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("cat.png")

Output:

The model is multimodal, so the response is structured as a list of parts that can contain interleaved text and image data (inline_data). The code above iterates through these parts to extract and save the generated image.

4) Image Editing with Text and Image Inputs

Provide an existing image along with a text prompt to perform edits. The model excels at maintaining character and content consistency from the input image.

from google import genai
from PIL import Image
from io import BytesIO

client = genai.Client(api_key="YOUR_API_KEY")

prompt = """Using the image of the cat, create a photorealistic,
street-level view of the cat walking along a sidewalk in a
New York City neighborhood, with the blurred legs of pedestrians
and yellow cabs passing by in the background."""

image = Image.open("cat.png")

# Pass both the text prompt and the image in the 'contents' list
response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=[prompt, image],
)

for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("cat2.png")

Input and Output:

5) Photo restoration with Nano Banana

One of the model's powerful applications is photo restoration. With a simple prompt, it can restore and colorize old photographs with impressive results.

from google import genai
from PIL import Image
from io import BytesIO

client = genai.Client(api_key="YOUR_API_KEY")

prompt = "Restore and colorize this image from 1932"

image = Image.open("lunch.jpg")  # "Lunch atop a Skyscraper, 1932"

response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=[prompt, image],
)

for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("lunch-restored.png")

Original and Output:

6) Working with Multiple Input Images

You can provide multiple images as input for more complex editing tasks.

from google import genai
from PIL import Image
from io import BytesIO

client = genai.Client(api_key="YOUR_API_KEY")

prompt = "Make the girl wear this t-shirt. Leave the background unchanged."

image1 = Image.open("girl.png")
image2 = Image.open("tshirt.png")

response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=[prompt, image1, image2],
)

for part in response.candidates[0].content.parts:
    if part.text is not None:
        print(part.text)
    elif part.inline_data is not None:
        image = Image.open(BytesIO(part.inline_data.data))
        image.save("girl-with-tshirt.png")

Inputs 1 & 2 and Output:

7) Conversational Image Editing

For iterative refinement, you can use a chats session to maintain context across multiple requests. This allows you to edit images conversationally.

from google import genai
from PIL import Image
from io import BytesIO

client = genai.Client(api_key="YOUR_API_KEY")

# Create a chat
chat = client.chats.create(
    model="gemini-2.5-flash-image"
)

# Make the first image edit
response1 = chat.send_message(
    [
        "Change the cat to a bengal cat, leave everything else the same",
        Image.open("cat.png"),
    ]
)
# display / save image...

# Continue chatting and editing
response2 = chat.send_message("The cat should wear a funny party hat")
# display / save image...

Input and Outputs 1 & 2:

Tip: If you notice image features starting to degrade or "drift" after many conversational edits, it's best to start a new session with the latest image and a more detailed, consolidated prompt to maintain high fidelity.

8) Aspect ratios

You can control the aspect ratio of the output image using the aspect_ratio field under image_config in the request. You can find all supported aspect ratios in the docs.

from google import genai

client = genai.Client(api_key="YOUR_API_KEY")

prompt = """Create a photorealistic image of an orange cat
with a green eyes, sitting on a couch."""

response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=prompt,
    config={"image_config": {"aspect_ratio": "16:9"}},
)

Output:

If no aspect ratio is specified, the model defaults to matching the output image size to that of your input image, or otherwise generates 1:1 squares.

9) Image-only outputs

You can configure the response to return only images without text by setting the response_modalities config to ["Image"].

response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=prompt,
    config={"response_modalities": ["Image"]},
)

10) Best Practices and prompting tips for Nano Banana

To achieve the best results with Nano Banana, follow these prompting guidelines:

Be Hyper-Specific: The more detail you provide about subjects, colors, lighting, and composition, the more control you have over the output.
Provide Context and Intent: Explain the purpose or desired mood of the image. The model's understanding of context will influence its creative choices.
Iterate and Refine: Don't expect perfection on the first try. Use the model's conversational ability to make incremental changes and refine your image.
Use Step-by-Step Instructions: For complex scenes, break your prompt into a series of clear, sequential instructions.
Use Positive Framing: Instead of negative prompts like "no cars," describe the desired scene positively: "an empty, deserted street with no signs of traffic."
Control the Camera: Use photographic and cinematic terms to direct the composition, such as "wide-angle shot", "macro shot", or "low-angle perspective".

For a deeper dive into best practices, review the official blog post on prompting best practices and the prompting guide in the documentation.

11) Community Examples and Inspiration

Explore what the community is building with Nano Banana:

Shifting camera perspective by @henrydaubrez: X post
Few-shot learning for consistent character design by @multimodalart: X post
"What does the red arrow see" Google Maps transforms by @tokumin: X post
Generating images from stick figure annotations by @yachimat_manga: X post
Creating 3D models from still images by @deedydas: X post
Generating location-based AR experiences by @bilawalsidhu: X post
Converting a 2D map into a 3D graphic by @demishassabis: X post

12) Resources and Next Steps

This guide has covered the fundamentals of building with Nano Banana aka Gemini 2.5 Flash Image. You've learned how to set up your environment, generate and edit images, and apply advanced techniques. Now you're ready to start incorporating these powerful capabilities into your own projects.

For further reading, check out the official resources:

If you're building something cool with this, I'd love to see it! Feel free to DM or tag me on X: @patloeber.

My New Job as Developer Advocate!

Patrick Loeber — Fri, 18 Feb 2022 08:20:15 +0000

I'm incredibly excited to share that in February 2022, I joined AssemblyAI full-time as a Developer Advocate!

About AssemblyAI

AssemblyAI is a Deep Learning startup focused on a state-of-the-art Speech-To-Text API and additional Audio Intelligence features. One critical point for us is to make this technology accessible to everyday developers through a simple API and a great developer experience.

Last October, I created a video where I played around with the API for an automation project, and the transcription accuracy really surprised me.

Small promotion 😉: If you also want to play around with it, you can signup and get started FOR FREE.

So in the last couple of months, I got more involved with this technology, met with their team, and even created a few videos for their YouTube channel on the side. In the end, I was offered an exciting position that I couldn't turn down.

Developer Advocate role

What does a Developer Advocate do? How this role is interpreted can actually vary a lot between companies. My Twitter friend Gus summarized the different possible responsibilities pretty nicely:

🚀 Raise awareness for the brand/product
🖊️ Produce content/documentation
🤝 Engage w/ the Community
📢 Organize/Present on events
⌨️ Create Samples/Demos
🤓 Answer questions on SO/Forums
🤕 Be user Zero
👂🏾 Gather feedback
🗣️ Suggest improvements

In my case, I will be focusing on creating video content and blog posts, educating people about our product and also about general Machine Learning / Deep Learning topics, and growing a community of interested people in this field.

Since creating videos for the AssemblyAI YouTube channel the subscribers have been grown from almost 0 to 1500! We create beginner friendly weekly videos for you around ML / DL related topics. So if you want to learn about this, I would be very happy if you checked it out and subscribed!

If you are wondering how this affects my own channel, you don't have to worry! I don't have any restrictions, and I will still create regular content for myself!

First 2 weeks

The first two weeks have been amazing! I love the product, and the whole team is really nice and helpful. And I am given a lot of freedom in my content creation process, which allows me to create the best possible educational material.

I can already tell that being part of a startup from the US feels a lot different from every role I had before in Germany. There are no slow company processes, all teams are moving super fast, and you can feel the high energy everywhere!

If this sounds interesting, check out our 100% fully remote roles. This year is going to be big, and we are currently hiring like crazy!

10 Deep Learning Projects (Beginner & Advanced)

Patrick Loeber — Sun, 13 Jun 2021 12:41:15 +0000

Here are 10 deep learning projects from beginner to advanced that you can do with TensorFlow or PyTorch. For each project the links to the datasets are included.

1 MNIST

The MNIST dataset is a large set of handwritten digits and the goal is to recognize the correct digit. This project is fairly easy, it should make you comfortable with your deep learning framework and you should learn how you can implement and train your first Artificial Neural Network. It also teaches you how to do multiclass classification problems instead of just binary problems.

MNIST can be loaded directly from within TensorFlow and PyTorch.

http://yann.lecun.com/exdb/mnist/

2 CIFAR-10

This project is similar but a little bit more difficult than the first one. It contains color images of 10 different classes like airplanes, birds, dogs, and other objects. Here it’s a little bit harder to get a good classification model. Now instead of just using a simple neural net, you should implement a Convolutional Neural Net and learn how they work.

CIFAR-10 can be loaded directly from within TensorFlow and PyTorch.

https://www.cs.toronto.edu/~kriz/cifar.html

3 Dogs vs. Cats

The third project is the Dogs vs. Cats challenge on Kaggle. As the name suggests, the dataset only contains images of either a dog or a cat. This classification task is actually a little bit simpler than in the previous task, because now we only deal with a binary classification problem. But the challenging part could be to learn how to download the data and load it with the correct format into your model. If you are ambitious you can then submit your results to Kaggle and compete with other people.

To get a really good performance you could also have a look at a technique that is called Transfer Learning. This is a very important concept that you should learn sooner or later, so now would be a good point to try this. If you want to learn more about this then I have a tutorial for you here.

Dogs vs. Cats | Kaggle

4 Breast Cancer Classification

The medical field is one of the most common use cases of deep learning. There are many applications out there that help to detect diseases and help physicians to make their diagnosis. Here you can help to improve these applications and bring your knowledge to a good use.

The particular project I selected for beginners is about Breast Cancer Classification. Here you have to train a model to classify cancer subtypes based on 2D Medical Histopathology images. Breast cancer is the most common form of cancer in women, and accurately identifying and categorizing breast cancer subtypes is an important clinical task. If you can come up with a reliable automated method here then this can be used to save time and reduce errors in hospitals.

Breast Cancer Classification: Breast Histopathology Images | Kaggle

5 Natural Language Processing with Disaster Tweets

Up until now we had four computer vision projects. Now let’s switch the field and have a look at Natural Language Processing - or short NLP. This is another field where deep learning is widely used. Here we don’t deal with images but instead with words and sentences.

To get started I recommend the Disaster Tweet project. Again you find this on Kaggle in the NLP getting started category. You have to classify Twitter Tweets and predict if they are about real disasters or not.

This would be a nice time to learn about RNNs - Recurrent Neural Networks, and LSTMs - Long Short Term Memory. These are two special types of neural networks that are extremely important when working with text data. You can find a tutorial about them here.

Natural Language Processing with Disaster Tweets | Kaggle

6 Chatbot

Next I suggest a project I think almost everyone will enjoy. And this is about chatbots. Build your own chatbot from scratch and put it to test with a simple chat application.

To get data for this task I can point you to two large open source datasets which should be enough for the beginning.
The first is the Conversational Question Answering dataset provided by Stanford NLP, and the other one is the Google Natural Questions dataset.

If you don’t know how to get started then I can point you to my tutorial where we build a simple chatbot with an RNN together. Once you've understood the concepts of RNNs then creating a - maybe not advanced but decent - chatbot is not that hard anymore.

Google’s Natural Questions
CoQA: A Conversational Question Answering Challenge

7 Recommender System

Now let’s go to a task almost every company needs. Have a look at Netflix, YouTube, Instagram, Spotify, and all the other big names. They all need Recommender Systems. Based on the information they collect on each user they want to recommend other content that the user might enjoy.

To get started with this I suggest to build a movie recommender system, You can either use the MovieLens 100K Dataset or the official Netflix dataset on Kaggle.

This is also a good time to learn about a technique that is called collaborative filtering. You could solve this with „classical“ Data Science techniques, but you can also build deep recommender systems using deep learning.

MovieLens 100K Dataset
Netflix Prize data | Kaggle

8 Forecasting

Next let’s have a look at Forecasting. This is another interesting field where we deal with a time series and you can practice your knowledge about RNNs again. We want to predict the values of a time series in the future. A very popular example for this is stock price prediction.

As dataset here I actually encourage you to scrape or download the stock data yourself from Yahoo! Finance. This should not be too hard and there is also a python package (yfinance) that you can simply use.

So get some stock data, use the time data only up to a certain point in the past to train your model, then see how it predicts the prices from the rest of the data up to the present time, and then build a system to predict the prices in the future.

Yahoo! finance
yfinance Python Package

9 Object Detection

The last two projects are advanced Computer Vision tasks. First let’s have a look at Object Detection. The goal is to identify the specified objects and mark the positions in the image. So you have to check if there is an object, and then where it is, and also deal with possible multiple objects in an image.

This is indeed a very advanced task, and you could try to recreate the popular YOLO object detection model from scratch, but I recommend to just use a pertained model.

Then you still have to implement the whole object detection pipeline, and you should learn about OpenCV here. A very important Computer Vision library that is used here for example to draw the bounding boxes.

As datasets I can point you to the Raccoon dataset or the Annotated Driving Dataset that is used for self driving cars.

Raccoon Dataset
Annotated Driving Dataset

Helpful Articles:

Towardsdatascience - Build Your Own Object Detector
Towardsdatascience - Object Detection With TensorFlow Object Detector API

10 Style Transfer

As last project I suggest to have a look at style transfer, a very interesting use of deep learning. You train a model and can then feed a style image to this model, and after training it is able to apply this style to any other given image you want.

Here again you don’t have to implement this from scratch but can use an existing framework like the TensorFlow fast style transfer or the PyTorch fast neural style implementation.

To retrain your model for your own style images you should use the COCO dataset. COCO is a large-scale object detection, segmentation, and captioning dataset, and it’s one of the most important deep learning datasets for computer vision that you should definitely check out!

TensorFlow Fast Style Transfer
PyTorch Fast Neural Style

COCO dataset

Final Words

I hope you will enjoy these projects! And if you need help you can always join our community in the Discord server: https://discord.gg/FHMg9tKFSN

Machine Learning From Scratch in Python - Full Course [FREE]

Patrick Loeber — Sat, 30 Jan 2021 15:35:23 +0000

In this course we implement the most popular Machine Learning algorithms from scratch using pure Python and NumPy.

By the end of this course, you will have a deep understanding of the concepts behind those algorithms.

The course is available here:

Each part starts with a short theory section that explains the math and concepts behind the algorithm. Then we jump to the code and implement it in a clean, object oriented style. You will get a hands-on experience with Machine Learning algorithms and feel more confident using them in your own projects.

Prerequisites

Beginner Python skills and a little bit of math knowledge (Linear Algebra, Differential Equations) is required to follow the course. NumPy knowledge can be benefitial but is not a must. If you want to have a quick refresher, you can checkout my free NumPy handbook that covers all essential functions. You can get it here.

Note

This is a collection of my ML From Scratch playlist compiled into one single video. The code for all algorithms is available on GitHub. The jupyter notebooks are available on Patreon.

Course Overview

1) KNN
2) Linear Regression
3) Logistic Regression
4) Regression Refactoring
5) Naive Bayes
6) Perceptron
7) SVM
8) Decision Tree Part 1
9) Decision Tree Part 2
10) Random Forest
11) PCA
12) K-Means
13) AdaBoost
14) LDA
15) Load Data From CSV

5 Machine Learning BEGINNER Projects (+ Datasets & Solutions)

Patrick Loeber — Mon, 19 Oct 2020 11:46:29 +0000

I this tutorial I share 5 Beginner Machine Learning projects with you and give you tips how to solve all of them. These projects are for complete beginners and should teach you some basic machine learning concepts. With each project the difficulty increases a little bit and you'll learn a new algorithm.

For each project I give you an algorithm that you can use and include the links to the datasets, so you can start right away!

For all those projects I recommend to use the scikit-learn library. This is the go-to library in Python when it comes to machine learning. It's incredibly easy to get started with this library and to implement your own Machine Learning algorithms with it.

Regression vs. Classification

Before we go over the projects you should know about the 2 basic types of machine learning tasks: Regression vs. Classification.

Fundamentally, classification is about predicting a label, so a concrete class value while regression is about predicting a quantity, so a continuous value.

Project 1

As first project I recommend to start with a regression problem. For this problem I recommend to do actually 2 projects. One is a super simple project to predict the salary based on the number of years of experience. This only contains 2 variables, so you stay in 2 dimensions and this should give you a good understanding of how the model works. After that I recommend to do the Boston Housing dataset. Here you should predict the price of a home based on multiple different variables. The algorithm you should use here is the so-called Linear Regression model. This is one of the easiest algorithms and shouldn't be too hard to understand.

Datasets

Algorithm

Linear Regression

Project 2

After that I recommend to tackle your first classification problem. The dataset is the Iris dataset. This is probably the most famous dataset in the world of machine learning, and everyone should have solved it at least once. Here we have samples from 3 different flower species, and for each sample we have 4 different features that describe the flower. With this information we want to predict the species of the flower then. As algorithm I recommend to use the K Nearest Neighbor (KNN) algorithm. This is one of the simplest classification algorithms but works pretty well here. The species are very clearly distinguishable, so you should be able to train a good KNN model and reach 100% correct predictions.

I know everyone is using the Iris dataset as first example, so if you cannot see it anymore and want to have an alternative then you can check out the Penguin dataset where we want to predict the species of a penguin based on certain features.

Datasets

Algorithm

K Nearest Neighbor

Project 3

Next, I recommend to use the Breast Cancer dataset. This is another famous dataset with the interesting task to predict if a cancer cell is good or bad (or in medical terms: malignant or benign). Here we have 30 different features for each cancer cell that have been computed from medical images. This is certainly more complex and more difficult than the project before, but still you should be able to reach an accuracy of 95% here. As algorithm I recommend to try out the Logistic Regression model. This is similar to the Linear Regression model in the beginning. Don't be confused by the name, because even though it has Regression in its name, it is actually used for a classification task. The Logistic Regression algorithm also models a continuous value, but this is a probability value between 0 and 1 and can therefore be used for classification. I also recommend to have a look at another new technique that is called feature standardization. Because the 30 different features may have values in different ranges, and this might confuse the model a little bit. So play around with feature standardization here and see if you can improve your model even further with that. (Note: Feature standardization is not required for Logistic Regression, but it's still an important technique and can be important for other classifier here.)

Dataset

https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29

Algorithm

Project 4

The fourth project is interesting because it is implemented in everyones email client. Here we want to create a spam filter based on the Spambase dataset. In this dataset we have the frequency of different words and characters, so we calculate the total number of appearances of each word and divide it by the total number of words in the email. Spam emails clearly show certain key words more often than normal mails, so with this information we are able to create a spam classifier. As algorithm I recommend to have a look at the Naive Bayes algorithm here. The new challenge here is then not only to use this dataset and evaluate your model, but then after you have trained your classifier also apply it to a real application. So what do you do with a new email? What do you have to do before you pass it to the classifier? Here you somehow have to find out how to transform the text from the email to the same format that your classifier expects. This should give you a better understanding of how datasets are shaped and created.

Dataset

https://archive.ics.uci.edu/ml/datasets/spambase

Algorithm

Naive Bayes

Project 5

The last project I recommend is the Titanic dataset. This is the first beginner project that Kaggle recommends on their site in the Getting Started section. Here we have a list of all Titanic passengers with certain features like the age, the name, or the sex of the person, and we want to predict if this passenger survived or not. The Titanic dataset requires a little more work before we can use it, because not all information in this dataset are useful and we even have missing values. So here you should learn some preprocessing techniques and how to visualise, analyze, and clean the data. Up to this point we could use the datasets right away, but in real world applications this is actually almost never the case, so you should definitely learn how to analyze datasets. As algorithm I recommend to have a look at Decision Trees, and also at a second algorithm, the Random Forest algorithm, which extends decision trees. As another tip i recommend to have a look at the pandas library here. This makes your life a lot easier when it comes to data visualisation and processing the data beforehand.

Dataset

https://www.kaggle.com/c/titanic/data

Algorithm

Conclusion

If you complete all projects you should have a good understanding of 6 popular machine learning algorithms, and you should also have a feeling for different datasets and some knowledge of how to analyze and process the data.

11 Tips And Tricks To Write Better Python Code

Patrick Loeber — Tue, 28 Jul 2020 07:39:02 +0000

You can read the original article on my website:
https://www.python-engineer.com/posts/11-tips-to-write-better-python-code/

In this tutorial I show 11 Tips and Tricks to write better Python code! I show a lot of best practices that improve your code by making your code much cleaner and more Pythonic. Here's the overview of all the tips:

1) Iterate with enumerate() instead of range(len())
2) Use list comprehension instead of raw for-loops
3) Sort complex iterables with the built-in sorted() method
4) Store unique values with Sets
5) Save Memory With Generators
6) Define default values in Dictionaries with .get() and .setdefault()
7) Count hashable objects with collections.Counter
8) Format Strings with f-Strings (Python 3.6+)
9) Concatenate Strings with .join()
10) Merge dictionaries with the double asterisk syntax ** (Python 3.5+)
11) Simplify if-statements with if x in list instead of checking each item separately

1) Iterate with `enumerate()` instead of `range(len())`

If we need to iterate over a list and need to track both the index and the current item, most people would use the range(len) syntax. In this example we want to iterate over a list, check if the current item is negative, and set the value in our list to 0 in this case. While the range(len) syntax works it's much nicer to use the built-in enumerate function here. This returns both the current index and the current item as a tuple. So we can directly check the value here and also access the item with the index.

data = [1, 2, -3, -4]
# weak:
for i in range(len(data)):
    if data[i] < 0:
        data[i] = 0

# better:
data = [1, 2, -3, -4]
for idx, num in enumerate(data):
    if num < 0:
        data[idx] = 0

2) Use list comprehension instead of raw for-loops

Let's say we want to create a list with certain values, in this case a list with all the squared numbers between 0 and 9. The tedious way would be to create an empty list, then use a for loop, do our calculation, and append it to the list:

squares = []
for i in range(10):
    squares.append(i*i)

A simpler way to do this is list comprehension. Here we only need one line to achieve the same thing:

# better:
squares = [i*i for i in range(10)]

List comprehension can be really powerful, and even include if-statements. If you want to learn more about the syntax and good use cases, I have a whole tutorial about list comprehension here. Note that the usage of list comprehension is a little bit debatable. It should not be overused, especially not if it impairs the readability of the code. But I personally think this syntax is clear and concise.

3) Sort complex iterables with the built-in `sorted()` method

If we need to sort some iterable, e.g., a list, a tuple, or a dictionary, we don't need to implement the sorting algorithm ourselves. We can simply use the built-in sorted function. This automatically sorts the numbers in ascending order and returns a new list. If we want to have the result in descending order, we can use the argument reverse=True. As I said, this works on any iterable, so here we could also use a tuple. But note that the result is a list again!

data = (3, 5, 1, 10, 9)
sorted_data = sorted(data, reverse=True) # [10, 9, 5, 3, 1]

Now let's say we have a complex iterable. Here a list, and inside the list we have dictionaries, and we want to sort the list according to the age in the dictionary. For this we can also use the sorted function and then pass in the key argument that should be used for sorting. The key must be a function, so here we can use a lambda and use a one line function that returns the age.

data = [{"name": "Max", "age": 6}, 
        {"name": "Lisa", "age": 20}, 
        {"name": "Ben", "age": 9}
        ]
sorted_data = sorted(data, key=lambda x: x["age"])

4) Store unique values with Sets

If we have a list with multiple values and need to have only unique values, a nice trick is to convert our list to a set. A Set is an unordered collection data type that has no duplicate elements, so in this case it removes all the duplicates.

my_list = [1,2,3,4,5,6,7,7,7]
my_set = set(my_list) # removes duplicates

If we already know that we want unique elements, like here the prime numbers, we can create a set right away with curly braces. This allows Python to make some internal optimizations, and it also has some handy methods for calculating the intersections and differences between two sets.

primes = {2,3,5,7,11,13,17,19}

5) Save Memory With Generators

In tip #2 I showed you list comprehension. But a list is not always the best choice. Let's say we have a very large list with 10000 items and we want to calculate the sum over all the items. We can of course do this with a list, but we might run into memory issues. This is a perfect example where we can use generators. Similar to list comprehension we can use generator comprehension that has the same syntax but with parenthesis instead of square brackets. A generator computes our elements lazily, i.e., it produces only one item at a time and only when asked for it. If we calculate the sum over this generator, we see that we get the same correct result.

# list comprehension
my_list = [i for i in range(10000)]
print(sum(my_list)) # 49995000

# generator comprehension
my_gen = (i for i in range(10000))
print(sum(my_gen)) # 49995000

Now let's inspect the size of both the list and the generator with the built-in sys.getsizeof() method. For the list we get over 80000 bytes and for the generator we only get approximately 128 bytes because it only generates one item at a time. This can make a huge difference when working with large data, so it's always good to keep the generator in mind!

import sys 

my_list = [i for i in range(10000)]
print(sys.getsizeof(my_list), 'bytes') # 87616 bytes

my_gen = (i for i in range(10000))
print(sys.getsizeof(my_gen), 'bytes') # 128 bytes

6) Define default values in Dictionaries with `.get()` and `.setdefault()`

Let's say we have a dictionary with different keys like the item and the price of the item. At some point in our code we want to get the count of the items and we assume that this key is also contained in the dictionary. When we simply try to access the key, it will crash our code and raise a KeyError. So a better way is to use the .get() method on the dictionary. This also returns the value for the key, but it will not raise a KeyError if the key is not available. Instead it returns the default value that we specified, or None if we didn't specify it.

my_dict = {'item': 'football', 'price': 10.00}
price = my_dict['count'] # KeyError!

# better:
price = my_dict.get('count', 0) # optional default value

If we want to ask our dictionary for the count and we also want to update the dictionary and put the count into the dictionary if it's not available, we can use the .setdefault() method. This returns the default value that we specified, and the next time we check the dictionary the used key is now available in our dictionary.

count = my_dict.setdefault('count', 0)
print(count) # 0
print(my_dict) # {'item': 'football', 'price': 10.00, 'count': 0}

7) Count hashable objects with `collections.Counter`

If we need to count the number of elements in a list, there is a very handy tool in the collections module that does exactly this. We just need to import the Counter from collections, and then create our counter object with the list as argument. If we print this, then for each item in our list we see the according number of times that this item appears, and it's also already sorted with the most common item being in front. This is much nicer to calculate it on our own. If we the want to get the count for a certain item, we can simply access this item, and it will return the corresponding count. If the item is not included, then it returns 0.

from collections import Counter

my_list = [10, 10, 10, 5, 5, 2, 9, 9, 9, 9, 9, 9]
counter = Counter(my_list)

print(counter) # Counter({9: 6, 10: 3, 5: 2, 2: 1})
print(counter[10]) # 3

It also has a very handy method to return the most common items, which - no surprise - is called most_common(). We can specify if we just want the very most common item, or also the second most and so on by passing in a number. Note that this returns a list of tuples. Each tuple has the value as first value and the count as second value. So if we just want to have the value of the very most common item, we call this method and then we access index 0 in our list (this returns the first tuple) and then again access index 0 to get the value.

from collections import Counter

my_list = [10, 10, 10, 5, 5, 2, 9, 9, 9, 9, 9, 9]
counter = Counter(my_list)

most_common = counter.most_common(2)
print(most_common) # [(9, 6), (10, 3)]
print(most_common[0]) # (9, 6)
print(most_common[0][0]) # 9

8) Format Strings with f-Strings (Python 3.6+)

This is new since Python 3.6 and in my opinion is the best way to format a string. We just have to write an f before our string, and then inside the string we can use curly braces and access variables. This is much simpler and more concise compared to the old formatting rules, and it's also faster. Moreover, we can write expressions in the braces that are evaluated at runtime. So here for example we want to print the squared number of our variable i, and we can simply write this operation in our f-String.

name = "Alex"
my_string = f"Hello {name}"
print(my_string) # Hello Alex

i = 10
print(f"{i} squared is {i*i}") # 10 squared is 100

9) Concatenate Strings with `.join()`

Let's say we have a list with different strings, and we want to combine all elements to one string, separated by a space between each word. The bad way is to do it like this:

list_of_strings = ["Hello", "my", "friend"]

# BAD:
my_string = ""
for i in list_of_strings:
    my_string += i + " "

We defined an empty string, then iterated over the list, and then appended the word and a space to the string. As you should know, a string is an immutable element, so here we have to create new strings each time. This code can be very slow for large lists, so you should immediately forget this approach! Much better, much faster, and also much more concise is to the .join() method:

# GOOD:
list_of_strings = ["Hello", "my", "friend"]
my_string = " ".join(list_of_strings)

This combines all the elements into one string and uses the string in the beginning as a separator. So here we use a string with only a space. If we were for example to use a comma here, then the final string has a comma between each word. This syntax is the recommended way to combine a list of strings into one string.

10) Merge dictionaries with the double asterisk syntax ** (Python 3.5+)

This syntax is new since Python 3.5. If we have two dictionaries and want to merge them, we can use curly braces and double asterisks for both dictionaries. So here dictionary 1 has a name and an age, and dictionary 2 also has the name and then the city. After merging with this concise syntax our final dictionary has all 3 keys in it.

d1 = {'name': 'Alex', 'age': 25}
d2 = {'name': 'Alex', 'city': 'New York'}
merged_dict = {**d1, **d2}
print(merged_dict) # {'name': 'Alex', 'age': 25, 'city': 'New York'}

11) Simplify if-statements with `if x in list` instead of checking each item separately

Let's say we have a list with main colors red, green, and blue. And somewhere in our code we have a new variable that contains some color, so here c = red. Then we want to check if this is a color from our main colors. We could of course check this against each item in our list like so:

colors = ["red", "green", "blue"]

c = "red"

# cumbersome and error-prone
if c == "red" or c == "green" or c == "blue":
    print("is main color")

But this can become very cumbersome, and we can easily make mistakes, for example if we have a typo here for red. Much simpler and much better is just to use the syntax if x in list:

colors = ["red", "green", "blue"]

c = "red"

# better:
if c in colors:
    print("is main color")

Conclusion

I hope you enjoyed those tips and learned a few new things! If you have any feedback or other tips you can recommend, please reach out on Twitter or YouTube!

DEV Community: Patrick Loeber

Gemini Embedding 2: Our first natively multimodal embedding model

New modalities and flexible output dimensions

State-of-the-art performance

Unlocking deeper meaning for data

Start building today

Gemini 3.1 Flash-Lite: Developer guide and use cases

Setup

1. Translation

2. Transcription

3. Lightweight Agentic Tasks and Data Extraction

4. Document Processing & Summarization

5. Model routing

6. Thinking with Gemini Flash-Lite

7. Batch API

Conclusion

How to build with Nano Banana: Complete Developer Tutorial

1) Using Nano Banana in Google AI Studio

2) Project setup

Step A: Generate an API Key

Step B: Enable Billing

How much does Nano Banana cost?

Step C: Install the SDK

3) Image Generation from Text

4) Image Editing with Text and Image Inputs

5) Photo restoration with Nano Banana

6) Working with Multiple Input Images

7) Conversational Image Editing

8) Aspect ratios

9) Image-only outputs

10) Best Practices and prompting tips for Nano Banana

11) Community Examples and Inspiration

12) Resources and Next Steps

My New Job as Developer Advocate!

About AssemblyAI

Developer Advocate role

First 2 weeks

10 Deep Learning Projects (Beginner & Advanced)

1 MNIST

2 CIFAR-10

3 Dogs vs. Cats

4 Breast Cancer Classification

5 Natural Language Processing with Disaster Tweets

6 Chatbot

7 Recommender System

8 Forecasting

9 Object Detection

10 Style Transfer

Final Words

Machine Learning From Scratch in Python - Full Course [FREE]

Prerequisites

Note

Course Overview

5 Machine Learning BEGINNER Projects (+ Datasets & Solutions)

Regression vs. Classification

Project 1

Datasets

Algorithm

Project 2

Datasets

Algorithm

Project 3

Dataset

Algorithm

Project 4

Dataset

Algorithm

Project 5

Dataset

Algorithm

Conclusion

11 Tips And Tricks To Write Better Python Code

1) Iterate with enumerate() instead of range(len())

2) Use list comprehension instead of raw for-loops

3) Sort complex iterables with the built-in sorted() method

4) Store unique values with Sets

5) Save Memory With Generators

6) Define default values in Dictionaries with .get() and .setdefault()

7) Count hashable objects with collections.Counter

8) Format Strings with f-Strings (Python 3.6+)

1) Iterate with `enumerate()` instead of `range(len())`

3) Sort complex iterables with the built-in `sorted()` method

6) Define default values in Dictionaries with `.get()` and `.setdefault()`

7) Count hashable objects with `collections.Counter`

9) Concatenate Strings with `.join()`

11) Simplify if-statements with `if x in list` instead of checking each item separately