DEV Community: Hilman Ramadhan

Connect Sakana AI (Fugu) with Web Search API

Hilman Ramadhan — Thu, 30 Jul 2026 02:50:51 +0000

Sakana AI’s Fugu introduces a new way to work with AI models: instead of relying on a single model for every task, Fugu serves as a multi-agent system that coordinates specialized AI agents via a single OpenAI-compatible API.

Sakana Fugu — Multi-agent System as A Model

That makes it especially useful for workflows that need more than static model knowledge. When you connect Fugu with Web Search API, you can give your AI application access to real-time search data from Google and other search engines, then use Fugu to reason over that data, summarize it, compare results, or generate useful outputs.

How to use the Sakana AI API

The Sakana AI API is OpenAI-compatible. We can use it like this:

import os
from openai import OpenAI

base_url = "https://api.sakana.ai/v1"

client = OpenAI(
    api_key=os.environ["FUGU_API_KEY"],
    base_url=base_url,
)

response = client.responses.create(
    model="fugu",
    input="Create a sarcastic poem for vibe coder",
)

print(response.output_text)

Don't forget to export your FUGU_API_KEY on the terminal

export FUGU_API_KEY=YOUR_SAKANA_FUGU_API_KEY

The poem result, hope no one is offended 😄

basic response API call for Sakana AI

Streaming

You can also stream the results with a simple tweak

with client.responses.stream(
    model="fugu",
    input="Explain why vibe coding is useful in three short bullets.",
) as stream:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)

    # Full response object assembled from the stream.
    response = stream.get_final_response()

print()
print(response.output_text)

Two available models

We have two options to use as the model:

Fugu: for balancing performance and low latency. Can be used for common problems or tasks.
Fugu Ultra: for more complicated tasks. It will use a deeper pool of expert AI agents. It costs more and takes a longer time to finish.

Benchmark from sakana.ai website

Give real-time data to Sakana AI

Not just the simple completion or response built-in tools that available on OpenAI is also available here, except the advance options.

First, let's ask Sakana what the recent World Cup score is:


response = client.responses.create(
    model="fugu",
    input="What's the recent FIFA men's world cup score?",
)

print(response.output_text)

Here is the result:

Sakana AI has no real-time data access

As we know, world cup 2026 is currently running in USA, clearly the AI don't have access to recent information.

Basic Web Search tool

Let's connect with the built-in search tool

response = client.responses.create(
    model="fugu",
    tools=[{"type": "web_search"}], # one line addition
    input="What's the recent FIFA men's world cup score?",
)

print(response.output_text)

Now, we have the update from today's match (when this blog is written)

collect real-time news on Sakana AI

Advanced Web Search Tool

What if we ask more complicated questions that can't be answered with a simple Google search?

input="What's the ticket price for this weekend from Toronto to Los Angeles? one way trip"

The program got stuck or just took a very long time (I waited for 3 minutes). So, I'm trying something else. Maybe if I stream it, I can get the thought process:

with client.responses.stream(
    model="fugu",
    tools=[{"type": "web_search"}], 
    input="What's the recommended gift for Argentina football fans? share a link and the price",
) as stream:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)

    # Full response object assembled from the stream.
    response = stream.get_final_response()

print()
print(response.output_text)

Again, I don't see anything and had to wait ~3 minutes before I stopped the program

Sakana AI error on hard question that need real-time data

The issue and the solution

Similar to the OpenAI native web search tool, there are still some limitations. To solve this issue, we can connect the AI with SerpApi - search engine APIs.

Sample project

I'm going to ask the AI to return a recommended items alongside real time pricing and shopping link.

Sample usage:

query = "What's a good gaming laptop under $1000?"
run_shopping_assistant(query)

Sample result:

Sample result when Sakana AI has access to live e-commerce data

Preparation

Ensure you register at serpapi.com to get your free API key.
Understand the basics of function calling on OpenAI, as we'll build a custom function to call an API

Function calling | OpenAI API

If you want to try it directly, here is the source code link:

tutorials/python_projects/sakana-fugu-web-search at master · serpapi/tutorials

Code implementation

First, let's load everything: OpenAI API key, SerpApi API Key, and the required packages

import os
import json
import requests
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

base_url = "https://api.sakana.ai/v1"

client = OpenAI(
    api_key=os.environ.get("FUGU_API_KEY"),
    base_url=base_url,
)

SERPAPI_API_KEY = os.environ.get("SERPAPI_API_KEY")

Define the tool structure we're going to use

tools = [
    {
        "type": "function",
        "name": "google_shopping_search",
        "description": "Search for products on Google Shopping using SerpApi",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The product search query"
                },
            },
            "required": ["query"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]

We can add more if we need to access multiple APIs.

Define the actual search function Now, we're going to call the API, in this case, the Google Shopping API

def google_shopping_search(query):
    """
    Simple Google Shopping search using SerpApi
    """
    print("🔎 Checking SerpApi key...", flush=True)
    if not SERPAPI_API_KEY:
        return {
            "success": False,
            "error": "Missing SERPAPI_API_KEY environment variable",
            "query": query
        }

    print(f"🌐 Requesting Google Shopping results from SerpApi for: {query}", flush=True)
    params = {
        "api_key": SERPAPI_API_KEY,
        "engine": "google_shopping",
        "q": query,
    }

    try:
        response = requests.get("https://serpapi.com/search", params=params, timeout=30)
        print(f"🌐 SerpApi responded with HTTP {response.status_code}", flush=True)
        response.raise_for_status()
        print("📦 Parsing SerpApi response JSON...", flush=True)
        data = response.json()

        shopping_results = data.get("shopping_results", [])

        # Format results
        formatted_results = []
        for item in shopping_results[:5]:
            formatted_item = {
                "title": item.get("title", ""),
                "price": item.get("price", "N/A"),
                "link": item.get("product_link", ""),
                "source": item.get("source", ""),
                "rating": item.get("rating", "N/A"),
                "reviews": item.get("reviews", 0),
            }
            formatted_results.append(formatted_item)

        return {
            "success": True,
            "query": query,
            "results": formatted_results,
            "total_results": len(shopping_results)
        }

    except requests.exceptions.RequestException as e:
        return {
            "success": False,
            "error": f"API Error: {str(e)}",
            "query": query
        }

In this example, I'm getting the first 5 items from the Google Shopping API response.

Helper function

To maintain the readability of the code, let's break down some helper functions. First, to help us see the results in a readable format. You can skip this if you don't need it on your terminal:

def print_shopping_results(result):
    """
    Print SerpApi shopping results in a readable format.
    """
    if result["success"]:
        print(f"📊 Found {result['total_results']} products:\n")
        for i, item in enumerate(result["results"], 1):
            print(f"{i}. {item['title']}")
            print(f"   Price: {item['price']}")
            print(f"   Source: {item['source']}")
            if item["rating"] != "N/A":
                print(f"   Rating: {item['rating']} ⭐ ({item['reviews']} reviews)")
            print(f"   Link: {item['link']}\n")
    else:
        print(f"   Error: {result['error']}\n")

Next, to send the actual request to Fugu:

def create_response(input_data):
    """
    Create a non-streaming Fugu response.
    """
    print("🤖 Sending request to Fugu...", flush=True)
    return client.responses.create(
        model="fugu",
        tools=tools,
        input=input_data,
    )

Last one, to handle the tool call:

def handle_tool_call(tool_call):
    """
    Execute a Responses API function_call item.
    """
    if tool_call.name != "google_shopping_search":
        result = {
            "success": False,
            "error": f"Unknown tool: {tool_call.name}",
        }
    else:
        try:
            tool_input = json.loads(tool_call.arguments or "{}")
        except json.JSONDecodeError as e:
            tool_input = {}
            result = {
                "success": False,
                "error": f"Invalid tool arguments: {e}",
            }
        else:
            print(f"\n🔧 Calling function: {tool_call.name}")
            print(f"   Query: {tool_input.get('query')}\n")
            result = google_shopping_search(tool_input["query"])
            print_shopping_results(result)

    if not result.get("success"):
        raise RuntimeError(result.get("error", "Tool call failed"))

    return {
        "type": "function_call_output",
        "call_id": tool_call.call_id,
        "output": json.dumps(result),

Running the program

Now, is the actual function that triggers the whole thing:

def run_shopping_assistant(user_query):
    """
    Run the shopping assistant with Sakana AI Fugu
    """
    if not os.environ.get("FUGU_API_KEY"):
        raise RuntimeError("Missing FUGU_API_KEY environment variable")

    print(f"User: {user_query}\n")

    input_messages = [
        {"role": "user", "content": user_query}
    ]

    print("🧠 Asking Fugu whether a tool is needed...", flush=True)
    response = create_response(input_messages)
    print("✅ Fugu response received.", flush=True)

    max_iterations = 5
    for iteration in range(1, max_iterations + 1):
        print(f"🔁 Processing response step {iteration}/{max_iterations}...", flush=True)

        if response.output_text:
            print(response.output_text)
            print()

        input_messages += response.output

        tool_calls = [
            item for item in response.output
            if getattr(item, "type", None) == "function_call"
        ]

        if not tool_calls:
            print("✅ No tool calls requested. Done.", flush=True)
            return response.output_text

        print(f"🛠️ Fugu requested {len(tool_calls)} tool call(s).", flush=True)
        tool_outputs = [handle_tool_call(tool_call) for tool_call in tool_calls]
        input_messages += tool_outputs

        print("🤖 Recommendation:\n")
        print("🧠 Sending tool results back to Fugu...", flush=True)
        response = create_response(input_messages)
        print("✅ Fugu follow-up response received.", flush=True)

    raise RuntimeError("Reached maximum tool-calling iterations")

# Example usage
if __name__ == "__main__":
    try:
        # Example 1
        query1 = "What's a good gaming laptop under $1000?"
        run_shopping_assistant(query1)
    except RuntimeError as e:
        print(f"❌ Error: {e}")
        print("\nMake sure you have a .env file with:")
        print("  FUGU_API_KEY=...")
        print("  SERPAPI_API_KEY=...")

That's it.

Use Sakana AI for Coding

Want to use Fugu, the new Sakana AI model, as your coding partner? Here is an example from our Vibe Coding series on YouTube:

https://www.youtube.com/embed/YfGbtyVnGnE?feature=oembed

Here is the official guideline on using Sakana in Codex.

FAQ on Sakana AI

Some frequently asked questions:

Why is Sakana AI popular?

Sakana AI is popular because it takes a different approach to AI by focusing on systems that can combine and coordinate multiple models. Its Fugu model is especially interesting because it works through an OpenAI-compatible API and can orchestrate specialized AI agents for more complex tasks.

Is Sakana AI - Fugu model free?

It's not free. Fugu is available through paid subscription and pay-as-you-go plans, so you’ll need to check the Sakana AI Console for the latest pricing, available models, and usage limits.

How to use Sakana AI?

To use Sakana AI, sign in to the Sakana AI Console and connect to it using its OpenAI-compatible API. You can also combine it with SerpApi to give your AI app access to real-time search data from Google and other search engines.

If you want to use it as your coding partner, you can install codex-fugu to use it just like codex CLI.

Amazon ASIN Lookup API: Find and Fetch Product Details

Hilman Ramadhan — Wed, 29 Jul 2026 02:45:23 +0000

Amazon products are identified by an ASIN , short for Amazon Standard Identification Number. If you're building a product research tool, price monitoring system, competitor tracker, or an Amazon data workflow, the ASIN is usually the key identifier you need.

In this tutorial, we’ll learn how to:

Search Amazon products by keyword.
Extract the ASIN from Amazon search results.
Use that ASIN to fetch detailed product information.

We’ll use two SerpApi APIs:

Amazon Search API: to search Amazon and retrieve product results with ASINs.
Amazon Product API: to look up detailed product data using an ASIN.

Amazon ASIN Lookup API method

Video Tutorial

We also provide video tutorials for this guide.

Amazon Search Results Scraper:

https://www.youtube.com/embed/j3YXfsMPKjQ?feature=oembed

Amazon Product Detail Scraper:

https://www.youtube.com/embed/Yd17Jr45liw?feature=oembed

Why use an Amazon Search API?

Manually searching Amazon and copying product details is slow. Building your own scraper also means handling HTML changes, blocking, proxies, parsing, and maintenance.

With an API, you can programmatically look up products and get structured JSON data without the scraping headaches.

The Amazon Search API uses the endpoint https://serpapi.com/search?engine=amazon, and the query parameter is k, which works like a regular Amazon search query.

You can use this for:

Product research
Price monitoring
Competitor tracking
Marketplace analysis
Product catalog enrichment
Review and rating monitoring
Amazon product matching
AI shopping assistants

Step 1: Search Amazon products and get ASINs

We're using Python for the tutorial. You can use any programming language you want.

Install requests first:

pip install requests

Create a new main.py file. Let’s say we want to search for "coffee" in Amazon:

import requests
import json

SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "amazon",
    "amazon_domain": "amazon.com",
    "k": "coffee"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

print(json.dumps(response, indent=2))

This sends a request to the Amazon Search API and returns Amazon search results in JSON format.

The k parameter defines the Amazon search query, and amazon_domain lets you choose the Amazon marketplace, such as amazon.com.

Here is the example result:

Amazon ASIN scraper sample result

You can see that we receive the ASIN for each item.

Step 2: Extract ASINs from Amazon search results

Amazon Search API results can include product fields like title, link, thumbnail, rating, and reviews. Products inside Amazon results include an ASIN field and a serpapi_link pointing to the Amazon Product API lookup for that ASIN.

Let’s print a cleaner list of products:

import requests

SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "amazon",
    "amazon_domain": "amazon.com",
    "k": "coffee"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

for product in response.get("organic_results", []):
    title = product.get("title")
    asin = product.get("asin")
    price = product.get("price")
    rating = product.get("rating")
    reviews = product.get("reviews")

    print(f"Title: {title}")
    print(f"ASIN: {asin}")
    print(f"Price: {price}")
    print(f"Rating: {rating}")
    print(f"Reviews: {reviews}")
    print("-" * 50)

Example output:

Title: Amazon Fresh, Colombia Ground Coffee, Medium Roast, 32 Oz
ASIN: B072MQ5BRX
Price: $15.31
Rating: 4.5
Reviews: 7823
--------------------------------------------------

Now we have the ASIN. Next, we can use it to get more complete product details.

Step 3: Look up product details by ASIN

The Amazon Product API uses engine=amazon_product and requires an asin parameter. The request returns a product_results object containing fields like asin, title, description, tags, badges, variants, brand, links, thumbnails, and more.

Here’s how to look up a product by ASIN:

import requests
import json

SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "amazon_product",
    "asin": "B072MQ5BRX"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

print(json.dumps(response, indent=2))

Now let’s print only the most useful product details:

import requests

SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "amazon_product",
    "amazon_domain": "amazon.com",
    "asin": "B072MQ5BRX"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

product = response.get("product_results", {})

print(f"Title: {product.get('title')}")
print(f"ASIN: {product.get('asin')}")
print(f"Brand: {product.get('brand')}")
print(f"Rating: {product.get('rating')}")
print(f"Reviews: {product.get('reviews')}")
print(f"Price: {product.get('price')}")
print(f"Link: {product.get('link')}")

Complete example: Search Amazon, get the first ASIN, then fetch product details

In many cases, you don’t already know the ASIN. You only have a keyword, a product name, a brand, or a category.

Here’s a complete workflow:

import requests

SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"


def search_amazon_products(query):
    params = {
        "api_key": SERPAPI_API_KEY,
        "engine": "amazon",
        "amazon_domain": "amazon.com",
        "k": query
    }

    search = requests.get("https://serpapi.com/search", params=params)
    return search.json()


def get_amazon_product_by_asin(asin):
    params = {
        "api_key": SERPAPI_API_KEY,
        "engine": "amazon_product",
        "amazon_domain": "amazon.com",
        "asin": asin
    }

    search = requests.get("https://serpapi.com/search", params=params)
    return search.json()


search_results = search_amazon_products("coffee")

first_product = search_results.get("organic_results", [])[0]
asin = first_product.get("asin")

print(f"Found ASIN: {asin}")
print(f"Product title: {first_product.get('title')}")

product_details = get_amazon_product_by_asin(asin)
product = product_details.get("product_results", {})

print("\nProduct Details")
print(f"Title: {product.get('title')}")
print(f"Brand: {product.get('brand')}")
print(f"Price: {product.get('price')}")
print(f"Rating: {product.get('rating')}")
print(f"Reviews: {product.get('reviews')}")

JavaScript example

You can also do the same thing in JavaScript.

Install the SerpApi package:

npm install serpapi

Then create an index.js file:

const { getJson } = require("serpapi");

const API_KEY = "YOUR_SERPAPI_API_KEY";

async function searchAmazonProducts(query) {
  return new Promise((resolve, reject) => {
    getJson(
      {
        api_key: API_KEY,
        engine: "amazon",
        amazon_domain: "amazon.com",
        k: query,
      },
      (json) => {
        resolve(json);
      }
    );
  });
}

async function getAmazonProductByAsin(asin) {
  return new Promise((resolve, reject) => {
    getJson(
      {
        api_key: API_KEY,
        engine: "amazon_product",
        amazon_domain: "amazon.com",
        asin,
      },
      (json) => {
        resolve(json);
      }
    );
  });
}

async function main() {
  const searchResults = await searchAmazonProducts("coffee");

  const firstProduct = searchResults.organic_results?.[0];

  if (!firstProduct) {
    console.log("No product found.");
    return;
  }

  const asin = firstProduct.asin;

  console.log("Found ASIN:", asin);
  console.log("Product title:", firstProduct.title);

  const productDetails = await getAmazonProductByAsin(asin);
  const product = productDetails.product_results;

  console.log("\nProduct Details");
  console.log("Title:", product?.title);
  console.log("Brand:", product?.brand);
  console.log("Price:", product?.price);
  console.log("Rating:", product?.rating);
  console.log("Reviews:", product?.reviews);
}

main();

Useful parameters

For Amazon Search API:

{
  "engine": "amazon",
  "amazon_domain": "amazon.com",
  "k": "coffee"
}

Useful parameters include:

k: the Amazon search query.
amazon_domain: the Amazon marketplace to use.
language: the Amazon search language.
delivery_zip: ZIP or postal code for shipping-based results.
shipping_location: shipping country.
s: sorting option.

The Amazon Search API supports sorting options such as featured, price low to high, price high to low, average customer review, newest arrivals, and best sellers.

What data can you get from an ASIN lookup?

Depending on the product and Amazon page, the Amazon Product API can return structured data such as:

Product title
ASIN
Brand
Description
Product images
Price
Rating
Reviews
Badges
Variants
Product details
Product features
Buying options
Other sellers
Related products

Conclusion

An Amazon ASIN lookup workflow usually has two steps:

Use the Amazon Search API to search Amazon and find product ASINs.
Use the Amazon Product API to fetch detailed product information from a specific ASIN.

This makes it easier to build product research tools, price trackers, marketplace dashboards, AI shopping assistants, and catalog enrichment workflows without maintaining your own Amazon scraper.

You can try the APIs here:

Build an AI Agent with the Claude Agent SDK (Tutorial 2026)

Hilman Ramadhan — Tue, 28 Jul 2026 07:53:32 +0000

Many people are starting to use AI agents in production, and the Claude Agent SDK makes building them surprisingly easy. Unlike a basic chatbot that responds and forgets, an agent can reason through multi-step problems, call tools, handle errors, and keep going until a task is actually done.

Whether you're automating a research pipeline, wiring up a coding assistant, or orchestrating a fleet of specialized sub-agents, the SDK gives you everything you need without having to reinvent the wheel.

Claude AI Agent SDK tutorial

Claude Agent SDK features

The Agent SDK allows us to programmatically build an agent. If you're not a developer, you may want to consider a no-code solution like OpenClaw or n8n instead.

Claude Agent SDK includes built-in features such as reading and writing files, running commands, and understanding code. You can even search and fetch web content with it.

from claude.com - Agent SDK built in tools

We can create a custom function to extend its capabilities. Of course, we also have an option to connect the Agent directly with MCP.

All of the code mentioned in this blog post is available in this repository:

GitHub - serpapi/claude-agent-with-web-search: Demo of using the Claude Agent SDK with web search capability from SerpApi

Quickstart tutorial

At the moment, it offers an SDK for Python and Typescript. We'll use the Python SDK in this tutorial.

Prepare the Agent

Let's start fresh. Create a new directory and cd into it.

mkdir first-agent && cd first-agent

Create a new Python file

touch main.py

Install SDK

Here is the repo if you want to learn more about the Python SDK.

pip install claude-agent-sdk

Prerequisite: Python 3.10+

Set your API Key

Register a free account and get your API key here. Create a new .env file, then store the value there:

ANTHROPIC_API_KEY=your-api-key

Sample code

Here is a simple code to test if the SDK is installed properly

import asyncio
from claude_agent_sdk import query

async def main():
    async for message in query(prompt="What is the Claude Agent SDK capabilities?"):
        print(message)

asyncio.run(main)

Sidenote: asyncio is an asynchronous networking and concurrency library.

What is the query command?

Query is the entry point for the agentic loop. We have to use async for to stream the message.

It'll loop the process if there are any tools to call or other things to do; otherwise, it'll share the final answer.

The query command can also accept options besides a prompt.

options = ClaudeAgentOptions(
    system_prompt="You are an expert in marketing",
    max_turns=2
)

async for message in query(prompt="How to improve my SEO", options=options):
    print(message)

Allowed tools

We can choose a specific tool by adding allowed_tools option. For example, let's say we want to read a file and fix an error in it.

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, AssistantMessage, ResultMessage

USER_PROMPT = "Review oldfile.py for bugs that would cause crashes. Fix any issues you find. Please create a new file, newfile.py, with the fixes. Summarize your changes in a comment at the top of the new file."

async def main():
    # Agentic loop: streams messages as Claude works
    async for message in query(
        prompt=USER_PROMPT,
        options=ClaudeAgentOptions(
            allowed_tools=["Read", "Edit", "Glob"],  
            permission_mode="acceptEdits", # Auto-approve file edits
        ),
    ):
        # Print human-readable output
        if isinstance(message, AssistantMessage):
            for block in message.content:
                if hasattr(block, "text"):
                    print("Claude Reasoning:", end=" ")
                    print(block.text) # Claude's reasoning
                elif hasattr(block, "name"):
                    print(f"Tool: {block.name}") # Tool being called
        elif isinstance(message, ResultMessage):
            print(f"Done: {message.subtype}") # Final result


asyncio.run(main())

There's also disallowed_tools if you want to prevent the agent from using a specific tool.

WebSearch tool

Let's try the built-in web search tool to scrape the internet. We'll see the limitations of the native tool and the solution to them.

Add WebSearch in the allowed_tools parameter.

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, AssistantMessage, ResultMessage


PROMPT = "What's trending news in Indonesia at the moment?. Share article link for each."

async def main():
    # Agentic loop: streams messages as Claude works
    async for message in query(
        prompt=PROMPT,
        options=ClaudeAgentOptions(
            allowed_tools=["WebSearch"], # Allowing web search
        ),
    ):
        # Print human-readable output
        if isinstance(message, AssistantMessage):
            for block in message.content:
                if hasattr(block, "text"):
                    print("Claude Reasoning:", end=" ")
                    print(block.text) # Claude's reasoning
                elif hasattr(block, "name"):
                    print(f"Tool: {block.name}") # Tool being called
        elif isinstance(message, ResultMessage):
            print(f"Done: {message.subtype}") # Final result


asyncio.run(main())

It took me around 84.76 seconds to finish this. It's pretty slow for a simple question.

Native websearch tool on Claude Agent SDK

Next, I want it to handle more complex tasks, like searching for flight information. I switch the prompt to this:

PROMPT = "Share flight info from Jakarta to Bali on April 30th? Share price, time detail, flight number, and resource for each. Don't hallucinate."

This time. It took me 80.68s.. but with little useful information.

Issue with native websearch

It seems like we're not able to scrape the internet with more advanced tasks like this. The result is just basic knowledge, not really from any resources.

Let's see the solution in the next section.

Real-time data for AI tools: SerpApi

Meet SerpApi, it's the web search API for your AI tools, LLMs, and AI agents. As we've seen, the native web search tool is not very reliable.

SerpApi allows you to collect data from various search engine results in real-time, like:

Google Search
Google Maps
Amazon
Ebay
and more

It supports 100+ search engine APIs.

Connect Agent SDK with an external tool

It's possible to use a custom API with the agent. We have several ways to do it:

Connect an MCP server
Write our function manually
Use claude skill file

We'll see how to do all of them in this blog post.

Connect the Claude Agent with an MCP server

If the external tool you want to use already has an MCP server, good news! You can connect the server directly, without adding too much code.

I'll share an example of connecting the Claude Agent SDK with the SerpApi MCP server. Feel free to use any other MCP server that is relevant to your project.

Ensure to register for free at serpapi.com to get your API key. I'll put my API key in an .env file and load it in the main.py file.

Install python-dotenv to read .env file

pip install python-dotenv

Load the API key

# Connecting SerpApi key
import os
from dotenv import load_dotenv
load_dotenv()
SERPAPI_API_KEY = os.environ.get("SERPAPI_API_KEY")

Here is how to connect the MCP server:

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, ResultMessage

# load SerpApi API key from .env file
import os
from dotenv import load_dotenv
load_dotenv()
SERPAPI_API_KEY = os.environ.get("SERPAPI_API_KEY")

async def main():
    SYSTEM_PROMPT = "Always use the SerpApi MCP server for web search;" 
    USER_PROMPT = "Find me a hotel under $100 in Bay Area next week."

    options = ClaudeAgentOptions(
        system_prompt=SYSTEM_PROMPT,
        mcp_servers={
            "serpapi": {
                "type": "http",
                "url": "https://mcp.serpapi.com/"+ SERPAPI_API_KEY +"/mcp",
            }
        },
        allowed_tools=["mcp __serpapi__ *"], #Base name:mcp __<server-name>__ <tool-name>
    )

    async for message in query(
        prompt=USER_PROMPT,
        options=options,
    ):
        if isinstance(message, ResultMessage) and message.subtype == "success":
            print(message.result)


asyncio.run(main())

Based on my personal experience, Claude doesn't always use the MCP, so I need to specify the instructions for them, which I put in the SYSTEM_PROMPT variable above. The result is more consistent.

Here is the example result:

Connect Claude Agent SDK with MCP server

If you're using Claude Desktop, here is how to connect to the SerpApi MCP server there:

How to Connect SerpApi MCP to Claude Desktop

How to call a custom API

Regardless of whether you're using SerpApi, it's good to learn how to call a custom function in the Claude Agent SDK to make it more powerful.

The easiest way is to use MCP, as we've seen in the previous section. However, since not many tools support MCP at the moment, let's see how to connect a tool manually. You also have more control when declaring the custom function manually.

The steps are:

Define the custom tool with the actual function inside
Connect this tool using create_sdk_mcp_server function
Allow this tool to use allowed_tools parameter

Let's say we want to build a travel agent. First, we want to connect it with the Google Flights API.

Ensure to understand what are the parameters and how to call your API.

Here is an example of defining the Google Flights API function:

import json
import requests
import asyncio
from typing import Any
from claude_agent_sdk import tool, create_sdk_mcp_server, query, ClaudeAgentOptions, AssistantMessage, ResultMessage

# Connecting SerpApi key
import os
from dotenv import load_dotenv
load_dotenv()
SERPAPI_API_KEY = os.environ.get("SERPAPI_API_KEY")

# Custom tool: SerpApi Google Flights Search
@tool(
    name="search_flights",
    description="Search for flights using SerpApi. Input should be a JSON string with 'origin', 'destination', and 'date' fields.",
    input_schema={
        "type": "object",
        "properties": {
            "departure_id": {"type": "string", "description": "IATA code of the departure airport"},
            "arrival_id": {"type": "string", "description": "IATA code of the arrival airport"},            
            "type": {"type": "number", "description": "Type of flight search: 1 for round trip (default), 2 for one way"},
            "outbound_date": {"type": "string", "description": "Departure date in YYYY-MM-DD format"},
            "return_date": {"type": "string", "description": "Return date in YYYY-MM-DD format (required if type is 1)"},
        },
        "required": ["departure_id", "arrival_id", "type", "outbound_date"],
    },
)
async def search_flights(args: dict[str, Any]) -> dict[str, Any]:
    try:
        departure_id = args["departure_id"]
        arrival_id = args["arrival_id"]
        flight_type = args["type"] #args.get("type", 1) # Default to round trip if not provided
        outbound_date = args["outbound_date"]
        return_date = args["return_date"] #args.get("return_date", "")

    except (json.JSONDecodeError, KeyError) as e:
        return f"Invalid input: {e}"

    # Call SerpApi Google Flights Search
    params = {
        "engine": "google_flights",
        "api_key": SERPAPI_API_KEY,
        "departure_id": departure_id,
        "arrival_id": arrival_id,
        "type": flight_type,
        "outbound_date": outbound_date,
    }

    if flight_type == 1 and return_date:
        params["return_date"] = return_date

    response = requests.get("https://serpapi.com/search", params=params)

    if response.status_code == 200:
        results = response.json()
        # Extract relevant flight information (this is just an example, adjust as needed)
        flights = results.get("best_flights", []) #optional: other_flights
        if not flights:
            return "No flights found."

        print("--- Successfully fetched flight data") # Debugging statement

        return {
            "content": [
                {
                    "type": "text",
                    "text": json.dumps(flights, indent=2)  
                }
            ]
        }
    else:
        print("--- Error fetching flight data:", response.status_code, response.text) # Debugging statement
        return f"Error fetching flight data: {response.status_code}"

Define the tool with tool() helper. It should have the name, description, and input schema.
Declare the actual function after that.
We need to return a content array with a specified type. In this example, we're using text .

Connecting the custom tool

To connect a custom tool, we need to use the create_sdk_mcp_server like this:

flight_search_server = create_sdk_mcp_server(
    name="flight",
    version="1.0",
    tools=[search_flights],
)

This is not an actual MCP server. You can think of a local MCP, where we manually create the function.

Here's the prompt I want to test:

USER_PROMPT = "I want to find a flight from Jakarta to Singapore departing on 2026-04-20 and returning on 2026-04-30. Can you help me with that? If the tool is failed, write in detail what's the issue"

Pro tips: add this on prompt during debugging: "If the tool is failed, write in detail what's the issue". From my experience, many error could happen during connecting the tool with Claude. It'll help us debug the issue.

Here is the AI Agent function:

async def main():
    async for message in query(
        prompt=USER_PROMPT,
        options=ClaudeAgentOptions(
            mcp_servers={"flight": flight_search_server}, # Register the MCP server with the custom tool
            allowed_tools=["mcp __flight__ search_flights"] # Base: mcp __{server_name}__ {tool_name}
        ),
    ):
        # Print human-readable output
        if isinstance(message, AssistantMessage):
            for block in message.content:
                if hasattr(block, "text"):
                    print("Claude Reasoning:", end=" ")
                    print(block.text) # Claude's reasoning
                elif hasattr(block, "name"):
                    print(f"Tool: {block.name}") # Tool being called
        elif isinstance(message, ResultMessage):
            print(f"Done: {message.subtype}") # Final result

asyncio.run(main())

It shows much better results with much faster speed (25.69s). Compared to the built-in function that takes ~80s. That's almost three times faster.

Here is the full result:

Claude Agent SDK with web search function

We can compare the result with the actual Google Flights website:

Google Flights website data

As you can see, the AI agent is getting the answer from a real-time data source.

Use multiple tools in the Claude Agent SDK

If the steps are clear, we may not need an AI Agent. Instead, we can just build a program with a linear workflow, which is much faster and simpler.

An important reminder from the Anthropic blog:

When to use agents: Agents can be used for open-ended problems where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents' autonomy makes them ideal for scaling tasks in trusted environments.

By the way, we have tutorials on how to connect SerpApi with your LLMs.

Now, assuming the user is asking more complicated questions that require several tools to solve. Let's see how to add multiple tools to the agent.

The idea is the same, we need to declare the tool with @tool, connect it with the create_sdk_mcp_server and let the agent know about it.

I'll continue from our previous Flight API example. Let's say we want to build a travel agent. I'm now introducing a Google Hotels API to retrieve hotel information.


@tool(
    name="search_hotels",
    description="Search for hotels using SerpApi. Input should be a JSON string with 'q', 'check_in_date', and 'check_out_date' fields.",
    input_schema={
        "type": "object",
        "properties": {
            "q": {"type": "string", "description": "Search query, e.g., 'hotels in Singapore'"},
            "check_in_date": {"type": "string", "description": "Check-in date in YYYY-MM-DD format"},
            "check_out_date": {"type": "string", "description": "Check-out date in YYYY-MM-DD format"},
            "min_price": {"type": "number", "description": "Minimum price filter (optional)"},
            "max_price": {"type": "number", "description": "Maximum price filter (optional)"},
        },
        "required": ["q", "check_in_date", "check_out_date"],
    },
)
async def search_hotels(args: dict[str, Any]) -> dict[str, Any]:
    print("---Received search_hotels tool call with args:", args) # Debugging statement
    try:
        q = args["q"]
        check_in_date = args["check_in_date"]
        check_out_date = args["check_out_date"]
        min_price = args.get("min_price")
        max_price = args.get("max_price")

        print(f"Parsed input - Query: {q}, Check-in: {check_in_date}, Check-out: {check_out_date}, Min Price: {min_price}, Max Price: {max_price}") # Debugging statement

    except (json.JSONDecodeError, KeyError) as e:
        return f"Invalid input: {e}"

    # Call SerpApi Google Hotels Search
    params = {
        "engine": "google_hotels",
        "api_key": SERPAPI_API_KEY,
        "q": q,
        "check_in_date": check_in_date,
        "check_out_date": check_out_date,
    }

    if min_price is not None:
        params["min_price"] = min_price
    if max_price is not None:
        params["max_price"] = max_price

    response = requests.get("https://serpapi.com/search", params=params)

    if response.status_code == 200:
        results = response.json()
        hotels = results.get("properties", [])
        hotels = hotels[:2] # limit to top 2 hotels for save tokens; You can adjust this as needed
        if not hotels:
            return "No hotels found."

        print("--- Successfully fetched hotel data") # Debugging statement

        return {
            "content": [
                {
                    "type": "text",
                    "text": json.dumps(hotels, indent=2)  
                }
            ]
        }
    else:
        print("--- Error fetching hotel data:", response.status_code, response.text) # Debugging statement
        return f"Error fetching hotel data: {response.status_code}"

Next, we connect both tools like this:

travel_search_server = create_sdk_mcp_server(
    name="travel",
    version="1.0",
    tools=[search_flights, search_hotels], # You can register multiple tools under the same server
)

Here is how to use it on the Agent:

async def main():
     async for message in query(
        prompt=USER_PROMPT,
        options=ClaudeAgentOptions(
            system_prompt=SYSTEM_PROMPT,
            include_partial_messages=True,
            mcp_servers={"travel": travel_search_server}, # Register the MCP servers with the custom tools
            allowed_tools=["mcp __travel__ search_flights", "mcp __travel__ search_hotels"], # Base: mcp __{server_name}__ {tool_name}
            disallowed_tools=["Read"]
        ),
    ):
        # Print human-readable output
        if isinstance(message, AssistantMessage):
            for block in message.content:
                if hasattr(block, "text"):
                    print("\nClaude Reasoning:", end=" ")
                    print(block.text) # Claude's reasoning
                elif hasattr(block, "name"):
                    print(f"\nTool: {block.name}") # Tool being called
        elif isinstance(message, ResultMessage):
            print(f"\nDone: {message.subtype}") # Final result

Here is our prompt for testing. I'm also adding some instructions to ensure the agent uses the tools.

SYSTEM_PROMPT = """You're a helpful travel assistant. Help user for their trip.
- Search flights using the `search_flights` tool
- Search hotels using the `search_hotels` tool
- Combine the results to find the best options for the user based on their preferences and constraints.
- If you encounter any issues with the tools, explain the problem in detail."""

# Simple sample
USER_PROMPT = """ I have a developer conference in Singapore on April 30, 2026. I'm at Malaysia at the moment. I can spend 5 days there.
                Can you help me with that?"""

Here is the output of the thinking process:

Thinking process Claude Agent SDK

And here is the final response:

Multiple tools with Claude Agent SDK

It's succesfully running two APIs and know what to search on each, which makes it perfect for running an AI agent that can think for themself.

Use the Claude Skill in the Claude Agent SDK

Have you heard about Claude Skill? It's another way to add a tool to the AI by providing the skill information in a Markdown file. Yes! It's also possible to use the Claude skill in the SDK.

Assuming we already have the skill directory locally. Here is how to connect it:

async def main():
    options = ClaudeAgentOptions(
        cwd=".claude/skills/serpapi-basic-skill", # Project with .claude/skills/
        setting_sources=["user", "project"], # Load Skills from filesystem
        allowed_tools=["Skill", "Read", "Write", "Bash"], # Enable Skill tool
    )

    # USER_PROMPT = "What is your serpapi skill?
    USER_PROMPT = "Use the SerpApi skill. What is the trending topic in healthcare now? Share link for reference."
    async for message in query(
        prompt=USER_PROMPT, options=options
    ):
    ....

In this example, I'm just creating a new SKILL.md file inside this directory .claude/skills/serpapi-basic-skill .

You can try adding a new skill by adding a simple custom instruction on a SKILL.md file.

Multi-turn conversation with session feature

Sometimes, we have a follow-up question or task for the agent, and hardcoding the prompt in the code is not very convenient.

Claude Agent SDK provides Sessions feature, where we can maintain context after multiple exchanges. It'll remember our conversation history and data.

Here is the basic example:

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, SystemMessage, ResultMessage


async def main():
    session_id = None

    # First query: capture the session ID
    async for message in query(
        prompt="Why is it important to have access to real-time search engine data",
        options=ClaudeAgentOptions(allowed_tools=["Read", "Glob"]),
    ):
        if isinstance(message, SystemMessage) and message.subtype == "init":
            session_id = message.data["session_id"]

    # Resume with full context from the first query
    async for message in query(
        prompt="How to do it programmatically?", # "it" = auth module
        options=ClaudeAgentOptions(resume=session_id),
    ):
        if isinstance(message, ResultMessage):
            print(message.result)


asyncio.run(main())

Next, here is how I implement it on the travel agent example we did when we learn about multiple tools calls previously.

First, we prepare the logic to loop the conversation in the terminal:

async def main():
    print("=== Claude Terminal Chat ===")
    print('Type "exit" to quit.\n')

    session_id = None # None on first turn → fresh session

    while True:
        try:
            user_input = input("You: ").strip()
        except (EOFError, KeyboardInterrupt):
            print("\nGoodbye!")
            break

        if not user_input:
            continue

        if user_input.lower() in {"exit", "quit", "q"}:
            print("Goodbye!")
            break

        # Each turn resumes from where the last one left off
        session_id = await run_turn(user_input, session_id)


asyncio.run(main())

Next, we replace the previous function name with run_turn and add the session logic there:

SYSTEM_PROMPT = """You're a helpful travel assistant. Help user for their trip.
- Search flights using the `search_flights` tool
- Search hotels using the `search_hotels` tool
- Combine the results to find the best options for the user based on their preferences and constraints.
- If you encounter any issues with the tools, explain the problem in detail."""


async def run_turn(prompt: str, session_id: str | None) -> str | None:
    start_time = time.perf_counter()

    options = ClaudeAgentOptions(
        system_prompt=SYSTEM_PROMPT,
        include_partial_messages=True,
        mcp_servers={"travel": travel_search_server}, # Register the MCP servers with the custom tools
        allowed_tools=["mcp __travel__ search_flights", "mcp __travel__ search_hotels"], # Base: mcp __{server_name}__ {tool_name}
        disallowed_tools=["Read"],
        **({"resume": session_id} if session_id else {})
    )

    new_session_id = session_id
    print("Claude: ", end="", flush=True)

    # second call with session_id
    async for message in query(
        prompt=prompt,
        options=options
    ):
        # Capture session_id on the very first turn (init message)
        if isinstance(message, SystemMessage) and message.subtype == "init":
            new_session_id = message.data["session_id"]

        # Print human-readable output
        if isinstance(message, AssistantMessage):
            for block in message.content:
                if hasattr(block, "text"):
                    print("\nClaude Reasoning:", end=" ")
                    print(block.text) # Claude's reasoning
                elif hasattr(block, "name"):
                    print(f"\nTool: {block.name}") # Tool being called
        elif isinstance(message, ResultMessage):
            print(f"\nDone: {message.subtype}") # Final result

    # Log time
    elapsed = time.perf_counter() - start_time
    print(f"Elapsed time: {elapsed:.2f} seconds")

    print() # blank line before next prompt
    return new_session_id

We need to pass the resume: session_id when the id is exists in the options.
At the end, always return the session_id we use for the next conversation.

Summary

McKinsey reported in The State of AI in 2025 that more companies are experimenting with AI agents. If you're looking to build your first AI agent, consider using this SDK.

The Claude Agent SDK is a very powerful tool with a nice developer experience. We can easily use the built-in tools, call a custom tool, either through the MCP server or Claude-skill, or declare the function manually.

You can use SerpApi APIs to extend the Agent capability with real-time information from various search engines.

SerpApi vs. Firecrawl: Search Intelligence VS Website Content

Hilman Ramadhan — Mon, 27 Jul 2026 11:17:52 +0000

When comparing SerpApi vs Firecrawl, you're not comparing two solutions to the same problem. You're comparing two fundamentally different data extraction philosophies that happen to both involve web scraping.

Firecrawl is a website content scraper. You point it at a URL, techcrunch.com/article/some-news and it returns clean, LLM-ready markdown of that article. It crawls links, extracts text, converts HTML to structured formats, and handles JavaScript rendering. It's designed for AI applications that need to ingest website content for RAG systems, knowledge bases, or training data.

SerpApi is a search engine scraper. You query it with "best CRM software" and it returns what Google (and other top search engines) shows for that query: the top 10 organic results, the paid ads, the featured snippet, the "People Also Ask" box, and the local map pack. It doesn't scrape the content of those websites. It scrapes the search results themselves, the metadata, rankings, prices, ratings, and visibility signals that search engines display.

Firecrawl vs SerpApi

For generic content extraction like "Get me all blog posts from this competitor's site," Firecrawl works fine and is purpose-built for that task. But for business intelligence questions like "Who's ranking for this keyword?" or "What's the average price for this product across retailers?", Firecrawl is categorically the wrong tool.

The core issue isn't that Firecrawl is bad at what it does. It's that Firecrawl and SerpApi solve opposite problems. If your business depends on understanding search visibility, competitive positioning, or what consumers actually see when they search, then scraping individual websites doesn't give you that data. You need to scrape the search engines themselves.

The Search Intelligence Gap: What Firecrawl Cannot Do

While Firecrawl recently added a /search endpoint that can perform basic web searches—including basic location customization—it fundamentally differs from true SERP scraping. Search engine results change drastically depending on the user. SerpApi provides an extensive range of parameters (like granular geolocation, device type, and language settings) to fetch fully customized results exactly as your target audience sees them.

Furthermore, Firecrawl's search returns only simple result metadata like URLs, titles, and descriptions. It cannot extract the structured SERP features that drive business intelligence: paid ad blocks, featured snippets as structured objects, Knowledge Graph panels, People Also Ask sections, local pack results, shopping carousels, or any Google-specific SERP elements.

As Firecrawl's own documentation clarifies: "SERP APIs specifically scrape and reformat data from existing search engines like Google or Bing. All SERP APIs are Search APIs, but not all Search APIs are SERP APIs."

This eliminates entire categories of use cases:

SEO rank tracking with SERP features: While Firecrawl provides position numbers in search results, they don't extract structured SERP features (Featured Snippets, People Also Ask, Knowledge Graphs) as separate objects for comprehensive rank tracking
Local SEO analysis: You cannot extract Google Maps data, local pack rankings with review counts, or business verification status
Price comparison: You cannot scrape Google Shopping SERP features or compare prices across retailers as displayed in search

If your product roadmap includes any feature that starts with "Show me what appears when I search for," Firecrawl is not the tool. SerpApi is.

Read moe: SerpApi vs other Web Search API alternatives.

The Structured Data Problem

Even with Firecrawl's basic search capability, you get raw result lists without the rich structured data that powers business applications.

SerpApi returns JSON with specific parsers for each SERP feature. When you query "pizza near me", SerpApi's response includes structured fields for rating, review counts, position ranking, hours of operation, and price indicators. Firecrawl returns basic fields like title, URL, and description.

Example when querying "Pizza Near me"

You'd have to build your own parser for these SERP-specific data points, which is exactly what SerpApi has industrialized across many platforms including Google, Bing, YouTube, Amazon, with 100+ API endpoints.

Website Scraping vs. Search Scraping: A Concrete Example

Here's a real-world scenario that clarifies the distinction:

Use Case: You're building a competitive intelligence dashboard for e-commerce brands. Your users want to know:

Which competitors rank for their target keywords
What prices competitors show in Google Shopping
Which products have featured snippets or rich results
What ad copy competitors use in paid search

Using Firecrawl:

You could scrape your competitors' product pages to extract their prices and descriptions. But you have no way to know which competitors to scrape because you don't know who ranks for your keywords. You'd need a pre-existing list of competitor URLs. Even with that list, you wouldn't see their Google Shopping prices (which often differ from their website prices), their ad copy, or their SERP rankings.

Using SerpApi:

You query SerpApi's Google Shopping API with your target product keyword. You get back every retailer showing prices for that product in Google Shopping, their exact prices, their ratings, their shipping costs, and their position in the results. You access ad data to see which competitors are bidding on that keyword and what ad copy they're using. All structured data, one API call per query.

The first approach gives you website content. The second gives you market intelligence.

The "LLM-Ready" Misconception

Firecrawl markets itself heavily on returning "LLM-ready markdown." That's true, and for content ingestion use cases, it's valuable. If you're building a chatbot that needs to answer questions about your company's documentation, Firecrawl can crawl your docs and return clean markdown.

But "LLM-ready" doesn't mean "search-intelligence-ready." If your LLM-powered application needs to answer questions like "What are the top 5 CRM tools according to Google?" or "Which hotels in Miami are under $200 tonight?", the LLM needs search results data, not website content.

SerpApi's JSON responses are just as "LLM-ready" as Firecrawl's markdown, arguably more so because they're pre-structured. When you integrate SerpApi with OpenAI's Assistant API, your LLM can query real-time search data and receive structured answers that Firecrawl simply cannot provide because it doesn't access Google Maps or structured search results.

SerpApi as LLM's web search API

See how SerpApi can be integrated with various LLMs.

SerpApi vs Firecrawl: Feature-by-Feature Comparison

Here's how SerpApi vs Firecrawl stack up across the dimensions that matter for search intelligence:

Feature	SerpApi	Firecrawl
Primary Function	Scrapes search engine results pages (Google, Bing, etc.)	Scrapes individual website content
Data Type	Search rankings, ads, prices, local results, SERP features	Website text, markdown, HTML content
Search Engine Access	17+ platforms, 100+ API endpoints	Basic search endpoint (not true SERP scraping)
Ranking Data	Full organic and paid ranking positions	Not applicable
Shopping Data	Google Shopping prices, sellers, reviews, positions	Must scrape individual retailer sites
Local Business Data	Google Maps with reviews, hours, photos, Q&A	Must scrape business websites individually
SERP Features	Featured snippets, People Also Ask, Knowledge Graph, 40+ feature types	Not applicable
LLM Integration	Structured JSON for AI consumption	Markdown optimized for LLMs
Response Time	~0.73-1.75 seconds with Ludicrous Speed	Varies by page complexity
Pricing Model	Per search query (starts at $25/month for 1,000 searches)	Per page crawled (starts at $16/month for 3,000 credits)
Use Case Focus	Competitive intelligence, SEO, market research	Content extraction, documentation

Understanding the Pricing Models

The pricing structures reflect the fundamental difference in what each tool does. SerpApi charges per search query because each query hits a search engine and extracts structured results. A single query like "best CRM software" can return many data points (10 organic results, ads, related questions, etc.).

Firecrawl charges per page crawled because it's extracting content from individual URLs. A single-page scrape returns the markdown content of that one page.

They're not comparable on price because they're not comparable on function. If you need SERP data, you cannot use Firecrawl regardless of its pricing.

On top of that, SerpApi gets cheaper as you scale.

When to Use Each Tool

Choose Firecrawl when:

Your data lives on specific websites, and you know the URLs
You need full-text content for LLM consumption
You're building content aggregation or documentation scrapers
Your use case is "scrape these websites"

Choose SerpApi when:

Your data lives in search engine results, and you need to query by keyword
You need search visibility metrics, rankings, or SERP features
You're building SEO tools, price monitoring, or competitive intelligence
Your use case is "what does Google show when I search for X"

Conclusion: Choose Based on Your Data Source

The decision between SerpApi vs Firecrawl isn't about which is "better." It's about which data source your application requires.

The web has two distinct data layers: the content that lives on websites, and the metadata that search engines display about that content. Firecrawl accesses the first. SerpApi accesses the second. For business intelligence, competitive analysis, and understanding market dynamics, the second layer is what matters.

If you're building SEO tools, price monitoring, competitive intelligence, or local business analysis, you need search engine data. That's where SerpApi operates, and that's why it's the tool of choice for developers who need to understand not just what exists on the web, but what search engines choose to show their users.

Ready to build with real search intelligence? Start your free trial and access the data that drives business decisions.

Next Steps

After understanding the fundamental differences between SerpApi and Firecrawl, here are practical ways to get started with real-world search data:

Explore Search Data Types : Try SerpApi's interactive playground to see structured results from Google Search, Google Shopping, Google Maps, and many other platforms. Compare how different search engines structure their results.

Build Rank Tracking : Follow our guide to create a SERP tracking API and monitor keyword positions across locations.

Extract Competitive Intelligence : Learn to scrape Google Ads data to analyze competitor ad copy and bidding strategies. Track Google Shopping results for real-time price intelligence.

Integrate with AI Applications : Connect SerpApi to OpenAI's Assistant API for real-time search capabilities, or build an AI-powered SEO research agent that analyzes search trends automatically.

Access Local Business Data : Master Google Maps scraping for local SEO analysis and competitive research. Build location-based intelligence tools that track reviews, ratings, and local pack rankings.

Also, check various SerpApi's use cases.

SerpApi vs. Exa.ai: The Difference Between "Real-Time Truth" and "Neural Search"

Hilman Ramadhan — Sun, 26 Jul 2026 11:14:06 +0000

If you're building an AI agent or a tool that needs search data, you've probably seen Exa.ai pop up in your research. The marketing positioning is compelling: "AI-native semantic search." But here's the question that matters: does your application need "an" answer or "the" answer?

Exa describes itself as "the first meaning-based web search API powered by embeddings." What this actually means is that Exa searches its own proprietary index of the web using neural embeddings. It's like having a well-organized library catalog. You get results quickly because you're just querying a database.

Exa.ai vs SerpApi

SerpApi scrapes Google, Bing, and other search engines in real-time. It's not searching a catalog. It's reading the live dashboard of what search engines are displaying to users right now - like checking an instrument panel that shows current readings, not historical records. That's why SerpApi delivers real-time search intelligence, not cached snapshots.

This isn't a subtle difference. If you're building a RAG chatbot that needs to answer generic questions like "explain quantum computing," Exa might work fine. But if you're building anything that requires market intelligence, SEO rankings, price monitoring, ad verification, or local search results, you need real-time data that reflects the actual state of search engines. That's what SerpApi delivers.

This article breaks down the technical and business differences between these two approaches so you can make an informed decision.

Also, read the_ _best Web Search API options in 2026.

Data Freshness and "The Truth"

The core architectural difference between Exa and SerpApi is what they're actually querying. Exa uses a proprietary index of the web, trained on embeddings for semantic search. This means Exa crawls the web periodically, builds an index, and then serves queries against that index. The problem with this approach is staleness.

Exa landing page

Here's a concrete example. Let's say a user asks your AI agent, "What's the price of the iPhone 17 right now?" If you query Exa, you might get a blog post that mentions the iPhone 17's price, but that post could be from three days ago. The price might have changed. The product might be out of stock. You won't know because Exa is showing you what was true when it last crawled that page.

If you query SerpApi's Google Shopping API, you get the actual Google Shopping results. You're seeing the same data that appears when a user searches Google Shopping in their browser. That's real-time truth.

Sample iPhone 17 price search on Exa and SerpApi

This matters even more for SEO and local search. If you're building a rank tracking tool, your customers want to know where their site ranks today, not where it ranked when Exa last updated its index. SerpApi captures search results exactly as they appear in real-time, including Google Maps, Local Packs, Knowledge Graphs, and Shopping Carousels. Exa doesn't support any of these structured result types because it's searching its own index, not Google's live results.

Let's look at what data types each service actually supports:

Exa's Coverage:

Text content from web pages
Semantic search across its index
Basic filtering (date, domain, category)

SerpApi's Coverage:

Organic search results from Google, Bing, Yahoo, Baidu, Yandex, DuckDuckGo
Google Maps and Local Pack results
Shopping results (Google Shopping, Amazon, eBay, Walmart, Home Depot)
Image and video results
Knowledge Graphs and answer boxes
News results
Google Flights and Hotels
Google Trends

If your use case requires anything beyond basic text search, Exa won't work. You can't get local business data, you can't track shopping prices, you can't monitor ad placements. Those data types don't exist in Exa's index.

Performance Wars: "Neural Speed" vs. Real-Time Scraping

Critics point out that scraping Google takes longer than querying a database. That's technically true. Exa can be faster because you're just doing a database lookup.

But here's what that criticism misses: SerpApi has closed the performance gap. SerpApi's measured 30-day average response time is 1.09 seconds. For the majority of requests, you're looking at 1-2 second response times.

For speed-sensitive applications, SerpApi offers Ludicrous Speed mode (using 2x server resources) and Ludicrous Speed Max mode (using 4x server resources) that can deliver sub-second response times while maintaining the same real-time data quality. This means you can get speed comparable to Exa's database lookups without sacrificing data freshness.

Let's put this in perspective. If you're building a chatbot that responds to user queries, a 1-2 second difference in response time is usually acceptable. Users expect a slight delay when they ask a question that requires a web search. What they don't expect is outdated or incomplete information.

The real question is: would you rather have a faster response with potentially stale data, or a response with real-time, accurate data? For most enterprise use cases, the answer is obvious.

SerpApi also offers performance optimization options for speed-sensitive applications. The Google Light Fast API can achieve faster response times depending on your specific needs and the complexity of the search query.

Here's the performance breakdown:

SerpApi:

Average response time: ~1.09s (30-day measured average)
Ludicrous Speed modes available for sub-second performance
Success rate: 99.95% SLA

Exa:

Fast mode: 178ms P50, 252 P90

The other performance metric that matters is reliability. SerpApi maintains a 99.95% success rate across all requests. When you make a request, you get a result. That reliability matters when you're building production systems.

Feature-by-Feature Comparison

Here's a direct comparison table to help you evaluate the two services:

Feature	SerpApi	Exa
Data Source	Real-time scraping of 50+ search engines and platforms	Proprietary internal index
Search Engine Coverage	✅ Google, Bing, Yahoo, Amazon, eBay, Walmart, Yandex, Baidu, DuckDuckGo, Naver, and 40+ more	❌ Single proprietary index only
Freshness	Real-time (live mirror of search engines)	Updated every minute (news); incremental for other content
Supported Data Types	Organic results, Maps, Local Pack, Shopping, Images, Videos, Flights, Hotels, Knowledge Graphs, News, Ads	Text content from web pages, semantic search
Structured SERP Features	✅ Knowledge Graphs, Local Pack, Shopping Carousels, Answer Boxes, Featured Snippets, People Also Ask, Related Searches	❌ Limited to text content
E-commerce Platform Support	✅ Amazon, eBay, Walmart, Home Depot APIs	❌ None
Ad Data	✅ Paid search ads included in all results	❌ Not available
Local Business Data	✅ Google Maps, reviews, photos, hours, contact info	❌ Not available
Reliability	99.95% SLA (actual uptime: 99.967%)	Not publicly disclosed
Location Targeting	✅ Precise geo-targeting with 1000+ locations worldwide	✅ Worldwide coverage (less granular)
Device-Specific Results	✅ Mobile, tablet, desktop variations	❌ Not supported
CAPTCHA Handling	✅ Automatic CAPTCHA solving included	N/A (no scraping)
Browser Rendering	✅ Full browser simulation for JavaScript-heavy pages	N/A (index-based)
Historical Data	✅ Search Archive API available	❌ Not available
Caching	✅ Built-in (free repeated queries)	N/A
API Architecture	✅ JSON output, REST API	✅ JSON output, REST API
Client Libraries/SDKs	✅ 10+ languages (Python, Node.js, Ruby, PHP, Java, Go, Rust, .NET, Swift, C++)	✅ Python, JavaScript, curl
No-Code Integration	✅ Google Sheets add-on, Zapier, Make.com, n8n, Bubble	❌ API-only (requires coding)
Interactive Playground	✅ Test queries in browser before coding	✅ Available
Documentation	✅ Comprehensive docs and SDKs	✅ Comprehensive docs and SDKs
Enterprise Options	✅ Custom SLAs available (99.97% for enterprise)	✅ Custom SLAs available
Global Access	✅ Multi-region proxy networks	✅ Worldwide index coverage
Legal Protection	✅ US Legal Shield with up to $2M indemnification (Production+ plans)	❌ None (customers must indemnify Exa)
Pricing (1k searches)	Starts at $75/month for 5,000 searches. Getting cheaper as you you scale	$5 (Basic Search 1-25 results), $15 (Deep Search)
Use Cases	Data for AI Agent, SEO tracking, price monitoring, ad verification, local search, market intelligence, competitive analysis	Generic RAG, content discovery, semantic search, concept exploration

The pricing comparison requires context. Exa is cheaper on a per-search basis for basic semantic queries ($5 per 1,000 searches vs. SerpApi's $10-25 per 1,000 at standard tiers). However, SerpApi gets cheaper as you scale. When you factor in the value of real-time data, comprehensive coverage of SERP features, and legal protection, SerpApi's pricing makes sense for production use cases that require search engine accuracy.

Which API Is Right for You?

Choose Exa if you're building a system that benefits from semantic understanding and concept exploration, where speed is critical and data freshness within minutes or hours is acceptable. Exa works well for RAG applications and content discovery tools.

Choose SerpApi if you're building a business that needs data reflecting the actual state of search engines right now. If you need web search data for AI tools, SEO rankings, product prices, ad placements, local results, or any structured SERP features beyond basic text, SerpApi is the only option. If you're an enterprise that needs legal protection for data collection activities, SerpApi provides the US Legal Shield feature.

The fundamental question is this: are you building a system that needs to understand what the web says about a topic (Exa's strength), or are you building a system that needs to understand what search engines are showing users right now (SerpApi's strength)? For most production applications involving market intelligence or user-facing search data, you need the latter.

Don't settle for a "version" of the web when you need real-time accuracy. SerpApi has provided developers, researchers, and businesses with access to real-time search data for over eight years.

Next Steps

Once you've chosen your search API approach, here are the logical next steps based on your use case:

For SEO and Rank Tracking Projects:

Start building your rank tracking system using SerpApi's SERP Tracking API. This guide walks through creating a white-label rank tracker application that monitors search rankings in real-time.

For E-commerce and Price Monitoring:

Implement automated price tracking with the Amazon Price Tracking tutorial. Learn how to monitor product prices over time and detect market trends across multiple platforms.

For Multi-Platform Search Applications:

If you're migrating from another search API or expanding your data sources, review the Bing Search API Replacement guide to understand how SerpApi provides comprehensive coverage across search engines.

For Local Business Intelligence:

Build location-based applications using the Google Maps Data Scraping tutorial. This covers extracting business listings, reviews, and local pack data for market analysis.

For API Optimization and Cost Management:

Understand how to optimize your search credit usage with the Credit Usage Calculation guide. Learn which parameters affect credit consumption and how to structure queries efficiently.

For Technical Implementation:

Master the full range of search customization options in the Google Search Parameters documentation. This comprehensive reference covers all available parameters for fine-tuning your search queries.

SerpApi vs. Brave Search API: Why "Independent" Data Isn't Enough for Business Intelligence

Hilman Ramadhan — Sat, 25 Jul 2026 11:09:10 +0000

When comparing SerpApi vs Brave Search API, the fundamental question isn't about data quality in isolation; it's about data relevance. The Brave Search API queries a private, independent index of the web containing approximately 20 billion pages. Compare that to Google's index, which spans hundreds of billions of pages. Brave bills itself as privacy-first and independent from Big Tech. It's what you might call a "Sanitized Web", curated, lean, and intentionally isolated from the mainstream search ecosystem.

Brave Search API vs SerpApi

SerpApi takes a different approach. It scrapes the live results from Google, Bing, YouTube, and yes, even Brave if you need it. It's the "Real Web", the messy, commercial, algorithmically-ranked reality that your customers actually see when they search.

For generic retrieval-augmented generation (RAG) tasks like "What is the capital of France?" or "Explain photosynthesis," Brave works fine. The answer doesn't change based on which index you query. But for Business Intelligence questions like "Who is bidding on 'CRM software' in London?" or "What's the average price for wireless earbuds on Google Shopping right now?" Brave becomes statistically irrelevant.

The core issue isn't that Brave is bad. It's that Brave answers a different question. If your business depends on understanding what consumers actually see, where they click, what prices they compare, which local businesses appear in the map pack, then you're not just choosing between two APIs. You're choosing between building on top of the market or building in parallel to it.

The 1% vs. 90% Problem: Understanding Brave Search API Limitations

When evaluating Google Search API vs Brave, the market share statistics tell a sobering story. Google controls approximately 90% of the global search market. Bing sits at around 4%. Brave Search, despite its privacy credentials and growing user base, represents less than 1% of search traffic.

This creates what I call the "Vanity Metric" danger, one of the most critical Brave Search API limitations for commercial applications. Imagine you're building SEO rank tracking APIs for marketing agencies. Your clients want to know where their e-commerce site ranks for "running shoes" in New York. If you build this tool using the Brave Search API, you're reporting on a user base that effectively doesn't exist for most businesses.

Your dashboard shows their client is ranking #3 on Brave. That sounds great until you realize that Brave's <1% market share means the client is visible to a tiny fraction of potential customers. Meanwhile, on Google, where 90% of searches happen and where actual revenue gets generated, that same site might be on page 10. Your tool just gave them false confidence based on irrelevant data.

Example of different search results between Google Search and Brave Search

The problem compounds when you consider the algorithm gap. Brave's ranking algorithm is completely independent from Google's. It doesn't use PageRank, doesn't weight E-E-A-T signals the same way, and doesn't apply the same spam filters or helpful content system updates that Google deploys. The two search engines are fundamentally solving different optimization problems.

This means optimizing for Brave does not help you rank on Google. If you're an SEO tool provider giving recommendations based on Brave data, you're telling your users how to rank on a platform their customers don't use. There's no feedback loop between Brave performance and business outcomes because the traffic isn't there.

Only SerpApi gives you access to the ranking signals, ad positions, and SERP features from the platforms where revenue actually happens. When your customer asks "Why did our traffic drop 40% last week?" you need to see what Google changed in their SERP, not what Brave might hypothetically show to its tiny user base.

The use case for Brave data in SEO tooling is essentially academic. It's interesting for research on algorithmic diversity or privacy-preserving search. But if you're building a business that needs to predict, track, or optimize for search visibility that translates to revenue, you need Google data;. That's not an opinion, it's arithmetic.

The "Rich Result" Gap: Where Brave Search API Limitations Become Critical

Brave is primarily a text-link search engine. You query it, you get a list of URLs with titles and descriptions. For many use cases, that's sufficient. But the modern Google SERP is nothing like a list of ten blue links.

Google's "rich results" now dominate commercial queries. These include local packs, shopping carousels, knowledge panels, video results, featured snippets, and "People Also Ask" boxes. These elements aren't decorative; they're where clicks happen. Featured snippets can capture significant traffic on informational queries, often pulling clicks away from traditional organic results.

Brave has none of this. Here's what SerpApi captures that Brave misses:

Local SEO Data API Capabilities

Local Data: Brave has no equivalent to Google's Local Pack. When you search "coffee shop near me" on Google, you get a map, business hours, real-time busy times, photos, and review scores. SerpApi connects to Google Maps data, pulling these structured signals in real-time. If you're building a local SEO data API solution, a competitive analysis tool for restaurants, or a location intelligence platform, Brave literally cannot provide the data you need. It doesn't have relationships with local business databases or review platforms at the scale Google has built through Google Business Profile.

Shopping Data: Brave has no Shopping Graph. When you search "Sony WH-1000XM5" on Google, you see prices from a dozen retailers, shipping estimates, availability, and product ratings aggregated across the web. SerpApi scrapes Google Shopping to give you real-time price comparisons, seller reputations, and stock levels. If you're building a price monitoring tool, a dropshipping product research platform, or an AI shopping assistant, Brave gives you a link to Sony's website. SerpApi provides you with the competitive market landscape.

Visual Search: Brave's image search is generic web crawl data. SerpApi scrapes Google Lens and reverse image search, which use sophisticated computer vision models to identify products, landmarks, plants, and more. The difference is the layer of machine learning that Google applies to visual data, something Brave's index simply doesn't have at a comparable scale or accuracy.

Real-World Use Case Comparison

Here's a concrete example. You're building an AI travel agent. A user asks, "Find me a hotel in Barcelona near Park Güell under $150 per night."

Using Brave Search API, your agent searches "hotels near Park Güell Barcelona" and gets back a list of hotel websites and booking aggregators. It might scrape a few of those sites to find prices, assuming they're not blocked. You get generic results with no structured data.

Using SerpApi's Google Hotels integration, your agent gets back structured JSON with exact prices per night, availability for specific dates, review scores, photos, amenities, distance to Park Güell calculated by Google Maps, and direct booking links. You can filter by price programmatically before even showing results to the user.

The first implementation requires you to build your own scraping infrastructure for dozens of hotel sites. The second gives you production-ready data in one API call. This isn't a minor difference; it's the difference between a prototype and a product customers will pay for.

Performance & "Ludicrous Speed"

Let's acknowledge the elephant in the room: Brave is fast. Query response times are typically under 500 milliseconds because you're hitting a purpose-built API on Brave's infrastructure. There's no scraping, no proxy rotation, no parsing complexity.

SerpApi is scraping live web pages, which adds latency. That's a trade-off.

But here's the counter: SerpApi's Ludicrous Speed Max brings Google scraping down to approximately 1-2 seconds. For most business intelligence use cases, that's fast enough. If you're running hourly rank checks, daily price monitoring, or enriching a CRM with company data pulled from search, 1-2 seconds is completely acceptable.

Ask yourself this: Is saving 800 milliseconds worth missing 90% of the market? Is it worth losing access to local data, shopping data, and rich results that drive actual business decisions?

For real-time, user-facing search where latency is critical, consider using Brave as a fallback or for generic queries. But for the business intelligence layer, the data that informs product decisions, marketing spend, and competitive strategy, the marginal latency of SerpApi is worth the accuracy and completeness.

Speed is a feature. Relevance is a requirement.

SerpApi vs Brave Search API: Feature-by-Feature Comparison

Here's how SerpApi vs Brave Search API stack up across the dimensions that matter for business intelligence:

Feature	SerpApi	Brave Search API
Primary Data Source	Live scraping of Google, Bing, Yahoo, DuckDuckGo, Yandex, Baidu, YouTube, and 50+ sources	Brave's independent index (~20B pages)
Market Coverage	~90% (Google's market share)	<1% (Brave's market share)
Shopping Data	Full Google Shopping integration with prices, sellers, shipping, reviews	No shopping-specific data
Local Business Data	Google Maps API with reviews, photos, hours, busy times, Q&A	No local pack or structured business data
Rich Results	Featured snippets, People Also Ask, Knowledge Graph, video carousels	Basic organic results only
Legal Protection	$2M indemnification via US Legal Shield	Standard API terms (no scraping protection)
Average Response Time	Depend on the API and the speed mode	<500ms
Pricing	Starts at $75/month for 5,000 searches. Getting cheaper as you you scale	$5 per 1,000 queries (~$0.005 per query)
Supported Search Engines	50+ including Google, Bing, Yahoo, DuckDuckGo, Brave, Yandex, Baidu	Brave only
Image Search	Google Lens, reverse image search with ML identification	Generic image index

Understanding SerpApi Pricing

The pricing difference is real. Brave is more affordable per query at face value. But when evaluating SerpApi pricing, consider the value proposition. If those Brave queries return data that doesn't reflect your customers' reality, you're not saving money; you're buying irrelevant data at a discount. SerpApi's pricing structure reflects the complexity of delivering real-time, structured data from the search engines where actual business happens.

Conclusion: Do You Want "Clean" Data or "Right" Data?

Brave is a great backup search engine. It's a solid training dataset for generic knowledge retrieval. If you're building a RAG system for a chatbot that answers encyclopedia-style questions, Brave will work fine.

But Brave is an island. Its index is independent, its algorithm is independent, and its user base is a rounding error in the global search market. If your business depends on understanding what consumers see when they search, which is Google 90% of the time, you cannot afford to look at a different index.

The data might be clean, but it's not right. It's not representative. It doesn't show you where your customers' ads appear, where their competitors rank, what prices are displayed, or which local businesses get the map pack visibility.

When choosing between Google Search API vs Brave , or more accurately, between SerpApi (which provides Google data) and Brave's independent index, the choice becomes clear. SerpApi is the universal adapter for the actual digital economy. It connects to Google, where the revenue happens, Bing, where enterprise users search, YouTube, where product reviews live, Amazon, where e-commerce pricing is set, and yes, even Brave if you need it for comparison. You're not locked into one provider's view of the web; you get the web as it actually exists across platforms.

If you're building SEO rank tracking APIs , price monitoring , local SEO data API solutions, ad intelligence, or competitive analysis tools, your users don't care about Brave's independent index. They care about Google's SERP because that's where their traffic comes from. Give them data from the source that matters.

See the market as it really is. Get 100% accurate Google data with SerpApi. Start your free trial and query the real web, not a sanitized approximation.

Also read best Web Search API options in 2026.

Next Steps

After understanding the fundamental differences between SerpApi and Brave Search API, here are practical ways to get started with real-world search data:

Start Building with SerpApi : Try SerpApi's interactive playground to test queries and see structured results immediately. Explore our integration libraries for Python, Node.js, PHP, Ruby, Java, and other languages to get production-ready code snippets.

Track Your Rankings : Build a rank tracking application to monitor keyword positions across locations and devices. Learn how to calculate search credit usage to optimize your monitoring strategy.

Extract Competitive Intelligence : Discover how to scrape Google Ads data from competitors and analyze shopping results to understand pricing strategies in your market.

Build Location-Based Tools : Learn to scrape Google Maps data and reviews for local business intelligence, or create a hotel price tracker with our Google Hotels API.

Integrate with AI Workflows : Connect SerpApi to OpenAI's Assistant API for real-time web knowledge, or build an AI-powered SEO research agent that analyzes search trends automatically.

SerpApi is the Web Search API you need for your applications and AI agents.

SerpApi Alternatives: Best Web Search APIs in 2026

Hilman Ramadhan — Fri, 24 Jul 2026 09:39:07 +0000

If you’re building products that rely on web search — AI apps, research tools, lead collectors, price trackers, or internal dashboards — you’ve probably heard of SerpApi. It’s a popular choice because it turns messy search engine pages into clean, structured data with zero scraping pain.

That said, SerpApi isn’t the only option. You may have heard some of the Search API alternatives. In this article, we’ll walk through the most popular SerpApi alternatives as a Web Search API:

Brave Search API
Exa
Tavily
Firecrawl
Serperdev and more

We’ll break down what each tool is good at, who it’s best for, and when you might choose it over SerpApi.

SerpApi alternatives as a Web Search API

Why do many companies choose SerpApi?

Before jumping into alternatives, here’s why developers typically use SerpApi:

Complete Search Engine APIs: We cover more than 100 APIs that give you programmatic access to Google, Bing, DuckDuckGo, etc.
Real-time results: We always perform live scraping for you; it's not coming from a database.
Structured JSON responses (no HTML parsing). No more DIY that forces you to spend time and resources on keeping up to date with the latest layout. Use a simple API, and you are done.
Enterprise-ready: We're SOC2 Type II certified and can handle a high volume of searches for your team.
We've been in business since 2017. So, we faced many issues that our competitor hasn't. We're focusing on scraping search engine results, not just any websites. That keeps us focused on our vision and our daily work.
Location, language, device, and other parameters to narrow your search to what you need.
Use cases: AI agents, SEO tools, market research, price monitoring, and lead collection.

If all of that sounds good to you. You can register for free at serpapi.com

Best Web Search API Options

Now, let's see some options for web search API:

1. SerpApi

SerpApi provides public search results data from various search engines in a structured JSON format. We currently cover more than 100+ APIs.

We don't manipulate the data in any way; we leave it to the developers to decide what to do with it, including providing responses to AI tools for various use cases. No hidden algorithm.

SerpApi - web search API

Pros

Unedited real-time results
Support 100+ APIs, you can pick depending on your application needs
Simple API through a single HTTP GET request
Flexible use cases

Cons

Depending on the volume, SerpApi is not the cheapest option, as it prioritizes result quality.

2. Brave Search API

Brave Search API gives you access to Brave’s own search index, not Google or Bing results.

Pros

Uses Brave’s independent search index
Want an alternative to Google/Bing dependency

Cons

Limited to only Brave's own algorithm and database
Not a drop-in replacement for Google SERP and other engines scraping
Not ideal for product or business research
Less SEO-specific data (ads, rich snippets, local packs, etc.)

Brave Search API alternatives

When to choose Brave over SerpApi

👉 When your apps need generic knowledge from search results and want to be independent from Google, Bing, or other search engines.

3. Exa

Instead of returning raw SERP-like results, Exa focuses on semantic search or "natural language search query".

Pros

Uses Exa's own search index. It can be faster as the source is coming from a database

Cons

Not designed to perform live scraping on search engines
Limited to their own algorithm and database
Not ideal for product or business research
Less useful for SEO monitoring, rank tracking, or ad data

Exa AI Search API Alternatives

When to choose Exa over SerpApi

👉 When you’re building apps that don't really need real-time or raw data from a search engine.

4. Tavily

Tavily takes a different approach. It aggregates content from multiple web sources and processes it on its side before returning the response to the customer.

Pros

Aggregated search results

Cons

Returned results are based on their algorithm
Summarized search responses might not be what people want in many cases
Not ideal for product research or SEO tooling that needs real-time data.

Tavily alternatives

When to choose Tavily over SerpApi

👉 When your app doesn't need the exact live data results from search engines, and you're fine using Tavily's algorithm for the returned results.

5. Firecrawl

Firecrawl is a website content scraper. You point it at a URL, techcrunch.com/article/some-news and it returns clean, LLM-ready markdown of that article.

SerpApi is a search engine scraper. You query it with "best CRM software," and it returns what Google (and other top search engines) shows for that query: the top 10 organic results, the paid ads, the featured snippet, the "People Also Ask" box, and the local map pack.

Pros:

Return the full content of a link

Cons:

You have to supply the individual link/URL
You can't scrape specific search engine results

6. Serperdev

Serperdev is closer to SerpApi in how they return raw search engine results.

Please read our blog post on SerpApi vs Serper:

Alternatives to SerpApi 2026 (Serper, SearchApi, and ValueSERP)

When to choose Serper over SerpApi

👉 When you want something closer to traditional SERP scraping with simpler needs or lighter usage.

Final Thoughts

Every web search API has its place. Tools like Brave, Exa, Tavily, or others shine for specific use cases.

If your product depends on reliable, production-grade SERP data at scale across search, shopping, maps, reviews, and more, long-term stability matters. As you move from prototypes to real users, mature infrastructure, consistent parsers, strong uptime, and predictable data formats become critical.

This is where SerpApi stands out. It’s more than a web search API. It’s a battle-tested web data platform teams trust to power AI features, SEO tools, lead generation, price monitoring, and product research.

If you’re evaluating options, the best move is to try a few APIs side by side. But when reliability, coverage, and developer experience start to matter more than experiments, you’ll likely want something built for the long haul.

How to Scrape Google Flights Deals with a simple API

Hilman Ramadhan — Tue, 14 Jul 2026 06:34:07 +0000

Google Flights Deals helps travelers discover discounted flights without a specific destination. Instead of searching one route at a time, you can start with a departure airport or city and explore discounted destinations.

This data can be useful for building flight-deal trackers, travel discovery tools, price-comparison dashboards, or destination-recommendation apps. In this tutorial, we'll use SerpApi's Google Flights Deals API to retrieve flight deals as structured JSON with cURL, Python, and JavaScript.

Scrape Google Flights Deals website

How to use the Google Flights Deals API

A basic request uses these parameters:

Parameter	Requirement	Description
`engine`	Required	Set to `google_flights_deals`
`api_key`	Required	Your private SerpApi API key
`departure_id`	Optional	Departure airport code or location ID
`currency`	Optional	Currency used for returned prices

An airport ID is normally an uppercase three-letter IATA code such as:

CDG for Paris Charles de Gaulle Airport
AUS for Austin-Bergstrom International Airport
MAD for Adolfo Suárez Madrid-Barajas Airport

You can also provide a Google location ID beginning with /m/ or /g/. Multiple departure locations can be separated with commas.

Get your API key

Create a SerpApi account and retrieve your private key from the API key page.

Replace the following placeholder in each example:

YOUR_SERPAPI_API_KEY

Never expose your API key in public source code or commit it to a public repository.

The complete parameter reference is available in the official Google Flights Deals API documentation.

cURL implementation

Let's retrieve flight deals departing from Paris Charles de Gaulle Airport:

curl --get https://serpapi.com/search \
 -d engine="google_flights_deals" \
 -d departure_id="CDG" \
 -d currency="USD" \
 -d api_key="YOUR_SERPAPI_API_KEY"

The parameters in this request are:

engine=google_flights_deals selects the Google Flights Deals engine.
departure_id=CDG searches for deals departing from Paris Charles de Gaulle Airport.
currency=USD returns prices in US dollars.

The API returns JSON by default.

Python implementation

Create a new Python file:

main.py

Install the requests package:

pip install requests

Add the following code:

import requests


SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "engine": "google_flights_deals",
    "departure_id": "CDG",
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

response = requests.get(
    "https://serpapi.com/search",
    params=params,
    timeout=30,
)
response.raise_for_status()

data = response.json()

if "error" in data:
    raise RuntimeError(data["error"])

departure = data.get("departure_informations", {})

departure_name = (
    departure.get("airport_name")
    or departure.get("city")
    or "Unknown departure"
)

print(f"Flight deals from: {departure_name}")
print()

for deal in data.get("deals", []):
    duration = deal.get("flight_duration")

    if duration is not None:
        hours, minutes = divmod(duration, 60)
        formatted_duration = f"{hours}h {minutes}m"
    else:
        formatted_duration = "Not available"

    print("Destination:", deal.get("name", "Unknown"))
    print("Country:", deal.get("country", "Unknown"))
    print("Price:", deal.get("price", "Not available"))
    print(
        "Average price:",
        deal.get("average_price", "Not available"),
    )
    print(
        "Discount:",
        deal.get("discount_percentage", "Not available"),
    )
    print("Outbound date:", deal.get("outbound_date", "Not available"))
    print("Return date:", deal.get("return_date", "One-way flight"))
    print(
        "Route:",
        f"{deal.get('departure_airport_code', '?')} → "
        f"{deal.get('arrival_airport_code', '?')}",
    )
    print("Duration:", formatted_duration)
    print("Stops:", deal.get("stops", "Not available"))
    print("Airline:", deal.get("airline", "Not available"))
    print("Google Flights:", deal.get("flight_link", "Not available"))
    print("-" * 80)

The code prints selected fields from every item in the deals array. It also converts the documented flight_duration value from minutes into hours and minutes.

Here is the result

Flights Deals API scraper

Fields are accessed with .get() because some values are not guaranteed to appear in every deal.

Export the flight deals to CSV

You can also save the results to a CSV file for further analysis:

import csv


rows = []

for deal in data.get("deals", []):
    rows.append({
        "destination": deal.get("name", ""),
        "country": deal.get("country", ""),
        "destination_id": deal.get("destination_id", ""),
        "price": deal.get("price", ""),
        "average_price": deal.get("average_price", ""),
        "discount_percentage": deal.get(
            "discount_percentage",
            "",
        ),
        "outbound_date": deal.get("outbound_date", ""),
        "return_date": deal.get("return_date", ""),
        "departure_airport": deal.get(
            "departure_airport_code",
            "",
        ),
        "arrival_airport": deal.get(
            "arrival_airport_code",
            "",
        ),
        "flight_duration_minutes": deal.get(
            "flight_duration",
            "",
        ),
        "stops": deal.get("stops", ""),
        "airline": deal.get("airline", ""),
        "airline_code": deal.get("airline_code", ""),
        "google_flights_link": deal.get("flight_link", ""),
        "serpapi_flight_link": deal.get(
            "serpapi_flight_link",
            "",
        ),
    })

fieldnames = [
    "destination",
    "country",
    "destination_id",
    "price",
    "average_price",
    "discount_percentage",
    "outbound_date",
    "return_date",
    "departure_airport",
    "arrival_airport",
    "flight_duration_minutes",
    "stops",
    "airline",
    "airline_code",
    "google_flights_link",
    "serpapi_flight_link",
]

with open(
    "google_flights_deals.csv",
    "w",
    newline="",
    encoding="utf-8",
) as csv_file:
    writer = csv.DictWriter(
        csv_file,
        fieldnames=fieldnames,
    )
    writer.writeheader()
    writer.writerows(rows)

print(
    f"Saved {len(rows)} deals to "
    "google_flights_deals.csv"
)

This creates a file named google_flights_deals.csv with one row for each deal.

Exported result in CSV file

JavaScript implementation

SerpApi provides an official JavaScript package.

Install it with npm:

npm install serpapi

Create an index.js file:

const { getJson } = require("serpapi");

const SERPAPI_API_KEY = "YOUR_API_KEY";

getJson(
  {
    engine: "google_flights_deals",
    departure_id: "CGK",
    currency: "USD",
    api_key: SERPAPI_API_KEY,
  },
  (json) => {
    if (json.error) {
      console.error(json.error);
      return;
    }

    const departure = json.departure_informations || {};
    const deals = json.deals || [];

    console.log(
      `Flight deals from: ${
        departure.airport_name ||
        departure.city ||
        "Unknown departure"
      }\n`
    );

    deals.forEach((deal) => {
      console.log({
        destination: deal.name || "Unknown",
        country: deal.country || "Unknown",
        price: deal.price ?? "Not available",
        averagePrice:
          deal.average_price ?? "Not available",
        discountPercentage:
          deal.discount_percentage ?? "Not available",
        outboundDate:
          deal.outbound_date || "Not available",
        returnDate:
          deal.return_date || "One-way flight",
        route:
          `${deal.departure_airport_code || "?"} → ` +
          `${deal.arrival_airport_code || "?"}`,
        duration: deal.flight_duration ?? "Not available",
        stops: deal.stops ?? "Not available",
        airline: deal.airline || "Not available",
        flightLink: deal.flight_link || "Not available",
      });
    });
  }
);

Run the script:

node index.js

The official SerpApi JavaScript package exposes the getJson function and supports both callbacks and promises.

Customize Google Flights Deals results

The API provides several ways to customize the returned deals. Here are three useful examples.

Search using exact or flexible dates

For an exact round trip, provide one outbound_date and one return_date in YYYY-MM-DD format:

params = {
    "engine": "google_flights_deals",
    "departure_id": "FCO",
    "outbound_date": "2026-08-10",
    "return_date": "2026-08-17",
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

For flexible dates, use two comma-separated dates as the departure window:

params = {
    "engine": "google_flights_deals",
    "departure_id": "BCN",
    "outbound_date": "2026-09-01,2026-09-30",
    "travel_duration": 3,
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

Possible travel_duration values are:

Value	Duration
`1`	One week
`2`	Weekend
`3`	Two weeks

You can alternatively use trip_length for a custom duration:

params["trip_length"] = "5,10"

This requests trips lasting between 5 and 10 days.

travel_duration, trip_length, and return_date cannot be used together. For a round trip with one exact outbound date, one exact return date is required.

Filter by stops and maximum price

Use stops to control how many connections are allowed:

Value	Meaning
`0`	Any number of stops
`1`	Nonstop only
`2`	One stop or fewer
`3`	Two stops or fewer

You can combine it with max_price:

params = {
    "engine": "google_flights_deals",
    "departure_id": "CDG",
    "stops": 1,
    "max_price": 500,
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

This searches for nonstop deals costing no more than 500 in the selected currency.

Include or exclude airlines

Use include_airlines to return deals from selected airlines:

params = {
    "engine": "google_flights_deals",
    "departure_id": "/m/0947l",
    "include_airlines": "W4,FR",
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

The example includes:

W4 — Wizz Air
FR — Ryanair

Multiple airline codes are separated with commas.

You can use exclude_airlines instead when you want to remove specific airlines from the results:

params["exclude_airlines"] = "UA"

The include_airlines and exclude_airlines parameters cannot be used together.

The API also accepts the following alliance values:

STAR_ALLIANCE
SKYTEAM
ONEWORLD

Searching for one-way flight deals

Round trip is the default flight type. Set type to 2 to search for one-way deals:

params = {
    "engine": "google_flights_deals",
    "departure_id": "MAD",
    "type": 2,
    "currency": "USD",
    "api_key": SERPAPI_API_KEY,
}

The supported values are:

Value	Flight type
`1`	Round trip
`2`	One way

For a one-way search with a specific date, provide only outbound_date:

params["outbound_date"] = "2026-09-15"

Do not use return_date, travel_duration, or trip_length with one-way flights. One-way deal results do not contain an return_date.

What data can be retrieved from the Google Flights Deals API?

The API returns information about the departure location and a list of available flight deals.

The departure_informations object can include:

Departure airport name and IATA code
Departure city and city ID
Departure country
Airport latitude and longitude

The deals array can include:

Destination name and country
Destination Google Knowledge Graph ID
Deal price
Average price for the route
Discount percentage
Outbound and return dates
Departure and arrival airport codes
Flight duration in minutes
Number of stops
Airline name and code
Destination description and highlights
Destination thumbnail
Direct Google Flights link
SerpApi link for retrieving detailed flight information

Some fields, including average_price, discount_percentage, airline, and description, are not always present. The return_date field is also omitted for one-way flights.

Here is a simplified example of a deal:

{
  "destination_id": "/m/0k02q",
  "name": "São Vicente",
  "country": "Cape Verde",
  "price": 468,
  "average_price": 2023,
  "discount_percentage": 77,
  "outbound_date": "2026-09-21",
  "return_date": "2026-09-27",
  "departure_airport_code": "CDG",
  "arrival_airport_code": "VXE",
  "flight_duration": 350,
  "stops": 0,
  "airline": "Cabo Verde Airlines",
  "airline_code": "VR",
  "flight_link": "https://www.google.com/travel/flights?..."
}

The main results are available inside the deals array.

Scraping Google Flights

If you're interested in scraping the regular Google Flights website, we also have an API for that: Google Flights API

Here is the blog tutorial on how to scrape Google Flights.

Here is a video tutorial if you prefer to watch

https://www.youtube.com/embed/zqZSvuj7JMs?feature=oembed

Conclusion

The SerpApi Google Flights Deals API makes it possible to retrieve Google Flights deal discovery results as structured JSON.

With one request, you can collect destinations, deal prices, average prices, discount percentages, travel dates, airport codes, airlines, flight durations, stops, and links to Google Flights.

You can also customize the search using exact dates, flexible date ranges, trip durations, maximum prices, stop limits, travel classes, and airline filters.

For the complete and current parameter list, review the official Google Flights Deals API documentation.

Grok AI API tutorial - from Chat to Web Search

Hilman Ramadhan — Mon, 30 Mar 2026 02:03:37 +0000

The xAI Grok API provides access to powerful frontier models like Grok 4 series, supporting chat completions (text + vision), image generation, tool calling (function calling + built-in tools like web search), and more advanced features.

Quick Intro

Sign up at https://x.ai/api
Generate an API key from the console
Install: pip install xai-sdk
Set env var: export XAI_API_KEY="your_key_here"
Models list: https://docs.x.ai/developers/models

I'll share some samples in Python.

Basic Chat API Call

Let's first prepare our project before making the API call

Install the xai-sdk

pip install xai-sdk

Set env var: export XAI_API_KEY="your_key_here" or use .env file

Now, create a new file and this basic setup:

import os
from xai_sdk import Client
from xai_sdk.chat import user, system

from dotenv import load_dotenv
load_dotenv()
XAI_API_KEY =  os.environ.get("XAI_API_KEY")

client = Client(api_key=XAI_API_KEY)

Ensure you can print out your XAI_API_KEY correctly at this stage.

Next, let's call the chat function

...
model = "grok-4-1-fast-non-reasoning"
chat = client.chat.create(model=model)
chat.append(system("You are Grok, a highly intelligent, helpful AI assistant."))
chat.append(user("How can I be a good developer?"))

response = chat.sample()
print(response.content)

Feel free to switch the model based on your needs or preferences.

Here is an example output:

Image Generation API

Let's see how to generate an image with Grok API. We'll need to use the "grok-imagine-image" model for this.

...
response = client.image.sample(
    model="grok-imagine-image",  
    prompt="detective cat searching on website"
)

print(f"Generated image: {response.url}")

The output is a URL like this:

Video Generation API

Generating a video is as easy as generating an image with Grok API. We'll need to use the "grok-imagine-video" model for this.

response = client.video.generate(
    prompt="A glowing crystal-powered rocket launching from the red dunes of Mars, ancient alien ruins lighting up in the background as it soars into a sky full of unfamiliar constellations",
    model="grok-imagine-video",
    duration=10,
    aspect_ratio="16:9",
    resolution="720p",
)
print(response.url)

You can set the duration, aspect ratio, and resolution.

The xAI Grok API features powerful tool calling capabilities, allowing Grok to go far beyond simple text generation. It can take real actions such as performing web searches, running code, retrieving information from your own data sources, or invoking any custom functions you've defined.

Let's start by calling a custom function, as it'll help us call any internal or external API or function.

Let's say we want to call a function to look for an item's price. First, we need to define the function, such as adding the name, description, and parameters.

...
import json
from xai_sdk.chat import user, tool, tool_result
...

# Define tools
tools = [
    tool(
        name="get_item_price",
        description="Get the price of an item from the store",
        parameters={
            "type": "object",
            "properties": {
                "item_name": {"type": "string", "description": "Name of the item to get the price for"},
            },
            "required": ["item_name"]
        },
    ),
]

Upon calling the client method, we now need to include the tool we declared above.

chat = client.chat.create(
    model="grok-4.20-reasoning",
    tools=tools,
)
chat.append(user("What is the price of a laptop?"))
response = chat.sample()

print("========= response ===========")
print(response)
print("==========================")

Important: At this stage, Grok doesn't care if we have the actual function to check the price or not. The AI simply wants to know "what tools are available" for them to use.

Try to run the code to see the output from the chat call.

As you can see, Grok can detect the tool we need to call. You can see it from outputs > message > tool_calls . It consists of the name of the function and the arguments that are extracted from the user's prompt, so it'll be dynamic.

Function call simulation

Next, let's create a fake function to call. In real life, it could be a call to a database or APIs.

def get_item_price(item_name):
    prices = {
        "laptop": 999.99,
        "smartphone": 499.99,
        "headphones": 199.99,
    }
    return {"item_name": item_name, "price": prices.get(item_name, "Item not found")}

Following up from the latest code, we can check if the response has a "tool_calls" object or not. If so, we'll call the actual function we just declared above.

# Handle tool calls
if response.tool_calls:
    chat.append(response)
    for tc in response.tool_calls:
        args = json.loads(tc.function.arguments)
        result = get_item_price(args["item_name"])
        chat.append(tool_result(json.dumps(result)))


    response = chat.sample()

print(response.content)

We need to loop through the tool_calls object
We need to extract the argument to pass to the function
Call the actual function alongside the argument value
Add the information back to our chat method

Now, calling the chat.sample() method, will include all the information we received from calling the "fake function" before.

Let's try with a different prompt

chat.append(user("I need to buy two laptops and a smartphone. Can you tell me how much that will cost?"))

Here is the result

Web Search API

Grok can access real-time information through this feature, so you can get up-to-date content. Unlike the function calling above, we don't need to declare a custom function, as it's an internal tool. Here is a simple example:

import os
from xai_sdk import Client
from xai_sdk.chat import user
from xai_sdk.tools import web_search

from dotenv import load_dotenv
load_dotenv()
XAI_API_KEY =  os.environ.get("XAI_API_KEY")

client = Client(api_key=XAI_API_KEY)

chat = client.chat.create(
    model="grok-4.20-reasoning",  # reasoning model
    tools=[web_search()],
    include=["verbose_streaming"],
)

chat.append(user("Grok VS OpenAI API"))

is_thinking = True

for response, chunk in chat.stream():
    for tool_call in chunk.tool_calls:
        print(f"\nCalling tool: {tool_call.function.name} with arguments: {tool_call.function.arguments}")
    if response.usage.reasoning_tokens and is_thinking:
        print(f"\rThinking... ({response.usage.reasoning_tokens} tokens)", end="", flush=True)
    if chunk.content and is_thinking:
        print("\n\nFinal Response:")
        is_thinking = False
    if chunk.content and not is_thinking:
        print(chunk.content, end="", flush=True)

print("\n\nCitations:")
print(response.citations)

Use tools=[web_search()]
To show what's happening in the process, we use include=["verbose_streaming"],
is_thinking variable is to check if the process is still running (a boolean variable)

As you can see, it'll perform several searches on the internal with different queries. It'll then visit a specific URL after that to get more context.

Allowed domains

You can search only in specific domains using allowed_domains.

 tools=[
        web_search(allowed_domains=["grokipedia.com"]),
    ],

Exclude Domains

Vice versa, you can exclude specific domains

chat = client.chat.create(
    model="grok-4.20-reasoning",
    tools=[
        web_search(excluded_domains=["grokipedia.com"]),
    ],
)

Better Web Search API

While you can specifically choose the domain, the keyword Grok uses to find answers on the internet is random. For example, when I'm asking for "Top 3 pizza restaurants from Google Maps in Boston. Share some reviews and ratings for each place."

This is what I saw from the thinking process:

It needs to perform multiple queries before returning the answer.

Another sample, when asking simply for 3 images:

It runs on multiple pages, and unfortunately, the links are not valid. Grok may hallucinate at this point.

Web Search API alternative

In some cases, AI-generated keywords are fine, but if you're building an app where you want efficiency and full control of the process, the native "Web Search Tool" can be replaced with a simple API call to a specific API your app needs. For example, to find answers on the internet, SerpApi provides 100+ APIs for you.

Need a generic Google answer? We have:

- Google Search API

- Google AI Overview

- Google AI Mode

and more!

See how SerpApi is the Web Search API for your AI apps, LLM, and agents.

Using Grok API with SerpApi

To get some idea of how SerpApi works, feel free to test the results on our playground. You can play with different parameters and directly see the JSON sample we return.

Sample case

Let's say we want to find images via Google Image API like this:

Step 1: Preparation

You can register for free at serpapi.com to get your API key.

Step 2: Parsing keyword

Let's say we need 3 images from Google. Since users can type anything, we need to parse the keyword, as SerpApi simply performs a search using a particular keyword.

USER_QUERY = "Show me 3 cute cat images from the internet"

# Step 1: Ask Grok to extract a search keyword from the user's natural language
keyword_chat = client.chat.create(model="grok-3-fast")
keyword_chat.append(system("Extract the most relevant search keyword or phrase from the user's message. Reply with only the keyword, nothing else."))
keyword_chat.append(user(USER_QUERY))

keyword_response = keyword_chat.sample()
search_keyword = keyword_response.content.strip()
print(f"Extracted keyword: {search_keyword}")

Step 3: Search via SerpApi

We now have the keyword. Let's run a search on SerpApi

# Step 2: Search via SerpAPI using simple requests (Google Images)
serpapi_params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "google_images",
    "q": search_keyword,
    "hl": "en",
    "gl": "us",
}

serpapi_url = "https://serpapi.com/search"
serpapi_response = requests.get(serpapi_url, params=serpapi_params)
results = serpapi_response.json()

At this stage, you already have the answers you're looking for.

Step 4: (Optional) Filter results

Sometimes, we don't need all the information. It's good to filter it programmatically first, so we don't use too many tokens.

For example, I'm only interested in the top 5 answers:

image_results = results.get("images_results", [])
[:5]
formatted_results = "\n".join(
    f"- {img.get('title', 'No title')}: {img.get('original', img.get('thumbnail', 'No URL'))}"
    for img in image_results
)
print(f"\nSerpAPI results:\n{formatted_results}")

We can also format the answer as a bonus.

Step 5: (Optional) Reply in a natural language

Depending on your application, you may want to answer the user back in natural language. We just need to pass the answers above back to the AI:

# Step 3: Feed results back to Grok for a final response
final_chat = client.chat.create(model="grok-3-fast")
final_chat.append(system("You are a helpful assistant. Use the provided search results to answer the user's question."))
final_chat.append(user(f"User question: {USER_QUERY}\n\nSearch results from SerpAPI:\n{formatted_results}\n\nPlease answer the user's question based on these results."))

final_response = final_chat.sample()
print(f"\nFinal Response:\n{final_response.content}")

Final result:

Sidenote

It's also possible to call the API with the OpenAI SDK. Sample:

from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1",
)

How to scrape 100 search results on Google (num parameter)

Hilman Ramadhan — Thu, 25 Sep 2025 01:48:54 +0000

Finally, there is a solution to scrape 100 search results from Google after they dropped the num parameter: introducing the Google Fast Light API

Google recently discontinued support for showing up to 100 results per page in search, a parameter that many SEO professionals, data companies, and researchers have relied on for years. This change disrupts established workflows, making large-scale data collection, competitor analysis, and keyword research less efficient.

API to scrape 100 search results from Google

Introducing Google Light Fast API

With this API, you can scrape 100 search results from Google with one single request. This API only contains the organic_results , which makes it faster compared to our regular Google Search API.

You can access this API with a single GET request:

https://serpapi.com/search.json?engine=google_light_fast&q=coffee&api_key=YOUR_API_KEY&num=100

You can replace coffee with any keyword you want to search for.
To get your API key, you can register for free at SerpApi.

Scraping other rich results from Google

If you need the complete Google results on the first page, you can still use our Google Search API in order to scrape AI Overview answer, knowledge graph, and so on. So, if you need both 100 organic results and complete rich results , you only need to perform two searches:

1 to get the 100 organic results via the new Google Fast Light API
2 to get the rich results on the Google Search API.

No need to paginate the results or run 10 searches anymore.

Conclusion

The removal of the num=100 (&num=100) parameter from Google has a lot of impacts on many SEO companies that need to serve ranking data for their clients. Many search API providers fail to scrape 100 search results, which results in many errors and misinformation on their platform.

This issue was also mentioned a couple of times on X (Twitter) by many search engine professionals,

Barry Schwartz - founder of the Search Engine Roundtable

Brodie Clark - Independent SEO consultant

SEO companies looking for a way to track their clients’ SEO rankings programmatically, as well as AI companies seeking more real-time data from Google, should try this new API.

How to scrape Tripadvisor (2025 Tutorial)

Hilman Ramadhan — Fri, 19 Sep 2025 02:17:42 +0000

Scraping Tripadvisor listings allows you to gather detailed information about various attractions, hotels, or restaurants, including descriptions, amenities, pricing, and user-generated ratings. This data can be instrumental in understanding market trends, identifying competitors, and optimizing your own listings or offerings in the travel industry.

Luckily, you can do this easily using our brand new Tripadvisor API. We'll see how to scrape it in cURL, Python, and JavaScript. We can scrape restaurants, things to do, hotels, destinations, vacation rentals, and forum information.

Available data on the Tripadvisor Search API

Here is the list of data you can retrieve from this Tripadvisor Search API:

title
description
rating
reviews
location
thumbnail
highlighted overview

This is perfect if you need to collect place data from Tripadvisor.

How to scrape the Tripadvisor website?

Now, let's see how to use this simple API from SerpApi to collect the data!

Get your API Key

First, ensure you register at serpapi.com to get your API Key. You can get 250 free searches per month. You can use this API Key to access all of our APIs, including the Tripadvisor Search API.

Available parameters

On top of running the basic search, you can see all of our available parameters here.

cURL Implementation

Here is the basic implementation in cURL:

curl --get https://serpapi.com/search \
 -d api_key="YOUR_API_KEY" \
 -d engine="tripadvisor" \
 -d q="Rome"

The q parameter is responsible for the search query.

Sample response from cURL request

Scrape Tripadvisor search results in Python

Next, let's see how to scrape the Tripadvisor search results in Python.

Preparation for accessing the SerpApi API in Python

Create a new main.py file
Install requests with:

pip install requests

Here is what the basic setup looks like:

import requests
SERPAPI_API_KEY = "YOUR_REAL_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, #replace with your real API Key
    # soon
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

With these few lines of code, we can access all of the search engines available at SerpApi, including the Tripadvisor Search API.

import requests
SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "tripadvisor",
    "q": "indonesia"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

To make it easier to see the response, let's add indentation to the output.

import json

# ...
# ...
# all previous code

print(json.dumps(response, indent=2))

Running this Python file should show you the listing for that keyword from Tripadvisor website.

Print specific information

Let's say we only need the title, description, rating, and reviews. This is how we can print specific columns from the locations results:

# ...
search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

for result in response.get("locations", []):
    title = result.get("title")
    rating = result.get("rating")
    reviews = result.get("reviews")
    description = result.get("description")

    print(f"Title: {title}")
    print(f"Rating: {rating}")
    print(f"Reviews: {reviews}")
    print(f"description: {description}")
    print("-" * 10)

Export data to a CSV file

Let's see how to export this Tripadvisor product data into a CSV file in Python

# ...

import csv

with open("tripadvisor_results.csv", "w", newline="", encoding="utf-8") as csvfile:
    fieldnames = ["title", "rating", "reviews", "description"]
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

    writer.writeheader()
    for result in response.get("locations", []):
        writer.writerow({
            "title": result.get("title"),
            "rating": result.get("rating"),
            "reviews": result.get("reviews"),
            "description": result.get("description")
        })
print("Data exported successfully.")

JavaScript implementation

Finally, let's see how to scrape the Tripadvisor search results in JavaScript.

Install the serpapi package:

npm install serpapi

Run a basic query:

const { getJson } = require("serpapi");
getJson({
  engine: "tripadvisor",
  api_key: API_KEY, // Put your API Key
  q: "indonesia"
}, (json) => {
  console.log(json["locations"]);
});

Other programming languages

While you can use our APIs using a simple GET request with any programming language, you can also see our ready-to-use libraries here: SerpApi Integrations.

How to customize the search?

We provide many filters to customize your search.

tripadvisor_domain: Parameter defines the Tripadvisor domain to use. It defaults to tripadvisor.com. Head to Tripadvisor domains for a full list of supported domains.
lat and lon: Specify the lat and lon GPS coordinates when performing the search

Example using a different Tripadvisor domain:

curl --get https://serpapi.com/search \
 -d api_key="YOUR_API_KEY" \
 -d engine="tripadvisor" \
 -d q="cafe" \
 -d tripadvisor_domain="wwww.tripadvisor.ca"

Advanced Parameters:

ssrc: This parameter specifies the search filter you want to use for the Tripadvisor search.

Available options:

a - All Results

r - Restaurants

A - Things to Do

h - Hotels

g - Destinations

v - Vacation Rentals

f - Forums

How to paginate the results?

You can scrape beyond the first page using the offset and limit parameter.

limit defines how many results you want to receive per page. By default, it's 30. Maximum value is 100.
Offset skips the given number of results. For example, 0 for the first page, 30 for 2nd page, 60 for 3rd page, and so on. Or if you're using 100 as the limit, then the 2nd page will be 100, 3rd page will be 200, and so on.

Frequently Asked Questions (FAQs)

Is it legal to scrape the Tripadvisor website?

Scraping publicly available data from websites like Tripadvisor is generally permitted under U.S. law.

How much does it cost?

Register at serpapi.com to start for free. If you want to scale, we offer tiered plans based on your usage.

Why do you need to scrape Tripadvisor listing?

Scraping Tripadvisor listings can provide valuable data for market analysis, competitive insights, and understanding customer preferences in the travel and hospitality industry. This information can help businesses optimize their offerings and improve customer engagement.

Closing

That's it! Thank you very much for reading this blog post. You can play around for free on our playground here.

Scrape YouTube videos in Python

Hilman Ramadhan — Fri, 08 Aug 2025 08:10:37 +0000

Scraping YouTube videos enables developers and businesses to extract detailed YouTube video metadata at scale, including titles, descriptions, view counts, thumbnails, channel names, related videos, and comments/replies. It streamlines what would otherwise require complex scraping and anti‑blocking measures.

Get your API Key

First, ensure to register at SerpApi to get your API Key. You can get 250 free searches per month. Use this API Key to access all of our APIs, including the YouTube Video API.

Available parameters

In addition to running the basic search, you can view all YouTube Video API parameters here.

How to scrape YouTube video data with Python

Create a new main.py file
Install requests with:

pip install requests

Here is what the basic setup looks like:

import requests
SERPAPI_API_KEY = "YOUR_REAL_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, #replace with the actual API Key
    # soon
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

With these few lines of code, we can access all of the search engines available at SerpApi, including the YouTube Video API.

import requests
SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "youtube_video",
    "v": "j3YXfsMPKjQ" # YouTube video ID
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

To make it easier to see the response, let's add indentation.

import json

# ...
# ...
# all previous code

print(json.dumps(response, indent=2))

Here is the result:

Here is how to scrape the comments on a video.

From the request we performed previously, you should be able to see this in the response.

We can scrape the "Top comments" or "Newest first" comments using the token that is available for each. We can put this token in the next_page_token parameter.

Here is an example:

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "youtube_video",
    "v": "j3YXfsMPKjQ", # YouTube video ID
    "next_page_token": "Eg0SC2ozWVhmc01QS2pRGAYyOCIRIgtqM1lYZnNNUEtqUTABeAIwAUIhZW5nYWdlbWVudC1wYW5lbC1jb21tZW50cy1zZWN0aW9u"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(json.dumps(response, indent=2))

Here is the result:

If you're interested in scraping the reply on the comment, you can repeat the same action, but this time using the replies_next_page_token.

Bonus

You can also scrape YouTube search results using our YouTube Search API. Here is how to scrape YouTube search results using Python.

Why Use It?

Real‑time data: Fetch up‑to‑date video details.
Rich metadata access: Retrieve comments and nested replies, related videos, and viewer stats in one request.
Automated proxy and captcha handling: SerpApi manages the complexities of scraping so you can focus on analysis.

Key Features

Feature: Comprehensive video metadata
- Benefit: Includes title, description, duration, chapters, views, publishing date, channel, thumbnails
Feature: Comments & replies
- Benefit: Access nested comments for engagement analysis
Feature: Related videos
- Benefit: Discover context and competition around a video