How to scrape Walmart products with Python

Scraping Walmart product pages becomes more useful when you can collect information beyond the price. You may need the product name, brand, seller, availability, condition, and ratings, especially when working with multiple products.

In this tutorial, you'll learn how to scrape Walmart products with Python, retrieve their details as structured JSON, and export the results to a CSV file. We'll also handle failed URLs so that one unavailable product doesn't interrupt the entire process.

We'll work with real product data from Walmart and use the Spidra Python SDK to retrieve it.

What data can you extract from Walmart?

A Walmart product page contains several pieces of information you can collect:

• Product information: Title, brand, product ID, description, and specifications.
• Pricing: Current price, currency, and available offers.
• Seller details: The seller offering the product and, where available, alternative sellers.
• Availability: Whether the product is in stock.
• Product condition: New, open-box, restored, or another condition shown on the listing.
• Reviews: Ratings, review counts, and customer feedback where available.

One detail to understand is that a Walmart product can have multiple offers.

For example, consider the Sony WH-1000XM5 wireless headphones. The product data collected on October 28, 2026, showed a featured price of $298 from Focus Camera. The same listing also had another new offer at $288 and lower-priced open-box and restored offers.

If you're collecting product data for price comparisons, these distinctions matter. A restored product priced at $159.99 shouldn't automatically be treated as the cheapest new version of the product.

For this tutorial, we'll collect the featured price along with the seller and condition. The additional offers remain available in the JSON response if you need to analyze them separately.

Set up your Python project

You'll need Python 3.9 or later, a code editor, and a Spidra API key.

You can use Python libraries such as Requests and Beautiful Soup to parse HTML when a website serves the information you need. However, building a scraper around HTML selectors also means maintaining those selectors as the website changes.

For this example, we'll use Spidra because it returns Walmart product data as structured JSON. This lets us focus on processing the information rather than writing HTML parsing logic.

Install the Spidra Python SDK

Create a directory for your project and move into it:

mkdir walmart-scraper
cd walmart-scraper

Install the Python SDK:

pip install spidra

Next, generate an API key from the Spidra dashboard.

Store the key in an environment variable rather than adding it directly to your Python code.

On macOS or Linux:

export SPIDRA_API_KEY="spd_YOUR_API_KEY"

On Windows PowerShell:

$env:SPIDRA_API_KEY="spd_YOUR_API_KEY"

Now we're ready to extract our first product.

Scrape your first Walmart product

Let's start with the Sony WH-1000XM5 headphones.

The product URL is:

https://www.walmart.com/ip/386006068

Create a file named scrape_walmart.py and add the following code:

# scrape_walmart.py
import os
from spidra import Spidra

client = Spidra(api_key=os.environ["SPIDRA_API_KEY"])

url = "https://www.walmart.com/ip/386006068"

result = client.scrape(url, output="json")

product = result.content

print(product["title"])
print(f"${product['price']:.2f} {product['currency']}")
print(product["seller"]["name"])

Run the script:

python scrape_walmart.py

For the product data collected on October 28, 2026, the output is:

Sony WH-1000XM5 The Best Wireless Noise Canceling Headphones, Black
$298.00 USD
Focus Camera

The client.scrape() method accepts the product URL and returns the completed scraping result.

Setting output="json" returns structured product information rather than the page's Markdown content. For supported Walmart product pages, the fields are already named, so we don't need to define a JSON schema or write an extraction prompt.

The returned price reflects the listing at the time of collection. Running the same request later may return a different value.

Inspect the JSON response

So far, we've printed only three fields. Let's examine the data itself.

Replace the three print() statements with:

import json

print(json.dumps(product, indent=2))

The product response contains information similar to this excerpt:

{
  "type": "product",
  "usItemId": "386006068",
  "title": "Sony WH-1000XM5 The Best Wireless Noise Canceling Headphones, Black",
  "brand": "Sony",
  "price": 298,
  "currency": "USD",
  "condition": "New",
  "inStock": true,
  "seller": {
    "name": "Focus Camera"
  },
  "marketplace": "walmart.com"
}

This is a shortened version of the product's actual response. The full result includes additional product details and offer information.

Notice that the seller is a nested object. That's why we use product["seller"]["name"] to access the seller's name.

The price field is numeric, which means you can use it directly for calculations without first removing a dollar sign from a string.

The response also includes the Walmart item ID, 386006068. This gives us a stable identifier for the listing, which is useful when saving product information to a database or joining it with other datasets.

Extract the fields you need

When you're working with product data, you probably won't need every field returned by the API.

For example, if you're building a product catalog, the title, brand, price, seller, availability, and product URL may be enough.

Let's extract those fields into a smaller Python dictionary.

product_data = {
    "id": product.get("usItemId"),
    "title": product.get("title"),
    "brand": product.get("brand"),
    "price": product.get("price"),
    "currency": product.get("currency"),
    "condition": product.get("condition"),
    "in_stock": product.get("inStock"),
    "seller": (product.get("seller") or {}).get("name")
}

print(json.dumps(product_data, indent=2))

For the Sony product, the resulting record is:

{
  "id": "386006068",
  "title": "Sony WH-1000XM5 The Best Wireless Noise Canceling Headphones, Black",
  "brand": "Sony",
  "price": 298,
  "currency": "USD",
  "condition": "New",
  "in_stock": true,
  "seller": "Focus Camera"
}

We're using .get() here because some product fields may be missing or return null.

For example, a product may not have a price available when it is out of stock. Handling missing values early makes the data easier to process later.

At this point, we can extract a product and transform the response into a format suitable for our application.

Next, let's collect multiple products.

Scrape multiple Walmart product URLs with Python

If you have dozens or hundreds of Walmart product URLs, calling client.scrape() manually for each one isn't convenient.

Instead, we'll create a CSV file containing the URLs we want to process and use Spidra's Batch scrape method to retrieve their data.

Batch scraping processes up to 50 URLs in one request, returning a separate result for each URL.

Create a CSV file with product URLs

Create a file named walmart_urls.csv:

url
https://www.walmart.com/ip/386006068
https://www.walmart.com/ip/20182712890

The first URL points to the standard Sony WH-1000XM5 listing.

The second points to a bundle containing the headphones and additional accessories. It gives us another product record to collect while demonstrating why similar product names don't necessarily represent identical items.

You can add more authorized product URLs to the file, one per row.

Read the URLs from the CSV file

Python includes a csv module, so we don't need to install another library.

Here's how to read the URLs:

import csv

with open("walmart_urls.csv", newline="", encoding="utf-8") as file:
    reader = csv.DictReader(file)
    urls = [row["url"] for row in reader]

print(urls)

The result is a Python list containing the two product URLs:

[
    "https://www.walmart.com/ip/386006068",
    "https://www.walmart.com/ip/20182712890"
]

We can now pass that list to the Batch scrape method.

Retrieve multiple product pages

The basic batch request looks like this:

batch = client.batch_scrape(urls, output="json")

for item in batch.items:
    if item.status == "completed":
        print(item.url, item.result)
    else:
        print(item.url, item.error)

Each item includes the URL, its status, and either the result or an error.

Notice that we check item.status before accessing the product data.

A batch can finish processing even when some individual URLs fail. For example, a product page might have been removed or may no longer be available.

Instead of discarding the entire batch, we can save the successful products and record the URLs that failed.

Let's put everything together.

Build a complete Walmart product scraper

We'll now create a script that reads product URLs from CSV, collects their data, and exports two files:

• walmart_products.csv, containing successful product records.
• failed_urls.csv, containing URLs that couldn't be processed and their errors.

The script also removes duplicate URLs from the input file and processes the URLs in groups of up to 50.

Create walmart_products.py:

# walmart_products.py
import csv
import json
import os
from decimal import Decimal, InvalidOperation
from pathlib import Path

from spidra import Spidra, SpidraError

INPUT = Path("walmart_urls.csv")
OUTPUT = Path("walmart_products.csv")
FAILED = Path("failed_urls.csv")

COLUMNS = [
    "item_id", "title", "brand", "price", "currency",
    "condition", "in_stock", "seller", "rating",
    "rating_count", "url",
]


def price_text(value):
    if value is None or isinstance(value, bool):
        return ""

    try:
        price = Decimal(str(value))

        if not price.is_finite():
            return ""

        return f"{price:.2f}"
    except (InvalidOperation, ValueError):
        return ""


def product_row(data, url):
    if not isinstance(data, dict) or data.get("type") != "product":
        raise ValueError("Expected Walmart product data")

    seller = data.get("seller") or {}

    seller_name = (
        seller.get("name", "")
        if isinstance(seller, dict)
        else str(seller)
    )

    return {
        "item_id": data.get("usItemId") or "",
        "title": data.get("title") or "",
        "brand": data.get("brand") or "",
        "price": price_text(data.get("price")),
        "currency": data.get("currency") or "",
        "condition": data.get("condition") or "",
        "in_stock": data.get("inStock"),
        "seller": seller_name,
        "rating": (
            data.get("rating")
            if data.get("rating") is not None else ""
        ),
        "rating_count": (
            data.get("ratingsCount")
            if data.get("ratingsCount") is not None else ""
        ),
        "url": url,
    }


def main():
    with INPUT.open(newline="", encoding="utf-8") as source:
        urls = list(dict.fromkeys(
            row["url"].strip()
            for row in csv.DictReader(source)
            if row.get("url") and row["url"].strip()
        ))

    if not urls:
        raise ValueError("No product URLs found in walmart_urls.csv")

    client = Spidra(api_key=os.environ["SPIDRA_API_KEY"])
    products, failures = [], []

    for offset in range(0, len(urls), 50):
        group = urls[offset:offset + 50]

        try:
            batch = client.batch_scrape(group, output="json")
        except SpidraError as exc:
            failures.extend(
                {"url": url, "error": str(exc)}
                for url in group
            )
            continue

        for item in batch.items:
            if item.status != "completed":
                failures.append({
                    "url": item.url,
                    "error": str(item.error or item.status),
                })
                continue

            try:
                data = item.result

                if isinstance(data, str):
                    data = json.loads(data)

                products.append(product_row(data, item.url))
            except (TypeError, ValueError) as exc:
                failures.append({
                    "url": item.url,
                    "error": str(exc),
                })

    with OUTPUT.open("w", newline="", encoding="utf-8") as dest:
        writer = csv.DictWriter(dest, fieldnames=COLUMNS)
        writer.writeheader()
        writer.writerows(products)

    with FAILED.open("w", newline="", encoding="utf-8") as dest:
        writer = csv.DictWriter(
            dest, fieldnames=["url", "error"]
        )
        writer.writeheader()
        writer.writerows(failures)

    print(f"Saved {len(products)} products to {OUTPUT}")
    print(f"Saved {len(failures)} failed URLs to {FAILED}")


if __name__ == "__main__":
    main()

Run the script:

python walmart_products.py

The script writes the successful product data to walmart_products.csv, with a separate file for failures.

Here's what the main columns contain for the Sony listing from our earlier product data:

item_id	title	price	condition	seller
386006068	Sony WH-1000XM5 The Best Wireless Noise Canceling Headphones, Black	298.00	New	Focus Camera

The full CSV also includes the brand, currency, stock status, rating, rating count, and source URL where those fields are available.

The resulting file can be opened in Excel, loaded into pandas, or imported into a database.

How the script handles multiple products

There are a few details in the code worth understanding.

First, we use dict.fromkeys() to remove duplicate URLs while preserving their original order.

Next, we split the list into groups of 50:

for offset in range(0, len(urls), 50):
    group = urls[offset:offset + 50]

This means that a CSV containing 120 unique URLs will be processed as three batches: 50, 50, and 20 URLs.

For each completed item, we pass its result to product_row(), which converts the product information into a flat dictionary.

The price_text() function uses Python's Decimal type to format valid prices to two decimal places. Missing or invalid prices become empty cells instead of causing the CSV export to fail.

Finally, we collect errors separately. If a product page cannot be retrieved, its URL and the reported error are written to failed_urls.csv.

This makes it easier to inspect problems or retry failed URLs without processing the entire dataset again.

Understand Walmart's multiple seller offers

One thing to be careful about when working with Walmart data is how you interpret prices.

A product's top-level price field represents the featured price in the returned listing. However, the same product page may include other offers.

The Sony WH-1000XM5 example illustrates this:

Seller	Condition	Price
Focus Camera	New, featured offer	$298.00
Video & Audio Center	New	$288.00
Beach Camera	Open Box	$189.00
Secondipity	Restored: Like New	$159.99

These prices come from the product data collected on October 28, 2026.

If your goal is to create a product catalog, the featured price might be enough.

But if you're building a price comparison application, you may want to inspect the offers array and distinguish new products from used, restored, or open-box offers.

You should also account for the seller and any location-specific availability or pricing. The lowest number in the response isn't always the offer you want to compare.

For this tutorial, we've deliberately kept the CSV focused on the featured product information. You can extend the script to create another CSV containing each seller's offers.

What can you build with the extracted data?

Once the product details are available in CSV or JSON, you can use them in several projects.

For example, you could build a product catalog that combines listings from multiple sources, create reports showing the prices and availability of selected items, or save daily snapshots to track how product prices change.

You could also compare product information across marketplaces. In that case, you'll need to match products using identifiers such as brand, manufacturer part number, and product variant instead of relying on titles alone.

These projects build on the same foundation: collect structured data, validate the fields that matter, and store the result in a format your application can use.

If you're extending the workflow to a larger set of URLs, the Spidra batch scraping documentation covers additional options for processing and retrying requests.

Final thoughts

We've gone from a single Walmart product URL to a Python script that processes a list of product pages and exports their details to CSV.

The script handles duplicate URLs, missing fields, and failed product requests. It also preserves the seller and condition information needed to interpret product prices correctly.

From here, you can connect the resulting dataset to a database, build reports, or use it as the starting point for a product-tracking application.

If you're implementing this against Walmart, make sure your data collection is authorized under the site's terms.