DEV Community

huynhtanan
huynhtanan

Posted on

Track Walmart Rollback deals with Python and SQLite

#python #webscraping #sqlite #walmart

If you buy a lot on Walmart.com, you've seen those orange Rollback tags — a temporary price cut that Walmart itself slaps on an item, usually a few dollars to a few tens of percent off. There's no newsletter listing every active Rollback across the catalog, and the "deals" pages are curated and partial. If you want them all in one table, you have to scrape search.

This post shows how to pull every Rollback deal from a Walmart search in Python, filter by discount percent, and track the drops day over day with a 60-line SQLite logger.

Code is in github.com/huynhtanan/walmart-deals-tracker (MIT). Disclosure: I built the Apify scraper the examples call; it's pay-per-result at $1 per 1,000 products and new Apify accounts get free monthly credit covering tens of thousands of items.

Setup 

pip install "apify-client>=3"
export APIFY_TOKEN=...   # https://console.apify.com/settings/integrations (free account)
Enter fullscreen mode Exit fullscreen mode

1. All Rollback items on one search page, with discount %

Walmart's search results already include the strikethrough price when there is one. The scraper exposes it as wasPrice, so computing the discount is (wasPrice - price) / wasPrice.

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("zet46/walmart-search-scraper").call(run_input={
    "queries": ["coffee maker", "headphones", "vacuum"],
    "maxItemsPerQuery": 30,
    "includeSponsored": False,
})

items = list(client.dataset(run.default_dataset_id).iterate_items())
rollbacks = [i for i in items
             if i.get("badge") == "Rollback" and i.get("wasPrice") and i.get("price")]
rollbacks.sort(key=lambda x: -(x["wasPrice"] - x["price"]) / x["wasPrice"])

for r in rollbacks[:10]:
    pct = (r["wasPrice"] - r["price"]) / r["wasPrice"] * 100
    print(f'-{pct:5.1f}%  {r["query"]:12}  {r["name"][:50]:50}  '
          f'${r["price"]:>6.2f}  (was ${r["wasPrice"]:>6.2f})')
Enter fullscreen mode Exit fullscreen mode

Real output (ran today, 90 products across the three queries, 13 of them Rollbacks):

- 81.6%  headphones    2 Pack iPhone wired earbuds USB C Headphones Wired  $ 13.99  (was $ 75.99)
- 44.6%  vacuum        Eureka MaxSwivel Deluxe Upright Vacuum with Swivel  $ 49.86  (was $ 89.99)
- 34.1%  vacuum        Shark Navigator Lift-Away Upright Vacuum, Pet Frie  $118.00  (was $179.00)
- 31.8%  vacuum        BISSELL PowerForce Helix Turbo Pet, Bagless, Uprig  $ 74.97  (was $109.99)
- 28.0%  vacuum        Shark Upright Vacuum, Rotator Pet, Pet Friendly, P  $183.00  (was $254.00)
- 25.2%  headphones    Skullcandy Smokin' Buds XT True Wireless in-Ear Ea  $ 14.88  (was $ 19.88)
- 23.8%  headphones    Apple AirPods 4                                     $ 99.00  (was $129.99)
- 21.0%  headphones    Skullcandy Crusher XT Wireless Foldable over-Ear H  $ 79.00  (was $100.00)
- 15.6%  vacuum        BISSELL PowerClean FurFinder 200W Self-Standing Co  $168.00  (was $199.00)
- 15.5%  coffee maker  Keurig K-Duo Essentials Hot & Iced Single Serve K-  $109.00  (was $129.00)
Enter fullscreen mode Exit fullscreen mode

A few things that are easy to miss unless you look at the data:

  • Not every strikethrough is a Rollback. 46 of the 90 items in the run had a wasPrice, but only 13 carried the Rollback badge. The rest are "clearance", seller-promoted "reduced price" or marketplace resets. If you only want Walmart's own Rollbacks, filter on badge == "Rollback"; if you want all discounts, filter on wasPrice is not None.
  • The top deal is almost always a marketplace seller, not Walmart.com. The 81% off wired earbuds at $13.99 is sold by a third-party store. That's a signal worth filtering on — seller == "Walmart.com" keeps you in Walmart-fulfilled stock.

The scraper also returns rating, reviewCount, fulfillment (SHIPPING / STORE / PICKUP), availability, sponsored, category (Walmart's own product-type leaf, e.g. Upright Vacuums) and the usItemId you need for the /ip/<usItemId> canonical URL.

2. Daily rollback tracker in SQLite

What you actually want for a watchlist is "same item, cheaper than last time". That takes a tiny SQLite table and a JOIN on usItemId.

# track_rollbacks.py — github.com/huynhtanan/walmart-deals-tracker/blob/main/track_rollbacks.py
import os, sqlite3
from datetime import date
from pathlib import Path
from apify_client import ApifyClient

QUERIES = ["coffee maker", "headphones", "vacuum", "monitor", "office chair"]
MAX_PER_QUERY = 60
DB = Path("walmart_rollbacks.sqlite3")

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("zet46/walmart-search-scraper").call(run_input={
    "queries": QUERIES,
    "maxItemsPerQuery": MAX_PER_QUERY,
    "includeSponsored": False,
})
items = list(client.dataset(run.default_dataset_id).iterate_items())
today = date.today().isoformat()

with sqlite3.connect(DB) as con:
    con.execute("""CREATE TABLE IF NOT EXISTS prices (
        run_date TEXT, us_item_id TEXT, query TEXT,
        name TEXT, price REAL, was_price REAL, badge TEXT, seller TEXT,
        rating REAL, review_count INTEGER, url TEXT,
        PRIMARY KEY (run_date, us_item_id))""")
    for it in items:
        if it.get("sponsored"):
            continue
        con.execute("INSERT OR REPLACE INTO prices VALUES (?,?,?,?,?,?,?,?,?,?,?)",
                    (today, it["usItemId"], it.get("query"), it["name"],
                     it.get("price"), it.get("wasPrice"), it.get("badge"),
                     it.get("seller"), it.get("rating"),
                     it.get("reviewCount"), it["url"]))

    print(f"Rollbacks today ({today}):")
    for row in con.execute("""
        SELECT name, price, was_price,
               ROUND(100.0 * (was_price - price) / was_price, 1) AS pct,
               seller
        FROM prices
        WHERE run_date = ? AND badge = 'Rollback' AND price < was_price
        ORDER BY pct DESC LIMIT 20""", (today,)):
        print(f"  -{row[3]:>5}%  {row[0][:60]:60}  ${row[1]:>7.2f}  (was ${row[2]:>7.2f})  {row[4]}")
Enter fullscreen mode Exit fullscreen mode

3. "Deeper than yesterday" alerts

After two runs, this one SQL query gives you every item whose Rollback got deeper:

WITH deals AS (
  SELECT us_item_id, name, price, was_price, badge, run_date,
         LAG(price) OVER (PARTITION BY us_item_id ORDER BY run_date)   AS prev_price,
         LAG(badge) OVER (PARTITION BY us_item_id ORDER BY run_date)   AS prev_badge
  FROM prices)
SELECT run_date, name, prev_price, price, was_price,
       ROUND(100.0 * (was_price - price) / was_price, 1) AS pct_today,
       ROUND(100.0 * (prev_price - price) / prev_price, 1) AS dropped_today
FROM deals
WHERE badge = 'Rollback'
  AND prev_price IS NOT NULL
  AND price < prev_price
ORDER BY dropped_today DESC;
Enter fullscreen mode Exit fullscreen mode

And "new Rollback today" (wasn't on sale yesterday):

WITH t AS (
  SELECT us_item_id, badge, run_date,
         LAG(badge) OVER (PARTITION BY us_item_id ORDER BY run_date) AS prev_badge
  FROM prices)
SELECT p.name, p.price, p.was_price, p.url
FROM t JOIN prices p USING (us_item_id, run_date)
WHERE t.badge = 'Rollback'
  AND (t.prev_badge IS NULL OR t.prev_badge != 'Rollback')
  AND p.run_date = DATE('now');
Enter fullscreen mode Exit fullscreen mode

The repo has a one-file wrapper that does both and prints them in a terminal-friendly table.

4. Fetch a specific product by URL

If you already have the /ip/<slug>/<usItemId> URL (from an email, a share link, a competitor's page), the scraper enriches it with brand, category, shortDescription, seller, rating, reviewCount and both prices:

run = client.actor("zet46/walmart-search-scraper").call(run_input={
    "productUrls": [
        "https://www.walmart.com/ip/BISSELL-PowerForce-Helix-Bagless-Upright-Vacuum/55251527",
    ],
})
Enter fullscreen mode Exit fullscreen mode

Point this at a competitor's hero product and you have a one-call price watcher.

Cost

$1 per 1,000 products returned. Watching the top 60 of five queries daily is 300 products = $0.30/day = $9/month. Running once a week is $1.20/month.

Only billed on items the scraper actually returned. Skipped items (sponsored, deduplicated by usItemId across pages, below minRating, etc.) are free.

A few things that bit me while building this

  • Walmart only ships prices to US IPs. From anywhere else, the __NEXT_DATA__ block arrives with price: null. The scraper pins traffic through a US Apify data-center proxy by default; you don't have to configure that unless you want residential.
  • priceLines is an array, not an object. The current SSR shape is priceDetails.priceLines[0].values[{lineType: "COMPARISON", key: "WAS_PRICE", price, ...}]. If you're parsing __NEXT_DATA__ yourself and following old blog posts that read .priceDetails.wasPrice.price directly, you'll get None on every row — Walmart changed this in late 2024.
  • Category is nested on /ip/ detail pages. Search results give you a flat category leaf name (e.g. "Laptop Computers"). The product page gives it as a breadcrumb path; the scraper walks to the deepest node and returns the same string, so you can mix search and detail data in one category column.
  • The Rollback badge disappears when the sale ends — the wasPrice disappears with it, not just the badge. Keep historical rows so you can tell "the sale ended" apart from "the item was replaced".

Feedback welcome

If you'd like a specific feature — Target / Costco side-by-side, Discord/Slack webhook alerts, a flat-file CSV mode, filtering by brand or price band — the repo issues are open and I ship fixes within a day.

Code and more examples: github.com/huynhtanan/walmart-deals-tracker. Scraper: apify.com/zet46/walmart-search-scraper.

Related tutorials

Top comments (0)