Zepto is India's 10-minute grocery app. Every product card on its website carries the selling price, the printed MRP, the discount and the number of units the dark store has on the shelf. Getting that into a CSV takes two Python scripts and one library, but there are three traps on the way: an empty 202, a parser that silently drops out-of-stock products, and prices that are 100 times too high.
The block: an empty 202
Send a plain Python request to a Zepto product page:
import requests
url = "https://www.zepto.com/pn/bingo-original-style-chilli-sprinkled-flat-cut-spicy-potato-chips-pack-for-snacks/pvid/cc0c8026-9b71-4bdd-bc88-b55d08e50185"
chrome = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"
for label, headers in [("python-requests UA", None), ("Chrome UA", {"User-Agent": chrome})]:
r = requests.get(url, headers=headers)
print(label, r.status_code, len(r.content), r.headers.get("x-amzn-waf-action", "-"))
| Request | Status | Body |
|---|---|---|
requests, default user agent |
429 | 0 bytes |
requests, Chrome user agent |
202, x-amzn-waf-action: challenge
|
0 bytes |
| Same, 20 times in a row | 202 on 20 of 20 | 0 bytes |
| Through Scrape.do, no parameters | 200 | 526,755 bytes |
Zepto sits behind AWS WAF. A browser runs the challenge script, gets an aws-waf-token cookie and reloads. A script gets nothing.
The trap: 202 means "Accepted", so raise_for_status() passes. A scraper that only checks for errors writes empty pages and reports no failures. Check status_code != 200 instead.
Routing the request through Scrape.do with no extra parameters returns the full page for 1 credit. geoCode=in and super=true changed nothing on this site, and render=true is the wrong tool: the data is in the HTML before any JavaScript runs.
import csv
import json
import re
import urllib.parse
import requests
TOKEN = "<your_token>"
def fetch(url):
api_url = f"https://api.scrape.do/?token={TOKEN}&url={urllib.parse.quote(url, safe='')}"
response = requests.get(api_url, timeout=120)
if response.status_code != 200:
print(f"Request failed with status {response.status_code}: {url}")
return ""
return response.text
safe='' encodes the / and ? in Zepto's URL so they are not read as part of the Scrape.do query.
The data: a Next.js stream in the HTML
Zepto runs on the Next.js App Router. A subcategory page such as Chips & Crisps makes no product API call: every card is streamed into the HTML as self.__next_f.push([1, "..."]) script chunks. Each chunk is a JavaScript string literal, and JSON strings use the same escaping, so json.loads decodes it:
PUSH_RE = re.compile(r'self\.__next_f\.push\(\[1,"((?:[^"\\]|\\.)*)"\]\)')
def decode_rsc(html):
# Join the stream chunks back into one text
return "".join(json.loads('"' + chunk + '"') for chunk in PUSH_RE.findall(html))
On the Chips & Crisps page, 6 chunks decode into 388,684 characters with one JSON object per product card.
Trap 2: the out-of-stock products your parser misses
Card keys are sorted alphabetically, so an in-stock card starts with availableQuantity. Anchor on that and parse with json.JSONDecoder.raw_decode, and it looks right. It is not: an out-of-stock card has no availableQuantity key, gains "outOfStock": true, and starts with baseProductId instead.
Anchored on availableQuantity: 29 products
Anchored on availableQuantity or baseProductId: 32 products
3 of 32 lost at the default store; in a Mumbai store checked in a browser, the same mistake dropped 15 of 32. Anchor on both and dedupe on the variant id:
ITEM_START_RE = re.compile(r'\{"(?:availableQuantity|baseProductId)":')
def extract_items(flat):
decoder = json.JSONDecoder()
items = {}
for match in ITEM_START_RE.finditer(flat):
try:
obj, _ = decoder.raw_decode(flat, match.start())
except json.JSONDecodeError:
continue
variant = obj.get("productVariant") or {}
if variant.get("id") and obj.get("product"): # skip look-alike objects
items.setdefault(variant["id"], obj)
return list(items.values())
Trap 3: prices in paise
The Bingo! card reads "discountedSellingPrice": 1900 and "mrp": 2000. That is ₹19 against an MRP of ₹20: Zepto stores every amount in paise.
def rupees(paise):
return paise / 100 if isinstance(paise, (int, float)) else ""
def product_urls(flat):
links = re.findall(r"https://www\.zepto\.com/pn/[a-z0-9-]+/pvid/[0-9a-f-]{36}", flat)
return {link.rsplit("/", 1)[1]: link for link in links}
def to_row(item, store_id, urls):
product = item.get("product") or {}
variant = item.get("productVariant") or {}
rating = variant.get("ratingSummary") or {}
return {
"variant_id": variant.get("id", ""),
"name": product.get("name", ""),
"brand": product.get("brand", ""),
"pack_size": variant.get("formattedPacksize", ""),
"price_inr": rupees(item.get("discountedSellingPrice")),
"mrp_inr": rupees(item.get("mrp")),
"discount_pct": item.get("discountPercent", 0),
"in_stock": not item.get("outOfStock", False),
"available_qty": item.get("availableQuantity", 0),
"rating": rating.get("averageRating", ""),
"rating_count": rating.get("totalRatings", ""),
"subcategory": item.get("primarySubcategoryName", ""),
"store_id": store_id,
"url": urls.get(variant.get("id", ""), ""),
}
Walking a whole category
Every subcategory page links its siblings, all sharing the category cid. Start from a subcategory URL (the category link on the homepage is not a listing), keep only paths under the start URL's own category slug (Top Picks also appears under /cn/snacks/... and renders nothing), and dedupe on scid:
START_URL = "https://www.zepto.com/cn/munchies/chips-crisps/cid/d2c2a144-43cd-43e5-b308-92628fa68596/scid/df4f5100-c02f-4906-83b8-ddb744081a7a"
html = fetch(START_URL)
flat = decode_rsc(html)
cid = re.search(r"/cid/([0-9a-f-]{36})", START_URL).group(1)
sub_re = re.compile(r"/cn/[a-z0-9-]+/[a-z0-9-]+/cid/" + cid + r"/scid/[0-9a-f-]{36}")
category_slug = START_URL.split("/cn/")[1].split("/")[0]
subcategories = {}
for path in [START_URL.split("zepto.com")[1]] + sub_re.findall(flat):
if path.split("/")[2] == category_slug:
subcategories.setdefault(path.rsplit("/", 1)[1], path)
rows = {}
for path in subcategories.values():
page = html if path in START_URL else fetch("https://www.zepto.com" + path)
page_flat = decode_rsc(page)
store = re.search(r'"storeServiceableResponse":\{[^}]*?"storeId":"([0-9a-f-]{36})"', page_flat)
urls = product_urls(page_flat)
for item in extract_items(page_flat):
row = to_row(item, store.group(1) if store else "", urls)
rows.setdefault(row["variant_id"], row)
rows = list(rows.values())
with open("zepto-category.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
writer.writeheader()
writer.writerows(rows)
print(f"Saved {len(rows)} products to zepto-category.csv")
On Munchies: nine subcategory pages, 9 credits, 232 unique products. 42 are out of stock and still in the file, 178 sell below MRP, and the deepest discount is 60%.
The ceiling: a subcategory page holds at most 32 products. The stream says "endOfPage": true and ?page=2 returns the same 32. For wider coverage, walk more subcategories and categories.
Product details: seller, shelf life, FSSAI licence
The product page adds the catalog sheet. A schema.org Product block in JSON-LD holds name, brand, category path, ratings, images and additionalProperty pairs (seller, shelf life, licence). Zepto writes the id attribute before type, so the regex must not depend on attribute order:
LD_RE = re.compile(r'<script[^>]*type="application/ld\+json"[^>]*>(.*?)</script>', re.S)
def product_jsonld(html):
for block in LD_RE.findall(html):
try:
data = json.loads(block)
except json.JSONDecodeError:
continue
if data.get("@type") == "Product":
return data
return {}
def variant_data(flat, variant_id):
# MRP and order limit for this exact pack size
decoder = json.JSONDecoder()
for match in re.finditer(r'"productVariant":\{', flat):
obj, _ = decoder.raw_decode(flat, match.end() - 1)
if obj.get("id") == variant_id:
return obj
return {}
url = rows[0]["url"]
page = fetch(url)
ld = product_jsonld(page)
variant = variant_data(decode_rsc(page), url.rsplit("/", 1)[1])
props = {p.get("name"): p.get("value") for p in ld.get("additionalProperty", [])}
print(ld.get("name"), "|", props.get("Seller"), "|", props.get("Shelf life"),
"| FSSAI", props.get("FSSAI licence number"), "| MRP", rupees(variant.get("mrp")))
Do not take price and stock from the product page. Its availableQuantity was 3 on all 10 pages tested while the listing showed 12 to 115 units, and two products out of stock in the listing still said InStock in their JSON-LD. Use the listing for price and stock, the product page for everything else.
One store per request
Each price and stock count belongs to one dark store, recorded in the store_id column. Without a location, Zepto serves a default store in east Bengaluru. In a browser a user_position cookie moves you between stores (a 5 kg Surf Excel pack went from ₹583 to ₹602 in Mumbai), but through Scrape.do the cookie either failed uncharged or was ignored, so these scripts collect one store.
Skip the boilerplate
The same logic is packaged as an open-source library, zepto-scraper on GitHub and PyPI:
pip install zepto-scraper
export SCRAPEDO_TOKEN=your_token
zepto-scraper category "<subcategory-url>" -f csv -o zepto-category.csv
zepto-scraper product -i zepto-category.csv --limit 10 -f csv -o zepto-products.csv
There is also a no-code version on Apify: Zepto Scraper.
Run it at scale with Scrape.do
Every Zepto page here is a plain Scrape.do request: no browser, no proxy setup, 1 credit per page. The free plan covers 1,000 pages a month. Get 1000 free credits and start scraping with Scrape.do.
The scripts read category (/cn/) and product (/pn/) pages, which Zepto's robots.txt allows, and never touch search, API or account paths. Product data is not personal data; read Zepto's terms for your use case and keep request rates modest. This isn't legal advice.
Top comments (0)