DEV Community

Qovia
Qovia

Posted on Originally published at qovia.ai

Pull JD and Taobao product reviews into a CSV with Python

I build Qovia, the API used in this post. The paging loop, CSV writer and rating summary are plain Python; they work with any source that returns the same review fields. The two HTTP paths below are Qovia's.

If you are comparing products, tracking a launch, or feeding reviews into a notebook, JD.com and Taobao/Tmall are the two marketplaces you will hit most often in China. Each site exposes reviews on its own product page, but the identifiers differ and the public pages are awkward to script against.

This tutorial writes a small Python script that:

  1. pulls JD.com buyer reviews with GET /v1/raw/jd/comments and a sku_id,
  2. pulls Taobao and Tmall buyer reviews with GET /v1/raw/taobao/comments and an item_id,
  3. pages until has_more is false,
  4. writes a combined CSV you can open in a spreadsheet or a notebook.

A short section at the end notes a second Taobao route that returns the platform-shaped JSON, including popular review keywords on the first page.

Setup

You need Python 3.9+ and requests:

pip install requests
export QOVIA_KEY=sk-qovia-...   # key from the Qovia console
Enter fullscreen mode Exit fullscreen mode

Get a key from the Qovia console. Requests authenticate with Authorization: Bearer <key> (an x-api-key header also works). A walkthrough of both paths, with the same example IDs, is on the product reviews use-case. New accounts get 50 trial calls; after that, successful calls are billed from a prepaid balance. The JD and Taobao API reference pages list every parameter with a response example.

The IDs used below (sku_id=100006271091, item_id=611261922146) are catalog example placeholders from those pages. Replace them with the product you are actually looking up.

Step 1: a GET helper

Successful calls return HTTP 200 with {"request_id", "data", "meta"}. The payload you care about is in data. Billing metadata lives in meta (meta.charged_usd, meta.billed, meta.empty). An empty result is still 200, with meta.empty: true, and on these two endpoints it is not charged. Errors are {"error": {"code", "message", ...}} and are never charged. If the server returns 503 with upstream_busy, wait for the Retry-After header and try once more — that failed call is not charged:

import os
import time
import requests

BASE = "https://api.qovia.ai"
HEADERS = {
    "Authorization": f"Bearer {os.environ['QOVIA_KEY']}",
}


def get(path, params):
    """GET JSON; honour Retry-After once on 503."""
    for attempt in range(2):
        r = requests.get(
            f"{BASE}{path}",
            headers=HEADERS,
            params=params,
            timeout=60,
        )
        if r.status_code == 503 and attempt == 0:
            time.sleep(int(r.headers.get("Retry-After", "10")))
            continue
        r.raise_for_status()
        payload = r.json()
        return payload.get("data") or {}
Enter fullscreen mode Exit fullscreen mode

Unknown query parameters are rejected with 400 invalid_params, so only send documented keys.

Step 2: JD.com reviews by SKU

sku_id is the number in item.jd.com/<SKU>.html. Reviews come back newest first, 10 per page. Optional only_this_sku defaults to true (this SKU only); set it to false to include every variant of the product. Raise page up to the previous max_page and stop when has_more is false. A SKU without reviews, or one that does not exist, answers empty and is not charged.

Each item includes text, a 1–5 rating, created_at, sku / sku_id, a mostly masked user_name, anonymous, ip_location, images, likes and shop_reply.

def fetch_jd(sku_id, only_this_sku=True, max_pages=5):
    rows = []
    page = 1
    while page <= max_pages:
        data = get(
            "/v1/raw/jd/comments",
            {
                "sku_id": sku_id,
                "page": page,
                "only_this_sku": only_this_sku,
            },
        )
        items = data.get("items") or []
        for it in items:
            rows.append(
                {
                    "platform": "jd",
                    "product_id": sku_id,
                    "review_id": it.get("id") or it.get("cid"),
                    "created_at": it.get("created_at"),
                    "rating": it.get("rating"),
                    "text": it.get("text") or "",
                    "sku": it.get("sku") or "",
                    "user_name": it.get("user_name") or "",
                    "shop_reply": it.get("shop_reply") or "",
                    "likes": it.get("likes") or 0,
                }
            )
        if not data.get("has_more"):
            break
        page += 1
    return rows
Enter fullscreen mode Exit fullscreen mode

Cap max_pages while you are testing. max_page in the first response tells you how far paging can go; it is often much smaller than total.

Step 3: Taobao and Tmall reviews by item ID

item_id is the id= in the item URL, and the same field works for Taobao and Tmall — for example item.taobao.com/item.htm?id=611261922146. Reviews come back newest first, 20 per page. Optional skip_default_reviews defaults to true: it skips default positive reviews that have no text, but keeps those that have a follow-up. Stop when has_more is false. An item without reviews, or one that does not exist, answers empty and is not charged.

Ratings are rating_level: good, neutral or negative (not a 1–5 star). Items also include sku, a masked user_name, user_level, images, videos, shop_reply and a followup object when the buyer added a later review.

def fetch_taobao(item_id, skip_default_reviews=True, max_pages=5):
    rows = []
    page = 1
    while page <= max_pages:
        data = get(
            "/v1/raw/taobao/comments",
            {
                "item_id": item_id,
                "page": page,
                "skip_default_reviews": skip_default_reviews,
            },
        )
        items = data.get("items") or []
        for it in items:
            follow = it.get("followup") or {}
            rows.append(
                {
                    "platform": "taobao",
                    "product_id": item_id,
                    "review_id": it.get("id") or it.get("cid"),
                    "created_at": it.get("created_at"),
                    "rating": it.get("rating_level"),
                    "text": it.get("text") or "",
                    "sku": it.get("sku") or "",
                    "user_name": it.get("user_name") or "",
                    "shop_reply": it.get("shop_reply") or "",
                    "likes": it.get("likes") or 0,
                    "followup_text": follow.get("text") or "",
                }
            )
        if not data.get("has_more"):
            break
        page += 1
    return rows
Enter fullscreen mode Exit fullscreen mode

Taobao's total may be approximate (total_exact can be false). Trust has_more to stop, not the headline count.

Step 4: write a CSV and a quick summary

import csv
from collections import Counter

JD_SKU = "100006271091"          # placeholder — replace me
TAOBAO_ITEM = "611261922146"     # placeholder — replace me

rows = fetch_jd(JD_SKU) + fetch_taobao(TAOBAO_ITEM)

fieldnames = [
    "platform",
    "product_id",
    "review_id",
    "created_at",
    "rating",
    "text",
    "sku",
    "user_name",
    "shop_reply",
    "likes",
    "followup_text",
]
with open("product-reviews.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=fieldnames, extrasaction="ignore")
    w.writeheader()
    w.writerows(rows)

print(f"wrote {len(rows)} reviews")
for platform, group in (
    ("jd", [r for r in rows if r["platform"] == "jd"]),
    ("taobao", [r for r in rows if r["platform"] == "taobao"]),
):
    print(platform, dict(Counter(r["rating"] for r in group)))
Enter fullscreen mode Exit fullscreen mode

JD ratings will look like 5, 4, 3; Taobao ratings will look like good, neutral, negative. Keep the platform column so you do not mix the two scales.

Optional: Taobao reviews as platform-shaped JSON

If you need the popular review keywords (first page only) or the platform's own field names, GET /v1/raw/taobao/comments_v2 takes the same item_id and page, returns 20 comments per page, and stops when hasNextPage is false. The list is comments, not items, and the text lives in content. It usually answers in about three seconds. The V2 reference documents the shape. For a clean CSV, the V1 route in step 3 is the simpler default.

What you have

You now have one script that talks to both marketplaces, a CSV of review text, ratings, SKUs, shop replies and follow-ups, and a per-platform rating tally. From here you can filter by created_at, keep only reviews that include images, or pass text into a classifier — the fetch-and-page part is done.

More context, including FAQ and related endpoints, is on the product reviews use-case. Qovia is also an MCP server at https://api.qovia.ai/mcp if you would rather call the same endpoints from an AI coding tool.


This article was written with AI assistance and reviewed against Qovia's OpenAPI. I work on Qovia.

Top comments (0)