Douban has no public API anymore — here's how I get movie & book ratings as JSON

#webscraping #python #api #datascience

If you've ever tried to add Chinese movie or book ratings to a dataset, you've hit the wall: Douban (豆瓣) — China's IMDb + Goodreads — closed its public API to new developers years ago. The old api.douban.com/v2 endpoints are gone, and most GitHub scrapers break within months.

I needed Douban scores for a side project, so I built a small scraper and published it as an Apify Actor. Below is how to use it from Python in ~10 lines, plus a mini analysis comparing Douban scores across a few films.

Disclosure: I wrote the Actor. It's pay-per-result ($3 / 1,000 items); Apify's free tier credit covers small experiments.

What it returns

Public subject data only — no reviews, no user profiles:

{"id":"1292052","type":"movie","title":"肖申克的救赎","originalTitle":"The Shawshank Redemption",
 "year":"1994","rating":9.7,"ratingCount":3354005,"genres":["剧情","犯罪"],
 "directors":["弗兰克·德拉邦特"],"countries":["美国"],"tags":["犯罪","越狱","励志"],
 "url":"https://movie.douban.com/subject/1292052/"}

Setup

pip install apify-client pandas
export APIFY_TOKEN=...   # console.apify.com → Settings → Integrations

Search by keyword

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
ACTOR = "certified_astronomy_6vi/douban-movie-book-scraper"

def douban(keywords, type_="movie", max_items=10):
    run = client.actor(ACTOR).call(run_input={
        "keywords": keywords, "type": type_, "maxItems": max_items})
    return list(client.dataset(run["defaultDatasetId"]).iterate_items())

films = douban(["Interstellar", "Parasite", "流浪地球"], max_items=6)
for f in films:
    print(f"{f['title']:<20} {f.get('originalTitle',''):<25} {f['rating']}  ({f['ratingCount']:,} votes)")

Chinese and English keywords both work. For books, pass type_="book" (e.g. ["三体"] for The Three-Body Problem).

Exact items by URL or ID

If you already have Douban IDs (e.g. from a MovieLens-style mapping), skip search:

run = client.actor(ACTOR).call(run_input={
    "startUrls": ["1292052", "https://movie.douban.com/subject/1291546/"], "type": "movie"})

Quick analysis: which genres does Douban rate highest?

import pandas as pd
df = pd.DataFrame(douban(["科幻", "爱情", "喜剧", "悬疑"], max_items=80))
df = df.explode("genres")
print(df.groupby("genres")["rating"].agg(["mean", "count"]).sort_values("mean", ascending=False).head(10))

Notes on scraping responsibly

It only reads publicly visible subject pages, at ~1 request / 2 s by default (delaySecs).
No login, no cookies, no personal data. Check Douban's terms for your use case.
Export to CSV/Excel straight from the Apify console if you don't want code.

Link: https://apify.com/certified_astronomy_6vi/douban-movie-book-scraper

Feedback and field requests welcome in the comments — e.g. if you need IMDb ID mapping, tell me.

DEV Community