Over 500 million posts are published on X (formerly Twitter) every single day. Reddit hosts more than 100,000 active communities discussing everything from machine learning to local weather. Instagram's 2 billion users generate a continuous stream of product opinions, lifestyle signals, and trend indicators.
Social media data is the closest thing the internet has to real-time public opinion. It powers:
Sentiment analysis — What do people think about your product, brand, or competitor right now?
Trend detection — What topics are accelerating before they go mainstream?
Academic research — Studying misinformation, political discourse, crisis communication
Market intelligence — Tracking competitor mentions, industry conversations
AI training datasets — Social text is gold for fine-tuning conversational models
The challenge in 2026: platforms have made scraping harder than ever. The official X API now starts at $100/month and the free tier is almost useless for research. Instagram aggressively blocks headless browsers. Reddit has rate-limited its API severely after the 2023 controversy.
This guide gives you working approaches for all three platforms — from DIY Python scrapers to managed tools — with honest assessments of what each method can and cannot do.
The 2026 Reality Check: What's Still Scrapable
Before writing a single line of code, here's the honest state of social media scraping in 2026:
| Platform | Public Posts | Profile Data | Followers | DMs | Difficulty |
|---|---|---|---|---|---|
| X / Twitter | ✅ (with effort) | ✅ | ✅ | ❌ | Hard |
| ✅ (via API/PRAW) | ✅ | ❌ | ❌ | Easy–Medium | |
| ⚠️ Public only | ⚠️ Public only | ❌ | ❌ | Very Hard | |
| TikTok | ✅ (via mobile API) | ✅ | ❌ | ❌ | Hard |
| ⚠️ (see Blog 02) | ⚠️ (see Blog 02) | ❌ | ❌ | Hard |
Key principles:
Never scrape private data, DMs, or anything behind a login you don't own
Always respect robots.txt and Terms of Service for your use case
For academic or commercial research, official APIs or licensed providers are the safer path
Part 1: Scraping X (Twitter) in 2026
The Landscape
X's public guest API was effectively removed in 2023. Almost every endpoint now requires a logged-in session and a CSRF token. The internal GraphQL API that underpins X's own web interface is the main DIY scraping target — but its endpoint identifiers change regularly.
Three working approaches exist in 2026:
| Method | Cost | Volume | Reliability | Best for |
|---|---|---|---|---|
| X API v2 (official) | $100+/month | Limited | High | Authorised developers |
| Playwright + session | Free | Low-medium | Medium | Research, personal use |
| ScrapeGraphAI | Paid per req | High | High | Production |
Method 1: X API v2 (Official)
For authorised projects, the official API is always the right choice. The free tier gives 500k tweets/month read access:
pip install tweepy
import tweepy
import pandas as pd
from datetime import datetime, timezone, timedelta
# Get credentials at developer.twitter.com
client = tweepy.Client(bearer_token="YOUR_BEARER_TOKEN")
def search_recent_tweets(
query: str,
max_results: int = 100,
days_back: int = 7
) -> pd.DataFrame:
"""
Search tweets from the last 7 days using X API v2.
Args:
query: Search query. Supports operators like:
- AND/OR: "python scraping OR web crawling"
- Exact: '"machine learning" tutorial'
- Exclude: "python -is:retweet"
- Language: "python lang:en"
- Has media: "python has:images"
"""
start_time = datetime.now(timezone.utc) - timedelta(days=days_back)
# Append filters to reduce noise
full_query = f"{query} -is:retweet lang:en"
tweets_data = []
paginator = tweepy.Paginator(
client.search_recent_tweets,
query=full_query,
tweet_fields=[
"created_at", "public_metrics", "author_id",
"lang", "context_annotations", "entities"
],
user_fields=["name", "username", "public_metrics", "verified"],
expansions=["author_id"],
start_time=start_time,
max_results=min(100, max_results),
)
users_map = {}
for response in paginator:
if not response.data:
break
# Build user lookup from includes
if response.includes and "users" in response.includes:
for user in response.includes["users"]:
users_map[user.id] = user
for tweet in response.data:
user = users_map.get(tweet.author_id)
metrics = tweet.public_metrics or {}
tweets_data.append({
"tweet_id": str(tweet.id),
"text": tweet.text,
"created_at": tweet.created_at,
"author_id": str(tweet.author_id),
"username": user.username if user else None,
"name": user.name if user else None,
"followers": user.public_metrics.get("followers_count") if user else None,
"retweets": metrics.get("retweet_count", 0),
"likes": metrics.get("like_count", 0),
"replies": metrics.get("reply_count", 0),
"quotes": metrics.get("quote_count", 0),
"impressions": metrics.get("impression_count", 0),
"url": f"https://x.com/{user.username if user else 'i'}/status/{tweet.id}",
})
if len(tweets_data) >= max_results:
break
df = pd.DataFrame(tweets_data)
print(f"Collected {len(df)} tweets for query: '{query}'")
return df
# Example: track brand mentions
df = search_recent_tweets(
query="python web scraping 2026",
max_results=200,
days_back=7
)
df.to_csv("tweets.csv", index=False)
print(df[["username", "text", "likes", "retweets"]].head(10))
Method 2: Playwright-Based X Scraper (No API Key)
For personal research without API access, Playwright can scrape public search results and profiles using a saved session — exactly as described for LinkedIn in Blog 02:
import asyncio
import random
import json
from playwright.async_api import async_playwright
from playwright_stealth import stealth_async
async def save_x_session():
"""Log in to X manually once and save cookies. Run this first."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=False)
context = await browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120"
)
page = await context.new_page()
await page.goto("https://x.com/login")
print("Log in manually, then press Enter...")
input()
cookies = await context.cookies()
with open("x_cookies.json", "w") as f:
json.dump(cookies, f)
print(f"Saved {len(cookies)} cookies.")
await browser.close()
async def scrape_x_search(query: str, scroll_times: int = 10) -> list[dict]:
"""
Scrape X search results using a saved session.
Returns a list of tweet dicts.
"""
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=True,
args=["--disable-blink-features=AutomationControlled"]
)
context = await browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120",
viewport={"width": 1280, "height": 900}
)
# Load saved session
with open("x_cookies.json") as f:
await context.add_cookies(json.load(f))
page = await context.new_page()
await stealth_async(page)
# Navigate to search
encoded = query.replace(" ", "%20")
await page.goto(
f"https://x.com/search?q={encoded}&src=typed_query&f=top",
wait_until="domcontentloaded"
)
await asyncio.sleep(random.uniform(2, 4))
tweets = []
seen_ids = set()
for scroll in range(scroll_times):
# Extract tweet data from the current viewport
tweet_articles = await page.query_selector_all("article[data-testid='tweet']")
for article in tweet_articles:
try:
# Get tweet text
text_el = await article.query_selector("[data-testid='tweetText']")
text = await text_el.inner_text() if text_el else None
# Username
user_el = await article.query_selector("[data-testid='User-Name'] a")
href = await user_el.get_attribute("href") if user_el else ""
username = href.strip("/").split("/")[-1] if href else None
# Display name
name_els = await article.query_selector_all(
"[data-testid='User-Name'] span"
)
display_name = None
for el in name_els:
txt = await el.inner_text()
if txt and not txt.startswith("@"):
display_name = txt
break
# Engagement stats
async def get_stat(testid):
el = await article.query_selector(f"[data-testid='{testid}']")
if el:
txt = await el.inner_text()
return txt.strip() or "0"
return "0"
likes = await get_stat("like")
replies = await get_stat("reply")
retweets = await get_stat("retweet")
# Time
time_el = await article.query_selector("time")
posted_at = await time_el.get_attribute("datetime") if time_el else None
# Unique ID to deduplicate
tweet_id = f"{username}_{posted_at}"
if tweet_id in seen_ids or not text:
continue
seen_ids.add(tweet_id)
tweets.append({
"username": username,
"display_name": display_name,
"text": text,
"likes": likes,
"replies": replies,
"retweets": retweets,
"posted_at": posted_at,
})
except Exception:
continue
# Scroll down for more tweets
await page.evaluate("window.scrollBy(0, window.innerHeight * 1.5)")
await asyncio.sleep(random.uniform(1.5, 3.0))
print(f" Scroll {scroll+1}/{scroll_times} — {len(tweets)} tweets collected")
await browser.close()
print(f"\nTotal unique tweets: {len(tweets)}")
return tweets
# Run
tweets = asyncio.run(scrape_x_search("python scraping 2026", scroll_times=8))
df = pd.DataFrame(tweets)
df.to_csv("x_search_results.csv", index=False)
Part 2: Reddit Scraping with PRAW
Reddit is the most developer-friendly major social platform for data collection. Its official Python wrapper PRAW (Python Reddit API Wrapper) is free, well-documented, and gives access to posts, comments, user profiles, and subreddit data at no cost.
pip install praw pandas
Register a Reddit app at reddit.com/prefs/apps (takes 2 minutes, free) to get your client ID and secret.
Scraping subreddit posts
import praw
import pandas as pd
from datetime import datetime, timezone
reddit = praw.Reddit(
client_id="YOUR_CLIENT_ID",
client_secret="YOUR_CLIENT_SECRET",
user_agent="PythonResearchBot/1.0 by u/your_username",
)
def scrape_subreddit(
subreddit_name: str,
sort: str = "hot", # "hot", "new", "top", "rising"
time_filter: str = "week", # "hour", "day", "week", "month", "year", "all"
limit: int = 500
) -> pd.DataFrame:
"""
Scrape posts from a subreddit.
Best for: topic research, trend monitoring, content analysis.
"""
sub = reddit.subreddit(subreddit_name)
# Choose sort method
if sort == "hot":
posts_gen = sub.hot(limit=limit)
elif sort == "new":
posts_gen = sub.new(limit=limit)
elif sort == "top":
posts_gen = sub.top(time_filter=time_filter, limit=limit)
elif sort == "rising":
posts_gen = sub.rising(limit=limit)
else:
posts_gen = sub.hot(limit=limit)
records = []
for post in posts_gen:
records.append({
"post_id": post.id,
"title": post.title,
"text": post.selftext[:500] if post.selftext else None,
"author": str(post.author) if post.author else "[deleted]",
"score": post.score,
"upvote_ratio": post.upvote_ratio,
"num_comments": post.num_comments,
"url": post.url,
"permalink": f"https://reddit.com{post.permalink}",
"flair": post.link_flair_text,
"is_self": post.is_self,
"created_utc": datetime.fromtimestamp(post.created_utc, tz=timezone.utc).isoformat(),
"subreddit": subreddit_name,
})
df = pd.DataFrame(records)
print(f"Scraped {len(df)} posts from r/{subreddit_name}")
return df
# Example: research Python discussions
df = scrape_subreddit("learnpython", sort="top", time_filter="month", limit=500)
df.to_csv("reddit_learnpython.csv", index=False)
print(df[["title", "score", "num_comments"]].head(10))
Scraping post comments (deep dive)
def scrape_post_comments(
post_url: str,
max_comments: int = 200,
include_replies: bool = False
) -> pd.DataFrame:
"""
Scrape all comments from a Reddit post.
Useful for sentiment analysis, topic deep-dives, building datasets.
"""
submission = reddit.submission(url=post_url)
# Replace "MoreComments" objects to get all comments
submission.comments.replace_more(limit=0)
all_comments = submission.comments.list() if include_replies else submission.comments
records = []
for comment in list(all_comments)[:max_comments]:
if not hasattr(comment, "body"):
continue
records.append({
"comment_id": comment.id,
"author": str(comment.author) if comment.author else "[deleted]",
"body": comment.body,
"score": comment.score,
"depth": comment.depth,
"created_utc": datetime.fromtimestamp(
comment.created_utc, tz=timezone.utc
).isoformat(),
"is_op": comment.is_submitter,
"awards": comment.total_awards_received,
})
return pd.DataFrame(records)
# Example: deep dive on a specific post
comments_df = scrape_post_comments(
"https://www.reddit.com/r/MachineLearning/comments/example/",
max_comments=300
)
print(f"Top comment: {comments_df.sort_values('score', ascending=False).iloc[0]['body'][:200]}")
Multi-subreddit keyword monitoring
def monitor_keyword_across_subreddits(
keyword: str,
subreddits: list[str],
limit_per_sub: int = 100
) -> pd.DataFrame:
"""
Search for a keyword across multiple subreddits.
Great for competitive intelligence and brand monitoring.
"""
all_posts = []
for sub_name in subreddits:
print(f"Searching r/{sub_name} for '{keyword}'...")
try:
sub = reddit.subreddit(sub_name)
for post in sub.search(keyword, sort="new", limit=limit_per_sub):
all_posts.append({
"subreddit": sub_name,
"title": post.title,
"text": post.selftext[:300] if post.selftext else None,
"author": str(post.author) if post.author else "[deleted]",
"score": post.score,
"comments": post.num_comments,
"permalink": f"https://reddit.com{post.permalink}",
"created_utc": datetime.fromtimestamp(
post.created_utc, tz=timezone.utc
).isoformat(),
})
except Exception as e:
print(f" Error on r/{sub_name}: {e}")
df = pd.DataFrame(all_posts)
# Summary by subreddit
summary = df.groupby("subreddit").agg(
posts=("title", "count"),
avg_score=("score", "mean"),
total_comments=("comments", "sum")
).sort_values("posts", ascending=False)
print(f"\n── Results for '{keyword}' ──")
print(summary.to_string())
return df
# Track "Python scraping" across tech subreddits
df = monitor_keyword_across_subreddits(
keyword="python scraping",
subreddits=["Python", "learnpython", "webdev", "datascience", "MachineLearning"],
limit_per_sub=50
)
df.to_csv("reddit_keyword_monitor.csv", index=False)
Part 3: Instagram Scraping (Public Profiles Only)
Instagram is the most aggressively protected major platform in 2026. The public guest API was removed years ago. Almost all data requires a logged-in session. The GraphQL API underlying Instagram's web interface changes identifiers constantly.
This section covers only public profile and hashtag data — the bare minimum available without login — and the Playwright approach for logged-in access to your own account's data.
Scraping public profile metadata
import httpx
import json
import asyncio
async def get_instagram_profile(username: str) -> dict | None:
"""
Fetch public profile data for an Instagram username.
Uses the semi-public shared data endpoint.
Only works for public accounts.
"""
url = f"https://www.instagram.com/{username}/?__a=1&__d=dis"
headers = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 Chrome/120 Safari/537.36"
),
"Accept": "*/*",
"Referer": "https://www.instagram.com/",
"X-IG-App-ID": "936619743392459", # Instagram Web app ID
}
async with httpx.AsyncClient(headers=headers, follow_redirects=True) as client:
try:
r = await client.get(url, timeout=15)
if r.status_code == 200:
data = r.json()
user = data.get("graphql", {}).get("user", {})
return {
"username": user.get("username"),
"full_name": user.get("full_name"),
"biography": user.get("biography"),
"followers": user.get("edge_followed_by", {}).get("count"),
"following": user.get("edge_follow", {}).get("count"),
"posts": user.get("edge_owner_to_timeline_media", {}).get("count"),
"is_verified": user.get("is_verified"),
"is_business": user.get("is_business_account"),
"profile_url": f"https://www.instagram.com/{username}/",
}
except Exception as e:
print(f"Error fetching @{username}: {e}")
return None
# Batch scrape public profiles
async def batch_profile_scrape(usernames: list[str]) -> pd.DataFrame:
results = []
for username in usernames:
print(f"Fetching @{username}...")
profile = await get_instagram_profile(username)
if profile:
results.append(profile)
await asyncio.sleep(random.uniform(2, 5))
return pd.DataFrame(results)
profiles_df = asyncio.run(batch_profile_scrape([
"natgeo", "nasa", "python.learning"
]))
print(profiles_df[["username", "followers", "posts", "is_verified"]])
Playwright-based Instagram scraper (logged-in session)
For your own account's data or influencer research on public accounts:
import asyncio, json, random
from playwright.async_api import async_playwright
from playwright_stealth import stealth_async
async def save_instagram_session():
"""Log in manually and save session cookies. Run once."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=False)
context = await browser.new_context(
user_agent="Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) "
"AppleWebKit/605.1.15 Mobile/15E148 Safari/604.1",
viewport={"width": 390, "height": 844},
is_mobile=True,
)
page = await context.new_page()
await page.goto("https://www.instagram.com/accounts/login/")
print("Log in manually, then press Enter...")
input()
cookies = await context.cookies()
with open("ig_cookies.json", "w") as f:
json.dump(cookies, f)
print("Session saved.")
await browser.close()
async def scrape_instagram_posts(username: str, max_posts: int = 30) -> list[dict]:
"""
Scrape recent posts from a public Instagram profile.
Requires a saved session from save_instagram_session().
"""
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=True,
args=["--no-sandbox", "--disable-blink-features=AutomationControlled"]
)
context = await browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120",
viewport={"width": 1280, "height": 900}
)
with open("ig_cookies.json") as f:
await context.add_cookies(json.load(f))
page = await context.new_page()
await stealth_async(page)
await page.goto(
f"https://www.instagram.com/{username}/",
wait_until="domcontentloaded"
)
await asyncio.sleep(random.uniform(2, 4))
posts = []
# Scroll to load posts
for _ in range(max_posts // 12 + 1):
post_links = await page.query_selector_all("article a[href*='/p/']")
for link in post_links:
href = await link.get_attribute("href")
if href and href not in [p.get("href") for p in posts]:
posts.append({"href": href, "username": username})
await page.evaluate("window.scrollBy(0, window.innerHeight)")
await asyncio.sleep(random.uniform(1, 2.5))
# Deduplicate and limit
seen = set()
unique_posts = []
for p in posts:
if p["href"] not in seen:
seen.add(p["href"])
unique_posts.append(p)
posts = unique_posts[:max_posts]
await browser.close()
print(f"Found {len(posts)} posts for @{username}")
return posts
Part 4: Sentiment Analysis on Collected Data
Once you've collected social media data, analysing sentiment turns raw text into actionable signals. The transformers library from HuggingFace provides excellent pre-trained sentiment models:
pip install transformers torch
from transformers import pipeline
import pandas as pd
# Load a pre-trained sentiment model (downloads ~67MB on first run)
# "cardiffnlp/twitter-roberta-base-sentiment-latest" is trained on tweets
sentiment_pipeline = pipeline(
"sentiment-analysis",
model="cardiffnlp/twitter-roberta-base-sentiment-latest",
tokenizer="cardiffnlp/twitter-roberta-base-sentiment-latest",
max_length=512,
truncation=True,
)
LABEL_MAP = {
"LABEL_0": "negative",
"LABEL_1": "neutral",
"LABEL_2": "positive",
}
def analyse_sentiment_batch(texts: list[str], batch_size: int = 32) -> list[dict]:
"""
Run sentiment analysis on a list of social media texts.
Returns list of {label, score} dicts.
"""
results = []
for i in range(0, len(texts), batch_size):
batch = texts[i:i + batch_size]
# Clean texts — remove URLs and excessive whitespace
cleaned = [
" ".join(word for word in t.split() if not word.startswith("http"))[:512]
for t in batch
]
preds = sentiment_pipeline(cleaned)
for pred in preds:
results.append({
"sentiment": LABEL_MAP.get(pred["label"], pred["label"]),
"confidence": round(pred["score"], 3),
})
print(f" Processed {min(i + batch_size, len(texts))}/{len(texts)}")
return results
def sentiment_report(df: pd.DataFrame, text_col: str = "text") -> pd.DataFrame:
"""
Enrich a DataFrame with sentiment scores and produce a summary report.
"""
texts = df[text_col].fillna("").tolist()
print(f"Analysing sentiment for {len(texts)} posts...")
sentiments = analyse_sentiment_batch(texts)
df["sentiment"] = [s["sentiment"] for s in sentiments]
df["confidence"] = [s["confidence"] for s in sentiments]
# Summary
summary = df["sentiment"].value_counts(normalize=True).mul(100).round(1)
print("\n── Sentiment Distribution ──")
for label, pct in summary.items():
bar = "█" * int(pct / 2)
print(f" {label:10s}: {bar} {pct}%")
return df
# Apply to Reddit data
reddit_df = pd.read_csv("reddit_learnpython.csv")
reddit_df = sentiment_report(reddit_df, text_col="title")
reddit_df.to_csv("reddit_with_sentiment.csv", index=False)
# Apply to X data
tweets_df = pd.read_csv("tweets.csv")
tweets_df = sentiment_report(tweets_df, text_col="text")
tweets_df.to_csv("tweets_with_sentiment.csv", index=False)
Part 5: Building a Brand Monitoring Dashboard
Combine all sources into a unified brand monitoring pipeline:
import asyncio
import pandas as pd
from datetime import datetime, timezone
async def run_brand_monitor(brand_name: str, competitor: str = None) -> dict:
"""
Collect brand mentions from Reddit and X, analyse sentiment,
and produce a unified brand health report.
"""
report = {
"brand": brand_name,
"run_at": datetime.now(timezone.utc).isoformat(),
"reddit": {},
"twitter": {},
"sentiment": {},
}
# ── Reddit ────────────────────────────────────────────────
print(f"\n[1/3] Scraping Reddit for '{brand_name}'...")
reddit_df = monitor_keyword_across_subreddits(
keyword=brand_name,
subreddits=["technology", "Python", "webdev", "datascience", "programming"],
limit_per_sub=50
)
reddit_df = sentiment_report(reddit_df, text_col="title")
report["reddit"] = {
"total_posts": len(reddit_df),
"avg_score": round(reddit_df["score"].mean(), 1),
"total_comments": int(reddit_df["comments"].sum()),
"top_post": reddit_df.sort_values("score", ascending=False).iloc[0]["title"],
"top_subreddit": reddit_df["subreddit"].value_counts().index[0],
}
# ── Sentiment ─────────────────────────────────────────────
all_sentiments = pd.concat([
reddit_df[["sentiment", "confidence"]],
])
sentiment_counts = all_sentiments["sentiment"].value_counts(normalize=True).mul(100)
report["sentiment"] = {
"positive_pct": round(sentiment_counts.get("positive", 0), 1),
"neutral_pct": round(sentiment_counts.get("neutral", 0), 1),
"negative_pct": round(sentiment_counts.get("negative", 0), 1),
"overall": sentiment_counts.idxmax(),
}
# ── Print Report ──────────────────────────────────────────
print(f"\n{'═'*50}")
print(f" BRAND MONITOR: {brand_name.upper()}")
print(f" Run at: {report['run_at']}")
print(f"{'═'*50}")
print(f"\n Reddit mentions: {report['reddit']['total_posts']}")
print(f" Avg post score: {report['reddit']['avg_score']}")
print(f" Most active sub: r/{report['reddit']['top_subreddit']}")
print(f"\n Sentiment:")
print(f" ✅ Positive: {report['sentiment']['positive_pct']}%")
print(f" 😐 Neutral: {report['sentiment']['neutral_pct']}%")
print(f" ❌ Negative: {report['sentiment']['negative_pct']}%")
print(f" Overall: {report['sentiment']['overall'].upper()}")
print(f"\n Top post: \"{report['reddit']['top_post'][:80]}...\"")
return report
# Run
report = asyncio.run(run_brand_monitor("python scraping"))
Rate Limits and Platform Rules Reference
| Platform | Daily free limit | Rate limit reset | Key restriction |
|---|---|---|---|
| X API v2 (Free) | 500k tweets/month read | 15 min windows | No historical data |
| X API v2 (Basic, $100/mo) | 10M tweets/month | 15 min windows | 30-day history |
| Reddit PRAW | 1,000 req/10 min | Rolling | Public posts only |
| Instagram (no auth) | ~200 req/hour | Hourly | Public profiles only |
| Instagram (with session) | 200–500 actions/day | Daily | Avoid aggressive scraping |
FAQ
Q: snscrape is broken in 2026 — what should I use instead? snscrape's X scraper broke in 2023 when X removed the guest API and has not been reliably fixed. Use Tweepy with X API v2 for authorised projects. For DIY scraping, use the Playwright approach above.
Q: Can I scrape private Instagram posts? No — and you shouldn't. Scraping private account data without consent violates Instagram's Terms of Service, GDPR, and potentially criminal hacking laws in many jurisdictions. Only collect publicly available data.
Q: What's the best model for social media sentiment analysis in 2026?cardiffnlp/twitter-roberta-base-sentiment-latest is trained specifically on tweets and outperforms generic models. For Reddit, which is longer-form, distilbert-base-uncased-finetuned-sst-2-english works well. For multilingual content, use nlptown/bert-base-multilingual-uncased-sentiment.
Q: Reddit changed its API — is PRAW still free? Yes. PRAW's free tier (500 requests per 10 minutes) was not affected by the 2023 API changes. Only third-party apps accessing certain endpoints at very high volume were impacted. Standard research use via PRAW remains free.
Q: How do I handle deleted posts in my Reddit dataset? Posts with [deleted] author or [removed] body are common. Filter them before analysis: df = df[df['author'] != '[deleted]'] and df = df[df['text'] != '[removed]'].
Summary
| Platform | Best Method | Key Library | What You Can Get |
|---|---|---|---|
| X / Twitter | Tweepy (API v2) | tweepy | Tweets, metrics, author data |
| X / Twitter | Playwright + session | playwright | Search results, profiles |
| PRAW | praw | Posts, comments, subreddits | |
| httpx + cookies | httpx | Public profiles, post counts | |
| All platforms | HuggingFace | transformers | Sentiment, emotion, topics |
Originally published on ZyVOP
💡 For more articles like this, subscribe to the ZyVOP newsletter!
Top comments (0)