Note that to select Read and write in App permissions under User authentication settings.
client = tweepy.Client(
bearer_token=BREARER,
consumer_key=API_KEY,
consumer_secret=API_KEY_SECRET,
access_token=ACCESS_TOKEN,
access_token_secret=ACCESS_TOKEN_SECRET,
return_type = dict,
wait_on_rate_limit = True
)
response = client.search_recent_tweets(
"artifical intelligence OR machine learning lang:en",
max_results = 10,
tweet_fields = ['author_id','created_at','text']
)
Top comments (1)
Good minimal Tweepy example, but two small corrections will save debugging time. BREARER appears misspelled, and the query contains artifical intelligence. More importantly, wait_on_rate_limit pauses execution but does not guarantee dataset completeness. Persist the pagination token and the last successful page.
For recurring Twitter data scraping, also store the original query and collection timestamp. Deduplicate by tweet ID before downstream analysis. Those steps make reruns reproducible.
Xquik provides a managed Twitter search API with structured JSON and pagination cursors. It also covers profiles, timelines, replies, and followers. Pay-as-you-go starts with $10, and credits never expire. Tweepy remains the right client for official API access. Xquik is useful when a cheaper Twitter API alternative fits the public-data portion of the pipeline.