The Twitter API allows developers and researchers to get Twitter data programmatically. Researchers interested in studying the conversation around a topic often use the search endpoints to get Tweets on a topic. Each of these Tweets returned from the search endpoints may have important follow-up conversations happening in replies to them, and in the Tweets quoting them, which may provide additional information to better understand the entire conversation. In this tutorial, you will learn how to get the entire conversation for a Tweet.
Step one: Search for Tweets about a topic
In the first step, you can search for Tweets on a topic, containing certain keywords and meeting certain criteria using the search endpoints. Currently, there are 2 search endpoints available in the Twitter API v2:
- Recent search, available to all developers and gives Tweets from the last 7 days
- Full-archive search, currently only available to academic researchers and gives Tweets from the entire archive of public Tweets
The search endpoints require developers to specify a search query to return Tweets meeting a certain criteria. Optionally, you can specify the time period from which you want the Tweets for.
By default, you will get 10 Tweets (Tweet ID and Tweet text) per request. Per request, using the max_results parameter, you can get upto:
- 100 Tweets for the recent search and quote Tweets lookup endpoints
- 500 Tweets for the full-archive search endpoint
If you want additional Tweets, you can use pagination with the next_token.
Below is an example of searching for Tweets from the TwitterDev account, using the recent search endpoint in Python using the Tweepy package.
Note: You will first need to sign up for a developer account and have your bearer token to use in the code samples below. Follow the steps in this tutorial to get your bearer token. Also, you will need to install the Tweepy package in Python to run the code sample below. Follow the instructions in this guide to install Tweepy.
import tweepy
client = tweepy.Client(bearer_token='REPLACE_ME')
# Replace with your own search query
query = 'from:TwitterDev -is:retweet'
# Replace the limit=1000 with the maximum number of Tweets you want
for tweet in tweepy.Paginator(client.search_recent_tweets, query=query,
tweet_fields=['context_annotations', 'created_at'], max_results=100).flatten(limit=1000):
print(tweet.id)
Once you have the Tweets on the topic of your choice, for each Tweet you can get all its replies.
Step two: For a Tweet, get all replies to it
You can get all replies to a Tweet using the conversation_id parameter available in the search endpoints. The value for this conversation_id is the Tweet ID of the original Tweet. The example below shows how to get all replies for the Tweet with ID 1503863303709286407
import tweepy
client = tweepy.Client(bearer_token='REPLACE_ME')
# Replace with your own search query
query = 'conversation_id:1503863303709286407 -is:retweet'
# Replace the limit=1000 with the maximum number of Tweets you want
for tweet in tweepy.Paginator(client.search_recent_tweets, query=query,
tweet_fields=['context_annotations', 'created_at'], max_results=100).flatten(limit=1000):
print(tweet.id)
Step three: For a Tweet, get all its Quote Tweets
To get Quote Tweets for a Tweet, you can use the Quote Tweets lookup endpoint. You can pass it the Tweet ID and set the exclude parameter to retweet. This will give you all original Quote Tweets of a Tweet. The example below shows how to get all Quote Tweets for the Tweet with ID 1503863303709286407.
import tweepy
client = tweepy.Client(bearer_token='REPLACE_ME')
# Replace the limit=1000 with the maximum number of Tweets you want
for tweet in tweepy.Paginator(client.get_quote_tweets, id=1503863303709286407, max_results=100,
exclude='retweets').flatten(limit=1000):
print(tweet.id)
That’s it. You now have the entire conversation for a Tweet including all replies and quote Tweets. The conversation_id and the quote Tweets lookup endpoints make it really easy for developers and researchers to understand the entire conversation around a Tweet. If you have any questions or feedback about this tutorial, please reach out to me on Twitter.
Top comments (0)