How to search for Tweets about various 'Topics' using the Twitter API v2

The Twitter API v2 allows developers and researchers to search for Tweets programmatically using the recent search endpoint (available for all developers) and full-archive search endpoint (available in the academic researcher product track). In order to search for Tweets using these endpoints, you have to specify your request in terms of a search query. In your search query you can use specify that keywords that you want the Tweets returned for and also use various operators such as from:, has:, is: etc. The Twitter API v2 supports a new search operator: context:. This operators let you search for Tweets on certain topics and entities that are available in the Twitter API as part of Tweet Annotations.

So for example, if you want to search for all Tweets that are about the LA Lakers, instead of building a search query like 'LA Lakers OR #LakeShow' etc., your can be 'context:12.706083845846597632' and it will give you all the Tweets about the Los Angeles Lakers. The question is, how do you know what the value is for the Los Angeles Lakers entity, in order to use the context: operator? In this guide, I will show you how to get the ID value for a topic to use with the context: operator to search for Tweets about a topic.

Step 1: Find a Tweet that represents a topic that you want to search for Tweets on

To do so simply, search for that topic using a keyword of your choice on Twitter. In this example, I want all Tweets about the Los Angeles Lakers so I simply searched for a Tweet with the hashtag #LakeShow on Twitter. I got a Tweet with the following URL: https://twitter.com/Lakers/status/1582026396917866497.

Step 2: Paste this link in the Tweet entity extractor tool

I put together a sample app that gives you the key & value combination to use with the context: operator. To use this app, open your browser and go to: https://tweet-entity-extractor.glitch.me/. In this app, pase the Tweet URL from step 1, and click on Get context ID! This will give you a response like the one shown below:

[
  {
    "context": "3.10000607734",
    "entity_name": "NBA Basketball"
  },
  {
    "context": "11.706083902411055104",
    "entity_name": "Basketball"
  },
  {
    "context": "12.706083845846597632",
    "entity_name": "Los Angeles Lakers"
  },
  {
    "context": "26.706083889454813185",
    "entity_name": "NBA"
  },
  {
    "context": "46.1557697289971322880",
    "entity_name": "Sports & Fitness Business"
  }
]

From this response, you can see that the ID to use for the Los Angeles Lakers topic is 12.706083845846597632.

Note: Some Topics may have multiple key.value pairs so you can chose one of those.

Step 3: Use the ID value from the previous step in your search query & get Tweets for a Topic

Now that you have the ID for the topic of your choice, simply use it in your search query. Below is a code snippet that shows how you can use context: operator in Python to get Tweets on a topic (Los Angeles Lakers in this case) using the key value pair from the previous step. (The code below uses the Tweepy package in Python, so make sure you have it installed, using the steps shown here)

import tweepy

client = tweepy.Client(bearer_token='REPLACE_ME')

response = client.search_recent_tweets(query='context:12.706083845846597632')

for tweet in response.data:
    print(tweet.id)

Note: Not all Tweets are annotated. You can find more information and FAQs on Tweet annotations here.

I hope you are now able to get Tweets based on topics of your interest. The Tweet Annotations support various topics, so feel free to explore the available topics using the steps show above. If you have any questions, feel free to DM me on Twitter.