Getting started with Twitter's Account Activity API

#twitter #webhooks #tutorial

Have you ever wanted to follow account events on Twitter in realtime? This tutorial will help you get started with Twitter's webhook-based Account Activity API.

You can watch the tutorial here, or continue reading below.

Twitter's Account Activity API sends you events related to a Twitter user's account activities in realtime. For example, the Account Activity API delivers a JSON event payload every time a subscribed user creates or deletes a Tweet, Retweets, receives or sends a Direct Message, follows a new user, or likes a post. You can find a full list of the types of activities served by the Account Activity API in Twitter's developer documentation.

A webhook-based API

The Account Activity API is different to Twitter's REST and streaming APIs, because it uses a combination of REST endpoints for setup and management, and webhooks for data delivery. If you've never used them before, webhooks deliver data by "pushing" (aka POSTing) events to the webhook URL of a registered web application. This is done via an HTTP call in real-time when a specific event (aka an account activity) happens.

Twitter's more common APIs are based on a client-server approach, where you would create a client application that makes a connection to Twitter's servers to receive some information. This can be based on a streaming model, where your client connection to Twitter's servers always stays open to receive data in realtime, or, it can be a client-triggered request, for example if you wanted to retrieve a specific Tweet.

With the Account Activity API, this pattern is inverted. Instead of your client application making a connection request to Twitter's servers, it's Twitter's servers that make a connection back to your web application listening for data. In this case, Twitter acts as the client and opens a connection to your server to post activities to that webhook every time an event occurs.

There are many advantages to using webhook-based APIs:

From a computing-resourcing perspective, webhooks require less resources than if you have an ongoing open connection to a streaming API.
There is confirmation on both sides of an activity being delivered per POST with an HTTP 200 response code.
If an activity is not successfully delivered, there is a built-in retry process to re-POST, which reduces the chance of missed data.
With webhooks, your application no longer has to make periodic polling requests to an API to check for new data to be available.
New data will automatically be sent to your server as soon as it's available. Therefore, the data received is almost always the most up-to-date.

What you need to get started

You'll need the following:

A developer account is required to use any of Twitter's APIs. You can apply for one if you don't have one already.
A Twitter developer app with the right permissions. You can create a new one if needed, by selecting the "Create an app" button under the Apps section of your developer portal.
- Make sure to enable "Read, Write, and Direct Messages", on the permissions tab of your app page. Also, note that permission settings are not retroactive for authorized users with previous lower permission settings.
- On the "Keys and Access token" tab, take a note of your app's Consumer API Key and API Secret. This information will enable you to generate a bearer token.
- You will also have to generate your app's access token and access token secret. You will need these when you register your webhook URL.
A developer environment, which you can set up in the Dev Environments section of your developer portal. Select your app and choose an environment name. The name you choose will replace the :env_name token when you start using the different Account Activity API endpoints.
A specific, secured web application with URL for receiving data through webhooks. A tool like ngrok could be useful for creating a local web address to test your webhooks in development.

Securing your webhooks

In order to receive activities from Twitter via webhooks, you'll need to create and secure a web application that can receive HTTP (POST and GET) requests on a URL of your choice. This app needs to be secured with the same Twitter app credentials that you set up in your developer environment for the Account Activity API, and coded to create the crc_token using these credentials. Twitter will regularly send your webhook challenge response check (known as CRC) GET requests, for which your web app needs to return the crc_token generated using the Twitter application credentials that match those of your whitelisted developer environment app.

You can register a webhook URL for the Account Activity API using the POST account_activity/all/:env_name/webhooks endpoint.

Set up for the Account Activity API is done with API requests - there's no user interface built into the developer portal. We've provided an example dashboard tool on our TwitterDev GitHub page, which you might find useful for testing: https://github.com/twitterdev/account-activity-dashboard

Your registered webhook URL is where the Account Activity API will send account activities, in the form of HTTP POST requests containing the event data.

Authentication

Authentication can be tricky with the Account Activity API, because there are different auth methods for different functions of the API. Most of the webhook setup is done through application-only authentication with a bearer token. Check the docs for more information on how to build one.

Subscriptions are set up via application-user authentication for each user. If you want to receive events for your own Twitter account, you can use the access token generated with the API key and API secret key from your developer portal, and this is a great way to get started and test the API. If you want to receive events relating to another Twitter account, the owner of that account will have to grant you permission via the 3-legged OAuth process. This process is also something that you will need to implement in your server-side code, as this will enable you to receive a user's access tokens. In the example dashboard app, we use Sign in with Twitter to get the tokens for each user, so take a look if you want to learn more.

Subscriptions

Once you have a user's access token from the 3-legged OAuth process, you can use it with application-user authentication to create a new subscription for that user.

At this stage, you should have these things:

A running server-side web application ready to receive webhook events.
A registered webhook URL where Twitter will send events (also known as account activities) in the form of POST requests.
One or more authorised users in the form of registered subscriptions for whom you will receive account activities.

OK! You're ready to start receiving events!

Events

You'll start receiving events for a user as soon as they are subscribed to your webhook URL. So, what exactly is an event? In this case, an event corresponds with an activity related to the Twitter account of a subscribed user. Each event is delivered in the form of a JSON payload. In the example below, a "message create" event is delivered when the user receives a Direct Message. There is a "text" field that contains the content of the message, and other information.

{
    "for_user_id": "1102321381",
    "direct_message_events": [{
        "type": "message_create",
        "id": "1133674120380657669",
        "created_timestamp": "1559123947602",
        "message_create": {
            "target": {
                "recipient_id": "1102321381"
            },
            "sender_id": "1029741837916012544",
            "message_data": {
                "text": "Good morning ☀️☕",
                "entities": {
                    "hashtags": [],
                    "symbols": [],
                    "user_mentions": [],
                    "urls": []
                }
            }
        }
    }],
    "users": {
        "1102321381": {
            "id": "1102321381",
            "created_timestamp": "1358552743000",
            "name": "Aurelia Specker",
            "screen_name": "AureliaSpecker",
            "location": "London, England",
            "description": "partner engineer @TwitterUK 👩‍💻 • instructor @CodeFirstGirls • outdoor enthusiast 🌄",
            "protected": false,
            "verified": false,
            "followers_count": 589,
            "friends_count": 745,
            "statuses_count": 521,
            "profile_image_url": "http://pbs.twimg.com/profile_images/1025560966434443264/vsVPbXiT_normal.jpg",
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1025560966434443264/vsVPbXiT_normal.jpg"
        },
        "1029741837916012544": {
            "id": "1029741837916012544",
            "created_timestamp": "1534344560624",
            "name": "Aurelia",
            "screen_name": "re_testing",
            "location": "London, England",
            "description": "I like traveling 🐪🌴☀️",
            "protected": false,
            "verified": false,
            "followers_count": 4,
            "friends_count": 10,
            "statuses_count": 52,
            "profile_image_url": "http://pbs.twimg.com/profile_images/1029742489090121728/-83RqB5N_normal.jpg",
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1029742489090121728/-83RqB5N_normal.jpg"
        }
    }
} {
    "for_user_id": "1102321381",
    "direct_message_indicate_typing_events": [{
        "created_timestamp": "1559123947551",
        "sender_id": "1029741837916012544",
        "target": {
            "recipient_id": "1102321381"
        }
    }],
    "users": {
        "1102321381": {
            "id": "1102321381",
            "created_timestamp": "1358552743000",
            "name": "Aurelia Specker",
            "screen_name": "AureliaSpecker",
            "location": "London, England",
            "description": "partner engineer @TwitterUK 👩‍💻 • instructor @CodeFirstGirls • outdoor enthusiast 🌄",
            "protected": false,
            "verified": false,
            "followers_count": 589,
            "friends_count": 745,
            "statuses_count": 521,
            "profile_image_url": "http://pbs.twimg.com/profile_images/1025560966434443264/vsVPbXiT_normal.jpg",
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1025560966434443264/vsVPbXiT_normal.jpg"
        },
        "1029741837916012544": {
            "id": "1029741837916012544",
            "created_timestamp": "1534344560624",
            "name": "Aurelia",
            "screen_name": "re_testing",
            "location": "London, England",
            "description": "I like traveling 🐪🌴☀️",
            "protected": false,
            "verified": false,
            "followers_count": 4,
            "friends_count": 10,
            "statuses_count": 52,
            "profile_image_url": "http://pbs.twimg.com/profile_images/1029742489090121728/-83RqB5N_normal.jpg",
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/1029742489090121728/-83RqB5N_normal.jpg"
        }
    }
}

It is not possible to choose to receive certain types of events and exclude others - you'll always receive all event types related to the subscribed user(s). A full list of the event types can be reviewed in the developer documentation.

If you need retries, you can sign up for a paid option to enable them. With retries, if the activity is not delivered and your webhook does not return a successful response, the Twitter server will try sending the activity again. Enterprise customers also have additional redundancy and recovery coverage with the Account Activity Replay API, providing a way to resend activities for a requested period of time in the previous five days.

The more subscriptions you have, the more important capacity planning becomes, to ensure that your systems are capable of receiving a large number of events.

Further resources

Twitter's developer docs contain further resources, including:

Guides
Frequently asked questions
API Reference
Account Activity API reference index
Quick start guide (for enterprise customers)

Join our Developer Community or Tweet us at @TwitterDev if this tutorial inspires you to build anything. Let us know what you think on our feedback channel.

Top comments (3)

Javier Aguirre • May 31 '19

Thank you for the article Aurelia! It reminded me an example I made with Node and the Twitter streaming api some time ago, I published it on GitHub in case it’s interesting.