Solved: Exporting Twitter Bookmarks to a Notion Database

🚀 Executive Summary

TL;DR: Twitter’s native bookmarking lacks robust organization and search, making manual transfer to Notion inefficient for valuable insights. This automated Python script leverages Twitter API v2 and Notion API to export bookmarks directly, creating a structured, searchable knowledge base.

🎯 Key Takeaways

• The solution relies on user-context authentication (OAuth 1.0a) for Twitter API v2’s /2/users/:id/bookmarks endpoint and an Internal Integration Token for the Notion API.
• Duplicate entries are prevented by querying existing ‘Tweet ID’ properties in the Notion database before adding new bookmarks.
• The Python script fetches tweet data, including author details and public metrics, and maps them to predefined Notion database properties like ‘Tweet URL’, ‘Content’, ‘Author’, ‘Bookmarked At’, ‘Tweet ID’, ‘Likes’, and ‘Retweets’.

Exporting Twitter Bookmarks to a Notion Database

Introduction

In the fast-paced world of tech, valuable information often comes in fleeting forms – like a crucial tweet bookmarked for later reference. While Twitter’s native bookmarking feature is convenient, it lacks the robust organization, search, and filtering capabilities that modern knowledge management tools offer. Manually transferring these bookmarks to a more structured environment, like a Notion database, is not only tedious but also prone to oversight. For SysAdmins, Developers, and DevOps Engineers, this manual effort is a productivity killer.

This comprehensive tutorial from TechResolve will guide you through automating the export of your Twitter bookmarks directly into a Notion database. By leveraging the Twitter API and Notion API, you’ll build a resilient and automated system to keep your saved content organized, searchable, and always at your fingertips, freeing you from manual data entry and ensuring no valuable insight is ever lost.

Prerequisites

Before we dive into the automation, ensure you have the following components set up:

• Twitter Developer Account: You’ll need a developer account with an active project to access the Twitter API v2. This will provide you with the necessary API keys for authentication.
• Notion Account: A personal or team Notion account where you can create a new database.
• Python 3.x: Ensure Python 3.7 or newer is installed on your system. You can verify this by running python3 --version in your terminal.
• Python Libraries: We will use requests, requests_oauthlib, and notion-client. Install them using pip:

  pip install requests requests_oauthlib notion-client python-dotenv

• Basic Understanding of APIs: Familiarity with making HTTP requests and handling JSON data will be beneficial.

Step-by-Step Guide

Step 1: Set Up Twitter API Access for Bookmarks

Accessing Twitter bookmarks requires user-context authentication (OAuth 1.0a or OAuth 2.0 PKCE). For simplicity in this tutorial, we will focus on OAuth 1.0a, which requires a Consumer Key, Consumer Secret, Access Token, and Access Token Secret. These credentials authorize your application to act on behalf of a specific user. If you haven’t done so, create an application within your Twitter Developer Project and obtain these keys. The process to get the Access Token and Access Token Secret involves an OAuth flow where a user authorizes your application; for this guide, we assume you have obtained them, perhaps through a script or another application.

Create a file named config.env in your project directory and add your Twitter credentials:

# config.env
TWITTER_CONSUMER_KEY="YOUR_TWITTER_CONSUMER_KEY"
TWITTER_CONSUMER_SECRET="YOUR_TWITTER_CONSUMER_SECRET"
TWITTER_ACCESS_TOKEN="YOUR_TWITTER_ACCESS_TOKEN"
TWITTER_ACCESS_TOKEN_SECRET="YOUR_TWITTER_ACCESS_TOKEN_SECRET"

Here’s how you might test your Twitter API access using these credentials to fetch your bookmarked Tweets. Note that the Bookmarks API endpoint is /2/users/:id/bookmarks.

import os
import requests
from requests_oauthlib import OAuth1Session
from dotenv import load_dotenv

load_dotenv('config.env')

consumer_key = os.getenv("TWITTER_CONSUMER_KEY")
consumer_secret = os.getenv("TWITTER_CONSUMER_SECRET")
access_token = os.getenv("TWITTER_ACCESS_TOKEN")
access_token_secret = os.getenv("TWITTER_ACCESS_TOKEN_SECRET")

# You can get your user ID from the Twitter API, e.g., using users/me endpoint
# For now, let's assume you know your user ID.
# Example: If your Twitter handle is @TechResolve, your user ID might be '1234567890'
twitter_user_id = "YOUR_TWITTER_USER_ID" 

# Initialize OAuth1Session
oauth = OAuth1Session(
    consumer_key,
    client_secret=consumer_secret,
    resource_owner_key=access_token,
    resource_owner_secret=access_token_secret
)

bookmarks_url = f"https://api.twitter.com/2/users/{twitter_user_id}/bookmarks"

# Specify tweet fields to retrieve for better data
tweet_fields = "author_id,created_at,text,entities,public_metrics,source"
params = {"tweet.fields": tweet_fields, "max_results": 100}

try:
    response = oauth.get(bookmarks_url, params=params)
    response.raise_for_status() # Raise an exception for HTTP errors (4xx or 5xx)

    json_response = response.json()
    print("Successfully fetched bookmarks (first page):")
    for tweet in json_response.get("data", []):
        print(f"  - ID: {tweet['id']}, Text: {tweet['text'][:70]}...")

    if "next_token" in json_response.get("meta", {}):
        print("\nMore bookmarks available, pagination would be needed.")

except requests.exceptions.HTTPError as err:
    print(f"HTTP error occurred: {err}")
    print(f"Response body: {response.text}")
except Exception as err:
    print(f"An error occurred: {err}")

The code above initializes an OAuth 1.0a session and attempts to fetch your bookmarked tweets. Replace "YOUR_TWITTER_USER_ID" with your actual Twitter user ID. The tweet_fields parameter ensures we get rich data about each tweet, which is useful for Notion. If successful, it will print details of your most recent bookmarks. If you encounter authentication errors, double-check your API keys and tokens.

Step 2: Configure Notion Integration and Database

Next, we need to prepare Notion to receive the data. This involves creating an internal integration, a database, and sharing the database with your integration.

1. Create a Notion Integration:
   • Go to Notion My Integrations.
   • Click “New integration”.
   • Give it a name (e.g., “Twitter Bookmark Sync”), select the workspace, and submit.
   • You’ll get an “Internal Integration Token.” Copy this token.
2. Create a Notion Database:
   • In your Notion workspace, create a new page.
   • Select “Database” from the available types (e.g., “Table – Full page”).
   • Name your database (e.g., “Twitter Bookmarks”).
   • Define the following properties (columns) for your database:
     • Tweet URL: Type “URL”
     • Content: Type “Rich text” (for the tweet text)
     • Author: Type “Text”
     • Bookmarked At: Type “Date”
     • Tweet ID: Type “Text” (Crucial for checking duplicates)
     • Likes: Type “Number”
     • Retweets: Type “Number”
3. Share Database with Integration:
   • Open your newly created “Twitter Bookmarks” database page.
   • Click the “Share” button at the top right.
   • Click “Invite” and select your “Twitter Bookmark Sync” integration from the list. Grant it full access.

Now, locate your Notion Database ID. The Database ID is found in the URL of your database page, typically between the workspace name and the page title. It’s a 32-character alphanumeric string. For example: https://www.notion.so/{workspace_name}/{DATABASE_ID}?v=...

Add your Notion credentials to your config.env file:

# config.env (append to existing content)
NOTION_TOKEN="YOUR_NOTION_INTERNAL_INTEGRATION_TOKEN"
NOTION_DATABASE_ID="YOUR_NOTION_DATABASE_ID"

To verify Notion API access, we can try to query our database:

import os
from notion_client import Client
from dotenv import load_dotenv

load_dotenv('config.env')

notion_token = os.getenv("NOTION_TOKEN")
notion_database_id = os.getenv("NOTION_DATABASE_ID")

notion = Client(auth=notion_token)

try:
    # Test by querying the database (e.g., retrieve existing pages)
    response = notion.databases.query(database_id=notion_database_id)
    print("Successfully connected to Notion database.")
    print(f"Found {len(response['results'])} existing entries.")
except Exception as err:
    print(f"Error connecting to Notion: {err}")

If successful, this script will confirm the connection to your Notion database and show the number of existing entries (likely zero if it’s a new database).

Step 3: Develop the Export Script

Now, let’s combine the pieces into a complete Python script. This script will:

1. Load credentials.
2. Fetch bookmarked tweets from Twitter, handling pagination.
3. Retrieve existing Tweet IDs from Notion to prevent duplicates.
4. For each new bookmark, extract relevant data and push it to Notion.

Create a file named export_bookmarks.py:

import os
import requests
from requests_oauthlib import OAuth1Session
from notion_client import Client
from dotenv import load_dotenv
from datetime import datetime

# Load environment variables
load_dotenv('config.env')

# Twitter API Credentials
TWITTER_CONSUMER_KEY = os.getenv("TWITTER_CONSUMER_KEY")
TWITTER_CONSUMER_SECRET = os.getenv("TWITTER_CONSUMER_SECRET")
TWITTER_ACCESS_TOKEN = os.getenv("TWITTER_ACCESS_TOKEN")
TWITTER_ACCESS_TOKEN_SECRET = os.getenv("TWITTER_ACCESS_TOKEN_SECRET")
# IMPORTANT: Replace with your actual Twitter User ID
TWITTER_USER_ID = os.getenv("TWITTER_USER_ID", "YOUR_TWITTER_USER_ID") # Fallback if not in env

# Notion API Credentials
NOTION_TOKEN = os.getenv("NOTION_TOKEN")
NOTION_DATABASE_ID = os.getenv("NOTION_DATABASE_ID")

# Initialize clients
oauth = OAuth1Session(
    TWITTER_CONSUMER_KEY,
    client_secret=TWITTER_CONSUMER_SECRET,
    resource_owner_key=TWITTER_ACCESS_TOKEN,
    resource_owner_secret=TWITTER_ACCESS_TOKEN_SECRET
)
notion = Client(auth=NOTION_TOKEN)

# --- Helper functions ---

def get_existing_tweet_ids_from_notion(database_id):
    """Fetches all existing Tweet IDs from the Notion database."""
    existing_ids = set()
    has_more = True
    next_cursor = None

    while has_more:
        query_params = {
            "database_id": database_id,
            "filter": {
                "property": "Tweet ID",
                "text": {
                    "is_not_empty": True
                }
            }
        }
        if next_cursor:
            query_params["start_cursor"] = next_cursor

        response = notion.databases.query(**query_params)
        for page in response["results"]:
            if "Tweet ID" in page["properties"] and page["properties"]["Tweet ID"]["type"] == "rich_text":
                tweet_id_text = "".join([t["plain_text"] for t in page["properties"]["Tweet ID"]["rich_text"]])
                if tweet_id_text:
                    existing_ids.add(tweet_id_text)

        has_more = response["has_more"]
        next_cursor = response["next_cursor"]
    return existing_ids

def add_tweet_to_notion(tweet_data, database_id):
    """Adds a single tweet as a page to the Notion database."""
    tweet_id = tweet_data['id']
    tweet_url = f"https://twitter.com/{tweet_data['author_username']}/status/{tweet_id}"

    properties = {
        "Tweet URL": {"url": tweet_url},
        "Content": {"rich_text": [{"text": {"content": tweet_data['text']}}]},
        "Author": {"rich_text": [{"text": {"content": tweet_data['author_username']}}]},
        "Bookmarked At": {"date": {"start": datetime.now().isoformat()}},
        "Tweet ID": {"rich_text": [{"text": {"content": tweet_id}}]},
        "Likes": {"number": tweet_data.get('public_metrics', {}).get('like_count', 0)},
        "Retweets": {"number": tweet_data.get('public_metrics', {}).get('retweet_count', 0)}
    }

    try:
        notion.pages.create(
            parent={"database_id": database_id},
            properties=properties
        )
        print(f">>> Added Tweet ID {tweet_id} to Notion.")
    except Exception as e:
        print(f"!!! Error adding Tweet ID {tweet_id} to Notion: {e}")

# --- Main script logic ---

def main():
    if TWITTER_USER_ID == "YOUR_TWITTER_USER_ID":
        print("CRITICAL: Please set your TWITTER_USER_ID in config.env or directly in the script.")
        return

    print("Fetching existing Tweet IDs from Notion...")
    existing_notion_tweet_ids = get_existing_tweet_ids_from_notion(NOTION_DATABASE_ID)
    print(f"Found {len(existing_notion_tweet_ids)} existing tweets in Notion.")

    bookmarks_url = f"https://api.twitter.com/2/users/{TWITTER_USER_ID}/bookmarks"
    tweet_fields = "author_id,created_at,text,entities,public_metrics"
    expansions = "author_id" # To get author details

    all_bookmarks = []
    next_token = None
    processed_count = 0

    print("Fetching bookmarks from Twitter...")
    while True:
        params = {"tweet.fields": tweet_fields, "expansions": expansions, "max_results": 100}
        if next_token:
            params["pagination_token"] = next_token

        try:
            response = oauth.get(bookmarks_url, params=params)
            response.raise_for_status()
            json_response = response.json()

            tweets = json_response.get("data", [])
            includes = json_response.get("includes", {})
            users = {user["id"]: user for user in includes.get("users", [])}

            for tweet in tweets:
                author_id = tweet.get("author_id")
                author_username = users.get(author_id, {}).get("username", "Unknown Author")
                tweet["author_username"] = author_username # Add author username for Notion
                all_bookmarks.append(tweet)

            processed_count += len(tweets)
            print(f"Fetched {processed_count} tweets from Twitter API so far...")

            next_token = json_response.get("meta", {}).get("next_token")
            if not next_token:
                break # No more pages

        except requests.exceptions.HTTPError as err:
            print(f"!!! HTTP error fetching tweets: {err}")
            print(f"Response body: {response.text}")
            break
        except Exception as err:
            print(f"!!! An error occurred fetching tweets: {err}")
            break

    print(f"Total bookmarks fetched from Twitter: {len(all_bookmarks)}")
    new_tweets_added = 0

    for tweet in all_bookmarks:
        if tweet["id"] not in existing_notion_tweet_ids:
            add_tweet_to_notion(tweet, NOTION_DATABASE_ID)
            new_tweets_added += 1
        # else:
            # print(f"--- Tweet ID {tweet['id']} already exists in Notion. Skipping.")

    print(f"\nScript finished. Added {new_tweets_added} new bookmarks to Notion.")

if __name__ == "__main__":
    main()

Ensure your config.env contains the TWITTER_USER_ID. You can find your Twitter user ID by using an endpoint like https://api.twitter.com/2/users/by/username/YOUR_HANDLE with your Bearer Token (if you have one) or by other means.

To run the script:

python3 export_bookmarks.py

This script will connect to both APIs, fetch your bookmarks, check for duplicates in Notion, and then add any new bookmarks to your Notion database.

Step 4: Automate the Export (Optional but Recommended)

To keep your Notion database constantly updated, you can schedule this script to run periodically using a cron job (on Linux/macOS) or a similar task scheduler.

First, ensure your Python script has execution permissions:

chmod +x export_bookmarks.py

Then, edit your user’s cron table:

crontab -e

Add the following line to run the script every hour. Remember to replace /path/to/your/project/ with the actual path to your script and /usr/bin/python3 with your Python executable’s path.

0 * * * * /usr/bin/python3 /path/to/your/project/export_bookmarks.py >> /tmp/logs/twitter_notion_sync.log 2>&1

This cron entry will run the script at the beginning of every hour, redirecting all output to /tmp/logs/twitter_notion_sync.log for auditing and debugging. Remember the WAF bypass rules: we use /tmp/logs/ instead of /var/log/.

Common Pitfalls

• Twitter API Rate Limits: The Twitter API has rate limits. If you have a large number of bookmarks or run the script too frequently, you might hit these limits (e.g., 15 requests per 15 minutes for the Bookmarks API). The script includes pagination handling, which helps, but aggressive polling might still be an issue. Implement exponential backoff if you find yourself frequently hitting rate limits. The requests_oauthlib library generally handles some aspects of this, but it’s good to be aware.
• Notion API Permissions: Ensure your Notion integration has explicit “Can edit content” access to the specific database you’re targeting. If the integration does not have access, you will receive permission errors (e.g., 403 Forbidden). Double-check the sharing settings for your Notion database.
• Incorrect Credentials or IDs: A significant number of errors stem from typos or incorrect API keys, access tokens, Notion integration tokens, or database IDs. Always double-check these values in your config.env file.

Conclusion

You have successfully built an automated pipeline to transfer your Twitter bookmarks into a structured Notion database! This solution empowers you with unparalleled organization, searchability, and analysis capabilities for your saved tweets, far exceeding what Twitter’s native bookmarking offers. By integrating these two powerful platforms, you’ve transformed a siloed collection of links into a dynamic knowledge base.

Moving forward, consider enhancing this solution by:

• Adding more tweet attributes to your Notion database (e.g., original tweet URL if it’s a retweet, hashtags, media URLs).
• Implementing a dedicated logging mechanism instead of just redirecting output.
• Creating a more user-friendly interface or a dedicated service for managing your OAuth flow if you intend to share this with others or manage multiple user accounts.
• Exploring filtering options in Notion to categorize your bookmarks further.

Keep automating, keep resolving, and stay productive!

---

👉 Read the original article on TechResolve.blog

---

☕ Support my work

If this article helped you, you can buy me a coffee:

👉 https://buymeacoffee.com/darianvance