DEV Community

Cover image for How to Scrape Tweets from Twitter with Python using Tweeds
Achyuth Jois M
Achyuth Jois M

Posted on

How to Scrape Tweets from Twitter with Python using Tweeds

What is Tweeeds?🌱

Tweeds is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter. The name might be funny but "Don't Judge a book by it's cover"😏

GitHub logo achyuthjoism / tweeds

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a Tweets and more while evading most API limitations.

Tweeds - Twitter Scraping Tool

Scrape tweets from any Twitter user profile. Twitter API alternative to scrape Twitter hashtags, threads, images, videos, statistics and Twitter history. Export data in JSON and CSV format. This project enables you to extract large amounts of data from Twitter It lets you do much more than the Twitter API, because it doesn't have rate limits and you don't even need to have a Twitter account, a registered app or Twitter API key.

Do not forget to star this project.🌟 😍

πŸ’‘ Features

  • No API Key required
  • No Limit
  • No Authentication required
  • Get tweets from specific user
  • JSON and CSV export for further Data Analysis
  • And the best part it is OpenSource πŸ˜‰

βœ”οΈ Requirements

  • Python >= 3.9

βš™ Installation

pip install tweeds
Enter fullscreen mode Exit fullscreen mode

πŸ’ƒ Usage

achyuthjoism-MBP ~ %tweeds
        β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–ˆβ–ˆβ•—β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—
        β•šβ•β•β–ˆβ–ˆβ•”β•β•β•β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•”β•β•β•β•β•β–ˆβ–ˆβ•”β•β•β•β•β•β–ˆβ–ˆβ•”β•β•β–ˆβ–ˆβ•—β–ˆβ–ˆβ•”β•β•β•β•β•
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β•šβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•‘β•šβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ•‘β–‘β–ˆβ–ˆβ•”β•β•β•β–‘β–‘β–ˆβ–ˆβ•”β•β•β•β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β•šβ•β•β•β–ˆβ–ˆβ•—
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β–‘β•šβ–ˆβ–ˆβ•”β•β–‘β•šβ–ˆβ–ˆβ•”β•β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•
        β–‘β–‘β–‘β•šβ•β•β–‘β–‘β–‘β–‘β–‘β–‘β•šβ•β•β–‘β–‘β–‘β•šβ•β•β–‘β–‘β•šβ•β•β•β•β•β•β•β•šβ•β•β•β•β•β•β•β•šβ•β•β•β•β•β•β–‘β•šβ•β•β•β•β•β•β–‘

        by Achyuth Jois M

usage: tweeds [-h] [-u U] [-s
…
Enter fullscreen mode Exit fullscreen mode

Why should I use Tweeds?🐐

  1. You don't need a Twitter API for scraping tweets
  2. You can scrape all the tweets(Twitter API limits 3000 tweets only😒!)
  3. No Twitter credentials required
  4. No limitation
  5. It's OpenSource πŸ˜ƒ

Prerequisites βœ…

  • Python >= 3.9

Installation βš™

pip install tweeds
Enter fullscreen mode Exit fullscreen mode

And you are done! Now you can use this tool in your terminal or in your python project. In this post we will discuss both the use case.😎

Command Line Interface πŸ’»

  • Scrape all the tweets from Elon Musk(including replies).
tweeds -u elonmusk
Enter fullscreen mode Exit fullscreen mode
  • Scrape all Tweets from the Elon's timeline containing pineapple.
tweeds -u elonmusk -s pineapple
Enter fullscreen mode Exit fullscreen mode
  • Scrape all Tweets from user's tweeting about Coffee near Berlin.
tweeds -s Coffee --near Berlin
Enter fullscreen mode Exit fullscreen mode
  • Want to add a limit just add --limit argument.
tweeds -s Coffee --near Berlin --limit 1000
Enter fullscreen mode Exit fullscreen mode
  • Scrape all Tweets from Elon's timeline since 2022-12-10.
tweeds -u elonmusk --since 2022-12-10
Enter fullscreen mode Exit fullscreen mode
  • Scrape Tweets from a radius of 1km around a place in Paris.
tweeds --geocode "48.880048,2.385939,1km" --limit 10000  
Enter fullscreen mode Exit fullscreen mode
  • Want to export the data into CSV or JSON for further analysis just add --csv or --json argument with the filename.
tweeds --geocode "48.880048,2.385939,1km" --csv paris.csv --limit 10000
Enter fullscreen mode Exit fullscreen mode

Refer Docs for more example.

Modules ⚑

Now Tweed can be used as a module and allows custom formatting. Example:

import tweeds

query = tweeds.Query()

query.search = "Yash Boss"
query.limit = 10
query.verified = True

tweeds.search(query)
Enter fullscreen mode Exit fullscreen mode

Remember you have 20 options to fiddle with🀯.

Conclusion πŸ‘‹

This project was made only for educational purpose only. Thank you for following this tutorial to the end. Hopefully this article helps you, see you in the next article.

GitHub logo achyuthjoism / tweeds

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a Tweets and more while evading most API limitations.

Tweeds - Twitter Scraping Tool

Scrape tweets from any Twitter user profile. Twitter API alternative to scrape Twitter hashtags, threads, images, videos, statistics and Twitter history. Export data in JSON and CSV format. This project enables you to extract large amounts of data from Twitter It lets you do much more than the Twitter API, because it doesn't have rate limits and you don't even need to have a Twitter account, a registered app or Twitter API key.

Do not forget to star this project.🌟 😍

πŸ’‘ Features

  • No API Key required
  • No Limit
  • No Authentication required
  • Get tweets from specific user
  • JSON and CSV export for further Data Analysis
  • And the best part it is OpenSource πŸ˜‰

βœ”οΈ Requirements

  • Python >= 3.9

βš™ Installation

pip install tweeds
Enter fullscreen mode Exit fullscreen mode

πŸ’ƒ Usage

achyuthjoism-MBP ~ %tweeds
        β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–ˆβ–ˆβ•—β–‘β–‘β–‘β–‘β–‘β–‘β–‘β–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—
        β•šβ•β•β–ˆβ–ˆβ•”β•β•β•β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•”β•β•β•β•β•β–ˆβ–ˆβ•”β•β•β•β•β•β–ˆβ–ˆβ•”β•β•β–ˆβ–ˆβ•—β–ˆβ–ˆβ•”β•β•β•β•β•
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β•šβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•‘β•šβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–‘
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β–‘β–ˆβ–ˆβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ•‘β–‘β–ˆβ–ˆβ•”β•β•β•β–‘β–‘β–ˆβ–ˆβ•”β•β•β•β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β•šβ•β•β•β–ˆβ–ˆβ•—
        β–‘β–‘β–‘β–ˆβ–ˆβ•‘β–‘β–‘β–‘β–‘β–‘β•šβ–ˆβ–ˆβ•”β•β–‘β•šβ–ˆβ–ˆβ•”β•β–‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•
        β–‘β–‘β–‘β•šβ•β•β–‘β–‘β–‘β–‘β–‘β–‘β•šβ•β•β–‘β–‘β–‘β•šβ•β•β–‘β–‘β•šβ•β•β•β•β•β•β•β•šβ•β•β•β•β•β•β•β•šβ•β•β•β•β•β•β–‘β•šβ•β•β•β•β•β•β–‘

        by Achyuth Jois M

usage: tweeds [-h] [-u U] [-s
…
Enter fullscreen mode Exit fullscreen mode

Top comments (3)

Collapse
 
inirayjo profile image
Umat Manusia

just asking, can we get a tweet by id ? i have bunch of dataset that only have the id of the tweet (50K Dataset). it will took a long time to collecting all of them. can you tell me how we can manage it ? so i could collect by id

Collapse
 
tradinghkb profile image
TraderHKB

It's amazingly great but needs to be faster if possible !

Do you have an idea how I can get in contact with you ? Do you use discord buddy ?

Collapse
 
sbouwnsv profile image
Sukhman

Wow some useful tool