Wikiquote Twitter Bot

Damien LaRocque — Thu, 21 May 2020 20:08:55 +0000

Context

When you learn to code, it's always a good idea to work on a project that keeps you motivated. I always had a passion for IoT and APIs, so I decided to work on a small project that would allow me to use APIs. This project is a Wikiquote Twitter Bot that tweets inspiring quotes from Wikiquote once a day. You can find the source code here :

IamPhytan / wikiquote-twitter-bot

A simple way to tweet famous quotes from Wikiquote

Wikiquote Twitter Bot

A simple way to tweet famous quotes from Wikiquote

Motivation

I wanted to have a bot that would tweet different quotes from Wikiquote each morning to start the day on a good note. This small scripts uses Tweepy's API and fredericotdn's Wikiquote API to scan quotes on Wikiquote and tweet them .

Installation

The following steps explain how to configure the script before running it:

Clone the repo: git clone https://github.com/IamPhytan/wikiquote-twitter-bot.git
Download the requirements (or create a virtualenv before) : pip3 install -r requirements.txt
Follow this tutorial to get your :
- Consumer key
- Consumer secret
- Access token
- Access token secret
Add these values to the configuration file
Insert in the same configuration file the time you want to send quotes and the names of the people who inspire you.
Run the script: python3 main.py

License

MIT

View on GitHub

Development

This project was small enough to be completed in one day. I coded in Python, as this language is easy enough and has the perfect libraries for it. I planned to run the bot on a Raspberry Pi, so it was perfect to develop in Python. The project has two main parts :

Wikiquote Interface and quotes retrieval
Twitter Interface and tweet posting

Each of this part of the project was built in a module. Through this arrangement, I learned how to organize Python projects. Let me explain how each module was developed.

Retrieving the quotes from Wikiquote

For this part, I originally planned to scrape data from Wikiquote with requests and Beautiful Soup (bs3, at that time), two popular web scraping libraries... until I found wikiquote, a python library to retrieve quotes from Wikiquote. This library is available through PyPi, so it's easy to deploy on another device (a Raspberry Pi, in this case).

This project was the first where I had to create a configuration file. For that, I used a json file. This json file contains the names of the Wikiquote pages to scrape :

{
  "WIKIQUOTE": {
    "FAMOUS_PEOPLE": [
      "Albert Einstein",
      "Isaac Newton",
      "Marie Curie",
      "René Magritte",
      "Michel de Montaigne",
      "Friedrich Nietzsche",
      "Emmanuel Kant",
      "Platon",
      "René Descartes",
      "Jean-Jacques Rousseau",
      "Thomas Hobbes",
      "Aristote",
      "Socrate",
      "Baruch Spinoza",
      "Pierre Bourdieu",
      "Épicure",
      "Michel Foucault",
      "Jean-Paul Sartre",
      "Blaise Pascal",
      "Linus Torvalds",
      "Richard Stallman"
    ]
  }
}

Quotes from each author are then saved in a .txt file, so we can keep track of all the quotes that were scraped.

Posting the quotes to Twitter

The best and most popular library to tweet with Python is Tweepy. This library is available through PyPi, so it's stil easy to deploy on another device (a Raspberry Pi, in this case).

Before posting to Twitter, we need to configure the API with secret keys. My code for that part was mainly inspired by a tutorial from Nael Shiab. This tutorial is not available anymore, so here is another one that explains well how to set up your Twitter account.

As always, the Twitter keys were added to the json configuration file :

{
  "TWITTER": {
    "CONSUMER_KEY": "<consumer-key>",
    "CONSUMER_SECRET": "<consumer-secret>",
    "ACCESS_TOKEN": "<access-token>",
    "ACCESS_TOKEN_SECRET": "<access-token-secret>"
  }
}

My goal is to tweet quotes that weren't tweeted before by the bot. For that, each quote is identified by an index in a list and I made a list, called used_tweets_idxs, that contains all the indexes of tweeted quotes. Each day, a quote is chosen randomly. If this quote was already tweeted, another quote is chosen randomly. The function responsible to choose the quote is therefore called recursively until the chosen quote was not tweeted before.

main.py : The conductor

The two previous parts are called from main.py. This script is therefore the central part of the project that conducts everything. Its main components are the following :

Assertion of the interpreter (must be run with Python 3)
Loading the config from the json configuration file
Call of the wikiquote module to retrieve quotes
Call of the twitter module to create the Twitter API Wrapper
Tweet quote if current time is equal to the time in configuration file and while not all quotes were tweeted

Next steps

This project was a great beginner project, but as all beginner projects, it has some lacks, in particular the fact that this script runs permanently. This is problematic when we run this bot on small computers with low processing power (i.e. : Raspberry Pi Zero), especially if we want to deploy multiple bots in parallel.

I since learned the existence of crontab. This UNIX tool can be used to run scripts in the background at regular intervals. Removing the while loop and running the script with crontab would release processing power and a thread for other programs.

Also, the use of lists to keep data is a bad idea, as it is hard to scale up (i.e. : if we have more than a million quotes to tweet). Now that the script should not run permanently, all the lists will not keep the values after each execution of the code. The quotes should therefore be saved in a file, maybe a json file.

In that case, it would also be beneficial to save those quotes in two different json keys, depending on whether they were posted on Twitter. This change would avoid to call functions recursively.

Finally, it would a good idea to implement the functions of the modules in classes, so that variables (i.e. : the Twitter API Handler) can be used by multiple functions (now, methods) instead of having to call functions with lots of parameters.

Conclusion

I hope that this post allowed you to learn how to avoid making some beginner mistakes. Lots of googling was involved through this whole project and it was quite challenging, even if this was a small project.

Keep coding ! ⌨️

DEV Community: Damien LaRocque

Wikiquote Twitter Bot

Context

IamPhytan / wikiquote-twitter-bot

A simple way to tweet famous quotes from Wikiquote

Wikiquote Twitter Bot

Motivation

Installation

License

Development

Retrieving the quotes from Wikiquote

Posting the quotes to Twitter

main.py : The conductor

Next steps

Conclusion