How I built SetProgress - Part 3 Automation

In the previous post, I explained how I built SetProgress.

In this post, I am going to explain how I automated progress updates.

Automation was not part of my initial plan. I just wanted to create a simple app which allows a user to design & push a progress bar to Twitter.

Sumudu Siriwardana asked me if the app could calculate & update progress for her #100DaysOfCode challenge.

I thought about it, and I realized that the app doesn't offer much automation. User has to update their progress manually everyday or every week.

Automating #100DaysOfCode

Let me give you a bit of context on 100DaysOfCode.

It has 2 rules

1. Code minimum an hour every day for the next 100 days.
2. Tweet your progress every day with the #100DaysOfCode hashtag.

Anyone who is doing this challenge will tweet everyday tagging 100DaysOfCode and a little bit about what they did that day.

Solution

We can fetch all the tweets of a user and check if any one of those have the hashtag 100DaysOfCode.

I used the Twitter search tweets API to get the tweets of a user containing a specific hashtag and in a given date range.

This is how the request looks.

https://api.twitter.com/1.1/search/tweets.json?q=from%3Asdinesh91%20since%3A2021-12-24%20until%3A2021-12-25%20#100DaysOfCode%20-filter%3Aretweets

These are the query parameters I used

from:sdinesh91 since:2021-12-24 until:2021-12-25 #100DaysOfCode -filter:retweets

This query returns tweets of a specific user, tweeted within a specific date range and having the specified hashtag. We also don't need to fetch the retweets, so I filter them out.

We don't have to count multiple tweets containing the hastag in a single day. 100DaysOfCode is progress made in days and not in the number of times tweeted.

Having fetched the tweets it is very simple to count the progress. If I find a tweet for any given day, I increment the count. Once I have the count, I update the progress in the database and push the new progress bar to Twitter.

One important aspect of writing a backend job like this, is to make it Idempotent. Being idempotent, ensures that we obtain the same result irrespective of the number of times we run the code. This is critical as backend jobs might fail or timeout. In such cases, we should be able to just rerun the job and things should work as expected.

To make tracker job idempotent, I store the tracker's last run date to each user's config. This allows the tracker to resume processing from where it last left by comparing the last run date.

Tech

I have hosted the tracker on Netlify as a function. I also use EasyCron to trigger the tracker function every day.