DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Inspired by @thevuedev, I created @TheReactDev
Juliano Rafael
Juliano Rafael

Posted on

Inspired by @thevuedev, I created @TheReactDev

This weekend I stumbled upon this article:

And I immediately thought: "Hey, this is an awesome idea. Let's make one about React as well, cause why not?"

So I did. I created a bot that checks dev.to/t/react/latest every minute and tweets any new article. Here it is: @TheReactDev

TypeScript? Nah.

I choose not to do TypeScript on this one. It is a super small function and I felt like JavaScript would get the job done faster.

Serverless

You have many options when you do serverless. You can go AWS, Azure, GCloud, etc. The article explains in detail how to setup the Azure function and because I had never used Azure before, I followed along. The integration of Azure with VSCode is pretty cool, can't deny it, but I might switch up later on for the serverless framework to be sort of platform agnostic. Not that it matters for a project like this though. πŸ˜„

UPDATE: I had some headaches with azure. Probably because I don't have much experience with. So after a couple of hours I switched to AWS with the serverless framework for the tooling.

Scraping

Because dev.to does not have an api yet, I had to scrape away the data from the url. I had used puppeteer before, but I absolutely loved the x-ray lib. The API is so clean. I'll definitely use more of it.

Problems I had

At first, when I fetched the recent tweets from the @TheReactDev, the twitter api didn't always returned the posted urls on the entities.

[
  //...
  [ { url: 'https://t.co/AA3sKrLsDZ',
      expanded_url:
       'https://dev.to/peterj/run-a-react-app-in-a-docker-container-kjn',
      display_url: 'dev.to/peterj/run-a-r…',
      indices: [Array] } ],
  [ { url: 'https://t.co/hptME9oAgf',
      expanded_url: 'https://twitter.com/i/web/status/1102686466436681731',
      display_url: 'twitter.com/i/web/status/1…',
      indices: [Array] } ],
  // ..
]

Because of this, I started checking for the article title and author on the tweet body. The problem is that the twitter API sometimes truncate the body. The solution to this was to use the tweet_mode: "extended" on the request for the tweets.

But, just as I was writing this post, I observed that this also solves the problem of the links. With this mode, the twitter API not only returns the full_text but also returns the full extended_url as well. πŸ˜…

Checking for the URL is way more reliable so I'm rewriting my function to use this as well.

Telegram

I also added a telegram bot that sends a message to a channel in case things go wrong. I choose telegram because it'll probably be the fastest way that the notification reaches me.

The repo

GitHub logo frontendwizard / TheReactDev

The function that feeds @TheReactDev twitter.

TheReactDev

The function that feeds @TheReactDev twitter.

This project was inspired by this article.

This function depends on a few environment variables:

  • dev_tag: the dev.to tag to which the crawler will get the latest articles.
  • twitter_bot_screen_name: the name of the account that will tweet the articles.
  • Twitter keys
    • twitter_consumer_key
    • twitter_consumer_secret
    • twitter_access_token
    • twitter_access_token_secret
  • telegram_bot_token: the telegram bot who's going to log messages for maintenance/debug purposes
  • telegram_chat_id: the channel where the bot will send messages

You just need to create a env.yml file and put those values in, like this:

dev_tag: "react"
twitter_bot_screen_name: "NameOfTheTwitterAccount"
twitter_consumer_key: "..."
twitter_consumer_secret: "..."
twitter_access_token: "..."
twitter_access_token_secret: "..."
telegram_bot_token: "..."
telegram_chat_id: "..."

It's worth noting that I used telegram here because it was simple and it probably is the…

Like it?

Well, it was super fun doing this. The feeling that you get when you see the function publishing new articles on twitter is amazing. πŸ˜„

If you like it, be sure to follow the @TheReactDev on twitter!

Thank you for your time and until the next one!

Top comments (4)

Collapse
ben profile image
Ben Halpern

πŸ˜„

Let’s all work together to make these accounts as useful as possible!

Collapse
danielelkington profile image
Daniel Elkington

Awesome job! Reading up on tweet_mode: extended, I think this relates to Twitter's change in 2017 to go from 140 characters to 280 - so that there were no breaking changes the API by default returns 140 characters with a link to view the full tweet. You have to 'opt-in' to enabling more than 140 characters with tweet_mode: extended. I suspect I haven't had this problem with @TheVueDev yet because I'm not (yet) tweeting tags and the article titles haven't been too long, so all tweets have so far been under 140 characters - but I'll definitely make the change to include this parameter! Thanks for finding this future bug for me πŸ˜ƒ

Collapse
turnerj profile image
James Turner • Edited on

FYI, there is actually an API for articles etc for dev.to though it isn't really documented anywhere as far as I have seen. I found out about it from a comment Ben made on another post.

dev.to/api/articles?tag=react

There are a few other endpoints too if you want to get articles for a specific person etc.

For a specific article, you can use this if you know the ID: dev.to/api/articles/88679

Collapse
frontendwizard profile image
Juliano Rafael Author

That can eliminate the scraping :)
Will look into it today.

🌚 Browsing with dark mode makes you a better developer by a factor of exactly 40.

It's a scientific fact.