loading...
Cover image for Joyful Tweets

Joyful Tweets

hammertoe profile image Matt Hamilton Updated on ・4 min read

Grant for the Web Hackathon (2 Part Series)

1) Joyful Tweets 2) Further Developing Joyful Tweets

What I built

There are a lot of pretty bad things going on in the world right now. Whilst not wanting to ignore or minimise them, sometimes it would be nice to just take a break for our own mental health. Wouldn't it be great if we could look at just the "joyful" tweets in our Twitter timeline?

I have build a small web app that authenticates to Twitter and fetches your Twitter timeline and filters it to analyse each of the tweets to just show the ones that might be "joyful" (and remove those that are classified as "angry").

Submission Category:

Creative Catalyst

Demo

Screenshot of Joyful Tweets

A working demo is running at:

https://joyfultweets-daring-wildebeest-eq.eu-gb.mybluemix.net/

This service may not be up all the time as is using a paid tone analyser service. So may be stopped to minimize costs. The idea is that WM will help fund services in the future.

Link to Code

GitHub logo hammertoe / JoyfulTweets

A simple web app to filter your Twitter timeline to just show Joyful Tweets.

Joyful Tweets

There are a lot of pretty bad things going on in the world right now. Whilst not wanting to ignore or minimise them, sometimes it would be nice to just take a break for our own mental health. Wouldn't it be great if we could look at just the "joyful" tweets in our Twitter timeline?

screenshot of Joyful Tweets

I have build a small web app that authenticates to Twitter and fetches your Twitter timeline and filters it to analyse each of the tweets to just show the ones that might be "joyful" (and remove those that are classified as "angry").

This was a part of the Grant for the Web / DEV Hackathon and was live-coded on a Twitch stream, a recording of which is at: https://www.cinnamon.video/watch?v=333183211095983753




Edit: When the hackathon was extended I went for a stretch goal and built my own tone analysis service, which the site now uses:

GitHub logo hammertoe / JoyDetector

A service that uses a pre-trained neural network model to detect tone in text

How I built it

A complete video of the build session (2.5 hours) is available on Youtube or Cinnamon:

https://www.cinnamon.video/watch?v=333183211095983753

The app is built using the following technologies:

  • Node.js, and Express, hosted on IBM Cloud Foundry
  • Passport.js for the OAuth dance with Twitter
  • Axios for making async parallel HTTP requests
  • IBM Watson Tone Analyzer to detect the tone of the tweets
  • Twitter API to fetch and render the tweet widgets
  • Masonry.js to layout the screen
  • And of course, Web Monetization via Coil

The browser makes a request to the backend which then directs it to do an OAuth dance with Twitter in order to obtain an API key to access the user's tweets.

app.get('/tweets',
        require('connect-ensure-login').ensureLoggedIn(),
        function(req, res) {

            var T = new Twit({
                consumer_key: process.env['TWITTER_CONSUMER_KEY'],
                consumer_secret: process.env['TWITTER_CONSUMER_SECRET'],
                access_token: req.user.token,
                access_token_secret: req.user.tokenSecret,
                timeout_ms:           60*1000,  // optional HTTP request timeout to apply to all requests.
                strictSSL:            true,     // optional - requires SSL certificates to be valid.
            })

            T.get('statuses/home_timeline', { count: 20,
                                              tweet_mode: 'extended' },

The backend then downloads the tweets from Twitter and then makes requests to the Watson Tone Analyzer:

                      const agent = axios.create({
                          timeout: 1000,
                          auth: {username: 'apikey',
                                 password: tone_key},
                          adapter: throttleAdapterEnhancer(axios.defaults.adapter, { threshold: 1000 })
                      });

                      let tweets = await Promise.all(data.map(async tweet => {
                          let status = tweet.retweeted_status || tweet;
                          let text = status.full_text;

                          // connect to tone analyser
                          try {
                              let tones = await agent.post(tone_url, {text: text});
                              tweet.tones = tones.data.document_tone.tones;
                          } catch (error) {
                              console.error(error);
                          }
                          return tweet;
                      }))

The tweets are then filtered to remove any that are protected, or have the anger tone, and just returns the ones that have the joy tone:

                          let joy_tweets = tweets.filter(tweet => {
                          if (tweet.tones) {
                              for (let i=0; i<tweet.tones.length; i++) {
                                  if(tweet.tones[i].tone_id == 'anger') {
                                      return false;
                                  }
                                  if(tweet.user.protected) {
                                      return false;
                                  }

                              }
                              for (let i=0; i<tweet.tones.length; i++) {
                                  if(tweet.tones[i].tone_id == 'joy') {
                                      return true;
                                  }
                              }
                          }
                      })

The tweets are then returned to the browser and the client-side JS then generates the <blockquote> tags needed for the Twitter widget to work, puts a colourful border around them and lays them out with Masonry.js:

   <script>
      $(document).ready(function() {
          $.getJSON("https://joyfultweets-daring-wildebeest-eq.eu-gb.mybluemix.net/tweets", data => {
              let i = 0;
              let colours = ['one', 'two', 'three', 'four', 'five'];
              data.tweets.forEach(tweet => {
                  let colour = colours[i % colours.length];
                  let url = `https://twitter.com/${tweet.user.screen_name}/status/${tweet.id_str}`;
                  i++;
                  console.log("tweet: ", url)
                  $("#container").append(`<div class="grid-item border ${colour}"><blockquote class="twitter-tweet"><a href="${url}"></blockquote></div>`)
              })
          }).then(() => {
              return twttr.widgets.load(
                  $("#container")
              );
          }).then(() => {
              return $("#fetching").hide();
          }).then(() => {
              return $('.grid').masonry({
                  // options
                  itemSelector: '.grid-item',
                  columnWidth: 560
              });
          })
      })
      </script>

Generally it all went well, even though I'm more comfortable in Python than Node.js. I decided to use Node as would be easier to do multiple parallel async HTTP requests out to the APIs. However this turned out be a blessing and a curse as I quickly overwhelmed the APIs and had to implement rate limiting to prevent overloading the API.

Additional Resources/Info

My plan is to continue developing this further after the submission and train my own neural network to detect tweet emotions. This will be written in Python and likely also hosted on Cloud Foundry and present a similar API to Watson Tone Analyzer than I can then redirect this app to.

Edit: due to the hackathon extension I went ahead and did this, see link above in code section.

I will not be submitting this to the main Grant for the Web process, as I am already submitting a larger, different project to that.

Grant for the Web Hackathon (2 Part Series)

1) Joyful Tweets 2) Further Developing Joyful Tweets

Posted on Jun 3 by:

hammertoe profile

Matt Hamilton

@hammertoe

Software developer. Developer Advocate at @IBMDeveloper. Tinkerer of motorbikes, vans, Arduinos, etc.

Discussion

markdown guide