How to do Twitter authentication with devise_token_auth

Josh Chen — Thu, 16 Dec 2021 07:09:15 +0000

Acknowledgement

I would like to express my sincere gratitude to @risafj for writing the Guide to devise_token_auth: Simple Authentication in Rails API.

I have implemented an authentication system for my Rails API by following @risafj's guide, and I would like to add Sign in with Twitter on top of it. I can't find any good tutorial on how to add Twitter authentication using Omniauth (gem) with devise_token_auth. So instead, I decided to make up (hack) my own solution.

Note: Guide is for Linux or MacOS.

Demo video

User workflow

You click the "Sign in with Twitter" button
You get redirected to a consent screen
You click "Authenticate app"
The page redirects to the actual application

Authentication workflow

Creating a new Twitter application

To make it possible for us to add Twitter authentication to our website, we need to register a new Twitter application on Twitter. This is a necessary step for getting the key and secret code for all communication between our application and Twitter. This is necessary for our application to confirm our identify to Twitter.

We can create a new Twitter application here by clicking Create Project button.

Next, we need to enter our application details.

After we have registered our application, we will get a screen where we can find the key and the secret of our Twitter application, the tab Keys and Access Tokens. Later we will need to use these keys, so save them in your computer for now.

Once this is done, you can use the left side menu bar to navigate to your project to config User authentication settings.

Pick the v1 API

On the bottom of the page we can find the Callback URI / Redirect URL, and Website URL. Fill them up and press the Save button to update the settings. Our Twitter application is now ready.

Open your Rails project, and create a .env file in the project root (if you don't have one yet). Add the Twitter keys you obtain earlier.

# .env

TWITTER_API_KEY=KEY FORM TWITTER
TWITTER_API_SECRET=KEY FORM TWITTER
TWITTER_API_BEARER_TOKEN=KEY FORM TWITTER
TWITTER_API_CALLBACK=http://localhost:4200/twitter_login_oauth_callback

The TWITTER_API_CALLBACK is pointing to your client app, and need to match with the one we submitted to Twitter earlier.

Add Twitter OAuth helper file

Create a twitter.rb file inside app/services/Oauth/ and copy and paste the code below to the file.

# app/services/Oauth/twitter.rb
require 'net/http'

module Oauth
  class Twitter
    def initialize(params)
      @callback = params[:callback]
      @app_key = params[:app_key]
      @app_secret = params[:app_secret]
    end


    def get_redirect
      # our front end will go to the callback url
      # and user will need to login from there

      # e.g.
      # https://api.twitter.com/oauth/authenticate?oauth_token=Bviz-wAAAAAAiEDZAAABdOLQn-s
      tokens = get_request_token
      oauth_token = tokens["oauth_token"]
      oauth_token_secret = tokens["oauth_token_secret"]

      callback_url = "https://api.twitter.com/oauth/authenticate?oauth_token=#{oauth_token}"

      return {
        "oauth_token": oauth_token,
        "url": callback_url,
        "oauth_token_secret": oauth_token_secret
      }

    end 

    def obtain_access_token(oauth_token, oauth_token_secret, oauth_verifier)
      tokens = get_access_token(oauth_token, oauth_token_secret, oauth_verifier)
    end

    private

    def get_access_token(oauth_token, oauth_token_secret, oauth_verifier)
      method = 'POST'
      uri = "https://api.twitter.com/oauth/access_token"
      url = URI(uri)
      oauth_timestamp = Time.now.getutc.to_i.to_s
      oauth_nonce = generate_nonce

      oauth_params = {
        'oauth_consumer_key' => @app_key, # Your consumer key
        'oauth_nonce' => oauth_nonce, # A random string, see below for function
        'oauth_signature_method' => 'HMAC-SHA1', # How you'll be signing (see later)
        'oauth_timestamp' => oauth_timestamp, # Timestamp
        'oauth_version' => '1.0', # oAuth version
        'oauth_verifier' => oauth_verifier,
        'oauth_token' => oauth_token
      }

      oauth_params['oauth_callback'] = url_encode(@callback+"\n")
      oauth_callback = oauth_params['oauth_callback']

      base_string = signature_base_string(method, uri, oauth_params)      
      oauth_signature = url_encode(sign(@app_secret + '&', base_string))

      authorization = "OAuth oauth_callback=\"#{oauth_callback}\", oauth_consumer_key=\"#{@app_key}\", oauth_nonce=\"#{oauth_nonce}\", oauth_signature=\"#{oauth_signature}\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"#{oauth_timestamp}\", oauth_token=\"#{oauth_token}\", oauth_verifier=\"#{oauth_verifier}\", oauth_version=\"1.0\""
      # authorization = 'OAuth oauth_callback="http%3A%2F%2Flocalhost%3A9000%2Ftwitter_connection%0A", oauth_consumer_key="QJImAUogu5MUalOP2Tv5jRt3X", oauth_nonce="a9900fe68e2573b27a37f10fbad6a755", oauth_signature="Y6y8dg4ENFXorvDPu7kyjrdbVYI%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1601796648", oauth_token="NPASDwAAAAAAiEDZAAABdOzo3sU", oauth_verifier="KiPMEx5rkceLjH1sCV3LfIVsxko0sBrc%0A", oauth_version="1.0"'

      http = Net::HTTP.new(url.host, url.port)
      http.use_ssl = true
      http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      request = Net::HTTP::Post.new(url)
      request["authorization"] = authorization

      response = http.request(request)

      parse_response_body(response)
    end

    def get_request_token
      # https://wiki.openstreetmap.org/wiki/OAuth_ruby_examples
      # http://www.drcoen.com/2011/12/oauth-1-0-in-ruby-without-a-gem/
      # http://www.drcoen.com/2011/12/oauth-with-the-twitter-api-in-ruby-on-rails-without-a-gem/

      method = 'POST'
      uri = "https://api.twitter.com/oauth/request_token"
      url = URI(uri)
      oauth_timestamp = Time.now.getutc.to_i.to_s
      oauth_nonce = generate_nonce

      oauth_params = {
        'oauth_consumer_key' => @app_key, # Your consumer key
        'oauth_nonce' => oauth_nonce, # A random string, see below for function
        'oauth_signature_method' => 'HMAC-SHA1', # How you'll be signing (see later)
        'oauth_timestamp' => oauth_timestamp, # Timestamp
        'oauth_version' => '1.0' # oAuth version
      }

      oauth_params['oauth_callback'] = url_encode(@callback+"\n")

      base_string = signature_base_string(method, uri, oauth_params)
      oauth_signature = url_encode(sign(@app_secret + '&', base_string))

      oauth_callback = oauth_params['oauth_callback']

      authorization = "OAuth oauth_callback=\"#{oauth_callback}\", oauth_consumer_key=\"#{@app_key}\", oauth_nonce=\"#{oauth_nonce}\", oauth_signature=\"#{oauth_signature}\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"#{oauth_timestamp}\", oauth_version=\"1.0\""
      puts authorization

      http = Net::HTTP.new(url.host, url.port)
      http.use_ssl = true
      http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      request = Net::HTTP::Post.new(url)
      request["authorization"] = authorization
      response = http.request(request)

      parse_response_body(response)

    end

    def url_encode(string)
      CGI::escape(string)
    end

    def signature_base_string(method, uri, params)
      encoded_params = params.sort.collect{ |k, v| url_encode("#{k}=#{v}") }.join('%26')
      method + '&' + url_encode(uri) + '&' + encoded_params
    end

    def sign(key, base_string)
      # digest = OpenSSL::Digest::Digest.new('sha1')
      digest = OpenSSL::Digest::SHA1.new
      hmac = OpenSSL::HMAC.digest(digest, key, base_string)
      Base64.encode64(hmac).chomp.gsub(/\n/, '')
    end

    def generate_nonce(size=7)
      Base64.encode64(OpenSSL::Random.random_bytes(size)).gsub(/\W/, '')
    end  

    def parse_response_body(response)
      ret = {}
      body = response.read_body
      body.split('&').each do |pair|
        key_and_val = pair.split('=')
        ret[key_and_val[0]] = key_and_val[1]
      end

      ret
    end
  end
end

OAuth step-by-step

Step 1. User clicks the "Sign in with Twitter" button in the client app

Step 2. Redirect to the Twitter consent screen to authenticate our app.

In my case, clicking the button calls the oauth_sessions_controller's get_login_link method, which then redirect the page.

Create a new file oauth_sessions_controller.rb, and put it under your controllers/api/v1 directory

# controllers/api/v1/oauth_sessions_controller.rb

module Api::V1
  class OauthSessionsController < ApplicationController

    # GET /v1/twitter_sign_in_link
    def get_login_link
      redirect_to OauthSession.get_sign_in_redirect_link
    end
  end
end

Add a line in your config/routes.rb file to setup the /v1/twitter_sign_in_link endpoint.

# config/routes.rb

scope module: 'api' do
  namespace :v1 do
    ...

     get 'twitter_sign_in_link' => 'oauth_sessions#get_login_link'

    ...
  end
end

You might ask. What is OauthSession.get_sign_in_redirect_link? This is going to return us a redirect URL to the Twitter consent screen.

OauthSession

To implement the get_sign_in_redirect_link, we will first generate the OauthSession model.

Execute this from your command line.

rails g model OauthSession

This will do many things, including:

Create a OauthSession model, which stores information such as oauth_token, and oauth_token_secret, and a corresponding migration file

Add the following code to the newly generated migration file, which looks something like YYYYMMDDTTTT_create_oauth_sessions.rb

class CreateOauthSessions < ActiveRecord::Migration[6.1]
  def change
    create_table :oauth_sessions do |t|
      t.string :provider
      t.string :oauth_token
      t.string :oauth_token_secret

      t.timestamps
    end
  end
end

Migrate database by running rails db:migrate.

Implement the get_sign_in_redirect_link class method to OauthSession.

# models/oauth_session.rb

class OauthSession < ApplicationRecord
  def self.get_sign_in_redirect_link
    params = {callback: ENV['TWITTER_API_CALLBACK'], app_key: ENV['TWITTER_API_KEY'], app_secret: ENV['TWITTER_API_SECRET']}
    twitter_service = Oauth::Twitter.new(params)
    result = twitter_service.get_redirect // {oauth_token, oauth_token_secret, url}


    oauth_token = result[:oauth_token]    
    oauth_token_secret = result[:oauth_token_secret]

    # Save the token and secret to the table, so that we can use it later
    twitter_auth_session = OauthSession.where("oauth_token = ? and provider = ?", oauth_token, "Twitter").first
    unless twitter_auth_session
      twitter_auth_session = OauthSession.new()
    end

    twitter_auth_session.oauth_token = oauth_token
    twitter_auth_session.oauth_token_secret = oauth_token_secret
    twitter_auth_session.provider = "Twitter"
    twitter_auth_session.save

    return result[:url]
  end
end

A quick explanation of the function above:

Make an API to request to Twitter to get oauth_token, oauth_token_secret and url to consent screen.
Save the oauth_token, and oauth_token_secret to the table.

Step 3. The user clicks "Authenticate app" on the consent screen

Step 4. The page redirects to your callback_uri with oauth_token and oauth_verifier.

Example URL to redirect to your callback_uri:

http://localhost:4200/twitter_login_oauth_callback?oauth_token=NPcudxy0yU5T3tBzho7iCotZ3cnetKwcTIRlX0iwRl0&oauth_verifier=uw7NjWHT6OJ1MpJOXsHfNxoAhPKpgI8BlYDhxEjIBY

Our client app will need to make an HTTP POST request to our Rails API endpoint (/v1/twitter_sign_in) to pass oauth_token and oauth_verifier back to our backend server.

Add a line in your config/routes.rb file to setup the /v1/twitter_sign_in endpoint.

# config/routes.rb

scope module: 'api' do
  namespace :v1 do
    ...

     post 'twitter_sign_in' => 'twitter_auths#sign_in'

    ...
  end
end

Create a twitter_oauths_controller.rb and save to /controllers/api/v1/

# twitter_oauths_controller.rb

module Api::V1
  class TwitterOauthsController < ApplicationController

    # POST /v1/twitter_sign_in
    def sign_in
      # converting the request token to access token
      oauth_token = params[:oauth_token]
      oauth_verifier = params[:oauth_verifier]

      if oauth_token && oauth_verifier
        user = TwitterAuth.sign_in(oauth_token, oauth_verifier)
        if user
          header = user.create_new_auth_token
          response.set_header('access-token', header["access-token"]) 
          response.set_header('token-type', header["token-type"]) 
          response.set_header('client', header["client"]) 
          response.set_header('expiry', header["expiry"]) 
          response.set_header('uid', header["uid"]) 

          render json: {data: user}, status: :ok
        else
          res = {
            "success": false,
            "errors": [
              "Invalid login credentials. Please try again."
            ]
          }
          render json: res, status: :unauthorized         
        end
      else
        res = {
          "success": false,
          "errors": [
            "Failed to sign in with Twitter"
          ]
        }         
        render json: res, status: :unauthorized
      end

    end

  end
end

A quick explanation of the code above:

If oauth_token and oauth_verifier are available, then we will try to use them to sign in to our application.
If the user is available, then use create_new_auth_token method from devise_token_auth to generate the header meta

Step 5. Exchange the oauth token for an user

To implement the TwitterAuth.sign_in(oauth_token, oauth_verifier), we will first generate the TwitterAuth model.

Execute this from your command line.

rails g model TwitterAuth

Add the following code to the newly generated migration file, which looks something like YYYYMMDDTTTT_create_twitter_auths.rb

class CreateTwitterAuths < ActiveRecord::Migration[6.1]
  def change
    create_table :twitter_auths do |t|
      t.string :twitter_user_id
      t.string :screen_name

      t.references :user, foreign_key: true
      t.timestamps
    end
  end
end

Add the user has one twitter_auth relationship.

In the user.rb, add has_one :twitter_auth
In the twitter_auth.rb, add belongs_to :user

Migrate database by running rails db:migrate.

Implement the sign_in class method to TwitterAuth.

# /models/twitter_auth.rb

class TwitterAuth < ApplicationRecord
  belongs_to :user


  def self.sign_in(oauth_token, oauth_verifier)
    params = {callback: ENV['TWITTER_API_CALLBACK'], app_key: ENV['TWITTER_API_KEY'], app_secret: ENV['TWITTER_API_SECRET']}

    # 1. look up the oauth_token_secret by oauth_token
    twitter_auth_session = OauthSession.where("oauth_token = ? and provider = ?", oauth_token, "Twitter").first
    if twitter_auth_session
      twitter_service = Oauth::Twitter.new(params)
      oauth_token_secret = twitter_auth_session.oauth_token_secret
      twitter_resp = twitter_service.obtain_access_token(oauth_token, oauth_token_secret, oauth_verifier)

      twitter_user_id = twitter_resp["user_id"]
      screen_name = twitter_resp["screen_name"]

      # Here is the fun part
      # 1. we look up if the twitter_user_id exist
      # if yes, then it's an existing user
      # if no, then it's a new user

      # for new user we will create a new account
      # for existing user, we will look up the user info, and return the header for auth

      twitter_auth = TwitterAuth.find_by_twitter_user_id(twitter_user_id)
      if twitter_auth
        # return the user basic on the twitter id
        user = twitter_auth.user        
      else
        # insert a new twitter_auth and also create a new user account        
        str = (0...8).map { (65 + rand(26)).chr }.join
        password = Digest::SHA256.base64digest "#{twitter_user_id}#{screen_name}#{str}".first(8) # generate a password

        user = User.create(email: "#{screen_name}@yourwebsite.com", password: password)

        twitter_auth = TwitterAuth.new()
        twitter_auth.user_id = user.id
        twitter_auth.twitter_user_id = twitter_user_id
        twitter_auth.screen_name = screen_name
        twitter_auth.save        
      end

      twitter_auth_session.delete # remove the auth session 

      return user
    else
      return nil
    end
  end
end

That's all you need to setup a Sign in with Twitter for your Rails API.

In this post, we handled everything manually without replying on gems like omniauth-twitter. This is more a "hack" than a solution, it might be easier and better to use the omniauth-twitter gem. I hope you have enjoyed this tutorial, There is much more to be done, but this should get you started. Thanks for reading!

Add chapter markers to podcast/audio using FFmpeg

Josh Chen — Wed, 29 Sep 2021 06:26:03 +0000

Chapter markers are an easier way for listeners to navigate your podcast, see what's coming up, or skip over spoilers they don't want to hear. So I want to add the support for users to add chapter markers on JustCast. To do this, I turn to my weapons of choice, FFmpeg (v4.4).

Add chapter markers

FFmpeg can take metadata from a file and save it to the audio file.

ffmpeg -i "input-ep1-hello-world.mp3" -i "ep1-hello-world-metadata.txt" -map_metadata 1 -codec copy "output-ep1-hello-world.mp3"

ep1-hello-world-metadata.txt

;FFMETADATA1

[CHAPTER]
TIMEBASE=1/1000
START=0
END=60000
title=chapter 1

[CHAPTER]
TIMEBASE=1/1
START=60
END=90
title=chapter 2

We need to ensure two things: 1. end time needs to be greater than the start time, 2. the chapter start time needs to be greater than the last chapter end time.

https://ffmpeg.org/ffmpeg-formats.html#Metadata-1

fluent-ffmpeg

Fluent-FFmpeg

ffmpeg(url)
      .input(metafile)
      .audioCodec('copy')
      // .outputOptions('-metadata', 'title=song x')
      .outputOptions([
        "-map_metadata 0" // # 0 means copy whatever in the existing meta, 1 means ignore the existing
      ])
      .toFormat('mp3')
      .saveToFile(this.outputFilePath)   
      .on('codecData', (audioData) => {
        // console.log(getSecondsFromHHMMSS(audioData.duration))
        this.duration = getSecondsFromHHMMSS(audioData.duration);
      })

DEV Community: Josh Chen