DEV Community: Galih

How I Built a Chrome Extension That Parses Any Job Site Without Scraping

Galih — Wed, 20 Aug 2025 12:13:48 +0000

Nine months ago, I got laid off. During my brief spell looking for a job (before deciding to go indie), as I have always been, I wanted to do it in an organized manner.

I used to rely on Airtable base for collecting all the job postings I found online, to then take notes and track activities and so on, but this time decided to look for a ready-made solution. Found some cool products but they are too expensive for my liking, costing between $20-40 a month!

That is how the idea of HuntingPad was born and the core feature seemed simple: clip job posts with one click from anywhere in the browser!

Spoiler of what I ended up achieving

What I didn't expect was that this "simple" feature would lead me through a maze of technical decisions - from expensive webscraping APIs to LLM token optimization.

What I wanted to have as the UX is something like this:

whenever user is viewing a job posting, using just one click the user should be able to save the job post immediately and have the important info parsed and structured nicely without manual intervention using AI.

There are couple of questions/challenges about this:

What APIs available to a chrome extension that enables this flow? I know at least one, but what are the options?
Once we’re able to trigger the clipping, how do I go about to get the job posting content?

I decided two study the question #2 first since that seemed more critical.

✂ Getting the content of the job posting

Here are the alternatives that I was considering:

Build my own webscraping system.
Use webscraper API → feed the content to LLM → structured data.
Use regex to parse job data, skipping webscraper API → feed the content to LLM → structured data.
Use LLM directly to both fetch, parse, and structure the job posting in one or two calls.

I wanted to enable users on a free plan to use the extension because I believe it should be useful for job seekers from all economic backgrounds. This makes cost the most important criteria in choosing the solution, we need to come up with economically-viable one.

Option #1 was immediately out of the question—too large in scope for a project with no guarantee of ROI. It would be fun, don't get me wrong, but fun pays no bills unfortunately :).

The Webscraper API initially seemed like the obvious choice. But after researching some providers, I discovered most API providers have pricing that gets quite steep. This option is not the most viable.

Next, I considered writing regex patterns to parse job data from different sites. LinkedIn has one structure, Indeed another, and company career pages vary wildly. After thorough discussions with my friend Claude, I realized this approach was doomed.

This left me with just one alternative from my original list: having the LLM alone handle scraping, parsing, and structuring the job posting. My instinct said it would be prohibitively expensive. I proceeded to test it on three job postings—one from LinkedIn, one from Indeed, and another from a company career page.

The results confirmed my suspicion: too expensive.

This forced me to reconsider the webscraping API option, so I decided to go with it (though this wasn't the end of the story).

▶ The Trigger

With the fetching job posting content settled, I researched how to actually trigger the job posting clipping flow. From my experience with other extensions, I knew we could click on the extension icon in the browser bar, but I wondered what other options were available.

Then I found chrome.contextMenus .

This is what I want, for the user on any job posting page to just right-click and select one option from the menu like “Clip job post with HuntingPad” and it will just work. I was sold. I was very glad this one is a much easier decision to reach.

↩ Back to the idea of scraping

Remember about how I settled with the Webscraping API decision earlier? Having researched about the context menu, I thought of something.

What if we let users "scrape" the job posting themselves? We could ask them to highlight the text they want to clip before right-clicking and selecting "Clip job post with HuntingPad." The extension's background script would detect this selection and extract the highlighted HTML tags before sending them to the backend for processing.

While this approach deviates from my ideal UX flow, it offers significant benefits: eliminating the cost of a scraping API and removing a potential source of latency and errors from integrating with another API provider.

(Let's be honest—it was mainly about the cost 😅).

But just when I thought we were in the clear, I discovered another challenge that threatened to derail our progress!

🚧 The challenge in user-led highlighting

On our background script, we need to determine both the start and end parts of the html page to pick up. So we need to find the smallest/closest common ancestor that the start (usually the html tag of the job title heading) and the end part (the last sentence of the job description content).

function findCommonParent(elementA: HTMLElement, elementB: HTMLElement): HTMLElement | null {
  // Get all parents of element A
  const parentsA: HTMLElement[] = [];
  let currentA: HTMLElement | null = elementA;

  while (currentA && currentA.tagName !== 'BODY' && currentA.tagName !== 'HTML') {
    parentsA.push(currentA);
    currentA = currentA.parentElement;
  }

  // Check if any parent of element B matches a parent of element A
  let currentB: HTMLElement | null = elementB;
  while (currentB && currentB.tagName !== 'BODY' && currentB.tagName !== 'HTML') {
    // Check if this element is in the parents of A
    const index = parentsA.indexOf(currentB);
    if (index >= 0) {
      // We found a common parent, now check if it's a block element
      const commonParent = parentsA[index];

      // Is it already a block-level element?
      const style = window.getComputedStyle(commonParent);
      if (style.display === 'block' || style.display === 'flex' || style.display === 'grid') {
        return commonParent;
      }

      // If not a block element, try to find its closest block parent
      const blockParent = findClosestBlockParent(commonParent);
      if (blockParent) {
        return blockParent;
      }

      // If no block parent found, just return the common parent
      return commonParent;
    }

    currentB = currentB.parentElement;
  }

  // No common parent found (shouldn't happen as body/html would be common)
  return null;
}

I know that job boards do not share common job posting pages structure, but what I did not expect was how varied they are in a way that highlighting/selecting the job posting text from title to the end can basically end up fetching almost 90 percent of the page content depends on how the pages are structured.

Illustration:

<body>
  <h1 id="job-title">Senior Backend Engineer at Anthropic</h1>
  // buttons, filters, and so on

  // left sidebar

  // scripts (yes, scripts)

  // code comments (expected, we're all humans) 

  <div id="job-description">
     Start of Job desc details
     // right sidebar

     // Another job recommendations

     // random empty html tags (p, div, hr, br)

     End of relevant job details
  </div>  
</body>

So when user highlights from top to bottom of the job posting, as we expect them to, the background scripts will pick too many irrelevant items. Trimming is needed then.

This is what I end up building:

function pruneJobAdHtml(container: HTMLElement): HTMLElement {
  // Create a deep clone of the element to avoid modifying the original DOM
  const clone = container.cloneNode(true) as HTMLElement;

  // 1. [Easy Part] Remove unwanted tags and all their descendants
  // using a constant of TAGS_TO_REMOVE and iterate over it
  // this includes removing tailwind or other styling classes

  // 2. Process images - replace with text nodes
  // I decided to not pick up any images for now

  // 3. Remove HTML comments, no need to explain

  // 4. Remove empty elements and attributes

  // 5. Flatten redundant containers
  // 6. Convert to semantic HTML
  // 7. Merge adjacent text nodes
}

The trimming step of #5, #6, and #7 were added later as refinements to further reduce the size of the “scraped” content so that the input token consumption by the LLM can be reduced.

From my observation when testing the effectiveness of the clipping, using both the selection function (findCommonParent) and the pruning above, the clipping can effectively reduces up to 80% of the content to feed to the LLM.

Illustration of the full flow:

⛳ Closing

This solution isn't perfect. Users have to select text instead of just clicking a button. But it taught me something valuable: sometimes the best technical decision isn't the most elegant one - it's the one that ships.

By requiring users to select the job posting text, we eliminated webscraping costs entirely and reduced LLM token usage by 80%. The extension stays free for job seekers who need it most, and the parsing is actually more accurate because users naturally select only the relevant content.

Are you building a chrome extension right now? Or thinking to build one and don’t know where to start? What walls have you hit building browser extensions? Feel free to drop a comment

PS. If any of you want to try using the Pro version of HuntingPad, let me know in the comment and I will happily give you discount

Easy SaaS subscription flow with Stripe Checkout in Rails

Galih — Sun, 18 Sep 2022 07:00:42 +0000

What we are going to build is a Stripe checkout integration for subscription flow of a SaaS app like this:

So the idea of the sequence is:

User views landing page

user clicks on one of the plans on pricing table

user gets redirected to the Stripe Checkout page

User fills the form and completes the checkout

user gets redirected to the registration form on our website

user adds password for authentication

user registered and authenticated, with subscription already provisioned.

In this guide, were assuming well sell a subscription of our web app in three tiers and only serve customers with USD and EUR as the currencies.

All the code in this article is available on this github repo.

Modelling user and account

On this guide, the assumption is that we are using devise for our authentication,

rails g devise:install

that means by running the above command, well already have users table and User model.

To model the Stripe Customer object

rails g model Account email stripe_customer_id
rails g model AccountUser user:references account:references role


class AccountUser < ApplicationRecord
    belongs_to :account
    belongs_to :user
end


class User < ApplicationRecord
    has_many :account_users
    has_many :accounts, through: :account_users
    # "devise" line added automatically by devise, no big deal. But notice we remove :confirmable
    devise :database_authenticatable, :registerable, :recoverable, :rememberable, :validatable, :lockable, :trackable
end


class Account < ApplicationRecord
    has_many :account_users
    has_many :users, through: :account_users, source: :user
end

Modeling the product and plan

For you not familiar with Stripe, Product object refers to something that you charge your customer for, basically things you sell be it a tangible goods like clothes or intangible things like subscription. Plan (or Price in Stripe) on the other hand, is merely the pricing unit for the Product, usually depicted in form of currency (USD, EUR, etc) and interval (monthly, quarterly, yearly).

For the Product, since there wont be many changes we expect to make for this kind of records, lets just model it into static values using a ruby class. You can make this in form of Ruby PORO, but to make it easier and cleaner, lets use ActiveHash gem for this.

Gemfile

gem 'active_hash'

Add the above line to your Gemfile then run bundle install.

After that create a file called product.rb inside the app/models/ directory.

class Product < ActiveHash::Base
  fields :stripe_product_id, :name, :tier, :unit_amounts
  create id: 1, stripe_product_id: 'LITE', name: 'Lite', tier: 'lite', unit_amounts: {
        month: { eur: 1000, usd: 1200 },
        year: { eur: 10000, usd: 12000 }
    }
  create id: 2, stripe_product_id: 'PRO', name: 'Pro', tier: 'pro', unit_amounts: {
        month: { eur: 2000, usd: 2400 },
        year: { eur: 20000, usd: 24000 }
    }
  create id: 3, stripe_product_id: 'PREMIUM', name: 'Premium', tier: 'premium', unit_amounts: {
        month: { eur: 4000, usd: 4800 },
        year: { eur: 40000, usd: 48000 }
    }
end

Now for the pricing object which is the Plan, we can do similar thing like the Product, but for this we can predict there will be more dynamism to the data, hence we go for full-blown database table. So lets run Rails scaffold for generating the model.

rails g model Plan product_id interval currency nickname unit_amount stripe_price_id

The scaffold command will automaticall generate the app/models/plan.rb, make sure after that you make that class similar to this below:

class Plan < ApplicationRecord
  # to link the db-backed Plan model with static ActiveHash model of Product
  extend ActiveHash::Associations::ActiveRecordExtensions
  belongs_to_active_hash :product

  delegate :tier, to: :product, prefix: true
end

Modeling the subscription and link it to the user

By running the command below in terminal, you will have subscription table and model automatically generated.

rails g model Subscription account:references plan:references stripe_subscription_id status

Go to the subscription model app/models/subscription.rb and make sure the model looks like this.

class Subscription < ApplicationRecord
  belongs_to :account
  belongs_to :plan

  delegate :product_tier, to: :plan, prefix: false
end

Before we forget, lets check the account and user models and add the lines below to make it easier to interact with the newly-created subscription model.

class Account < ApplicationRecord
    ## add this
    has_many :subscriptions
    has_one :current_subscription, -> { order(id: :desc) }, class_name: "Subscription"
end

class User < ApplicationRecord
    ## add this
    has_many :subscriptions, through: :accounts, source: :subscription
end

after all this, dont forget to run rails db:migrate .

Install ruby Stripe and Dotenv gems

We need to install Stripe ruby SDK gem, so shortly we can have with the Stripe API!

Add this to the Gemfile

gem 'stripe'
gem 'dotenv-rails', groups: [:development, :test]

Then run bundle install.

To store sensitive values such as API key for interacting with Stripe API, we can use .env file. Create .env file on the root of your repo and add your Stripe API key that you can retrieve from the developer dashboard.

STRIPE_SECRET_KEY=sk_test_111111111

Now that we have added the API key on the environment variable file, then create initializer file for the Stripe gem config/initializers/stripe.rb to set that key automatically.

Stripe.api_key = ENV['STRIPE_SECRET_KEY']

Seed Stripe product & price

To make it easier for setting and synchronizing up the data both in our database and Stripes one, we should have a script for this. You can make it as a rake task, but for the simplicity of running it, well use Rails seeds file.

db/seeds.rb

Product.all.each do |product|
  begin
    Stripe::Product.create(
      id: product.stripe_product_id,
      name: product.name
    )
  rescue Stripe::StripeError => error
  end

  # Fetch existing stripe prices of this product
  existing_stripe_prices = Stripe::Price.list(product: product.stripe_product_id)
  existing_stripe_prices.data.select do |price|
    plan = Plan.where(
      interval: price.recurring.interval,
      currency: price.currency.to_s,
      unit_amount: price.unit_amount,
      product_id: product.id
    ).first_or_initialize

    # this will enable us to sync the db records with Stripe
    plan.stripe_price_id = price.id
    plan.save
  end

  product.unit_amounts.each do |interval, data|
    data.each do |currency, amount|
      plan = Plan.where(
        interval: interval, currency: currency.to_s, unit_amount: amount, product_id: product.id
      ).first_or_initialize

      # skip creating the price in Stripe if already synced
      next if plan.stripe_price_id.present?

      stripe_price = Stripe::Price.create(
        product: plan.product.stripe_product_id,
        currency: plan.currency,
        unit_amount: plan.unit_amount,
        nickname: plan.nickname,
        recurring: { interval: plan.interval }
      )
      plan.update(stripe_price_id: stripe_price.id)
    end
  end
end

Whenever we want to seed or sync data to our database, we would just need to run this command:

rails db:seed

Do this if you have not, by the way, to continue follow along the tutorial. By now, we expect you already have records in both the Product and the Plan tables in the database, and also the matching records in Stripe.

Embed Stripe Pricing tables and Checkout

Create pricing table on Stripe Dashboard.

Add the pricing table on your landing page or pricing page, but in our case well add it to the landing page directly

<script async src="https://js.stripe.com/v3/pricing-table.js"></script>
<stripe-pricing-table pricing-table-id="prctbl_1LZ8xoxoxoxox"
publishable-key="pk_test_90Jbzzzzzzzzzzzzzz">
</stripe-pricing-table>

Obviously we should not commit those static values of the pricing-table-id and publishable-key, so let's utilize the .env variable we created earlier.

STRIPE_PUBLISHABLE_KEY=pk_test_90Jbzzzzzzzzzzzzzz
STRIPE_PRICING_TABLE_ID=prctbl_1LZ8xoxoxoxox


<body>
  <h1>We offer plans that help any business!</h1>
  <!-- Paste your embed code script here. -->
  <script
    async
    src="https://js.stripe.com/v3/pricing-table.js">
  </script>
  <stripe-pricing-table
    pricing-table-id="<%= ENV['STRIPE_PRICING_TABLE_ID'] %>"
    publishable-key="<%= ENV['STRIPE_PUBLISHABLE_KEY'] %>"
  >
  </stripe-pricing-table>
</body>

Add webhooks for listening to the checkout

Before we start writing the code for the webhook, make sure we have installed the Stripe CLI so that we can receive the webhook locally. You can do that easily by following the steps written on this URL. Once you have finished the setup, let's continue.

To be ready for the webhook, lets add a new endpoint to the routes.rb.

config/routes.rb

    resources :stripe_events, only: :create

Then of course create new controller for working as the webhook.

class StripeEventsController < ActionController::Base
    # The skip below here is very important
    skip_before_filter :verify_authenticity_token

  def create
        payload = request.body.read
      event = nil

      begin
        event = Stripe::Event.construct_from(
          JSON.parse(payload, symbolize_names: true)
        )
            stripe_id = event.data[:object][:id]
      rescue JSON::ParserError => e
        # Invalid payload
        render status: 400
        return
      end

      # Handle the event
      case event.type
        when 'checkout.session.completed'
      else
        raise "Unhandled event type: #{event.type}"
      end

      render status: 200
  end
end

For this skeleton endpoint, there are a number of important things:

The StripeEventsController inherit from ActionController::Base instead of ApplicationController, this is to ensure that the controller will not share any existing callbacks listed on ApplicationController. Otherwise youd have to skip those unrelated, redundant callbacks manually on this controller.
The second is the skip_before_filter :verify_authenticity_token line which is important, because otherwise youd get 422 Unprocessable Entity as response when the webhook endpoint gets hit by Stripe.

when 'checkout.session.completed'
  stripe_checkout = event.data.object
  stripe_customer_id = stripe_checkout.customer
  stripe_subscription_id = stripe_checkout.subscription

  account = Account.where(
    stripe_customer_id: stripe_customer_id,
    email: stripe_checkout.customer_details.email
  ).first_or_create

  stripe_subscription = Stripe::Subscription.retrieve(stripe_subscription_id)

  # make a loop of this if you expect that the subscription can contain many plans
  stripe_price = stripe_subscription.items.data[0].price
  plan = Plan.find_by(stripe_price_id: stripe_price.id)

  subscription = Subscription.where(stripe_subscription_id: stripe_subscription_id).first_or_initialize
  subscription.assign_attributes(
    plan_id: plan.id,
    account_id: account.id,
    status: stripe_subscription.status
  )
  subscription.save

Since were using Stripe Checkout, we can actually achieve what weve done above with only one event, which is the checkout.session.completed event, instead of two. The reason youd want to use the checkout.session.completed event instead would be the simplicity of having to listen to and handle only one event instead of more.

Handle redirect after checkout

So the idea is to redirect the customer to the registration form so they can set the password to their new account, once the payment is complete. Since were using Devise on this tutorial, then well use Devises existing form page and controller to do that. If youre using different authentication engine, then you will need to adjust accordingly.

That is to generate the form page, if you havent already done so. For the controller, this is the command:

rails generate devise:controllers users

config/routes.rb

Rails.application.routes.draw do
    ###
    devise_for :users, controllers: {
    registrations: 'users/registrations'
  }
  get '/checkouts/:id/complete', to: redirect('/users/sign_up?stripe_checkout_id=%{id}')
    ###
end

Notice that we specify the registrations of devise to be handled by the controller we generated from the rails generate devise:controllers users

Go back to the pricing table on the Stripe dashboard and edit it. Once you click continue, you can see the field for the redirect url. Fill it to your local development and add the route `http://lvh.me:3000/checkouts/{CHECKOUT_SESSION_ID}/complete. For this tutorial, we are assuming that you will be usinglvh.me` for your local development.

Now that well be redirecting them to the Devise registration controller, we will need to adjust the registration controller just a little bit.

class Users::RegistrationsController < Devise::RegistrationsController
    before_action :memoize_checkout, only: :new
    after_action :relate_account, only: :create

    # Add these lines on the bottom of the controller
    private

    # This method is for prefilling the email field for the registration form
    # with the email the customer used to checkout on Stripe Checkout earlier
    def build_resource(hash = {})
    self.resource = resource_class.new_with_session(
      hash.merge(
        email: session['stripe_checkout']['customer_details']['email']
      ),
      session
    )
  end

    # This method is for storing the checkout session object to the session,
    # once the customer gets redirected from Stripe Checkout
    def memoize_checkout
    return unless params[:stripe_checkout_id]
    session[:stripe_checkout] ||= Stripe::Checkout::Session.retrieve params[:stripe_checkout_id].as_json
  end

    # This is for hooking up the newly created user account after registration is successful
    # with the account object, to link user and the paid subscription
    def relate_account
        # alternative 1: find account by email
        # account = Account.find_by_email signup_params[:email]
        # alternative 2: find account by retrieving Stripe customer id from Stripe
        account = Account.find_by(stripe_customer_id: session['stripe_checkout']['customer'])
        # Associate the matching Stripe customer object, our Account object, and the newly-registered User object.
        account.account_users << resource
        account.save
    end
end

Before we start testing, we might want to redirect the user to different page after registration is successful, we can do this by:

# config/routes.rb
Rails.application.routes.draw do
    authenticate :user do
    get "dashboard" => "home#dashboard", as: :user_root
  end
end

# app/controllers/home_controller.rb
class HomeController < ApplicationController
    # add this endpoint
    def dashboard
  end
end

# app/views/home/dashboard.html.erb
<h2>Dashboard</h2>
<p>Thank you for registering!</p>

One last thing

By this point, you should have a complete end-to-end integration from the checkout and the registration, and the eventual authentication. Your customer would have their payments related correctly to their user record via the account object. So the last step below is entirely optional since it is a nice-to-have.

To make it easier to reference the current_user s account and subscription, we will create some helpers. First, we go back to the registration controller.

    def relate_account
        account = Account.find_by(stripe_customer_id: session[:stripe_customer_id])
        account.account_users << resource
        # we just change the last line of the method
        if account.save
            session[:current_account_id] = account.id
            session.delete(:stripe_checkout)
        end
    end

The current_account_id session key will be used to determine the account of the currently logged in user. So this will be just as useful as current_user to scope resources/records. For example, only

class ApplicationController < ActionController::Base

    def current_account
        @current_account ||= current_user.accounts.find(session[:current_account_id])
    end
    helper_method :current_account

    def current_subscription
    @current_subscription ||= current_account.current_subscription
  end
    helper_method :current_subscription
end

With this, for example if you want to check if the current_user is allowed to access certain pages/dashboards based on their current subscription:

class ArticlesController < ApplicationController
    before_action :allow_only_premium_users, only: :restricted_page

    def index
    end

    def restricted_page
  end

    private

    def allow_only_premium_users
        redirect_to '/404' if current_subscription.product_tier != 'premium'
    end
end

Now, in most apps, there are two places where we authenticate the user, the first is the registration and the other one is the login (session create). So we then need to ensure the current_account_id session key gets set everytime customer re-login back to the app.

class Users::SessionsController < Devise::RegistrationsController
    after_action :set_current_account_id, only: [:create]

    private

    def set_current_account_id
        # find the last used account by relying on the timestamp
        # this way, for example if you implement multi-accounts for a given user,
        # you just need to update the timestamp everytime a user swtiches the account
        session[:current_account_id] = current_user.user_accounts.order(updated_at: :desc).first.id
    end
end

If you want to be super clean, you can refactor this a bit better:

class UserAccount < ApplicationRecord
    ###
    scope :order_by_recently_accessed, -> { order(updated_at: :desc) }
    ###
end


class User
    has_one :recent_account_user, -> { order_by_recently_accessed}, class_name: 'AccountUser'
end

then the previous after_action becomes like this:

    def set_current_account_id
        # find the last used account by relying on the timestamp
        # this way, for example if you implement multi-accounts for a given user,
        # you just need to update the timestamp everytime a user swtiches the account
        session[:current_account_id] = current_user.recent_account_user.id
    end

Thats it, now weve got a super integrated registration flow with checkout easily enabled using Stripe Pricing Table + Stripe Checkout combo! Remember that you can always access the code here.

End-to-end Paypal Checkout with Rails (Part 3 of 3)

Galih — Sun, 06 Dec 2020 19:01:07 +0000

If you want to code along with this post, you can do so by checking out from this commit on indiesell repo.

Getting the order creation to Paypal dynamic

First of all, we want to ask payment from our customers according to the product they choose to buy, right? So that is our first objective, and where we'll capitalize on our hardwork of turning the paypal button into Vue component.

We can easily pass the attributes from our products, that were created on the backend, to the front-end, which is our Paypal button:

From:

app/views/store/products/_product.html.erb

    <!-- TODO: Put the paypal button here -->
    <div class="buynow">
      <paypal-button
        refer="paypal-container-<%= product.id.to_s %>"
      />
    </div>

To:

app/views/store/products/_product.html.erb

    <!-- TODO: Put the paypal button here -->
    <div class="buynow">
      <paypal-button
        refer="paypal-container-<%= product.id.to_s %>"
        currency-code="<%= product.price_currency %>"
        price-str="<%= product.price.to_s %>"
        product-description="<%= product.name %>"
        product-id="<%= product.id %>"
      />
    </div>

Here we have added the currency, price, product description, and also the id of the product, so that it can be used in the component.

app/javascript/components/paypal_button.vue

export default {
  props: {
    refer: {
      type: String,
      required: true
    },
    // Pass the product attributes to be used here
    currencyCode: {
      type: String,
      required: false,
      default() {
        return 'USD'
      }
    },
    priceStr: {
      type: String, // should be like "100.00"
      required: true
    },
    productDescription: {
      type: String,
      required: true
    },
    productId: {
      type: String,
      required: true
    }
  },

// REDACTED

The data that we pass from the rails template as props, will override our default Paypal order payload to trigger checkout process using the smart payment buttons:

app/javascript/components/paypal_button.vue

// REDACTED
  mounted: function() {
    // These 3 lines are what we add here
    this.order.description          = this.productDescription;
    this.order.amount.currency_code = this.currencyCode;
    this.order.amount.value         = Number(this.priceStr);

    // IMPORTANT: to cause the paypal button be loeaded and rendered
    this.setLoaded();
  },

// REDACTED

Now if you refresh, when you click one of the payment buttons, you will see that the amount we charge our customers is dynamic, as per set for the product selected.

So by this point, we are able to ask payment from our customers correctly, but the any successful, valid payment, will still not trigger anything on our app. So let's change that on!

Setup the Paypal Capture endpoint to capture payment

First, because we want to also store the successful payments that our customers made on Paypal from the smart buttons, we need to record it as "Purchase" on our DB. And we can achieve just that by creating an endpoint to do just that, and hook it to the "onApprove" callback from the smart button.

So the implementation is up to you, but for indiesell, I implemented something like this:

app/controllers/api/v1/store/paypal_purchases_controller.rb

# frozen_string_literal: true

module API
  module V1
    module Store
      class PaypalPurchasesController < ApplicationController
        # We'll remove this line below, i promise to you
        skip_before_action :verify_authenticity_token

        def create
          # TODO: this is where we put the magic
        end
      end
    end
  end
end

app/controllers/api/v1/store/paypal_purchases_controller.rb

def create
  # TODO
  purchase                      = Purchase.new
  purchase.gateway_id           = 1
  purchase.gateway_customer_id  = params[:customer_id]
  purchase.customer_email       = params[:customer_email]
  purchase.product_id           = params[:product_id]
  purchase.token                = params[:token]
  purchase.is_paid              = params[:is_successful]

  # Because price_cents is string of "20.00", we need to 
  # parse the string to money. To do that we need to build the compatible money string,
  # should be like "USD 20.00"
  money_string = "#{params[:price_currency]} #{params[:price_cents]}"
  parsed_money = Monetize.parse money_string

  purchase.price_cents          = parsed_money.fractional # 2000
  purchase.price_currency       = parsed_money.currency.iso_code # USD

  if purchase.save
    render status: :ok, json: { purchase_code: purchase.id }
  else
    render status: :unprocessable_entity, json: {}
  end
end

So on the endpoint, we should be prepping the purchase record based on the payload that we receive from the "onApprove" callback on the paypal_button.vue.

After prepping, we then try to save it. If it is successful, then we declare status 200, if not then 422, as the json response.

Now that the endpoint is ready, let's hook it to the vue component to have an end to end process setup.

app/javascript/components/paypal_button.vue


methods: {
  setLoaded: function() {
    paypal
      .Buttons({

        // REDACTED

        onApprove: async (data, actions) => {
          const order = await actions.order.capture();
          // for complete reference of order object: https://developer.paypal.com/docs/api/orders/v2

          const response = await fetch('/api/v1/store/paypal_purchases', {
            method:   'POST',
            headers:  {
              "Content-Type": "application/json"
            },
            body:     JSON.stringify(
              {
                price_cents:    this.priceStr,
                price_currency: this.currencyCode,
                product_id:     this.productId,
                token:          order.orderID,
                customer_id:    order.payer.payer_id,
                customer_email: order.payer.email_address,
                is_successful:  order.status === 'COMPLETED'
              }
            )
          });

          const responseData = await response.json();
          if (response.status == 200) {
            window.location.href = '/store/purchases/' + responseData.purchase_code + '/success';
          } else {
            window.location.href = '/store/purchases/failure?purchase_code=' + responseData.purchase_code;
          }
        },
        onError: err => {
          console.log(err);
        }
      }).render(this.selectorContainer);
  }
}

I know it seems a lot, and I do apologize if this step is a bit overwhelming. But don't worry, we'll discuss it one by one.

The receiving of the callback from paypal

onApprove: async (data, actions) => {
  const order = await actions.order.capture();

So the order constant is basically the "capture" result, meaning that when the customer that checks out using our Smart Payment buttons, Paypal knows to where the successful payment callback should be posted to, we just need to capture it and store it.

The acknowledgment of successful payment for our app

Now that Paypal knows our customer has successfully paid the bill, then we need to acknowledge it also, hence this action of sending POST request to the endpoint we created earlier

// REDACTED
          const response = await fetch('/api/v1/store/paypal_purchases', {
            method:   'POST',
            headers:  {
              "Content-Type": "application/json"
            },
            body:     JSON.stringify(
              {
                price_cents:    this.priceStr,
                price_currency: this.currencyCode,
                product_id:     this.productId,
                token:          order.orderID,
                customer_id:    order.payer.payer_id,
                customer_email: order.payer.email_address,
                is_successful:  order.status === 'COMPLETED'
              }
            )
          });

Take a good look on the JSON object with the :body key, that is essentially the payload that we will be processing on the endpoint that we made. So you can just customize, add, or remove any data as you see fit.

Notify/Redirect user

          const responseData = await response.json();
          if (response.status == 200) {
            window.location.href = '/store/purchases/' + responseData.purchase_code + '/success';
          } else {
            window.location.href = '/store/purchases/failure?purchase_code=' + responseData.purchase_code;
          }

So again, this is entirely up to you, where or how you want to notify your customers that the payment, aside from being completed in Paypal, have also been acknowledged by your database.

In the case of Indiesell, I redirect the customers to success page if successful and failure page if there is something wrong on the endpoint. The successful and failure page have been made beforehand, so I will not cover that on this post.

Finishing: Enabling submitting CSRF token

So last but not least, remember about the promise I made to you on this post earlier?
app/controllers/api/v1/store/paypal_purchases_controller.rb

# redacted
      class PaypalPurchasesController < ApplicationController
        skip_before_action  :verify_authenticity_token

        def create
          # redacted

Yes, that bit. That bit actually is unsafe for production, since it bypasses one of security features from Rails. I skipped that bit just to keep things simpler to complete our checkout development, but now we're done, let's get on it then.

First, remove that unsafe line.

app/controllers/api/v1/store/paypal_purchases_controller.rb

# redacted
      class PaypalPurchasesController < ApplicationController
        def create
          # redacted

Now with this, our checkout system will fail once more during the capture callback. What we need to do is to submit CSRF token created by rails for POST request that we send to our endpoint

So first we create a mixin to specifically fetch the CSRF token from the HTML:
app/javascript/mixins/csrf_helper.js

var CsrfHelper = {
    methods:{

      findCsrfToken() {
        let csrf_token_dom = document.querySelector('meta[name="csrf-token"]');
        let csrf_token = "";

        if (csrf_token_dom) {
          csrf_token = csrf_token_dom.content;
        }

        return csrf_token;
      }

    }
};
export default CsrfHelper;

Then, we must not forget to import that mixin and declare it in our paypal_button.vue component

app/javascript/components/paypal_button.vue

<template>
  <div :id="refer"></div>
</template>

<script>
// MIXINS
// For grabbing the CSRF token to be used to submit to internal API endpoint
import CsrfHelper from '../mixins/csrf_helper.js';
export default {
  mixins:[CsrfHelper],

Once done, use it by calling it before we send the POST request:

app/javascript/components/paypal_button.vue

// REDACTED
const response = await fetch('/api/v1/store/paypal_purchases', {
  method:   'POST',
  headers:  {
    "Content-Type": "application/json",
    "X-CSRF-Token": this.findCsrfToken()  // taken from the mixins/csrf_helper.js
  },
  body:     JSON.stringify(

// REDACTED

And we're done. If you have been coding along, please refresh the page and try to complete a purchase.

Or if you want to check the source code for this series of posts, you can do so by checking out this branch on the indiesell repo.

Happy coding, cheers!

Use Paypal Smart Button as Vue Component (Part 2 of 3)

Galih — Mon, 30 Nov 2020 14:22:08 +0000

Right, from the previous post we have managed to add the Paypal Smart payment buttons easily. But the buttons, though are functioning, not really do much in terms of providing a way for our customers to pay for our goods or services.

In this post, we will start the journey of making the buttons dynamic, ie. able to charge our customers correctly based on the price of the goods, and once they completed the payment, allow them to access the download link.

If you want to code along with this post, you can do so by checking out from this commit on indiesell repo.

Convert the button to vue component

Remove the second script that we added on the layout template so that it will only look like this:

app/views/layouts/store/application.html.erb

  </footer>

    <script
    id="paypal-sdk-js-src"
    src="https://www.paypal.com/sdk/js?client-id=<%= ENV['paypal_client_id'] %>"
    data-sdk-integration-source="button-factory"
  >
  </script>
</body>

Only keep the script source from Paypal, then we add the button container, by replacing the previous button html on the product item partial html

app/views/store/products/_product.html.erb

    <!-- TODO: Put the paypal button here -->
    <div class="buynow">
      <paypal-button
        refer="paypal-container-<%= product.id.to_s %>"
      />
    </div>

Now if you refresh, the button will disappear, our mission for the next 5 mins is to make that re-appear

Intro for the vue pack: store.js

This is the current state of the Vue pack that will be responsible to hold all the javascripts needed to easily get our Paypal button to be dynamic.

app/javascript/packs/store.js

import Vue from 'vue/dist/vue.esm'

document.addEventListener('DOMContentLoaded', () => {
  var storeList = new Vue({
    el: '', // Add the css identifier for the div that will load the paypal button
    components: {
      // Add paypal button here
    }
  })
})

This Vue pack is already hooked to our rails HTML template on the layout template for the storefront.

app/views/layouts/store/application.html.erb:13

%= javascript_pack_tag     'store' %>

Add Paypal JS SDK using yarn

yarn add @paypal/sdk-client

then add to the store.js

import Vue from 'vue/dist/vue.esm'
// add this single line below here to use the Paypal JS SDK we've just installed
import { unpackSDKMeta } from '@paypal/sdk-client';

document.addEventListener('DOMContentLoaded', () => {

Create the vue component: paypal button

app/javascript/components/paypal_button.vue

<template>
  <div :id="refer"></div>
</template>

<script>
export default {
  props: {
    // This is for giving dynamic container css 'ID' to be called by the Paypal button renderer
    refer: {
      type: String,
      required: true
    }    
  },
  data() {
  },
  mounted: function() {
  },
  computed: {
  },
  methods: {
  }
}
</script>

The last thing for this step is to hook this newly-created component onto the store.js pack

import Vue from 'vue/dist/vue.esm'
import { unpackSDKMeta } from '@paypal/sdk-client';
// Add this single line below
import PaypalButton from '../components/paypal_button.vue';

document.addEventListener('DOMContentLoaded', () => {
  var storeList = new Vue({
    el: '#store-list', // Add the css identifier for the div that will load the paypal button
    components: {
      // Add paypal button here
      PaypalButton
    }
  })
})

Notice that we have just added this line:

    components: {
      PaypalButton
    }

and this line:

el: '#store-list',

but for this line to be working, we need to remember to add this element identifier onto the rails template also

app/views/store/products/index.html.erb

<div
  class="section section-demo"
  id="store-list" <!-- This is what we add -->
>
  <div class="container">
    <%= render partial: "product", collection: @products, as: :product %>
  </div>
</div>

Two lines, that's it. We will not be adding anymore to this file.

So the setup is ready, all we need to do now is to get the component working.

Setup the vue component: paypal button

app/javascript/components/paypal_button.vue


// REDACTED //

<script>
export default {
  props: {
    refer: {
      type: String,
      required: true
    },
    // This is where will pass more product attributes to be used in this vue component from the rails html template
  },
  data() {
    return {
      // Defaults template for the order JSON payload that will be sent to Paypal
      order: {
        description: "",
        amount: {
          currency_code: "",
          value: 0
        }
      }
    };
  },
  mounted: function() {
    // IMPORTANT: this method causes the paypal button be loeaded and rendered
    this.setLoaded();
  },
  computed: {
    selectorContainer() {
      return '#' + this.refer;
    }
  },
  methods: {
    setLoaded: function() {
      paypal
        .Buttons({
          createOrder: (data, actions) => {
            return actions.order.create({
              purchase_units: [
                this.order
              ]
            });
          },
          onApprove: async (data, actions) => {
          },
          onError: err => {
            console.log(err);
          }
        }).render(this.selectorContainer);
    }
  }
};
</script>

Up to this point, your component should be ok enough that it will re-appear and working once more.

On the next part, we will have the maximum fun by turning all of these hard work into fruition by tying the checkout button to our backend.

PS: reminder, if you don't want to wait, you can view the full post at this blog post

Integrate Paypal Smart Payment Buttons in Rails (Part 1)

Galih — Mon, 30 Nov 2020 14:21:28 +0000

This will be the first of a three-posts-series related to how you can easily develop an end-to-end checkout using Paypal Smart Payment buttons in a Vue-powered Rails app. If you want to view the full series in one page, you can view it on my blog instead.

So this is what we aim to have built at the end of this post using only the html template in Rails:

The second post and third post will feature how we convert this simple integration to complete the checkout system, helped by VueJS.

To make the learning process easier, you can code along using this branch that I made in the repo of Indiesell. You can just donwload the repo on that commit and use that to follow the tutorial series until the end.

Before we start, don't forget to run these commands for your convenience:

bundle install
yarn
rake db:create
rake db:migrate
rake db:seed

Get Paypal Client ID and Secret

First of all, you should get sandbox Client ID (the secret is not needed for this post, but you might need it) from Paypal beforehand, you can find the direction to do just that on their official documentation here

Then, create application.yml using figaro and use it to store the ID, so that you don't (accidentally) commit the id and secret to the public.

You can just run this command to create the yml file.

bundle exec figaro install

Then, update it:
config/application.yml

development:
  paypal_client_id: PUT_YOUR_PAYPAL_SANDBOX_CLIENT_ID_HERE
  paypal_client_sec: PUT_YOUR_PAYPAL_SANDBOX_CLIENT_SECRET_HERE

test:
  paypal_client_id: PUT_YOUR_PAYPAL_SANDBOX_CLIENT_ID_HERE
  paypal_client_sec: PUT_YOUR_PAYPAL_SANDBOX_CLIENT_SECRET_HERE

PRO TIP: Please remember to RESTART your server ^_^

Integrate Paypal smart button to html views

Add the paypal button container on the product item html partial, below the TODO: comment

app/views/store/products/_product.html.erb

    <!-- TODO: Put the paypal button here -->
    <div id="paypal-button-container"></div>

Add script source from PAYPAL to the bottom of the body element, after the footer

app/views/layouts/store/application.html.erb

  <script
    id="paypal-sdk-js-src"
    src="https://www.paypal.com/sdk/js?client-id=paypal_client_id"
    data-sdk-integration-source="button-factory"
  >
  </script>
  <script>
    paypal.Buttons({
      env: 'sandbox', // Valid values are sandbox and live.
      createOrder: function(data, actions) {
      // This function sets up the details of the transaction, including the amount and line item details.
      return actions.order.create({
        purchase_units: [{
          amount: {
            value: '0.01' // This will be replaced, I promise
          }
        }]
      });
    },
    onApprove: async (data) => {}
  }).render('#paypal-button-container');
  </script>

Reference the client id from application.yml app/views/layouts/store/application.html.erb

  <--- REDACTED --->
  <script
    id="paypal-sdk-js-src"
    src="https://www.paypal.com/sdk/js?client-id=<%= ENV['paypal_client_id'] %>"
    data-sdk-integration-source="button-factory"
  ></script>
  <--- REDACTED --->

After adding this, you can reload and you will see that the Paypal button shows up and working. You can even click it and start making (mock) payment!

If you click the Pay with paypal button:

And if you click the "Debit or Credit Card" button, it will render a form like this for free:

But we are not done, since completing this checkout will not trigger anything (even though it will be successful) and the checkout amount is mocked to USD0.01 at the moment. So the rest of the post, will be for getting our checkout working as normal, even production-ready.

Excited? We'll do that on the next part, don't worry.

PS: If you want to view the full series in one page, you can view it on my blog instead