DEV Community: Sahil Gadimbayli

Fetching ancestor/sibling nodes per locale in ActiveRecord

Sahil Gadimbayli — Thu, 23 Apr 2020 14:37:47 +0000

There are times when we need to work with parent records
within the database.

A practical example we will use today is Pages table with a
parent_id column, referencing another Page record.

This will be used in building page breadcrumbs(parents) and fetching
recommended pages(siblings).

pizza_recipe_page = Page.last

recipes_page = pizza_recipe_page.parent

This builds up hierarchical structure, and we would most
probably need a way to fetch the page's parent or related pages
when working in our application.

pages_for_breadcrumbs = page.self_and_parent_nodes
recommended_pages = page.siblings

For straightforward cases, we may opt out to use a gem such as
ancestry and be done
with the task. This will work, assuming you are using one column
such as parent_id to set a parent for the record.

class Page
  # assumes your parent record
  # reference is based only on a single column
  has_ancestry
end

However, you would run into a slight inconvenience
if you are using multiple columns to set parents,
such as for different languages(parent_id_en, parent_id_es, parent_id_it).

Pages	Field
parent_id_es	Parent page for spanish locale
parent_id_en	Parent page for english locale
parent_id_it	Parent page for italian locale

Now, imagine querying for related pages per locale,
and generating breadcrumbs... Not so straightforward!

Thus, we have to build functionality on our own to enable fetching ancestors/siblings per locale.

PostgreSQL recursive queries for the rescue!

This assumes you are using Active Record. However,
you may use the same concept regardless of the ORM.

class Page
  # Starting with the direct parent
  # iterate through the chain, querying
  # parent of the found record as we go higher
  # terminating when nothing is found.

  # Maintains the correct order of results.
  # So, ids will be self_id, parent_node_id,
  # greatparent_node_id, greatgreatparent_node_id...

  # @returns ActiveRecord::Relation
  def self_and_parent_nodes(locale:)
    query = <<~SQL
        WITH RECURSIVE parent_nodes AS (
          SELECT id, parent_id_#{locale}
          FROM #{self.class.table_name}
          WHERE id = #{id}
          UNION
          SELECT c.id, c.parent_id_#{locale}
          FROM #{self.class.table_name} c
            JOIN parent_nodes p ON p.parent_id_#{locale} = c.id
      ) SELECT id FROM parent_nodes;
    SQL

    self_and_parent_node_ids =
      ::ApplicationRecord.connection
                         .execute(query)
                         .map { |record| record.fetch('id') }

    # Fetch AR objects from returned collection of ids

    self.class
        .where(id: self_and_parent_node_ids)
        .ordered_by_ids(self_and_parent_node_ids)
  end

  # Find records that share parent with self
  #
  # @returns ActiveRecord::Relation

  def siblings(locale:)
    self.class
        .where("parent_id_#{locale} = ? AND #{self.class.table_name}.id != ?",
               public_send("parent_id_#{locale}"), id)
  end

  # Maintain order of results
  #
  # @returns ActiveRecord::Relation

  def ordered_by_ids(ids)
    order_results_query =
      ids.map.with_index { |id, position| "WHEN #{id} THEN #{position}" }.join(' ')

    order_sql =
      Arel.sql(
        "CASE #{self.table_name}.id #{order_results_query} ELSE #{ids.size} END, #{self.table_name}.id"
      )

    order(order_sql)
  end
end

Now you are able to fetch ancestor and sibling nodes per locale. Go ahead and create/test some records.

page = Page.create!(
  name: 'Test Page',
  parent_id_en: id_of_some_english_page,
  parent_id_es: id_of_some_spanish_page
)

pages_for_breadcrumbs =
  page.self_and_parent_nodes(locale: 'en')

recommended_pages_for_spain =
  page.siblings(locale: 'es')

This post has also been published at ramblingcode

That's it for today! If you see an issue or have any question
please give me a shout.

Processing SEPA transactions with Ruby

Sahil Gadimbayli — Thu, 16 Apr 2020 17:27:39 +0000

SEPA,
also known as Single European Payment Area, is a system of tools/technologies/regulations,
facilitating simplifacation of bank transfers denominated in EUR.

There are quite a few steps involved in processing
SEPA transactions from start to end. These include:

management of EBICS users
generation of XML documents with different formats; depending on your needs
communicating XML files to financial institutions
reading and parsing bank statements to perform updates in your system (if needed)

Building all of the above from scratch would take a considerable
amount of time. I have encountered a few awesome tools in ruby, that
may help us with just that, and save some of your precious time.

We will go through tools that we can use for each step, also
noting some gotchas along the way.

Management of EBICS users

First of all, to submit transactions and fetch bank statements,
we would need to have the EBICS users contracted, configured and confirmed.

This post assumes that you have done a contractual part
with your bank/financial institution and have details
at hand for the user. If not, please contact your
financial institution to provide you with an EBICS user.

EBICS users can be an SRZ type or not. The difference is that the prior
may process transactions on behalf of other customers of
the bank, given the authorization. Normal users (NON-SRZ) may only
process transactions for bank account it is associated with.

So, before asking for an EBICS user, it's good to know what your
needs are for its utilization.

Configuring and confirming an EBICS user involves 3 steps.

Create an EBICS user with data given by your financial institution; saving keys, and other user-related data.
Print INI file received after creation, and physically sign/send to bank
Confirm the user when you receive ok from your bank

Folks at Rails Love built an awesome
gem Epics, to do just that.

Let's get started.

Setting up the user initially...

# initialize user

user =
  Epics::Client.setup(
    "some-secure-password",
    "https://ebics.sandbox",
    "EBICS_HOST_ID",
    "EBICS_USER_ID",
    "EBICS_PARTNER_ID"
  )

# save keys (important!)

user.save_keys('path/to/my.key')

# submit keys to the bank
# sends the signature key
user.INI

# sends the encryption and authentication keys
user.HIA

# save ini file (you will need to print, sign and send this to the bank)

user.save_ini_letter( 'Bank Name', "/path/to/ini.html" )

Once your financial institution confirms that you may proceed
with the user, you will have to confirm the user again(HBP order type)
and download public keys of the bank.

Epics gem becomes handy in this part as well.

# initialize the client you have set up above

user = Epics::Client.new(
  File.open('/path/to/my.key'),
  'some-secure-password',
  'https://ebics.sandbox',
  'EBICS_HOST_ID',
  'EBICS_USER_ID',
  'EBICS_PARTNER_ID'
  )

# confirm and fetch the bank's public keys

user.HBP

# save new keys
# (you will use these keys from now on to initialize client)

user.save_keys('path/to/my.key')

# Try something to see if user works.

# this will return available order
# types for the user. However, from experience
# this list is not always correct :)
user.HAA

Now, we have got the EBICS user creation out of the
way, let's go through how we can build an XML from
list of transactions with correct format.

Generating XML documents consisting your transactions

Folks at SalesKing, wrote
an awesome gem Sepa King
to build XML files for direct debit and credit transactions
with legacy and latest schema format.

Note that, Sepa King will suit your needs only if you are
using a NON-SRZ user.

It is due to differences between container and non-container
XML schema, which Sepa King currently does not support.

Explaining container/non-container format is not part of this
blog post, thus we skip to usage of Sepa King.

Identify if you would like to create a debit or credit
transaction, and create an XML out of it by providing data
to Sepa King.

Have a look at their Readme,
as it is pretty straight forward.

Once you have generated the XML, we need to communicate this to
the bank.

Communicating XML to financial institutions

Now we have the XML file from Sepa King, we can use the
Epics gem again to initialize our EBICS user, and communicate
this XML to the bank.

# Generate the XML file
xml_file =
  'your_xml_output_received_from_sepa_king...'

# initialize EBICS user

user =
  Epics::Client.new(
    File.open('/path/to/my.key'),
    'some-secure-password',
    'https://ebics.sandbox',
    'EBICS_HOST_ID',
    'EBICS_USER_ID',
    'EBICS_PARTNER_ID'
  )

# Submit to bank

puts user.CDD(xml_file) # for direct debit
puts user.CCT(xml_file) # for direct transfer

Response is transaction_id and order_id of the order.
Mind that the order type(CDD|CCT) is important to distinguish
as they are used for seperate things( direct debit vs direct transfers).

It may happen that some order types are not available for
your user, and you may get errors due to not being authorized
to perform such orders. In that case, contact your financial
institution to enable the required order types.

Fetching and parsing bank statements to perform updates

It may be that, you need to follow up with the transactions
you have submitted previously by reading bank statements, parsing
it to perform updates in your system or to send out notifications.
You decide...

Parsing XML documents and digging for data using Xpath is always
painful. To ease their pain for common good, folks at Barzahlen
wrote another awesome gem Camt Parser,
which parses bank statements and supports several formats(C52, 53, 54).

Let's get to use it.

# Initialize EBICS user

user =
  Epics::Client.new(
    File.open('/path/to/my.key'),
    'some-secure-password',
    'https://ebics.sandbox',
    'EBICS_HOST_ID',
    'EBICS_USER_ID',
    'EBICS_PARTNER_ID'
  )

# Request C53 bank statement

collection_of_statements =
  user.C53(from_date = "2020-04-14", to_date = "2020-04-15")

# Pass it to camt_parser

parsed_document =
  CamtParser::String.parse(collection_of_statements[0])

# You may also parse files
parsed_document =
  CamtParser::File.parse '/path/to/your_statement.xml'

# Now, you can iterate over statements

parsed_document.statements.each do |statement|
  puts statement.account.iban
  statement.entries.each do |entry|
    # Access individual entries/bank transfers
    puts entry.amount
    puts entry.debitor.iban
  end
end

See other accessors and how an example C53 statement looks
like here.

That's all for now.

We have gone through:

creation and management of EBICS users
creation of Direct Debit and Transfer XML files
communication with banks
parsing and using bank statements

There are lots of small stuff that we did not cover in this
post such as batch booking, differences in statements, container
and non-container XML formats. However, this should get you started.

Wish you chargeback-free statements :), and let me
know if you have any suggestions for the post or questions about SEPA.

This post has also been published @ramblingcode.

Get last returned value in the console

Sahil Gadimbayli — Fri, 21 Jun 2019 13:39:42 +0000

It happens that when we are working on the console we need to make use of last return value. It is also annoying when we forget to assign it to a variable to make use of it and have to type/copy it all over again.

When this happens, we can conveniently use underscore _ to retrieve the last return value.

$ irb

['I am last returned array']
_
['I am last returned array']

Convert Base64 images to PDF with Prawn in Ruby

Sahil Gadimbayli — Mon, 20 May 2019 12:57:14 +0000

A couple of weeks ago, I was working on a feature in Camaloon, which I ended up not using. However, I thought it would still be fun to share.

So, let's get to the issue without the boilerplate as it's late at night and I have to rise early :).

Problem:
I was trying to implement a third-party API and hopefully get a Base64 encoded PDF file of different documents I was trying to fetch. Unfortunately, the third party API didn't provide the PDF files but instead, images.

So, I thought alright I will just use Prawn and shove all images inside the Prawn pdf document and be done with it. Buttttt, images were in GIF format which I had to convert them to Prawn supported formats such as PNG/JPEG.

So, seeing these issues, Base64ImageToPdfConverter was born. I wanted to keep the class generic in the case in the future we need to reuse it.

Solution: Accept an array of Base64 encoded images, check if they need a conversion to a PNG(Prawn supported format).Convert them to PNG in case they need conversion and include it in the prawn document, if not, include them directly.

Let's create our class.

require 'prawn'
require 'tempfile'
require 'base64'

class Base64ImageToPdfConverter
  PRAWN_SUPPORTED_MIME_TYPES= ['image/png', 'image/jpeg'].freeze

  def initialize(base64_images:)
    @base64_images = base64_images
  end

  def run!
    return if base64_images.empty?

    open_tempfile(['combined-pdf', '.pdf']) do |pdf_file|
      base64_images.each do |base64_image|
        with_image_path(base64_image) do |image_file|
          # Process and Insert image to Prawn document
        end
      end
      # Render content to Prawn Document
      # Encode PDF in base64
    end
  end

  private
  attr_reader :base64_images

  def open_tempfile(*args)
    f = Tempfile.new(*args)
     yield(f)
    ensure
     f.close!
  end

  def with_image_path(base64_image)
    open_tempfile('image') do |f|
      f.write Base64.decode64(base64_image)
      f.flush
      yield f
    end
  end
end

So what the hell was that? Here, we are creating one temp file for final PDF that we will combine all images in and a temp file for each image, decoding Base64 and writing to it as we will need it to work with ImageMagick conversion soon.

Let's add our conversion logic.

require 'prawn'
require 'tempfile'
require 'base64'
class Base64ImageToPdfConverter
  PRAWN_SUPPORTED_MIME_TYPES= ['image/png', 'image/jpeg'].freeze

  def initialize(base64_images:)
    @base64_images = base64_images
  end

  def base64_pdf
    return if base64_images.empty?

    open_tempfile(['combined-pdf', '.pdf']) do |pdf_file|
      base64_images.each do |base64_image|
        with_image_path(base64_image) do |image_file|
          insert_image(image(image_file.path))
        end
      end
      # render and return the Base64 encoded PDF.
      prawn_document.render_file pdf_file.path
      base64_encoded(pdf_file.path)
    end
  end

  private

  attr_reader :base64_images
  attr_accessor :prawn_document

  def insert_image(string_io)
    prawn_document.start_new_page
    prawn_document.image string_io, position: :center
  end

  def image(file_path)
    if need_conversion?(file_path)
      # We are using imagemagick convert command to convert to png
      system "convert #{file_path} #{file_path}.png"

      # As we do not need a Tempfile but an object that responds to
      # read and write to use in Prawn document. We can use StringIO and
      # unlink tempfile.

      string_io= StringIO.new(File.read("#{file_path}.png"))
      File.unlink("#{file_path}.png")
    else
      string_io= StringIO.new(File.read(file_path))
    end

    string_io
  end

  def need_conversion?(file_path)
    !PRAWN_SUPPORTED_MIME_TYPES.include?(mime_type(file_path))
  end

  def mime_type(file_path)
    `file #{file_path} --mime-type -b`.strip
  end

  def prawn_document
    @prawn_document ||= Prawn::Document.new margin: 0,
                                            skip_page_creation: true
  end

  def base64_encoded(file_path)
    Base64.encode64(File.read(file_path))
  end

  def open_tempfile(*args)
    f = Tempfile.new(*args)
    yield(f)
  ensure
    f.close!
  end

  def with_image_path(base64_image)
    open_tempfile('image') do |f|
      f.write Base64.decode64(base64_image)
      f.flush
      yield f
    end
  end
end

base64_gifs = ['R0lGODlhAQABAIAAAAUEBAAAACwAAAAAAQABAAACAkQBADs=']

converter = Base64ImageToPdfConverter.new(base64_images: base64_gifs)
puts converter.base64_pdf

So here we go, we check for each image's Mime Type, convert them using ImageMagick's convert command to PNG and include it in Prawn document. We return the Base64 encoded PDF.

You can write the result to a file if you would like to create one:

base64_gifs = ['R0lGODlhAQABAIAAAAUEBAAAACwAAAAAAQABAAACAkQBADs=']
File.open "your_pdf_document.pdf", "wb" do |file|
  file.write Base64.decode64(Base64ImageToPdfConverter.new(base64_images: base64_gifs)
                                                      .base64_pdf)
  file.close
end

Credits to Roger for the review of the PR and shaping the class to be a bit better than its first iteration :D

Ruby's retry method

Sahil Gadimbayli — Sun, 12 May 2019 15:26:34 +0000

Salud to Dev.to community and eager Ruby lovers! It has been a busy month @Camaloon and I will be writing several posts from challenges and solutions in the following weeks.

Today's topic would be Ruby's retry method which I was not familiar until now which came in handy recently.

retry is used when we want to re-execute a code block. This is useful especially when we are trying to send an HTTP request to third-party API and would like several retries in case there are any failures.

As I was not aware of this feature before, when I wanted to re-execute a code block, I would've gone with loops.

Let's look at an example with loops:

We are sending an HTTP request and would like to have 3 retries if it fails due to connection errors.

require 'http'

def send_request(params)
  with_connection_retry { HTTP.post("http://example.com/resource", params: 
    params) }
end

def with_connection_retry
  retries = 0

  loop do
    yield
  rescue HTTP::ConnectionError => e
    retries += 1

    raise e if retries >= 3
  end
end

This code looked ugly with loop. Let's have a look with retry.

def send_request(params)
  with_connection_retry { HTTP.post("http://example.com/resource", params: 
    params) }
end

def with_connection_retry
  retries ||= 0

  begin
    yield
  rescue HTTP::ConnectionError => e
    retries += 1
    raise e if retries >= 3
    retry
  end
end

So, here you are, using Ruby's native API instead of relying upon loops for re-executing code blocks.

Custom binstub for Rubocop to run only on changed files

Sahil Gadimbayli — Tue, 09 Apr 2019 20:53:27 +0000

Salud to Dev.to community! A couple of weeks back I have posted first time here in Dev.to, motivating myself to post articles about Ruby, Rails, Javascript and Development in general.

Today's short write up will be to run Rubocop only on changed files and creating binstubs.

When starting a project, setting up and following style guides is pretty easy. However, when we inherit a legacy codebase or try to apply them on already existing medium to large scale applications, we quickly get frustrated as there are already hundreds if not thousands of failures that we have to fix.

A better way to do this is to progressively run robocop as we continue with our development. Each time you touch part of your codebase, Rubocop will only analyze and lint the given changed files.

So, let's start.

First, we will have to create a custom binstub to run Rubocop instead of default rubocop command. Then we will be writing logic inside the binstub to find changed files using git and apply robocop on them.(Aneeta has a handy article about this here)

To create a binstub Bundler provides bundler --binstubs gemname but we will not need any of the boilerplate, so let's just create an empty file named rubocop in our bin/ directory.

Once, we have created the file, we will have to write logic to compare current changes to the master branch(or the branch of your choice) as below:

#!/usr/bin/env ruby
# frozen_string_literal: true

class RubocopRunner
  COMPARE_TO_BRANCH = 'master'
  def self.check_branch_validity
    current_branch = `git branch | grep \\* | cut -d ' ' -f2`.strip
    if current_branch.empty? || current_branch == COMPARE_TO_BRANCH
      puts "No git found, or you are still on #{COMPARE_TO_BRANCH} branch."
      exit 0
    end
  end

  def self.check_non_committed_changes
    system("git ls-files -m | xargs ls -1 2>/dev/null | grep '\.rb$' | xargs bundle exec rubocop #{ARGV.join(' ')}")
  end

  def self.check_committed_changes
    system("git diff-tree -r --no-commit-id --name-only head origin/master | xargs bundle exec rubocop #{ARGV.join(' ')}")
  end
end

RubocopRunner.check_branch_validity
RubocopRunner.check_non_committed_changes
RubocopRunner.check_committed_changes

Now we can run bundle exec bin/rubocop and see it only apply on changed files. And don't forget to add your .rubocop.yml configuration file to root directory of your project.

Hope it helps, and let me know if there is anything that does not work as expected!

See you soon!

Time to write

Sahil Gadimbayli — Thu, 28 Mar 2019 21:46:15 +0000

Salud to Dev.to community! I have been wandering around the platform for a while and it became a daily part of my routine to check out amazing posts from amazing people out here!

I have always been a fan of writing down stuff to learn, but the words have always found themselves on good old papers.

Today, I have decided to contribute with posts to Dev.to every two to three weeks with whatever I found interesting while working on my day to day work or side projects.

I will be writing mostly about Ruby, Rails, Javascript and Web Development.

See you around!