DEV Community: Faisal Shaikh

Run artisan command instantly in the background using Symfony process component

Faisal Shaikh — Mon, 23 May 2022 06:28:19 +0000

Sometimes we want to optimize an API response time but we can't optimize it because the API has operations that can't be moved to background jobs, we need to perform them in the foreground.

In this type of situation, the Symfony process component can help.

When the API is called, we will perform the most important steps like creating/updating main table records, rest of the steps can be moved to an artisan command which will be run instantly in the background using the Symfony process component.

Code:

function runBackgroundProcess($command, $data)
{
 $phpBinaryFinder = new PhpExecutableFinder();

 $phpBinaryPath = $phpBinaryFinder->find();

 $process = new Process ([$phpBinaryPath, base_path('artisan'), $command, $data]); // (['php', 'artisan', 'foo:bar', 'json data'])

 $process->setoptions(['create_new_console' => true]); //Run process in background 
 $process->start();
}

Just call the above function in your regular PHP code flow and we can achieve asynchronous kind of functionality with this approach.

Tip: Use a table to track process failure.

Vuejs dynamic components

Faisal Shaikh — Sat, 13 Mar 2021 13:21:19 +0000

Vue dynamic components enable users to switch between two or more components and even retain the state of data when switching back to the initial component.

Some use cases are as follows:

Show or hide nested components based on the application state.
Tabbed interface like bootstrap tab pill.
You can even achieve router functionality without using the Vue router (not recommended for large projects). For eg. on click of the home link, we can load the home component dynamically.

Declaring a dynamic component:

Vue provides a special element to place dynamic components called 'component'. You can pass props too.

Syntax:

<component v-bind:is=”currentComponent” :prop1='foo' :prop2='bar'></component>

Here currentComponent is the data property that returns the component name which needs to be mounted.

To maintain the state of the data object of components while switching between them we can use a special element called 'keep-alive'.

Syntax:

<keep-alive>
      <component v-bind:is="currentComponent" :prop1='foo' :prop2='bar'/>
</keep-alive>

Full example:

<template>
  <div id="app">
    <keep-alive>
      <component v-bind:is="currentComponent" />
    </keep-alive>
    <button v-on:click="mountOne">One</button>
    <button v-on:click="mountTwo">Two</button>
  </div>
</template>
<script>
import One from './components/One.vue'
import Two from './components/Two.vue'
import Three from './components/Three.vue'
export default {
  name: 'app',
  components: {
    One,
     Two
  },
  data (){
    return {
      currentComponent:'Three'
    }
  },
  methods: {
    mountOne(){
        this.currentComponent = 'One';
    },
    mountTwo(){
        this.currentComponent = 'Two';
    }
  }
}
</script>

Custom validation error message using Laravel blade component

Faisal Shaikh — Sat, 09 Jan 2021 06:07:15 +0000

There are multiple ways to display a custom error message for a failed validation in Laravel. We can even use the blade component to achieve this.

Just put this component below the input you want to validate. Pass the name of the input to the for attribute and your custom message to the custom-message attribute. Boom you are done!!

Usage:

<x-input-error for="foo_name" custom-message="Foo is required" />

Component:

@props(['for', 'customMessage'])
@error($for)
<p {{ $attributes->merge(['class' => 'text-sm text-red-600 mt-1']) }}>{{ $customMessage ?? $message  }}</p>
@enderror

Build an email tracking service using Laravel, AWS SES & SNS services.

Faisal Shaikh — Sun, 18 Oct 2020 05:41:00 +0000

AWS setup:

Go to the SNS dashboard and create a new topic of which we want to receive notifications. This topic will serve as a bridge between SNS and SES. For example, I have created an email-notifications topic.
Create a subscription for this topic where we will define our web app's public URL and process the notification data.
Select the topic we created. The protocol can be https. The endpoint will be our web app's public route. We want a notification in JSON format so no need to enable raw message delivery. As soon as we create the subscription SNS will send a notification for confirmation of the link. We have to visit a subscription URL for confirming the subscription after that we will start receiving the notification. The code for confirming the subscribe URL is done below in the Laravel setup. If we miss confirming the subscription we can request a reconfirm.
Set up the SES configuration set from the SES dashboard. Add a new configuration set, name it, and then edit it for defining the destination of email notifications like delivery, opened, etc. Select SNS from the add destination option, type the name, select the event types we want SES to send it to SNS, and the topic which we created in the first step. Now set up for SES and SNS is done.

Laravel setup:

To use the Amazon SES driver for sending mail we have to install the Amazon AWS SDK for PHP, by adding the "aws/aws-sdk-php": "~3.0" to your composer.json file's require section and running the composer update command.
Set the default option in your config/mail.php configuration file to SES and in config/services.php add key, secret, region and in options, array add configuration set name which we created in AWS setup.

'ses' => [
       'key' => env('AWS_ACCESS_KEY_ID'),
       'secret' => env('AWS_SECRET_ACCESS_KEY'),
       'region' => env('AWS_DEFAULT_REGION', 'ap-south-1'),
       'options' => [
           'ConfigurationSetName' => env('AWS_SES_CONFIG_SET', 'email-notifications'),
       ],
   ],

Whenever we send an email we will be getting a unique message-id through an X-SES-message-ID header which is added by AWS SES, we have to store this id in the database table for matching it with SNS notification messageId. So I have created an Amazon Controller and sendEmail method for sending the email and persisting the message-id and email details in the sent_emails table.
Add a public route that we define as an endpoint for our SNS subscription in web.php. We will receive notifications as a post request. I have created an emailNotifications method in Amazon controller for handling this post request.
Route::post(‘amazon-sns/email-notifications’, ‘AmazonController@emailNotifications’);
As the data comes as a post request we have to tell our verify csrf token middleware to exclude this route. So in VerifyCsrfToken.php add ‘/amazon-sns/email-notifications’ in the except array.
We receive 2 types of notifications, a subscription confirmation which is for confirming the URL endpoint when we create a subscription for the first time from the SNS dashboard and the other one is just normal notification with SNS events.
So in the emailNotifications method, we take all the JSON from the request, then check the value of the ‘Type’ key if it is ‘SubscriptionConfirmation’ then we get the value of ‘SubscribeURL’ key and then using file_get_contents() method we visit the URL and confirm the subscription.
If the ‘Type’ key is ‘Notification’ then we take the value of ‘Message’ key and decode it to an array, then check the value of ‘eventType’ key which can be ‘Bounce’, ‘Open’, ‘Delivery’, etc. Now we take the ‘Mail’ key whose value is an array and take the value of ‘messageId’ key from it and update the sent_emails table records according to the event by using this messageId and persisted messageId.

JSON data for Subscription confirmation example:

array (
 'Type' => 'SubscriptionConfirmation',
 'MessageId' => 'c9f90bc3-6b7a-4ab9-8156-5b27b17797f8',
 'Token' => '2336412f37fb687f5d51e6e2425f004aee4aa0c75a56575973dba0d4ca2fa9f2d12a4ee88495d74faebbdda39e034e91fc520e03aa05796e452ab5c12cc81a666e25fe99529b3975bb34a500d84d692dbcace3d64cc36273504da5b0464a954b7d48239c61198e2eb161f6cfcfa6304b566c685c407b9c65b7c07d7d2d11da32',
 'TopicArn' => 'arn:aws:sns:ap-south-1:499826649778:email-notifications',
'Message' => 'You have chosen to subscribe to the topic arn:aws:sns:ap-south-1:499826649778:email-notifications.
To confirm the subscription, visit the SubscribeURL included in this message.',
 'SubscribeURL' => 'https://sns.ap-south-1.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:ap-south-1:499826649778:email-notifications&Token=2336412f37fb687f5d51e6e2425f004aee4aa0c75a56575973dba0d4ca2fa9f2d12a4ee88495d74faebbdda39e034e91fc520e03aa05796e452ab5c12cc81a666e25fe99529b3975bb34a500d84d692dbcace3d64cc36273504da5b0464a954b7d48239c61198e2eb161f6cfcfa6304b566c685c407b9c65b7c07d7d2d11da32',
 'Timestamp' => '2020-07-21T11:26:03.334Z',
 'SignatureVersion' => '1',
 'Signature' => 'QVIAlzHwf4DkxvtwBpKp9SByF9Z9Yh88t46ksTUkYA3Vr3nqkaKgVDmIE6wjdLn/B3LlkDU27zl9R+O2Sn4X9CSw9TT7B221zFCciEm6lbrQpIAX+KJ81blvaanR45pQY1VH9ageUv+2fk8VJvz1VnXw1+0fNIGx7k67aQbI1OttagbRr9wUIay/bwGq8LlP+rTviU8qg+AWbJgfg0iLhyoYSw5VaVL4aFyu7v5tLACMpRfPRDfIMaejmCJQBRfxWLfSFmALgkWcTIcUcWPllrw1sFEqgP+PzXlFfSfEkCzkcmqdy8++XfzEjME9lHky3kcOqgcaWse8Fs4GTFEsiQ==',
 'SigningCertURL' => 'https://sns.ap-south-1.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem',
)

JSON data for Notification of delivery event example:

array (
 'Type' => 'Notification',
 'MessageId' => '81d5d58e-7838-5a2a-8fac-821ee35fc2e1',
 'TopicArn' => 'arn:aws:sns:ap-south-1:499826649778:email-notifications',
 'Subject' => 'Amazon SES Email Event Notification',
 'Message' => '{"eventType":"Delivery","mail":{"timestamp":"2020-07-21T07:28:36.625Z","source":"foo@gmail.com","sourceArn":"arn:aws:ses:ap-south-1:499826649778:identity/foo@gmail.com","sendingAccountId":"499826649778","messageId":"0109017370463711-e110c671-cfe4-48d2-9f16-a6ee18b6edad-000000","destination":["bar@gmail.com"],"headersTruncated":false,"headers":[{"name":"Message-ID","value":"<c48ec65d8e3ffcc42020b52f3c28ecb5@127.0.0.1>"},{"name":"Date","value":"Tue, 21 Jul 2020 07:28:36 +0000"},{"name":"Subject","value":"SES TEST"},{"name":"From","value":"Email_Tracker <foo@gmail.com>"},{"name":"To","value":"bar@gmail.com"},{"name":"MIME-Version","value":"1.0"},{"name":"Content-Type","value":"text/html; charset=utf-8"},{"name":"Content-Transfer-Encoding","value":"quoted-printable"}],"commonHeaders":{"from":["Email_Tracker <foo@gmail.com>"],"date":"Tue, 21 Jul 2020 07:28:36 +0000","to":["bar@gmail.com"],"messageId":"0109017370463711-e110c671-cfe4-48d2-9f16-a6ee18b6edad-000000","subject":"SES TEST"},"tags":{"ses:operation":["SendRawEmail"],"ses:configuration-set":["email-notifications"],"ses:source-ip":["103.88.223.122"],"ses:from-domain":["gmail.com"],"ses:caller-identity":["root"],"ses:outgoing-ip":["76.223.180.12"]}},"delivery":{"timestamp":"2020-07-21T07:28:39.465Z","processingTimeMillis":2840,"recipients":["bar@gmail.com"],"smtpResponse":"250 2.0.0 OK  1595316519 r188si12628852pfc.330 - gsmtp","reportingMTA":"c180-12.smtp-out.ap-south-1.amazonses.com"}}
',
 'Timestamp' => '2020-07-21T07:28:39.537Z',
 'SignatureVersion' => '1',
 'Signature' => 'I1OrPbUNXeFM681n6E9K0NlJovNUGTBOpk3OIylVAufwJQCeUF0/Syrn9RlUbDXcfe5z/HaI5T8SGB0Opnh53om7FU7++R5Nte2Zltp/MoC2DCVxGv+ycSlP4Zv4csftHLcrVu/XtXDGRA6JHystxeznY4iUnFY4IKz84d0l+eOAdTi/Z71qb3swMpzhGybLeNBhMUyBzv4FFlSbIYF2z5/gDCCPjDSIkvFE1nm4pl7Gk7YcU3Dyw8zlT05LbRMQRNR8jHzNhTzOXBLCA6Ej+mTZ+F2qWe+7dDRiSSOoVk2nO4NMo+9r0RmdslpRJzAaas/8e8i1uAwz9WskS/aX1A==',
 'SigningCertURL' => 'https://sns.ap-south-1.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem',
 'UnsubscribeURL' => 'https://sns.ap-south-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:ap-south-1:499826649778:email-notifications:c4f4e401-f0c6-4643-9fd4-8ba36fe34c8f',
)

In the above JSON, only ‘eventType’ changes according to the event that happens and the rest remains the same.

Link to the source code: aws-ses-email-app

Manipulating data object of the Vuejs component through JQuery events

Faisal Shaikh — Mon, 31 Aug 2020 04:21:53 +0000

JQuery uses its own event system. We cannot catch events emitted from jQuery with Vuejs, we have to catch them with jQuery and call Vuejs from the handler.

Here is the code:
In our root Vue component binding the event:

mounted() {
    $("body").on("custom-event", this.methodForSettingDataObject);
},
beforeDestroy() {
    $("body").off("custom-event", this.methodForSettingDataObject);
}

Trigger event

var event = jQuery.Event("custom-event");
event.foo = fooArray; 
event.bar = barArray;
$('body').trigger(event);

Conclusion:
It is considered a bad practice to use jQuery with Vuejs but it is not wrong as long as it solves a problem.

Recognizing text from an image using Ruby

Faisal Shaikh — Sat, 01 Feb 2020 07:45:10 +0000

We can recognize the character's from an image using rtesseract gem and tesseract-ocr. So tesseract-ocr is an open-source OCR(optical character recognition) engine and rtesseract gem is a ruby wrapper for using tesseract-ocr.

To make rtesseract gem work we need tesseract-ocr, leptonica libraries, headers, and a C++ compiler. The tesseract-ocr gem will provide all of this. If you're on a distribution that separates the libraries from headers, remember to install the -dev package. On Debian, you will need to install libleptonica-dev and libtesseract-dev.

Installation steps are as follows:
First, install tesseract-ocr by using the following command "gem install tesseract-ocr" and check the installation by using "tesseract --version".
Then install rtesseract gem that's it start using the powerful tesseract for recognizing text in an image.

For example:

  image = RTesseract.new('my_image.jpg', lang: 'eng') #lang is optional
  image.to_s # Getting the text value

Here RTesseract takes image URL which is required and lang is the type of language we want to scan and the default is eng - English, the "to_s" method returns the text and we can also convert the image into pdf by using "to_pdf" on the image object. It supports many more options and methods you can check it here.

The reason I used this gem because I wanted to scan a folder of images and check whether an image consists of "No image" text if yes then move the image to the "no_image" folder.
So below is the ruby code of how I achieved this.

require 'fileutils'
require 'rtesseract'

class ExtractNameFromImage

  def move_images_from_folder(path)
    count = 0
    Dir.glob("#{path}/*") do |file|
      image = RTesseract.new(file)
      text = image.to_s
      count += 1
      if text.include? "No Image" 
        FileUtils.mv(file, 'pathtomove') #/home/user/no_image_folder
        puts "Moved #{file} scanned #{count}"
      else
        puts "Ok #{file} scanned #{count}"
      end
    end
  end
end
puts "Scanning directory #{ARGV[0]}"
extracter = ExtractNameFromImage.new
if File.directory?(ARGV[0])
  extracter.move_images_from_folder(ARGV[0])
else
  puts "No such directory exist, try again"
end

Command to run this file "ruby image_name.rb home/user/pathtoimagefolder".

Conclusion:
Using this set up I can scan thousands of image without manually opening each of them one by one.

Streaming CSV files in Ruby On Rails

Faisal Shaikh — Wed, 01 Jan 2020 06:58:34 +0000

Here I will show you how to achieve the streaming of CSV files using Ruby On Rails and Postgres. We will use concerns for our model and controllers so that we can include them wherever needed. Model concern file generates the data for CSV and the controller concern file handles the file exporting on the client's end.

Model concern file generate_csv_data.rb contains this code as follows:

  module GenerateCsvData
   extend ActiveSupport::Concern
    class_methods do
     def stream_query_rows(sql_query, options="WITH CSV HEADER")
      conn = ActiveRecord::Base.connection.raw_connection
      conn.copy_data "COPY (#{sql_query}) TO STDOUT #{options};" do
      while row = conn.get_copy_data
       yield row
     end
    end
  end
 end

Here stream_query_rows method accepts a query and options and yields the data. COPY (#{sql_query}) TO STDOUT #{options}; is a Postgres command which copies the data return by the query and output it to a specific format (here the format is CSV).

Controller concern file stream_file.rb contains this code as follows:

  module StreamFile
   extend ActiveSupport::Concern
    def stream_file(filename, extension)
     response.headers["Content-Type"] = "application/octet-stream"
     response.headers["Content-Disposition"] = "attachment; filename=# 
     {filename}-#{Time.now.to_i}.#{extension}"
     yield response.stream
   ensure
    response.stream.close
   end
  end

Here stream_file method accepts a filename and extension of the file and then yields the response and sends the file as an attachment to the client.
We can then include our concerns wherever needed.

Example:
We want all the data from the product's table into a CSV file and send the file as an attachment to the client. We will include the model concern in the product model and controller concern in products controller respectively:

Product Model:

  class ProductMaster < ApplicationRecord
   include GenerateCsvData
  end

Product Controller:

  class ProductsController < ApplicationController
   include StreamFile
   def export
    respond_to do |format|
    format.csv { stream_csv_report }
   end
  end

  private
  def stream_csv_report
    query = Product.some_query(query_params).to_sql
    query_options = "WITH CSV HEADER"
    # Note that if you have a custom select in your query
    # you may need to generate the header yourself. e.g.
    # => stream.write "Name,Created Date,Price,# of Products"
    # => query_options = "WITH CSV" # note the lack of 'HEADER'
    stream_file("products", "csv") do |stream|
      Product.stream_query_rows(query, query_options) do |row_from_db|
       # row_from_db will be ordered according to the select
       # e.g.
       # => Given: "COPY (SELECT product.name as name, price, created_at 
            FROM sales INNER JOIN ...) TO STDOUT WITH CSV"
       # => row_from_db will look like 'Bag, 500,2010-05-11 
            13:27:58.580456'
       stream.write row_from_db
      end
    end
   end
 end

Benefits:

Generating CSV using a gem and ActiveRecord for large data of more than 1 lakh records would take lots of minutes and slow down the server until it finishes the process but with Postgres, we can achieve this under 2-3 minutes.
Postgres is fast generating large data in CSV.
Data is immediately downloaded on the client-side no need to save anywhere if you don't want to.