DEV Community: Deepak Chauhan

Preventing Concurrent Migrations in a Distributed Environment

Deepak Chauhan — Thu, 25 May 2023 16:59:41 +0000

Introduction:

In a distributed environment, where multiple apps share a single database, running migrations simultaneously can lead to conflicts and deployment failures. In this article, we'll explore a solution to prevent concurrent migrations and ensure smooth deployments. We'll discuss the issue, the approach used, and the code changes implemented to tackle this problem.

Problem:

When running migrations in Rails, they are meant to run sequentially based on their timestamps. However, in a distributed environment like Heroku, multiple apps can trigger migrations simultaneously, causing conflicts and errors. This occurs when one app tries to run a migration while another migration is already in progress, resulting in an ActiveRecord::ConcurrentMigrationError exception.

Approach:

To address this issue, we'll use a combination of advisory locks and environment variables to control the execution of migrations during deployment. The basic idea is to allow only one app to run migrations while the others skip them.

Implementation:

Let's walk through the code changes needed to implement this solution.

Step 1: Create a release-tasks.sh Script
We'll start by creating a shell script called release-tasks.sh in the project's root directory. This script will handle the execution of migrations during the release phase.

bash

#!/bin/bash
set -e

if [ "$RUN_MIGRATIONS" = "true" ]; then
  echo "Running migrations"
  bundle exec rake db:migrate
else
  echo "Skipping migrations"
fi

The script checks the value of the RUN_MIGRATIONS environment variable. If it is set to "true," the script runs the migrations using bundle exec rake db:migrate. Otherwise, it skips the migrations.

Step 2: Grant Execution Permissions to the Script
To allow execution of the release-tasks.sh script, we need to grant it execution permissions. Run the following command:

bash

chmod +x release-tasks.sh

Step 3: Update the Procfile
In the project's Procfile, we need to modify the release command to use our custom script. Open the Procfile and update the line as follows:

release: ./release-tasks.sh

This change ensures that our release-tasks.sh script is executed during the release phase.

Step 4: Set Environment Variables for Each App
In Heroku, go to each app's settings and set the RUN_MIGRATIONS environment variable. For the app that should run migrations, set RUN_MIGRATIONS to "true". For the other apps that should skip migrations, either leave it unset or set it to any other value.

Conclusion:

By implementing this solution, we have effectively prevented concurrent migrations in our distributed environment. By using advisory locks and environment variables, we ensure that only one app runs migrations while others skip them during deployment. This approach promotes a more stable and error-free deployment process.

Remember to test this solution thoroughly in a non-production environment before applying it to your live applications. It's crucial to consider the specific requirements and constraints of your own setup.

I hope this article helps you in handling concurrent migrations in your distributed environment. Feel free to reach out if you have any questions or need further assistance.

Happy coding!

Automating Git Commit Messages with OpenAI's GPT-3: A Step-by-Step Guide

Deepak Chauhan — Mon, 17 Apr 2023 05:44:20 +0000

Automating Git commit messages with OpenAI's GPT-3

Have you ever struggled to come up with a good commit message for your changes? Or have you found yourself writing the same type of commit message over and over again? With OpenAI's GPT-3 language model, you can now automate the process of generating commit messages for your changes.

This blog post will guide you through a simple Python script that uses OpenAI's GPT-3 to generate commit messages based on the changes made to your code.

GitHub repo

The code for this blog post can be found on GitHub at gpt-commit-message-helper.

The script is written in Python and uses the OpenAI API to generate commit messages. Before using the script, you will need to set up an OpenAI API key and install the openai Python library.

Overview of the script

The script works by generating a summary of the changes made to each file, and then using those summaries as prompts for the OpenAI API. The API generates a commit message based on the prompts, which is then returned to the script.

Here's a high-level overview of the script:

Get the list of files that have been changed (using Git).
For each file, generate a summary of the changes made.
Use the summaries as prompts for the OpenAI API to generate a commit message.
Print the suggested commit message and prompt the user to approve it or enter a custom one.
Commit the changes using the selected commit message.
Let's dive into the code.

Importing the necessary libraries and setting up the OpenAI API key

The first few lines of the script import the necessary libraries and set up the OpenAI API key:

import openai
import os
import sys
import subprocess
from requests.exceptions import RequestException

openai.api_key = os.environ.get("OPENAI_API_KEY")
The openai library is used to interact with the OpenAI API. The os, sys, and subprocess libraries are used to interact with the operating system and Git. The RequestException class is imported from the requests library to handle errors when making API requests.

The OpenAI API key is loaded from an environment variable, which should be set up before running the script.

Setting up default values for the OpenAI API parameters

Next, the script sets up default values for the OpenAI API parameters:

openai_engine = os.getenv("OPENAI_ENGINE", "text-davinci-002")
openai_max_tokens = int(os.getenv("OPENAI_MAX_TOKENS", "60"))
openai_temperature = float(os.getenv("OPENAI_TEMPERATURE", "0.7"))
openai_stop = os.getenv("OPENAI_STOP", None)

These parameters control how the OpenAI API generates the commit message. The openai_engine parameter selects which OpenAI model to use. The openai_max_tokens parameter controls how many tokens (words or punctuation) are generated in the message. The openai_temperature parameter controls the "creativity" of the message. The openai_stop parameter can be used to add custom stop words that will cause the API to stop generating text.

Generating a summary of the changes made to a file

The generate_summary function takes a file path as input and generates a summary of the changes made to the file:

def generate_summary(file_path):
    # Get git diff of the file
    output = subprocess.run(['git', 'diff', '--cached', file_path], capture_output=True, text=True)

    # Extract added/modified/deleted lines from git diff
    added_lines = []

Devise and SendGrid integration into rails 6

Deepak Chauhan — Thu, 19 Nov 2020 16:30:54 +0000

Devise is a flexible production-ready authentication system supported in rails. The authentication system basically consists of three functionalities i.e. Signup, Login, and Logout. Further, in Signup, the password has to be a hashed password rather than a normal string and a confirmation link has to be sent on the registered email id for authenticating the user. In Login, a forget password functionality has to be there for recovery of password, and remember me functionality has to be there for saving credentials of the user in browser cookies for providing auto-filled credentials.

Instead of coding all these functionalities from scratch, use Devise. It is pre-equipped with all these functionalities and all of it can be accomplished by just installing the Devise gem. Devise is used extensively with so many functionalities that can be customized but in this article, I will only write about authenticating the registered user through a verification link. After registration, a verifying link has to be sent on the registered email id of the user On clicking it the user can verify himself.

HOW TO ACHIEVE THIS?

Well, we know that all these email operations are performed by the SMTP(Simple Mail Transfer Protocol) server. However, to set up an SMTP server for a beginner is quite back-breaking. Instead, we can use Sendgrid. SendGrid is a cloud-based SMTP provider that allows you to send email without having to maintain email servers. SendGrid manages all of the technical details and left you with just reaping the benefits of someone else’s effort.

LET’S CODE THIS

First, let’s include Devise gem and bundle install it in our gemfile.

gem 'devise'

After bundle install. You need to run the generator:

rails generate devise:install

The generator will install an initializer which describes ALL of Devise’s configuration options. By default, there are no views and models. You have to generate them. Let’s now generate a model User

rails generate devise User

You can also generate the corresponding Devise views.

rails generate devise:views users

You can find a migration file that has been generated. Pull up that migration file and uncomment the 4 lines under confirmable:

t.string :confirmation_token
t.datetime :confirmed_at
t.datetime :confirmation_sent_at
t.string :unconfirmed_email

Pull up the user.rb model file under app/models and in the line for devise, add in a:

:confirmable

, after :registerable, entry Run your migration now to create the users table

rails db:migrate

Devise will create some helpers to use inside your controllers and views. To set up a controller with user authentication, just add this before_action

before_action :authenticate_user!

Now, Let’s hop onto sending email in production

for beginners, we are using Heroku as a production server. First, add your credit card details to your Heroku account. (Don’t worry this will not cost you a penny). Then enter in:

heroku addons:create sendgrid:starter

This will add a SendGrid add-on to your deployed Heroku project. You can log in to your SendGrid account through this addon and in your SendGrid account menu you have to create an API key. After creating an API key, Now, we have to set the SendGrid API key credentials you created for Heroku.

heroku config:set SENDGRID_USERNAME=apikey
heroku config:set SENDGRID_PASSWORD=enterintheapikey

In the SENDGRID_PASSWORD you have to enter the API key that you have created from SendGrid above Rails provides you to securely store your API credentials in your credentials.yml file. All the data inside this file is encrypted so security is ensured. The encrypted credentials.yml.enc file will be decrypted in production, using a key stored in the RAILS_MASTER_KEYenvironment variable For people using the VS Code as an editor. You can edit the credentials.yml file by enabling the edit mode as below

EDITOR='code --wait' rails credentials:edit

After issuing this command you can store your Sendgrid credentials in this file

 sendgrid:
   user_name: apikey
   password: Your API key

Under config/environment/production.rb file add in the following code

 ActionMailer::Base.smtp_settings = {
      :address => 'smtp.sendgrid.net',
      :port => '587',
      :authentication => :plain,
      :user_name => Rails.application.credentials.dig(:user_name),
      :password => Rails.application.credentials.dig(:password),
      :domain => 'heroku.com',
      :enable_starttls_auto => true
    }
    config.action_mailer.delivery_method = :smtp
  config.action_mailer.default_url_options ={:host => 'yourherokuapp.herokuapp.com', :protocol => 'https'}

Now update the development.rb file under config/environments folder and add the following lines

 ActionMailer::Base.smtp_settings = {
      :address => 'smtp.sendgrid.net',
      :port => '587',
      :authentication => :plain,
      :user_name => Rails.application.credentials.dig(:user_name),
      :password => Rails.application.credentials.dig(:password),
      :domain => 'heroku.com',
      :enable_starttls_auto => true
    }

    config.action_mailer.delivery_method = :smtp
    config.action_mailer.default_url_options ={:host => 'http://localhost:3000'}

This setup configuration is both for the production and development environment.
Now it is all set up. When you sign up now SendGrid will send a verification link to the registered email of the user. The confirmation link will look something like this.

Sendgrid verification link

If you want to redirect the user to a specific URL after they clicked the link in the confirmation email, override the after_confirmation_path_for in your confirmations_controller:

Create a new confirmations_controller.rb in app/controllers directory:

class ConfirmationsController < Devise::ConfirmationsController
  private
  def after_confirmation_path_for(resource_name, resource)
    sign_in(resource) # In case you want to sign in the user
    your_new_after_confirmation_path
  end
end

In config/routes.rb, add this line so that Devise will use your custom ConfirmationsController. This assumes Devise operates on the users table (you may edit to match yours).

devise_for :users, controllers: { confirmations: 'confirmations' }

Restart the webserver, and you should have it.