DEV Community: Kartikey Tanna

How to deploy a React app with Kamal (formerly known as MRSK) & GitHub Action

Kartikey Tanna — Mon, 17 Jun 2024 05:49:34 +0000

I recently helped Sidecar Learning{:target="_blank"} move their legacy application built with React front-end and Rails back-end migrate from Heroku and AWS to VPS over at DigitalOcean.

This guide is outcome of that migration. React front-end was quite simple and do not have much moving parts so it's quite simple to do this. If you are looking to deploy monolith application then you can read the following posts I have written:

Kamal Config

First, let's create a simple deploy.yml for Kamal. The configuration is mostly boilerplate. I am using GHA cache{:target="_blank"} to speed up the deploys.

# config/deploy.yml

service: react-app
image: username/image-name

servers:
  web:
    - 123.456.789.012

registry:
  username: docker_username
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    FOO: BAR
    NPM_CONFIG_PRODUCTION: false

builder:
  multiarch: false
  cache:
    type: gha
    options: mode=max

If you wish to, you can use "registry" cache as well. I have noticed that it's faster by 5-10 seconds but if you are using free Docker Hub account then you only get one free image. That's why I prefer to use "gha" as cache back-end. Kamal only supports{:target="_blank"} "gha" and "registry". If you would like to use the "registry" cache, use the following builder config:

builder:
  multiarch: false
  cache:
    type: registry
    options: mode=max,image-manifest=true,oci-mediatypes=true

Dockerfile

Our app is using quite old version of NodeJS. This is why I like Kamal more than anything. No platform dependency or requirement to fulfil. You create your own environment and Kamal provides thin wrapper around the Docker commands. As simple as it could be.

Here, we first create a build step. Once the build is ready, no need to include all those node_modules in our final application. We keep it as small as possible so that it's easier to perform operations.

In our app, we have a small Express app that serves the static file. That's why I have created a separate package.runtime.json file. You can modify that part as per your own need. The next session will explain how the express server works.

# Dockerfile

FROM node:11.10.1 as build

WORKDIR /app

ENV NODE_ENV="production" \
    NPM_CONFIG_PRODUCTION="false"

COPY . .

RUN npm ci

RUN npm run build

FROM node:11.10.1

WORKDIR /app

ENV NODE_ENV="production" \
    NPM_CONFIG_PRODUCTION="false"

COPY --from=build /app/build ./build
COPY --from=build /app/server.js ./server.js
COPY --from=build /app/package.runtime.json ./package.json

RUN npm install --production

EXPOSE 3000

CMD ["node", "server.js"]

Express Server

We have a tiny express server that serves the static files with appropriate headers. We have configured Cloudflare to provide SSL support and cache the static assets. Only index.html file is not cached, and bundled assets will be served by this Express server only once. Then Cloudflare will take the load. All of this is deployed on our $4 DigitalOcean droplet and it is handling our moderately busy app very well.

I have configured{:target="_blank"} the Webpack to generate bulid with hash, so with every new build, the new hash will rename the file - same as Propshaft{:target="_blank"}.

I have added a route for Kamal healthcheck as well even though the health-check step has been removed{:target="_blank"} in Kamal 1.6.0.

// server.js

const compression = require('compression');
const express = require('express');
const path = require('path');
const port = process.env.PORT || 3000;
const app = express();

// Use compression middleware
app.use(compression());

// Serve static files with cache control headers
app.use(express.static(path.join(__dirname, 'build'), {
  maxAge: '1d', // Cache for 1 day
  setHeaders: (res, path) => {
    // Set cache-control headers based on file types
    if (path.endsWith('.js')) {
      res.setHeader('Cache-Control', 'public, max-age=31536000'); // 1 year
    } else if (path.endsWith('.css')) {
      res.setHeader('Cache-Control', 'public, max-age=31536000'); // 1 year
    } else if (path.endsWith('.html')) {
      res.setHeader('Cache-Control', 'public, max-age=0'); // No cache
    } else {
      res.setHeader('Cache-Control', 'public, max-age=86400'); // 1 day
    }
  }
}));

// Kamal health check route
app.get('/up', (req, res) => {
  res.send(`<!DOCTYPE html><html><body style="background-color: green"></body></html>`);
});

// Send all requests to index.html to handle routing in React Router
app.get('*', (req, res) => {
  res.sendFile(path.resolve(__dirname, 'build', 'index.html'));
});

// Start the server
app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

We only need ExpressJS and the compression packages to run this small server. That's why we are including only these two packages in the runtime package file.

package.runtime.json

{
  "name": "your-app",
  "version": "1.0.0",
  "main": "server.js",
  "dependencies": {
    "compression": "^1.7.4",
    "express": "^4.16.4"
  },
  "scripts": {
    "start": "node server.js"
  }
}

GitHub Action

It's a headache to run the deployment from your own machine. The following GitHub Actions config will solve it for you. I have added a concurrency config too that will let only one deploy run at a time. It's useful when you make multiple commits in a short time.

# .github/workflows/deploy.yml

name: Deploy to production

# To make sure that only one deploy runs at a time. Deploy lock will not let simultaneous deployments.
concurrency:
  group: ${{ github.workflow }}

on:
  push:
    branches: [master]

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      KAMAL_REGISTRY_PASSWORD: ${{ secrets.KAMAL_REGISTRY_PASSWORD }}

    steps:
      - name: Set up Docker Buildx for cache
        uses: docker/setup-buildx-action@v3

      # Since we are using GHA cache, we need to expose the cache to the runtime
      - name: Expose GitHub Runtime for cache
        uses: crazy-max/ghaction-github-runtime@v3

      - name: Checkout code
        uses: actions/checkout@v3

      # Ruby is only needed to install Kamal
      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.3'

      - name: Install dependencies
        run: gem install kamal

      # This is to facilitate Kamal with the private key to access server(s)
      - uses: webfactory/ssh-agent@v0.7.0
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}

      - name: Run deploy command
        run: kamal deploy

Understanding the role of the “schema.rb” file in Ruby on Rails development

Kartikey Tanna — Sun, 06 Aug 2023 00:00:00 +0000

I recently published a post in a Ruby on Rails group on LinkedIn about a rake task I bring to most of my projects. This Rake task deletes the schema.rb file and regenerates it. I got many reactions from fellow developers and that motivated me to write this post.

Understanding the Purpose of Rails’ schema.rb File

The schema.rb file is essentially a blueprint of your Rails application’s database. Imagine your app as a house - the schema.rb is the floor plan you’d show your architect. It outlines your database’s structure, detailing tables, columns, their types, primary keys, and even relationships between them. Migrations update and generate this file automatically.

Why is it important? Think of it this way: a new team member joins and needs to understand your database structure. Rather than making them trawl through countless migrations (a process as thrilling as watching paint dry), they can reference the schema.rb file. This file provides a concise summary of your database structure, the Cliff Notes version, if you will.

The schema.rb file in a new Rails application contains the following comment:

# This file is auto-generated from the current state of the database. Instead  
# of editing this file, please use the migrations feature of Active Record to  
# incrementally modify your database, and then regenerate this schema definition.  
#  
# This file is the source Rails uses to define your schema when running \`bin/rails  
# db:schema:load\`. When creating a new database, \`bin/rails db:schema:load\` tends to  
# be faster and is potentially less error prone than running all of your  
# migrations from scratch. Old migrations may fail to apply correctly if those  
# migrations use external dependencies or application code.  
#  
# It's strongly recommended that you check this file into your version control system.

Decoding the Authority Over the Database in Rails

The Rails docs’ interpretation of the authority over the database schema has seen notable changes over the years. To illustrate this, let’s look at two significant commits on the Rails GitHub.

The first version of the docs from 2012 claimed that the db/schema.rb or an SQL file generated by Active Record was the authoritative source for your database schema.

What are Schema Files for?

Migrations, mighty as they may be, are not the authoritative source for your database schema. That role falls to either db/schema.rb or an SQL file which Active Record generates by examining the database. They are not designed to be edited, they just represent the current state of the database.

There is no need (and it is error prone) to deploy a new instance of an app by replaying the entire migration history. It is much simpler and faster to just load into the database a description of the current schema.

However, the follow-up version from 2018 which we read today clearly states that your actual database remains the authoritative source.

What are Schema Files for?

Migrations, mighty as they may be, are not the authoritative source for your database schema. Your database remains the authoritative source. By default, Rails generates db/schema.rb which attempts to capture the current state of your database schema.

It tends to be faster and less error prone to create a new instance of your application’s database by loading the schema file via rails db:schema:load than it is to replay the entire migration history. Old migrations may fail to apply correctly if those migrations use changing external dependencies or rely on application code which evolves separately from your migrations.

The commit was made with the following comment from the contributors:

**Update `schema.rb` documentation [CI SKIP]**

The documentation previously claimed that `db/schema.rb` was “the  
authoritative source for your database schema” while simultaneously  
also acknowledging that the file is generated. These two statements are  
incongruous and the guides accurately call out that many database  
constructs are unsupported by `schema.rb`. **This change updates the**  
**comment at the top of `schema.rb` to remove the assertion that the file**  
**is authoritative.**

The documentation also previously referred vaguely to “issues” when  
re-running old migrations. This has been updated slightly to hint at the  
types of problems that one can encounter with old migrations.

In sum, this change attempts to more accurately capture the pros, cons,  
and shortcomings of the two schema formats in the guides and in the  
comment at the top of `schema.rb`.

[Derek Prior & Sean Griffin]

While db/schema.rb and migrations play vital roles in managing your database structure, neither are the definitive descriptors of your schema. The db/schema.rb file is a Rails-generated snapshot of your database structure, and is useful for setting up new instances of your database quickly. Migrations, however, are used to implement incremental changes to your database over time.

As developers, ensuring your migrations are up-to-date and error-free is paramount. These migrations should be able to recreate db/schema.rb accurately. In the event of any breaking changes, such as class name alterations, best practices must be followed to ensure your migrations remain reversible and unaffected by such changes. Understanding these principles will help manage your database schema effectively in Rails.

Working with schema.rb

While it might be tempting to manually modify schema.rb, resist the urge! Any changes should be made through migrations. This ensures that the schema.rb file can be regenerated correctly.

When working on a new or unreleased project, it might make sense to adjust existing migrations rather than creating new ones. This way, you keep the migration history lean and focused. However, if a project is already in production, creating new migrations for each database change is the standard practice. This ensures that your database changes are properly tracked, and each team member can understand when and why the database structure was changed.

In new and unreleased projects, I often opt to modify existing migrations rather than creating new ones. In older projects where migrations are being rectified, I use a custom rake task to streamline the process. Here’s a snippet of the task for resetting the database by dropping the schema:

# lib/tasks/complete_reset.rake

namespace :db do
  desc 'reset the database by dropping the schema'
  task complete_reset: :environment do
   raise unless Rails.env.local?

    FileUtils.rm_f('db/schema.rb')
    Rake::Task['db:drop'].invoke
    Rake::Task['db:create'].invoke
    Rake::Task['db:migrate'].invoke
    Rake::Task['db:seed'].invoke
    Rake::Task['dev:prime'].invoke
  end
end

This task handles the removal of db/schema.rb, drops the database, creates it anew, migrates and seeds it, and primes it for development. Be extremely cautious when choosing the environment this task is applied to; it should not be used on deployed servers once your app has hit staging or production stages.

This task becomes particularly useful in the early stages of a project when specifications are frequently changing. When switching between branches, conflicts in the schema.rb file can often arise due to these rapid changes. This task helps you avoid creating excessive migrations and ensures a smooth, efficient development process by resolving such conflicts.

It’s important to note that this task also invokes another task, dev:prime. This primes the development database with test data, providing a consistent starting state after a reset. So, firing complete_reset on the development environment not only clears any schema conflicts, but also populates your database with test data for a fresh start.

FAQs

Should I modify the `schema.rb` file manually?

As a rule of thumb, you should not manually modify the schema.rb file. Instead, you should use ActiveRecord Migrations to alter your database schema. This ensures that changes are properly tracked and the schema.rb is updated correctly. In rare, exceptional cases with legacy code or failed migrations, direct modification could be considered but it’s a risky approach and should only be done with extreme caution.

What are some best practices for managing the `schema.rb` file in a team environment?

Version Control : Always keep the schema.rb file in your version control system to keep track of its changes over time.
Don’t Modify Manually : Avoid making manual modifications to the schema.rb file. Use ActiveRecord migrations instead.
Review Changes : Before committing changes to the schema.rb file, review them to make sure they align with the changes made in your migrations.
Sync with Database : Always make sure your schema.rb file is synchronized with the current state of your database schema.

What happens if the `schema.rb` file is deleted or modified?

If the schema.rb file is deleted or modified manually, it may lead to inconsistencies between the actual state of your database and the Rails application’s understanding of the database schema. This can lead to unexpected errors or bugs in your application. Therefore, it is advised not to delete or manually modify the schema.rb file.

How to enable Traefik dashboard with Kamal (formerly known as MRSK)

Kartikey Tanna — Wed, 12 Apr 2023 13:10:00 +0000

Kamal, formerly known as MRSK uses Traefik as a dynamic reverse-proxy. Traefik has a beautiful dashboard to visually display the configuration. Kamal is not configured to show the dashboard by default.

Traefik dashboard can run in two modes - secure and insecure. Traefik recommends secure mode but this post wil cover how to enable both the modes.

First, let’s see the unsecure mode. Unsecure mode is very easy to configure. It is unsecure because it exposes Traefix API on the entrypoint. It means that after configuring the dashboard in unsecure mode, the dashboard will be available on the port 8080 of the host.

Adding the following snippet in the config/deploy.yml file will enable the unsecure dashboard.

# config/deploy.yml
traefik:
  options:
    publish:
      - 8080:8080
  args:
    api.dashboard: true
    api.insecure: true

After adding the configuration, run the kamal traefik reboot command to apply the configuration. This command will stop, remove and start new container again with the latest configuration. The dashboard should be visible on the port 8080 of the host server now e.g. http://99.99.99.99:8080

Now, let’s see about the secure mode. It’s called secure mode because the API is not expose on the entrypoint. We need to create a router rule that uses api@internal service. Let’s look at the configuration for the secure model.

traefik:
  options:
    publish:
      - 8080:8080
  args:
    api.dashboard: true
  labels:
    traefik.enable: "true"
    traefik.http.routers.dashboard.rule: Host(`traefik.example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
    traefik.http.routers.dashboard.service: "api@internal"
    traefik.http.routers.dashboard.middlewares: "auth"
    traefik.http.middlewares.auth.basicauth.users: test:$2y$05$H2o72tMaO.TwY1wNQUV1K.fhjRgLHRDWohFvUZOJHBEtUXNKrqUKi

Here, we have configured Traefik dynamically with help of Docker labels. First of we have created a router. Then we attached the router with the api@internal service because in the secure mode we have to do this manually. After that, we added the auth middleware to Trafeik. In the last, we conifgured this auth middleware to use HTTP Basic Authentication and provided it with the credentials. You can read more about the rules in the details on the Traefik docs.

The credentials are in the “username:hashed_password” format. The credentials are generated with the htpasswd command. Let’s say you want to create a user with the username “admin” and the password “super_strong_password” then you can use the following command:

htpasswd -nb admin super_strong_password
# output: admin:$apr1$2FGO09Gu$PSZdmmJqyrXWYvidWAm6p0

You will get the password hash in the output. Just copy paste the output with the username:password in the labels. The official Traefik docs mention that you need to escape the $ character but you don’t need if you are using Kamal but Kamal escapes the $ sign for these labels.

That’s it! Don’t forget to reboot the Trafeik container with the kamal traefik reboot comamnd. After that, the dashboard should be accessible on the http://traefik.example.com/dashboard endpoint.

P.S. I would not recommend running the dashboard permanently if you are using a small single server with multple apps. Because that would eat up the valuable server resources.

How to deploy a NodeJS application using MRSK

Kartikey Tanna — Mon, 10 Apr 2023 08:25:00 +0000

Kamal, formerly known as MRSK, is created by DHH - founder of Rails, but the tool is not limited to deploy only Rails applications. Kamal is a simple tool that automates some Docker related commands. That is why it can support any platform/language. In this post, let’s see how to deploy a NodeJS application on a small DigitalOcean VPS using Kamal.

First of all, let’s generate an express app. I installed express-generator and generated a new app using the express express-app command.

Then let’s create a Dockerfile:

FROM node:16

# Create app directory
WORKDIR /usr/src/app

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

RUN npm install
# If you are building your code for production
# RUN npm ci --omit=dev

# Bundle app source
COPY . .

EXPOSE 3000
CMD ["node", "bin/www"]

Now, let’s use Kamal to generate the config files. Make sure you have Kamal installed on your machine. If not, you can follow these instructions on README.

Let’s init the configuration with kamal init. It will create config/deploy.yml and .env files in your root directory. My config file looks like this:

service: express-app

image: user/express-app
servers:
  - 146.190.86.77

registry:
  username: user
  password:
    - KAMAL_REGISTRY_PASSWORD

healthcheck:
  path: /
  port: 3000

The config file I have created here is minimal. It defines a service, provides the server IP, Docker Hub registry info and healthcheck information. After successful deploy, Kamal will ping the “/” path on port “3000” and will expect the “200 OK” response from the server.

Now, let’s deploy the app with the kamal deploy command. Kamal will deploy your app and you can check if it is live or not by entering the server IP address in the browser. If you want to connect your app with the database, or host multiple environment of the application like staging, production then you can check out my previous posts. They are platform agnostic and applicable on NodeJS apps as well.

How to deploy multi-environment(staging, production) application using Kamal (preivously MRSK)

Kartikey Tanna — Sun, 09 Apr 2023 08:45:00 +0000

In the previous post, I described how to host a Rails app and a database on a single server. This post will describe what if you want to host multiple environments i.e. staging, production of the same application using Kamal - previously known as MRSK.

Kamal uses Traefik as a reverse proxy. It means that any incoming request will be handled by Traefik. Traefik will handover the request to appropriate server and Docker container based on the configuration. For example, let’s assume that we have deployed the staging and the production version of our apps on the server. Then we have to configure Traefik in such a way that it points “staging.myapp.com” and “production.myapp.com” to the staging and the production Docker containers respectively.

Kamal uses Traefik with “docker” as a provider. Traefik rules are configured by providing certain lables to Docker containers. You can read more about it in their docs.

Let’s start configuring Kamal. Kamal has a feature called “destination”, we will use that to create two separate destinations - “staging” and “production”. First, we will create the config/deploy.yml file with common options like service name, image name, registry, common environment variables, and Postgresql database as acessory.

# config/deploy.yml
service: myapp
image: user/myapp

registry:
  username: user
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    DB_HOST: 99.99.99.99
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:15
    host: 99.99.99.99
    port: 5432
    env:
      clear:
        POSTGRES_USER: 'myapp_db_user'
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

Staging config

Now, let’s create a destination named “staging” with the config/deploy.staging.yml file.

# config/deploy.staging.yml
servers:
  - 99.99.99.99

labels:
  traefik.http.routers.myapp-web-staging.rule: Host(`staging.myapp.com`)

env:
  clear:
    POSTGRES_DB: myapp_staging
    IS_STAGING: true

In this file we have declared the server for the staging environment, environment variable spcific to the environment and we have used the labels key that will attach the labels to the staging Docker container.

Let’s understand these rules with little more details. Kamal applies some default Traefik labels to each container in the “service-role-destination” format. With the help of these labels Traefik defines a service (not to be confused with Kamal service). With the rule mentioned in the deploy.staging.yml file above, we are overriding the default labels. Since we have not specified any role, Kamal will assign the web role to the service. Anyways, there has to have one “web” role if we are specifying roles.

Don’t forget to replace “staging.myapp.com” with your domain. The domain should be configured to point to the IP of the server.

Production config

# config/deploy.production.yml
servers:
  - 99.99.99.99

labels:
  traefik.http.routers.myapp-web-production.rule: Host(`rails.myapp.com`)

env:
  clear:
    POSTGRES_DB: myapp_production

The deploy.production.yml is similar to the staging configuration. Since we are going to deploy both the staging and the production as well as Postgresql(the database) on the same server, the same IP is configured under the hosts key. Again, don’t forget to configure your domain.

That’s it! You should have both the staging and the production as well as the database deployed on the same or different server(s) and accessible from the domains you have set for each environment!

Deploy Rails app and Postgres with Kamal(previously MRSK) on single DigitalOcean server

Kartikey Tanna — Wed, 05 Apr 2023 13:08:00 +0000

For me, it’s been always cumbersome to host a Rails side project or personal application. Sure, Heroku is straightforward but it’s not as cheap as I would like it to be. And I don’t like the uptime limitation their free plan has. I have been making my way through Dokku and Capistrano. Luckily, a new door opened when Kamal, previously known as MRSK was launched by DHH recently.

In this post, I am not getting into how Kamal works and the benefits of using it. DHH has created this fantastic introduction video for that.

Kamal is capable to deploy applications on multiple servers as well as a single server. In this post, I will describe how we can deploy a Rails application and Postgresql on a single VPS.

First of all, install and init Kamal on your local machine with instructions on the README.

The key to the single server setup is to use the same hosts for the web as well as accessories. Here is what the deploy.yml file looks like for that setup:

service: my-app
image: user/my-app

servers:
  web:
    - 134.209.111.91

registry:
  username: tannakartikey
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    DB_HOST: 134.209.111.91
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:15
    host: 134.209.111.91
    port: 5432
    env:
      clear:
        POSTGRES_USER: 'my_app'
        POSTGRES_DB: 'my_app_production'
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

I am using the default Dockerfile that is generated with Rails 7.1. Create and place the Dockerfile in your root directory if it does not exist already. You also need to create a .env file in the root of the repository and also need to make minor changes in your config/database.yml file.

I have created this sample repository that has all the code described in this post. This commit holds all the Kamal setup-related changes.

Be sure NOT to include your .env file in Git or your Dockerfile. If you are using Docker Hub or any other registry, make sure to make the image private because it holds a copy of the application code.

Next, we need to set up the server since we are deploying to the server for the first time. We can do it with:

bin/kamal setup

The setup command will set up all the accessories and deploy the app on the server.

That’s it! You should have a running Rails app on the server which is connected to Postgresql on the same server. For subsequent deploy you can use kamal deploy or kamal redeploy based on your needs.

I have been using the USD4 droplet on DigitalOcean. If you are using a similar configuration I would recommend creating a swap partition on the server. I am assuming that if you are doing this, you are not expecting loads of traffic on day one. But without a swap partition, some simple operations like migrating the database may also fail.

It can also be possible to host the same code multiple times for different environments i.e. staging, production etc. For that, the “role” and “destination” functionality of Kamal can be used. I might cover that in a separate post.

The surpising behaviour of Rails' default_scope method

Kartikey Tanna — Sun, 10 Jul 2022 14:45:06 +0000

It is very convinient to declare default_scope in the Rails models. Specially for something like deleted_at. But it has an interesting side effect that has to be kept in mind.

Let's take an example model Bookwith an attribute approved.

This is how the column is defined in the migration:

t.boolean :approved, null: false, default: false

This is how the default_scope method is declared in the model:

class Book
  default_scope -> { where(approved: true) }
end

Whenever a new record is created, the approved is going to be true for the record even though the migration has the default value set as false. Let's see it in action:

book = Book.new
book.approved # true

To conclude, the default_scope overrides the default value set at the migration level. That has to be kept in mind while declaring default_scope in a model.

Memory efficient way of reading and downloading a large file in Ruby

Kartikey Tanna — Fri, 02 Oct 2020 00:00:00 +0000

To read a large file from the disk

File.foreach method reads the file line by line; that is why it is safe to use for large files.

It can accept the block to execute each line of a file.

Example:

File.foreach('example.jsonl') { |line| JSON.parse(line) }

However, in the Ruby documentation, you will not find this method defined on the File class. It is defined on IO, which is a superclass of File.

To download the large files from the Internet

We can use IO.copy_stream to download the large files from the Internet. It will create a stream instead of loading the whole file into memory before writing.

Example:

IO.copy_stream(open('http://example.com/file.jsonl'), 'file.jsonl')

To sum it up…

Memory friendly methods:

File.foreach, IO.copy_stream

Other methods to read a file:

File.read, File.readlines

Convert API Response Into a Model in Rails

Kartikey Tanna — Wed, 30 Sep 2020 00:00:00 +0000

Let’s get down straight to the business.

Let’s take an example of showing the invoices to the user from Stripe.

First, let’s write what we would like to have without worrying about the implementation.

I like short and clean controllers. Something like the following would do.

app/controllers/invoices_controller.rb

class InvoicesController < ApplicationController
  def index
    @invoices = Invoice.find_all_by_user(current_user)
  end

  def show
    @invoice = Invoice.new(params[:id])
    render
  end
end

Below is what I would like in my view: app/views/invoices/_invoice.html.erb

<% unless (invoice.total == 0) %>
  <tr>
    <td><%= link_to invoice.number, invoice_path(invoice.id) %></td>
    <td><%= invoice.date %></td>
    <td><%= number_to_currency(invoice.total, negative_format: "(%u%n)") %></td>
    <td><%= invoice.period_start %> to <%= invoice.period_end %></td>
    <td><%= invoice.paid? ? 'Paid' : 'Unpaid' %></td>
  </tr>
<% end %>

I have experienced that writing the interface first, as we did above, gives me a lot of clarity during implementation. Now, let’s start implementing it in a model.

app/models/invoice.rb

class Invoice

  attr_reader :stripe_invoice

  def self.find_all_by_user(user)
    if user.present?
      stripe_invoices_for_user(user).map do |invoice|
        new(invoice)
      end
    else
      []
    end
  end

  def initialize(invoice_id_or_object)
    if invoice_id_or_object.is_a? String
      @stripe_invoice = retrieve(invoice_id_or_object)
    else
      @stripe_invoice = invoice_id_or_object
    end
  end

  def to_partial_path
    "invoices/#{self.class.name.underscore}"
  end

  def id
    stripe_invoice.id
  end

  def number
    stripe_invoice.number
  end

  def total
    cents_to_dollars(stripe_invoice.total)
  end

  def date
    convert_stripe_time(stripe_invoice.date)
  end

  def paid?
    stripe_invoice.paid
  end

  def subscription
    stripe_invoice.subscription
  end

  def period_start
    convert_stripe_time(stripe_invoice.period_start)
  end

  def period_end
    convert_stripe_time(stripe_invoice.period_end)
  end

  def user
    @user ||= User.find_by(stripe_customer_id: stripe_invoice.customer)
  end

  def balance
    if paid?
      0.00
    else
      amount_due
    end
  end

  def amount_due
    cents_to_dollars(stripe_invoice.amount_due)
  end

  def subtotal
    cents_to_dollars(stripe_invoice.subtotal)
  end

  def amount_paid
    if paid?
      amount_due
    else
      0.00
    end
  end

  def plan
    stripe_invoice.lines.data[0].plan.name
  end

  def plan_amount
    cents_to_dollars stripe_invoice.lines.data[0].plan.amount
  end

  def pay
    stripe_invoice.pay
  end

  def self.pay_if_pending(user)
    invoices = find_all_by_user(user)
    unless invoices.empty? || invoices.first.paid?
      invoices.first.pay
    end
  end

  def self.upcoming(user)
    new(Stripe::Invoice.upcoming(customer: user.stripe_customer_id) || nil )
  end

  private

  def self.stripe_invoices_for_user(user)
    Stripe::Invoice.all(customer: user.stripe_customer_id).data
  end

  def retrieve(invoice_id)
    Stripe::Invoice.retrieve(invoice_id)
  end

  def convert_stripe_time(time)
    Time.zone.at(time).strftime('%D')
  end

  def cents_to_dollars(amount)
    amount / 100.0
  end
end

Here, the Stripe gem takes exposes many methods on the response. But we can take any vanilla JSON response any do the same.

Many years ago, I learned this pattern form Upcase. Since then I am always parsing API responses like this. Upcase is free now and definitely worth a look.

Download the code as gist.

Many-to-many self join in Rails

Kartikey Tanna — Tue, 29 Sep 2020 00:00:00 +0000

Let’s say we have “products” and we want to prepare “kits” of those products. Kits are nothing but the group of the products.

We can use many-to-many relationship here because products can be in many kits, and kits can be associated with many products.

Also, since the kits are just grouping the products, we can use self-joins. There are multiple ways we can implement self-joins.

Using has_and_belongs_to_many

You can read more about has_and_belongs_to_many on Rails docs.

Migration

class CreateJoinTableProductKits < ActiveRecord::Migration[6.0]
  def change
    create_table :product_kits, id: false do |t|
      t.references :product, null: false, foreign_key: true, index: false
      t.references :kit, null: false, foreign_key: { to_table: :products }, index: false
      t.index [:product_id, :kit_id], unique: true
    end
  end
end

Model

class Product < ApplicationRecord
  has_and_belongs_to_many :kits,
    join_table: :product_kits,
    class_name: 'Product',
    association_foreign_key: 'kit_id'
end

Using has_many through

This approach is better because later on in your project you can add more fields and validations in ProductKit model.As you know, our projects are always dynamic and most of the time(all the time) we end up modifying the flow. So, it isbetter to be prepared and use has_many :through from the beginning.

More on, has_many :through on Rails docs.

Migration

class CreateJoinTableProductKits < ActiveRecord::Migration[6.0]
  def change
    create_table :product_kits do |t|
      t.references :product, null: false, foreign_key: true, index: false
      t.references :kit, null: false, foreign_key: { to_table: :products }, index: false
      t.index [:product_id, :kit_id], unique: true

      t.timestamps
    end
  end
end

Model `app/models/product.rb`

class Product < ApplicationRecord
  has_many :product_kits, dependent: :destroy
  has_many :kits, through: :product_kits
end

Model `app/models/product_kit.rb`

class ProductKit < ApplicationRecord
  belongs_to :product
  belongs_to :kit, class_name: 'Product'
end

Challenges I face as a freelancer and how I overcome them

Kartikey Tanna — Sun, 20 Sep 2020 00:00:00 +0000

My initial days as a freelancer were rather difficult and confusing. There were multiple challenges that I learned to overcome the hard-way.

Many people assume that the biggest challenge while freelancing is about finding work. Once the ball starts rolling, selecting, managing, and planning the work becomes a bigger concern.

Finding work Finding good work

Initially, as a freelancer, when I would receive an offer, I used to be apprehensive about things like:

What if I get a better offer later?

Should I go for a short-term contract with higher hourly rates or a long-term contract with lesser hourly rates?

How long will the project go on?

What if the client went out of funds earlier than they had foreseen?

I’d either not accept any offer and lose time and money or I’d accept multiple offers and deliver inferior work quality according to my standards.

So in the past few years, I have learned to overcome these concerns by not accepting a less paying job out of desperation. I keep my hourly rates at par with the market and my skills. Being realistic, neither underestimating nor overestimating myself.

Additionally, communication is the key! I outrightly started asking the client regarding the project’s duration and financial status. I convey to them how these aspects of the project can affect me.

In-between projects Personality update available

No matter how careful I am, sometimes, there is no full-time project in hand for a few days. This initially bothered me as the cash flow would take a hit. Saving and having knowledge about investment came into play after such incidents. The Intelligent Investor turned out to be a good start.

I also realized that this is the time to improve my online presence and resume, contribute to open source, work on personal projects, and learn new things. These things help me in acquiring the next job sooner and at a higher hourly rate.

Exploiting Appreciating flexible timings

Another challenge is prioritizing! As a freelancer, flexible working hours come as a boon, but the same can be a curse if you keep procrastinating. I observed that when I didn’t set a target then I’d either hardly work or would keep working endlessly. I’d either make the entire week a weekend or keep scrolling through assigned tickets, responding to messages even while having dinner with family, and reading feature suggestions till late night. All of this only decreased my work efficiency. To enjoy the perks of flexible timings, I don’t let important tasks become urgent, and I have a goal that I have to achieve by Friday so that I can have a good weekend with my family.

Being outdated updated

As a freelancer, it is also vital that we keep learning. If you are not updated, then you are outdated.

I upgrade my knowledge by contributing to open-source, lots of reading, participating in conferences, online courses, and working on personal projects. I have subscribed to portals like Frontend Masters, Egghead, TestingJavascript, Pirple, Coding-Gnome, and regularly take courses there (links are not affiliated/sponsored).

If you liked this article, check out my other articles on freelancing.

The 3 Roles of a Freelance Developer

Kartikey Tanna — Sun, 06 Sep 2020 00:00:00 +0000

According to Hindu mythology, the three cosmic functions of creation, sustaining, and transformation/destruction of the universe are performed by Lord Brahma, Lord Vishnu, and Lord Shiva, respectively. Similarly, to stand apart from others, we have to perform these three Godlike functions in our human profession! Let’s look into it from a freelance developer’s point of view.

Being Brahma: the creator

Creating need not necessarily refer to just building web apps. There are many other tasks that you have to look into, like:

Building a remarkable online presence to get new clients
Creating a network of clients
Drafting proposals
Coming up with creative ideas to improve the products.
A strategy to follow throughout the project.

Being Vishnu: the sustainer

Once new projects start rolling, there is quite an amount of maintenance required, viz.:

Maintaining a healthy relationship with the clients, including the previous ones.
Stick to the strategy decided and maintain the pace of the project.
Updating documentation, fixing bugs, upgrading dependencies, keeping an eye on the performance.
Managing your finances and maintaining good health. (Do not underestimate this!)

Being Shiva: the transformer/destroyer

Update to improvise!

Transform and upgrade your work-style and strategy if needed.
Troubleshooting technical challenges
Refactor the code and get rid of the clutter.

There might be various other things apart from the above that a freelance developer is required to do, but, interestingly, all of it falls under these three categories.

If you are a freelance developer, its a good idea to regularly introspect which of these three categories you expertise at and which you need to work upon.

DEV Community: Kartikey Tanna

How to deploy a React app with Kamal (formerly known as MRSK) & GitHub Action

Kamal Config

Dockerfile

Express Server

GitHub Action

Understanding the role of the “schema.rb” file in Ruby on Rails development

Understanding the Purpose of Rails’ schema.rb File

Decoding the Authority Over the Database in Rails

Working with schema.rb

FAQs

Should I modify the schema.rb file manually?

What are some best practices for managing the schema.rb file in a team environment?

What happens if the schema.rb file is deleted or modified?

How to enable Traefik dashboard with Kamal (formerly known as MRSK)

How to deploy a NodeJS application using MRSK

How to deploy multi-environment(staging, production) application using Kamal (preivously MRSK)

Staging config

Production config

Deploy Rails app and Postgres with Kamal(previously MRSK) on single DigitalOcean server

The surpising behaviour of Rails' default_scope method

Memory efficient way of reading and downloading a large file in Ruby

To read a large file from the disk

To download the large files from the Internet

Convert API Response Into a Model in Rails

Many-to-many self join in Rails

Using has_and_belongs_to_many

Migration

Model

Using has_many through

Migration

Model app/models/product.rb

Model app/models/product_kit.rb

Challenges I face as a freelancer and how I overcome them

Finding work Finding good work

In-between projects Personality update available

Exploiting Appreciating flexible timings

Being outdated updated

The 3 Roles of a Freelance Developer

Being Brahma: the creator

Being Vishnu: the sustainer

Being Shiva: the transformer/destroyer

Should I modify the `schema.rb` file manually?

What are some best practices for managing the `schema.rb` file in a team environment?

What happens if the `schema.rb` file is deleted or modified?

Model `app/models/product.rb`

Model `app/models/product_kit.rb`