How We Created a Multi-site Setup for a Vehicle Marketplace: Part 2

This article was originally published on Codica Blog.

Today, we want to provide you with the second part of the article dedicated to creating a multi-site setup for a multi-vendor vehicle Marketplace. Here we cover such challenges as testing and deployment.

If you missed the first part, here is the link for your convenience.

Let's get started and see what difficulties we faced.

Testing

Goal: Provide high-quality and timely testing of 89 websites

Taking into account the fact that the websites’ number was steadily increasing, we understood that the only effective option is automated testing. It is much more time-efficient than manual testing and is easy-to-maintain because we have a single code base.

The first step towards automated testing was creating test cases, as we had no documentation for the website maintenance.

After some time we realized that test cases couldn’t cover all the project needs, and then we decided to take advantage of the following tools and practices.

Repository configured CI

Initially, we had a goal to find a way to track the created tests and their execution. To achieve it, we have appealed to Continuous Integration configured to a GitLab repo that tracks created tests. Using Continuous Integration may become extremely beneficial in case automation is essential in your workflow.

Consequently, it shows us whether a code review was executed positively or negatively that gives us an opportunity to avoid time-consuming tasks and spend much less time on repeated actions.

Ruby Gems

RSpec refers to Behaviour Driven Development for Ruby. It gives an opportunity to track the customer’s behaviour on the website. It is of utmost importance to check what is working and what’s not. Our development does testing many languages and currencies that gives an opportunity to track the marketplace working capacity and check the setup performance.

Capybara is a test framework for web applications. It clearly cooperates with RSpec as its add-on. With the help of this tool, we do integrated tests that stimulate how a real user would interact with an app. We can describe the authorization scenario in several lines, and then the gem provides different handy methods for tests debugging.

Here you can see how Capybara works with RSpec:

describe "the sign-in process", type: :feature do
  before: each do
    User.make(email: '[e-mail]', password: 'password')
  end

  it "signs me in" do
    visit '/sessions/new'
    within("#session") do
      fill_in 'Email', with: '[e-mail]'
      fill_in 'Password', with: 'password'
    end
    click_button 'Sign in'
    expect(page).to have_content 'Success'
  end
End

Using RSpec and Capybara, we can run the system of 120 automated tests per 5 minutes.

Deployment

Challenge: Time-efficient deployment of the multi-site setup

The best time-efficient solution is to create automated deployment scripts. Though the number of sites was increasing, we wanted to advance the deployment process. Let’s check how we achieved this goal.

Step 1. 16 websites: Capistrano

Firstly, we have used Capistrano gem - a framework for building automated deployment scripts. After a while, the platform was growing and reached the number of 13-16 websites. Subsequently, it became pretty time-consuming to deploy more sites at one go and we decided to adopt Mina.

Step 2. 40 websites: Mina

By the time the websites’ number reached 40, each site deployment took 2-3 min that was too much. For your guidance, the whole platform deployment took 2 hours. It pushed us to move to Mina - a deployer and server automation tool.

With the help of this tool, we spent only 40-90 seconds per each site for the deployment. In a meanwhile, the sites quantity grew to 80+. That’s why we decided to parallelize the deployment into several streams to make it time-efficient.

Step 3. 80+ websites: Mina-multideploy

After a while, we realized that we need a parallel deployment tool due to the fact the number of websites was constantly growing. Thus, we built Mina-multideploy for parallel deployment. This tool runs Mina on many servers.

Now, deployment is performed in several streams with the help of Parallel gem. It gives us an opportunity to reach a huge amount of 89 websites simultaneously. Consequently, the deployment of the whole platform takes approximately 10 minutes.

How Mina-multideploy works:

Here you can see the deployment performance (average time) that was reached at each step. An improvement

Tools	Average deployment time
Capistrano Gem	120-180 seconds / site
Mina	40-90 seconds / site
Mina-multideploy	8-12 seconds / site (10-15 min / platform)

Conclusion

Here we presented you the second part of the article covering such challenges of marketplace development like testing and deployment. The third (final) part is coming soon and there we will tell you about adding new websites and monitoring challenges.

Stay tuned and read the full article version here: How We Created a Multi-site Setup for a Vehicle Marketplace.