DEV Community: DevGraph

Evolution of Encrypted Credentials in Rails 6.2

DevGraph — Wed, 27 Oct 2021 10:51:32 +0000

Your blog The concept of encrypted secrets evolved and acquired a better shape with each update of the Rails version. Recently, Rails 6.2 has brought in many such updates to the credentials feature. Let us analyze and discuss encrypted credentials further, how to read them, the advantages, and finally, how to manage a secret key base.

Evolution of Encrypted credentials

Encrypted secrets were introduced with Rails 5.1 in a view to bringing more security to the secrets handled. In this version, these were referred to as ‘secrets’ and were referenced by

The secrets.yml.enc file handles the secrets along with an encryption key.

Handling secrets before Rails 5.1

Before this version of Rails, there were two methods to commit the secrets.

The first method was to store secrets onto the secrets.yml file, read secrets from environment variables, and commit the secrets.yml file to the repository. Though this method was easy to operate, it had a high-security risk as any gem used can dump environment variables. Since the data is open and not encrypted, this can be read by anyone accessing the repository.
The second method was to store all secrets in the secrets.yml file and not commit them to the repository.

Handling secrets in Rails 5.1

By default, from this version of Rails, the secrets were passed as secrets.yml file along with an encryption key. Without this key, the secrets stored in the file will look like some junk characters. To initiate using secrets, the user needs to run:

This would create two files: config/secrets.yml.key and config/secrets.yml.enc. The key file will hold the secret key to decrypt data in the enc file.

Encrypted Credentials in Rails 5.2

An update to the older secret handling, this Rails version removed plain text secrets, and only encrypted credentials were allowed. Credentials were stored in config/credentials.yml.enc, and the key was stored on config/master.key. Thus users could deploy code and credentials together and store all credentials in one place.

Here, multi-environment credentials were handled by specifying explicitly, and the configuration was accessed by mentioning the access_key_id.

Encrypted Multi-environment Credentials in Rails 6.1

The latest update made it to separate credential files for each environment. This built-in feature necessitates separate encryption key for each credential file, thus guaranteeing more security.

A global credential file is enough for multiple environments. And when the environment is passed, two files would be created:

Here’s an example of how it works:

If the environment file is missing or not created, the default credentials.yml.enc file will be used.

Also, the config/credential/prod.yml.enc file would be committed to the repository, whereas the config/credential/prod.key file would not.

Add-ons in Rails 6

It is now possible to store the credentials in a location of your choice. This is explicitly mentioned by running:

config.credentials.content_path and_ _

config.credentials.key_path

Make sure to save the valid key and the credentials to avoid errors while running the code.

Handle local environment credentials using

config/credentials/environment.key and_ _

config/master.key

The below command tells Rails to search the credentials file in path config/credentials/local.yml.enc instead of config/credentials/development.yml.enc

Our latest guide Rails encrypted credentials on 6.2 offer an interesting peek into the Rails credentials.

Advantages of encrypted credentials

The main advantages of encrypted multi-environment credentials are as follows:

Safety: The separate encryption key for each environment makes it safer
Easy deployments: since the variables can be moved along with the code, deployments become easier
A single upload of the key is enough
The solution applies to any of the Ruby on Rails applications

Conclusion

Rails is constantly improving the efficiency and scalability of the framework. With the multi-environment credentials enabled, applications used in multiple platforms and POD find it easier to keep the codes simple and accessible.

Know how to use Engine Yard Kontainers to connect to your database, enabling the Rails credentials for improved security.

To learn more about the older versions of credentials, check out Encrypted Rails Secrets on Rails 5.1

Rewrite Rules in Nginx

DevGraph — Thu, 21 Oct 2021 10:31:14 +0000

By Ritu Chaturvedi

Rewrite rules modify a part or whole of the URL. This is done for two reasons. First, to inform clients about the relocation of resources, and second, to control the flow to Nginx. The two general-purpose methods used widely for rewriting URLs are the return directive and the rewrite directive. Of these, the rewrite directive is more powerful. Let's discuss why it is so, as well as how to rewrite the URLs.

Having a better understanding of NGINX will make it easier to follow this blog.

Return directive

Return is the easiest way to rewrite a URL declared in the server or local machine.

Return in Server:

Suppose your site is migrated to a new domain and all existing URLs are to be redirected here; run the below code to direct any new request to your site.

This directs all requests that hit www.previousdomain.com to www.currentdomain.com. The www.previousomain.com will send out a '301' error as soon as the above code is run, and a new access request is generated. The two variables, $scheme, and $request_uri, get data from the input URL. 'Listen 80' indicates that the block applies to both HTTP and HTTPS requests.

Return in local

If you want to redirect pages in place of a complete domain, you can use the return directive under the location block.

Knowing how to create the Nginx rewrite rules can save a lot of your effort and time.

Rewrite directive

Just like the return directive, the rewrite directive can also act in both server and local. Compared to the return directive, the rewrite directive can handle complex replacements of URLs. The following is the syntax of the rewrite:

The regex is a regular expression that matches against incoming URI.

The replacement_url is the string used to change the requested URI.

The value of the flag decides if any more redirection or processing is necessary.

Static page rewrite

Suppose you want to redirect the page https://example.com/tutorial to https://example.com/new_page. The directive will be:

The line location = /tutorial defines that any identification of the tutorial is to be replaced. The rewrite command says to replace the phrase within notations ^ and $ with 'new_page.html' and then break the command. The notation '?' is termed as a non-greedy modifier, after which the pattern search is stopped.

Dynamic page rewrite

Consider rewriting the URL https://www.sample.com/user.php?id=11 to https://www.sample.com/user/11. Here, the user=11 is to be replaced. By using the static rewrite method, it would require writing the rewrite command 10 times. Instead, let's do it in a single go.

The line location = /user.php asks Nginx to check for the prefix'/user'. As said earlier, the Nginx will search for phrases between the start and end notations as ^ and $ along with the non-greedy '?' modifier. The phrase in our example is a range of users. It is mentioned inside the square brackets as [0-9]+. The back-reference in this expression is noted within the parenthesis and is referred to by the $1 symbol. So, for our example, the rewrite will happen for all users automatically.

One special case under the dynamic reference is the multiple back-references.

Now, we have discussed how to write the rewrite rules for simple and complex URLs.

Understand the detailed working of rewrite rules through some examples handling various scenarios.

Directive Comparison

Let's analyze both directives by comparing them and find out why the rewrite derivative is more powerful.

Return directive

It is simple to use and understand.
It can be used in both server and location contexts.
It explicitly mentions the corrected or updated URL so that the client can use them in the future.
Return directives can include multiple error codes as well.
For codes 301, 302, 303, and 307, the URL parameters define the redirect URL.

return (301 | 302 | 303 | 307) url;

For other codes, the text is to be explicitly mentioned by the user.

return (1xx | 2xx | 4xx | 5xx) ["text"];

For example: return 401 "Access denied because the token is expired or invalid";

This directive can be used in scenarios where the return URL is correct for both server and location block, and rewritten URL is built with Nginx variables.

Rewrite directive

It can accommodate more complex URL modifications where capturing elements without Nginx variables or an update in the elements in the path is required.
It can be used in both server and location contexts.
The rewrite directive can return only code 301 or 302. To accommodate other codes, explicitly add the return directive after the rewrite directive.
It may not send the redirect details to the client.
The Nginx request processing is not halted.

Conclusion

The return and rewrite directives can be used to redirect URLs in both server and location contexts. Though the return directive is much simpler, the rewrite directive is widely used as it can also handle complex modifications/updates to the URLs.

Code Concurrency and Two Easy Fixes

DevGraph — Tue, 12 Oct 2021 07:08:07 +0000

By Ritu Chaturvedi

Code concurrency is a default in any Rails code. A threaded web server will simultaneously serve many HTTP requests, and each will hold its controller instance. Threaded active job adapters and action cable channels also handle multiple requests simultaneously. Even when the global process space is shared, work for each instance is managed separately. For scenarios where the shared space is not altered, this process will run smoothly.

Let us see in detail how this is managed in our Rails applications.

Executor: Executors separate the framework and application codes by wrapping the application code.

The wrapping of code was more complex before Rails 5.0. To protect a code, either separate Rack middleware classes were used or direct wrapping was used. With recent updates in Rails, the executor handles the wrapping in a single step that is easier to understand.

Call the executor to wrap the application code as you invoke it.

For example:

One attractive feature of the executor is its reentrancy.

Two callbacks of the executor are 'to run' and 'to complete.' This enables wrapping code in parts when it is not possible to wrap the code as blocks.

The current thread is moved to the 'running' mode, temporarily blocking the thread. So, the thread will not be accessible by any other request that tries to do so.

Reloader: The Reloader functions similarly to the Executor. The code that is to be protected is wrapped before another request is hit. This is used in scenarios where application code is invoked multiple times by any long-running process. Most often, in Rails, the web requests and Active Jobs are by default wrapped. So the Reloader is rarely used.

A Reloader is employed when there is a need to reload the application. When the requested condition asks for reloading, the Reloader delays the application reload until it is secure. And for scenarios where application reloads are mandated, the Reloader waits until the current block is executed and allows the reload. Thus, the code is protected from errors.

The 'to_run' and 'to_complete' callbacks are used by the Reloader also.

A class unload is involved in the Reloader process. Here, all of the auto-loaded classes are removed and set ready to be further loaded. The application reload is to happen only before or after the class unload, and so, the two additional callbacks of Reloader are 'before_class_unload' and 'after_class_unload'.

Executor and Reloaders are used by the Rails framework components as well. ActionDispatch::Executor and ActionDispatch::Reloaderare included in the default application stack. Whenever there is a code change, the Reloader serves a fresh copy of the application for an HTTP request. Active Job feature also utilizes Reloaders, whereas Action Cable uses Executor. Action Cable also uses the before_class_unload of Reloader to disconnect all the connections.

As discussed, code concurrency is handled by default with the threaded active jobs and action cables features of Rails codes. With recent Rails updates, Executors and Reloaders are also added to improve the code concurrency handling.

Also, check out 7 Ruby Gems to keep in your toolbox.

Sending iOS Push Notifications via APNs

DevGraph — Mon, 04 Oct 2021 11:46:05 +0000

By Ritu Chaturvedi

User engagement is of the highest importance in today's world, no matter what you sell or offer to your clients. And mobile phone notifications play the masterstroke in this aspect. By regular interactions with your clients through push notifications, it is possible to release timely updates and stay connected with them. So, let us discuss setting up an Apple push notification service (APN) with a Node.js application today.

In this blog, we will discuss in detail the APN services, enable and register for remote notifications, device token ids, and APN node package with the help of a sample iOS. Make sure to have a physical iOS device, as push notifications won't work with the simulator. Also, an Apple Developer Program Membership account is a must for creating a Push Notifications Certificate.

Getting to know about the Remote Notification Service

Apple devices use a remote notification service to send and receive notifications in any of the iOS, tvOS, or macOS. The remote notification setup is to be configured in your Apple device with proper device tokens and certificates. Otherwise, there are possibilities that the notification services don't work as expected.

The main component of this remote notification setup is the Apple Push Notification service, also known as APNs. APN in actuality is a collection of services that allows the developer to send the notifications from their server to the targeted iOS devices. The APNs are robust and secure methods to establish a connection between the provider server and individual Apple devices.

APN contains two components, the Gateway component, and the Feedback component, both of which are must-haves.

Gateway component establishes the TLS connection from a provider side which enables the provider server to send messages to the Apple, which will be processed by Apple and then forwarded to the respective device. It is recommended to keep the connection in an 'always-on' mode. The Feedback Component is established only occasionally to identify and remove the devices which no longer receive notifications for specific applications. This is a mandatory component in any Apple device.

To receive and handle the remote notifications, the app you provide must have four components as basics. They are:

Remote connection enabled
A registered Apple Push Notification Service (APN) and device token
Sending device token to the notification provider
Establish support to handle incoming notifications in the device

Once the push notification setup is complete in your apps and the provider server-side, the providers can send notification requests to APNs. they convey the notification payloads to each targeted device. As the notification is received, the payload is delivered to the respective app and user communications are managed. It is great to note that the APNs hold the notifications if the device is powered off and waits till the device turns power on while trying multiple times to deliver the message to the appropriate app in the device. This makes it perfect for the user and the provider not to miss any important communication.

It is good to have a clear and strong understanding of the APNs, direct from Apple’s guides.

The provider is entitled to many responsibilities. Some of the major ones are below:

Receive globally unique, app-specific device tokens and all relevant data pertaining to your app on user’s devices.
Determine when the remote notifications are to be sent to each device.
Building and sending notification requests to APNs.

Only with proper entitlements to talk to APNs, can the app handle the remote notifications. Without these entitlements, the apps are rejected by the App Store. Know the details of entitlements required for your app in the ‘Enable push notification’ section of the Xcode Help page on Apple’s website. Once notification pushes are enabled, app registration is a must every time the app is launched. The process of app registering includes the below steps, for all Apple devices.

Your app asks to be registered with APNs
APNs sends app-specific device tokens
The system delivers the device to your app by calling a method in your app delegate
The device token is shared with the app’s associated provider

A method registerForRemoteNotifications is called at the launch time by your app. This initiates the app object to contact APN for app-specific device tokens. Below code shows how to fetch your device token:

Swift:

Know more about the Remote and Local notificationson Apple’s page.

Thus, the system establishes an encrypted and persistent IP connection between the app and APNS, whenever the app starts on a device. This connection setups and enables the device to receive notifications, thus requires configuration on the Apple development account and necessary certificate creation.

For our servers to communicate with the APNs, a lot of connections and configurations are to be done in Node.js. It is easier to opt for the apn packages with preset code to establish error-free connections. This ready-to-use package is one good choice for the production environment. The package is based on HTTP/2 API and is capable of maximizing the processing and notification delivery. It also collects the not-sent notifications when an error has occurred. The official repository of the apn packagewill gives a better understanding of this topic.

Creating Certificates

One main step in setting up the notification services is the certificate creation, specific to your app and device. This certificate tells the APN that our server has legal permissions to send and receive notifications of our app. That means the SSL certificate establishes a secure connection with the APNs. The certificate for our is created by us and is specific to our app alone.

Proceed with the below steps to create a certificate:

Open the project from Xcode, and select the Capabilities tab from Target.
Enable Push Notification switch and wait for automatic configuration to be completed.
In the developer center, search for our app in our developer account. Tap Edit.
Open the Push Notification section on this screen.
Open ‘Create Certificate’ in the Development SSL Certificate tab
Follow the procedure in the upcoming steps on the screen to create a certificate. The certificate will be ready for download once created successfully.
Download the certificate aps_development.cer.
Select the certificate and private key from the list and export a p12 file. Type in the password and note to remember the newly created password as it will be needed by Apple to send push notifications.

Obtaining device token

In the above section, we saw the general code to fetch device tokens. Let us consider a scenario-specific code that we can further use for Node.js script creation.

Our sample app is titled SimplRTApp. We need to identify the device tokens to push notifications from the sample app. So, to get the device token, add the following methods on our AppDelegate.swift.

View the raw code for the below snippets in the github library.

Also, check the final app code for SimpleRTAppfor further details.

The Node.js Script Creation

We now have a p12 file to establish a secure connection with the APNs and a device token to send push notifications to this device from our SimpleRTApp. All we need now is to create our program on Node.js and test the push notifications.

To create the Node.js script:

Open the terminal app.
Need to install the module that handles APN connections. For that, type npm install apn.
Copy or save the p12 file to the folder same as that of our Node.js script.
Create the below script(available from Github).

In the script, the <device-token> and <p12-password> are replaced by the values specific to our app. The above script does the following to set up a real-time push notification when triggered.

Establish the connection
Instate the provider object
Create Notification and send with the send() method
Finish execution when there is a response for either success or error scenario

With the above script, the push notification setup is complete. Now is the time to test the script. For testing, type node send_push.js in the terminal and wait for the notification to arrive in the mentioned Apple device.

Complete script for the push notification service in check in the GitHub and is available for your reference.

It is a good practice to include a custom action with the notifications. It allows the user to communicate quickly with a proper preset response. Examples for custom actions include the ‘Reply’ or ‘Mark as read’ button in a message notification, ‘Install’ or ‘Snooze’ options in an OS update notification, etc. in our case, one easy and simple custom action would be ‘Retweet’.

Simple edits on the Node.js code shared above will enable these custom actions. Let’s see how to proceed.

While setting the notification object, include below:

Also, to let the iOS app know it is an allowed action with the below code:

Thus, for our AppDelegate.swift, the script would be as follows:

Conclusion

As we come to the end of this blog, we have discussed all the below points in detail

The remote notifications
APNs
Certificate creation
Fetching Device tokens
Node.js code for delivering push notification

Building a Vagrant Box: Setting up your Environment

DevGraph — Mon, 27 Sep 2021 05:50:59 +0000

By Ritu Chaturvedi

If setting up a virtual development environment is your goal, here is a guide on how to utilize the vagrant box and virtual machine for this purpose. Let’s discuss the process in detail with this blog.

Here, we will build a vagrant on top of a virtualization engine, as VirtualBox.

Creating the vagrant box

Download VirtualBox and create a directory. You will also need a base operating system. Let’s use ‘ubuntu/trusty64’ for now. Now, we need to initiate the use of the vagrant directory as ‘vagrant init’ and the operating system name.

So, the code would be:

mkdir vagrant_demo --create directory

cd vagrant_demo --enter into the directory

vagrant init ubuntu/trusty64 --initiate vagrant operating system

A VagrantFile, which explains the configuration of the vagrant environment, is created with that last command. Five specs in this file can be modified to suit your requirements. If no modifications are done, the default setup will be used.

-config.vm.box -- defining vagrant Operating System.

-config.vm.provider -- defining the base (Virtual Machine). With this command, you can also manipulate the number of CPUs used.

-config.vm.network -- defining IP address and ports of application. Rails applications usually default to port 3000. In our case, the host is our computer, and the guest is a virtual machine.

-config.vm.synced.folder -- defining how the guest accesses files in the host. Thus, the project files can be modified on your computer, and then they will be automatically synced to the virtual machine.

-config.vm.provision -- defining the virtual environment setup.

Thus, the final version of our vagrant file is as follows:

Once the vagrant file is created, start and ssh into it to be in a completely active yet isolated OS. Now that the virtual machine and vagrant file are up and running, define and install everything you need for developing your application. In our example, we need to do some installations as below:

install git

install curls

load rvm

install ruby

set default ruby version

verify ruby version

install and check your rails version

install bundler

bundle your gems

install nodejs

install your database.

Make sure to have a properly aligned database.yml file and set the database names in this file. The alignment of this yml file is very important because it can be read wrongly otherwise. Another important factor to consider is the database names. The database names in this yml file should be the database names of your rails application.

Finally, the vagrant environment configuration is complete, and now create your application from scratch.

Creating the application

Creating an application from scratch is done in your local machine and not in a vagrant.

In your local machine, open a new tab in the same directory as your vagrant file.

Now, to create a new application, run:

rails new your_app_name database = postgresql

The application is now created. Configure the database with names and permissions, as discussed above.
Open the terminal where the vagrant is running.
Move into your synced directory, which is “/vagrant_files” in our example.
Move into the new app’s directory.
Run ‘bundle install’.
Run ‘rails s’ and your app should be up and running at the port or URL as per your definition.

Using the existing application

Open a new terminal in the directory of your vagrant file. Bring the code of your existing app from GitHub.

Now, update the database with proper names.

Open the terminal where the vagrant is running.
Move into your synced directory, which is “/vagrant_files” in our example.
Move into the new app’s directory.
Run ‘bundle install’.

Run ‘rails s’ and your app should be up and running at the port or URL as per your definition.

Conclusion

At first, the process of creating and setting up a virtual machine and a vagrant box will be tiring. But once tuned in, there is no going back.

There are options to use existing vagrant boxes and also use packers instead of manually doing all the setup. But either way, it is always better to know the logic and working of any predefined schemes for better understanding and easy troubleshooting. So, try out your versions in an above-discussed manner before creating your app. Once you get into the detailed flow, you can skip the basic steps by utilizing predefined packers and move up the ladder for further challenges.

There are chances to replace virtual machines with Docker containers as well. Want to know more about virtual machines and Dockers? Check out our blog post - Docker Vs Virtual Machines Explained.

To deploy and start your Ruby apps on Engine Yard, start your free trial now.

Credits:

Images source: DEV Community

Rails encrypted credentials on 6.2

DevGraph — Mon, 13 Sep 2021 07:18:16 +0000

By Ritu Chaturvedi

Any rails program would have secrets to be stored, for at least the secret key base with tokens for third-party APIs. Post version updates, handling secrets has become easier.

Initially, there were two methods to handle secrets.

The first method stored secrets in the environment variable (secret_key_base) and committed the secrets.yml file to the repository. This had many safety constraints in place.
The alternate method saved secrets in the secrets.yml file and avoided committing it to the repository.

In the 5.1 version, encrypted secrets were introduced, and were handled by the secrets.yml.enc file along with the encryption key control. The encryption key enabled us to commit the secrets to the repository safely.

With rails 5.2, the plain text credentials became obsolete. Since then, only encrypted credentials were in place and the same were stored and accessed by two files: credential.yml.enc and master.key .

Credentials were stored in config/credentials.yml.enc and the key was stored on config/master.key. This feature enabled deploying code and credentials together and also storing all credentials in one place.

Handling multi-environment credentials before rails 6

Before rails 6, credentials and configurations corresponding to all environments were saved in a one-way file, with the environment as a major key and multi-environment credentials were handled by specifying explicitly.

Development:

aws:

access_key_id: 123

secret_access_key: 345

and the configuration was accessed by mentioning the access_key_id.

Handling multi-environment credentials in rails 6

The Rails 6 version has taken further steps to improve the scalability of the rails framework by including multi-environment credentials. Instead of keeping one credential file to handle the secrets for all environments, separate credential files for each environment and point of deliveries are created. Though this necessitates separate encryption keys per environment, this feature brings more safety and clarity. The in-built feature of multi-environment credentials also facilitates one-way-time uploading of the encryption/decryption key to the server.

With this update, a global credential file is enough for multiple environments. And when the environment is passed, two files were created -

_config/credentials/#{environment}.yml.enc and _

_config/credentials/#{env}.key. _

Let us consider an example. The below command

_rails credentials:edit --environment prod _

would create the credential files for the production environment as

config/credential/prod.yml.enc and

_config/credentials/prod.key _

and if the environment file is missing or not created, the default file credentials.yml.enc file will be used.

Also, the config/credential/prod.yml.enc file would be committed to the repository, and the config/credential/prod.key file would not be committed.

Credentials in rails

Rails understands the credential files to be used in a specific environment. If the environment-specific credentials are defined, this will be considered over the global credentials.

As for the above example,

rails.application.credentials.config {:aws=>{:access_key_id=>"123", :secret_access_key=>"345"} }} rails.application.credentials.aws[:access_key_id]=> "123"

Added features in rails 6

Rails 6 also added a feature to explicitly specify the location where the credential file is stored. Committing to a known path in the repository would make handling these files easier. To save the files in a specific path of your choice, config.credentials.content_path and config.credentials.key_path are used. While using this, one must make sure to upload the valid key as well as credentials to avoid errors/interruptions. If the respective key is not available on the path, the encrypted credentials would remain as a bunch of meaningless characters.

To handle local environment credentials, config/credentials/environment.key and simple config/master.key are to be used. The scenario varies in the production environment. For such a scenario, the rails_master_key can be used and the encryption keys from the .key file are stored here.

The below command tells rails to search the credentials file in path config/credentials/local.yml.enc instead of config/credentials/development.yml.enc

_config.credentials.content_path = ‘config/credentials/local.yml.enc’ _

Conclusion

The Rails family puts constant effort into improving the efficiency and scalability of the framework. With the multi-environment credentials enabled, applications used in multiple platforms and pods find it easier to keep the codes simple and accessible.

You can use Engine Yard to connect to your database enabling the Rails credentials for improved security. Click hereto learn how to proceed.

Tutorial on how to use Active Storage on Rails 6.2

DevGraph — Mon, 13 Sep 2021 07:13:34 +0000

By Ritu Chaturvedi

Understanding Active Storage in Rails 6.2

Active storage is an inbuilt gem in Rails that developers widely use to handle file uploads. Combined with the encrypted credentials feature in the latest releases of Rails, active storage is a safe and easy method to upload, serve, and analyze files onto cloud-based storage services as well as local storage.

To start with, we need to install the active storage gem. This step is followed by declaring attachment associations, uploading attachments, processing attachments, and adding validations.

Active storage gem Installation

With any new application, the first step to enable active storage is to install the gem. Run the below to install this migration to create the three basic tables automatically:

bin/rails active_storage:install

Run the migration by below:

bin/rails db:migrate.

This creates the three tables for your application as active_storage_blobs, active_storage_variant_records, and active_storage_attachments. Out of these three, the active_storage_attachments is a polymorphic join table.

The blobs table holds some straightforward details about the uploaded file like filename and content type. It also stores the encoded key that points towards the uploaded file in the active storage service.
The active_storage_attachments table is a polymorphic join table that holds references to a blob and a record. The polymorphic option is set to true in t.references:record, null: false, polymorphic: true, index: false line when this table was created. This is set this way because the record_id column has details on what type of data it is holding. The identification is made clear with the help of a foreign key and a class name defined in the table.
Usually, active storage stores original copies, but it also lets the user accommodate modifications like resizing file size. The active_storage_variant_records table holds details about all these modified files.

Functionalities of Active Storage

Active storage gem is used to attach, remove, serve, and analyze files.

Attaching files: Files can be attached as a single file or multiple files. Use macros like ‘has_one_attached’ and ‘has_many_attached’ accordingly.

Below are the sample codes to add attachments.

One attachment:

class User < ApplicationRecord

has_one_attached :avatar

End

Many attachments:

class Message < ApplicationRecord

has_many_attached :images

End

Active storage enables attaching files and data to record on storage services. If we expand the ‘has_one_attached’ declaration, we can see that there is an avatar_attachment and an avatar_blob to get through the avatar_attachment association.

Third-party services are used to open, view, and operate the attached files from storage services. The content type of attachment decides the kind of service to be used.

If the avatar_attachment is an image file attachment, here’s how you can upload an image to this model.

<%= f.label :avatar %>

<%= f.file_field :avatar %>

In order to display the uploaded image, run:

<%= image_tag event.avatar %>

It is always advisable to add custom validations to the files uploaded since the Active storage feature does not include in-built validations. File type and the file size must be validated before the upload, to avoid errors and complications.

As discussed above, Rails allows modification of the uploaded files and stores the data in the variants table. For example, to process image files, the image_processing gem of Rails can be used.

Remember to encrypt your storage service credentials before uploading to the cloud. Encrypted credentials are a safe way to handle cloud-based storage services like Amazon S3.

Removing files: The attached files can also be removed from the records by using the purge command.

User. avatar. purge

Serving files: The uploaded files can be served by active storage. Two methods are used for this - the redirecting method and the proxying method. The redirect method uses the file’s blob URL to serve the file, and the proxying method will download data in files from the storage service.

As mentioned, active storage uses third-party software to enable file processing. You can download and install libvipsor ImageMagick v8.6+ for image analysis and transformations, ffmpeg v3.4+ for video/audio analysis and video previews, and poppleror muPDFfor PDF previews separately, as Rails will not install this software.

Major active storage updates in recent releases

In the recent releases of Rails, the active storage gem has seen notable updates. The major ones are as follows:

In ffmpeg, the user can configure all those parameters used for generating a video preview image under config.active_storage.video_preview_arguments.
No error is raised when the mime type is unrecognizable.
When an image previewer cannot generate a preview image, ActiveStorage::PreviewError is raised.
Even with no service selected, Blob creation will not crash.
Like MuPD previewer, Poppler PDF previewer also uses the original document's crop box to display a preview image.
Configure attachments for the service you want to store them in.
Use any URL, private or public, for blobs. The latest Rails update on active storage extends support for this feature and ensures the public URLs are permanent.

Now that you have seen how to install the active storage gem, explore its functionalities, releases, updates, and uploading files, we hope that this blog helped you gain a better understanding of active storage in Rail 6.2.

On what parameters dev.to features stories in its feed?

DevGraph — Thu, 26 Aug 2021 05:29:26 +0000

ARM-Based Cloud Computing: Inexpensive and Fast

DevGraph — Wed, 18 Aug 2021 04:13:39 +0000

By Ritu Chaturvedi

Many of you have heard about home automation with Raspberry Pi or that the latest smartphones are more clever than some desktops. You may have been wondering why tiny computers are not used on an industrial scale outside of the portable gear.

They are. The market for alternatively architectured computers is not restricted to devices for private use. Nowadays, it is possible to equip cloud computing facilities with such machines.

We are talking about ARM technology. You can use it for CI/CD on a corporate level but need to prepare your development routines for this transition.

An Introduction to ARM Machines

A Brief History of ARM

ARM is an acronym that has kept its meaning but changed the underlying abbreviated components. Originally, it stood for Acorn RISC Machine. Acorn Computers Ltd. used to be a British manufacturer of microcomputers founded back in 1978. In 1990, after a few years of cooperative experimental projects with Apple, one section of the company was separated and established as a new firm: Arm Ltd.

Today, ARM stands for Advanced RISC Machines.

Arm Ltd. seldom appears in the news. The reason is that Arm Ltd. mainly focuses on the development of the RISC architecture — we will come to that in a moment — and not on the end-user products. It sells licenses to other companies that manufacture computer processors and sell them to third parties or incorporate them into their own products. Raspberry Pis have ARM cores, as do iPads and a wide range of their tablet competitors.

By the way, the term “ARM core” is applied not only to CPUs manufactured under Arm Ltd.’s license. It can also be a Qualcomm computer chip that was designed independently.

What Is RISC?

RISC stands for reduced instruction set computer. To help you understand RISC and its hidden potential for cloud computing, we will directly introduce its counterpart: a complex instruction set computer called CISC. CISC computers are basically all personal computers and the biggest part of the data center hardware. In other words, the CISC architecture is what we know to be a usual computer.

What Is the Difference?

As we know, a CPU is an electronic circuitry. Once a set of instructions specified in a software program is executed, an oscillating clock signal is issued. When you choose hardware for end-users, you usually look at the number accompanied by MHz, standing for megahertz. Hertz measures the number of clock cycles, which is how often the clock signal was issued within one second. We are accustomed to taking the higher number for the better.

This is not completely wrong. But CISC computers made the race for higher clock speed their main direction for development. The reason is that a CISC processor executes multiple instructions within only one clock cycle by merging them into a single instruction set. On the contrary, RISC processors execute only one instruction per cycle and, therefore, they break complex instructions down into simple instructions. RISC processors impose restrictions on the size of a single instruction but they do not hunt for the highest clock speed.

Here comes the main difference between the two architectures.

RISC architecture aims to use simple hardware and simple software that needs fewer clock cycles to execute. “Fewer” and “simpler” means that you need fewer transistors and, consequently, less electric power to run an ARM-based machine.

Unfortunately, an ARM-based processor needs more memory since simple instructions add more work to compilers. In the early days, manufacturing memory components was difficult and expensive. That’s why the evolution of end-user computers took a different path, and RISC architecture had to wait decades for a new rise.

RISC vs CISC: Further Advantages and Disadvantages

You can run complex applications on simple processors — ARM-based ones — but such a machine will require a larger memory cache.

In addition to that, two disadvantages have evolved historically. They are not per se drawbacks of the ARM technology, but they are rather the impact of the computer industry following a non-RISC path for years.

First, RISC remained a less popular niche for programmers. As a result, nowadays, it is more difficult to find a good developer who can write applications for ARM-based computers. Second, creating an application for RISC architecture means more effort since you need to transform complex instructions into simpler ones. And most of the current software applications are made complex due to their usage with the complex CPUs.

But, in general, RISC architecture means cheaper and smaller hardware components that run faster and consume less energy. Once you’ve managed to adjust the legacy code to the new type of hardware, you can count on a severe cost reduction.

ARM and Cloud Computing

Arm Ltd. has willingly done its homework in the field of cloud computing and came up with two convincing arguments that ARM-based data centers can offer:

Single threading
Linear scalability

Single threading means that a CPU processes one thread at a time. Or that one thread means one CPU, with no shared cores that slow down performance. Since many web applications open one thread per request, and since most of them run in the cloud, ARM-based machines have good chances to replace their CISC rivals.

Scalability was previously a curve-shaped thing: in the beginning, it goes up, but then it slows down and grows in steps. An ARM-based server scales up in a linear manner: the pace remains the same and continues to climb with the permanent speed. You do not need to run complex predictions or prognoses.

Load/Store Architecture

Apart from this, ARM-based machines seem to have overcome the memory problem by integrating the load/store architecture.

During any computing operation, the mainstream CISC architecture directly accesses the memory, takes the data from it, changes it, and stores the result.

RISC architecture separates memory access operations from computing operations, making each calculation a two-step instruction.

Small pieces of an ARM-based processor’s fast temporary memory are called registers. The stored instructions take data from a register and store it in the main one. The load operations do the opposite: access the main memory and move data into a register.

The load/store architecture decreases memory access latency.

The Future of Computing

RISC architecture can change the future of software development. A boom of ARM-based processors would promote applications that are close-to-the-metal and directly access RAM and hardware registers. These applications are programmed in low-level languages, such as the C family, that approximate the commands, making them intermittently understandable to the hardware they run on.

Besides, ARM processors offer a 40% better price and performance. The cost-saving factor makes them particularly attractive.

The Modern Market of ARM Hardware for Cloud Computers

We hope to have made you highly enthusiastic about ARM-based (cloud) computing. By now, you may be asking yourself, “How do I get this magic hardware?” For building your own ARM-based data center, you can consider one of the following options.

Arm Limited Neoverse Family

We will briefly describe the official offer of Arm Ltd. As we mentioned before, it basically sells the blueprints that you need to implement into your hardware design and then manufacture.

Neoverse is the family of systems on a chip (SoC). Contrary to traditional computing, ARM processors do not have motherboards that are coupled with other important components into a circuit board. An SoC is an integrated circuit that already contains all those components: a CPU, storage, input/output ports, and sometimes a GPU.

The ARMv8-A extensions solved the old problem of incompatibility between ARM-based machines and 64-bit applications and operating systems. Besides, the ARMv8-A architecture has enhanced capabilities for performing calculations with integer and float numbers due to Scalable Vector Extensions (SVE). It also allows efficient memory partitioning and monitoring and has various security features.

Neoverse outperforms Intel chips by 40% so far. Combined with less power consumption and smaller sizes that reduce the physical facilities you need to rent, an ARM data center seems to offer a bright future of distributed computing.

Apple M1: Maybe

In November 2020, Apple announced its divorce from Intel and the beginning of a new era. It would replace all traditional chips with SoCs in their Mac machines within the next two years. After less than one year, only three Mac models have undergone the transformation.

But a lot of software developers started to hope for the emergence of M1-based data centers. That would definitely eliminate the second challenge around ARM-based clouds.

The challenge is that you would not only need to take care of the manufacturing and installation of your hardware, but also to optimize your existing applications to make them run in your new ARM-based data center. With the M1 transformation finished, the applications can be developed on ARM machines and then directly deployed into an ARM-based cloud.

The Realistic Option: AWS Graviton Processor

Those who use Amazon Elastic Compute Cloud (Amazon EC2) can see Neoverse cores in action: they are implemented as AWS Graviton2 processors.

With this option, you simply subscribe to Amazon EC2 and use it usually, but configure it to run on AWS Graviton. It is highly recommended for applications that can benefit from smaller but faster cores, including but not restricted to web applications, containerized microservices, eject-transform-load (ETL) pipelines, online games, video encoding applications, and high-performance computing in general.

AWS Graviton-based EC2 instances open the doors for more cost savings. Their power allows you to use open-source databases, such as MySQL, MariaDB, and PostgreSQL. They are known to be memory-intensive but, as any open-source, are free to use. You can deploy them on AWS EC2 and let them run on ARM-based capacities.

AWS Graviton is not limited to memory-consuming technologies. You can use it for burstable general-purpose workloads, as well as compute-intensive workloads including real-time analytics, since, as any ARM, it can scale quickly.

Indeed, if you decide to subscribe to an Amazon pay-as-you-go service, you will need to fit into the Amazon architecture and Amazon pricing model. We are not discouraging you from doing this. But we have a better alternative — or a compromise, depending on your needs. Before we explain it in detail, let us walk you through an important preparation step that lies between you and your dream ARM-based cloud.

No worries, our alternative solution will include this preparation as well. We still want to show you the standard way of doing things.

How to Prepare for Leveraging This Technology

As with any cloud migration, you will need to do some adjustments to your existing applications before you can move them into the cloud. A new ARM-based environment requires even more fundamental changes since we know that the very CPU architecture of ARM-based servers is different.

But we do not mean that those are changes you won’t manage to perform. You need to ensure that your applications will be compatible with the new environment. And you do not need to re-build them completely. Instead, use containerization technology.

Your First Container for ARM: A Short Overview of the Steps

Containers are virtual envelopes for software applications that include the application source code and a minimum set of environmental elements — libraries and selected operating system components — that allow running these applications under any operating system and in the cloud, making them OS-agnostic.

Docker is the most popular tool for creating containers. In 2019, Docker and ARM established a partnership to help their users in moving containerized applications to ARM-enabled platforms. Since then, Docker has implemented a multi-architecture imaging feature. It allows you to build containers that can be deployed on any CPU architecture.

The whole procedure is done with the single “docker build” command, provided that you have installed the Docker. To generate a container that is compatible with RISC architecture, simply use the plug-in Docker Buildx together with the build command and specify which platform you need it for. Buildx must be set as the default builder. It is described in more detail in the Docker documentation.

Every time you build a container, it is stored in a GitHub repository, from which you need to download it, in the GitHub language, to clone the repository to your local machine.

Indeed, when you have many applications or small microservices, it starts to look like a lot of manual work prone to human errors that you cannot allow in an enterprise environment.

Now, let us present the alternative we were talking about before. You can deploy your containerized applications on AWS with a Platform-as-a-Service solution: Engine Yard Kontainers.

Deploy and Manage Your AWS Applications

Engine Yard is more than just a platform. We are a team that can help you to perform a transition to the new ARM-based Amazon EC2 infrastructure. At Engine Yard, we assist you in migrating your applications to containers and then to the ARM-based cloud. We share with you predefined templates and detailed documentation. If there are any questions left, you can always approach our expert team.

Deployments happen with a simple Git push and take only minutes to complete. You can fine-tune regions to deploy and have full control over the memory and your CPU usage.

With Engine Yard’s log aggregation feature, you can monitor your failed applications from a single universal cockpit. We enable you to dig into the failures and anomalies in resource usage and set up notifications to act fast on any troubles.

We can also help you in moving your databases into AWS services. Engine Yard offers automated backup and recovery for any kind of database.

Once the migration is done, you can manage your AWS instances yourself or outsource this task to our team. You can create a clone or copy of your environments to separate production and development stages.

Pricing Advantages: Transparency and Full Control

Although ARM-based cloud computing is generally cheaper, we understand that you still want to have a clear overview of your costs. With Engine Yard, you not only customize your working environments but can also set business rules that allow you to stop worrying about provisioning going out of control. You can scale up your applications without skyrocketing your costs.

Conclusion

In this article, we tried to touch on the long story of two types of CPU architectures — RISC and CISC — and how the potential of the former remained dormant for years but is rapidly unfolding now.

Reduced instruction set computers are back on stage. This is an inspiring trend in cloud computing that promises radical cost reduction and better computing power.

However, the underlying differences between the two architecture types would require you to modify your applications before moving them into an ARM-based cloud. An easier way to prepare your applications is to containerize them. Containerization preserves the core functionalities and makes your apps OS- and CPU-architecture agnostic.

For a secure transition, you may need a deployment management platform that is simple to operate and allows you to monitor your cloud costs.

With Engine Yard, you can move your applications into the ARM-based Amazon cloud and deploy and manage newly built applications with a simple PaaS solution.

To learn more about ARM support and performance benefits, reach out to ARM - Developer support or arm community

The Developers Guide To Scaling Rails Apps

DevGraph — Thu, 12 Aug 2021 06:27:05 +0000

By Ravi Duddukuru

From Airbnb to Zendesk, a ton of really great apps were built using the Ruby programming language and the Rails web framework. Albeit a less popular option than other front-end frameworks such as React, Angular, and Vuejs, Rails still holds substantial merit in modern software development.

Ruby on Rails (RoR) is open-source, well-documented, fiercely maintained, and constantly extended with new Gems — community-created open-source libraries, serving as “shortcuts” for standardized configuration and distribution of Ruby code.

Rails is arguably the biggest RoR gem of them all — a full-stack server-side web application framework that is easy to customize and scale.

What is Rails? A Quick Refresher

Rails was built on the premises of Model-View-Controller (MVC) architecture.

This means each Rails app has three interconnected layers, accountable for a respective set of actions:

Model: A data layer, housing the business logic of the app
Controller: The “brain center”, handling application functions
View: Defines graphical user interfaces (GUIs) and UI performance

In essence, the model layer establishes the required data structure and contains codes needed to process the incoming data in HTML, PDF, XML, RSS, and other formats. The model layer then communicates updates to the View, which updates the GUI. The controller, in turn, interacts both with models and views. For instance, when receiving an update from a view, it notifies the model on how to process it. At the same time, it can update the view too on how to display the result for the user.

(Basic Rails app architecture. Image source: Medium

The underlying MVC architecture lends several important advantages to Rails:

• Parallel development capabilities — one developer can work on View, while others handle the model subsystem. The above also makes Ruby on Rails a popular choice for rapid application development (RAD) methodology.

• Reusable code components — controllers, views, and models can be packaged to share and reuse across several features with relative ease. When done right, this results in cleaner, more readable code, as well as faster development timelines. Also, Ruby on Rails is built on the DRY principle (don’t repeat yourself), prompting more frequent code reuse for monotonous functions.

• Top security — the framework has a host of built-in security-centric features such as protection against SQL injections and XSS attacks, among others. Moreover, there’s plenty of community-shipped gems addressing an array of common and emerging cybersecurity threats.

• Strong scalability potential — there’s a good reason why jumbo-sized web apps such as GitHub, Twitch, and Fiverr are built on Rails. Because it scales well when the overall app architecture and deployment strategy are done right. In fact, one of the oldest Rails apps, Shopify, scales to processing millions of requests per minute (RPM).

In spite of this, many Rails guides still make the arbitrary claim that Rails apps are hard to scale. Are these true? Not entirely, as this post will showcase.

3 Common Problems With Scaling Rails Apps

The legacy lore once told that scaling Rails apps is like passing a camel through the eye of a needle — exasperating and exhausting.

To better understand where these concerns are coming from, let’s first recap what scalability is for web apps.

Scalability indicates the application’s architectural capability to handle more user requests per minute (RPM) in the future.

The keyword here is “architecture”, as your choices of infrastructure configuration, connectivity, and overall layout are determinants to the entire system’s ability to scale. The framework(s) or programming languages you use will only have a marginal (if any) impact on the scalability.

In the case of RoR, developers, in fact, get a slight advantage as the framework promotes clean, modular code that is easy to integrate with more database management systems. Moreover, adding load balancers for processing a higher number of requests is relatively easy too.

Yet, the above doesn’t fully eradicate scaling issues on Rails. Let’s keep it real: any app is hard to scale when the underlying infrastructure is subpar.

Specifically, Ruby scaling issues often pop up due to:

• Poor database querying
• Inefficient indexing
• Lack of logging and monitoring
• Subpar database engine selection
• Sluggish caching
• Overly complex and spaghetti code

Over-Engineered App Architecture

RoR supports multi-threading. This means the Rails framework can handle concurrent processing of different parts of code.

On the one hand, multi-threading is an advance since it enables you to use CPU time wiser and ship high-performance apps.

At the same time, however, the cost of context switching between different threads in highly complex apps can get high. Respectively, performance starts lagging at some point.

How to Cope

By default, Ruby on Rails prioritizes clean, reusable code. Making your Rails app architecture overly complex (think too custom) indeed can lead to performance and scalability issues.

This was the case with Twitter circa 2007.

The team developed a Twitter UI prototype on Rails and then decided to further code the back-end on Rails too. And they decided to build a fully custom, novel back-end from scratch rather than modifying some tested components. Unsurprisingly, their product behaved weirdly and times and scaling it was challenging, as the team admitted in a presentation. They ended up with a ton of issues when partitioning databases because their code was overly complex and bloated.

Image Source: SlideShare

Interestingly, at the same time, another high-traffic Rails web app called Penny Arcade was doing just fine. Why? Because it had no funky overly-custom code, had clearly mapped dependencies, and hailed well with connected databases.

Remember: Ruby supports multi-processing within apps. In some cases, multi-process apps can perform better than multi-thread ones. But the trick with processes is that they consume more memory and have more complex dependencies. If you inadvertently kill a parent process, children processes will not get informed about the termination and thus, turn into sluggish “zombie” processes. This means they’ll keep running and consume resources. So watch out for those!

Suboptimal Database Setup

In the early days, Twitter had intensive write workloads and poorly organized read patterns, which were non-compatible with database sharding.

At present, a lot of Rails developers are still skimming on coding proper database indexes and triple-checking all queries for redundant requests. Slow database queries, lack of caching, and tangled database indexes can throw any good Rails app off the rails (pun intended).

Sometimes, complex database design is also part of deliberate decisions, as was the case with one of our clients, PennyPop. To store app data, the team set up an API request to the Rails application. The app itself then stores the data inside DynamoDB and sends a response back to the app. Instead of ActiveRecord, the team created their own data storage layer to enable communication between the app and DynamoDB.

But the issue they ran into is that DynamoDB has limits on how much information can be stored in one key. This was a technical deal-breaker, but the dev team came up with an interesting workaround — compressing the value of the key to a payload of base64 encoded data. Doing so has allowed the team to exchange bigger records between the app and the database without compromising the user experience or app performance.

Sure, the above operation requires more CPU. But since they are using Engine Yard to help manage and optimize other infrastructure, these costs remain manageable.

How to Cope

Granted, there are many approaches to improving Rails database performance. Deliberate caching and database partitioning (sharding) is one of the common routes as your app grows more complex.

What’s even better is that you have a ton of great solutions for resolving RoR database issues, such as:

• Redis — an open-source in-memory data structure store for Rails apps.

• ActiveRecord — a database querying tool standardizing access to popular databases with built-in caching capabilities.

• Memcached — distributed memory caching system for Ruby on Rails.

The above three tools can help you sufficiently shape up your databases to tolerate extra-high loads.

Moreover, you can:

• Switch to UUIDs over standard IDs for principle keys as your databases grow more complex.

• Try other ORM alternatives to ActiveRecord when your DBs get extra-large. Some good ones include Sequel, DataMapper, and ORM Adapter.

• Use database profiling gems to diagnose and detect speed and performance issues early on. Popular ones are rack-mini-profiler, bullet, rails_panel, etc.

Insufficient Server Bandwidth

The last problem is basic but still pervasive. You can’t accelerate your Rails apps to millions of RPMs if you lack resources.

Granted, with cloud computing, provisioning extra instances is a matter of several clicks. Yet, you still need to understand and account for:

• Specific apps/subsystems requirements for extra resources

• Cloud computing costs (aka the monetary tradeoff for speed)

Ideally, you need tools to constantly scan your systems and identify cases of slow performance, resources under (and over)-provisioning, as well as overall performance benchmarks for different apps.

Not having such is like driving without a speedometer: You rely on a hunch to determine if you are going too slow or deadly fast.

How to Cope

One of the lessons we learned when building and scaling Engine Yard on Kubernetes was that the container platform sets no default resource limits for hosted containers. Respectively, your apps can consume unlimited CPU and memory, which can create “noisy neighbor” situations, where some apps rack up too many resources and drag down the performance of others.

The solution: Orchestrate your containers from the get-go. Use Kubernetes Scheduler to right-size nodes for the pods, limit maximum resource allocation, plus define pod preemption behavior.

Moreover, if you are running containers, always set up your own logging and monitoring since there are no out-of-the-box solutions available. Adding Log Aggregation to Kubernetes provides extra visibility into your apps’ behavior.

In our case, we use:

• Fluent Bit for distributed log collection

• Kibana + Elasticsearch for log analysis

• Prometheus + Grafana for metrics alerting and visualization

To sum up: The key to ensuring scalability is weeding out the lagging modules and optimizing different infrastructure and architecture elements individually for a greater cumulative good.

Scaling Rails Apps: Two Main Approaches

Similar to others, Rails apps scale in two ways — vertically and horizontally.

Both approaches have their merit in respective cases.

Vertical Scaling

Vertical scaling, i.e., provisioning more server resources to an app, can increase the number of RPMs. The baseline premises are the same as for other frameworks. You add extra processors, RAM, etc., until it is technically feasible and makes financial sense. Understandably, vertical scaling is a temp “patch” solution.

Scaling Rails apps vertically makes sense to accommodate linear or predictable growth since cost control will be easy too. Also, vertical scaling is a good option for upgrading database servers. After all, slow databases can be majorly accelerated when placed on better hardware.

Hardware is the obvious limitation to vertical scaling. But even if you are using cloud resources, still scaling Rails apps vertically can be challenging.

For example, if you plan to implement Vertical Pod Autoscaling (VPA) on Kubernetes, it accounts for several limitations.

During our experiments with scaling Ruby apps, we found that:

• VPA is a rather disruptive method since it busts the original pod and then recreates its vertically scaled version. This can cause much havoc.

• You cannot pair VPA with Horizontal Pod Autoscaling.

So it’s best to prioritize horizontal scaling whenever you can.

Horizontal Scaling

Horizontal scaling, i.e., redistributing your workloads across multiple servers, is a more future-proof approach to scaling Rails apps.

In essence, you convert your apps in a three-tier architecture featuring:

• Web server and load balancer for connected apps
• Rails app instances (on-premises or in the cloud)
• Database instances (also local or cloud-based)

The main idea is to distribute loads across different machines to obtain optimal performance equitably.

To effectively reroute Rails processes across server instances, you must select the optimal web server and load balancing solution. Then right-size instances to the newly decoupled workloads.

Load balancing

Load balancers are the key structural element for scale-out architecture. Essentially, they perform a routing function and help optimally distribute incoming traffic across connected instances.

Most cloud computing services come with native software load balancing solutions (think Elastic Load Balancing on AWS). Such solutions also support dynamic host port mapping. This helps establish a seamless pairing between registered web balancers and container instances.

When it comes to Rails apps, the two most common options are using a combo of web servers and app servers (or a fusion service) to ensure optimal performance.

• Web servers transfer the user request to your website and then pass it to a Rails app (if applicable). Essentially, they filter out unnecessary requests for CSS, SSL, or JavaScript components (which the server can handle itself), thus reducing the number of requests to the Rails app to bare essentials.

Examples of Rails web servers: Ngnix and Apache.

• App servers are programs that maintain your app in memory. So that when an incoming request from a web server apps appears, it gets routed straight to the app for handling. Then the response is bounced back to the web server and, subsequently, the user. When paired with a web server in production, such a setup lets you render requests to multiple apps faster.

Examples of app servers for Rails: Unicorn, Puma, Thin, Rainbows.

Finally, there are also “fusion” services such as Passenger App (Phusion Passenger). This service integrates with popular web servers (Ngnix and Apache) and brings in an app server layer — available for standalone and combo use with web servers.

Image Source: Phusion Passenger

Passenger is an excellent choice if you want to roll out unified app server settings for a bunch of apps in one go without fiddling with a separate app server setup for each.

In a nutshell, the main idea behind using web and app servers is to span different rails processes optimally across different instances.

Pro tip: What we found when building our product is that AWS Elastic Load Balancer often doesn’t suffice. A major drawback is that ELB can’t handle multiple vhosts.

In our case, we went on with configuring an NGINX-based load balancer and configured auto-scaling on it to support ELB. As an alternative, you can also try HAProxy.

App Instances

The next step of scale-out architecture is configuring communication between different app instances, where your Rails workloads will be allocated.

App servers (Unicorn, Puma, etc.) help ensure proper communication between web servers and subsequently increase the throughput of requests processed per second. On Rails, you can allocate an app server to handle multiple app instances, which in turn can have separate “worker” processes or threads (depending on which type of app server service you are using).

It’s important, however, to ensure that different app servers can communicate well with the webserver. Rack interface comes in handy here as it helps homogenize communication standards between standalone app servers.

When it comes to configuring the right instances for containers, keep in mind the following:

You have four variables to min/max CPU and min/max memory to regulate pod size
Limit the resources using [minimum requirement + 20%] formula
Use average CPU utilization and average memory utilization as scaling metrics
Mind the timing. Pods and clusters take 4 to 12 minutes to scale up on Kubernetes.

P.S. If you don’t want to do the above guesswork every time you are building a new pod/cluster, Engine Yard comes with a predictive cluster scaling feature, which helps you scale your infrastructure just-in-time without ballooning the costs.

Database Scaling

Transferring databases to a separate server, used by all app instances, is one of the sleekest moves you can do to scale Rails apps.

First of all, this can be a nice exercise in segregating your data and implementing database replication for improving business continuity. Secondly, doing so can reduce the querying time since the request will not have to travel through multiple database instances where different bits of data are stored. Instead, it will go straight to a consolidated repository.

Thus, consider setting up a dedicated MySQL or PostgreSQL server for your relational databases. Then scrub them clean and ensure optimal instance size to save costs.

For example, AWS RDC lets you select among 18 types of database instances and codify fine-grain provisioning. Choosing to host your data in a cheaper cloud region can drive substantial cost savings (up to 40% at times!).

Here’s how on-demand hourly costs differ across AWS regions:

US East (Ohio)

• db.t3.small — $0.034 per hour
• db.t3.xlarge — $0.272 per hour
• db.t3.2xlarge — $0.544 per hour

US West (LA)

• db.t3.small — $0.0408 per hour
• db.t3.xlarge —$0.3264 per hour
• db.t3.2xlarge — $0.6528 per hour

Europe (Frankfurt)

• db.t3.small — $0.04 per hour
• db.t3.large — $0.16 per hour
• db.t3.2xlarge — $0.64 per hour

Asia Pacific (Seoul)

• db.t3.small —$0.052 per hour
• db.t3.large — $0.208 per hour
• db.t3.2xlarge — $0.832 per hour

Another pro tip: opt for reserved instances over on-demand when you can to further slash the hourly costs.

Caching

Database caching implementation is another core step to accelerating your Rails apps, especially when it comes to database performance. Given the fact that RoR comes with a native query caching feature that caches the result set returned by each query, it’s a shame not to profit from this!
Caching can help you speed up those slow queries. But prior to implementing, investigate! Once you’ve found the “offenders”, consider trying out different strategies such as:

• Low-level caching — works best for any type of caching to retrieve database queries.
• Redis cache store — lets you store keys and value pairs up to 512 MB in memory, plus provides native data replication.
• Memcache store — another easy-to-implement in-memory datastore with values limited at 1 MB. Supports multi-thread architecture, unlike Redis.

Ultimately, caching improves data availability and, by proxy, your application’s querying speed and performance.

Database sharding

Lastly, at some point in your database scaling journey, you’ll inevitably face the decision to shard your relational databases.

Data sharding means slicing your DB records horizontally or vertically into smaller chunks (shards) and storing them on a cluster of database nodes. The definitive advantage is that querying should now happen faster since a large database gets split in two and has twice more memory, I/O, and CPU to run.

The tradeoff, however, is that sharding can significantly affect your app’s logic. The scope of each query is now limited to either DB 1 or DB 2 — there’s no commingling. Respectively, when adding new app functions, you need to carefully consider how to access data across shards, how sharing relates to the infrastructure, and what’s the best way to scale out the supporting infrastructure without affecting the app’s logic.

To Conclude: Is There An Easier Solution to Scaling Rails Apps?

Scaling Rails apps is a careful balancing act of ensuring optimal instance allocation, timely resource, provisioning, and careful container orchestration. Keeping tabs on all the relevant metrics across a portfolio of apps and sub-services isn’t an easy task when done manually. And it shouldn’t be.

You can try Engine Yard Kontainers (EYK) — our NoOps PaaS autoscaling services for containerized apps. In essence, we act as your invisible DevOps team. You code your apps and deploy them to EYK, and we take over auto-scaling implementation, container orchestration, and other infrastructure right-sizing tasks from there.

Learn more about Engine Yard.

Move Your Database to the Cloud With Zero Downtime

DevGraph — Thu, 05 Aug 2021 05:33:06 +0000

By Ravi Duddukuru

A downtime-free migration with ScaleArc: decouple your database from the application layer and move it safely to the cloud with our load balancing tool.

The average cost of database downtime has risen from $300,000 to almost $1,000,000 worldwide between 2014 and 2020, say Gartner and Statista, respectively.

Downtimes are one of the main risks that accompany database migrations to the cloud. But with the right tools in your hands, you can eliminate them and arrive safely at the advantages of cloud computing.

Cloud Databases Are Key to Your Business Success

By the end of 2021, 59% of all companies will be using the cloud for their main workloads, according to IDG. This shift is mostly driven by the three main factors:

• The simplicity of the maintenance: you do not need to deal with purchasing and upgrading the hardware.

• Better scalability: if you need more computing power, you can get it in minutes.

• Cost savings: as a result of the previous two, you have fewer personnel costs and far fewer hardware expenditures.

On your way to a cloud database, you should be aware of the obstacles.

Database Downtimes

Database downtimes can be unnerving even for your permanent customers and a decisive disappointment for the new ones. When your application is not showing up-to-date information, users cannot work with it properly. Downtimes may have various reasons, for instance, application errors.

Application Code Change

Applications remain bound to their databases through the SQL queries that run in the background. Once you’ve renamed an entity in the application source code, you have to reflect them in the database or the queries that the application sends over to the database.

Such changes may be quickly overlooked, and even if they are not, their implementation takes time.

Ideally, no adjustment in one layer should ever influence the other, especially during the cloud transition, to prevent database downtimes.

The Solution

Thus, the first step on the way to a seamless database migration to the cloud is decoupling the application from the database. It can be done by creating an additional tier between the database and the application, making them independent from each other and more robust against failures.

But before we proceed with this, let’s take some time to go through the database migration issue in detail and outline possible solutions for the pitfalls that may be waiting for you during the transition period. As a bonus, we’ll show you a shortcut.

Cloud Migration: Database-Related Obstacles

If you decide to move your application and/or the underlying database to the cloud, the whole thing may get tricky. First of all, by moving, we obviously mean creating a copy of the original database and moving the data into the new one.

Basically, there are three widely used scenarios for doing this. Their main differences are the duration of a possible downtime and the number of business risks you have to accept in exchange for reducing the database downtime.

Strategies for Migrating a Database to the Cloud

Offline Copy Migration

As we mentioned before, a database receives many queries that may be updating and reading the same records with a minimum time lag. During the migration, the challenge is to create a complete copy of your data catching the most recent updates.

Offline copy migration solves this issue by simply shutting down the database — and, consequently, the application — for the time a copy has to be generated by means of a simple export and import of the data. Within this downtime, the original database will be disconnected from the application and the new copy connected to it.

In reality, this may mean a few hours of the database and application unavailability. It may work if you migrate an internal application used only by your employees and you do it on a weekend or during the night hours. For commercial applications, however, this may lead to revenue reduction and a poor customer experience.

Master/Read Replica Switch Migration

In this case, you create a read copy of your original database. The master copy gets updated and sends updates to the replica. Once you are ready to turn on the switch, you swap them, granting write access to the former replica.

Indeed, in this scenario, you still have some short downtime. You also run a risk that the original database failed to handle a few last-minute queries and they won’t show up in the new database. This may be acceptable for small businesses, internally-used applications, or applications that demonstrate a clear top and bottom in the amount of traffic during the day.

Master/Master, or Dual-Write, or Online Migration

Contrary to the previous method, you can write data to both databases and “turn off” one of them at any time. The databases need to be synched to keep identical records.

Here, a lot can get lost in transition. You never know which one is the most up-to-date. Thus, the synchronization process requires continuous observation. Together with the complexity of the maintenance, it can lead you to postpone the final switch.

The downtime is close to zero but the risks associated with this strategy are the highest among the three scenarios.

Still, the temptation for a quick switch is high, and there is a solution for this: incremental batch reading. This method adds a new column to each table in the database that identifies which records were already synced and which not. This allows you to at least spot the gaps, eliminating them later with a manual one-time synchronization.

Strategy-Independent Obstacles and Solutions

Apart from the difficult strategic choice, there are other factors that can influence your application downtime and, therefore, your approach to database migration. We gathered the three most common challenges and a few life hacks for dealing with them.

Database Schema Changes

Preparing a database migration is often a good point to do some inventory. For instance, to re-think the way your data is organized in the database and to re-design the latter accordingly. That manifests in the database schema changes.

As a result, your source and target databases have different schemes. The initial setup of a replica, as well as a routine synchronization between the two, will require a schema translation tool. Alternatively, you can use an eject-transform-load (ETL) tool, provided that you selected the first or second scenario.

Data Discrepancies

When you move the entire application to the cloud, you may decide to do some re-factoring that leads to changes in how your application works. Consequently, the application data will change. For instance, instead of using UTC in a time column, you start using your local time format. Again, you’ll need a synchronization and ETL solution to transfer the records correctly.

Other Database Functionality Differs

Indeed, cloud migration means a functional upgrade. For instance, your old database may have lacked one-to-many relationships between tables, or did not allow triggers or materialized views, or was not so good in partitioning.

The new cloud one may have this. The question is how to match the data, especially, if you choose the master/master scenario where you have to perform two-way updates all the time.

In very rare cases, it can be the other way around, and you perform a functional downgrade. For instance, if due to some cost factors you decided to adopt a cloud database that offers only essentials, then you have to cover discrepancies before the final switch happens. For instance, solve the many-to-many issue by adding a few more tables and establishing connections between them, or by getting rid of the old data instead of partitioning the tables.

A Smooth Migration: Managing Your Replicas Successfully

The use case with the master and a replica of a database is not only limited to database migration. You can use the master (the primary database) and one or more replicas, or secondaries or slaves, for other purposes. For instance, you might want to split the traffic between them and write the data only to the master and read the data only from the slaves to enhance your database performance.

When you manage your master and secondary databases in a clever way, this can help you not only to speed up the routine work of your application but also to secure your cloud migration.

The magic cure is a database load balancer. It can be applied to the databases without a master-slave relationship and to all databases that reside on more than one server.

What Is a Database Load Balancer?

Obviously enough, it balances the load of your database. Load balancers belong to middleware. They are placed between applications and database server farms.

With a load balancer, you have a universal endpoint for your application and enable the resource optimization of your database. It accelerates query throughput and lowers latency.

The load balancer distributes the queries more efficiently using one of the following methods:

• Round robin sends queries to the servers in a linear manner: the first query to the first server, the second to the second, and so on. It ensures primitive queue management of the incoming queries.

• Weight-based balancing means that the balancer keeps in mind the actual capacities of each server specified as a proportion of the network traffic and the amount of its current load. When a new query comes in, it re-directs it to the server that still has some free capacity.

• The least connection method only takes into consideration how many connections each server has already established, and re-directs a newly arrived request to the least burdened one.

When you want to migrate your database into the cloud, a load balancer can be used in the following scenario. But, indeed, not every load balancer is made for this. We explain to you how ScaleArc performs this task.

How ScaleArc Works During a Migration

Usually, your database is tightly coupled to the application. Instead, you can make the coupling looser and more flexible by putting ScaleArc between the database and the application, and then proceed with the migration.

Inside ScaleArc, you need to create a cluster or use an existing one for the source and target databases. Then, you can add a read-and-write target database to that cluster. It will be put into standby mode and won’t receive any traffic until the cutover.

ScaleArc will move all the data to the target database and keep synching the new data. Once you are ready to make the target your primary database, switch the connection between ScaleArc and the new database. The old one will still be there but not receiving any traffic. Your application will read and write data from the former replica.

Since ScaleArc remains connected to your application during the whole migration period, this method has no downtime.

A Short Step-by-Step Tutorial

The ScaleArc console has an intuitive cockpit where you can configure your database clusters before migration. We reduced the number of configuration parameters to a sufficient minimum. You need just a couple of minutes to fill in the form and start the migration process.

In the ScaleArc console, do the following to begin your migration:

• Click on “Set Up ScaleArc Cluster”
• Specify the database type
• Enter user credentials to allow ScaleArc to log in into the database
• Specify source and target database server names
• Wait till ScaleArc validates these host names and create a new cluster
• Click “Start Migration”
• Specify roles for each database server
• Enter an email address to receive notifications

While your migration is happening, you will get notified by email about important status changes and the completion of the migration. You can also check the progress manually in the console. You can monitor how many tables have already been written to the new database.

Last but not least, you can schedule the cutover time.

ScaleArc: More Than Just Balancing

With ScaleArc, you have a full grip on your migration risks. A seamless switchover is made possible thanks to the integration with Microsoft SQL Server™ AlwaysOn technology and MySQL™ automatic failover.

As we mentioned before, ScaleArc’s main purpose is routine database load balancing. We would like to highlight a few of its advantages to show you why this can be of great help to you even after you’ve finished your migration.

Load balancing works in the three main directions:

• traffic re-routing
• transaction queueing
• queries throttling

It prevents downtime maintenance, doubles your website performance, and boosts your revenue. ScaleArc stands out among other tools since it offers a few particular features on top of the basic functionality.

Automatic Failover

If you have data-heavy applications, such as webshops, or booking or fintech platforms, you may keep one or a few secondary standby databases that can support the main one when it is down. Outside of the database migration, a switch from the primary to the secondary database is called failover.

ScaleArc offers an automated failover process. It means that not only do you get notified when a database is interrupted, but ScaleArc automatically re-directs the traffic to the secondary databases. The users won’t even notice the difference, and can keep working with your application normally.

Surge Queuing

During a failover, the primary and secondary databases are not synchronized. This leads to a replication lag. Once the failover ends, the primary database gets a lot of new requests, resulting in service outages.

It is important to queue the incoming requests to prevent new troubles. The issue is often addressed by surge queuing, which is a ScaleArc feature.

Splitting Read and Write

Write requests are very important since they can change the returned results of a read request that follows afterward. That’s why it is often better to separate these two types of requests completely and only perform write operations on the primary database, while using the secondary one for the read requests.

In this case, the information gets updated quicker. This feature boosts the overall performance of your application.

Connection Pooling and Multiplexing

ScaleArc will collect inactive connections and store them temporarily in a pool instead of closing them immediately. This works faster than establishing a completely new connection every time.

Multiplexing goes beyond that and allows the re-use of a connection for multiple clients.

Query caching

ScaleArc caches — saves — responses to the most common queries. This allows for quicker delivery of user content since the query does not need to be processed anew when it arrives again. This reduces waiting time for your applications, making them more user-friendly.

Analytics and Logging

Indeed, ScaleArc is a highly transparent tool that allows you to collect data about your database performance and all failovers and downtimes.

At ScaleArc, we have thought about your need for a real-time view of all processes. We provide a live monitor for your database load. You can break down query traffic, handpick problematic queries and candidates for caching, and perform bucketing on your databases.

You have all your log data in one place. You do not have to generate logs for each database and then bring them together in a third-party analytics tool.

Moreover, you can retrieve historical data at any time and create forecasts based on it.

Integrations with RESTful API

If you still want to use your tool for analyzing logs, ScaleArc supports REST API integrations with monitoring and management tools. You can customize your log analytics to catch weak points.

Browse to www.scalearc.com to learn more about ScaleArc Database Migrations!

Don’t pass on PaaS

DevGraph — Thu, 22 Jul 2021 08:58:50 +0000

By Darren Broemmer

Gartner expects that the PaaS market will double in size between 2018 to 2022 growing at a 26.6 percent rate to about $58 billion by 2022. As per IDG, almost two-thirds of organizations today use PaaS. Post Covid-19, we expect this momentum to continue due to the shift toward remote work.

However, the market is highly fragmented. Gartner notes; “As of 2019, the total PaaS market contains more than 360 vendors, offering more than 550 cloud platform services in 21 categories. The market remains short on standardization, established practices, and sustained leadership.” In this scenario, it is very difficult to choose the right PaaS provider.

PaaS: Under the hood
To get an application running in the cloud, you can spend long hours downloading, compiling, installing, configuring, and connecting all sorts of components— and that’s just on a single virtual server instance. Not only is this costly and time consuming, it takes away from time your team could have spent innovating and improving your application.

There’s a better way. Technologies between the infrastructure and your application have evolved — the platform layer — making cloud computing easier. Rather than downloading and building all those platform-level technologies on each server instance and then having to repeat the process as you scale, you can go to a simple Web user interface, click a few options, and have your application automatically deployed to a fully provisioned cluster.

As application usage grows, you can add more capacity using the auto scaling capabilities built into your PaaS. When you need to set up increasingly sophisticated architectures with high availability and disaster recovery, you can do this from the same Web interface.

As all the constituent platform components evolve, they are automatically updated for you with no effort required. That is what Platform-as-a-Service (PaaS) is all about.

A PaaS consists of three main components.

First are the software layers your application runs on — “the stack.” These include the various libraries, frameworks, and services the developer uses to build the application, which are present in the runtime environment. The stack consists of the language interpreter or virtual machine (VM), application framework (e.g. Rails, Lithium), HTTP server, load balancer, caching mechanisms, databases, and container orchestration frameworks. A given PaaS may offer several stack combinations to choose from, such as different stacks for different languages or frameworks. The diagram shows a view of a stack based on Kubernetes and containers.

Second is the deployment machinery that packages and deploys containers provisioned with your application stack. This machinery operates directly from your CI/CD pipeline via a Git push, and it gets out of the way once deployment is complete and your application is up and running.

This machinery is itself code, perhaps a combination of scripts and Web services and may use an off-the-shelf technology such as Puppet or Chef. The way this machinery is architected, the particular parameters it exposes, and the functions it makes available to an overlying graphical user interface (GUI) or command line interface (CLI) are important differentiators between a good PaaS and a bad one.

Third is the user interface and the overall user experience (UX). A particular PaaS may provide Web GUI, a CLI, or both. The ordering of the screens, the choices, the logic of how multiple applications and environments are organized and presented—all these factors are make-or-break for the usability of a given PaaS. The goal is to make it easy to change the things you care about and hide the things you don’t care about. The right trade-offs between simplicity and flexibility, constraint and freedom, and opaqueness and transparency are critical.

When you may need a PaaS
Those unfamiliar with PaaS options may ask, “What’s the true benefit of using a Platform as a Service?” They elaborate by saying, “I can install Ruby (or Node.js, PHP, MySQL, PostgreSQL, etc.), deploy my application, and monitor the systems myself!” This is definitely true. Thousands of companies are doing their own DevOps today, and that pattern works for them.

Where a PaaS can really save you money is when the company doesn’t have the developer resources, in-house expertise, or contractor budget to efficiently manage their production infrastructure.

A PaaS allows a development team of any size to focus on the application instead of the infrastructure, making them more productive and providing more “bang for the buck” with development dollars spent.

Here are 5 basic scenarios when you should consider a PaaS platform to deploy your applications to the cloud.

You Don’t Have In-house DevOps Resources: Setting up platform-level software to run your application is time-consuming and complex. Take a look at this video that compares deploying an application directly to AWS vs using a PaaS. By simplifying, automating, and, in many cases, eliminating the steps associated with setting up the foundation for your application, you can get your application deployed much more quickly in the first place, and you can iterate, adapt, and extend it more rapidly over time. Your developers can focus on development and leave the deployment and management to your PaaS provider.
You Want to Improve Infrastructure Performance: Infrastructure knowledge and expertise is built over time. It's only when you have spent decades deploying applications on Ruby that you have the knowledge to provision the best performing infrastructure stack for Ruby. That's where PaaS providers come in - they have specialized knowledge and expertise about the best database, load balancer, web server, cache etc, to deliver the best infrastructure stack for your application.
You Want to Standardize Infrastructure: With the number of choices available today, it's very difficult to decide what infrastructure choices to make. This has resulted in the creation of bespoke permutations and combinations that work great until vendors’ upgrades require you to keep up with tens or hundreds of configuration changes. Why not let the vendor’s experts make the choices and create standard best-in-class operations that can be managed easily?
You Want to Reduce Infrastructure Costs: The majority of organizations have high levels of idle or underutilized infrastructure. This is because they try to overprovision infrastructure to deliver performance at peak periods. A robust PaaS has inbuilt autoscaling that enables your infrastructure to scale up or down based on demand. This can reduce your infrastructure costs by up to 50 percent over time while providing excellent application performance.
Your Applications Need to Be Managed Round the Clock: A critical consideration for most companies is the type of support an application needs. Is one day of downtime acceptable in your business environment? What about two days? One of the key benefits a good PaaS solution provides is 24x7 monitoring and support so you incur no downtime when there are issues with your application. It's critical here to choose a PaaS partner that provides these benefits.

Reasons to use a PaaS instead of doing it yourself
There are really 3 core benefits investing in a PaaS provides versus doing it yourself.

#1 Increase Agility

Using a PaaS to deploy and run your application enhances your agility and time to market. This is because a PaaS greatly simplifies and accelerates the deployment process. Instead of spending hours or even weeks setting up and configuring a production strength cluster you set it up in a matter of minutes. This improves time to market and helps developers be more productive and focus on what they do best-building apps. You can get your apps to market faster and you can iterate and adapt faster with the same development resources as before.

Eliminating much of the overhead to deploy and manage applications doesn’t just mean you can do certain things faster. It means you don’t have to do certain things at all --. which allows you to be even better at knowing how to do the things that differentiate your business, like building applications with innovative features and exceptional user experiences.

Another challenge of deploying your application on a self-built stack is the sheer number of components that need to be maintained and updated over time. When you need to swap in an update to the app server or the load balancer you may find yourself in a nightmare of reconfiguration. This fear leads many do-it-yourselfers to remain indefinitely on an increasingly outdated stack for fear of rocking the boat. With PaaS, you not only get the best possible stack as of the moment you deploy, you also get a stack that keeps up with you over time, ensuring that your application is always running on the latest and greatest.

#2 Optimize Costs

One of the biggest challenges with provisioning infrastructure to maintain high app performance is knowing how much infrastructure to provision-how many CPUs, RAM, storage etc. that are optimum to guarantee performance at peak periods. Most companies end up over provisioning infrastructure with high levels of idle/underutilized resources. This results in excessive spending on AWS usage, which can be controlled simply by rightsizing the infrastructure. A robust PaaS platform uses intelligent monitoring of application performance in production to rightsize infrastructure. It also autoscales based on demand resulting in higher utilization levels and lower AWS costs.

The other big area where you save is of course the costs of hiring a dedicated person to manage your applications 24x7. The reality here is that such resources are expensive and scarce and you have to ask yourself where the money is better spent. So do the math and see for yourself what's cheaper especially when you factor in the AWS usage savings. All this does not even count the savings in the developer’s time spent on setting up and configuring the application on the cloud as well as ongoing monitoring and management, which can be a 24x7 job for a mission critical app.

There are also less obvious hidden costs, such as the cost of downtime when one of your administrators makes a mistake configuring your application server, and no one can access your Web application for hours. According to a study by Uptime Institute, 70 percent of data center downtime is caused by human error. Consider both the hard costs of downtime, such as lost business and unexpected support costs, and the soft costs, such as idled employees and a tarnished reputation.

#3 Better App Performance

The benefit of economies of scale doesn’t simply stop at getting the same thing for less money. What you actually end up with is something better, for less money. The stack and platform-level technology you would build yourself will almost never be as good as what a top PaaS will provide. Few companies have both the ability to pay and the attractiveness to hire the world’s best platform builders. A PaaS employs specialists who constantly tune, optimize, load-balance, reconfigure, and so on. The result is faster application performance.

The best PaaS vendors embed technologies and techniques in their products to keep availability high enough that they can offer service-level agreements (SLAs) at or above 99.9 percent availability.

One of the key benefits baked into a best in class PaaS platform is autoscaling - scaling up or down based on demand. When building a platform yourself, you basically have three choices: you can optimize for the scale you’re at now, you can optimize for a scale you expect to be at a later date, or you can invest a lot in building your own scaling mechanism. In the first case you risk having to redo your platform and incur downtime when you outgrow your initial set-up. In the second case you will likely waste resources due to overprovisioning. And in the third case, you will likely spend a lot of opportunity cost building something that ends up not nearly as good as what you can get from a PaaS. With a PaaS, on the other hand, you get the benefit of a great scaling mechanism developed by experts over time and in response to the needs of many customers. On top of that, the PaaS scaling mechanism leverages the underlying infrastructure’s elasticity but presents it in an easy-to-use way, abstracting the complexity of the mechanism’s details.

Security showcases another distinct advantage of the PaaS model. With the sheer volume and the diversity of security threats on an upward spiral, protecting against attacks is best left to specialists. A PaaS offering provides continual security updates for individual stack components as they are issued.

Finally, if your application is mission critical and any downtime is unacceptable then outsourcing to a PaaS vendor makes sense. A good PaaS vendor will offer 24x7 support and specialized domain experts who have dealt with hundreds of problems in the same domain as yours. You speak to someone who has access to—may even be sitting next to—some of the leading experts in the community, whether for core Ruby or PHP language stack components or complementary open source projects.

If your core expertise is in developing software not deployment and infrastructure, a PaaS may be the answer to your prayers.

Any PaaS you choose needs to deliver well – not just deliver – in these areas:

• Infrastructure Expertise: Look for a provider with expertise in the language your application is developed in. If you have a PHP application, ensure that they have expertise in PHP infrastructure and can make good platform choices for you.
• Setup & Configuration Time: A good PaaS platform should reduce your setup and configuration time with a preconfigured stack and enable you to push your code directly from Git.
• Horizontal AutoScaling: Look for the ability to scale automatically in response to traffic and usage demands so you don't have to worry about scaling your infrastructure based on planned usage. This will ensure that you don’t need to provision excess capacity and waste valuable dollars on idle infrastructure.
• Support & SLA: If you don't have in house resources then the PaaS provider is your support arm. It's one of the most important choices you have to make so choose a provider with a reputation for robust support. Examine their SLA and response times critically.
• Uptime Guarantee: An uptime guarantee of 99 percent is fairly standard in the business. Ensure that there is no downtime during deployments.
• Redundancy, Failover & Backups: Ensure that the provider takes responsibility for replication, backups, and recovery.as well as keeping your platform up to date with patches and new functionality.
• Security: What kind of security does your PaaS provide? Are you on a private cluster or do you have shared resources? Public clusters are less secure and can be susceptible to noisy neighbor issues where one or more users can hog all the available resources degrading performance.
• Pricing: How well does the pricing scale with usage? One of the common complaints that PaaS users have is the fact that they can become expensive really fast. Ensure that the pricing is scalable and delivers the value vs DIY.

When you should not consider a PaaS
While 80 percent of scenarios lend themselves well to a PaaS there may be some situations where it is important to make all the infrastructure decisions yourself. In these cases a PaaS may not be the best solution for you.

Legacy Systems: PaaS may not be a plug-and-play solution for existing legacy apps and services that are not designed for the cloud. Instead, several customizations and configuration changes may be necessary for legacy systems to work with a PaaS service. The resulting customization can result in a complex IT system that may limit the value of the PaaS investment altogether.
On-premise Integrations: The complexity of connecting the data stored within an onsite data center or off-premise cloud is increased, which may affect which apps and services can be adopted to the PaaS. When not every component of a legacy IT system is built for the cloud, integration with existing services and infrastructure may be a challenge.
Highly Customized Configurations: A robust PaaS platform tends to promote standardized configurations and limit some flexibility. Although this is intended to reduce the operational burden on developers, it may not be appropriate for you if you want to extensively customize your environment.
Enormous Scale of Operations: If your operations are at the scale of a Netflix or AOL, you probably have a large in house team and you may want to make all the infrastructure decisions yourself rather than leave them to a third party.

In summary, PaaS simplifies application deployment and management, improves agility and time to market and reduces deployment and management costs. If you are a software developer who wants to deploy applications to the cloud, and don't have an in-house DevOps team, a PaaS platform may be the answer.

As seen on ITProPortal.com