It's late at night, after a week of struggle I managed to configure my deployment properly. The purpose of this short article is to show you some problems that I came across with dependencies in my Ruby app during deployment with Codedeploy, and solutions to them.
Prerequisites: Basic knowledge of AWS Codedeploy configuration (If you went through the documentation you will do just fine).
From the beginning...
I have my Ruby app. It is using Bundler to install gem dependencies. I put it on AWS EC2 and it works just great.
Note: On my EC2 I am using Amazon Linux AMI 2018.03.0 (HVM), SSD Volume Type, since it comes with Ruby and some CLI tools installed out of the box.
There is just one thing missing to make me happy - I would like to deploy my application when a particular branch is updated on my bitbucket repository.
Let's assume I have VM running and codedeploy-agent installed on it.
I recommend this tutorial to set up deployment pipelines for your bitbucket repository.
There is one thing not covered in the tutorial though: installing app dependencies with Bundler.
The problem #1
So what's the big deal?
I can use gem install bundler and then add bundle install in my scripts ran in my AppSpec deployment hooks, right?
| version: 0.0 | |
| os: linux | |
| files: | |
| - source: app.rb | |
| destination: /home/ec2-user/ | |
| - source: Gemfile | |
| destination: /home/ec2-user/ | |
| - source: Gemfile.lock | |
| destination: /home/ec2-user/ | |
| hooks: | |
| BeforeInstall: | |
| - location: scripts/install_dependencies | |
| timeout: 300 | |
| runas: root | |
| AfterInstall: | |
| - location: scripts/bundle_install | |
| timeout: 300 | |
| runas: root | |
| # That's only an example. In the real app, you probably will want to add ApplicationStop and ApplicationStart hooks |
| #!/bin/bash | |
| # That's the location of my app | |
| cd /home/ec2-user | |
| bundle install --path vendor/bundle |
| #!/bin/bash | |
| cd /home/ec2-user | |
| yum update -y | |
| # Installing ruby version 2.4 | |
| yum install ruby24 -y | |
| # Setting ruby version 2.4 as default | |
| alternatives --set ruby /usr/bin/ruby2.4 | |
| gem2.4 install bundler |
| # This is just to show the gems that I had issues with | |
| source 'https://rubygems.org' | |
| gem 'dotenv' | |
| gem 'json' | |
| gem 'mail' | |
| gem 'pry' |
(App code is not really important here)
But I get an error when I deploy my code:
[2019-07-07 10:01:40.378] [d-0N2XZ4E6A]LifecycleEvent - AfterInstall
[2019-07-07 10:01:40.379] [d-0N2XZ4E6A]Script - scripts/bundle_install
[2019-07-07 10:01:40.391] [d-0N2XZ4E6A][stderr]/opt/codedeploy-agent/deployment-root/d0556b49-b646-4579-9d8e-20fca9397a2e/d-0N2XZ4E6A/deployment-archive/scripts/bundle_install: line 4: bundle: command not found
[2019-07-07 10:03:58.992] [d-8Y4VW7E6A]LifecycleEvent - ApplicationStop
[2019-07-07 10:03:58.992] [d-8Y4VW7E6A]Script - scripts/stop_server
It looks like bundle binary is not found.
Solution
So let's just install the binary somewhere in the environment PATH:
| #!/bin/bash | |
| cd /home/ec2-user | |
| yum update -y | |
| # Installing ruby version 2.4 | |
| yum install ruby24 -y | |
| # Setting ruby version 2.4 as default | |
| alternatives --set ruby /usr/bin/ruby2.4 | |
| gem2.4 install bundler --bindir /usr/bin |
Problem #2
All right, so will it work now?
2019-07-07 18:00:20 [stdout]Fetching json 2.2.0
2019-07-07 18:00:20 [stdout]Installing json 2.2.0 with native extensions
2019-07-07 18:00:20 [stderr]Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
2019-07-07 18:00:20 [stderr]
2019-07-07 18:00:20 [stderr]current directory:
2019-07-07 18:00:20 [stderr]/home/ec2-user/vendor/bundle/ruby/2.4/gems/json-2.2.0/ext/json/ext/generator
2019-07-07 18:00:20 [stderr]/usr/bin/ruby2.4 -r ./siteconf20190707-26032-sw7ot0.rb extconf.rb
2019-07-07 18:00:20 [stderr]mkmf.rb can't find header files for ruby at /usr/share/ruby/include/ruby.h
2019-07-07 18:00:20 [stderr]
2019-07-07 18:00:20 [stderr]extconf failed, exit code 1
2019-07-07 18:00:20 [stderr]
2019-07-07 18:00:20 [stderr]Gem files will remain installed in
2019-07-07 18:00:20 [stderr]/home/ec2-user/vendor/bundle/ruby/2.4/gems/json-2.2.0 for inspection.
2019-07-07 18:00:20 [stderr]Results logged to
2019-07-07 18:00:20 [stderr]/home/ec2-user/vendor/bundle/ruby/2.4/extensions/x86_64-linux/2.4/json-2.2.0/gem_make.out
2019-07-07 18:00:20 [stderr]
2019-07-07 18:00:20 [stderr]An error occurred while installing json (2.2.0), and Bundler cannot continue.
2019-07-07 18:00:20 [stderr]Make sure that `gem install json -v '2.2.0' --source 'https://rubygems.org/'`
2019-07-07 18:00:20 [stderr]succeeds before bundling.
2019-07-07 18:00:20 [stderr]
2019-07-07 18:00:20 [stderr]In Gemfile:
2019-07-07 18:00:20 [stderr] json
Not really. Funny thing is that some gems got installed, while some did not - in my case it's json gem that cannot be build.
Solution
I needed to install another two things to make it work - ruby development kit and c++ garbage collector
| #!/bin/bash | |
| cd /home/ec2-user | |
| yum update -y | |
| # Installing ruby version 2.4 | |
| yum install ruby24 -y | |
| # Installing ruby development kit (needed to build gems) | |
| yum install ruby24-devel.x86_64 -y | |
| # Installing c++ garbage collector (needed to build gems) | |
| yum install gcc-c++ -y | |
| # Setting ruby version 2.4 as default | |
| alternatives --set ruby /usr/bin/ruby2.4 | |
| gem2.4 install bundler --bindir /usr/bin |
Why putting it all in hooks scripts?
Good question. Why not just connect to EC2 instance and install all this stuff by hand?
Because it will not work. I have not found a precise cause, but I have found out that the environment under which deployment is made (at least before copying/replacing files of the working app) is different than the one we have access to when connecting to our VM.
I hope you found the above information useful. If you did, please like, share or leave a comment.
That's my first post, so any constructive feedback will be appreciated :)
Top comments (1)
Marcin, thanks-
I'm fighting through some version upgrades, a lot of stale gems, etc. and I never considered actually setting the ruby version in the deploy scripts. We use rbenv instead of yum, but the principle is the same and it saved me a lot of time. Thanks again!