Why you should deploy on Friday without fear

I have heard a lot of times: “Don’t deploy on Friday!”

Well, I understand why, but I disagree with 99% of the cases.
To understand my point of view, I would like to share some concepts about software.

I am always trying to create quality code and when I start to work on a legacy project or a starting project, usually I ask the team:
How often do you deploy the software?

Many times when you talk about deploy with developers you can see fear on their faces because sometimes deploying an application can be complicated or very delicate.

I have seen many different types of problems with deployment, and now I can share some stories.

This is a story about a legacy project that takes a lot of data from renewable energy systems every minute and generates a monthly report.
This software is used to understand how much energy is generated and how much clients have to pay.

After the introduction, I was starting to ask something about deploy to the team.

Me: “Well, tell me, How often do you deploy the software?”
Developer: “Uhm, usually once a week but not on Friday! You know, don’t deploy on Friday :)”
Me: “Ok, the frequency could be fine, and how do you deploy your software?”
Developer: “Well, we open our FTPClient and we transfer updated and new files into that, and all it’s done!”

In this example, the deployment frequency could be fine if you don’t make a lot of features or fixes, but in my opinion, the method to deploy manually is wrong because you have to know which files you need to upload or enter into the server to restart a process, for example.

You can make an error because you forget to upload a file or the internet connection goes down when you are uploading files (it happens).
So remove manual processes if it’s possible. It’s ideal to remove all manual processes, if possible.

Now we can explore another story, this story is about a financial project where a team of 5 developers created a lot of features.
Those features are deployed into a test server together, and after the customer approves them, they can deploy all the features to production.

Me: “Well, tell me, how often do you deploy the software?”
Developer: “We deploy our features usually once a month because we need approval from the customer”
Me: “Why does the approval only come once a month?”
Developer: “because the customer can review only twice a month and we develop a lot of features and it takes a long time, and sometimes there are some bugs that need to be solved”

In this example, there is a problem with the quantity of work developed, the approval time, and the frequency of deployment.

To solve this problem I can advise customers to check features every week at least and approve only those features.
When there is approval the team can deploy those features and restart the process.

Try to avoid a long time for the approval to limit errors, deployment frequency, and possible errors in production because if you need to check a lot of features the error % increase because you don’t pay the right attention to each feature.

Remember: small and frequent deployment can reduce errors and wasting time.

There are a lot of other examples but with these two we can now understand some deployment problems.

To avoid these problems we can try to deploy more frequently with continuous integration and continuous delivery.

Continuous integration (CI)

Continuous Integration is a development practice where developers merge code into a repository whenever possible, for example, many times a day.
Every merge is verified by an automated build that can try to understand if there is some problem.

Another advantage of the CI is that you have a solid build process where you trust it to understand if your code can be deployed or not in production. You can reduce or remove the time to check manually your features and reduce integration problems.

How to do it:

• every team developer has the source code inside their local machine
• when the developer completes a task or an important piece of feature, he commits to a specific branch for the feature
• CI whenever a push event is emitted, it runs tests, it runs code analysis
• if there is something wrong CI sends a failure message or an email, and the team fixes the problem
• if all is okay the code is ready to be deployed

Continuous delivery (CD)

Continuous Delivery is very close to Continuous Integration and usually, it refers to the practice of releasing every time there is a successful build.
When all automated tests are passed, and every check is okay, it’s time to deploy your application automatically.

It’s a set of capabilities that enables us to integrate features, bug fixes, and configuration changes into production in a safe and quick way.
It’s important to understand that it is a culture and all people inside the project are responsible.

We need to work in small batches to reduce errors and become more efficient in delivering value.
Implementing CD means creating different feedback loops for high-quality software to be delivered to users frequently.

Applying continuous integration and continuous delivery to your process can reduce risks and find bugs more quickly.

https://www.slideshare.net/IzzetMustafaiev/fabric8-cicd

Creating a CD / CI culture and process means improving the quality of your software and the health of your teams.

To release your software more frequently, I have seen that limiting the “Work in progress” (WIP) improves your lead time.

By lead time, I mean the time from the moment that you start a feature, to the moment when that feature is released.

Tools

Usually when we have to create a CD/CI build we try to use external services that can be integrated into our process.
In my opinion, it’s important that every team can use its preferred tools for processes because every project is different, so they can need a different process.

The team can choose which software, check, and steps need to be used and implemented for a specific application because it depends on many things.
It’s not important to find the best choice immediately because after each deploy you can ask your team how to improve the process or if there is something wrong.

Sometimes we create little experiments to try new tools, processes, or other things.
Those experiments are small and in time-boxing to understand whether that improvement can be integrated into our process or not.
This enables teams to explore new solutions and improve the quality of the software.

Don’t be afraid of failure because you are trying to improve something in a little experiment without big consequences.

Example of a common CD/CI process in our project

Usually, we create a repository in GitHub.
Each developer has cloned the repository into his local machine and has coded new features into separate branches.

Every time a developer pushes inside a branch we have configured GitHub actions to run several processes like launch tests, run static code analysis

Example of Github actions configuration:

.github/ci.yml

name: ProjectName

on:
  push:
    branches:
      - '*'

  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Validate composer.json and composer.lock
      run: composer validate

    - name: Cache Composer packages
      id: composer-cache
      uses: actions/cache@v2
      with:
        path: vendor
        key: ${{ runner.os }}-php-${{ hashFiles('**/composer.lock') }}
        restore-keys: |
          ${{ runner.os }}-php-
    - name: Install dependencies
      #if: steps.composer-cache.outputs.cache-hit != 'true'
      run: composer install --prefer-dist --no-progress

    - name: Coding Standard Checks
      run: ./bin/php-cs-fixer fix --dry-run -v

    - name: Static Analysis
      run: ./bin/psalm

    - name: Test
      run: ./bin/phpunit

With this configuration file, every time a developer pushes on a branch it creates an environment with the latest ubuntu, checkouts the project, launches composer install to install dependencies, and after that, launches tools like php-cs-fixer (for coding standard), psalm (for static analysis) and phpunit (for tests).

If all is okay the branch could be merged into the main branch of the project.

When a branch is merged into the main branch we have created another process to automatically deploy the software with AWS in this project.

With AWS there are many systems to deploy your application and we have chosen for our project: CodeDeploy and CodePipeline.

To create an automatic deployment with these tools, you need to configure them on your AWS account and creates two files. For example:

File: appspec.yml is used to manage each deployment as a series of lifecycle event hooks, which are defined in the file

version: 1.0
os: linux
files:
  - source: /
    destination: /var/www/yourProject/
permissions:
  - object: /var/www/yourProject/var/
    owner: data
    group: data
    mode: 755
    type:
      - directory
hooks:
  BeforeInstall:
    - location: scripts/deploy/before-install-production.sh
      timeout: 180
  AfterInstall:
    - location: scripts/deploy/after-install-production.sh
      timeout: 180
  ValidateService:
    - location: scripts/deploy/validate-service.bat    
      timeout: 900 

File: buildspec.yml is a collection of build commands and related settings, in YAML format, that CodeBuild uses to run a build

version: 0.2
phases:
 install:
 pre_build:
   commands:
     - apt-get update -y
     - apt-get install -y software-properties-common
     - add-apt-repository ppa:git-core/ppa -y
     - apt-get update -y
     - apt-get install -y build-essential python-pip git
 build:
   commands:
     - make backend_test_ci
     - make frontend_test_ci

I don’t want to go deep inside AWS processes, I think that I will write another article about that.
I have only shown you some examples to make it explicit that it’s not so difficult to create a CD/CI system with those tools.

Another way to deploy software is to use Kubernetes and deploy the docker images into production, we have used this solution for some projects with success.

All these processes and tools can help you create a solid automated build for your software that you can trust and improve your projects because it enables you to launch tests, static analyses, and other checks asynchronously.
You can improve your project step-by-step and code to create quality software day after day until you can feel comfortable to deploy on Friday without fear because you trust in your build and in your code.

But if you deploy something with an error?

Well, with these tools you can rollback to the previous version in a few moments and try to understand the problem locally if you want.
When you find a build problem (it happens) you should fix it and improve again your build by adding, for example, more checks or other processes and steps to avoid problems.

It’s better to add a check inside a build than to see bugs that appear in many parts of the software.
When you feel confident to deploy every time you want and on Friday also, you can feel less stressed and proud of your build and code.

The deploy frequency is one of the most important things that I try to understand when I analyze a new or legacy projects because it’s a powerful metric that can show you immediately the quality of the software.
In my opinion the deployment process is a mindset and culture that must be embraced and shared by the team.

For these reasons I believe that you should deploy on Friday without fear because if you trust in your code and your build process, it’s very easy to deploy and be calm when new code is going to production.

I can recommend a very interesting book that treats all these and more concepts: Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations.

If you need more help or try to improve or create an automated deploy feel free to contact me or the company where I actually work: Flowing