DEV Community: AnaisUrlichs

Comparison: Code Analysis Tools

AnaisUrlichs — Tue, 09 Feb 2021 16:55:47 +0000

Code analysis tools are essential to gain an overview and understanding of the quality of your code. This post is going to cover the following

While these tools target similar use cases, they differ in their implementation, ease of use, and documentation just to name a few.

This post provides an overview of each tool as well as a detailed comparison to help analyse and decide which tool is best suited for your needs.

Overview

Code analysis tools make it possible to access, group, and merge analysis reports of your application. Test coverage reports used in combination with code analysis tools provide comprehensive insights on the quality of your code.

This allows developers to know their code quality today and improve it tomorrow. Specifically, to

Improve poorly written code;
Develop robust and secure applications;
Receive insights into the code quality over time;
Ensure test coverage is done throughout the codebase.

In the best case, you would want to have all the insights on your code quality in one place. This is made possible by Codacy, Codecov, and Coveralls.

The next sections will compare distinct aspects of each tool with its competitors. We tried all 3 tools with our own project in order to gain a better understanding of how they work. Here is each blog post if you want to see what we did:

Features

Now that we discovered that all three tools have the same goals in mind, let’s take a look at the features each tool pursues and the ways they differ.

Each tool has been evaluated based on the following criteria

Feature	Description	Clear Difference
Pricing	The flexibility of the pricing options and the extent to which those are justified	Coveralls is the only tool of the three that charges per repository rather than per user
Ease of Use	Whether or not it was intuitive to get started with the tool	Coveralls was the easiest to integrate within our project
Documentation	The quality of the overall documentation, whether examples were provided, integration with CI/CD providers, and the clarity of the conveyed information	Codecov had the most comprehensive documentation that left little to no room for ambiguity, making it easy set-up and expand the integration with our project
Dashboard	The functionality of the Dashboard and the way information is displayed.	While the Codacy Dashboard is not the easiest to navigate, it did provide the best data visualisation
Historical Data	Whether it is possible to access historical data, view trends, and analyse code changes	All tools enable this feature in a different format
CI/CD Integration	Ease of automating the process of creating code coverage reports and forwarding those to a CI platform	Codecov did not require extensive custom set-up to integrate with Codefresh

Pricing

Both Codacy and Codecov provide a free version, a pro-version and an on-premises version. In the case of Codecov, all pricing options are detailed on their website. In comparison, Codacy does not clearly state the service provided with the free version. Additionally, both platforms charge per user, who receives access to an organization’s repository.

Coveralls, on the other hand, has more segmented pricing options with 5 different tiers. Additionally, it provides free analysis for open source. The remaining options are priced depending on the number of repositories that receive coverage reports rather than the number of users, who need access to the platform.

Given the information provided by each tool, we would recommend startups and open-source projects to go with Coveralls. While they currently do not provide extensive support to projects, they clearly state their belief in the open-source ethos and provide a flexible pricing model. Additionally, paying Coveralls per repository provides more predictable pricing. Depending on the size of your team and the number of repositories that have to be covered, Codecov will provide the best option for its price. However, if your company would like to receive additional features, it might be worth looking into the enterprise and on-premises solutions provided by Codecov and Codacy since both provide a few more features than Codecov.

Reports and Analysis

All three platforms provide a comprehensive dashboard through which code analysis reports can be accessed. This section will look at the features of each platform before providing a comparison. Note that we uploaded to each platform the same coverage report.

The coverage reports uploaded to each platform were based on a Typescript application. Thus, reports had to be in either of the following formats: json, xml, or txt.

Both Codecov and Coveralls suggest different testing and coverage report frameworks depending on the language used throughout a comprehensive list of examples. In comparison, Codacy provides a list of suggestions that are not directly managed by Codacy. This makes it a bit more difficult to understand the potential and implementation of Codacy. Once we have uploaded our coverage report, we can view the information in the respective Dashboards.

Codecov

The Codecov Dashboard is divided into several sections with a top menu bar. Each coverage report is classified in either of the following categories:

hit indicates that the source code was executed by the test suite.
partial indicates that the source code was not fully executed by the test suite; there are remaining branches that were not executed.
miss indicates that the source code was not executed by the test suite.

This is visible in the files section of the Dashboard in the form of a street-light system.

Coveralls

Coveralls follows a unique style throughout its platform. Similar to Codecov, Coveralls uses the percentage of the project covered by tests to analyze its code quality. Furthermore, with every new code coverage upload, Coveralls makes a separate entry into the recent builds section of the Dashboard. This entry includes the coverage percentage of the repository. While a graph for enhanced visualisation of code coverage improvements would be nice, it is not necessary to understand the coverage change over time.

Codacy

Similar to Codecov, Codacy focuses on enhancing its code analysis by connecting coverage reports with the information recorded on your git repository. This includes issues, pull requests and commits. From the main Dashboard, users can access specific information on the repository, such as file coverage. However, Codacy does not provide you with an option to view coverage reports directly. The pro version allows users to set-up security alerts, which is a feature that is unique to Codacy.

Separately, Codacy provides additional tools such as user management, separate configuration for file support, and setting quality standards.

Comparison

Coveralls was the most intuitive platform of the three. However, this can be considered a positive side-effect of Coveralls focusing on the core features needed to create curated coverage reports rather than providing too many options that might confuse users.
In contrast, both Codecov and Codacy place more focus on the visualisation of the coverage reports that enhance the Dashboard.
An additional feature worth highlighting is the option to compare branches directly within the Codecov UI.
In the case of Codacy, the complex UI used to accommodate all features was a problem. The Codacy UI has two settings taps with similar features. When setting up the integration, it was not clear which API token should be used. More on this in the integration section.
Overall, there is not much to set-up or to customise within the UI, which results in a similar workflow while using each tool.

Historical Trends

Historical Trends are necessary to reason between events, changes done to the code base and the results of the code coverage. Depending on the way historical trends are displayed, it becomes easier for developers to identify good practices that led to higher code quality.

All platforms provide historical trends. While Codacy provides the best visualisation of reports over time as an average, it makes it difficult to access coverage reports directly from within the tool. Looking at Codecov and Coveralls, also the simplified Dashboards makes it possible to reason about changes in code coverage within respective files and repositories.

Summarising, the information provided by Codacy is better when looked at high-level but does not provide an advantage when users want to analyse specific files.

Code Analysis

Using a code coverage solution makes it possible for users to access additional information on the quality of their code. The following screenshots showcase the analysis interface of the file that we ran tests on. It is worth highlighting that Codacy analyses the code quality of all files within the git repository by default. Resulting, also files that are not part of the code coverage report are automatically analysed.

Codecov

Coveralls

Codacy

Note that Codacy has some additional logic that did not allow us to access the js file created from the ts file. Thus, we used the ts file instead to analyse our code quality.

In this category, Codacy clearly provided the most information on each line of code, highlighting the improvements that can be made. However, by doing so, the platform enforces specific practices onto your codebase. For instance, in the case of JavaScript, there is no need to provide semicolons but their tool will flag those as missing and downgrade the code quality. By not flagging distinct coding styles, Codacy could be used to enforce a coding style within the organisation.

Documentation

Your need for good documentation will vary, depending on the way you want to use the tools and access code coverage reports. Sadly, the documentation is often that part of a product that is least taken care of. Good documentation allows users to get started quickly, avoid ambiguity, and optimise the use of the platform over time.

When comparing the documentation by Codecov, Coveralls and Codacy, the following became apparent

Codecov has the most comprehensive documentation since it covers every aspect of the product with great detail.
Coveralls’ documentation was the easiest to navigate given that it focuses on the essential operations that users would want to do.
Both Codecov and Coveralls provide a comprehensive list of examples that users can choose from depending on the programming language and framework used in their application. This was greatly missed within Codacy, which is further detailed in the next section.

Use and Ease of Implementation

The use and ease of implementation is arguably the most important aspect of each platform. Depending on the tool that is added to your existing workflow, you do not want to change your entire processes to accommodate one tool. Thus, it is necessary for code coverage and analysis tools to understand at what point coverage reports are created in the development lifecycle. This will allow the tool to provide seamless integration into the testing and coverage process that a team has already established.

Using Codefresh to connect our code coverage reports to the respective tool makes it possible to automate the process of running tests, creating coverage reports, and uploading those to the code analysis tool. This section will detail our experience integrating each analysis tool within our Codefresh pipeline.

Integrate with each tool

The connection between the tool and your application is made through a repository token. Independent of the tool you are using, you will be provided with a repository token once you grant the platform access to a new git repository. The repository token is set as an environment variable either in your local installation or within your CI/CD platform.

In the case of Coveralls, it was necessary to install a node package and a script was added to the repository. The script can be modified to provide additional information on the application. Further details are provided in the Coveralls specific blog post.

Codecov merely required a specific set-up of the code coverage tool used.

Codacy was a bit special in this regard. Instead of installing a script or package within the application, users have to call a remote script. While it might seem more user friendly at the beginning, it has several disadvantages. Namely:

The script has to be called every time that the code coverage report is pushed to the Codacy account. If the script changes, it might result no longer be compatible with the coverage reports used and fail to upload those.
In case of an error, users do not have insights on what has gone wrong. In our case, Codacy was unable to access the right file within the repository and as a result, did not display coverage resorts. However, the script behaved as if it ran successfully.

Pipeline Creation

One heavily promoted feature within each code coverage tool is the possibility to integrate it directly within your existing CI/CD platform. To evaluate the use of each tool, we set-up a CI/CD pipeline for each tool within Codefresh. The process was highly similar across tools and resulted in the following pipeline for Codecov:

Note that we have access to the live logging through the ‘Output’ section in the Codefresh UI.

This section specifically details the use of integrating either of the code coverage tools into Codefresh pipelines. The first steps of the Codecov pipeline are provided as an example. In Codefresh, each step is its own Docker image. At the beginning of standard pipelines, the git repository is cloned, a container image created and pushed to the user's container registry. These steps were the same for each tool.

version: "1.0"
# Stages can help you organize your steps in stages
stages:
  - "clone"
  - "build"
  - "test"

steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "codefresh-contrib/codecov-sample-app"
    # CF_BRANCH value is auto set when pipeline is triggered
    # Learn more at codefresh.io/docs/docs/codefresh-yaml/variables/
    revision: "${{CF_BRANCH}}"
    git: "github"
    stage: "clone"

  build:
    title: "Building Docker image"
    type: "build"
    image_name: "codefresh-contrib/codecov-sample-app"
    working_directory: "${{clone}}"
    tag: "${{CF_BRANCH_TAG_NORMALIZED}}"
    dockerfile: "Dockerfile"
    stage: "build"

Next, we had to run the tests, create coverage reports of our projects, and forward those to each platform. This is done in its own container image. For our application, we used the node image within each pipeline. The implementation for forwarding coverage reports between our Codefresh pipeline varied between Codecov, Coveralls, and Codacy.

 
test:
    title: "Running test"
    type: "freestyle" # Run any command
    image: "anaisurlichs/codecov-sample-app:master" # The image in which command will be executed
    working_directory: "${clone}" # Running command where code cloned
    commands:
      - "npm run test"
    stage: "test"

upload:
    title: "Pushing to Codecov"
    type: "freestyle" # Run any command
    image: "node:15.2" # The image in which command will be executed
    working_directory: "${clone}" # Running command where code cloned
    commands:
      - "ci_env=`curl -s https://codecov.io/env`"
      - "npm install codecov -g"
      - "codecov -t ${CODECOV_TOKEN} -f ./coverage/clover.xml"
    stage: "test"

Note that it would have been best if each tool would provide their own container image that can be used within CI/CD pipelines instead of using the generic node image. This would have reduced the need for a custom workaround and makes it possible to optimize the process.

Let’s Recap

This post provided an overview and comparison of three different code analysis tools. Namely, Codecov, Coveralls and Codacy. All three tools have the same goal of accessing code coverage reports and generating curated insights over time.

When explored in more detail it becomes visible that all tools have a different focus on their implementation of features. This results in a tradeoff between access to features and ease of use. While Codacy is the most advanced tool of the three, it was more difficult to set-up and to navigate its UI. In contrast, Coveralls focuses on the minimum functionality needed to access and analyse code coverage reports. Comparing the free version of each tool, there is no significant difference between tools.

If you are interested in each tool specifically, we highly recommend you to have a look at our previous blog posts on:

Each tool can be automated through Codefresh; simply by creating a pipeline for your project that integrates with the respective code coverage tool.

Create Your Free Account today and try it out!

#14 DevOps Diary: Learning Go & More Podcasts!

AnaisUrlichs — Sun, 24 Jan 2021 15:39:35 +0000

Welcome beautiful people!

Motivation comes from doing ♾️

The more I learn about the cloud-native space, tools and platforms, the more excited I get about learning more 🎉

On other news, I was finally able to customise my YouTube domain! A BIG shout-out to my subscribers for supporting me along my learning journey! You rock 🙌🏼

Now let's get started with this week's content...

Videos🎥

This week, I will highlight a few YouTube channels that are quite amazing in different ways

Jeff Geerling's YouTube channel where he shows lots of Raspberry Pi videos + tutorials; he also has a specific Kubernetes 101 tutorial. Have a look if you enjoy hands-on learning.
Dave Farley's Channel on Continuous Delivery — Definitely a variety of insights that I would not get in most blog posts or by other YouTubers + he aims to make the content more entertaining by connecting it with current events across industries.

And next, two amazing, growing channels — check those out if you are into podcast style content

The Popcast by Dan is going into its second season, featuring lots of amazing people across the cloud-native space.

View the entire newsletter here

#13 DevOps Diary: Events are back

AnaisUrlichs — Sun, 17 Jan 2021 18:05:07 +0000

Welcome beautiful people!

This week is yet another week full of Kubernetes learning resources. Personally, I started learning Go and I am consistently surprised by the level of support from the community; this is just one example.

Hope you enjoy this week's newsletter 😊

If you have any comments, suggestions, anything you would like to have included in next week's edition, please do let me know.

Top Kubernetes Learning Resources 📖

This is an open source documentation, linking to Kubernetes resources and providing explanations. You can also contribute here.

Videos🎥

This week, I had the chance to spontaneously record an episode on Kubernetes Services with David McKay and Philipp Strube in which I asked all those questions newbies might be wondering about, getting started with Kubernetes. Also, have a look at David McKay's YouTube channel.
If you are new to Kubernetes, or also, if you have already been using Kubernetes for some time, you might want to check out k9s as part of my #100DaysOfKubernetes series. It is a terminal based UI that allows you to easier manage all your cluster resources.

View the full newsletter here!

Code Coverage Reports using Codacy and Codefresh

AnaisUrlichs — Thu, 14 Jan 2021 14:07:06 +0000

Where do you usually track your code coverage? If you are not sure about the answer to this question or you would like to explore other options to the ones that you are currently using, then this post is for you.

Specifically, this post details how you can use Codacy in your Codefresh pipeline to create and send coverage reports of your repository with every pipeline build.

To follow along, make sure to have a Codacy and a Codefresh account. If not, now is the time to set-up a fresh account for free!

Codacy Overview

Codacy is a code review tool that allows for automatic analysis, code coverage tracking, and extensive reports that allow you and your team to improve your code quality over time.

The goals of Codacy is to provide insights on

How is your code quality today and
How is your code quality evolving over time

Additional features include:

User management: Mirroring GitHub permissions
Extensive configuration
Additional integration with other tools, such as notifications
Receive analysis reports per file
Depending on the standards that your team wants to enforce, you can add additional parameters such as coverage thresholds

Codacy has a free version, a pro version, and an on-premises version. The latter two have a free trial, which allows you to test all features over the course of two weeks. You can sign-up via GitHub, Bitbucket, or GitLab.

It supports over 30 different languages. In our example, we'll be using TypeScript. If you would like to follow along with a different language make sure it is supported by Codacy.

When you log into Codacy for the first time, it will ask you to provide access to a repository. At this stage, Codacy will not download any code from your repository but merely access its names. You can then either provide access to selective repositories or your entire Git repository.

After adding a repository, it will automatically update the information recorded about your application in the Dashboard. Codacy’s UI has the following taps:

The Dashboard with an overview of issues and code coverage reports
The Commits, which provides a history of all commits that are associated with this repository
The Files within the repository and their respective issues
The issues that Codacy identified in our repository
Open Pull requests
And lastly, as part of the pro features, users can set-up security alerts.

We will stick for now with the Dashboard tap.

As you might be able to tell, the Dashboard does not have access to code coverage reports by default. The next section will provide a guide on connecting Codacy to your code coverage reports.

Set-up Codacy

To connect Codacy either to our local set-up or to Codefresh, we need an API token. The token can be accessed through the taps in the project. For this, select your project => go to settings => integrations => add integration => select “Project API”. Make sure that you select the API token from here and not your general project settings:

Next, we are going to prepare our project. In our case, we are using a simple Typescript application and uses jest to produce coverage reports.

git clone git@github.com:anais-codefresh/codacy-sample-app.git

# now cd into the cloned repository
cd codacy-sample-app

You should then find yourself in the following project structure:

To run the application, you can use the following commands:

npm install
npm run build

And to test:

npm run test

This should produce the following test coverage report:

Once we have our testing and coverage reports set-up, we can push our changes to our Git repository.

Set-up Codefresh

The previous section showed how we can set-up our node project with code coverage reports. However, that would not be scalable to continue across machines, versions, or our application, and team members. In this section, we will set-up our Codefresh pipeline to push our code coverage reports to Codacy automatically.

To follow this section, please have your Codefresh account ready! And integrate with a container registry e.g. Docker Hub or the GitHub container registry.

Once we enter the Codefresh platform, we want to create a new Pipeline. Make sure to select the repository in which we set-up our coverage reports:

The pipeline will come with a basic codefresh.yml file. If you are following along, make sure to delete everything that is in the current inline YAML. If you are using your existing pipeline, the next section will show you how to modify the YAML to integrate with Codacy. In the end, we will have the following pipeline:

To use Codacy, we will have to modify our pipeline to generate the code coverage reports and then push those to our Codacy account. First, we are going to run our tests inside a node image and then connect to Codacy. Our full pipeline will look as such:

version: "1.0"
# Stages can help you organize your steps in stages
stages:
  - "clone"
  - "build"
  - "test"

steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "codefresh-contrib/codecov-sample-app"
    # CF_BRANCH value is auto set when pipeline is triggered
    # Learn more at codefresh.io/docs/docs/codefresh-yaml/variables/
    revision: "${{CF_BRANCH}}"
    git: "github"
    stage: "clone"

  build:
    title: "Building Docker image"
    type: "build"
    image_name: "codefresh-contrib/codecov-sample-app"
    working_directory: "${{clone}}"
    tag: "${{CF_BRANCH_TAG_NORMALIZED}}"
    dockerfile: "Dockerfile"
    stage: "build"

  tests:
      title: "Running test"
      type: "freestyle"
      working_directory: '${{clone}}'
      arguments:
        image: 'node:15.2'
        commands:
          - "npm install --save-dev jest"
          - "npm run test"
      stage: "test"

  codacy:
        title: "Pushing reports to codacy"
        type: "freestyle"
        working_directory: '${{clone}}'
        arguments:
          image: 'alpine:3.8'
          commands:
            - "export CODACY_PROJECT_TOKEN=${{CODACY_PROJECT_TOKEN}}"
            - "wget -qO - https://coverage.codacy.com/get.sh | sh"
        stage: "test"

Make sure to
Change image_name in the build step to your image name in the following format:
Container-registry/image-name,
Change the tag of the image to fit your tag,
And lastly, replace the image and the commands in the “tests” stage to use the commands of your repository to run tests and create coverage-reports.

The last step, called “codacy” in this case, is responsible for running the Codacy script. To set the env variable, we are going to export our Codacy project token and add the token in the toolbar that you can find on the right of the UI. Once the variable is called through the Codefresh YAML syntax, it will automatically use the value provided within the variables section. Note that you could also provide your token directly in the YAML; however, once we add the pipeline to our git repository, we do not want to worry about accidentally exposing secrets.

Once added, we can save and run our pipeline.

You can follow the execution of the pipeline in the output section. The next time that we commit to our repository, a new coverage report will be generated and pushed to our Codacy account through our Codefresh pipeline.

View the reports in Codacy

Now we can head over to our Codacy account and see the updated data:

Selecting the files tap and a specific file to analyze we can find more information on our code quality:

Let’s Recap

Codacy provides a fast way to integrate code coverage into your Codefresh pipeline. It supports over 30 different languages and an extensive dashboard that also displays information on issues, PRs, and your commit history.

In this tutorial, we set-up coverage reports in our node application, set-up our project in Codacy, and create a Codefresh pipeline with two additional steps.

If you are new to Codefresh, now is the time to set-up a fresh account for free! Also, do join our forum and ask away if you have any questions.

#12 DevOps Diary: Computer Science and K8s resources

AnaisUrlichs — Sun, 10 Jan 2021 16:28:53 +0000

Welcome beautiful people!

I hope you are all doing well. I was really excited to write this week's newsletter and share these amazing resources with you. I hope you like them/find them interesting 😊

Remember, if you discover anything that you would like to share, just message me, I would love to hear from you.

Let's get started ⛳

Top Kubernetes Learning Resources 📖

Kubernetes has several interactive tutorials on their Documentation

Let's stay on the practical resources, these websites provide great example projects to learn Kubernetes:

katacoda: Learn Kubernetes using Interactive Browser-Based Scenarios
Kubernetes by example; by Red Hat OpenShift

Videos🎥

Under the theme "You can learn anything", I am currently doing #100DaysOfKubernetes. If you enjoy this newsletter, chances are high that you would get lots of value from my current and upcoming videos. Have a look 😊

Have a look at the full newsletter here

#11 DevOps Diary: Let's kick off 2021🎉

AnaisUrlichs — Sun, 03 Jan 2021 16:17:07 +0000

Welcome beautiful people!

First of all, congratulations! Congratulations for making it through 2020. It was certainly a tough year for most of us and while we focus on expanding our tech skills and collaborating on tech projects, we shall not forget that we are all human with emotions and human struggles. Let's be kind to each other and support each other throughout 2021.

Happy New Year! 🎉

Top Learning Resources 📖

If you would like to learn more about the fundamentals of Computer Science, this website provides a comprehensive list of high quality resources, including lectures, e-books and more.
Are you currently preparing for your CKS Exam? This exam series by Killer Shell provides practice questions and answers in video format.
Kubernetes Resource thread by Daniele Polencic, sharing his 2020 learning journey.

Videos🎥

Get started contributing to open source projects in 2021. Contributing to open source is a great way to contribute to real world projects and to expand your skills. I would highly recommend the content by Eddie Jaoude to get stated 🙌🏼

You can view the entire newsletter here

#10 DevOps Diary: #100DaysOfKubernetes Let's go

AnaisUrlichs — Sun, 27 Dec 2020 17:07:24 +0000

Hey everyone,

I hope you are doing well and thank you for joining me to week 10 of the DevOps Diary newsletter 🎉

If you are celebrating Christmas, I hope you have/had a lovely time.

Now for the news that I mentioned in my last newsletter, here it comes:

The News 📢

In this week's newsletter, I would like to share with you my new challenge:

✨#100DaysOfKubernetes ✨

My goal is to learn something new about Kubernetes each day. Every day (except weekends), I will be posting a video on my YouTube about the new content that I came across.

I would love for you to join me on this challenge. If you have any suggestions on what I should spend some time learning, please do let me know.

My goal is to inspire others to start exploring topics they are interested in; whether those are related to DevOps or not, there is so much to learn and to explore.

And now to this week's content.

Podcast 🎤

StackRox launched a Kubernetes-focused podcast: ‘Screaming in the Cloud’

The first episode is focused on Eliminating Security Risks in Kubernetes.

Read the full newsletter here

Code Coverage Analysis Using Codecov and Codefresh

AnaisUrlichs — Tue, 22 Dec 2020 16:08:26 +0000

Coveralls is a web service that allows users to track the code coverage of their application over time in order to optimize the effectiveness of their unit tests.

Once you are managing your application and associated resources within a CI/CD platform like Codefresh, you want to receive insights on the test coverage automatically with every pipeline build.

This post provides an overview of how this can be achieved with Coveralls and Codefresh. To make the best out of this post, make sure you have both a Coveralls and a Codefresh account set-up.

Overview of Coveralls

Coveralls has several features that allow you to optimize the coverage of your repositories, analyze trends, and see how changes in specific files have affected the overall project over time.

Some of the advantages of running code coverage reports with Coveralls:

Set it up once and receive additional insight from code coverage reports with every build
Coveralls support 22 different language integrations
Access detailed information on the coverage of every file; Coveralls breaks down the information per file for additional insights
The updates page, integrated within the UI, provides additional information on how individual commits to your project have accepted the code coverage
Add a nice coverage badge to your project read-me to show everyone how well written your code is
Little plus point: The maintainers behind Coveralls show their humour in the documentation (what jokes can you spot?)

Separately, Coveralls has a pretty comprehensive pricing model; whereby open-source projects are free. We have found that coveralls integrates seamlessly with our projects.

Let’s get started

Once signed-up to Coveralls, you will have to provide Coveralls with a git provider to access your repositories. The integration is required to generate repository-specific access tokens that will allow us to push the coverage reports to Coveralls. Ideally, you are using the same git provider in Coveralls as in Codefresh. If you are using the normal account, you can choose between either of the following git providers: GitHub, GitLab, or BitBucket. The enterprise account supports further options.

In this example, we are going to be using Coveralls in this node application. However, you could use your own application, as long as it is supported by Coveralls.

First, we have to make sure that our application is set-up correctly with a testing tool that produces coverage reports. To demonstrate the set-up we will first generate test coverage reports manually in our local environment and then optimize the process with Codefresh so that a coverage report is generated automatically with every pipeline build.

In our case, we are going to use jest. Alternatively, you could use Mocha in combination with several tools; more information is provided in their example. Have a look at their documentation for other languages to get an idea of which testing tools are supported.

Set-up testing and code coverage reports in our node repository

If your Node project does not have a testing tool installed, install jest:

npm install --save-dev jest

Next, we are going to add a file, called jest.config.js to our root project folder with the following content:

module.exports = {
 preset: 'ts-jest',
 testEnvironment: 'node',
};

To complete the configuration, we are going to add the following lines to our package.json file:

"jest": {
   "coverageDirectory": "./coverage/",
   "collectCoverage": true
 },

And add a script to run jest:

"scripts": {
   "test": "jest --coverage",

Now, when you run ’npm run test’, you should see an output similar to the following:

Once our application produces coverage reports, we have to add coveralls to our packages:

npm install coveralls --save-dev

Coveralls requires a script that takes standard input and sends it to coveralls.io to report your code coverage. Depending on the framework that you are using, you will have to add a different script to your application.

Any coverage reports can be forwarded that are within a lcov data format (including mocha's LCOV reporter). For this, we are going to set-up a “bin” folder, and within the folder a coveralls.js file that contains the following content:

#!/usr/bin/env node
 
'use strict';
 
const { handleInput } = require('..');
 
process.stdin.resume();
process.stdin.setEncoding('utf8');
 
let input = '';
 
process.stdin.on('data', chunk => {
 input += chunk;
});
 
process.stdin.on('end', () => {
 handleInput(input, err => {
   if (err) {
     throw err;
   }
 });
});

Lastly, we will have to change our testing script:

"test": "jest --coverage && coveralls < coverage/lcov.info",

Again, for further information on how to set-up Coveralls on your testing framework, please refer to their examples.

Set environment variables to connect to Coveralls

If you don’t have a Coveralls account yet, now is the time to set-one up. Once you are signed into Coveralls, you should either see a list of already connected repositories (like in my case), or an empty dashboard.

One last thing that we have to do before running the coverage report and sending it to Coveralls is to define the environment variables. Those will also become useful once we are using Coveralls in Codefresh.

export COVERALLS_SERVICE_NAME="codefresh"
export COVERALLS_GIT_BRANCH="insert the branch that you will be using with your application"
export COVERALLS_REPO_TOKEN="insert the secret repo token from coveralls.io"

Like so:

Connect your repository in Coveralls and receive a secret repository token

To create the secret repository token, head over to your Coveralls account and add the current repository:

Once you turn on any of the repositories (box on the left side next to the repository), you will enable tracking of code coverage reports. It is then that you will see the secret repository token that has to be used within the environment variable above.

Connect your local instance with Coveralls

If you have been following along, go ahead and call the tests:

npm run test

Now refresh the Coveralls site of your repository to see the Code Coverage reports.

To unlock the full benefits of using Coveralls and to submit coverage reports automatically with every build, we are now setting up our Codefresh pipeline to do the work for us.

Using Coveralls in your Pipeline

You know the drill, if you don’t have a Codefresh account yet, let’s go ahead and create your free one. Once signed in, you should be able to create a project and within the project, we will now create our pipeline.

Once you ’create’ the pipeline, a standard codefresh.yml file is generated with three steps:

The first step will clone your repository;
The second step will both, build and push your repository to the container registry that you have connected with Codefresh;
And the third step currently does not do much.

Thus, let’s go ahead and modify it.


  test:
    title: "Running test"
    type: "freestyle" # Run any command
    image: "node:15.2" # The image in which command will be executed
    working_directory: "${{clone}}" # Running command where code cloned
    commands:
      - "export COVERALLS_SERVICE_NAME=${{COVERALLS_SERVICE_NAME}}"
      - "export COVERALLS_GIT_BRANCH=${{CF_BRANCH}}"
      - "export COVERALLS_REPO_TOKEN=${{COVERALLS_REPO_TOKEN}}"
      - "npm install --save-dev jest"
      - "npm run test"
    stage: "test"

For reference, you can find the entire pipeline in the git repository.

As you can see, we specify several variables within this step. Those, which start with ’CF’ are Codefresh-specific steps and the value is automatically provided by Codefresh once you run the pipeline.

However, we have to tell Codefresh Coveralls specific variables. You could place the values directly within the step; however, since the secret repository access token should not get accidentally pushed to your public git repository, we will provide the variables within the variables section.

Let’s go ahead and run the pipeline. You can view the steps and commands executing within the output section of the pipeline.

Moving over to Coveralls, we can now see our code coverage report.

Let’s Recap

In this post, we

First saw an introduction to Coveralls, its features, and use cases;
Then, we prepared our git repository;
We set up our Coveralls and our Codefresh Account;
Set-up our Codefresh pipeline;
And lastly, ran our tests within Codefresh and pushed the coverage reports to Coveralls!

To connect Codefresh to Coveralls through your pipeline Create Your Free Codefresh Account today!

As a bonus, we are going to add our Coveralls code coverage button to our README

To do so, you can access various formats directly from the Coveralls UI, within your repository:

Adding it to our README, and tadaa:

#9 DevOps Diary: Tips and Tricks

AnaisUrlichs — Fri, 18 Dec 2020 14:16:54 +0000

Welcome beautiful people!

Can you believe that this is already week 9! I have something special prepared for next week, so make sure to stay tuned for that.

In the meantime, we will look at more Kubernetes resources and some great video tutorials and talks that I came across within the past days.

Videos🎥

Red Hat Says Farewell to CentOS
How to install Nginx Ingress Controller in Helm 3
Kelsey, Kubernetes, and GitOps - GitHub Universe 2020 — Kelsey Hightower's presentation at GitHub Universe
5 Simple Tips for Troubleshooting Your Kubernetes Pods

Blog Posts (and other written content)🗞️

Get started with Chaos engineering; specifically with Litmus
A lot of times we learn about concepts and tools so fast, that we miss out on the basics — here is a nice article on Kubernetes namespaces

View the entire newsletter here

#8 DevOps Diary: No more Docker?

AnaisUrlichs — Sun, 13 Dec 2020 15:48:57 +0000

Hey everyone, I hope you have/are having a lovely day and Welcome to week 8 🥳

This week's newsletter is out later than usual since life kind of got in the way. I hope you enjoy it anyway and you are having a great Sunday.

News 📣

Kubernetes is dropping Docker support — don't freak out. Unless you have a self-managed cluster, it likely does not affect your use of Kubernetes nor your Kubernetes workflow.

What does this mean ❓

Video Explanation by TechWorld with Nana "Kubernetes is dropping Docker support - What does it mean for YOU?"
Let's have a look at some of the alternatives that you can use to Docker — Docker images without Docker

Blog Posts (and other written content)🗞️

Adding to #7 of this newsletter; some amazing people have started collections of free resources on GitHub to help you prepare for the CNCF Certified Kubernetes Security Specialist - CKS (or just to learn cool stuff about security)
ConfigMaps in Kubernetes: how they work and what you should remember

View the full newsletter on my Blog

Using the DigitalOcean Container Registry with Codefresh

AnaisUrlichs — Wed, 09 Dec 2020 10:58:41 +0000

DigitalOcean has just announced the DigitalOcean Container Registry, which is directly integrated into the DigitalOcean Dashboard. While you can use any Container Registry in your DigitalOcean resources, the goal of the Container Registry is to provide for easy connection between the Container Registry and DigitalOcean Kubernetes resources.

DigitalOcean Container Registry Dashboard

Whether you have already integrated DigitalOcean cloud resources into your Codefresh pipelines or not, you could use the DigitalOcean Container Registry like any other container registry in your Codefresh Pipeline.

Overview

The DigitalOcean Container Registry provides:

Three subscription plans that offer different allowances for repositories, storage, and bandwidth
1-click method to configure DigitalOcean Kubernetes Cluster to use the registry
Garbage collection using the command line or the API
Ability to see the current storage usage in the control panel
Automatic global load balancing across caching endpoints in multiple geographic regions
Ability to store any artifact that is compatible with OCI-based registries, such as Helm charts

Furthermore, the DigitalOcean Container Registry is not just for containers. It allows users to store any artifacts that are compatible with the OCI specifications. This allows you to keep Helm Charts in your DigitalOcean Container Registry (note that Helm’s OCI support is still experimental).

Building and pushing a Container image with DigitalOcean and Codefresh

The set-up will only take 4 steps, assuming that you have a

DigitalOcean Account
Codefresh Account -- If not Create Your Free Account today!

First, we have to set-up our Container Registry in DigitalOcean. This can be done by opening your account and accessing the “Container Registry” section:

Since we are already in DigitalOcean, we can go ahead and create our access token. Open API in the bottom left of the Account page (below Settings). And create a new token at Tokens/Keys. The token needs to have both read and write permissions. Make sure to take note of the token since we will need it in the next step.

Now we can add the DigitalOcean Container registry to our Docker registries in Codefresh. For this, open:

Codefresh Account => Account Settings (bottom left) => Docker Registries => Integrations => Docker Registries => Add Registry Provider => Other Registries

This should get you to the following screen. As you can see, Codefresh supports a variety of Container Registries:

Next, we have to provide the configuration information for the DigitalOcean Container Registry. For this, you will need our DigitalOcean Access Token. The token is used both as username and as password.

The URL is going to be registry.digitalocean.com -- you can then provide the registry name in the advanced options (alternatively, you could include it in your build step before the image name).

That’s it! Now we can push images from our Codefresh Pipeline directly to the DigitalOcean Container Registry. Note that if DigitalOcean is not set as your default registry, you have to define the registry in your build step:

registry: "digital-ocean"

Our full build step will look as follow:

  build:
    title: "Building Docker image"
    type: "build"
    image_name: "anais-codefresh/react-article-display-do-registry"
    tags:
      - "1.0.0"
    dockerfile: "Dockerfile"
    registry: "digital-ocean"

Different to other platforms, the Codefresh Build Step both builds AND pushes the image to the registry.

Running the image, we can see the step execute in the output section of our pipeline:

And view the Container image directly in DigitalOcean

Try it out TODAY!

We made it super easy for you to connect DigitalOcean Container Registry to your Codefresh pipeline. You can find more details in the Codefresh documentation.

Wait, you are not a Codefresh user yet? Create Your Free Account today!

This process works on all Codefresh accounts. You can set it up in minutes to try it out.

If you have any questions, feel free to ask in our community forum, we would love to hear from you.

#7 DevOps Diary: DevSecOps and how to make the best of learning DevOps in 2021

AnaisUrlichs — Wed, 02 Dec 2020 17:32:05 +0000

Hey everyone, I hope you have/are having a lovely day and Welcome to week 7 🥳

For the better or worse, we have reached December and the year is soon going to be over. What are your goals in the last month? Maybe reaching some last-minute yearly goals of 2020?

Let me give you some inspiration in this week's newsletter for next year.

Inspiration: DevOps 2021, something to look forward to 🎉

Make a list of conferences that you would like to attend (most of them are likely online or hybrid for most of 2021)
Join me in #100DaysOfKubernetes
What is something that you always wanted to learn and were never able to take the time for? Maybe the resources in this newsletter provide some inspiration
Collaborate with other amazing people in the space; whether on blog posts or videos — I tried and about half of the folks I contacted were quite excited about the idea 🙌🏽

Videos 🎥

The cd Foundation has an amazing webinar series going on that provide interesting insights into tools, best practices, and more
Get started learning more about security with this introduction to Snyk

Blog Posts (and other written content)🗞️

Staying with the topic on security,

Snyk also has a great learning hub to get started with DevSecOps and a comprehensive list of security-focused resources
IBM's X-Force Threat Intelligence Research Hub with tons of research papers
Found on Reddit, DevSecOps University has a comprehensive list of resources to get started with — however, the more advanced content is highly focused on AWS. I highly enjoyed this Cloud Security Architecture Workshop by Dave Shackleford
Monthly "Getting into DevOps" thread on Reddit
And lastly some tips on how to evaluate Kubernetes monitoring solutions