Every software development cycle involves three major phases-building, testing, and deployment. A lag in any of these three phases will result in a delay in the launch of the product. In order to avoid such delays, organizations rely on CI/CD tools to automate these processes. But lately, with rapid demand for CI/CD tools there’s a proliferation of choices, and choosing the right tool can be a daunting task!
Two popular CI/CD tools, which you should certainly keep an eye on are, Jenkins and TeamCity, each with their own set of unique features. These two CI/CD tools are widely used to automatically push frequent code changes into the main branch whilst performing DevOps testing at every step to ensure that there are fewer errors in the build.
In this article on TeamCity vs.Jenkins, I make an attempt to help you choose the right CI/CD tools for your needs. Before getting started with the TeamCity vs. Jenkins comparison, I’ll start with the basics of CI/CD. In case you are already aware of it, you can directly head to TeamCity vs. Jenkins section.
What Is CI/CD & Why Is It Important?
In Continuous Integration (CI), changes are integrated into a shared repository on a frequent basis, where regular builds and DevOps testing (an optional step) is performed for early bug detection, thereby improving the product quality. You can get a faster and stable build when you combine it with continuous delivery.
Continuous Delivery (CD) happens only after the completion of CI and consists of an automatic pipeline where code changes can be deployed from one staging environment to another. It includes all the necessary steps that are required for making the software deployable (i.e. quality check using testing tools, software-signing, deployment to pre-production environments, and more). This makes the deployment easier and faster, along with minimization of spending on bug fixes and enhancements.
CI/CD is often used with Selenium test automation for testing builds as they are pushed in a new environment or integrated together, this is referred to as continuous testing or DevOps testing. CI/CD when used in conjunction with TDD (Test Driven Development) can be a powerful combination, as it can be used to identify business-related bugs in the product.
In our previous blogs, we’ve covered the best practices of CI/CD pipeline where we discussed how you can achieve a faster go-to-market launch with the help of CI/CD tools best practices.
As I mentioned earlier, there are several tools available, some of the popular CI/CD tools are:
- Jenkins
- TeamCity
- Travis CI
- Circle CI
- Bamboo
- GitLab CI
- AWS CodePipeline
- Azure Pipelines
With so many CI/CD tools available in the market, it becomes a challenge to select the one that meets the needs of your project. If you are on the lookout for the best CI/CD tools, you have landed at the right place, as here we do a TeamCity vs Jenkins comparison.
LambdaTest, a cloud-based cross browser testing platform offers integration with popular CI/CD tools such as Jenkins, TeamCity, Travis CI, and more so that you can build a robust delivery pipeline to accelerate the go-to-market launch.
Introduction to Jenkins
Jenkins is one of the widely used open-source CI/CD tools available. It was started as a side project by one of the engineers in Sun and gradually evolved as one of the most popular open-source CI tools used for automation of deployments.
It is based on Java and can automate tasks related to build, test, deploy, and delivery of software. Jenkins is available for use on Windows, macOS, and different Unix versions such as Red Hat, Ubuntu, OpenSUSE, and more. Along with installation using native installation packages, it can be installed as a Docker or as a standalone on any machine that has JRE (Java Runtime Environment) installed.
The Jenkins project produces two release lines – Weekly and LTS (Long Term Support). Jenkins is highly extensible and a rich array of plugins, also installation is easy. With Jenkins, developers can focus on their core activities, as a major part of integration and testing is managed by the tool.
At the time of this article, the latest version of Jenkins for LTS was 2.235.1 and weekly was 2.242 respectively.
Mobile emulators online from LambdaTest allows you to seamlessly test your mobile applications, websites,and web apps on mobile browsers and mobile devices.
Introduction to TeamCity
TeamCity is a Java-based CI/CD tool, created by JetBrains, the producer of other useful tools such as PyCharm, IntelliJ Idea, RubyMine, ReShaper, and more. Small teams can use TeamCity for free.
The product’s slogan is ‘Powerful Continuous Integration Out of the Box’ and it stands true to its promise as TeamCity offers source control, build chain tools, and detailed build history – features are not available in a free tool like Jenkins.
It can be installed on Windows and Linux servers. It also provides support for the .Net framework and can be integrated into IDEs such as Visual Studio and Eclipse. At the time of this article, the latest version of TeamCity was 2020.1. Conditional build steps are supported by this version, you can also launch build agents in a Kubernetes cluster. You also get the option to integrate with popular project management tools such as Azure DevOps and Jira.
Let’s take a deep dive into the detailed compassion of the popular CI/CD tools – TeamCity vs Jenkins.
TeamCity vs. Jenkins – Detailed Comparison
Below is the Google Trends report (past 12 months) of TeamCity vs. Jenkins, which indicates that Jenkins is more popular in comparison to TeamCity.
Here are major parameters for performing the TeamCity vs. Jenkins comparison.
1. Installation & Configuration
Installing and configuring the TeamCity server is easy, as it only involves downloading the appropriate TeamCity server installation and performing the install (or upgrade) instructions. The extensive documentation on TeamCity’s official site makes the task easier.
Jenkins being a self-contained Java program is easy to install and can be run out-of-the-box for OS X, Windows, and Unix-based OS. There are three installation steps if Java and Apache TomCat are already installed. On the whole, the process to set up Jenkins is easy. The configuration of Jenkins is performed using a web interface that includes built-in help and on-the-fly error checks.
On the whole, both Jenkins and TeamCity are fairly simple to install, if you follow the necessary steps for installation.
2. Extensibility & Customization
Both Jenkins and TeamCity offer RESTful API for extensibility. With TeamCity, you get several ways for customization, interaction, and extending your server. There are options to perform interaction via RESTful APIs, using service messages in build scripts, and creating plugins for TeamCity using Open API. The level of extensibility would also depend on the type of TeamCity being used. Further details about extensibility and customization for TeamCity is available here.
The remote-access API of Jenkins comes in three flavors – XML, Python, and JSON with JSONP support. The APIs in Jenkins is used to get information to trigger a new build to create or copy jobs. The page on the Jenkins website has detailed documentation on Jenkins APIs that can help in extensibility.
Through this usability testing tutorial, you will learn how usability testing is a great way to discover unexpected bugs, find what is unnecessary or unused before going any further, and have unbiased opinions from an outsider.
3. Plugin Ecosystem
The plugin ecosystem of Jenkins is much more mature in comparison to TeamCity. The primary reason is the community involvement in the development of Jenkins. TeamCity has around 400 plugins (393 to be more precise) crowd developed plugins. These plugins are available for download on the plugin page for TeamCity. The plugins have to be installed separately, as they are necessarily not a part of the commercial product.
With the use of Open API, developers can create plugins for integration with version control systems, build tools, IDEs, notification programs, and server health reports. You can check the official video from TeamCity that helps in the installation of plugins.
In comparison, Jenkins thrives on the community and its rich plugin ecosystem. At the time of this article, Jenkins offers 1500+ plugins which are community supported and supports build, deployment, and automation in a project. As plugins are ranging from build tools to language-specific development tools, it makes customization tasks simple and cost-effective since you would not require costly in-house customization.
4. Cloud & Key Integrations
TeamCity offers integrations with popular cloud solutions via plugins and non-bundled plugins. It takes advantage of cloud computing by dynamically scaling its build agents on popular cloud platforms.
Few examples of cloud integrations via plugins are Amazon EC2, VMWare vSphere, and Kubernetes (supported in version 2020.1). The non-bundled plugins for cloud integration are Windows Azure, Google Cloud, Digital Ocean Cloud, and more. There is detailed documentation for developers interested in the implementation of cloud support in TeamCity. Detailed information about TeamCity integration with cloud (IAAS) solutions is available here.
Apart from cloud integrations, it supports key integrations with Docker, Maven, Visual Studio Team Services, NuGet, VCS Hosting Services, and more.
Jenkins can also be integrated with popular cloud platforms such as Amazon EC2, VMWare vSphere, Google Cloud, Atlassian Cloud, etc. using plugins. Detailed information about the installation and configuration of Jenkins for popular cloud platforms is below:
5. Community Support
Both TeamCity and Jenkins have an active community forum. The advantage with Jenkins is that it is in development and usage for a much longer time than TeamCity. The active and growing community is one of the major plus-points of Jenkins.
General Topics and Plugin Development are the two sections of the community in TeamCity. Community helps in knowledge sharing and problem sharing. Along with the community forum, TeamCity also has an active TeamCity blog where the TeamCity team provides information related to product and feature updates.
In the TeamCity vs. Jenkins comparison, Jenkins has an upper edge over TeamCity as far as the community is concerned. As Jenkins is an open-source project, users of Jenkins rely on the community for most of the requirements (i.e. installation, troubleshooting, and more). Some of the popular ways of engaging with the Jenkins project and community are experience sharing, coding, translation, documentation, review, design, etc.
TeamCity and Jenkins have a powerful set of features that makes the product ideal for CI/CD.
Technology Awareness is a key selling point of TeamCity. It is dubbed as ‘Intelligent CI Server’ due to the approach used for integration. With the support for VS projects, you get support for testing frameworks, code coverage, automatic detection of tool versions, static code analysis, and more All this support is provided out-of-the-box and does not require modification in build scripts or additional plugin installation.
With access to ReShaper & IntelliJ IDEA code analysis & inspections for Java and .NET code, it helps to ensure better code quality.
Due to the availability of plugins, Jenkins is not only used for building code but analyzing code as well. There is support for almost all version control systems and build environments. Many collaborative initiatives are implemented under the Jenkins project that aid in the expansion of Jenkins.
7. Ease Of Use
In comparison to Jenkins, TeamCity provides a better and clear interface. The interface can be easily customized as per the requirements. This does not mean that Jenkins is not usable, the main usability difference being that Jenkins is focused more on functionality than usability (i.e. functionality comes before appearance).
The primary component of TeamCity is its the server, and the browser-hosted interface is used to administer projects, agents, and project configurations.
The experience might differ if your team is well versed with Jenkins and its plugin ecosystem. If your team is on the lookout for a more friendly option (i.e. easy to configure and straightforward to use), you should give TeamCity a brief look.
8. Hosting
TeamCity is available only in the on-premise version. If you intend to setup TeamCity as on a self-hosted server, you can do the same with a reverse proxy (Nginx). Instead of going through these additional configurations for hosted TeamCity, it would be better to try out Jenkins.
Jenkins is available in the cloud-based and on-premise favors. Jenkins is suited for installation in the cloud to run self-hosted pipelines. The detailed article on architecting for scale using Jenkins is a good reference point for leveraging the advantage of on-premise CI/CD with Jenkins.
9. Release Cycle
TeamCity and Jenkins are equally good when it comes to releasing cycles. TeamCity 2020.1.1 was released on June 23, 2020. The changelog is well documented and you have the option to download or fallback on the previous releases. The entire changelog for TeamCity releases is available here.
Jenkins is also good with releases. It follows the LTS (Long-Term Support) release concept that is also used in Ubuntu. The latest stable release of Jenkins is 2.235.1 that was released on June 17, 2020. There is a separate LTS changelog and weekly changelog.
10. Distributed Run (or Execution)
Both TeamCity and Jenkins can run distributed i.e. tasks run through a different machine without causing an impact on the GUI (Graphical User Interface).
It is important to note that only Jenkins can use the same instance on which it is running GUI related tasks. This may result in poor performances with Jenkins.
11. Parallelism
Parallelism for TeamCity vs. Jenkins comparison means that tasks can be concurrently run on the same machine, whereas distributed indicates that tasks can be scaled on different machines.
You can run parallel builds both on TeamCity and Jenkins for different builds and environments. Jenkins does this through parallel stages, a feature introduced in the product a few years back.
Parallelization works to the extent until the tools that run the builds support it. TeamCity does not restrict tools that support parallelism from using it. However, users of TeamCity have reported here & here that enabling parallelization in TeamCity is not as easy as it is in Jenkins. The feature request for implementation of a feature to run several build steps in parallel with a single build is open for the last ten years!
On the other hand, parallelization in Jenkins is just about enabling the Parallel Test Executor plugin. A user of TeamCity has also cited Jenkins to be more feature-friendly when it comes to parallelism.
Jenkins supports parallel builds but they share the same environment, which can cause issues with shared resources. On the whole, Jenkins wins hands down for parallel builds(or parallelization) when it comes to TeamCity vs. Jenkins comparison.
12. Open Source or Commercial
Jenkins is open-source (covered under the MIT License) and free to use. Since it is free, many smaller organizations prefer Jenkins to TeamCity. The only costs involved would be in running the infrastructure. The extensive range of plugins and online community are the key pillars of Jenkins.
There are two versions of the TeamCity (Professional Server License and Build Agent License) and you can choose the appropriate one as per your requirements. Here is the gist of pricing structure of TeamCity:
- Free for 100 build configurations and 3 build agents.
- Additional build agent licenses cost $299 and include one additional build agent.
There are additional options – a steep 50 percent discount for startups and free for open-source projects. Further details about TeamCity pricing are available here.
If the price is a factor, you should choose Jenkins over TeamCity, as Jenkins is open-source and free to use.
At LambdaTest, we offer integration with popular CI/CD tools for you DevOps testing efforts, to help you build a robust delivery pipeline for a faster go-to-market launch.
Black Box testing? Don’t worry; we will be covering what is Black box testing, merits, demerits, types & techniques.
TeamCity vs. Jenkins Comparison Snapshot
Here is the snapshot of TeamCity vs. Jenkins comparison
| Feature | TeamCity | Jenkins |
|---|---|---|
| Product Type | On-Premise | Self-hosted/On-Premise |
| Setup and Installation | Easy | Easy |
| Ease of use | User-friendliness available with out-of-the-box usability | Comparatively less user friendly as the focus is on functionality than appearance |
| Official Support | Yes | No official support but extensive support on IRC, Jenkins Forum, and other support channels. |
| Plugin ecosystem | Not so rich plugin ecosystem | Thrives on plugins (close to 1500+ plugins currently available) |
| Parallelization | Yes | Yes (partial) |
| Reporting | Yes | Yes |
| Build Pipelines | Yes, Allows defining pipelines using a Kotlin-based DSL (Domain Specific Language) | Yes, Support for custom pipelines through Jenkins Pipeline DSL. |
| Integration | Yes, Amazon EC2, VMWare vSphere, Google Cloud, etc.Key Integrations: Docker, Maven, Visual Studio Team Services, NuGet, VCS Hosting Services | Yes, Amazon EC2, VMWare vSphere, Google Cloud, Atlassian Cloud, Slack, etc. through Jenkins Plugins |
| Online Documentation | Yes, Blogs, Support Forum, Youtube channel, and more. | Yes, Blogs, Support Forum, IRC, events, and more. |
| Open Source or Commercial | Free for 100 build configuration | Free (open-source) |
There is not much difference in terms of feature comparison between the two products and you need to make a conscious call of what product type suits the project. Do you intend to go with an open-source CI/CD tool like Jenkins that is feature-rich but less friendly to use? Do you want to go with a premium alternative like TeamCity that is also feature-rich and user-friendly? The answer should be circled back with your expectations (from tool per se) and overall project requirements (including the budget).
This smoke testing tutorial covers what smoke testing is, its importance, benefits, and how to perform it with real-time examples.
All In All
I had a detailed look at two popular CI/CD tools- Jenkins and TeamCity. The selection of the right CI/CD tool is pivotal for a faster go-to-market launch. Also lately DevOps testing is playing a crucial role in order to ensure the quality of your build. For DevOps testing of web applications, you can opt for Selenium to perform automated browser testing & by using a cloud Selenium Grid, you can also scale your testing efforts. It is as important as choosing the right test automation framework to accelerate your testing activity.
I hope you find this article on TeamCity vs Jenkins informative, feel free to share your views in the comment section below. Also, remember to share this article with your peers! That’s all for now. Happy Testing!!!😊
Top comments (0)