DEV Community

Cover image for Universal Artifact Management for Go development community
Gowtham Neerukonda
Gowtham Neerukonda

Posted on

Universal Artifact Management for Go development community

Need for DevOps acceleration

The need for adaptive software applications increases exponentially in the current landscape. We see a spike in movement from traditional CI/CD tools to the advanced DevOps platforms with COVID-19 bringing more urgency to adapt to the market, financial, consumer spending habit changes, etc.

 As software applications compete to adapt to the ever changing needs of consumers, the leader will be the one that can deliver these changes in time. The same logic applies to the other end of the leaderboard where there are significant delays to develop, test, automate and deliver software applications, ultimately leading to failure.

Tools that can help you accelerate Go development

For the Go development community, the recent introduction of Go modules streamlines the process of consuming Go dependencies. GOPROXY also provides more control to the sources of these dependencies and assures that builds are deterministic and secure. A private GOPROXY is one you install to store both public and private Go modules on your own infrastructure. This will ensure that your Go artifacts are secure, local, immutable and always available to readily serve you.

Also, more tools are available free of cost by various community champions to help enable easier integration, usage and development. GoCenter (search.gocenter.io), a free central public Go module repository offered by JFrog helps developers to resolve their Go  dependency modules from a single source. It also secures third party dependencies by providing information on security vulnerabilities found in various modules.

Alt Text

Software development and security practices do not stop at securing third party dependencies. Organizations rely on more than one programming language for software development, which introduces the need for a universal artifact management solution. The process of implementing open source security checks must be embedded into your CI/CD pipelines and build systems, rather than being a checkbox scan before deploying your artifacts into production.

This is where a free JFrog Cloud subscription of the JFrog DevOps Platform will be helpful. The free subscription includes Artifactory, a central repository to store all your artifacts including private Go modules that are proprietary to your organization. Artifactory supports unlimited repositories, unlimited users and 27+ packages including Go, PyPi, RPM, Debian, Docker, NPM, etc. The free subscription also includes Xray, an integrated scanning tool that can help you scan your private Go modules for vulnerabilities that you might be inheriting through open source dependencies. You can sign up for a free JFrog Cloud subscription, hosted on the major public cloud provider you select.

To help ease the problem of fixing vulnerabilities at the last minute, which is a tedious and time taking process, developers can utilize JFrog's VScode extension,  which provides live vulnerability information about every public Go module. The sooner you address a given security issue, the easier it will be and consumes less time.

How JFrog DevOps Platform fits into your CI/CD pipelines?

When introducing any solution into your DevOps toolset, the first things you need to worry about are integration and automation. JFrog provides various tools to easily onboard your organization and integrate with your build systems.

Artifactory integrates with various CI/CD systems and captures information about the the artifact i.e., 

  • Inception - dependencies utilized, system and environment variables, build name and number 

  • Production Readiness - any specific properties that users have tagged the artifact with (unit, integration, system and performance testing status) or you can have different repositories for each maturity level to store artifacts 

  • Statistical metadata - total number of downloads,  uploaded by, date added and  modified, etc..

The beauty of the solution is that it associates these data points to the actual artifact all along the stages of development, integration, system testing, staging, and production storing them with the artifact. Each stage in your DevOps environment will utilize the data points published by the previous stage, essentially building up a system of record which release engineers can utilize as a reference to determine the status of a given artifact or build.

This metadata stored with artifacts gives the ability to qualify/disqualify the artifact to move to the next stages of maturity, perform precise cleanup of artifacts, pickup the right release candidates, etc. You can also automate this process by utilizing the build promotion API. Artifactory supports easy integration with CI/CD tools and build systems.

For Go development, Artifactory can act as a GOPROXY that serves both private and public Go modules. Using Artifactory's remote repositories, you can cache a public GOPROXY along with private modules from their VCS repos. Artifactory ensures immutability and availability for both public and private Go modules. Artifactory's local repositories can help you publish Go modules.

Alt Text

Automating artifact lifecycle management using the JFrog DevOps Platform

When working on creating an application using DevOps best practices, as much as it is important to plan, code, build, test, release, deploy, operate and monitor, it is also important to maintain and manage lifecycle information associated with the application and its artifacts. This provides full visibility into your SDLC workflows and increases your speed of DevOps cycles, ultimately leading to your organization's success.

Alt Text

The Go client might not allow you to publish Go modules to Artifactory. The JFrog CLI, a lightweight command-line utility tool supports publishing Go modules to Artifactory. The JFrog CLI can generate go.mod, and generate build information for your Go builds.

Continuous vulnerability scanning for your Go applications

Scanning capabilities exist as a part of the JFrog DevOps Platform, performing a continuous and deep recursive scan on all the Artifacts stored in the Artifactory. It can perform powerful DevSecOps operations like failing a given build and blocking the download of an Artifact using a completely granular and customizable policy engine. You can also ensure that only approved sets of Artifacts are deployed to production.

Alt Text

The vulnerability data is used to generate a unique impact analysis graph that identifies which artifact layers are affected by a given vulnerability.

You can integrate JFrog Xray vulnerability information for IDEs, including Eclipse, IntelliJ, and Visual Studio, to flag risky dependency versions as you code.

Conclusion

Onboarding and adopting these free tools for Go development takes a lot of time and effort. Here, we’ve identified some of the integration points where you can introduce automation and the benefits you can achieve at each instance. It always helps to evaluate the pain points in your workflows and see where you can improvise before actually introducing the solution to get the best results in development and delivery time.

Discussion (0)