DEV Community: Argonaut

Cloud Credits: A Guide for Startups To Maximize Benefits and Avoid Pitfalls

Argonaut — Tue, 25 Jul 2023 09:38:54 +0000

Scaling your startup using the cloud is easier than ever before. With the availability of cloud credits, startup founders can cover their cloud operational costs for up to two years, choosing their preferred provider. The startup programs offered by the leading cloud providers have played a crucial role in helping numerous startups experience exponential growth and deliver modern cloud-based solutions to customers worldwide.

Both AWS and GCP offer cloud credits to startups ranging from bootstrapped ventures to those at the Series A stage. If you haven't acquired credits yet, we recommend checking out our article on how to obtain free credits for AWS and for GCP.

In this article, we will explore the ways in which these credits can benefit startups. We will then delve into a detailed understanding of the programs and provide insights on how to maximize their potential. Lastly, we will highlight common pitfalls to avoid when utilizing cloud credits.

How do cloud credits help startups?

The importance of startup cloud credits for early-stage businesses lies in the following aspects:

Offsets Initial Infrastructure Costs: Startups can optimize their budget by utilizing startup credits provided by cloud service providers, which help offset initial infrastructure costs such as computing resources, storage, network infrastructure, and essential services. This strategy enables startups to reduce upfront expenses and allocate their limited financial resources toward crucial areas such as product development, marketing, and talent acquisition.
Experimentation with Advanced Cloud Services: Startup credits opportunities to explore and experiment with the diverse range of services and tools offered by cloud platforms. Including the latest in AI, ML, Big data, and serverless. Access to such cutting-edge tools and technologies can give startups a competitive edge, help them innovate, and enhance their overall capabilities.
Scalability and Flexibility: Cloud computing provides startups with the ability to scale their operations quickly and efficiently. Startup credits allow early-stage businesses to access scalable resources and infrastructure, ensuring they have the capacity to accommodate growth as their user base and demand increase. The flexibility provided by cloud services helps startups avoid upfront investments in fixed infrastructure and allows them to adjust their resources based on fluctuating requirements.
Enhanced Reliability and Security: Startups, especially those in their early stages, may lack the necessary expertise or resources to implement robust security measures on their own. By utilizing startup credits, they can benefit from the established security practices and robust infrastructure provided by cloud providers.

Overall, startup credits provide a valuable opportunity for early-stage businesses to minimize infrastructure costs, experiment with cloud services, and scale their operations efficiently. By leveraging these credits, startups can accelerate their growth, focus on core business activities, and leverage the capabilities and expertise of established cloud service providers.

How do these programs work?

Application review timeline

AWS reviews applications within 7-10 business days. Microsoft reviews applications within 5-7 business days. GCP reviews applications within 3-5 business days. You will receive an email with further details on redeeming the credits for your respective cloud. If you are rejected, you can reapply after 14 days (Microsoft). Google and AWS don’t openly specify terms for reapplication. You can reach out to their support team to take another shot at the credits. You will usually also receive a reason for rejection along with the rejection email.

Tiered credits

Most of the startup credit programs have a tiered credit offering. It starts off by creating an account with the cloud provider of your choice, then you enroll in the credits program, which is usually by filling out a form, and then your application is reviewed. Usually, a basic amount, say $5000 worth of credits, is given to the startup for their first few months of usage. However, if you have more demanding usage, you can request higher credit which will then be evaluated.

Requesting more credits

In the case of tiered credits, you are allowed to request credits multiple times until you reach the maximum amount ($100,000 for AWS). If you had already been offered $5,000 and in your subsequently applied for $25,000, you will receive the difference between the two credit awards, i.e., $20,000. Each request has to be for an amount higher than the previous request. Requesting can be done through the same form as your initial request.

💡 See the requirements for the various levels of Microsoft for Startups Founders Hub.

Credits expiry

All cloud credits come with an expiry date. If you received credit codes from a developer community program, they would have to be redeemed within 60 days from when you received them. The expiry date of the credits is usually one year from the moment you apply them to your account. You can also check the credit utilization and expiry date on the billing console. Users will also receive emails when they are about to run out of credits or get closer to the expiry date.

If you have multiple credits expiring at different times, the one expiring sooner will be applied first to your upcoming bills. More on that here.

Pitfalls to avoid

Turning a blind eye to cloud costs

With thousands in cloud credits, you may start adding more services from the provider. You may also end up over-provisioning (choosing bigger instance types than you actually need). These practices don’t sting you until the day your credits expire. Some may continue paying for cloud services, which could become wasteful spending. It would also be a bigger and more time-consuming effort to evaluate and optimize these resources later on in your product’s lifecycle.

Our recommendation is to start off small, only provision the resources, and use services that are essential for your operations. Try to stay within the free tier of certain services whenever possible. Be in control of who has access to provide services and for what purposes. Imagine you are actually paying for the cloud services so that it doesn’t end up as a shocker expense once your credits run out.

Using latest versions

Though most cloud services nowadays are launched with sufficient documentation, it is recommended to wait 6 months to a year before trying out new services in a prod setting. Dealing with difficulties in setting up or finding public guides to troubleshoot your issues might become more difficult for newer services.

Our recommendation is to stick to services that you and your team are familiar with or can easily adopt. Choose solutions that have been GA for at least a few months, and have sufficient community support.

Poor product quality

With free credits for years and investor money in hand, it would surely feel like anything is possible. You can deploy your app frequently and add 20 new features to the roadmap each quarter. This might also lead to you hiring more engineers and creating a false sense of growth. The day you run out of investor money and cloud credits, your bills skyrocket, and you are left firing people and having incomplete features everywhere.

Our recommendation is to have a lean product management approach and focus on building features that work well for your customer today. Put in equal efforts in marketing and sales to achieve a product market fit before taking on more ambitious feature additions to your product.

Overcomplicating your solution

When utilizing cloud credits for your startup, it's crucial to be cautious about overcomplicating the solution. This can happen if there aren’t many experienced engineers/architects on the team. While the cloud offers scalability, flexibility, and cost savings, it's important not to take these advantages for granted. Poorly managing your cloud setup can result in various problems, such as wasted resources, security vulnerabilities, and unnecessary complexity. Moreover, there is a possibility that your credits may run out, leaving you with unexpected expenses or even service outages.

Our recommendation is to have proper oversight and management of your cloud infrastructure from day one. This entails establishing robust monitoring and alerting systems to effectively track resource usage, performance metrics, and potential security threats. Additionally, if you have limited familiarity with cloud technology, opt for simpler solutions like DigitalOcean.

Ignoring best practices

If you believe you are a startup and hence you have to do everything differently, think again. Ignoring best practices in matters such as GitOps, testing, CI pipelines, security, observability, and Infrastructure as Code will only set you back in the long run.

Our recommendation is that you try and incorporate best practices for each of these from day one and put in sufficient research before getting started with running your apps on the cloud. Many of these services and their providers also have additional costs associated with them, be mindful of that and include it in your cloud budgeting.

Not preparing for the first bill

Your cloud credits will inevitably come to an end, and you may not be ready to digest the huge amount that shows up on your first cloud bill. If you have had a product for several months now or even years, it is likely that you already have a way of funding your operations.

Our recommendation is to be prepared for the bill by keeping your team informed about the usage and costs incurred. Export the cost data from the start and constantly monitor how it changes as you scale. Explore if there are other cheaper alternatives and experiment with them before going into your first bill.

Use open-source solutions where possible and third-party data stores so you can step away from the cloud provider and leverage another year or two of free credits from another cloud provider.

Conclusion

In conclusion, leveraging cloud credits has become a game-changer for startups looking to scale their operations. By utilizing these credits, startups can offset initial infrastructure expenses, experiment with advanced cloud services, and benefit from scalability and flexibility. Moreover, cloud credits provide enhanced reliability, security, and access to cutting-edge technologies.

However, it's important to be mindful of potential pitfalls such as overlooking cloud costs, overcomplicating solutions, and ignoring best practices. By being proactive, prepared, and strategic in utilizing cloud credits, startups can accelerate their growth, focus on core business activities, and maximize their potential for success in the cloud computing realm.

Argonaut platform enables you to build your apps on AWS and GCP, while acting as a central place for managing infra resources, deployment pipelines, app configurations, and more!

Helm Guide: An Introduction to the Kubernetes Package Manager

Argonaut — Fri, 16 Jun 2023 04:51:33 +0000

Helm has become an essential part of the Kubernetes ecosystem. By using Helm, one can simplify the process of creating and deploying Kubernetes resources. In this article, we walk through the basic components of Helm, its architecture, and the benefits of using Helm. Then we have a tutorial on deploying Helm charts using Argonaut.

What is Helm?

Helm is a package manager for Kubernetes that simplifies application deployment and management. It enables users to define, install, and upgrade complex applications with a single command. Helm offers a user-friendly design suitable for beginners and experts and a vast library of ready-to-use charts for effortless installation and management of diverse applications.

Key concepts in Helm

Helm manages the deployment lifecycle of applications using Helm Charts, which ensures consistency across different environments and users. Users can create their own Helm charts for deployment or utilize charts for third-party and open-source tools from public repositories, such as artifacthub, bitnami charts, harbor, and chart museum.

Helm chart

A Helm chart is a collection of files describing the resources and dependencies needed to deploy an application on Kubernetes. It allows for modularization and versioning, making application distribution, sharing, and management more accessible across various clusters and users.

The package consists of multiple files and directories, each with a specific function. Helm reads the chart and generates the necessary Kubernetes manifests based on the provided configurations (values.yaml file).

Helm charts can have dependencies, called subcharts which are stored in the charts/ directory.

Structure of a Helm chart

Name	Type	Function
charts/	Directory	Location for chart dependencies managed manually.
templates/	Directory	These template files written in Golang are merged with values.yaml configuration data
templates/NOTES.txt (optional)	File	A plain text file containing short usage notes
crd/	Directory	Store CRDs that will be installed during a helm install
Chart.yaml	File	Metadata about the chart, such as the version, name, search keywords, etc.
LICENSE (optional)	File	Chart's license in plain text format.
README.md (optional)	File	Important information for using the chart in a human-readable format.
requirements.yaml (optional)	File	A list of chart’s dependencies
values.yaml	File	The default configuration values
values.schema.json (optional)	File	A JSON Schema for imposing a structure on the values.yaml file

Helm releases

The next important component in the Helm architecture are releases. Releases in Helm represent the instances of a deployed Chart within a Kubernetes cluster. A release constitutes of all the Kubernetes objects and resources, such as deployments, services, and ingress rules, which are created as part of the specified configuration in the chart.

Helm chart repository

Helm chart repositories, or repos, are dedicated HTTP servers that host and serve charts alongside an index.yaml file, which provides information about a collection of charts and their download locations.
A Helm client can connect to multiple chart repositories, initially configured with none by default. Using the helm repo add command, users can effortlessly configure and add new chart repositories, enabling seamless access to and management of various charts for their Kubernetes deployments.

Popular chart repositories are artifcathub, bitnami charts, harbor, and chart museum.

Chart version

Every chart must have a version number. Packages in repositories are identified by name plus version. Helm charts are versioned according to the SemVer2 spec. For example, an nginx chart whose version field is set to version: 1.2.3 will be named: nginx-1.2.3.tgz

The version number is found in the Chart.yaml file and is used by various Helm tools, including the CLI. When creating a package, the helm package command uses the version number from the Chart.yaml in the package name. The system expects the version number in the chart package name to match the one in the Chart.yaml, and any discrepancy will cause an error.

Chart dependency

In Helm, one chart may depend on any number of other charts. These dependencies can be added in two ways - by dynamically linking using the dependencies field in Chart.yaml or by bringing it into the charts/ directory and managing manually.

Example using dependencies field:

dependencies:
  - name: apache
    version: 1.2.3
    repository: https://example.com/charts
  - name: mysql
    version: 3.2.1
    repository: https://another.example.com/charts

Example using charts/ :

wordpress:
  Chart.yaml
  # ...
  charts/
    apache/
      Chart.yaml
      # ...
    mysql/
      Chart.yaml
      # ...

Release number (release version)

A release can be modified several times. To keep track of these changes, a continuous counter is utilized. Upon the initial helm install, the release number is set to 1. With each subsequent upgrade or rollback, the release number increases by 1. This history is useful if one needs to roll back to a previous release number.****

Helm Rollbacks

The helm rollback <RELEASE> [REVISION] [flags] command can be used to roll back to any previous version of the release. Note: a rolled back release will receive a new release number. You can find a list of flags here.

Helm library (or SDK)

The Helm Library (or SDK) refers to the Go code that interacts directly with the Kubernetes API server to install, upgrade, query, and remove Kubernetes resources. It can be imported into a project to use Helm as a client library instead of a CLI.

This is an advanced technique released in Helm 3. You can check the official docs here. And examples here.

Helm architecture

This diagram better explains how Helm uses charts and value files to manage releases (deployed resources) in your Kubernetes cluster.

This diagram shows the OpenTelemetry Operator Helm chart workflow. Here you see how the AWS Observability team builds and maintains the Helm chart in a public repo, and it can be seamlessly downloaded and deployed to users’ clusters.

This process also has several benefits compared to the previous methods of deploying an OpenTelemetry operator.

Added functionality such as installing/uninstalling packages, upgrades, rollbacks, and customized installations.
User flexibility to configure values through the values.yaml file, you determine which values to pass to the OpenTelemetry Operator Helm chart configuration. You can override multiple values with one command.
It’s the easiest way to deploy the Operator to Kubernetes.

For more information, check out this AWS blog.

Why you should use Helm?

There are several ways to deploy and manage resources on Kubernetes; why should you choose Helm? Popular alternatives are Kustomize, Tanka, and Carvel, all of which have less mature communities than Helm and lack the number of publicly available charts (packages).

Helm has emerged as the clear winner with its ability to handle both simple and complex configurations, versioning, reusability, etc.

There are also ways to use Helm and Kustomize together.

Here are the key benefits of Helm:

Simplicity: Defining, installing, upgrading, and rolling back complex Kubernetes applications can be done with a single command. This greatly simplifies the management and deployment of Kubernetes resources.
Reusability: Helm charts are essentially packages of pre-configured Kubernetes resources. The charts can be reused across projects and shared with the wider community.
Configurability: Helm provides a highly configurable structure with Charts(templates) and values(configs). Just by changing a few parameters, we can use the same chart for deploying on multiple environments like stag/prod or multiple cloud providers.
Consistency: Helm charts provide a standardized way of packaging and deploying Kubernetes resources. This can help ensure consistency across different environments and reduce the risk of errors or inconsistencies in deployment.
Scalability: With Helm, you can easily scale your Kubernetes applications up or down by adjusting the values in the values.yaml file.
Community: Helm has a large and active community that is constantly developing and improving the tool. This means that there are many resources and best practices available to help you get the most out of Helm.

Tutorial - Deploy Helm charts using Argonaut

Here’s a quick demo of deploying a Helm chart of OSS and third-party tools to your Kubernetes cluster using Argonaut. Argonaut’s platform helps you simplify cloud deployments and infra management. You can choose any publicly available Helm chart and deploy it to your Kubernetes cluster on AWS and GCP cloud.

The beauty of using Argonaut is that you don’t need to use any Helm commands. You can set the chart configs, version, and edit values.yaml using our simple UI. Your Kubernetes resources are then deployed using GitOps best practices using ArgoCD under the hood. Also, when you create a cluster using Argonaut, we automatically add essential apps such as Keda, Kubernetes events exporter, Nginx-Ingress, Cert-manager, Prometheus, and metrics-server.

In this example, we will be deploying rabbitmq.

Pre-requisites

Argonaut account
AWS/GCP account connected to Argonaut
A k8s cluster created using/imported to Argonaut

Deploying Helm chart

Select your environment from the Argonaut dashboard
In your selected cluster, click on the Add-ons button to see the pre-installed apps
Click on Application +
Choose From Library
Ensure your environment and cluster names are correct
Select the Custom-Apps option
1. Use the default tools namespace
2. Set release name as my-rabbitmq
3. Set chart name as rabbitmq
4. Repo URL as https://charts.bitnami.com/bitnami
5. The latest chart version will be automatically fetched; you can change to an older version if needed
6. Click on load values.yaml to see the values file
7. You can modify as needed and add overrides or annotations
Click Install. Rabbitmq is now added to your cluster!

It’s just as simple to install all other apps using Helm charts. Just ensure the chart name is the same as the one in the repo. And to find the correct Helm repo URLs for your apps, you can search https://artifacthub.io/.

💡 If you want to try installing it using Helm CLI commands, here is a useful tutorial.

Conclusion

Helm has proven to be an invaluable tool for managing Kubernetes applications. With its powerful features, such as simplicity, reusability, configurability, consistency, and scalability, Helm enables users to harness the full potential of Kubernetes without a steep learning curve. Additionally, the active and growing community behind Helm ensures that it will continue to improve and remain the go-to solution for deploying and managing complex Kubernetes resources.

By using tools like Argonaut alongside Helm charts, you can simplify your cloud deployments even further and streamline your infrastructure management processes. As demonstrated in our tutorial on deploying RabbitMQ with Argonaut.

Ultimately, if you're looking to deploy and manage Kubernetes applications efficiently while minimizing errors and inconsistencies in deployment processes, consider adopting Helm as your go-to package manager.

References for this article:

Engineering: A Technical Exploration of Argonaut's Notifications System

Argonaut — Tue, 30 May 2023 04:54:49 +0000

This article is written with massive help from Prajjwal Dimri who built the Notifications feature.

We are excited to add Notifications to Argonaut. Notifications allow users to get instant updates on any build and deploy actions that occur in a user’s org. This blog is a walk-through of our approach, details of our architecture, along with use cases for this feature.

There are three layers in our architecture: The transformation layer, the processing layer, and the fanout layer. We built the transformation and processing layer in house and used Novu for the fanout layer. It helped us save time by avoiding reengineering this component, which is fairly consistent in most notification systems across various products. It provided us with additional benefits:

Easy integrations with communication channels that our users are on.
Web widget out of the box, making the integration into Argonaut seamless.
User level control for subscribe/unsub for in-app and email notifications.

Notifications play a crucial role in keeping you and your team informed about build and deploy stage updates in real-time. We recognize that the effectiveness of notifications hinges on their promptness, delivery through optimal channels, and appropriate frequency. Consequently, we have designed a customizable notifications system that allows you to receive only the desired notifications through your preferred medium, such as Slack, Teams, Discord, or email.

Use cases

You get a lot of useful information at a glance! The notifications look something like this. And clicking on any notification takes you directly to the respective screen.

We currently provide different kinds of updates for Application Developers. With the addition of events catering to infra coming soon.

Role in your company	Get updates on	So that you can
Application Developer	High-priority notifications in the case of my builds or deployment pipelines failing	Fix the errors quickly
Application Developer	High-priority notifications in the case of pods failing to start after a new deployment	Fix the errors quickly
Application Developer	Opt-in to receive notifications when pipelines succeed	Be confident that the deployments went through
	👇 Coming soon 👇
DevOps Engineer	Pods take a very high amount of resources (close to the set limit) in the cluster	Get this fixed by allocating higher limits or getting the dev team to lower the resource usage of the pod
DevOps Engineer	Pods getting evicted	Fix it by allocating higher limits or getting the dev team to lower the resource usage of the pod
DevOps Engineer	Cluster spinning up more nodes than the desired nodes config	Check why more nodes are getting spun up and how this is affecting our costs in our cloud provider
DevOps Engineer	Get high-priority notifications if any infra-CRUD operations fail	Fix and restart the infra operation
A Manager	Get notified when environments are created or deleted.	Be notified of resource usage
A Manager	VCS & cloud account integrations are added or deleted	Be in sync about users’ actions in the org

Notifications architecture

Here’s a quick overview of our implementation. The process can be broken down into three layers: The transformation layer, the processing layer, and the fanout layer.

Events

Actions by users, such as creating new apps, updating a build or deploy config, and triggering a pipeline, are all considered to be events. We use event driven programming in our backend, where our internal services communicate with each other through publishing and consuming events from a queue. For example, when a build is triggered, running, gets completed or fails, the build service publishes these events to the queue which can be consumed by various other services.

Transformer

The notification transformer listens to a particular topic in the queue, processes the message, and transforms it into a standard format (including information about the org and the user).

Transformers have the capability to deal with various types of events getting generated in the system and will also have the logic to assign priorities to various types of events. (priority queue coming soon)

Standard notification format

Key Name	Key Type	Key Description
type	EventType (Enum)	Type of event that caused this payload to be created
priority	number	Priority for the event.
users	Array (User Ids)	The users for which this event is relevant
organizations	Array (Org Ids)	The organizations for which this event is relevant
createdAt	Date-Time	Event creation timestamp
metadata	JSON	Additional metadata and information relevant to the event

We use Novu subscriber in the fanout layer. Therefore, we internally map each org/user with the Novu subscriberId.

Processor

The processor takes the information about the user and the org and retrieves the map, and changes the ID to match the Novu subscriber ID, which is required in the next step to connect with the Fan Out layer and ensure the notifications are routed to the right users.

Fanout layer

The final layer is the Fanout layer, which is managed by Novu. Using Novu made our integration with Slack, Teams, email, and Discord a breeze. The fanout layer (Novu) receives the payload and sends it to the subscribers (members in the same workspace).

Note: Every user and org ID is mapped out to respective subscriberIds.


func (d defaultFanOutService) TriggerFanOut(ctx pkgCommons.IContext, eventId string, subscriberIds []string, payload map[string]interface{}) errors.IError {

novuClient := d.getNovuClient()

_, err := novuClient.EventApi.Trigger(context.Background(), eventId, novu.ITriggerPayloadOptions{

To: subscriberIds,

Payload: payload,

})

Journey of a Notification

An event occurs and is published in Argonaut’s internal backend service.
The event is added to the SNS queue.
The appropriate mapped notification transformer retrieves the event.
Transformer decides if the event should be processed or not.
The transformer fetches all the users and organizations for which this event is relevant. For example, In case of a failed build event, the relevant org and its users will be fetched.
A notification processor (goroutine)is spawned and it picks up the notification.
Processor loops over the given users and organizations.
Every iteration, it fetches the subscription preferences, creates a payload to trigger Novu, and dispatches the payload.
Novu does its magic and sends notifications to users through their selected channels.

Experience using Novu

In our setup, Novu forms the fanout layer, and its main responsibility is sending out notifications to various platforms, e.g., Slack, Email, UI, etc. We are currently using Novu Cloud, which has a free tier supporting up to 10K events per month.

We use Templates offered by Novu, which ties in all the different channels under a single entity. This provides a consistent experience for both the developer and the user, regardless of the channel they choose to receive the notification on.

Entry of notifications and preferences of subscribers is stored in Novu’s MongoDB Database. Notification Aggregation (Digest) is also handled by Novu.

We preferred Novu’s solution over Courier & Knock as it was more economical and it is also an Open Source project.

Novu’s active discord community was supportive when we had queries regarding implementation. We were also able to add our contributions to their Go lang SDK.

We’ve also used Novu’s front-end react client to show notifications within the Argonaut UI. It was easy to use and configure according to our brand colors and design, the configuration of which took under 1 hour.

Upcoming features

External event integrations where we would add a Webhook processor to receive notifications from external services and third-party tools.
Adding a priority queue so that high-priority issues like pod failures and cost escalations are shared with the user instantly.
Notifications for more events catering to different types of users. Such as infra updates, cost, pod utilization, billing, etc.

Conclusion

In summary, adding Notifications to Argonaut has greatly improved the user experience by giving quick updates on build and deploy actions. Using Novu, we've created a smooth and personalized notification system with options for updates through channels like Slack, Teams, Discord, or email.

Our implementation includes the Process layer, Transform layer, and Fan out layer, making sure each notification is useful, prompt, and correct. Working with Novu has led to a more cost-effective solution. We're sure our users will appreciate this new feature, and we're eager to keep enhancing and growing our offerings to meet the various needs of our users

Set Up Your Internal Developer Platform With These Open-Source Solutions

Argonaut — Wed, 19 Apr 2023 07:22:38 +0000

Internal Developer Platforms (IDPs) are revolutionizing the way developers work by automating repetitive tasks, standardizing workflows, and reducing the time spent on infrastructure management. This not only leads to increased productivity but also fosters a culture of collaboration, innovation, and growth within the team. In this article, we dive into the world of open-source solutions that help you elevate your development process, streamline workflows, and improve efficiency. In other words, these solutions put together can provide you with the benefits of an IDP.

While there are several SaaS products like Argonaut, there might be org specific needs because of which bespoke tooling is needed. In such cases, open-source solutions can be leveraged to build your IDP. In this article, we explore the tools that can be used to build an IDP.

By leveraging open-source software to build your IDP, you benefit from a vast community of developers and documentation. However, in reality, there is no plug-and-play OSS IDP and some assembly is required. Some of the reasons are:

Unique requirements: Your organization’s requirements vastly differ from other organizations that use IDPs. Matching these features to feature would usually require building a custom solution or severely modifying an available OSS.
Open-source solutions rely on community support. Therefore, ensuring there’s an active community or the resources required to keep it up-to-date is important.
Integration challenges: It is unlikely to find open-source solutions that seamlessly integrate with your company's existing tools, technologies, and infrastructure. This could lead to additional time and effort spent on customizing and integrating the solutions into your IDP, which may not always be feasible. Some OSS tools also lack proper documentation and don’t offer support, which could make implementing them even more challenging and costly.

Tooling for an Internal Developer Platform

The minimal requirements for an effective Internal Developer Platform (IDP) are an effective IaC solution, GitOps tooling, and a service catalog. There can be several other tools, such as monitoring, databases, CI tools, and security, as a part of your IDP. These together help your dev team automate repetitive tasks and standardize workflows.

Here we explore popular tools in each of these categories, along with alternatives.

Defining Infrastructure as Code (IaC)

An Infrastructure as Code tool is an essential component of an IDP; it enables managing infra at scale and in a declarative way. It provides several benefits, like collaborating more effectively, managing complex cloud complexities, and improving consistency.

Crossplane

Crossplane is a framework for building cloud-native control planes without the need to write code. It provides the building blocks that enable you to provision, compose, and consume infrastructure with the Kubernetes API. Its ability to work as a control plane, interact with multiple services across vendors, and create custom and composite resources make it suitable for IDPs.

Setting up

Set up Crossplane in your Kubernetes cluster (Helm Install) to manage and provision infrastructure resources. You can then define custom resources for the cloud services and infrastructure components you want to manage.

Alternatives

Terraform is a powerful and flexible Infrastructure as Code (IaC) tool that can be used to build, manage, and scale internal developer platforms. Its extensibility, multi-cloud support, declarative configuration, Kubernetes-native architecture, and composability make it an ideal choice for organizations looking to streamline their infrastructure management processes and empower their development teams.
Pulumi is a versatile IaC tool designed with cloud-native applications in mind. It allows developers to manage and provision infrastructure using familiar programming languages. Its versatility, multi-cloud support, and integration with Kubernetes make it an ideal solution for constructing internal developer platforms (IDPs). By leveraging Pulumi's language-specific SDKs and reusable components, teams can efficiently collaborate, standardize infrastructure configurations, and create customized resources tailored to their specific needs. > 💡 For more IaC tools, check out our Top IaC tools article. And here’s an in-depth comparison between Pulumi, Terraform, and AWS CloudFormation.

Continuous Delivery (CD)

Continuous delivery tools are crucial in IDPs, enabling faster and more reliable software releases. By automating the deployment process, teams can minimize human errors, enhance collaboration, and maintain consistent quality throughout the development lifecycle. This ultimately accelerates innovation and increases overall efficiency within an organization.

ArgoCD

ArgoCD is an open-source Continuous Delivery tool for automating application deployment in Kubernetes clusters. Utilizing the GitOps methodology, it monitors Git repositories to synchronize the desired state with the live environment, ensuring efficient and reliable application delivery.

Setting up

Set up a Git repository containing the desired state of your applications and infrastructure, including the custom resources defined using Crossplane. ArgoCD can then be used to manage the deployment and configuration of your applications and infrastructure resources using GitOps methodology.

Alternatives

Flux is a popular GitOps alternative to ArgoCD. It helps you manage deployments, resources, and integrations with various Git providers and provides multi-tenancy support. It uses a cluster operator to start deployments in Kubernetes, so there's no need for another CD tool.
Gimlet is a command line tool and a dashboard that packages a set of conventions and matching workflows to manage a GitOps developer platform effectively. It is built on top of Helm and Flux and provides you with a paved path, a set of best-practices, so you can focus on your task at hand. > 💡 In-depth comparison between ArgoCD and Flux.

Service Catalogs

A service catalog, such as backstage.io, serves as a developer portal that offers a comprehensive view of various applications, services, and resources managed by engineering teams. It includes information about ownership, metadata, and essential service-related links. While it primarily benefits engineers and engineering managers, it does not directly address the need for quicker collaboration between Operations and engineering teams.

Backstage

Backstage is a comprehensive platform designed to streamline the development process for product teams by centralizing software catalogs, infrastructure tooling, services, and documentation. Key features include the Backstage Software Catalog for managing various software types, Software Templates for creating new projects in alignment with organizational best practices, and TechDocs for seamless technical documentation. Additionally, Backstage offers an expanding ecosystem of open-source plugins for enhanced customization and functionality.

Setting Up

Install and configure Backstage to provide a unified interface for your developers. Integrate Backstage with Crossplane and ArgoCD, so developers can easily discover, manage, and deploy applications and infrastructure resources through the Backstage portal.
Leverage Backstage’s custom plugins and integrations to further tailor the platform to your organization's needs. This can include integrating with other tools, services, and APIs used in your development processes.

Alternatives

Port can be used as an alternative to Backstage.io. It offers a way to build internal developer portals with context-rich software catalogs with maturity and quality
Compass by Atlassian aids engineering teams in managing software sprawl by offering a single source of truth for distributed architecture. It enables understanding of built components, ownership, operational health, and applied policies. Compass provides insights into problem areas and changes over time, enhancing architecture and development velocity.
OpsLevel is a uniform interface that lets developers manage everything from one place, including their tools, services, and systems.

Here we’ve used Crossplane for IaC and as an infrastructure control plane, ArgoCD for Continuous Delivery and GitOps best practices, and Backstage as a service catalog. This combination provides a powerful, customized IDP that simplifies the management of your cloud infrastructure while providing a seamless experience for your developers. Here’s a video demo of one such setup.

The setup and its complexity may vary depending on your company’s size and the tools you are currently using. It will also require a significant time and development commitment to build and maintain the platform.

Add-on capabilities for your IDP

While the combination of the above three tools provides a basic solution for IDP, the below tools can provide more add-on capabilities.

Kubernetes control planes

Shipa.io is a Kubernetes control plane that provides an abstraction level for deploying clusters while maintaining the same user experience. It requires YAML files for configuration and CI pipeline connections but lacks dynamic workload and environment creation. Shipa is suitable for governance purposes but may not be ideal for building an IDP.
Kubermatic Kubernetes Platform (KKP) is an open source project that helps centrally manage the global automation of thousands of Kubernetes clusters across multicloud, on-prem and edge with unparalleled density and resilience. KKP is compatible with all major cloud providers even supports custom infrastructure setups. By offering a user-friendly, self-service portal for developers and IT teams, KKP simplifies the complexities of managing cloud-native IT infrastructure and multi-cloud operations.

Continuous Integrations (CI)

GitLab is a versatile application that many organizations depend on for tasks like source code management, continuous integration, and deployment. It provides the flexibility of creating and running pipelines with multiple CI/CD stages. Auto DevOps makes using GitLab a breeze.
GitHub Actions provides a wide range of pre-built actions, integration with third-party tools, and the ability to create custom actions, making it a versatile solution for implementing CI in any project. With CI, developers can automatically build, test, and validate their code whenever changes are made, ensuring that it is always in a deployable state.

Monitoring and observability

ELK stack: Elasticsearch, Logstash, Kibana is a popular open-source stack for log management. It is a powerful tool for collecting, storing, and analyzing log data. Kibana is quite useful for visualizing the data.
Grafana stack: Grafana, Loki, Tempo is an open-source stack to compose observability dashboards with everything from Prometheus and Graphite metrics to logs and application data. Grafana connects with a plethora of data sources, including Graphite, Prometheus, Influx DB, ElasticSearch, MySQL, and PostgreSQL. It helps to monitor and analyze data and track user and application behavior, including error type and frequency in pre-production and production environments. If you’re ok with the additional overhead, it’s a great way to monitor.

💡 For more observability tools, check out this article. Here’s an in-depth comparison between Datadog, New Relic, and Splunk.

Best Practices for Internal Developer Portal

Here are some best practices to consider while creating and maintaining your own Internal Developer platform (IDP).

Ensuring clear documentation - IDPs are meant to be easy to use and provide self-serve capabilities. Whether they are in-built or purchased solutions, having clear and concise documentation helps its users understand why and how to resolve their issues. A well-documented IDP would also make it easy for new team members to get up to speed and collaborate better.
Encouraging collaboration and communication among developers - Since IDPs are used by the entire team, understanding who uses what in an IDP is important. These controls can be set using RBAC in most IDPs.
Monitoring usage and performance to identify areas for improvement - By adding monitoring abilities, team leaders or Ops professionals can get a sense of cloud usage and associated costs. It is also essential to have observability over issues and logs of the various cloud services.
Regularly updating SDKs, libraries, and code samples - As a part of maintaining the IDP, maintaining the versions of the dependency libraries is important for the developers. This becomes even more important when the IDP is a combination of several tools and needs to meet security standards.
Integrating user feedback to enhance the overall developer experience - If you’re just starting out, getting feedback from your developer team is the best way to ensure that your IDP meets the organizational requirements.

IDPs are a way to highlight the golden paths for your team and include best practices when it comes to the development and deployment of both infra and app. Following these five best practices can help you get started in creating an effective and useable platform to elevate your dev team’s productivity.

Conclusion

The significance of Internal Developer Platforms (IDPs) for developer teams is immense, as they offer a wide range of advantages that help streamline various aspects of the development process. By automating repetitive tasks and standardizing workflows, IDPs allow developers to focus on critical aspects of their work, leading to increased productivity and growth. Additionally, IDPs help reduce the time and effort spent on managing infrastructure, allowing teams to concentrate on innovation and delivering high-quality applications.

To create an effective IDP that caters to their specific needs, organizations should follow best practices and actively seek user feedback. By doing so, they can ensure that their IDP addresses their unique requirements and enhances the overall developer experience. In today's competitive and rapidly evolving technological landscape, embracing the power of IDPs is crucial for organizations aiming to stay ahead of the curve. Adopting a well-designed and efficient IDP can make all the difference in empowering development teams to excel and drive the organization's success.

Argonaut is a DevOps automation platform designed to streamline the management of both applications and infrastructure, enabling engineering teams to accelerate delivery. By incorporating GitOps best practices, Argonaut simplifies the process of creating and maintaining complex cloud setups. With support for Kubernetes application deployment on AWS and GCP, Argonaut offers a single pane to manage all your cloud apps, infra, integrations, and deployment workflows, catering to a wide range of organizational needs. Try it out for free today!

Internal Developer Platforms: An In-Depth Guide to Unlocking Developer Productivity

Argonaut — Tue, 11 Apr 2023 18:04:05 +0000

What are Internal Developer Platforms (IDPs)?

Internal developer platforms (IDPs) are platforms that developers can leverage to build and deploy their applications to one of the environments. Their main purpose is to increase the pace of development and reduce the developer’s dependence on DevOps and Platform engineers through automation and developer self-service. The platform may be built in-house, usually, an adaptation of an open-source or purchased offering such as platform as a Service (PaaS).

The adoption has been both recent and rapid. Puppet’s state of platform engineering survey finds that over 51% of companies that have adopted Internal Developer Platforms have done it in the past three years. And an overwhelming majority of the respondents (93%) declared that IDP adoption is a step in the right direction.

Internal developer platforms are good at helping the team manage applications and infrastructure from one place, providing tight integration with the existing tools and services, and giving developers self-serve and collaboration capabilities.

Benefits of IDPs

IDPs offer several benefits to the organizations using them. The first relates to improvements in infrastructure and IT management, such as increased productivity stemming from reduced communication times between devs and infra teams. Secondly, they reduce the complexity of the cloud and make it easy for people in your org to pick up on the best practices set forward by your team. In larger organizations, they can also be an easy way of managing RBAC to deployments, infra creation and management, and more.

Responses from Puppet’s State of Platform Engineering 2023 report also show how IDP has improved DevOps KPIs:

An increase in development velocity (68%)
42% say development velocity has improved “a great deal” since they started doing platform engineering
Improved productivity (59%)
Improved system reliability (60%)
Improved security (55%)

Components of IDPs

Let’s dive deeper into the components of IDPs and discuss which parts of the application and infrastructure management process they improve.

Integration with existing tools and services IDPs are built by integrating with the existing set of tools and services that the company currently uses. These can be your source control systems (e.g., GitHub, GitLab), Continuous Integration and Continuous Deployment (CI/CD) pipelines, and observability tools such as monitoring and logging systems. In addition, they may also include some form of access control, observability dashboards, and a set of best practices for developers to use.
Application and infrastructure management IDPs can be used to automate a lot of the hassle of deployment. By setting up IDPs with GitOps best practices, users can automatically deploy application and infra changes with each commit to the Git code. Managing multiple environments is another popular use case of IDPs. They can be effective in creating, providing selective access, and managing the use of multiple different environments such as development, testing, pre-prod, and prod. Several tools set up preview environments on the fly. Their ability to monitor and view deployments across environments helps organizations scale dynamically to changing workloads and maintain consistent performance across their infrastructure.
Developer self-service capabilities IDPs empower developers with self-serve capabilities by offering a centralized, user-friendly interface to access tools, resources, and services on demand. These capabilities facilitate faster development cycles, reduce operations teams’ dependency, and enable greater autonomy in seamlessly creating, testing, and deploying applications. The streamlined workflow also makes it considerably easier for new employees to get started with your stack without diving into the internal workings. Some tools also add a collaboration layer on top that allows different members of a team to work together easily and review changes before deploying the code.
Collaboration and governance features IDPs can achieve enhanced security and compliance by providing Role-based access controls (RBAC), ensuring each team member has the appropriate permissions to access specific resources and perform certain actions. Overall, this minimized the risk of unauthorized access or accidental changes while also promoting collaboration and efficient workflows among cross-functional teams. IDPs help developer teams maintain transparency, accountability, and traceability throughout the development process by providing audit logs and history tracking. This enables easier identification and resolution of issues, as well as ensures compliance with regulatory and organizational requirements for data handling and change management.

Platform engineering

Companies that decide to build their own platform usually hire platform engineers, who are specialized professionals responsible for the creation and continuous improvement of the IDP. They work to implement and maintain tools, services, and best practices that streamline development processes, ensuring a smooth and efficient workflow for the organization's developers.

Platform engineering is the process of designing, building, and maintaining an IDP that provides a centralized, scalable, and efficient infrastructure for developers within an organization. An IDP simplifies and standardizes application development, deployment, and management, enabling faster and more reliable software delivery.

Software development trends that are catalysts for IDP adoption

Digital transformation: As businesses worldwide increasingly embrace digital transformation, there is a growing need for efficient and robust software development processes. IDPs play a crucial role in enabling companies to stay agile and quickly adapt to the dynamic digital landscape.
Need for faster software delivery: In today's fast-paced business environment, organizations must deliver new features and applications at an unprecedented speed. IDPs provide a standardized, automated, and centralized platform that accelerates development cycles, enabling companies to stay competitive and responsive to market changes.
The growing complexity of software architectures: With the rise of microservices, containers, and cloud-native technologies, managing and deploying software has become increasingly complex. IDPs help simplify this complexity by providing a unified platform where developers can build, test, and deploy applications with ease, regardless of the underlying architecture.
Demand for DevOps and CI/CD practices: The need for IDPs grows as more organizations adopt DevOps and Continuous Integration/Continuous Deployment (CI/CD) practices. IDPs enable seamless collaboration between development and operations teams, automating many manual tasks and ensuring smooth transitions throughout the software development lifecycle.
Scalability and flexibility: IDPs offer a scalable solution that can accommodate the growing needs of organizations, regardless of their size. They provide a flexible platform that can be easily customized and adapted to cater to the unique requirements of different teams and projects.
Cross-region collaboration: With businesses operating across multiple geographies, the need for a platform that supports cross-region collaboration is essential. IDPs empower development teams spread across the globe to work together seamlessly, enabling efficient knowledge sharing and fostering a culture of innovation.

In summary, the rising popularity of IDPs across companies of all sizes and regions worldwide can be attributed to their ability to streamline software development processes, simplify complex architectures, support DevOps and CI/CD practices, and facilitate cross-region collaboration. As the demand for digitalization and agility continues to grow, IDPs are poised to play a critical role in shaping the future of software development.

Commercially available internal developer platforms

Argonaut is a DevOps automation platform that helps engineering manage both the app and infra side of things and ship faster! Built with the GitOps best practices in mind, Argonaut reduces the complexities of creating and maintaining cloud setups. Whether it is Kubernetes app deployment to AWS or GCP, there are several runtimes, environments, regions, integrations, and app types to choose from.
Mia Platform offers a simple way to develop and operate modern cloud applications on Kubernetes. It can be adopted as either a self-hosted or PaaS option. There’s also a marketplace with essential plugins, templates, and applications making it easier to get started. Its features benefit not only the developers but also the platform engineers and CIOs.
Humanitec is an internal developer platform providing simplicity, automation, reusability, and self-service. It acts as a Platform Orchestrator that lets engineering teams remove bottlenecks by enabling them to build code-based golden paths (executable config files and templates) for developers. It can be used through the CLI or UI.
Opslevel provides engineering teams self-serve access to tools and information. It helps developers ensure operational excellence across services with its integrations that can be set up in a secure and compliant way.
Shipa is a Kubernetes application management platform that enables efficient deployment processes. Developers can leverage its standardized application and policy definitions that are platform-agnostic. It also has a GUI-based portal to manage apps after deployment and gain visibility into pipelines for smoother operations.
Port provides a context-rich software catalog with maturity and quality scorecards. It also supports comprehensive developer self-service actions while providing additional role-based access controls (RBAC). Their free-forever provides many key features and makes it a worthy contender.
Upbound, powered by Crossplane, offers an enterprise-grade control plane solution for multi-cloud and hybrid environments, enabling efficient management of cloud infrastructure. The Up command line and Upbound marketplace make it more effective and easier to get started.
DevOpsBox is an all-in-one DevOps platform that streamlines the application deployment process. It provides a comprehensive feature set in a modular manner that allows teams to focus fully on business functionality.

Evaluating Internal Developer Platforms

Despite its myriad of benefits, Internal Developer Platforms do not make sense for all teams. They can end up being overkill for certain types of engineering teams and a burden to build and maintain for companies with smaller teams.

When they don’t make sense

You have existing processes that are efficient. Don’t complicate things too early. Companies that are currently using PaaS or other managed offerings should continue doing so as long as possible.
Limited resources and team size. This could also mean your team is small, and most of your team is senior and comfortable writing scripts and managing infrastructure.
There is low development complexity. If you have just one app with a simple single-cloud setup. And, if your app is monolithic and doesn’t make use of the microservice architecture, there is little benefit from creating an IDP.
Incompatible organizational culture. If an organization’s culture is resistant to change or does not foster collaboration and communication, implementing an IDP might not be successful and could even lead to decreased efficiency and productivity.

When they make sense

You plan to start using Microservices. This usually also means a growing development team size or the complexity of projects you handle.
You have a small team, and not everyone feels comfortable with deployments, scripting, and infrastructure, and you have not yet hired a dedicated DevOps.
Dependencies on other colleagues are blocking your developers.
The cost of your existing setup (such as PaaS) is too high. Which is inevitable once you start to scale to meet new requirements.
You have plans to go multi-cloud, adopt more modern cloud services, and scale geographically.
You want to increase standardization and consistency across your teams. An IDP can help reduce errors, improve code quality, and ensure all developers work with the same set of tools and follow the same best practices.

ArgoCD in Action: A Behind-the-Scenes Tour of Argonaut's GitOps Approach

Argonaut — Tue, 11 Apr 2023 17:53:37 +0000

This article is written in collaboration with Prashant R who built the ArgoCD integration.

In our recent announcement about adding the pipeline features to Argonaut, we mention using ArgoCD for our Continuous Delivery workflows. This article aims to shed more light on our implementation of ArgoCD by touching upon some of the major changes we’ve made. These changes might interest those curious about the backend changes that went in to this integration or if you’re looking to implement GitOps practices in your org using ArgoCD.

What is ArgoCD?

ArgoCD is an open-source, declarative, and Kubernetes-native Continuous Delivery (CD) tool designed to automate and simplify application deployment and management within Kubernetes clusters. It follows the GitOps methodology, using Git repositories as the single source of truth for maintaining the desired state of applications and infrastructure. ArgoCD monitors the Git repository for changes, automatically synchronizing the declared state with the live environment, ensuring consistency, and enabling faster and more reliable application delivery.

We choose to go with ArgoCD because it offers Argonaut greater flexibility to build features on top of our existing offerings. With its extensive feature set, including built-in support for Helm, Kustomize, and other configuration management tools, we found ArgoCD the right choice to implement our GitOps pipelines. In the coming months, we will be able to provide more functionality for our users to leverage based on these toolsets.

Our implementation

There are two major new components that we’ve introduced at Argonaut that enabled this shift to ArgoCD-based workflows. First, a new internal Git repo for every workspace that uses Argonaut. Second, a manager ArgoCD controller that runs in Argonaut’s environment. Before we get into the details, let’s touch upon what our setup looked like up until now.

Previous backend setup

(Pre-migration on 30th March 2023)

In our previous setup, we had a single workflow that combined the build and deploys steps. Though this was acceptable for most use cases, there were several of our customers that had more complex requirements. Such as deploying an image that was already built, building once and deploying to multiple environments, and so on. There was also an issue regarding custom apps such as Datadog, which do not require a build step. Currently, we’ve been handling that using the Add-ons flow and differentiating them from your Git apps (Git-apps).

These workflows were also triggered from Argonaut each time using Helm. This meant there was more involvement of Argonaut’s backend to complete the tasks and ensure that all your deployments are being carried on as expected. Our backend had to be stateful for the most part and directly involved in keeping up with user deployments.

The Argonaut Bot on your Git repo had to install several workflow files on your Git repo to make it function. Small manual changes to these files could cause issues, and we even had to develop a three-way merge feature to fix such issues. This was a messy implementation, and many of our customers also requested a better way to do this.

New backend setup

The first major change we made was separating the build and deployment into two independent steps. This involved splitting the releases into the build and deploy steps, separating the build time and run time secrets while also ensuring they work well together without affecting the performance in any way.

Secondly, we now have an internal Argonaut GitHub with a unique repository for each workspace our user creates. This repo for your org will house all the config files and Helm values.yaml files. The ArgoCD instances for your org will watch for changes to this Git repo and automatically apply the changes when they notice them. By doing so, we can implement GitOps-powered deployment flows for your applications.

Third, we’ve reduced the number of config files we store in your repo. Each repo connected to an Argonaut org will now have just one workflow file. This file will be triggered in multiple ways based on the action you take from Argonaut’s UI.

Key decisions during migration

We’ve had to make several key decisions during the migration process and ensure a consistent user experience while also maintaining cost-effectiveness and planning for future scale from our end. Here we highlight a few key decisions.

Spinning up ArgoCD instances

All kinds of users visit and try out Argonaut. Some use it for their hobby projects, some power users use it for their work, and some are just exploring the product. We wanted to continue offering Argonaut as a free-to-use solution for all.

With the new ArgoCD instances approach, the initial costs would go up on Argonaut’s end as we spin up new ArgoCD instances for each user/org/environment. It was necessary to set up a milestone in a user’s journey where deploying an ArgoCD cluster for their app was viable. i.e., that they would actually use it, and we were confident that they would use Argonaut for their deployment.

We decided to spin up a new ArgoCD instance whenever users create their first k8s cluster in Argonaut. This would trigger a Helm install that would set up the ArgoCD instance in your own cluster.

Since every workspace has just one ArgoCD instance and one repo in Argonaut’s internal GitHub, creating the first Kubernetes cluster as a threshold point would be sufficient for all their future workflows and apps managed using Argonaut.

💡 Note: Though the ArgoCD instance is only installed on first cluster creation, the internal Git repo creation happens when a new org is created. The GitHub repo is essential to store all configs pertaining to the org.

Connecting ArgoCD and user clusters

Now that we have an ArgoCD manager app running on Argonaut’s cluster and an instance running in each of our users’ workspace, the next step is to ensure the right ArgoCD instances are watching the right repos and updating the changes as they come.

In our setup, the manager App is the one that runs in Argonaut’s cluster, and the rest of the apps are deployed directly in our user’s cluster during their first Kubernetes cluster creation.

The task of the Manager app is to make sure that all the child apps are in sync with the Git Repo. For example, a user wants to deploy the Battleships app. The user creates a new Kubernetes cluster, then Argonaut automatically creates a connection to the new cluster using ArgoCD. The Manager App on Argonaut’s cluster now spins up a child app (one for the Org) on the users’ cluster. And whenever there is a change to config in the Git repo, the child app watches it and ensures the cluster configs are in sync with the repo of that org.

The other issue we had to deal with was that the ArgoCD Manager runs on Argonaut’s cluster, but we need to deploy the apps with specified configs on the user’s selected cluster. To do this, we took the external cluster credentials approach.

Now, the cost for the ArgoCD controller running on our cluster and related components is all managed by Argonaut. This should show improved performance to our users at no additional cost.

Monitoring the deployment

For every app you deploy, Argonaut’s backend creates the values.yaml file, and commits it to the internal repo corresponding to your workspace. The ArgoCD instances on your clusters automatically pick up on these changes and deploy them to your cluster.

At this point, Argonaut doesn’t know the state of the deployment, as it is managed by ArgoCD. To get more visibility into this, we use ArgoCD’s Webhooks. As soon as the configuration is saved in our internal repo for your workspace, we listen to the callback from ArgoCD. This then continuously updates the status of your deployment in Argonaut UI. Everything happens in an async way. Each ArgoCD app has the same name as the Deployment ID, and we use these Deployment IDs to map the state of deployment and update it.

Managing secrets

Secrets are managed outside of the ArgoCD instance and Git. We use an internal secret manager for this, with plans to extend this implementation and integrate with other secret managers like Doppler and Hashicorp Vault in the future.

The container registry secrets are generated and stored outside of this as well, and are upserted as needed into the kubernetes cluster.

Conclusion

We’ve been working on this migration for a while now, and we’re excited to share it with you. This enables us to be truly GitOps-powered and also gives us the flexibility to serve scaling teams and infrastructure for our users as we grow.

Specifically, this new setup enables us to add functionality like:

Deploying to multiple clouds, environments, and clusters
Cloning apps and environments
Managing secrets
Managing custom build and deploy pipelines
Preview environments on demand
And more!

We’ve also been able to improve the user experience by making it easier to manage your deployments and secrets.

We’re excited to see what you build with Argonaut. If you have any questions or feedback, please reach out to us on Twitter.

Redpanda: Quickstart to selfhost Redpanda

Argonaut — Tue, 28 Mar 2023 11:00:40 +0000

⏰ Estimated Time: 4 minutes

This guide covers the steps to self host Redpanda on your Kubernetes cluster using Argonaut.

Redpanda is a distributed event streaming platform providing infrastructure for real-time data. It is designed to be fast, scalable, and easy to use. It is built on top of Apache Kafka and is fully compatible with the Kafka ecosystem, while being packaged as a single binary without the need for ZooKeeper or a separate control plane.

This makes for a lightweight and easily maintainable deployment. It provides a number of features that make it well-suited for modern streaming applications, including support for transactions, and a high-performance storage engine.

We will deploy both Redpanda and the Redpanda console.

Prerequisites for installation

Argonaut account with an environment created
AWS or GCP account connected to Argonaut
EKS or GKE cluster provisioned by Argonaut or any other imported Kubernetes cluster

Note: This installation requires Kubernetes 1.21+. Argonaut automatically maintains Kubernetes versions.

Setup Library App in Argonaut

You can add Redpanda as a custom app from Library to Argonaut. This will then be available in your cluster under Add-Ons.

💡 Each Redpanda broker runs on its own worker node and requires a minimum of 3 nodes. This can be changed in Helm chart configuration's podAntiAffinity rules by setting podAntiAffinity.type: soft.

Click on the Application + and the From Library button.
Choose Custom-Apps under configuration.
Ensure the selected environment is correct, then select the cluster you want to deploy the agent to. Important to ensure that the environment and region of this node is the same as all other nodes.
Set the configuration to Custom Apps (Helm)
1. Set namespace as the same name as the cluster (that's where your apps are deployed by default)
2. Release Name as redpanda
3. Chart Name as redpanda
4. Repo Url as https://charts.redpanda.com
The Helm version will be updated automatically.

Make the following changes to values.yaml file.

  # Logging
  logging:
    # Log level
    # Valid values (from least to most logging) are warn, info, debug, trace
    logLevel: warn

Notes:
1. Multiple redpanda instances running in the same cluster will need manual port tweaks to avoid conflicts. This is supported by the chart.
2. Secrets must be used for passwords in production environments.
3. Optionally, enable SASL authentication by editing this in the values.yaml file.
```
  # Authentication
  auth:
    sasl:
      enabled: true
      users:
        - name: superuser
          password: <secretpassword>
```
Click on Install to deploy the app.

Accessing Redpanda

The Kafka interface is exposed on port 9094 by default (listeners.kafka.external.default.port). The Redpanda console is exposed on port 9644 within the cluster (listeners.admin.port).

Components installed

The Redpanda Helm chart will install the following components into your Kubernetes cluster:

A StatefulSet with three Pods.
One PersistentVolumeClaim for each Pod, each with a capacity of 20Gi.
A headless ClusterIP Service and a NodePort Service for each Kubernetes node that runs a Redpanda broker.

Behind The Scenes: Integrating Infracost for Cost Visibility

Argonaut — Thu, 02 Mar 2023 09:32:02 +0000

This article is written with massive help from Prajjwal Dimri who built the infracost integration.

Argonaut enables app deployment and management in Kubernetes on AWS and GCP. Moreover, cloud infra like RDS, s3, CloudSQL, GKE, EKS, etc. can also be provisioned and managed alongside your apps in one place -- across all environments. We aim to provide more visibility into your cloud setup. This article goes through Argonaut’s integration with Infracost for giving users instant cost estimates for their infra resources.

To achieve more visibility into cloud resources, one of the main challenges is getting a sense of cloud costs. FinOps is a fast-growing cross-functional discipline covering tools and best practices to improve cost spending on the cloud. Argonaut also provides its users with real-time cloud cost estimates; this is done with reliable and updated resource pricing from Infracost.

Infracost.io is one of the leading open-source projects for cloud cost, with over 3 million cloud resources pricing across AWS, GCP, and Azure. It can be used by invoking the Cloud pricing API, CLI commands, or through one of its integrations. It can also be self-hosted and used with the Infracost Cloud subscription.

In July 2022, Argonaut launched the cost estimate feature by adopting Infracost to fetch the resource pricing of infra components shipped using Argonaut. We show infra cost for most of the infra components. Few resources having usage-based pricing like S3 are not supported.

Cost visibility in Argonaut

Users can see the cost of infrastructure resources in two places. First, pre-creation, while you create/update a new resource. Second, post-creation, for existing resources with generated terraform files.

Price pre-creation estimates shown before creating an object can be seen below. (Infra create page). This is a quick estimate based on the values you have entered by you. This is not as accurate as the post-creation cost.

The post-creation cost can be seen below. (Infra list page). This version is more accurate as it uses the terraform config files to estimate the cost.

Implementation

To achieve this, we built out a microservice, creatively named costly, that interacts with Argonaut's backend and the Infracost API to provide us with cost estimates. We have two different workflows that use Infracost and costly in different ways.

Note: We strongly profess using a monolith architecture for our backend. We have a single backend that handles all the requests from the UI. This is a conscious decision to keep the backend simple and easy to maintain.

We chose to build a separate microservice to interact with Infracost because we use REST APIs internally and Infracost is a GraphQL API.

`costly` microservice

costly is a microservice created by our team to facilitate easy cost visibility at various stages of infra operation. costly’s job is to aggregate the pricing. (Base price + Storage + compute cost). It also understands the resource type and calculates the total cost by including other factors like region, number of node groups, and attributes based on the service type (e.g., purchase options for Kubernetes clusters).

The costly microservice processes the logic to separate fixed and variable costs based on the resource type. This cost is returned to the user in the Argonaut UI. This cost estimate is a rough estimate based on the values you have entered by you. By hovering over the ? you can see the breakdown between Fixed and Variable costs.

Here is an example of a graphQL query we run in the costly microservice. This shows the processing of requests for an AWS DocumentDB resource. The Product properties are shared by the user, and the return value collected is the price in USD.

query ($region: String!, $instanceType: String!) {
  products(
    filter: {
      vendorName: "aws"
      region: $region
      service: "AmazonDocDB"
      attributeFilters: [{ key: "instanceType", value: $instanceType }]
    }
  ) {
    prices {
      USD
    }
  }

Cost estimation workflows

Argonaut uses two workflows to get you the costs of resources. They are described below:

Quick cost estimates (pre-creation)

This is shown to the user when creating a new infra resource on Argonaut. They appear in the step where you create new infra resources or update an existing infra resource. Perform the following actions to get a quick cost estimate.
Infra > Resource + > Add resource details > See cost estimate

We get the above estimate by using Infracost’s Cloud Pricing API.

This process happens in this sequence.

A request is sent from the UI to the Argonaut backend containing the infra resource parameters. Parameters such as
1. Vendor
2. Service
3. Product Family
4. Region
5. Attributes
This request is forwarded to the costly microservice.
costly microservice processes the logic to separate fixed and variable costs based on the resource type.
The Infracost cloud pricing API uses this information and returns the correct cost values based on the pricing charts available for the requested resource

Infra provisioning along with cost estimate (post-creation)

Actual costs refer to the cost of resources shown once they are created and active. These costs are usually more accurate than the initial quick estimates provided. They have a slightly different flow, as in this step, we use Terraform config files to estimate your infra resources costs.

costly has an additional step where the returned cost is stored securely. A time-series clickhouse database is used here, and each entry is mapped to your account and resource ID. This will be part of upcoming features that provide more ways to save costs.
This process happens in this sequence:

User specifies resource attributes and clicks create resource.
The backend receives the request and provisions the necessary Terraform (TF) files and resources.
Then, initiates the cost calculations using Infracost for all resources in the TF file.
Costs are forwarded to costly along with the resource_id, which is then stored securely by costly. costly also calculates total costs.
This cost is then displayed to the user.

Note: There are two additional scenarios where the post-creation workflow is triggered. These don’t require any user interaction and are automatically handled by Argonaut.

A weekly cron job updates all infra costs based on the latest Terraform files

A safeguard method that can initiate the cost estimation process if no value is found for the resource_id in the storage

Benefits to Argonaut users

Argonaut provides several benefits by integrating infra cost visibility right into the resource CRUD workflows.

It saves the user time and effort compared to visiting other tools such as AWS cost calculator to obtain the relevant costs.
The decision-making process is faster. By viewing the costs in-line, you can decide whether the resource fits within your budget for that service.
The process is automated and based on TF modules (GitOps single source of truth principles), leaving less room for human errors than cost estimates using third-party software.
Argonaut updates its infra and resource costs weekly with a custom cost update service,
Make cost estimates as part of your infra workflow, achieve more savings, and improve your cloud resource utilization.
Supported resources - Argonaut currently provides cost information for these AWS and GCP resources, respectively
1. AWS
  1. Aurora
  2. DocumentDB
  3. EKS
  4. OpenSearch (Elasticsearch)
  5. Elasticache
  6. MSK
  7. RDS
2. GCP
  1. CloudSQL
  2. CloudComposer v2
  3. GKE

Challenges faced during implementation

Here are a few challenges we faced and things we learned integrating costly and infracost. If you’re looking to add infracost to your workflows, do keep these in mind.

Figuring out queries for cloud pricing APIs can be tricky.
The documentation for some of the attributes and values is work in progress.
For Google’s CloudSQL queries, the API returned text-based responses having various attributes. Regex matching was challenging and required additional custom functions to parse the responses.

However, despite these small hurdles, the infracost solution was easy to use, and their active community helped us resolve the issue we had raised on GitHub in a timely manner.

Conclusion

Having visibility into your cloud costs is essential as you set up your cloud resources. Argonaut’s current cost visibility setup with Infracost is just the first step towards providing you with steps to save on cloud costs. Lots more features, such as historical cost insights, tips for cost savings, are planned for the next few months.

Do note that there are other hidden costs associated with cloud operations. More Kubernetes cost optimization strategies can be found here.

Huge shoutout to the Infracost team for building an amazing open-source tool to help us keep track of cloud costs across thousands of different services, products, regions, and configurations.

GitOps Tools: Popular CD and IaC Tools That Enable GitOps Processes

Argonaut — Thu, 02 Mar 2023 09:23:54 +0000

GitOps tools have transformed how teams develop and deploy applications, enabling teams to ship features faster and with greater confidence. Continuing our GitOps series, this piece focuses on the popular tools that help you establish an efficient and reliable GitOps deployment pipeline.

GitOps is an approach where the user can declaratively specify the desired state for both apps and infra in a Git repo. This approach to software development and operations enables teams to collaborate and manage their applications and infrastructure using Git.

By utilizing source control and automation, GitOps automates the entire deployment experience, from code to cloud, providing teams with the ability to manage their applications, infrastructure, and configuration from one centralized location. This has resulted in an improved deployment experience, with more reliable, secure, and faster deployments.

Check out our GitOps Primer article for more information on the basics. This article discusses two essential categories of tools that make GitOps possible - CD and IaC tools.

CD tools for GitOps

These tools enable the main CD pipeline that makes GitOps possible. We are talking about tools that ensure automated deployments in a secure manner and support GitOps Capabilities. These tools also have Kubernetes-native support out of the box.

We will have a future article that provides an in-depth comparison of ArgoCD and FluxCD.

ArgoCD

ArgoCD is a popular open-source solution that’s part of the Argo project. It is a declarative continuous delivery tool for Kubernetes, with a fully-loaded UI and a CLI tool. It stores all configuration logic in Git to allow developers to utilize the code development, review, and approval workflow already connected to Git-based repositories.

Some of its popular features are automating application deployment to target environments, supporting multiple configuration management and templating tools, and enforcing strong authorization with RBAC and multi-tenancy policies. It also offers real-time visibility into application activity, webhook integration, rollouts of complex deployments, access tokens, audit trails, and parameter overrides.

Flux

Flux is a powerful GitOps tool that helps you manage deployments, resources, and integrations with various Git providers and provides multi-tenancy support. It uses a cluster operator to start deployments in Kubernetes, so there's no need for another CD tool.

It does not require any CI access to your Kubernetes clusters, and it provides atomic and transactional changes with an audit log stored in Git for your convenience and security. Additionally, Flux provides an easy-to-use UI that allows you to track and manage the changes you make to your clusters. Its robustness and scalability make it a great choice for businesses of any size.

Several tools like D2iQ Kommander, Giant Swarm Kubernetes Platform, AWS EKS Anywhere are built on top of Flux.

GitLab CI/CD

GitLab CI/CD is a powerful, enterprise-grade CI/CD platform that's designed to help teams create and manage their GitOps pipelines. It provides a number of unique features that make it well-suited for GitOps, such as multi-project pipelines, custom workflows, and the ability to integrate with multiple CI/CD tools. GitLab has also announced its integration of Flux CD.

Additionally, it offers a built-in Kubernetes integration, so teams can leverage the power of Kubernetes to deploy applications and can easily track any changes made to their applications and infrastructure in the CI/CD pipeline. GitLab CI/CD also offers built-in security features, such as authentication and authorization, secrets management, and secure credential storage, that help teams ensure their applications and infrastructure are secure.

Werf

Werf is an open-source, GitOps-based CI/CD solution for Kubernetes developed by Flant. It enables teams to quickly and easily deploy their applications and infrastructure with GitOps and Kubernetes. Werf’s features, such as integrated container image building, automated application deployment, and support for multiple cloud providers, make it suitable for GitOps workflows.

Werf is designed to be easy to use and cost-effective, allowing teams to get up and running quickly and without any additional cost. It also offers a number of integrations with popular tools, such as Helm, Kubernetes, and Docker, as well as the ability to connect to multiple Git providers. Additionally, Werf offers support for advanced GitOps features, such as automated rollbacks, secure deployments, and multi-cluster synchronization.

Weave GitOps core

Weave GitOps Core is an extension to Flux. It provides insights into your application deployments and makes CD with GitOps easier to scale and adopt across your teams. It provides a dashboard with applications, sources, and a Flux Runtime view.

Weave GitOps Core is an open-source CD tool for Kubernetes and cloud-native applications that uses Git-based CD, is Kubernetes-native, has declarative automation, and includes integrations with various tools. Weaveworks also provides Weave GitOps Enterprise, a commercial solution based on the open-source Weave GitOps Core, and is also available via the AWS Marketplace.

IaC tools for GitOps

Infrastructure as Code (IaC) tools are essential for creating and managing the infrastructure that powers applications and services. The two main advantages of using IaC are that it helps automate the process of configuring and deploying cloud infrastructure and is stored as code, making it easier to version control, track changes, and collaborate on deployments.

In the context of GitOps, IaC tools help teams automate the process of configuring and deploying their applications and infrastructure, ensuring that the same version of the application is deployed in production as in development. This helps teams test the configuration of their applications and infrastructure in a reliable and repeatable manner.

An important thing to note is that IaC tools are not GitOps tools.
They are used to create and manage the infrastructure that powers applications and services. However, they can be used in conjunction with GitOps tools to automate the process of configuring and deploying applications and infrastructure. IaC tools in conjunction with a GitOps tool can help teams automate the process of configuring and deploying their applications and infrastructure, ensuring that the same version of the application is deployed in production as in development. This helps teams test the configuration of their applications and infrastructure in a reliable and repeatable manner. Notably, IaC tools do not account for drift by default. However, there are tools that can help you detect and fix drift, with continuous reconciliation. Crossplane has such features built in.

Here are some of the most popular open-source tools. Check out our IaC tools blog for more tools. And here’s an in-depth comparison of CloudFormation, Terraform, and Pulumi.

Terraform

Terraform by Hashicorp is an Infrastructure as Code (IaC) tool that is cloud platform-agnostic and allows users to define cloud and on-prem resources in a human-readable format. It also has plugins that interact directly with cloud providers and SaaS providers, and a registry of publicly available modules. Terraform also enables initializing infrastructure from scratch, defining network resources, and integrating with VCS like GitHub.

It offers features such as an intuitive UI, an extensive library of pre-defined modules, and the ability to define multiple environments for development, staging, and production. Additionally, Terraform supports a wide variety of cloud providers, making it one of the most popular IaC tools.

Pulumi

Pulumi’s Cloud Engineering Platform assists in the Build, Deploy and Manage your infrastructure. Building your IaC using Pulumi allows you to use the languages you love, such as TypeScript, JavaScript, Go, .NET, and YAML to model your cloud infrastructure. It also provides access to a full breadth of services from AWS, GCP, Azure, and over 60 other providers. Your cloud infra code can be reused in the form of Pulumi packages.

Founded in 2017 by Microsoft and Google veterans, Pulumi is a free, open-source infrastructure as code tool. The Pulumi Service offering makes managing infrastructure secure, reliable, and hassle-free. Pulumi service is available as both a self-host or SaaS option. You can easily deliver IaC through your existing CI/CD pipelines. There is also an option to enforce guardrails for security and compliance using policies in standard languages.

You can view the roadmap and contribute to the Pulumi open-source project here. One of their main reasons for success has been their ability to support and help teams migrate to modern containerized, serverless, and Kubernetes workloads easily. Find a detailed comparison between Pulumi and Terraform here.

Crossplane

Crossplane is a framework for building cloud-native control planes without the need to write code. It is built specifically for Kubernetes with support for multi-cloud, serverless, and containerized workloads.

Crossplane provides building blocks that enable you to provision, compose, and consume infrastructure with the Kubernetes API. These individual concepts work together to allow for a powerful separation so that each member of a team interacts with Crossplane at an appropriate level of abstraction.

For enterprises looking for more stability, better support, and reduced risk, there is the Universal Crossplane (UXP) product by Upbound.

Note: Cloud Service Provider (CSP) tools for Infrastructure as Code (IaC) offer an intuitive experience and good integration with existing cloud services. However, they may be limiting if you need to run solutions across multiple clouds.

AWS Cloud Development Kit

AWS Cloud Development Kit (CDK) is an open-source software development framework for defining cloud Infrastructure as Code (IaC) using modern programming languages such as Java, Python, .NET, and TypeScript. This allows developers to create and deploy their infrastructure and runtime code together.

It uses constructs, which are cloud components used to generate AWS infrastructure. AWS CloudFormation powers the provisioning, and you get all the benefits of CloudFormation, including repeatable deployment, easy rollback, and drift detection.

Some of its drawbacks and limitations are that AWS CDK does not currently have support for non-AWS cloud providers, so teams using it will be limited to AWS cloud services. AWS CDK is a relatively new tool, and it may still have some bugs and lack certain features that teams may need. You might need to use other IaC tools, such as Terraform, to supplement.

GCP Cloud Deployment Manager

Google's Cloud Deployment Manager is an infrastructure deployment service that automates the creation and management of Google Cloud resources. It provides users with the power to write flexible templates and configuration files, allowing them to create deployments that include various Google Cloud services, such as Cloud Storage, Compute Engine, and Cloud SQL.

Through the use of these files, users can deploy the resources they need and configure them to work together seamlessly, providing them with an easy and efficient way to manage their cloud infrastructure. Additionally, Cloud Deployment Manager allows users to make changes and updates to their deployments quickly and easily, ensuring that their cloud resources remain up-to-date and in line with their desired configurations.

Azure Resource Manager

Azure Resource Manager is the deployment and management service for Azure. It acts as a management layer that authenticates and authorizes requests from your Azure APIs, CLI, or SDKs before forwarding them to the respective Azure service. Operations such as creating, updating, and deleting Azure resources are supported, and features like access control, locks, logs, and tags help you organize your resources after deployment.

To implement Infrastructure as Code (IaC) for your Azure cloud resources, you use Azure Resource Manager (ARM) templates. These IaC templates are written in Bicep, a domain-specific language with declarative syntax. The Bicep code is converted into JSON files during deployment by the Bicep CLI. ARM templates can also be written directly in JSON.

Azure resources that share a common lifecycle are grouped into resource groups. You can then deploy to the target resource group. All the available resources for Azure services can be found here.

Considerations before using GitOps

To make the most out of GitOps, you will need a process and tooling shift. Here are some practical considerations before adopting GitOps.

Ensure that everyone in the organization is properly trained on the tools and processes used in the GitOps workflow (Git, CD tools, IaC, and Kubernetes).
Establish a clear policy for committing and merging changes into the repository. Especially when it comes to testing and reviewing the changes before they are committed. Make sure the conflicts or issues are identified and resolved quickly.
Maintain the security of the Git repository carefully with the right access controls, proper authentication and authorization, and how secrets or sensitive data are securely stored and managed.
Proper monitoring and backups of the Git repository are essential. This helps to ensure that any changes can easily be tracked and reverted if necessary.
Most often, choosing an open-source tool with an active community and support for multiple cloud vendors is the safer option. As it saves debugging time, is extensible, and can scale with your organization.

By taking the proper precautions and ensuring that everyone is properly trained and equipped with the right tools, organizations can reap the benefits of using GitOps and have a smooth and successful transition into the GitOps workflow.

Conclusion

GitOps is a powerful approach to software development and operations that helps teams collaborate and manage their applications and infrastructure using Git. It automates the deployment experience and helps teams achieve improved deployment experiences with more reliable, secure, and faster deployments.

We have this article provided you with an overview of some of the most popular tools and has helped you understand the benefits of GitOps and the tools available to help you adopt it.

Argonaut is a deployment automation tool that comes with all the benefits of IaC and CI/CD tools in-built. We follow GitOps best practices, provide a secret-management solution, and keep your configurations stored in your repo. Moreover, we have monitoring built-in for cluster and app-level metrics, along with providing access to multiple runtimes, environments, applications, and infra from a centralized dashboard.

Setup External Secrets with Hashicorp Vault on AWS EKS

Argonaut — Wed, 15 Feb 2023 11:23:47 +0000

external-secrets is one of the most efficient and secure ways manage Kubernetes Secrets. external-secrets integrates with many external secret stores like Hashicorp Vault, AWS Secret Manager, etc. and can be used to manage secret files and variables.

Key components

The main components of external secrets are as follows:

External Secrets Operator (ESO) is a collection of custom API resources - ExternalSecret, SecretStore, and ClusterSecretStore that provide a user-friendly abstraction for the external API that stores and manages the lifecycle of the secrets for you.

ExternalSecret - It is a declaration of what data has to be fetched from your external secret manager. It references a SecretStore, which knows how to access the data. You can do more things like set refresh interval, specify a blueprint for the resulting Kind=Secret, use inline templates to construct the desired config file containing your secret, set a target that will be created, and create and delete policies.

SecretStore - They are namespaced by design and cannot communicate with resources across namespaces. This is the file where you select your ESO controller, and cloud provider, along with the role and access IDs, and retry settings in case of connection failure.

The secrets are fetched from an external vault and made available in SecretStores, these are limited to that particular namespace. For use in multiple namespaces, use the ClusterSecretStore.

This guide will go through the process of setting up the External Secret Operator for your AWS EKS Kubernetes cluster.

To find out about other approaches to Kubernetes secrets management, check out this blog.

Pre-requisites

An existing Kubernetes cluster
Kubernetes v1.16.0 or later
Connected Apps or databases that require secrets to access
Hashicorp Vault as your external secret store
kubectl and helm CLI installed
Optional: an Argonaut account and (art cli)

Section I of the installation can be done using both Helm (1.a.) or Argonaut’s UI (1.b.). Choose one of the below section.

1.a. Install using Helm

To set this up, you will need your kubeconfig file. You can use a file created from your existing cluster or obtain this from Argonaut’s CLI.

1: Run art configure generate-aws-credentials

2: Run aws eks update-kubeconfig --name <clustername> --region <region>

Once this is ready, run the following commands from your Terminal.

For Mac and Linux:

export KUBECONFIG=/home/argo/.kube/clustername-kubeconfig.yaml

For Windows:

$env:KUBECONFIG = "C:\Users\argo\.kube\clustername-kubeconfig.yaml"

Confirm your current context by running the command:

kubectl config current-context

You will see an output with the name of your cluster argonaut-cluster. Once the context is set up, you can add external-secrets repo to Helm.

helm repo add external-secrets https://charts.external-secrets.io

You can run a helm update to make sure you’re on the latest version

helm repo update

You should see this output: "external-secrets" has been added to your repositories

You can now install the external-secrets repository

helm install external-secrets \
    external-secrets/external-secrets \
    -n tools \
    --create-namespace \
    --set installCRDs=true

You should now see an install successful message with the name and namespace specified.

1.b. Install with Argonaut

Open Argonaut, and navigate to your desired AWS EKS cluster.
Click on the Application + and the From Library option.
Choose Custom-Apps under configuration.
Ensure the selected environment is correct, then select the cluster you want to deploy the agent to.
Set the namespace you want to deploy the external secrets to be tools (or where you plan to use the secrets).
Set the release name of the application as external-secrets.
Set the chart name as external-secrets.
Set the chart repository as https://charts.external-secrets.io.
Leave the chart version blank. It will be automatically populated to the latest version.
Load the values.yaml file and make any changes (if required).
Then click Install. In just a minute, the external-secrets operator will be added to your cluster, and you will be able to see the outputs as follows.

Now your external-secrets operator is installed in the cluster as an Add-on application. You can view the logs, status, and update the configs by going clicking on external-secrets under Add-ons.

2. Connecting k8s CLI

External Secrets Operator (ESO) is now installed in your cluster. You can start to use it by connecting to your cluster from your Terminal and running the following kubectl commands.

To access your Kubernetes cluster, you will have to generate AWS credentials through art CLI and connect it to your EKS cluster. This is a quick two step process.

1: Run art configure generate-aws-credentials

2: Run aws eks update-kubeconfig --name <clustername> --region <region>

This connects and gives you access of your cluster in the specified region.

Note: The generated access tokens are valid for 12 hours. These inherit the same privileges granted to the Argonaut account for accessing the aws account and infra. To set up more apps such as Mirantis Lens and k9s, follow this docs page.

3. secret-store.yaml

The ClusterSecretStore is a file having the definition of how the external-secret operator can find the secrets from an external secret store. This includes two main things, an existing secret to provide authentication to the external secret store (AWS secret manager linked account) and the AWS project identifier of that particular secret that you wish to access. Make sure you have kubectl installed before proceeding further.

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: "vault-backend"
spec:
  provider: # provider field contains the configuration to access the provider
    vault:
      server: "https://your-domain:8200"
      path: "kv" # Path is the mount path of the Vault KV backend endpoint
      version: "v1"
      auth:
        tokenSecretRef: # static token: https://www.vaultproject.io/docs/auth/token
          name: "vault-token"
          key: "token"

Note: A SecretStore file is also similar but is namespaced and maps to exactly one instance of an external API.

kubectl apply --filename secret-store.yaml

4. external-secret.yaml

This file tells the external-secret operator what specific data is to be fetched from your external secret store. It has three sections:

Secret Store reference - This references the previous resource ClusterSecretStore we created. It tells external-secrets on how to access the resources.
Target - This is the target secret that should be created. This defines the name and type of secret that is created in Kubernetes secret. For example PostgreSQL.
DataFrom - This defines the name of the secret as it is stored in your external AWS secret store.

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: "my-first-secret"
spec:
    secretStoreRef:
    name: vault-backend
    kind: SecretStore  # or ClusterSecretStore
  refreshInterval: "15s" # How often this secret is synchronized
  target: # Our target Kubernetes Secret
    name: my-first-aws-secret # If not present, then the secretKey field under data will be used
    creationPolicy: "Owner" # This will create the secret if it doesn't exist. Options are 'Owner', 'Merge', or 'None'
        deletionPolicy: "Retain"
  data:
    - secretKey: my-first-aws-secret-key
      remoteRef:
        key:message # This is the remote key in the secret provider (might change in meaning based on your provider)
        property:value # The property inside of the secret inside your secret provider

kubectl apply --namespace argonaut --filename external-secret.yaml

Once this is done, the secrets from your external secret store are brought securely to your cluster and are available to access as Kubernetes secrets. This can be checked by executing the following command:

kubectl --namespace argonaut get ExternalSecret my-first-secret

Output:

NAME              STORE           REFRESH INTERVAL   STATUS
my-first-secret   vault-backend   15s                SecretSynced

Note: The secrets are now in Kubernetes and available to anyone with access to the cluster.

Whenever you update a secret directly on your external secret store, it takes a few moments for it to reflect in your Kubernetes cluster. This refresh interval can be set as a part of your configuration.

5. Conclusion

You have now successfully set up external-secrets. This powerful open source tool allows you to manage secrets in a more secure way. You can also choose from various other secret manager tools like the AWS Secrets Manager or GCP Secrets Manager.

The external-secrets approach allows you to securely manage secrets and bring them to your cluster as needed. There are also other approaches to the same that we discuss in this blog. external-secrets can do so much more like generating secrets, CRDs, controller classes, and multi-tenancy. And many more exciting things in their roadmap.

Argonaut has a native secret management solution for scaling teams. Argonaut integrations with third party secret providers is coming soon Q1CY23.

With Argonaut’s modern deployment platform, you can get up and running on AWS or GCP in minutes, not weeks. Our intuitive UI and quick integrations with GitHub, GitLab, AWS, and GCP make managing your infra and applications easy - all in one place.

GitOps Primer: The Benefits, Workflow, and Implementation of GitOps

Argonaut — Wed, 15 Feb 2023 11:12:55 +0000

GitOps is one of the biggest shifts in development methodologies in a long time. Since its introduction in 2017, there has only been a constant growth in this methodology of managing infra and apps. As we mentioned in our 7 cloud trends for 2023 blog, GitOps will continue to see a rise in adoption this year, and we will also see new features and enhancements from the popular tools.

This article is the first one in the series covering GitOps, GitOps tools, best practices, and a detailed comparison of popular GitOps tools.

What is GitOps?

GitOps is the practice of using Git as a single source of truth for declarative infrastructure and applications. It enables users to use Git version control and automate deployment, testing, and operations processes.

Even though some of the GitOps practices were followed for several years, the team GitOps was coined in 2017 by Alexis Richardson. Since its introduction, GitOps has seen rapid growth in adoption, with more and more organizations seeing the value in the improved collaboration, reliability, and security of their applications. In the years since, we have seen the launch of popular tools such as ArgoCD and Flux CD, which have made it easier to implement GitOps in organizations.

In 2022, these tools achieved CNCF graduate status, crossing the chasm and ready to take on enterprise workloads. As more organizations adopt GitOps, the tools, and processes surrounding it are constantly being improved, allowing organizations to reap the benefits of faster and more efficient deployments.

GitOps vs traditional deployments

GitOps is different from traditional deployment methods in a few key ways. Firstly, it utilizes a code repository as the source of truth, with all changes and updates reflected in the repository. This makes it easier to track changes, as well as to roll back any changes if needed. Secondly, it implements Continuous Integration and Deployment (CI/CD) pipelines to ensure that the code that is deployed is always in sync with the code in the repository. It also enables automated rollbacks and disaster recovery in case of unforeseen issues.

Finally, GitOps relies on principles such as immutable infrastructure, declarative configuration, observability, and auditability. These principles ensure reliable and secure deployments and improved collaboration and communication. All of these aspects make GitOps an attractive option for organizations that are looking to improve their DevOps practices.

Key benefits of GitOps

It improves collaboration between developers by allowing them to quickly and easily make changes to the codebase without fear of breaking the system.
It provides a unified platform for managing deployments, with all of the code stored in the same repository. This uniformity makes it easier for developers to understand the process and debug and troubleshoot any issues that may arise.
GitOps also increases reliability and security by enabling automated rollbacks and disaster recovery in case of unforeseen issues. This is possible due to the principles of immutable infrastructure, declarative configuration, observability, and audibility that GitOps relies on.
GitOps also enables faster and more efficient deployments, significantly reducing the risk of human error and allowing teams to get their applications up and running quickly.

The GitOps workflow

The GitOps workflow has been enabled by other foundational technologies, like the rise in the adoption of Kubernetes, the declarative paradigm of defining infrastructure using IaC tools, and containerization. Let us learn about the three main components of the GitOps workflow.

Single source of truth

A key component of GitOps is using a code repository as the source of truth. In this model, all changes to your IaC code (e.g. Terraform) and updates are reflected in the repository, which allows developers to easily track changes and roll back to a previous version if needed. Additionally, the repository being the source of truth, enables developers to share and collaborate on the codebase more efficiently.

CI/CD pipelines

This code repository is also used to set up continuous integration and deployment (CI/CD) pipelines that ensure that the code that is deployed is always in sync with the code in the repository. By utilizing these pipelines, developers can ensure that their code is always up-to-date and that any changes are properly tested and deployed.

The CI/CD pipelines can work on a pull-based or push-based model. A push-based pipeline means that code starts with the CI system and may continue its path through a series of encoded scripts or uses ‘kubectl’ by hand to push any changes to the Kubernetes cluster. In a pull-based pipeline, a Deployment Automator watches the image registry, and a Deployment Synchronizer residing in the cluster maintains its state. In the middle of it all, we have a single source of truth for manifests.

Automated rollbacks and disaster recovery

GitOps also enables automated rollbacks and disaster recovery in case of unforeseen issues. This is done by leveraging the code repository as the source of truth and using CI/CD pipelines to ensure that the code that is deployed is always in sync with the code in the repository. Thus, if any issues arise, developers can quickly roll back to a previous version and recover from any issues without having to manually intervene.

These processes together make GitOps a powerful way of developing and deploying code and infrastructure. Its benefits are even more visible as the complexity of one’s cloud setup grows.

Implementing GitOps

Once you have decided to implement GitOps in your organization, the implementation can be broken down into three main steps.

Set up GitOps pipelines
Automate deployments
Set up monitoring and logging

The GitOps pipelines: There are a few different ways to set up GitOps pipelines, depending on the tools and processes that your organization is using. The most popular tools for setting up GitOps pipelines are ArgoCD and Flux CD, both of which are open-source projects and have achieved CNCF graduate status.

To set up a GitOps pipeline with either of these tools, you need to first create a repository in a version control system such as GitHub. Then, you need to define the CI/CD pipelines with the tools, which will ensure that the code in the repository is always in sync with the deployed code.

Note: The implementation of GitOps requires different toolchains for Apps and for Infra. ArgoCD and Flux are primarily for deploying apps on kubernetes. Crossplane’s implementation is a very clever implementation that leverages Kubernetes control loops to handle both Apps and Infra resources. We’ll cover it in a future blog.

Automating Deployments with GitOps: Once you have set up the GitOps pipelines, you can start automating deployments using GitOps. This process involves setting up the deployment synchronizer, which will watch the image registry and ensure that the state of the cluster is maintained. The deployment synchronizer will then deploy the code from the repository to the cluster automatically, and you can verify the changes by viewing the logs in the repository.

Setting up Monitoring and Logging for GitOps: Once the GitOps pipelines and automated deployments are set up, it is important to set up monitoring and logging for the system. This is necessary to ensure that the system is working as expected and to identify any issues that may arise. Monitoring and logging can be set up using various tools such as Prometheus, Grafana, and ELK stack. These tools allow for tracking of metrics, logs, and errors in the system, which can then be used to identify and troubleshoot any issues that may arise.

Additionally, setting up proper alerting systems is also important to ensure that any issues are quickly identified and addressed. This will help ensure that the GitOps system runs smoothly and that any potential issues are quickly identified and resolved.

Pros and Cons of using GitOps in an organization

GitOps is becoming increasingly popular as an approach to DevOps and is seen as a valuable tool for modern organizations. While there are many benefits to using GitOps, there are also some drawbacks that should be considered before adopting this methodology.

Pros of using GitOps

Improved collaboration and communication between developers due to a single source of truth for the codebase
Processes are lightweight and vendor-neutral
Increased reliability and security due to automated rollbacks and disaster recovery
Faster and more efficient deployments due to automated CI/CD pipelines
Reduced risk of human error due to automation of processes
Improved observability and audibility of the system
Automating infra definition and testing reduces manual work and lowers cost
Faster environment duplication with immutable and reproducible deployments

Cons of using GitOps

Steep learning curve for developers new to the methodology
Dependency on Git and GitOps tools, which can be costly and complex to manage
Complexity of GitOps processes, which can be difficult to debug and troubleshoot
Does not come with a centralized secret management solution
Resistance to change in traditional organizations, as the adoption of GitOps requires a shift in mindset and processes
Git is not designed for programmatic updates, which might cause conflicts requiring manual resolution

In conclusion, adopting GitOps processes is a great way to modernize your DevOps practices. It enables organizations to quickly and easily make changes to the codebase without fear of breaking the system. Based on the above pros and cons list, you’ll see that the adoption of GitOps requires a shift in mindset and processes and can be complex to manage.

The future posts in this series will explore the tools that enable GitOps for app deployments along with an in-depth comparison between ArgoCD and FluxCD.

Start using Argonaut today to take advantage of GitOps best practices and streamline your deployment workflows. Argonaut also provides a single UI for both App and Infra deployments and improves team collaboration.

Secret Management Primer: Challenges, Standards, and Best Practices

Argonaut — Wed, 15 Feb 2023 11:07:48 +0000

Secret management is the process of securely storing and managing sensitive information such as keys, passwords, and tokens. Secrets are used to provide privileged access and are usually stored in secure environments using secret management tools. Secrets are essential to establish the right access and connections on both the app and the infra side of things.

What are secret management tools?

Secret management tools are made for this specific task - to manage, organize, safely store and retrieve secret information. We cover some popular tools in this blog and suggest actionable steps for choosing the right tool to suit your requirements. These tools help address the challenges across SaaS, LaaS, PaaS, private, and hybrid multi-cloud scenarios.

Challenges of secret management

The challenges of managing secrets can occur in the various stages of the secret’s lifecycle, from generation and storing to distribution and revocation.

Secrets at rest can be vulnerable, especially if they are a part of the automated CI/CD pipelines and must be exposed to external tools or stored in the code repository or local device.
Secret distribution to authorized users has to be done securely. This may involve the use of secure communication channels, specific tools (Doppler, 1Password, not slack) or physical measures such as security tokens.
Static secrets that don’t change in value can cause serious security issues as they may easily be shared, and recovering or updating their value can be difficult especially if the same value is used in multiple places.
Compliance: It is important to ensure one’s secret management practices comply with the legal requirements and regulations defined either in data privacy laws or industry standards. We will talk about some industry standards in the best practices section below. Manually managing secrets makes it hard to comply with such standards.
Scalability: As the company scales, the issue of secret sprawl arises, which makes credentials difficult to track and manage, hence more vulnerable to hacking. According to a report from Verizon, stolen credentials account for nearly half of all data breaches.
Human factors: People are an important part of secret management, but they can also be a weak point. Ensuring that individuals handle secrets responsibly and follow established procedures is critical to the system’s security.

Benefits of Secret Management Tools

Secret management tools like Vault, AWS Secret Manager, Doppler, and many others discussed here can help organizations overcome the various challenges of secret management practice discussed above.

Storing secrets: All secret management tools act as a central store for your secrets; this includes passwords, SSL certificates, and API keys. This makes it easy to manage and protect secrets.
Access controls: These tools can enforce access control to ensure that only authorized users access specific secrets. Types of access controls can be based on identity, location, or roles. You can also easily set limits on the count of clients and revoke or update access to any user.
Automated distribution: Secret management tools can automate the process of distributing secrets to authorized users and revoking access when necessary. This can help reduce the risk of human error and improve the efficiency of secret management.
Compliance: Secret management tools make it easier for organizations to comply with regulatory and industry standards, such as data privacy laws and PCI DSS.
Integrations: Their ability to integrate with other systems and tools, such as configuration management tools and CI/CD pipelines, helps streamline the management of secrets in a cloud environment.
Scalability: These tools are built to enterprise standards and grow as your organization grows. Having a centralized store with good access control and compliance policies also reduces the risks of secret sprawl.

Secret Management Best Practices

Once you’ve chosen the right secret management tool and are ready to make the most of the benefits, it’s time to check off these best practices.

Consolidate your secrets and secure them: If you’re using any kind of developer tool or cloud service, you will already be using secrets. Some of them may lie as plain text on your config files or encrypted and pushed to your Git repo. You must first bring them to a secret management tool to secure them.
Rotation: Secrets can be reset or changed based automatically on a schedule. It is a good practice to rotate your secrets constantly. In fact, many compliance standards require that secrets are rotated regularly.
Automation: Secrets no longer need to be hard-coded or embedded. They can be injected directly into the pipeline in most cases and made available for the tools/users that need access to a particular resource.
Create and enforce policy: Cloud security policies are essential components that provide a formal guideline for how a company operates in the cloud. Creating and enforcing these policies help companies reduce the risk and also assures their customers that their data is protected. Here’s how to create a cloud security policy.
Least privilege access: Enforcing least privilege access for machines and humans means that access to the secret is given a need-to-know, just-in-time access to the specific secret for a specified duration.
Scalable solution: Picking a tool that scales as your needs and secrets rise and can work just as efficiently with your future systems and secrets. There is an extra price, but it is worth paying for securely managing secrets.

Some Industry standards for Secret management

There are several industry standards and best practices for cloud secret management:

ISO 27001: Is an international standard that outlines the best practices for information security management, including the secure handling of secrets.
NIST SP 800-53: The National Institute of Standards and Technology (NIST) publishes guidelines for securing federal information systems, including recommendations for secret management.
PCI DSS: The Payment Card Industry Data Security Standard (PCI DSS) is a set of requirements for organizations that handle credit card information. It includes guidelines for securing secrets and protecting against unauthorized access.
Center for Internet Security (CIS) Controls: The CIS Controls are a set of best cybersecurity practices organized into 20 control families. Secret management is addressed in several of the control families, including Access Control and Maintenance.
Cloud Security Alliance (CSA) Cloud Controls Matrix (CCM): The CSA CCM is a framework for evaluating the security of cloud computing environments. It is a set of requirements to certify that all companies that collect and transmit credit card information maintain a secure environment. It also includes secret management controls such as using strong passwords and implementing access controls.
Service Organization Control (SOC 2): This compliance is intended to provide assurance to customers that a service organization has implemented the necessary controls to protect sensitive data. It includes risk assesssments, disaster recovery procedures, confidentiality and integrity control for sensitive data.
Health Information Privacy Protection Act (HIPAA): Under HIPAA, covered entities, such as health care providers and insurance companies, are required to implement policies and procedures for safeguarding protected health information (PHI). This includes encryption, access, and audit controls to protect electronic protected health information (ePHI).

It is important for organizations to understand and comply with relevant industry standards and best practices when implementing cloud secret management. Based on your region of operation, there may be local laws such as CCPA or GDPR.

Check out the other blogs in the secret management series where we talk about top cloud secret management tools and also about kubernetes secret management approaches.

Argonaut has a native secret management solution for small teams. Argonaut integrations with third party secret providers is coming in Q1CY23.