DEV Community: Abhimanyu Selvan

Otomi: Self-hosted PaaS for Kubernetes on Windows (minikube)

Abhimanyu Selvan — Wed, 13 Apr 2022 16:51:20 +0000

Otomi is a self-hosted PaaS for Kubernetes and offers a complete suite of integrated, pre-configured applications combined with automation. Otomi is optimized for running on managed Kubernetes services like AKS, EKS, and GCP, but now it can be installed on any K8s cluster, even on Minikube.

What's new?

The latest version of Otomi, by default, installs a minimal set of apps, called the Core. The core offers an advanced ingress architecture based on Istio, Nginx ingress controller, Keycloak as IdP, OAuth2 Proxy, and cert-manager. With the web UI (Otomi Console) you can add services to the mesh and securely expose them with just one click. All other integrated apps are now optional and can be activated by dragging them into the enabled apps section.

But can we install it locally on our machine?

Yes, you can! It is now possible to install Otomi on minikube running both on Mac and Windows. For Mac installation, check this blog.

In this post, I'll describe how to get started with Otomi on minikube running on Windows. If you ever have run Minikube, you'll probably have used the Docker driver. Unfortunately, you can't access the Minikube IP from your Windows PC when using the Docker driver. Only 127.0.0.1 can be used. This wouldn't be an issue in many cases, but Otomi requires an IP that is resolvable from your machine browser and from within the cluster itself.

Prerequisites

Make sure you have the following installed:

Kubectl
Helm
Docker
Minikube version 1.25.2 (tested)

Tip: If you are using PowerShell, make sure to run as Administrator. I used the chocolatey package manager to install the packages.

Configure Minikube

Open PowerShell as Administrator and run the following commands to configure minikube.

# Set the cpus and memory
minikube config set memory 8g
minikube config set cpus 8
# Enable calico if you want to check network policies
minikube start --driver=hyperv --kubernetes-version=v1.22.4 --cni calico

Enable metallb (network load balancer)

# Enable metallb
minikube addons enable metallb
# Get the IP
minikube ip
# Configure metallb with the IP as seen in the figure below
minikube addons configure metallb

Install Otomi using Helm

From the same PowerShell terminal run the following commands:

# Add the Otomi repo
helm repo add otomi https://otomi.io/otomi-core 
helm repo update
# Otomi install with minimal chart values
helm install otomi otomi/otomi --set cluster.k8sVersion="1.22" --set cluster.name=minikube --set cluster.provider=custom --set apps.host-mods.enabled=false

The helm chart deploys an installer job responsible for installing the Otomi platform on the minikube cluster.

# Monitor the job status
kubectl get job otomi -w
# Installer job logs
kubectl logs jobs/otomi -n default -f

At the end of the logs of the installer job, you will find the URL and the credentials to log into the Otomi console (as seen below)

Otomi Console

Since we install Otomi without providing a custom CA or using LetsEncrypt, the installer generated a CA. This CA is of course not trusted on your local machine. To prevent you from clicking away lots of security warnings in your browser, you can add the generated CA to your keychain. In the left menu of the console, click on "Download CA".

Now we only need to activate Drone:

In the side menu of Otomi Console under platform click on the Drone app
Click on the play button in the top right. A new tab will open for Drone
Sign in locally with as otomi-admin and the password provided in the logs of the installer job
Click on Authorize Application
Click on Submit on the Complete your Drone Registration page. You don't need to fill in your Email, Full Name, or Company Name if you don't want to
Click on the otomi/values repository
Click on + Activate Repository

Done! Now you can create a team, add services, expose them, configure network policies and explore Otomi.

Like what you see? Please support us by joining the stargazers: https://github.com/redkubes/otomi-core

How we leveraged nip.io and custom CA for Otomi

Abhimanyu Selvan — Tue, 15 Mar 2022 12:30:49 +0000

Otomi consists of a complete suite of applications that can be installed in one run on a Kubernetes cluster. It contains several open-source projects like Prometheus, Loki, Istio, cert-manager, external DNS, and much more. All of these applications are configured with sane defaults, making everything work out-of-the-box.

But some of these apps require advanced configuration and dependency management to make them work.

How can we have users try out Otomi with the least amount of effort and without any dependencies?

In this article, I’ll explain how Otomi is configured to be used without the need for a DNS zone. For this, Otomi uses nip.io for DNS name resolution combined with a service LoadBalancer IP, and cert-manager with a custom CA for creating certificates. We not only wanted to demonstrate this solution but also wanted to inspire you to create your test/demo set up by making use of the powerful open-source projects that are available.

The why?

You want your development environment to mirror production as closely as possible. When it doesn’t, you invite more issues showing up in production that didn’t show up in development. Running HTTP when your production site is HTTPS-only is an unnecessary risk.

If you have worked on greenfield cloud-native projects, you would have experienced the pain of testing your application with TLS enabled. It’s easy to spin up an application or micro-service and access this via an IP address or localhost, but this isn’t a fully qualified domain name and doesn’t quite reflect the real-life scenario.

Do you want to manually edit the /etc/hosts file for every new subdomain used for your applications?

Enter nip.io

The nip.io service is a “dead simple wildcard DNS for any IP Address”, and it allows you to map example domain names like otomi.{public-ip}.nip.io to {public-ip}. If you wonder how nip.io knows how to route otomi.{public-ip}.nip.io to my {public-ip}, well, it's because of how nip.io is designed.

It essentially forwards that external DNS request back, telling it to look at the {public-ip}. The advantage is that this is a very easy and powerful solution for generating domain names that map to a local app or Kubernetes cluster.

Otomi + nip.io

When you install Otomi with helm chart installation values, it uses nip.io for DNS name resolution and Self-signed certificates to securely access the application. By default, all the integrated applications are set to the nip.io base domain. Therefore, you can immediately test access without additional configuration such as external-dns.

To configure the application services, Otomi first obtains the external IP address of the load balancer and then uses the public-ip address to configure full qualified domain names (FQDN) in the format https://{application}.{public-ip}.nip.io as shown in the figure below.

Let's take a dig at it

You can see that an A record has been created with the public-ip: 34.147.113.155.

Pretty neat isn’t it?

Custom CA

TLS Certificate management is automatically handled by a cert-manager to manage certificates and issuers (installed with pre-configured chart values).

Otomi automatically generates a Certificate Authority (CA) and stores it as a Kubernetes secret {custom-ca}:

A ClusterIssuer of type CA is then created that references the custom-ca secret:

The custom-ca ClusterIssuer is used to sign new Certificate Sign Requests (CSRs) for each of the integrated applications separately:

Note: There is no manual configuration needed whatsoever and all these are handled automatically by Otomi.

Conclusion

Don't worry, we will test it in Production (NOT!!!)

So if you are want to test your web applications like it is in production, then you can make use of the same projects we use in Otomi to build a secure test/demo setup. The primary disadvantage of using a service like nip.io is that you are relying on an external service, which means your development or test loop is coupled with the SLA (or lack thereof) from this service. Also, the use of a Custom CA that is not publicly trusted is not encouraged but doesn't it make it a perfect solution for testing or evaluation purposes? Let us know what you think!

Useful Links:

The need for a GitOps powered Container Platform

Abhimanyu Selvan — Wed, 19 Jan 2022 16:28:44 +0000

Introduction

Enterprises are more rapidly adopting Kubernetes to accelerate digital transformation efforts in the wake of the pandemic. By implementing cloud-native and open source technologies like Kubernetes, organizations can increase agility and time-to-market.

A new study from D2iQ reinforced the importance of Kubernetes, with 77% of organizations claiming that the container automation system is a central part of their digital transformation strategy.

However, the same study found that while projects in production on Kubernetes are expected to rise 61% in the next two years, almost all organizations that use cloud-native technologies have run into challenges related to the complexity in building and maintaining a container platform.

Challenges with Kubernetes

The journey to Day 2 production operations and Kubernetes success is not an easy one. Day 2 is a DevOps concept that has been around for some time, referring to the development lifecycle phase that follows initial deployment to where the real application demands exist. Challenges in Day 2 are common and complex as operations teams increase the number of nodes and scale applications to keep pace with broader business goals.

The most common challenges organizations face when it comes to adopting Kubernetes are security concerns (47%), difficulty scaling up effectively (37%), and lack of IT resources (34%). Challenges with Kubernetes are not new for developers, as organizations often cite the same challenges for many IT deployments; however, Kubernetes deployment raises the stakes, as it often sits at the center of the cloud-native journeys that are critical to every digital transformation.

Kubernetes challenges are felt by almost everyone, with 94% of respondents claiming **that **Kubernetes is a source of pain or complexity for their organization. However, those in the Kubernetes trenches often feel the brunt of that pain, with 78% of developers and architects claiming that Kubernetes add-ons cause a great deal of pain and introduce complexity.

Challenges with Developers

Almost all (95%) organizations that use cloud-native technology have run into challenges, most commonly during the development phase (47%).

This can often mean long hours, shortening the time for business application development, and high-pressure situations that create a draining work environment on developer teams. According to the study, 38% of developers and architects claim their work makes them feel burnt out, 32% say that building cloud-native applications cause stress, and 28% admit that building applications are very frustrating.

These feelings of burnout, stress, and frustration can drive IT teams to consider drastic measures, including changing their scenery. 51% of developers and architects say building cloud-native applications makes them want to find a new job. This statistic and sentiment should be alarming to organizations as they need full, skilled developer teams to fuel the adoption of Kubernetes and drive accelerated digital transformations.

What is important for an organization?

Reduce developer(s) and team(s) burnouts
Increase team(s) productivity
Shorten time to market
Save costs
Avoid technical debt

The need for a Modern GitOps powered Platform

According to the 2020 State of DevOps Report, 63% of organizations developing software have adopted GitOps as a deployment model. The top benefit of adopting the platform model, according to the report, is the ability to enable self-service capabilities for application development teams. This would enable them to quickly create and provision the resources their code requires without having to have an Ops person do it for them. For the platform team, they can focus on continuous deployment and platform maintenance, and monitoring independently of the application team.

The platform team creates the platform that would be used to provision developer resources. They set up configuration templates for developers to create the resources they need on their own. These templates are in the form of Git repositories and are managed using the GitOps model. They also put in place limits and restrictions, and privileges for how a developer can customize the templates. If those need further customization, they reach out to the platform team. This approach needs some initial effort, but once set up, developers can provide the resources they need in a matter of minutes.

Key Benefits of a GitOps powered Platform

Developer Self Service
Reduces platform team’s workload
- By enabling teams to be self-sufficient(RBAC)
- Integrated logging and monitoring for individual teams
Leverages Automation

-Security is built-in & scalable

-Ready-made audit trail for compliance

MTTR(Mean Time To Repair) as low as minutes
Improves deployment velocity

We feel your pain and we might have the answers to most of your pain points.

Otomi: GitOps powered Container Platform

Otomi is an open-source and cloud-agnostic platform to run on top of Kubernetes to securely deploy, run and manage applications in an automated fashion.

Highlights

Easy to install
- Easily installed within minutes using helm chart install
Out-of-the-box experience
- otomi-console provides an intuitive user experience built on top of the git configuration repository that allows developers to create, configure, manage and monitor containerized applications
- Offers a GitOps way of working, where the desired state is reflected as code and the cluster state is automatically updated
- A baseline configuration for integrated open-source applications and add-ons to support the most common Kubernetes use-cases
Advanced ingress architecture
Multi-Cloud support
- Otomi supports the big 3 public cloud providers (AWS, Azure, and GCP) and on-premise Kubernetes.
- The otomi-quickstart can be used to install Otomi on managed Kubernetes from the cloud providers (Experimentation and Evaluation purposes only)
Security Best Practices
- Workload isolation
- Image vulnerability scanning
- Network policies
- Policy enforcement
- Pre-configured RBAC
- mTLS
- Secret Management

Talk is cheap, show me the code

Sure, you can schedule a demo with me. Let's connect and share the pain together ;)