DEV Community: Alex Vallejo

Google Managed SSL Certificates on Kubernetes

Alex Vallejo — Mon, 07 Mar 2022 22:53:48 +0000

Google Managed SSL Certificates on Kubernetes

Prerequisites: This overview provides a straightforward path for installing Google-managed SSL Certificates on your GKE-hosted application. This assumes you've created a Deployment which runs your uploaded Docker image. It also assumes you have the gcloud command-line tool installed as we'll be working with that to perform our network configurations right from our terminal.

SSL Certificate deployments can range from a simple certbot to a managed wildcard certificate with manual installation. For a Google Cloud hosted application on Kubernetes, you can certainly install and manage your own certificates through the platform, or you can use a Google-managed SLL certificate which will manage the provisioning and autorenewal for you. It's actually extremely easy to do:

managed-cert.yml

apiVersion: networking.gke.io/v1beta1
kind: ManagedCertificate
metadata:
    name: myCert
spec:
    domains:
        - myDomain.com

Create the cert

kubectl apply -f managed-cert.yml

Note the only downside is that only one domain name is permitted for each Google-managed SSL certificate.

That's it, your SSL certificate is now registered with a domain in the Google Cloud. Next we'll attribute the certificate to an Ingress service which will route our traffic for our domain. We use an Ingress object to define route mapping rules for routing HTTP and HTTPS traffic. It essentially creates an HTTPS load balancer to route all our traffic to the appropriate services.

We'll eventually want our domain hitting a static IP address so we'll reserve one and name it something we can reference:

gcloud compute addresses create myApp-ip --global

You can reference this IP through gcloud compute addresses describe myApp-ip --global or you can navigate to VPC Console / External IP addresses and find the IP listed as Static. You can now point your DNS A Record to this IP address, however we'll need to create an Ingress object to map our HTTP and HTTPS traffic.

Before we create our Ingress, we'll be creating a NodePort which provides a gateway port between our public-facing Ingress controller to our cluster's application. A NodePort is, in Google terminology, a Service which simply connects one pod to another. Depending on what port our application is listening on, we can map it to our Ingress via a NodePort. Our NodePort can map directly to our Workload and the cluster will autoscale accordingly.

nodeport.yml

apiVersion: v1
kind: Service
metadata:
    name: myApp-service
spec:
    type: NodePort
    selector:
        app: myApp-workload
    ports:
        - name: myApp-port
          protocol: TCP
          port: 80
          targetPort: 5000

Create the NodePort

kubectl apply -f nodeport.yml

The targetPort is whatever port our application is listening on. Because our Ingress will route traffic on port 80, we'll perform the mapping as such. Lastly, we'll configure the Ingress object which will tie this all together.

ingress.yml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myApp-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: showcase-mde-static
    networking.gke.io/managed-certificates: moviedecisionengine
spec:
  rules:
    - host: myDomain.com
      http:
        paths:
          - path: /
            backend:
              serviceName: myApp-service
              servicePort: myApp-port

kubectl apply -f ingress.yml

This is exciting. We've deployed our Ingress object and we're ready to check the provisioning status of our SSL certificate.

kubectl describe managedcertificate myCert

It may take up to 15 minutes for our SSL certificate to be provisioned on the server.

NGINX Ingress on Kubernetes

Alex Vallejo — Mon, 07 Mar 2022 22:41:21 +0000

NGINX Ingress on Kubernetes

https://www.nginx.com/products/nginx/kubernetes-ingress-controller/

By default, pods of Kubernetes services are not accessible from the external network, but only by other pods within the Kubernetes cluster. Kubernetes has a built‑in configuration for HTTP load balancing, called Ingress, that defines rules for external connectivity to Kubernetes services.

The Ingress controller can then automatically program a frontend load balancer to enable Ingress configuration. The NGINX Ingress Controller for Kubernetes is what enables Kubernetes to configure NGINX and NGINX Plus for load balancing Kubernetes services.

Deploy Ingress NGINX

https://kubernetes.github.io/ingress-nginx/deploy/

First, we'll need to initialize our user as a cluster-admin with the following command:

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin \
  --user $(gcloud config get-value account)

Now we can deploy to our cluster via:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

Running a check on the service might show a Pending status:

alexvallejo:showcase-mde$ kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-7b887b65b7-d99kh   0/1     Pending   0          22m

Quick Google search shows a potential fix that should address that:

https://github.com/kubernetes/ingress-nginx/issues/4775#issuecomment-558381452

kubectl label node --all kubernetes.io/os=linux

https://blog.rcmmd.com/www-to-non-www-with-kubernetes/

To the Ingress service yml:

annotations:
      nginx.ingress.kubernetes.io/ssl-redirect: 'true'

      // this is the required part
      nginx.ingress.kubernetes.io/from-to-www-redirect: 'true'

More Resources:

https://devopscube.com/setup-ingress-kubernetes-nginx-controller/

https://cloud.google.com/community/tutorials/nginx-ingress-gke

https://medium.com/google-cloud/setting-up-google-cloud-with-kubernetes-nginx-ingress-and-lets-encrypt-certmanager-bf134b7e406e

https://stackoverflow.com/questions/48763805/does-gke-support-nginx-ingress-with-static-ip

Phoenix deploys with Elixir 1.9 with systemd (no Docker)

Alex Vallejo — Thu, 21 May 2020 16:50:05 +0000

Elixir 1.9 was released in June 2019 and with it came the exciting feature of releases, which allows for the compilation of your Phoenix application into a releases directory that you can run your application on through a executable command.

The purpose of this article is to deploy a Phoenix app to a server without relying on Docker containers, third-party hosting platforms (e.g. Heroku, Gigalixir). Prerequisites: You should be able to ssh into your host server with sudo access and be comfortable writing environment variables and navigating a Unix environment.

Prior to 1.9, Elixir apps relied on some additional programs to compile the app for deployment. Distillery would be used for the compilation of the application, and could be used in conjunction with edeliver to ssh into the target host server, run the build, download the build locally, and then deploy and run the app on the same server.

Testing our release locally

The documentation is pretty robust for bootstrapping a Phoenix app with Elixir 1.9. The deployment documentation also sets us up for releasing our app in staging and production environments. These steps are quite important and can allow for the deployment of not just our application but also:

Handling environment variables at runtime
Environment-specific configurations (dev.exs and prod.exs)
Running database migrations every time we build a release

I'm not going to get into the details of writing our config files because these things are pretty well documented. When our configurations are all set up for production, we should be able to run the following and see the normal runtime server log and make sure the app is accessible at localhost:4000:

MIX_ENV=prod mix release
_build/prod/rel/my_app/bin/my_app start

Assuming this successfully runs locally, we can then move on to our remote host setup!

Remote Setup

We'll want to perform the following steps on our remote server to make sure we can start pulling code from our git repo and build the executable:

Install postgres if not installed and create a database for you application (if not stateless).
Create a deploy user (adduser deploy) that handles both ssh connections to our local machine and our git repository. Add ssh keys to `/home/deploy/.ssh/authorized_keys'.
Establish a directory that our build will live in, let's say /home/deploy/myApp and pull down the latest code to that directory.
Install Elixir 1.9 and Erlang on our remote server wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb && sudo dpkg -i erlang-solutions_1.0_all.deb
Run mix to make sure Elixir is installed properly
Navigate to our application directory, git pull our phoenix app, and build a release: MIX_ENV=prod mix release
Test the release out by running _build/prod/rel/myApp/bin/myApp start. Is it running as normal? Can you navigate to yourDomain:4000 and see your app? If yes, great! If not, try to work out what is wrong with the configuration. Is the port open on your server? There could be a host (pun) of reasons why our server isn't serving our app correctly so that's outside of this tutorial.

Now that our application can successfully run, we need a way to make it run in the background!! The documentation assumes we can just run a Docker container which is suitable for a lot of use cases, particularly when it comes to scaling, or building clusters via Kubernetes, etc. However, it should be perfectly suitable to build and host a Phoenix app without that isolated environment. You shouldn't need Docker containers to run your app! So let's continue.

Build script

The next step I highly recommend for Phoenix deploys is writing a build script. The documentation briefly covers this through the perspective of a Docker file but we can break this down into a simple bash script that we can run whenever we want to run a release on our remote server.

#!/usr/bin/env bash
# exit on error
set -o errexit

# Initial setup
mix deps.get --only prod
MIX_ENV=prod mix compile

# Compile assets
npm install --prefix ./assets
npm run deploy --prefix ./assets
mix phx.digest

# Build the release and overwrite the existing release directory
MIX_ENV=prod mix release --overwrite

# Perform any migrations necessary
_build/prod/rel/myApp/bin/myApp eval "MyApp.Release.migrate"

This is pretty straightforward. All we're doing is installing our mix dependencies, installing our npm packages, buiding our release, and running any migrations needed. Depending on our CI integration we can adopt this script to run a git pull beforehand too or adjust to however you want CI to be handled.

systemd

And this brings us to systemd. While the history of systemd is somewhat controversial, it has been the de facto system and service manager for Linux distributions since 2014 and is super easy to set up and initialize our application with.

For the basic application I want to set up, it accomplishes three critical things:

Runs the application in the background
Auto initializes the application if the server reboots
Provides logs for debugging and reference

I'm sure the use cases for a more complex configuration are out there, but my goal right now is limited to the above.

There actually is a mix library that allows you to generate a systemd unit file for your Phoenix application but it's simple enough that you can write one from scratch or copy and paste the following to /etc/systemd/system/myapp.service:

[Unit]
Description=myApp service
After=local-fs.target network.target

[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/home/deploy/build/myApp/_build/prod/rel/myApp
ExecStart=/home/deploy/build/myApp/_build/prod/rel/myApp/bin/myApp start
ExecStop=/home/deploy/build/myApp/_build/prod/rel/myApp/bin/myApp stop
EnvironmentFile=/etc/default/myApp.env
Environment=LANG=en_US.utf8
Environment=MIX_ENV=prod


Environment=PORT=4000
LimitNOFILE=65535
UMask=0027
SyslogIdentifier=myApp
Restart=always


[Install]
WantedBy=multi-user.target

We'll want to additionally create a myApp.env file at /etc/default so that our service can use runtime environment variables for our application. In my file I simply have the following:

PORT=4000
HOSTNAME="myApp.io"
SECRET_KEY_BASE="[output of mix phx.gen.secret]"
DATABASE_URL="ecto://postgres:password@myApp.io/[dbName]"

Now that our systemd is configured, we can start our service. Here are some useful commands to play around with:

After making changes to the systemd unit files:
sudo systemctl daemon-reload

Start your service:
sudo systemctl start myapp.service

List systemd services:
systemctl list-units --type=service

Check the status of your service:
systemctl status myapp.service

This one is key because it will tell you if the service was able to start successfully. If it has, your app should be up and running! If you app isn't properly running we can check the logs via this command:

sudo journalctl -f -u myapp.service
sudo journalctl -u myapp.service --since today

Seems to be a very flexible logging service as well.[^1](https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs)

Environment Variables

systemd scripts are executed normally at the root level, so the env vars need to be acccessible by root. Originally I had my deploy@dayoff.io env vars set for the deploy user but my systemd didn't have access to them.

To set my env vars for runtime, I placed them in /etc/default/dayoff.env, which included my PORT, DATABASE_URL, and SECRET_KEY_BASE. I'm still a little bit uncertain if you need your vars accessible at the build directory by the deploy user so you may additionally need to export these vars at ~/.profile or ~/.bash_profile.

Thanks for reading and I hope to get an article out next time around setting up nginx on the server (and maybe Nginx Ingress in Kubernetes).

Useful Links

https://hexdocs.pm/phoenix/releases.html#containers

https://github.com/cogini/mix_systemd

https://github.com/cogini/mix-deploy-example

https://render.com/docs/deploy-phoenix

https://github.com/SolarisJapan/lunaris-wiki/wiki/Deploy-Phoenix-Project

https://github.com/phoenixframework/phoenix/blob/master/guides/deployment/releases.md#runtime-configuration

https://gist.github.com/davoclavo/61b9d84f2248f182c95ae7738490ddd1

https://hexdocs.pm/phoenix/releases.html#ecto-migrations-and-custom-commands

https://elixirforum.com/t/elixir-apps-as-systemd-services-info-wiki/2400

https://medium.com/@mcsonique/deploying-elixir-phoenix-projects-to-production-44a236c643c

https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units