DEV Community: Porter

Deploy Strapi on AWS/GCP/Digital Ocean using Porter

Trevor Shim — Fri, 06 Aug 2021 16:39:42 +0000

Intro

Porter is a Platform as a Service (PaaS) that runs in your own cloud provider. It brings the convenience of platforms like Heroku, Netlify, and Vercel into a cloud provider of your choice. Under the hood, Porter runs on top of a Kubernetes cluster but abstracts away its complexity to the extent that you don't even have to know that it's running on Kubernetes.

This is a quick guide on how to deploy Strapi to a Kubernetes cluster in AWS/GCP/DO using Porter. This guide uses PostgresDB by default - to customize your database settings, modify the files in /app/config/env/production in the example repository.

Quick Deploy

Create an account on Porter.
One-click provision a Kubernetes cluster in a cloud provider of your choice, or connect an existing cluster if you have one already.
Fork this repository.
From the Launch tab, navigate to Web Service > Deploy from Git Repository. Then select the forked repository and Dockerfile in the root directory.
Configure the port to 1337 and set environment variable to NODE_ENV=production. Depending on your database settings, you might want to add more environment variables. More on this in the section below.
Set the assigned resources to Strapi's recommended settings (i.e. 2048Mi RAM, 1000 CPU), then hit deploy!

Deploying PostgresDB

Strapi instance deployed through Porter connects to a PostgresDB by default. You can connect Strapi instance deployed on Porter to any external database, but it is also possible to connect to a database that is deployed on Porter. Follow this guide to deploy a PostgresDB instance to your cluster in one click.
After the database has been deployed, navigate to the Environment Variables tab of your deployed Strapi instance. Configure the following environment variables:

NODE_ENV=production
DATABASE_HOST=
DATABASE_PORT=5432
DATABASE_NAME=
DATABASE_USERNAME=
DATABASE_PASSWORD=

To determine what the correct environment variables are in order to connect to the deployed database, see this guide.

Development

To develop, clone the example repository to your local environment and run npm install && npm run develop; from the app directory. Porter will automatically handle CI/CD and propagate your changes to production on every push to the repository.

Questions?

Join the Porter Discord community if you have any questions or need help.

Datadog on Kubernetes: Avoiding Common Pitfalls

Trevor Shim — Sat, 17 Jul 2021 00:08:21 +0000

Datadog is a powerful monitoring and security platform that gives you visibility into end-to-end traces, application metrics, logs, and infrastructure. While Datadog has great documentation on their Kubernetes integration, we've observed that there's some missed nuance that leads to common pitfalls.

This blog post will guide you through how to install the Datadog agent on Kubernetes and enable additional features such as DogStatsD and APM via Helm and Porter while avoiding these common pitfalls:

Not running the Datadog Daemonset on all nodes
Not adding admission label to your application pods you'd like to use DogStatsD and APM with.
Overriding environment variables that have been injected by the agent
Not setting resource limits on the agent to avoid crashing the node/kubelet

Basic Installation

We will use Helm to install the Datadog agent with the default set of values. The default values in Datadog's helm chart, along with Datadog's Autodiscovery feature, is sufficient to give visibility into all cluster level metrics.

These are the commands to install the Datadog agent in your cluster using Helm v3 with the default values. Make sure to copy your API key from Datadog dashboard in the install command.

helm repo add datadog https://helm.datadoghq.com
helm repo update
helm install <RELEASE_NAME> --set datadog.apiKey=<DATADOG_API_KEY> datadog/datadog

Common Pitfall #1: Daemonset not running on all nodes

The Datadog agent is installed as a Daemonset. This means that the agent is designed to run on every node of your Kubernetes cluster. By default, the Datadog agent will assume that none of your nodes has a taint. If you've added taints to some of your nodes, the agent will run and ingest data only on the nodes that do not have taints.

To ensure that the agent runs on every node, you must modify the value agents.tolerations to accommodate all your nodes. You can check the taints on your nodes by running the following command.

kubectl describe node <NODE_NAME>

For example, Kubernetes does not run daemonsets on the master node by default. If there is a master node in your cluster and application pods can be scheduled on it, you need to add the following toleration in the agents.tolerations value of the Datadog Helm chart to retrieve metrics from those pods. Make sure to add the appropriate tolerations based on your node taints.

agents:
    tolerations:
    - key: node-role.kubernetes.io/master
      operator: Exists
      effect: NoSchedule

Enabling DogStatsD and APM

The default installation above exports all cluster level metrics, but you might want to use DogStatsD to track custom application metrics and APM to collect application traces. If you've used DogStatsD or APM before without Kubernetes, you are probably used to manually injecting environment variables like DD_AGENT_HOST for your application code to consume.

On Kubernetes, the Datadog agent automatically injects DD_AGENT_HOST as an environment variable into all your containers so you don't have to make additional configuration to export custom metrics. Ensuring that the agent is running on every single node of your cluster is essential to make this work, because the IP address of the pod's host node will be injected.

To automatically inject these environment variables into your pods, you need to enable the Datadog admission controller. You can do this by modifying these values on the Datadog Helm chart.

clusterAgent:
   admissionController:
      enabled: true
      mutateUnlabelled: false

Common Pitfall #2: Not adding the admission label to your pods

For the admission controller to inject environment variables into your pod, you must add the following label to the pods you'd like to use DogStatsD in: admission.datadoghq.com/enabled: "true". If you'd rather inject the environment variables to all pods running on your cluster, you can also just set the admissionController.mutateUnlabelled value to true.

You can verify that the environment variable has been injected to the pod by running the following command:

kubectl exec -it <POD_NAME> -- printenv | grep "DD_AGENT_HOST"

If the admission controller is working as expected, you'll see the IP address of the node that the pod is running on. You can confirm this by running:

kubectl describe pod <POD_NAME>

The Node field should match the IP address printed above.

Common Pitfall #3: Overriding the injected environment variable

The environment variables injected by the admission controller do not take precedence over the environment variables set on the pod. If you already have DD_AGENT_HOST defined on the pod, that value will override what's automatically set by the Datadog agent. Make sure to remove both DD_AGENT_HOST and DD_ENTITY_ID from your pod configuration for the agent to work as expected.

Now that you have the admission controller working, modify the following values on the Helm chart to enable DogStatsD and APM:

dogstatsd:
   port: 8125
   useHostPort: true
   nonLocalTraffic: true
datadog:
   apm:
     enabled: true

Common Pitfall #4: Not setting resource limits on the agent

The default set of values on Datadog do not have any limits set on the resources it can consume. This may result in a node or kubelet crash if the agent consumes more resources than what its host node can accommodate. The resource limit you should set depends on the size of your nodes and the operations you are performing on the agent. There are many places that these resource limits can be set, but we've found that in most cases the agent pod itself is the primary consumer of resources. You can set the limits on the agents with the following values:

agents:
   containers:
      agent:
         resources:
            limits:
               cpu: 200m
               memory: 256Mi
            requests:
               cpu: 200m
               memory: 256Mi

Then run the install command again with the values above. You can check from your Datadog dashboard whether custom metrics and application traces are being received properly.

Summary

We went over how to install the Datadog agent via Helm and configure additional features like DogStatsD and APM while avoiding common pitfalls. This is the annotated values.yaml file that you can use as reference:

agents:
   containers:
      agent:
         resources:     
            limits:  # <-- set limits on the agent as necessary
            requests:
    tolerations:
    - key: node-role.kubernetes.io/master # <-- Add if nodes have taints
      operator: Exists
      effect: NoSchedule
clusterAgent:
   admissionController:
      enabled: true
      mutateUnlabelled: false # <-- `true` if you want to inject env vars to all pods
datadog:
   apiKey: <API_KEY> # <-- Your api key, can be found in Datadog Dashboard
   apm:
      enabled: true
   logs:
      enabled: true
      containerCollectAll: false # <-- `true` if you want to collect logs from all containers
dogstatsd:
   port: 8125
   useHostPort: true
   nonLocalTraffic: true

Installing via Porter

Porter is a platform as a service (PaaS) that runs in your own cloud, specifically in your own Kubernetes cluster. On Porter, you can install the Datadog agent as a one click add-on and enable DogStatsD, APM, and logging by a simple toggle.

1.Navigate to the launch tab and select the Datadog add-on.

2.Put in your ingestion key, toggle the features you'd like to use, then hit deploy!

Deploy Django on Kubernetes in a few clicks (without even Dockerizing your application)

Trevor Shim — Mon, 07 Jun 2021 22:08:49 +0000

This blog article is based on the documentation and example repository written by Porter's community member, jimcru21.

Kubernetes is a powerful container orchestrator that automates deployments, management, and scaling of containerized applications. Despite all these benefits of Kubernetes, however, there is typically a ton of overhead to it that is often not justified for simple applications. In this tutorial, we go over how to deploy Django applications on major cloud providers' Kubernetes offerings (e.g. EKS, GKE, DOKS) in a few clicks using Porter, without even having to containerize your applications.

What is Porter?

Porter is a Platform as a Service (PaaS) that runs in the user's own cloud. If you're familiar with Heroku/Vercel/Netlify, Porter brings the ease of use of those platforms into your own cloud, particularly into your own Kubernetes cluster. With Porter, you can deploy and scale Django applications on Kubernetes with minimal overhead without having to write a Dockerfile or YAML files.

Porter is open source. Check out the source code here.

Provisioning the Kubernetes Cluster

Before you can start deploying a Django application on Kubernetes, you must first provision a Kubernetes cluster. With Porter, it's possible to create a cluster on AWS, GCP, and Digital Ocean with a single click. Follow this guide to provision the cluster in the cloud provider of your choice.

Alternatively, you can also provision a cluster on your own and connect Porter to an existing cluster. These are the guides on how to create your own cluster for each cloud provider:

After you've created a Kubernetes cluster, you can connect to it via the Porter CLI per this guide.

Prepare Django Application

While it is not necessary to containerize your Django application to deploy it through Porter, you must follow these steps for a successful deployment.

Install django-allow-cidr (this is the middleware to enable the use of CIDR IP ranges in ALLOWED_HOSTS)

pip install django-allow-cidr

Go to Django Settings and add os.environ.get in allowed host.

ALLOWED_HOSTS = os.environ.get("DJANGO_ALLOWED_HOSTS", default='127.0.0.1').split(" ")

Add allowed CIDR networks. Put CIDR according to the Kubernetes kubelet CIDR. If you've provisioned the cluster through Porter, it is set to 10.99.0.0/16 by default. If you've provisioned the cluster yourself, visit your cloud console to find the CIDR.

ALLOWED_CIDR_NETS = os.environ.get("ALLOWED_CIDR_NETS", default='10.99.0.0/16').split(" ")

Add django-allow-cidr middleware to the application.

MIDDLEWARE = [
  'allow_cidr.middleware.AllowCIDRMiddleware',
  #'django.middleware.security.SecurityMiddleware',
]

Add Gunicorn.

pip install gunicorn

Add static folder and add your HTML and CSS files. Locate static URL settings and add static file dirs below:

STATICFILES_DIRS = (
  os.path.join(BASE_DIR, 'static'),
)
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

Add a Procfile to your repository. This is an alternative configuration to Dockerfile that uses Cloud Native Buildpacks, which have been popularized by Heroku. Porter uses the Procfile to build your images if a Dockerfile is not present.
Lastly, generate a requirements file.

pip freeze > requirements.txt

Sample repository for the above configuration: https://github.com/jimcru21/porter-sample-django-non-docker

Deploy Django on Porter

Visit the Porter Dashboard. Click Web Service > Launch Template.
Name your application.
When prompted for the deployment method, click Git Repository. Connect your GitHub account and select the repo you'd like to deploy from.
Select the branch (main in the example below), then hit Continue.
Porter will read your Procfile and prompt you for the name of the process you'd like to run. In the example above, the process is named web. By default, Porter stores your build artifacts in the registry that was provisioned by Porter. If you've connected to an existing cluster, you can also connect an existing container registry to Porter using our CLI per this guide.
In Additional settings, specify the container port that you use for gunicorn in the Procfile (in the example above, this is set to 8989). You can also configure custom domain per this guide.
From the Environment tab, set DJANGO_ALLOWED_HOSTS that we specify on Django settings. Then input the domain you have set for your application.
Click Deploy then wait for buildpack to finish and push to porter. Porter writes a GitHub Actions file to your repository for Continuous Integration. You can check the build progress on your GitHub repository under the Action tab.

Done!

Kubernetes can be a mountain to climb for newcomers, but many developers are drawn to its benefits despite the steep learning curve. Porter is a tool that eases that learning process to the extent that the user doesn't have to learn anything about Kubernetes, or even Docker, to get started.

If you have any questions about this tutorial, join our discord community and ask away!

Introducing Porter - move your localhost into the cloud

Trevor Shim — Wed, 05 Aug 2020 16:17:16 +0000

Think Heroku meets Codespaces. A quick tutorial on how to get started on Porter with a simple MERN stack.

Setting up development environments and deploying them into production are probably the two biggest wastes of time of any development cycle. Configuring and installing modules on your local machine often goes wrong, and we have all experienced the headache of trying to deploy an application that works on your own machine but breaks in production.

With platforms like Vercel and Netlify, developing and deploying JAM stack applications are getting easier and easier. But what if you want to work on something more complex that requires multiple databases, caches, and workers? Heroku and Render do a better job automating a lot of the DevOps when you are working on more complex projects, but you still have to do the work of maintaining a development environment on your local machine and shipping it to production.

What if, instead, development happened in the cloud so that you don't have to run or install anything on your local machine? What if you could ship your application into production with a single click from the same platform you were developing on?

What is Porter?

Porter is a platform that lets you develop applications in the cloud while using your favorite local IDE. You can also host the apps you've developed on Porter with a single click from the same platform.

If you are familiar with Heroku, you can think of apps on Porter as Dynos you can directly develop in. This means your localhost now lives on a URL that you can publicly share, and your application's entire lifecycle from development to production happens on a single platform with no local set-up whatsoever.

Let's see this in action with a simple MERN stack example.

1. Create Containers

Everything that runs on Porter runs as Docker containers. If you aren't familiar with containers, you can think of them as lightweight VM's that are completely isolated from your host machine. These containers let you develop and ship code in consistent and predictable environments. In Heroku terms, containers behave a lot like Dynos, except Dynos were created before Docker came around and ate up the software industry. Porter is a modern PaaS (Platform as a Service) built on top of the Docker ecosystem.

First, login to the Porter Dashboard. You will be prompted to create a project, then you will see the "Create a Container" button. When you click that button, you will see a variety of templates Porter provides by default. To spin up a MERN stack, simply select the React and MongoDB templates as shown below.

It may take up to a minute for these containers to load. Once they've loaded, each container will display a URL. This URL effectively acts as your localhost. In the case of the react container, the URL serves a simple create-react-app template as shown here: https://dev-to-react-porter-dev-bwxork4w.getporter.dev/ (Note that this URL is my development environment!).

In the case of the mongo container, the URL is a connection string that other containers can access as long as they are within the same project. We'll come back to this later.

2. Download the Porter CLI

I may have lied a little bit when I said you don't have to install anything on your local machine. You do need to install one tiny CLI to start developing in these remote containers, but I promise this is the only thing you'll need to install.

npm install -g porter-cli

The above command is for MacOS. For Windows and Linux, check here.

Porter lets you develop on remotely running containers without having to use a WebIDE. You can use whatever local editor you want to use. Our CLI will sync your local files with the files in the remote container so that every local edit you make gets propagated into the remote container.

First, login via the CLI:

porter login

Then create and navigate to an empty local directory and run

porter init

This will prompt you to select a container you want to link your local directory with. Select the name of the react container you have just created.

Once you have linked up the directory, you can start syncing your local file system with the remote container via

porter sync --bi-dir

This will start a bi-directional syncing process and download all the contents in the remote container into your local folder. Now, every change you make on your local directory will be instantly propagated into the remote container, and the container URL will hot-reload whenever there's a change.

To see the logs from the create-react-app process running inside the container, run

porter logs

3. Linking up MongoDB

Spinning up an entirely new MongoDB instance is arbitrary on Porter. If you want to start fresh, just make a new instance by clicking on the MongoDB template again. The connection string displayed on the container is available only to internal traffic - i.e. only the other containers that are within the same project can access this database. Copy this connection string by clicking on the icon boxed below.

To connect to database, you need to first npm install mongodb --save in the react container. To install new modules in this container, you simply need to run

porter enter sh

This will open up a shell inside the remote container, and you can run the commands you'd normally run on your local machine inside this remote shell, such as npm install. These kinds of commands might require more RAM and CPU than what is assigned to the container by default. You can allocate more resources from the "Resources" tab in container setting.

Now, you can use this connection string from the react container to connect to the database. To test the connection, let's go back to /index.js in the react container and add these lines of code.

var MongoClient = require('mongodb').MongoClient

// Connection URL you have copied
var url = 'mongodb://mongo-db-0.mongo-db.porter-dev-bwxork4w:27017/';

MongoClient.connect(url, function(err, db) {
  console.log("Connected successfully to mongodb");

  db.close();
});

You'll see the log message that confirms that proper connection has been established between the react and mongodb containers.

4. Using environment variables on Porter

Instead of using .env files, you can directly inject in environment variables into the container via the Porter Dashboard. As an example, click on the react container and navigate to the "Environment" tab. Write in a new MONGO_URL variable with the URL that you have copied from the mongodb container. The container will restart, and when you swap out the connection string and console.log the connection string, you will see the URL in the logs as you'd expect.

var MongoClient = require('mongodb').MongoClient

MongoClient.connect(process.env.MONGO_URL, function(err, db) {
  console.log(process.env.MONGO_URL);
  console.log("Connected successfully to mongodb");

  db.close();
});

In the logs, you will see that the connection has been established once again.

Try it out!

You can try setting up a simple MERN stack on Porter for free here: https://dashboard.getporter.dev. Here are the docs for your reference.

We can't wait to see the projects you create on Porter!

Wordpress Development on Porter

Trevor Shim — Mon, 03 Aug 2020 23:26:52 +0000

Local development for Wordpress often requires a lot of configuration and installation of tools like PhpMyAdmin, MySQL, and Wordpress itself. Docker helps compartmentalize each of these components with containers. Simply put, containers are lightweight VM’s that provide environments that are isolated from your host machine — if you are familiar with heroku, you can think of them as Dynos. Docker lets you spin up and manage Wordpress stacks for your various projects without the headache of maintaining and configuring these technologies on your computer.

Despite Docker’s ease of use, however, there is still a bit of learning curve that prevent Wordpress developers from taking advantage of containers. Porter lets you develop Wordpress themes and plugins using Docker, without having to configure or even install Docker.

What is Porter?

Porter is a remote development environment that lets you develop Docker containers in the cloud while using a local IDE of your choice. Using the Wordpress template provided on Porter, you can start developing themes and plugins without configuring or installing anything locally, then ship them into production with literally one-click.

1. Create containers on Porter

Everything on Porter runs as Docker containers. Let’s create Wordpress and MySQL containers on Porter using the default templates. You can do this by logging into Porter’s dashboard, then clicking on the “Create a Container” button and choosing the corresponding templates.

It may take up to a minute for these containers to load.

2. Connect the containers

Once you have created these containers, you need to make them talk to each other — this can be done by changing the environment variables of each container. Click on the MySQL container you have created and change the environment variables MYSQL_ROOT_PASSWORD, MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD to values of your choice from the “Environment” tab. All variables are initially set to "default”.

Once you have configured the MySQL container, copy the connection string displayed in the container by clicking on the icon boxed below.

Now click on the Wordpress container, then change the environment variable WORDPRESS_DB_HOST from “default” to the connection string you have just copied — make sure WORDPRESS_DB_PASSWORD also matches the MYSQL_ROOT_PASSWORD you have put in the MySQL container.

Once you hit save, the Wordpress container will restart and soon the URL on the container will display the familiar Wordpress installation page.

3. Develop on remote containers

The above URL is now effectively your localhost. But how do you develop on these remotely running containers? All you need to install is the porter-cli that will let you develop remotely using your own local IDE by syncing your local files with the files in the remote container. Let’s see this in action.

Install the CLI with npm i -g porter-cli if you are using MacOS. For other OS, see our docs.
Run porter login

Make an empty local directory and run porter init inside that directory. When prompted to choose a container, select the name of the Wordpress container you have created.
Run porter sync --bi-dir from the same directory — this will start a bi-directional syncing process between the directory and your remote container.

Once all the folders have been downloaded from the remote-container via sync, start developing your themes in the wp-content/themes directory that has just been downloaded.

Any local edit you make in that local directory will be instantly propagated into the remote container that you can view on the above URL. As long as the syncing process is ongoing, the remote container will behave exactly like your localhost. Just run porter sync --bi-dir again when you want to start developing.

You can share your works in progress with clients and collaborators by simply giving them the URL you are developing on. The containers you develop on Porter can also be promoted to production with your custom domain in just one-click.

You can try developing Wordpress themes on Porter for free here.