Introducing appfile: a declarative way of managing apps in DigitalOcean App Platform

I have been experimenting with DigitalOcean App Platform for a while and I like how it helps me focus on defining only what I need to run my apps. Using the app.yaml spec, I can declare the app components and store it within the project codebase. Soon though, I started to run into the problem of how to manage different environments for the same application (e.g. review, staging and production).

After unsuccessfully searching online for anything that would fit my use case, I figured I would solve the problem myself. I wanted a tool that would allow me:

• Declare the different environments for a given App specification
• Have diff capabilities
• Deploy multiple apps at once

After a couple of days of tinkering, I had an up and running the first version of appfile. If you want to go straight to the code, check the repo at renehernandez/appfile.

Ready? Ok, let's discuss what appfile is all about.

Features #

The main capabilities that I set out to have and are implemented as of the current version (v0.0.2) are outlined below:

• Declare the different environments for a given App specification
• Support templates to customize the final app specification based on the selected environment
• Have diff capabilities
• Deploy multiple apps at once

CLI #

The full CLI help can be seen by either typing on the terminal:

$ appfile

Or:

$ appfile --help

It will output help information like:

$ appfile

Deploy app platform specifications to DigitalOcean

Usage: 
  appfile [command]

Available Commands: 
  destroy Destroy apps running in DigitalOcean 
  diff Diff local app spec against app spec running in DigitalOcean 
  help Help about any command 
  sync Sync all resources from app platform specs to DigitalOcean

Flags: 
  -t, --access-token string API V2 access token
  -e, --environment string root all resources from spec file (default "default")
  -f, --file string load appfile spec from file (default "appfile.yaml")
  -h, --help help for appfile
  --log-level string Set log level (default "info")
  -v, --version version for appfile

Use "appfile [command] --help" for more information about a command.

The available sub-commands are:

• appfile sync: Sync all resources from app platform specs to DigitalOcean
• appfile diff: Diff local app spec against app spec running in DigitalOcean
• appfile destroy: Destroy apps running in DigitalOcean

Github Action #

There is also a Github Action that you can use to automate the deployment of Apps to DigitalOcean with appfile. Check the action-appfile Action at:

• Marketplace: https://github.com/marketplace/actions/github-action-for-appfile-cli
• Repository URL: https://github.com/renehernandez/action-appfile

Installation #

Currently, you would need to install appfile by downloading a corresponding release from the latest Github release for your platform of choice.

For Mac:

$ wget https://github.com/renehernandez/appfile/releases/latest/download/appfile_darwin_amd64
$ chmod +x appfile_darwin_amd64
$ mv ./appfile_darwin_amd64 /usr/local/bin/appfile

For Linux:

$ wget https://github.com/renehernandez/appfile/releases/latest/download/appfile_linux_amd64
$ chmod +x appfile_darwin_amd64
$ mv ./appfile_darwin_amd64 /usr/local/bin/appfile

For Windows:

> Invoke-WebRequest -Uri "https://github.com/renehernandez/appfile/releases/latest/download/appfile_windows_amd64.exe" -OutFile appfile.exe
> $env:Path += "./appfile.exe"

Usage #

Let's look at the following example to start seeing the power of appfile. We want to deploy a Rails application to the DigitalOcean App Platform. This Rails app would have different components depending if we are deploying to production or a review environment.

To start, we need to define our appfile.yaml spec:

# appfile.yaml
environments: 
  review:
  - ./envs/review.yaml
  production:
  - ./envs/production.yaml

specs:
- ./app.yaml

The above spec lays out that our App has 2 environments to get state values from: review and production from the ./envs/review.yaml and ./envs/production.yaml files respectively. It also defines that the App spec is located at ./app.yaml

Let's take a look a the app.yaml definition:

# app.yaml
name: {{ .Values.name }}

services:
- name: rails-app 
  image: 
    registry_type: DOCR 
    repository: <repo_name> 
    tag: {{ requiredEnv "IMAGE_TAG" }}
  instance_size_slug: {{ .Values.rails.instance_slug }}
  instance_count: {{ .Values.rails.instance_count }}
  envs: 
{{- range $key, $value := .Values.rails.envs }}
  - key: {{ $key }}
    value: {{ $value }}
{{- end }} 

{{- if eq .Environment.Name "review" }}
- name: postgres
  image:
    registry_type: DOCR
    repository: postgres
    tag: '12.4'
  internal_ports:
    - 5432
  envs:
{{- range $key, $value := .Values.postgres.envs }}
  - key: {{ $key }}
    value: {{ $value }}
{{- end }}
{{- end }}

jobs:
- name: migrations
  image:
    registry_type: DOCR
    repository: <repo_name>
    tag: {{ requiredEnv "IMAGE_TAG" }}
  envs:
{{- range $key, $value := .Values.migrations.envs }}
  - key: {{ $key }}
    value: {{ $value }}
{{- end }}

{{- if eq .Environment.Name "production" }}
databases:
- name: db
  production: true
  cluster_name: mydatabase
  engine: PG
  version: "12"
{{- end }}

As you can see, the app.yaml is leveraging templates to abstract the values that can change (e.g. tag: {{ requiredEnv "IMAGE_TAG" }}), as well as, determining which components need to be deployed based on the environment (e.g. the usage of a postgres container in review environments vs the usage of a managed database in production).

Next, we define the values for each of the environments that are going to be merged with the app.yaml to produce the final app specification. First, the values definition for the review environment:

# review.yaml
name: sample-{{ requiredEnv "REVIEW_HOSTNAME" }}

.common_envs: &common_envs
  DB_USERNAME: postgres
  DB_PASSWORD: password
  RAILS_ENV: production

rails:
  instance_slug: basic-xxs
  instance_count: 1
  envs:
  <<: *common_envs

postgres:
  envs:
    POSTGRES_USER: postgres
    POSTGRES_DB: mydatabase
    POSTGRES_PASSWORD: password

migrations:
  envs:
  <<: *common_envs`

And second, the values definition for the production environment:

# production.yaml
name: sample-production

.common_envs: &common_envs
  DB_USERNAME: postgres
  DB_PASSWORD: strong_password
  RAILS_ENV: production

rails:
  instance_slug: professional-xs
  instance_count: 3
  envs:
  <<: *common_envs

migrations:
  envs:
  <<: *common_envs

With all the required files in place, we can now proceed to deploy our app to DigitalOcean

As a review environment:

$ IMAGE_TAG='fad7869fdaldabh23' REVIEW_HOSTNAME='fix-bug' appfile sync --file /path/to/appfile.yaml --environment review

This would deploy a public Rails service, and internal Postgres service (the database running on a container) and would run the migration job. The final App spec to be synced to DigitalOcean would look like:

# final app specification with review environment values
name: sample-fix-bug

services:
- name: rails-app
  image:
    registry_type: DOCR
    repository: <app-repo>
    tag: fad7869fdaldabh23
  instance_size_slug: basic-xxs
  instance_count: 1
  routes:
  - path: /
  envs:
  - key: DB_PASSWORD
    value: password
  - key: DB_USERNAME
    value: postgres
  - key: RAILS_ENV
    value: production

- name: postgres
  image:
    registry_type: DOCR
    repository: postgres
    tag: '12.4'
  internal_ports:
    - 5432
  envs:
  - key: POSTGRES_DB
    value: mydatabase
  - key: POSTGRES_PASSWORD
    value: password
  - key: POSTGRES_USER
    value: postgres

jobs:
- name: migrations
  image:
    registry_type: DOCR
    repository: <migration-repo>
    tag: fad7869fdaldabh23
  envs:
  - key: DB_PASSWORD
    value: password
  - key: DB_USERNAME
    value: postgres
  - key: RAILS_ENV
    value: production

As a production deployment:

$ IMAGE_TAG='fad7869fdaldabh23' appfile sync --file /path/to/appfile.yaml --environment production

This would deploy a public Rails service and a migration job. Both components would connect to an existing database. The final App spec to be synced to DigitalOcean would look like:

# final app specification with production environment values
name: sample-production

services:
- name: rails-app
  image:
    registry_type: DOCR
    repository: <app-repo>
    tag: fad7869fdaldabh23
  instance_size_slug: professional-xs
  instance_count: 3
  routes:
  - path: /
  envs:
  - key: DB_PASSWORD
    value: strong_password
  - key: DB_USERNAME
    value: postgres
  - key: RAILS_ENV
    value: production

jobs:
- name: migrations
  image:
    registry_type: DOCR
    repository: <migration-repo>
    tag: fad7869fdaldabh23
  envs:
  - key: DB_PASSWORD
    value: strong_password
  - key: DB_USERNAME
    value: postgres
  - key: RAILS_ENV
    value: production

databases:
- name: db
  production: true
  cluster_name: mydb
  engine: PG
  version: "12"

Future steps #

There are several areas where the tool could move forward in the future:

• Provide packages for homebrew and chocolatey to ease the installation process in MacOS and Windows respectively.
• Providing a lint command, that would allow to validate the final spec without connecting to the DigitalOcean API. Usage would be: appfile lint -f <appfile.yaml> -e <env_name>
• Load App specs from a remote URL. That would be a first step towards a reusability of Apps in DigitalOcean and having access to pre-defined, customizable Apps
• Support secrets encryption through an integration with sops

Conclusion #

Let's recap quickly the post. First, I talked about the DigitalOcean App platform and the obstacle of customizing the App specification to suit different environments requirements, resulting on the creation of appfile. Next, I provided an overview of the tool, how to install it, main features and how to use. Finally, I mentioned some future ideas where I could see appfile evolving to.

appfile has been a very interesting project to work on for the past few days. I have learned a lot about the DigitalOcean API and the App Platform in particular. I see the value that it brings to developers and some of the directions where it could go in the future are pretty interesting.

To conclude, thank you so much for reading this post. Hope you enjoyed reading it as much as I did writing it. See you soon and stay tuned for more!!