January Gnarly Learnings #1: Local services with asdf and foreman

Upon learning of asdf plugins for backing services like Postgres, Redis, Elasticsearch and others, my first question was whether and to what extent it could replace Docker for providing those services in local development, where asdf already shines for versioning.

The first obstacle for most users will be that asdf has no equivalent of docker compose up for booting and monitoring an entire local environment. (As of January 2022 the asdf discussions page is recently dark but this was an active topic at the time of its disappearance.) Brainstorming the matter with a colleague, he pointed out we were essentially proposing Foreman. A few moments later he found this:

Managing Ruby on Rails development dependencies with asdf and foreman

[Note: We're talking about Foreman, not The Foreman.]

From the oldie-but-a-goodie department: Foreman is a simple tool that reads a list of command line tasks from a file, starts them, feeds their output to stdout with some nice formatting, and eventually shuts them down when you exit the process; i.e., exactly what asdf doesn't provide. As outlined above, there isn't even really any "integration" necessary for asdf and Foreman to play nicely. They just do their own things correctly.

I've been using the approach above where possible for a couple of weeks now, and it's nice. (Be sure to note the trick in the blog post for running Postgres under Foreman control!) Docker is, among other things, a significant resource commitment for local development. On a Mac, you're spinning up an entire separate Linux instance to run software that MacOS is quite capable of running itself. This might be a good tradeoff; for many the benefit of standardizing developer experience or duplicating an idiosyncratic production environment will outweigh other considerations. But for individual developers looking for lightweight solutions—or to avoid duplicating functionality already provided by the kernel, the shell, asdf and various language-specific dependency managers—this solution is very much worth a look.

Streamlining

I created the following in ~/bin/app, which will probably get (a little) more clever over time:

#!/usr/bin/env zsh

foreman start -f Procfile.local

I used Procfile.local to prevent conflicts with applications that already have a Procfile for production. In this case we're booting a Phoenix project using Postgres and Redis:

postgres: mkdir -p log/ && postgres-local
redis: redis-server
api: mix phx.server

Meanwhile in .tool-versions:

elixir main-otp-24
erlang 24.2
postgres 14.1
redis 6.2.6

Booting the application is nice and simple:

$ app
17:05:22 postgres.1 | started with pid 12345
17:05:22 redis.1    | started with pid 12346
17:05:22 api.1      | started with pid 12347
[...logging continues, ctrl-c to exit...]

Future possibilities, difficulties, etc.

The main pain point I see is running multiple applications simultaneously. Docker virtualizes an entire operating system, so if you have two applications, A and B, which both use the same version of (say) Postgres, they still get entirely separate installations, data directories, port 5432 to bind, and so on. With this approach, you have one version of Postgres.

One answer is to separate out the runtime specifics of the service from the codebase, which will presumably depend on the nature of the service itself, but follow a general shape:

• Environment variables
• Configuration files
• Persistent data storage (your database)
• Temporary data storage (logging, lock files)

If you can specify those uniquely for each instance in Procfile.local, it's likely feasible to run multiple instances of the same service simultaneously. My own research into that will occur when/if I need to run several unrelated applications simultaneously, which is not usually how I work.

Credit where credit is due

Big h/t to Kassio Borges for having thought of this before any of us over here. That link again: Managing Ruby on Rails development dependencies with asdf and foreman