Deploy windmill on Fly.io

Windmill is an open-source, self-hosted workflow engine (even a low-code platform) that you can deploy yourself. You can visit https://www.windmill.dev/docs/compared_to to see the official comparison with services like Zapier.

This article will show you how to deploy Windmill to Fly.io.

Overview

This article covers two ways to deploy Windmill, which I refer to as the minimal and full modes.

In the minimal mode, all Windmill components run in a single app. This is the simplest and most cost-effective solution, but it is not as flexible (for example, because the Server and Worker run in the same container, it is difficult to scale them horizontally).
In the full mode, the Windmill components are split into two separate apps: the Server and the Worker. This is more complex than the minimal mode but allows for more control over the distributed Workers.

Please clone the repository windmill-on-fly and follow the steps below.

LSP Component

LSP needs to be installed in both the minimal and full modes

LSP provides code completion functionality on the Windmill frontend. It needs to be deployed separately.

Launch & Deploy

To change the region, modify the primary_region field in the fly.toml file (default is sea). If you change the region, keeping the region consistent for all apps for optimal experience is recommended.

Navigate to the lsp directory and run fly launch to start the project. When prompted with Would you like to copy its configuration to the new app? (y/N), choose Y and give the app a name.

Replace my-windmill-lsp with the app name you set here in the subsequent steps.

In the following prompts, do not create the PostgreSQL and Redis services (choose N for all related prompts).

When asked Would you like to deploy now? (y/N), choose Y to proceed with the deployment.

Allocate IP

After the deployment is completed, run fly ips allocate-v6 --private to assign a private IP to the service (you can also remove all public IPs).

Scale

LSP has higher performance requirements. Based on my experience, I recommend allocating at least 4GB of memory to it for normal usage. You can run fly scale vm performance-2x to scale it.

To save costs, you can enable auto_start_machines=true and auto_stop_machines=true in the fly.toml file.

Deploy in Minimal Mode

Skip this step if you choose the full mode.

All the following operations are performed in the minimal directory.

Launch

Run fly launch to start the project. When prompted with Would you like to copy its configuration to the new app? (y/N), choose Y and give the app a name.

Replace my-windmill with the app name you set here in the subsequent steps.

In the following prompts, do not create the PostgreSQL and Redis services (choose N for all related prompts).

When asked Would you like to deploy now? (y/N), choose N to skip the deployment.

Configure the Database

Follow the Fly Postgres tutorial to create a database cluster.

Replace my-pg in the following commands with the name of your database cluster.

Run fly pg attach my-pg --superuser to create the database and user.

Configure LSP

Edit the Caddyfile and replace xxx-windmill-lsp with the name of your windmill-lsp service.

(Optional) Bind Custom Domain

Skip this step if you don't have or don't want to bind a custom domain.

Run fly certs add YOUR_DOMAIN to configure a custom domain (replace YOUR_DOMAIN with your domain) and configure the required CNAME record as instructed.

Configure Environment Variables

Open the fly.toml file and modify env.BASE_URL to your external access domain. If you're not using a custom domain, enter https://my-windmill.fly.dev here; otherwise, enter https://YOUR_DOMAIN.

If you want to modify the number of workers, you can change the value of the env.NUM_WORKERS variable.

Deploy

Run fly deploy to deploy the application (the deployment process will automatically build the required images and create the necessary volumes). Wait for the deployment to complete successfully and then access your instance to test logging in using the account [e-mail] and password changeme.

Deploy in Full Mode

Skip this step if you choose the minimal mode.

The full directory includes two subdirectories: server and worker, which store the configurations for the Server and Worker components, respectively.

Server - Launch

Navigate to the full/server directory and run fly launch to start the project. When prompted with Would you like to copy its configuration to the new app? (y/N), choose Y and give the app a name.

Replace my-windmill-server with the app name you set in the subsequent steps.

In the following prompts, do not create the PostgreSQL and Redis services (choose N for all related prompts).

When asked Would you like to deploy now? (y/N), choose N to skip the deployment.

Server - Configure the Database

Follow the Fly Postgres tutorial to create a database cluster.

Replace my-pg in the following commands with the name of your database cluster. Use windmill as the database-name and database-user in the commands, or modify them to your desired names.

Navigate to the full/server directory and run fly pg attach my-pg --superuser --database-name windmill --database-user windmill to create the database and user.

Take note of the DATABASE_URL printed during the execution of the above command for later use.

Server - Configure LSP

Navigate to the full/server directory and edit the Caddyfile. Replace xxx-windmill-lsp with the name of your windmill-lsp service.

(Optional) Server - Bind Custom Domain

Skip this step if you don't have or don't want to bind a custom domain.

Navigate to the full/server directory and run fly certs add YOUR_DOMAIN to configure a custom domain (replace YOUR_DOMAIN with your domain) and configure the required CNAME record as instructed.

Server - Configure Environment Variables

Navigate to the full/server directory and open the fly.toml file. Modify env.BASE_URL to your external access domain. If you're not using a custom domain, enter https://my-windmill-server.fly.dev here; otherwise, enter https://YOUR_DOMAIN.

Server - Deploy

Navigate to the full/server directory and run fly deploy to deploy the server (the deployment process will automatically build the required images and create the necessary volumes).

By default, the new version of a fly app without volumes will deploy two instances. For the Server, multiple instances are usually unnecessary. You can run fly scale count 1 to delete the extra instances.

Wait for the deployment to complete successfully. Then, access your instance to test logging in using the account [e-mail] and password changeme.

Worker - Launch

Navigate to the full/worker directory and run fly launch to start the project. When prompted with Would you like to copy its configuration to the new app? (y/N), choose Y and give the app a name.

Replace my-windmill-worker with the app name you set in the subsequent steps.

In the following prompts, do not create the PostgreSQL and Redis services (choose N for all related prompts).

When asked Would you like to deploy now? (y/N), choose N to skip the deployment.

Worker - Configure the Database

Navigate to the full/worker directory and run fly secrets set 'DATABASE_URL=xxx' --stage to configure the credentials for connecting to the database. (Replace xxx with the content of the DATABASE_URL printed during the previous steps).

Worker - Configure Environment Variables

Navigate to the full/worker directory and open the fly.toml file. Modify env.BASE_URL to the external access domain of your windmill-server. If you're not using a custom domain, enter https://my-windmill-server.fly.dev here; otherwise, enter https://YOUR_DOMAIN.

If you want to modify the number of workers, you can change the value of the env.NUM_WORKERS variable.

Worker - Deploy

Navigate to the full/worker directory and run fly deploy to deploy the worker.

Visit your instance's /workers route (e.g., https://my-windmill-server.fly.dev/workers) to see if your Worker is displayed.

Frequently Asked Questions

Upgrading (or specifying a version)

Currently, all version numbers use latest, which may not be suitable for a production environment (because each redeployment will upgrade to the latest version).

The version numbers are present in the following files:

• lsp/fly.toml
• minimal/Dockerfile
• full/server/Dockerfile
• full/worker/fly.toml

You can replace latest with the desired version number in these files, for example, 1.133.

The only important thing to note is to always keep all version numbers consistent; otherwise, there may be unknown issues.

To upgrade: Modify the version number and redeploy using fly deploy in the relevant directories.

Comments

[VladTabacariu]

thank you. i've been looking for this a couple of days ago. you dropped it right on time. what a sync.

One thing i have to mention for the minimal version. The one i did. But i think applies to full version as well. I had to open "run.sh" file with notepad++ and do a Unix(LF) EOL Conversion. Besides this everything was smooth.