DEV Community: Marc Auberer

Compose Generator - Update 1.5.0

Marc Auberer — Thu, 16 Dec 2021 13:43:53 +0000

If you haven’t read about Compose Generator yet, visit the introduction article here. This particular article shows which features / improvements were added in version 1.5.0 of Compose Generator.

Apart from optimization and bugfixes, there are a few new features to discuss in the following parts.

Watchtower integration

This update adds support for auto-updating of Docker images, using Watchtower. The requirement for that is, that the tag of the Docker image does not change with the update. If you set the tag latest you should be good to go!

Compose Generator asks you if you want to enable Watchtower after selecting the other service templates. It will ask you which services you want to enable Watchtower for. Any selected service will get the com.centurylabs.watchtower.enable label.

If Watchtower is enabled. The compose file contains an additional service:

companion-watchtower:
  command:
    - --interval
    - "30"
  depends_on:
    database-postgres:
      condition: service_started
    db-admin-adminer:
      condition: service_started
    frontend-accent:
      condition: service_started
    frontend-angular:
      condition: service_started
  image: containrrr/watchtower:latest
  restart: unless-stopped
  volumes:
    - type: bind
      source: /var/run/docker.sock
      target: /var/run/docker.sock

Major performance optimization for CCom

Compose Generator internally uses the CCom compiler toolchain to evaluate conditional sections and conditional statements, based on JSON input. CCom offers a CLI, which Compose Generator called previously. This CLI executable enriches the user input and then calls the compiler executable with all required inputs. After the last update we executed several benchmarks for CCom, which have shown that this additional executable fork costs a lot of performance.

As Compose Generator uses CCom very often (~20 times even if only one template was selected), CCom can be optimized quite a bit by calling the CCom compiler executable directly without going via the CLI.

Compose Generator - Update 1.3.0

Marc Auberer — Tue, 16 Nov 2021 19:57:42 +0000

If you haven't read about Compose Generator yet, visit the introduction article here. This particular article shows which features / improvements were added in version 1.3.0 of Compose Generator.

Apart from optimization and bugfixes, there are a few new features to discuss in the following parts.

CLI auto-completion

With version 1.3.0 Compose Generator for Linux starts to support cli auto-completion. For Linux installations with the package manager (apt-get, yum, dnf) the auto-completion script is installed by default with the new version. The dockerized Compose Generator does not support auto-completion, for Windows auto-completion will be shipped with one of the next releases.

To enable the cli auto-completion on Linux, execute the following two commands:

$ set PROG=compose-generator
$ source /etc/bash_completion.d/compose-generator

Command to delete custom templates

Up to now, you were only able to save and load custom templates, but not delete them. This feature is included in version 1.3.0 and you can now remove unused custom templates and save disk space.

To execute the new command, use compose-generator template delete <template-name> (Short form cg t d <template-name>). If you omit the optional argument template-name here, you will get prompted with a list of found templates, where you can select the one to delete.
For safety reasons, you will be prompted if you really want to delete a template when you execute the command.

Command to create blank predefined templates

After getting a feature request from a predefined template contributor, Compose Generator now brings a new command for creating a blank predefined service template to start from scratch with a template contribution much faster. The command only works in a development environment and is not included in the production version. Thus, you will have to clone the GitHub repo and build the Compose Generator executable by yourself using the build.bat / build.sh script.

Type compose-generator predefined-template new (Short form: cg p n) to use the new command.

Compose Generator - Update 1.2.0

Marc Auberer — Sat, 06 Nov 2021 23:36:19 +0000

If you haven't read about Compose Generator yet, visit the introduction article here. This particular article shows which features / improvements were added in version 1.2.0 of Compose Generator.

Apart from optimization and bugfixes, there are a few new features to discuss in the following parts.

Traefik as a new proxy template

Version 1.2.0 brings support for a second reverse proxy service in addition to JWilder Nginx / Let's Encrypt Companion called Traefik. Traefik is a highly customizable edge router which brings a massive amount of built-in features: Reverse proxying, automatic service-detection, load balancing, no-downtime upgrades/configuring and supports Docker, Docker Swarm, Kubernetes, AWS, etc.

For supporting Traefik, we added the proxy-labels field to the predefined service configuration spec, because configuring Traefik service detection works with Docker labels.

The most interesting feature about Traefik is the built-in web interface, which gets exposed on port 8080 (customizable). It gives you an overview which and how many services are running and if they are healthy.

Warning: The web UI is not password protected per default, so before you use it in production, please un-expose the web UI or enable a password protection.

Generating from remote config files

Since the beginning, Compose Generator supports generating stacks from configuration files (Example configuration). With this update it supports remote configuration files. Compose Generator will download the config file from the url and parse it directly. You can pass a remote config file the same way as a file on disk, but as an url instead of the filepath:
cg --config https://example.com/config.yml

Support for .yml and .yaml

Up to now, Compose Generator could only load docker-compose.yml and was not able to load docker-compose.yaml. This should have changed now. Compose Generator searches for docker-compose.yaml first. If this file does not exist, it falls back to docker-compose.yml. The default value is still docker-compose.yml to not cause any compatibility issues.

Compose Generator - Create Service Templates

Marc Auberer — Thu, 04 Nov 2021 23:11:06 +0000

Note: If you didn't read the first article of this series yet, please do this first. This article builds onto base knowledge about Compose Generator.

Compose Generator currently offers > 60 high quality predefined service templates, which you can use out of the box. You can find a list of all supported templates here.

There are a lot of awesome software projects out there that Compose Generator does not support yet. That's where you come into play (more about this at the end of the article) …

Structure of a predefined service template

A predefined service template consists of the following file structure:

<template-name>
├─volumes
│ ├─volume1
│ ├─volume2
│ ├─volume3
│ └─...
├─config.json
├─service.yml
├─environment.env
└─README.md

Configuration file (config.json): Holds the metadata for a service template like name, label, questions, volumes, secrets, etc. in JSON format. Example file, json spec

Service file (service.yml): Can be seen as a part of the docker-compose.yml file. It contains information about one particular service. It allows the usage of placeholders like ${{PLACEHOLDER_EXAMPLE}} which can be populated by defining question variables in the config.json file. Compose Generator replaces all placeholders in the generating process. Example file

Environment file (environment.env): Contains any sensitive data like database credentials or any kind of secrets. This file is included in the .gitignore file per default and allows placeholders as well. Example file

Readme file (README.md): Contains detailed information about the service template for the user. It is best practice to first introduce the software for which the service template is made for and subsequently explain the setup process. Take a look at existing service template Readme's to get some inspiration. Example file

The volumes directory is not mandatory but can be used for storing configuration files, which your template may use. If the volume directory does not exist but is bound to the container in the service.yml file, Compose Generator will create an empty directory.

Steps to create a predefined template

Choose a software which you want to create a template for.
Search an image registry where the Docker image of your software can be obtained from.
Research which volumes, environment variables, networks and companion services your service needs.
Fork and clone the Git repository of Compose Generator to your local dev system and create a new branch from the latest release/vX.Y.x branch. Best practice for the branch name is e.g service/adguard-home
Classify the type of the template to one of frontend, backend, database or db-admin. You will need this classification in the next step.
Go to the directory predefined-services/<your-type> and create a new sub-directory for the template (has to be kebab-cased).
Create the file structure above in the directory you just created. The config.json and the service.yml file can copied from an existing template and modified accordingly.
If your service template contains a Dockerfile, please optimize it for minimal image size and make use of multi-stage builds if appropriate.
If your service has companion services (e.g. PhpMyAdmin for MySQL), navigate to the config.json file of the companion service and change the property preselected to a CCom single condition. See the PhpMyAdmin config file to get an idea how this condition can look like.
Edit the table in docs/docs/supported-service.md and insert your service to the respective section. Mind the alphabetical order here.

Test the template

Build Compose Generator from source by executing build.sh (Linux) or build.bat (Windows) from the root dir of the repo. The build output can be found in the bin subdirectory.

For testing your local predefined service templates, you can execute ./compose-generator -irf (Linux) or .\compose-generator.exe -irf (Windows). Your service template should now show up in the respective list. Select it (and maybe some other random templates) and let Compose Generator generate the stack for you. After that, Docker Compose should launch your stack without any error message.

Contribute your own templates

The pledge of the project is to be extensible and community-driven. Therefore, feel free to create service templates by yourself for missing services and to hand in your predefined service templates in form of community contributions on GitHub. Visit the guide here for more information.

Compose Generator - Update 1.1.0

Marc Auberer — Sat, 23 Oct 2021 01:05:32 +0000

If you haven't read about Compose Generator yet, visit the introduction article here. This particular article shows which features / improvements were added in version 1.1.0 of Compose Generator.

Service adding / generating rework

Originally, the generate sub-command only supported generating a stack from predefined service templates whereas the add command could only handle fully custom services. If you needed a mixture of both e.g. an own frontend service and an own backend service, combined with the predefined MySQL service, you formerly would have to generate the MySQL without frontend and backend with the generate command and subsequently add the other two services with the add command.

With v1.1.0 the generate command offers the menu option Custom service in each predefined service selection list at the bottom. If you select it, you get an "empty" service template like with the add command. In the same move we added support for predefined templates for the add command.

Service template tester

Compose Generator v1.1.0 introduces a new testing workflow for predefined service templates. Up to now, only the JSON and YAML files were validated in the sense of syntax, whereas now, every service template gets tested fully-automated on GitHub Actions. The workflow runs on every pull request which touches any predefined template.

Logging and error handling mechanism

Up to v1.0.0 Compose Generator did not collect execution logs and printed the error messages + stack traces directy to the console. As this is not very consumer friendly, Compose Generator v1.1.0 introduces the maintainence of log files. These can be used to comprehend the execution path for reproducing bugs et al. The logfiles can be found at /usr/lib/compose-generator/log on Linux and ./log relative to the Compose Generator executable on Windows.

Compose Generator - Deploy your App in Seconds

Marc Auberer — Tue, 12 Oct 2021 09:38:22 +0000

Application deployment with Docker and Kubernetes is easier than ever before, but could it even be much faster and much more seamless? This is where Compose Generator comes into play by helping you with common Docker tasks with a command line interface.

Deploy a project

With Compose Generator you only have to do the following four steps:

Install Compose Generator (instructions below)
Clone repository of your project from a VCS
Run compose-generator install to install Docker and Docker Compose (can be skipped if you have Docker and Docker Compose already installed)
Runcompose-generator -r to generate your deployment configuration and run it subsequently

Lets dive a bit further into Compose Generator to better point out the various advantages and why you should consider using it.

Generate Docker Compose configurations

Compose Generator can be seen as a deployment assistant and acts like a wrapper around your Docker installation. As you see above, Compose Generator can be used to install Docker and Docker Compose on your system, so you only need to care about installing Compose Generator in the first place. The main feature of Compose Generator is - as the name might suggest - to generate the Docker Compose configuration for your project. Moreover, Compose Generator can generate secrets (like e.g. database credentials, etc.), manage volumes, networks, environment variables and much more.

In the following example, we’re going to deploy a basic Angular Application, connected to a Spring Boot API and a MySQL database. Compose Generator supports a wide variety of service templates (tendence growing), which you can find listed here.

Example stack deployment

However it is recommended to install Compose Generator on your host system as described here, for this beginners guide you can also run it as Docker container, by executing following command:

$ docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock 
     -v $(pwd):/cg/out chillibits/compose-generator

Docker pulls the image and runs it in interactive mode. The $(pwd) placeholder stands for the current path on your Docker host system and gets mapped to the /cg/out directory within the container. This means, that the resulting output files, like docker-compose.yml will later be located in the directory, in which you execute the command above.

Compose Generator asks you for the name of your project and if you want to create a production-ready configuration instead of the default development config, to hide your services behind a reverse proxy. Compose Generator offers a variety of predefined service templates, which you can arrange for your needs like in a building block system. For our example, we will select Angular in the frontend section, Spring with Maven in the backend section and MySQL in the database section. For each item you select from the templates list, Compose Generator asks you service-specific questions, that are important for setting them up. After having answered all questions for MySQL you can see, that Compose Generator pre-selected PhpMyAdmin, because it is the default db admin system for MySQL. You can deselect it or press enter to continue.

Tip: You can execute compose-generator -a to launch the Compose Generator Cli in advanced mode. Then you also get questions for advanced stack customization. Find more information about cli options of the generate sub-command here.

After having all questions answered, Compose Generator will generate following file structure in the directory where it was called in:

current directory
├─volumes
│ ├─volume1
│ ├─volume2
│ ├─volume3
│ └─...
├─.gitignore
├─.cg.yml
├─docker-compose.yml
└─environment.env

The docker-compose.yml file contains your Docker Compose configuration. If your containers require any secrets, they are stored in the environment.env file, which will be attached to all regarding containers at runtime. As you see above, Compose Generator also creates the volume directories for you. Depending on the stack template you’re choosing, there already will be files in the volume directories (such as configuration files, etc.) to take as much workload off your shoulders as possible.
The .gitignore file in the root directory contains the environment.env file to not accidentally committing any secrets to a VCS.
.cg.yml is a configuration file for Compose Generator itself to remember some selections of yours.

Removing services from a configuration

Imagine, you have generated a stack with Angular, Spring with Maven, MySQL and PhpMyAdmin, but you want to switch to MongoDB as you have to deal with more document-like data structures in your Spring application.
What you can do is to ask Compose Generator to remove the services MySQL and PhpMyAdmin and add MongoDB instead.

To remove the first two, you can execute the following command in the directory where the docker-compose.yml file for your configuration lives:

$ compose-generator remove

Select the two services MySQL and PhpMyAdmin and press the enter key. Compose Generator will remove those services from your compose configuration without leaving residues, e.g. unused volumes, networks, etc.

Adding services to a configuration

To now add the service for MongoDB, you can execute following command:

$ compose-generator add mongodb

After answering all questions, the docker-compose.yml file should now contain a service database-mongodb and the services database-mysql and db-admin-phpmyadmin should be gone.

DEV Community: Marc Auberer

Compose Generator - Update 1.5.0

Watchtower integration

Major performance optimization for CCom

Further reading about Compose Generator

Compose Generator - Update 1.3.0

CLI auto-completion

Command to delete custom templates

Command to create blank predefined templates

Further reading about Compose Generator

Compose Generator - Update 1.2.0

Traefik as a new proxy template

Generating from remote config files

Support for .yml and .yaml

Further reading about Compose Generator

Compose Generator - Create Service Templates

Structure of a predefined service template

Steps to create a predefined template

Test the template

Contribute your own templates

Further reading about Compose Generator

Compose Generator - Update 1.1.0

Service adding / generating rework

Service template tester

Logging and error handling mechanism

Further reading about Compose Generator

Compose Generator - Deploy your App in Seconds

Deploy a project

Generate Docker Compose configurations

Example stack deployment

Removing services from a configuration

Adding services to a configuration

Further reading about Compose Generator