DEV Community: Stephan Maximilian Huber

What's new in phabalicious 3.8?

Stephan Maximilian Huber — Mon, 22 Aug 2022 12:36:52 +0000

What has changed?

PHP 7.3 required

Phabalicious requires now PHP version 7.3. It should work with newer versions, including 8.1

Better integration with 1Password cli/ connect

Support for version 2 of the cli
added new script callback get_file_from_1password which will retrieve an encrypted file from 1password, decrypt it and store it somewhere, suitable when scaffolding.

Encryption and decryption

Phab supports now encryption and decryption of files and strings. It uses defuse/php-encryption,
a widely used library for encryption under the hood. You can encrypt files in a script with

secrets:
  name-of-secret:
    question: What is the password?

scripts:
  encrypt:
    - encrypt_files(path/to/files/or/folders/to/encrypt/*.ext, path/to/folder/to/store/encrypted/files, name-of-secret)
  decrypt:
    - decrypt_files(path/to/files/or/folders/to/decrypt/*.enc, path/to/folder/to/store/decrypted/files, name-of-secret)

The scaffolder has a new callback called decrypt_assets which works the same as copy_assets but with an additional
decryption step

scaffold:
  - decrypt_assets(targetFolder, dataKey, secretName, twigExtension)

You can also use decrypt() and encrypt() in your scaffold templates using the provided twig-functions:

test.yml

data:
  my_secret_data: "{{ decrypt(my_secret_data, 'name-of-secret') }}"
  encrypted: "{{ encrypt("Hello world", 'name-of-secret') }}"

New command `install:from-sql-file`

The command install:from and the new command install:from-sql-file get reworked to prevent double-work and to
streamline the process. You can now install the app from an existing sql-file, which translates into an optimized
flow of

phab -cmy-host-config install
phab -cmy-host-config restore:sql-from-file sql.gz
phab -cmy-host-config reset

Instead run

phab -cmy-host-config install:from-sql-file sql.gz

Refactoring of data-handling

Phab now handles data differently, by storing from which file or url a piece of data was loaded. This enables phab to
use relative includes, even for remote configurations. Now you can do things like

base.yml

hosts:
  test:
    foo: Hello world
    bar: Hello from base.yml

fabfile.yml:

inheritsFrom:
  - ./base.yml

needs:
  - local

hosts:
  test:
    foo: Hello mars

Running output will show the resolved inheritances:

❯ phab38 -ctest output --what host


Output of host-configuration `test`
===================================

  test:
    foo: 'Hello mars'
    bar: 'Hello from base.yml'
    needs:
      - local
      - script
    type: dev
    .
    .
    .

This enhances also the introspection possibilities:

The about command got a `-v` flag

The about command has another trick up its sleeve now. When inheriting configurations from multiple files, it can now
display the exact location from which a particular parameter is being picked. Thus, finding out if a value is coming from an inherited file from a remote server or some sensible defaults is at your fingertip.

Let's have a look at an example to see this in action. The command displays not only the active value for a particular
key but also the location from which it is picked up.

In our case the ouput of phab -ctest about will show the inheritance very in the second column:

❯ phab about -v -ctest

Configuration of test
=====================

+-----------------------------+-------------------------------------+---------------------------------------------------+
| Key                         | Value                               | Inherited from                                    |
+-----------------------------+-------------------------------------+---------------------------------------------------+
|   foo                       | Hello mars                          | /Users/stephan/Desktop/phab-38-blog/.fabfile.yaml |
|   bar                       | Hello from base.yml                 | /Users/stephan/Desktop/phab-38-blog/base.yml      |
|   needs                     |                                     | /Users/stephan/Desktop/phab-38-blog/.fabfile.yaml |
|     needs.0                 | local                               | /Users/stephan/Desktop/phab-38-blog/.fabfile.yaml |
|     needs.1                 | script                              | /Users/stephan/Desktop/phab-38-blog/.fabfile.yaml |
|   type                      | dev                                 | host defaults                                     |
.
.
.

New command `find:property`

Phab 3.8 contains a new command named find:property which provides a visual prompt where the user can start typing the
name of a property they are interested in. Phab supports you by offering properties via autocomplete. As an alternative
you can hit enter to get a list of suggestions. After you found the property you are interested in, hit return, and phab
will display the actual value of that property, its ancestors and from where it got read.

Here's a short walkthrough using the files and their config from above.

docs.phab.io

Enhancements to the database command

There's a new subcommand available db:query where you can run a query against the database of a given configuration
without starting a database-shell and run the query manually.

phab -cmy-host-config db:query "show tables"

It will try to get the database credentials from the configuration and apply them. This command is only implemented for
mysql/mariadb for now.

New command `restic`

Phab 3.7 got support for offsite backups via restic. But it was still a bit cumbersome to interact with the restic
repository for a given configuration using restic alone. That's why phab now has the restic command. It is applying the
configuration and options from the fabfile and you can concentrate on the commands you want to pass to restic. Some
examples:

phab -cmy-host-config restic snapshots
phab -cmy-host-config restic -- snapshots --host-name my-config
phab -cmy-host-config restic -- forget --keep-daily=14 --keep-weekly=4 --keep-monthly=6 --group-by host --prune

Experimental integration with ddev

ddev is a superb tool for managing local development environments and provides similar features as phabalicious. As it
takes a lot of time to maintain a local development stack (like multibasebox) we thought it might make more sense to
embrace existing open-source tools.

As ddev is also using docker and docker-compose under the hood it was very unproblematic to integrate ddev with phabalicious.
Phab will read the ddev configuration from .ddev/config.yaml and provide it under the global property ddev. So it's
very easy to provide a phab configuration for your local ddev instance. Instead of copy-pasting the boilerplate into
every project we store it on a central place and inherit from it in the fabfile:

Store this file in a central place, e.g. in your user-folder or on a remote-server. For demonstration purposes we store
this in the same folder as the fabfile

the ddev.yml-file:

requires: 3.8.0

dockerHosts:
  ddev:
    rootFolder: .
    runLocally: true


hosts:
  ddev:
    additionalNeeds:
      - ddev
    info:
      description: Local ddev instance
      publicUrls:
        - "https://%settings.ddev.name%.ddev.site"
      category:
        id: 01-local
        label: Local development
    executables:
      drush: /var/www/html/vendor/bin/drush

    shellProvider: docker-exec
    composerRootFolder: /var/www/html
    gitRootFolder: /var/www/html
    rootFolder: /var/www/html/web
    backupFolder: /var/www/backups
    docker:
      configuration: ddev
      name: ddev-%settings.ddev.name%-web
      projectFolder: .

the actual .fabfile.yml:

inheritsFrom:
  - ./ddev.yml

hosts:
  ddev:
    # its a drupal installation, so add the drupal-specfic stuff
    adminPass: admin
    replaceSettingsFile: false
    alterSettingsFile: false
    executables:
      drush: /var/www/html/vendor/bin/drush
    needs:
      - git
      - docker
      - drush
      - files

running phab list:hosts will show sth like this for your ddev-project:

❯ phab list:hosts -v

Available configurations for your project
=========================================

Local development
-----------------

 ‣ ddev
   Local ddev instance
   → https://my-ddev-site.ddev.site

You can run any command against the ddev site as usual. For controlling app lifetime please use ddev, e.g. ddev start
or ddev stop

New run contexts for yarn, npm and others

The configs for yarn, npm, composer and laravel changed. The composite property name like yarnBuildCommand is
deprecated, please move the config under a section named yarn. This allows us to add more options for the respective
commands by keeping a structure. Phab will support the old property-names until the next major release, for now you'll
get a friendly reminder.

One of the reasons for this change is the extension of the runContext, which got renamed into context which will adapt
the existing functionality of the script execution contexts. Build commands can now be multiline, but then please
provide the executable-name by yourself:

Old:

hosts:
  test:
    yarnBuildCommand: build:production
    yarnRunContext: docker-host

New:

hosts:
  test:
    yarn:
      buildCommand:
        - yarn build:production
        - yarn build:assets
      context: docker-image
      image: node:16

There are two new contexts available which pigypack on the existing functionality of script execution contexts:

docker-image will execute the yarn build in a docker image (see script execution contexts for configuration options)
docker-image-on-docker-host same as above, but it will be executed on the same instance where docker-commands are executed, usually where your dockerConfig points to.

Smaller enhancements

Passwords are now obfuscated in phabs output.
Update-check is now using a maintained [php
library](https://github.com/consolidation/self-update) and works more reliably. Please that the name of the option for getting a beta-version changed too.
When importing a sql-dump phab freezes the app, and unfreezes it afterwards. For drupal applications the site will be put in maintenance mode and back.

Switch to conventional commits and semi-automated releases

We switched to the conventional commits convention and introduced standard-release to help us creating proper releases.

Where to get it

See the documentation for installation

How to test a beta version of phabalicious in parallel

Stephan Maximilian Huber — Sat, 13 Aug 2022 11:42:00 +0000

Sometimes it would be neat to test if a new beta version of phabalicious works with my local projects, without breaking
my productivity if I encounter problems. Fortunately it is rather safe to have two phab versions installed on your local
computer.

Usually you keep phabalicious up-to-date by running phab self-update. The easiest way to install a beta-version side
by side to the official version are the following commands:

$ curl -L  https://github.com/factorial-io/phabalicious/releases/download/3.8.0-beta.16/phabalicious.phar --output /usr/local/bin/phab38
$ chmod +x /usr/local/bin/phab38
$ phab38 --version
phabalicious 3.8.0-beta.16

Please adapt the version in the url to your likings. After a successfull installation you can update the beta version using

phab38 self-update --preview

And if you want to test if your project is still working with the new beta version, just substitute phab with phab38, e.g.

phab38 -cmbb docker run

If you encounter any problems with a beta or a stable version please create an issue
To remove the beta-version just run rm /usr/local/bin/phab38

About the importance of including DevOps in your team

Stephan Maximilian Huber — Thu, 17 Dec 2020 16:17:42 +0000

Developers working on client projects are taking care of numerous tasks. When you think the list gets shorter, new things are added at the bottom. What you can do is provide an infrastructure that lets them tackle their issues. But this doesn’t include only the administrative side but also the technical requirements for the developers to work properly. What’s missing is someone who is familiar with the IT area of the developers and has the capacity to assist in operative processes aside from the usual management. This is where the DevOps enter the field.

The DevOps work to hold the development team’s back. They appear as the merge between operations and development, supporting the team by managing the software and applications, deployment of tools in order to simplify procedures and smoothen the development process. With his work the DevOps face delays and other performance interferences by reducing downtimes and system failures. The use of DevOps in your team not only assures quality and an optimized workflow, you also stick to the latest development trends and keep yourself and your team up to date.

What skill set do the DevOps need?

Let’s start with the most obvious one: The development skills. The differentiation element of the DevOps in comparison to an administrative manager are their skills regarding programming and scripting. This includes various programming languages, open source technologies and tools as well as knowledge in server systems and security concepts. The DevOps prove their expertise for the position through their understanding of source control systems, infrastructure automation tools, cloud services and testing skills. Aside from the technological competences required in this area, the DevOps need to combine this with a set of soft skills that they can come up with.

As they stand for the binding component between different teams an excellent communication is indispensable. Bearing in mind the amount of possible issues the DevOps need to solve, which is a challenge for itself, and to keep a cool head and show empathy and integrity is not easy. Last but not least the job requires more than any other the willingness to educate yourself. The digital environment is in a continuous change and so are the skills the DevOps have to maintain in order to stay an expert in their area. Being well-informed about new technologies, systems and procedures will prepare the DevOps for the never ending wave of issues and errors they need to solve to support their team.

How do we implement the DevOps role at Factorial?

We at Factorial divide the tasks of the development operations on various shoulders. In this case we not only have specific members of the team dealing with specific tasks, we also use administrative development programs in order to automate workflows. We established a way to centralise logging and monitoring and also administer our zoo of hosted applications like Jira, Confluence and Mattermost to constantly improve our corporate infrastructure. With this we work on new ways of streamlining our processes and think about new features like rapid rollouts of dev instances.

In addition to that we enhance and develop our local dev stack based on MultiBaseBox. Our devs are using docker for spinning up the needed dependencies for their projects. We are using MultiBaseBox and phabalicious for local development. The underlying haproxy container with automatic configuration is also used as an easy way to spin up remote dev instances, so clients can test new features. Our team is constantly enhancing and extending these tools.

phabalicious

By using phabalicious our devs are equipped with all necessary and project related information within one single configuration file. With this program our devs can connect to the actual app installation, be it locally installed, behind a ssh-jump-host or part of a kubernetes cluster and can execute common, every-day tasks as getting a database dump, deploying a new version of the application or just get a shell on that particular instance. Recently our engineers added support for kubernetes. It’s actively maintained by our team, and adapted to the ever changing needs of our developers and clients, while trying to be as open as possible.

kubernetes

Our devs can rely on kubernetes as a valuable tool to administer and create definition files for our clusters and the clients’ clusters. They can bring Drupal-, Laravel- or Node-based websites and applications into a kubernetes based cluster and monitor the applications over time.

GitLab & docker

As passionate developers we work with GitLab as our main code repository and make full use of its strong integration capabilities provided by Pipelines. In addition to that we use docker to provide the best working environment for our developers all across the globe.

Wrap-up

The DevOps appear as the perfect match to your development team. In fact, as we at Factorial see it, the development team isn’t complete without someone uniting the understanding of development processes and programs with the administrative knowledge that ties all together. The true value this role can bring to your business, is not only from an economical standpoint but also in terms of stress reduction and process optimization. If you haven’t thought working with this role yet, you might be surprised about the value it can contribute to your company.