DEV Community

Cover image for Create an auto-merging workflow on Github
Peter Strøiman
Peter Strøiman

Posted on • Edited on

1

Create an auto-merging workflow on Github

How do you prevent bad commits from entering the main branch?

You can add a workflow that builds and runs your test suite. This will notify you immediately when this has happened. But the bad commits are already there, possibly blocking other developers. Or worse, preventing a crucial hotfix from reaching production.

You can submit a pull request, a popular popular way for teams to work. But pull requests are intended to handle the process for feedback of proposed changes. For single-developer repos, or teams that don't need this feedback process, using pull requests just to prevent broken builds from entering the default branch easily overcomplicates things.

A much simpler process is to just push to a different branch, have a let github automatically merge the changes to your main branch if verification succeeds.

This article shows how to setup such a workflow that runs almost seamlessly, and how to deal with a few challenges along the way.

Set the permissions

This setup requires a github workflow that pushes to your repository. In order for this to work, you need to set the proper permissions for the project.

Go to the settings page for your project, and open the "Actions > General" settings, where you must configure "Workflow permissions" to "Read and write permissions".

Screenshot showing the workflow permissions being set to read and write premissions

Once this is done, workflow actions can push to your repository.

Create/update a verification workflow

You must have a verification workflow. I assume that you already have an existing workflow, and that it has a push trigger.

Normally, the trigger is not set to run on all branches, so you need to add the branch name to use. Here it's called auto-merge. If you don't already have a push trigger, be sure to add it.

Remember the name of the workflow, here it's Build. It is needed in the next step.

# .github/workflows/build.yaml
name: Build

on:
  push:
    branches: [ "main", "auto-merge" ]
Enter fullscreen mode Exit fullscreen mode

For a team setting, each developer could have their own branch, and you can use a wildcard to react to them all.

About github workflows

If you are new to workflows, here are some fundamentals.

A github workflow is started from a trigger. A common configuration is to use both the push trigger, to run the workflow when there has been pushed to a branch, and the pull_request trigger, which obviously triggers when a pull request is created, or new code pushed to the pull request.

There are many kinds of triggers, including triggers that are completely unrelated to code. E.g., I used a scheduled job trigger to renew server certificates from a github workflow.

A verification workflow would normally use actions/checkout to fetch the code from git. For a push trigger, the branch will by default be checked out. For a pull_request trigger, a merge commit will be checked out, i.e. it is the result of a merge with the default branch that is verified by the workflow.

A common default default is to have both push and pull_request triggers on the default branch, here main.

# .github/workflows/build.yaml
name: Build

on:
  push:
    branches: [ "main", "auto-merge" ]
  pull_request:
    branches: [ "main" ]


jobs:
  build:
    name: Build and test the code
    runs-on: ubuntu_latest
    steps:
    # Uses can use pre-made actions. 
    # actions/checkout will fetch your code.
    - uses: actions/checkout@v4
    # Frameworks can often be configured using other actions.
    # Github have good starting points for most project types.
    - name: Build and test
      # Make sure you run the steps necessary. 
      # Github have good starting points for most project types.
      run: ./build-and-test.sh
Enter fullscreen mode Exit fullscreen mode

Create a new workflow in the default branch.

The new workflow uses the workflow_run trigger, a trigger that can react to events of another workflow.

This workflow is not associated with a branch, it points to another workflow. Because of that, it is global to the github project and must exist in the default branch; typically named main or master.

The auto-merge workflow should be run when the verification workflow is completed, and the workflow was executed on the auto-merge branch.

# .github/workflows/auto-merge.yaml
name: Auto-merge
on:
  workflow_run:
    workflows: [Build]
    branches: [auto-merge]
    types: [completed]
Enter fullscreen mode Exit fullscreen mode

So while the workflow with a workflow_run trigger workflows on a project level, it can still filter on the branches that were the trigger a verification workflow run.

Note: You can use wildcards in your branch names if you have multiple auto-merge branches.

Create a job

A job does the actual work. While this is triggered on a completed verification workflow, a completed workflow can still have failed. To handle that, add a condition to check the outcome of the completed workflow.

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
Enter fullscreen mode Exit fullscreen mode

Fetch the code

To fetch the code, we use the actions/checkout action.

The first unexpected issue is that the action checks out the default branch, not the auto-merge branch; which was the original source of a trigger. While a push trigger will by default check out the branch that was updated, the workflow_run trigger does not. We must check out the right branch in the workflow ourselves.

The branch name exists as data on the associated event of the trigger. This contains information about the completed workflow, including the name of the branch that triggered the first workflow. This value is found in the variable github.event.workflow_run.head_branch.

Note: For a single auto-merging branch, we could just have duplicated the branch name, but for multiple branches, it's necessary to read this value from the event.

Shallow Clones

The next issue is that by default, the checkout action creates a shallow clone, i.e., there is no history. You cannot push to a branch, if you don't locally have all commits from, and including, the head of the target branch to your branch.

The easiest solution is to add fetch-depth: 0 to the action. This pulls the full history, problem solved.

    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.workflow_run.head_branch }}
          fetch-depth: 0
Enter fullscreen mode Exit fullscreen mode

For large repositories with a lot of history, or large binary files in the history, this can increase the runtime significantly. But there are ways to deal with this problem.

Smarter handling of shallow clones

Instead of fetching full history, we can fetch just enough history to be able to push.

Note: This is much more tricky, so for smaller repositories, the previous method would be advisable. Particularly, if you/the team don't feel comfortable with shell scripting.

The changes to make compared to the previous simple version are:

  1. Remove the ref and fetch-depth options.
  2. Use git remote set-branches ... to tell git there is a different remote branch we want to use.
  3. Use git fetch ... to fetch the relevant commits, i.e. just enough shallow history, to be able to push the changes.
  4. Checkout the source branch locally.
    steps:
      - uses: actions/checkout@v4
      - name: Tell git we want the `auto-merge` branch
        run: git remote set-branches --add origin ${{ github.event.workflow_run.head_branch }}
      - name: Fetch the target branch
        run: git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"
      - name: Checkout target branch
        run: git checkout ${{ github.event.workflow_run.head_branch }}
Enter fullscreen mode Exit fullscreen mode

Here is a more detailed explanation of the changes.

1. Remove ref and fetch-depth

Because we need to have all commits in the source branch not reachable from the target branch (default branch), we need the target branch in our working copy. The simplest solution is that start the process with the target branch checked out.

2. Add new remote branch

In a shallow clone, git doesn't fetch all remote branches, only the target branch. The following command tells git explicitly that this is a branch we want to work with.

git remote set-branches --add origin ${{ ... .head_branch }}
Enter fullscreen mode Exit fullscreen mode

Again, for a single branch, you could duplicate the branch name, but it's so simple to avoid it.

3. Fetch the remote branch.

With the remote branch added, we can fetch the branch using git fetch.

In a shallow clone, fetch will fetch commits that are after the current branch, i.e., it will fetch all the commits that are new in the auto-merge branch. But if the target branch is ahead of the source branch, i.e., the default branch contains commits not yet in the auto-merge branch, git fetch fetches the entire history, defeating the purpose of the shallow clone to begin with.

This scenario is less likely for a single-developer setup, but very likely to happen occasionally in a team setup where team members use individual auto-merge branches.

To keep the clone shallow, the command option --shallow-since="" is used to not fetch commits older than the current HEAD (which is the default branch). The commit timestamp for HEAD is found using git show --no-patch --format=%ci HEAD.

git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"
Enter fullscreen mode Exit fullscreen mode

Backticks here is a special shell feature that executes the git show ... command, and uses the textual output as input for the command, here for the arguments for the git fetch command

fetch will now fetch exactly all the commits that are necessary for the branch to be pushed if possible.

4. Check out the source branch

How we have just enough commits in the local database to push. We checkout the branch

git checkout ${{ github.event.workflow_run.head_branch }}
Enter fullscreen mode Exit fullscreen mode

Push

Both the simple, and the shallow clone workflow variations, have left us with the source branch
checked out. All that is left is to push it to the remote target branch, i.e. push auto-merge to main in this example. Here HEAD is used, so the script doesn't need to know what the branch actually is.

jobs:
  on-success:
    # ...
    steps:
      # ...
      - name: Push to main
        run: git push origin HEAD:main
Enter fullscreen mode Exit fullscreen mode

By default, git will only perform a fast-forward merge on push. So the workflow will fail if there are new commits on the target branch. In this case, you need to deal with the conflict, by either merging or rebasing your changes off the new master; just as you normally would.

The error could be one of two:

  • The push failed because it wasn't a fast-forward.
  • The push failed because you didn't have the remote target HEAD in your local git database. This was because they were not fetched as they were committed earlier than the target HEAD.

No matter which, the cause is still the same, there are new commits in the default branch not reachable from the HEAD of your local branch.

There is another potential error, you have rewritten the commit time in the history. That shouldn't be a normal workflow, so this workflow is not designed to handle that.

The full workflow file

This is the full workflow file. Adapt the following to your own workflow.

  • The verification workflow is named Build.
  • The branch to run on is named auto-merge.
  • The default branch is named main.
# .github/workflows/auto-merge.yaml
name: Auto-merge
on:
  workflow_run:
    workflows: [Build]
    branches: [auto-merge]
    types: [completed]

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    steps:
      - uses: actions/checkout@v4
      - name: Tell git we want the `auto-merge` branch
        run: git remote set-branches --add origin ${{ github.event.workflow_run.head_branch }}
      - name: Fetch the target branch
        run: git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"
      - name: Checkout target branch
        run: git checkout ${{ github.event.workflow_run.head_branch }}
      - name: Push current head to main
        run: git push -v origin HEAD:main


Enter fullscreen mode Exit fullscreen mode

Pushing to the right branch locally

You can push directly to the branch from the command line:

> git push origin HEAD:auto-merge
Enter fullscreen mode Exit fullscreen mode

Here, the remote is assumed to be named origin, the default for most workflows. But this can be simplified, so you just need to perform a normal git push.

To set that up, you need to make some changes in your local git repository configuration, found in the .git/config file of your working directory.

First sub-optimal solution

By adding a push refspec for the remote, you can configure git to push to a different remote branch.

# .git/config
[remote "origin"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
    push = refs/heads/main:refs/heads/auto-merge
Enter fullscreen mode Exit fullscreen mode

With this setting, when you pull the main branch from your local working directory, you get the main branch from the remote repository, but when you push the main branch, it will be pushed to the auto-merge branch on the remote. If your changes are good, your team mates will quickly get them when they pull.

Using this setup, you can work on the main branch locally exactly the same way you would normally.

But this also changes the default behaviour for other branches.

A less intrusive configuration

A solution to not break default behaviour for other branches is to create a new remote with the same url. Then you can configure the default to use this remote when pushing. Now other branches are not affected by the change to the push configuration, only the default branch has different behaviour.

# .git/config
[remote "origin"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[remote "origin-main"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
    push = refs/heads/main:refs/remotes/origin/auto-merge
[branch "main"]
    remote = origin
    pushRemote = origin-main
    merge = refs/heads/main
Enter fullscreen mode Exit fullscreen mode

The local workflow

With everything setup, your workflow should looks remarkably familiar:

> git pull # or git pull --rebase
> git commit -am "Change 1"
> git commit -am "Change 2"
> git commit -am "Change 3"
> git push
# In case of a conflict
> git pull # or git pull --rebase
# Fix merge conflicts
> git push
Enter fullscreen mode Exit fullscreen mode

The only differences are:

  • There is the delay of the build before you know what the outcome is.
  • You local branch will appear to be ahead of the default branch after a push, until you fetch again after a successful build.

But commits failing the verification will not appear in the main branch.

The git configuration is local

Remember, the repository configuration is local, i.e., it exists only on the local machine. Every developer need to set the configuration on every computer they use.

It's a simple solution to a simple problem

This setup will not prohibit bad commits from reaching the default branch. Developers can still push to directly to the default branch.

For a team project, every developer needs configure their git configuration to push to an auto-merge branch, requiring uncommon initial custom configuration.

But for those projects that don't need complicated processes, this small change can help prevent bad commits from reaching the default branch in a completely non-intrusive way.

Speedy emails, satisfied customers

Postmark Image

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (0)

AWS Security LIVE!

Tune in for AWS Security LIVE!

Join AWS Security LIVE! for expert insights and actionable tips to protect your organization and keep security teams prepared.

Learn More

👋 Kindness is contagious

Explore a sea of insights with this enlightening post, highly esteemed within the nurturing DEV Community. Coders of all stripes are invited to participate and contribute to our shared knowledge.

Expressing gratitude with a simple "thank you" can make a big impact. Leave your thanks in the comments!

On DEV, exchanging ideas smooths our way and strengthens our community bonds. Found this useful? A quick note of thanks to the author can mean a lot.

Okay